Entries Tagged 'wetware' ↓

Schrödinger used to have this quantum cat which was alive and dead at the same time as long as nobody opened the box, and it was the very act of looking at the cat that made it either alive or dead. Now, I’m not sure about this quantum stuff, but if you ask me you’d always find a dead cat upon opening the box, killed by the act of not looking. In fact, if you open any random box nobody was looking into, chances are you’ll find a dead cat there. Let me give an example.

I recently chatted with a former employee of a late company I’ll call Excellence (the real name was even worse). Excellence was a company with offices right across the street that kept shrinking until the recent financial crisis. It then had to simultaneously fire almost all of its remaining employees, carefully selected as their best during the previous years when other employees were continuously fired at a lower rate. Giving us a whole lot of great hires, including MV, the co-worker in this story (though he was among those who guessed where things were going and crossed the street a bit earlier).

Rumors tell that to the extent possible, Excellence used to live up to expectations created by its name. In particular, without being encouraged or forced by customers to comply to any Software Development Methodology such as the mighty and dreadful CMM, they had (as CMM puts it) not only Established, but rigorously Maintained an elaborate design, documentation and review process which preceded every coding effort. Other than praise, MV had little to say about the process, except perhaps that occasionally someone would invent something awfully complicated that made code incomprehensible, having gone just fine through the review process because of sounding too smart for anyone to object.

Now, in our latest conversation about how things were at Excellence, MV told how he once had to debug a problem in a core module of theirs, to which no changes had been made in years. There, he stumbled upon a loop searching for a value. He noticed that when the value was found, the loop wouldn’t terminate - a continue instead of a break kind of thing. Since the right value tended to be found pretty early through the loop, and because it was at such a strategic place, test cases everyone was running took minutes instead of seconds to finish. Here’s a dead cat in a gold-plated box for ya, and one buried quite deeply.

My own professional evolution shaped my mind in such a way that it didn’t surprise me in the slightest that this slipped past the reviewer(s). What surprised me was how this slipped past the jittery bodybuilder. You see, we have this Integration Manager, one of his hobbies being bodybuilding (a detail unrelated, though not entirely, to his success in his work), and one thing he does after integrating changes is he looks at the frame rate. When the frame rate seems low, he pops up a window with the execution profile, where the program is split into about 1000 parts. If your part looks heavier than usual, or if it’s something new that looks heavy compared to the rest, it’ll set him off.

So I asked MV how come that the cat, long before it got dead and buried, didn’t set off the jittery bodybuilder. He said they didn’t have one for it to set off. They were translating between the formats of different programs. Not that performance didn’t matter - they worked on quite large data sets. But to the end user, automatic translation taking hours had about the same value as automatic translation taking seconds - the alternative was manual translation taking weeks or months. So they took large-scale performance implications of their decisions into account during design reviews. Then once the code was done and tested, it was done right, so if it took N cycles to run, it was because it took N cycles to do whatever it did right.

And really, what is the chance that the code does everything right according to a detailed spec it is tested against, but there’s a silly bug causing it to do it awfully slowly? If you ask me - the chance is very high, and more generally:

• Though not looking at performance followed from a reasonable assessment of the situation,
• Performance was bad, and bad enough to become an issue (though an unacknowledged one), when it wasn’t looked at,
• Although the system in general was certainly “looked at”, apparently from more eyes and angles than “an average system”, but it didn’t help,
• So either you have a jittery bodybuilder specifically and continuously eyeballing something, or that something sucks.

Of course you can save effort using jittery automated test programs. For example, we’ve been running a regression testing system for about a year. I recently decided to look at what’s running through it, beyond the stuff it reports as errors that ought to be fixed (in this system we try to avoid false positives to the point of tolerating some of the false negatives, so it doesn’t complain loudly about every possible problem). I found that:

• It frequently ran out of disk space. It was OK for it to run out of disk space at times, but it did it way too often now. That’s because its way of finding out the free space on the various partitions was obsoleted by the move of the relevant directories to network-attached storage.
• At some of the machines, it failed to get licenses to one of the compilers it needed - perhaps because the env vars were set to good values with most users but not all, perhaps because of a compiler upgrade it didn’t take into account. [It was OK for it to occasionally fail to get a license (those are scarce) - then it should have retried, and at the worst case report a license error. However, the compiler error messages it got were new to it, so it thought something just didn't compile. It then ignored the problem on otherwise good grounds.]
• Its way of figuring out file names from module names failed for one module which was partially renamed recently (don’t ask). Through an elaborate path this resulted in tolerating false negatives it really shouldn’t.
• …

And I’m not through with this thing yet, which to me exemplifies the sad truth that while you can have a cat looking at other cats to prevent them from dying, a human still has to look at that supervisor cat, or else it dies, leading to the eventual demise of the others.

If you don’t look at a program, it rots. If you open a box, there’s a dead cat in it. And if a tree falls in a forest and no one is around to hear it, it sucks.

As a part of my continuous moral degradation and the resulting increasing alignment with the forces of Evil, I’m sharing an apartment with a gal who used to work in HR assessment. She recently got me acquainted to a friend of hers, BC, who works as a business consultant (names have been changed to protect the guilty).

BC’s primary educational background is in applied mathematics. Having put the math they teach in CS departments to relatively few uses as a working programmer, I asked her about the uses of applied mathematics in business consulting. BC cited the following two examples.

The first example involves compensation and its dependence on key performance indicators, affectionately known as KPI and estimated by HR assessors. One way of looking at this dependence is to consider how it affects compensation over time as an employee’s competence increases.

A psychological discussion is then possible on the relative merits of the different graphs plotting the compensation functions f(KPI). If f is linear (has a constant derivative), we make people struggle equally hard at every step. If f’s derivative increases over time (for instance, when f is exponential), we make elevation hard at first and then increasingly easy. If f’s derivative decreases over time (for example, if f is logarithmic), we make the last mile the hardest. And so on.

Through a psychological discussion of this sort, someone in the consulting company decided that what was really needed in some case or other was an S-shaped curve. The problem was that you couldn’t just draw an S-shaped curve - the plotting had to be done in Excel according to a formula; an S-shaped curve which just blithely goes through arbitrary points doesn’t cut it when you deliver a Compensation Model. But how do you make a formula to go up slowly, than fast, than slowly again? Exponents don’t work. Logarithms don’t work. A sine does look like an S, but it’s a wrong kind of S, somehow. What to do?

Enter BC with 6 years of math studies under her belt. A compact yet impressive formula is spelled out, and - presto! - Excel renders an S-shaped curve. (I guess she used the sigmoid function but I didn’t check.) The formula brought delight to management and fame to BC, and compensation payments issued according to its verdict keep adding up to scary numbers (BC’s agency works with some really big companies).

The second example involves the compensation of managers. Naturally, a good manager near the bottom is worth less to the firm than a bad manager near the top, and therefore the compensation function should now depend on the manager’s level in the hierarchy as well as his KPI (or better). Equally naturally, the numbers coming out of the compensation spreadsheet will under no circumstances arise through an externally conducted study of their psychological implications or any similarly unpredictable device. The numbers will result from nothing but the deep understanding of the organization possessed by the top management.

The development process of the managerial compensation function is thus complementary to that of the employee compensation function. Instead of producing numbers from a beautiful spreadsheet, what is needed here is to produce a beautiful spreadsheet from the numbers specified by the top management. The spreadsheet then promptly generates back these exact numbers from the input parameters.

The purpose of the spreadsheet is to relieve the top managers from the need to justify the numbers to their underlings. In order to guarantee that they are relieved from this need, the formula should not contain terms such as 1/(level^2), which could raise questions such as why not use 1/level, why not use 1/log(level) and other questions along these lines. Rather, the formula should contain terms which could raise no questions at all simply due to their size and shape.

BC faced this problem at an early stage of her career, and despite the natural stress, came up with an interesting Compensation Model, its key term being e raised to the power of something unspeakably grand, combining the trademark Gaussian look and feel with an obvious ability to deter the skeptics from asking questions. The only problem with that term was the very source of its utility, namely, the fact that it evaluated to 0 for all values of KPI and hierarchy level.

The deadline being close, BC told the manager of the consulting project in question about the status of her work and expressed her doubts regarding the delivery of the Computational Model. The manager told her that she just doesn’t get it, does she, it’s great, the right numbers come out and that’s all there is to it and we should send it right away. And so they did, to everyone’s complete satisfaction.

Her command of applied mathematics aside, BC is generally quite powerful.

For instance, she once got invited to consult some government agency about a project of theirs, while being on vacation and without it being explained to her that she was about to attend a formal meeting with the whole team. In order to maintain the reputation of the guy who somewhat clumsily brought her in, she had to improvise.

The project, worthy of a government agency, was related to some sort of war on corruption, the unusual thing being that they wanted to fight the corruption of other governments. Their weapon of choice was the training of representatives of another government, financed by the other government, in their supposedly superior methods of governance. While the general concept was impressive on many levels, the details were unclear.

BC had to speak, and she spoke based on a principle appearing in a book by some McKinsey alumni (she didn’t tell its name nor generally recommended it): whatever you tell people, it should contain 3 main points. Possibly 4. But preferably 3. More is overwhelming and less is boring. So she said: “At its core, your project is about teaching people. It is therefore essential to clearly understand three things:

• Whom you’re teaching,
• What you’re teaching them,
• And how you’re teaching it.”

And they started writing it down.

I asked BC whether there was some way to unleash her on the company employing me so that she grinds a few bullet points into them (a handsome Compensation Model being as good a place to start as any). She said something to the effect of “it only works on the weak-minded”; it was apparent, she said, that the government agency in question had little previous exposure to consulting.

BC says she (still) believes that business consulting is meaningful and valuable, which sounds paradoxically at this point. But, looked at from another angle, it really isn’t. Don’t view her stories as ones undermining the credibility of business consulting but rather as ones building her own credibility as a person aware of the actual meaning of things and willing to sincerely share that understanding (how many people would instead say that they Developed Cutting-Edge Compensation Models?) If she says there’s meaning to it, perhaps there is.

A few days ago a coworker stepped into the room of our Integration Manager, with his iPhone in his hand and his face glowing with happiness, telling he has this great new game on his phone. He asked for a permission to take a picture of the Integration Manager, and did so to the sound of dramatic, though somewhat repetitive, music. He then marked the areas around the eyes and the mouth of his subject.

Then the game started. The guy pressed some buttons, delivering virtual blows to the photographed subject, who couldn’t really fight back according to the rules of the “game”. The face on the photograph got covered with blood and wounds (especially the eyes and the mouth), and the speaker yelled out something designed to convey pain. And the guy stood there and kept punching and laughed his ass off.

Then he talked about how iPhone was this great thing and how iPhone app developers got rich. And I told him in a grim voice to go ahead and get rich off iPhone apps and he sensed disrespect, to him or to the iPhone or both. But mainly it wasn’t disrespect, it was desperation. I told him to go ahead and do these apps because he probably can, and I certainly can’t, so I have no choice but leave all this wealth for him to collect.

Take this punching app. Where do I fit in?

For instance, I know enough computer vision to eliminate the need to manually select the eyes and the mouth. You find the eyes using circular Hough transform, and then you get a good idea where to look for the mouth and then it can’t be too hard. How much value would this automation add? The happy user I observed didn’t seem bothered by the need to select, worse, I suspect having to do it made him happier - this way he actually worked to deliver his punches.

And then what I couldn’t do, and I couldn’t do this for the life of me, is to invent this app in the first place. It’s just something that I don’t think I’ll ever understand, something worse than, I dunno, partial differential equations which you can keep digging into, something more like the ability to distinguish between colors, which is either there or not.

Could someone PLEASE explain just how can such a profoundly idiotic activity yield any positive emotions in a human soul?

Perhaps it’s because our Integration Manager is a bodybuilder, with bulging biceps and protein milkshakes and stuff, so there’s no other way to beat him up for that guy? Well, he could probably beat me up, and he still enjoyed going through the process with me. So this just shows how childish my attempt to dissect the psychology of the phenomenon is - I’m not even close. How could I ever think of something I can’t even begin understanding after I’ve already seen it?

The serious, or should I say sad, side of this is that to develop something valuable to users, you need empathy towards these users, and the way empathy works is through identifying with someone, and so the best chance by far to develop something valuable is to develop something for yourself. But someone like me, who bought his first PC in 2007 and uses a cell phone made in 2004, with no camera, no Internet connection, no nothing, simply can’t develop something for himself because he clearly just doesn’t need software.

The only chance for a programmer who uses little software except for development tools is to develop development tools. Take Spolsky, for instance - here is a programmer who is, by the programmers’ standard, extremely user-aware and user-centric, as evident in his great UI book, among other things, and yet he eventually converged to developing software for programmers, having much less success with end-user products.

The trouble with development tools is that the market is of course saturated, with so many developers with otherwise entrepreneurial mindsets being limited by their lack of empathy to this narrow segment of software. This is why so many programs for programmers have the price of zero - not because copying software has a cost near zero; developing software has a cost far from zero so there aren’t that many free face punching apps, but with software for developers, there’s unusually fierce empathy-driven competition.

So every time I see an inexplicably moronic hit app, I sink into sad reflections about my destiny of forever developing for developers, being unable to produce what I can’t consume and watching those with deeper understanding of the human nature reaping all the benefits of modern distribution channels.

P.S. To the fellow programmer who, as it turns out, uses FarmVille - you broke my heart.

Suppose you have a sparse RAM API, something along the lines of:

• add_range(base, size)
• write_ram(base, bytes)
• read_ram(base, size)

People use this API for things like running a simulated CPU:

1. define the accessible memory with add_range()
2. pass the initial state to the simulator with write_ram()
3. run the simulation, get the final state with read_ram()

Suppose this API becomes a runaway success, with a whopping 10 programmers using it (very little irony here, >95% of the APIs in this world are used exclusively by their designer). Then chances are that 9 of the 10 programmers are API users, and 1 of them is an API wrapper. Here’s what they do.

API users

The first thing the first API user does is call you. “How do I use this sparse thing of yours?” You point him to the short tutorial with the sample code. He says “Uhmm. Errm…”, which is userish for “Come on, I know you know that I’m lazy, and you know I know that docs lie. Come over here and type the code for me.” And you insist that it’s actually properly documented, but you will still come over, just because it’s him, and you personally copy the sample code into a source file of his:

add_range(0x100000, 6) # input range
add_range(0x200000, 6) # output range
write_ram(0x100000, "abcdef")
# run a program converting the input to uppercase
print read_ram(0x200000, 6) # should print "ABCDEF"

It runs. You use the opportunity to point out how your documentation is better than what he’s perhaps used to assume (though you totally understand his frustration with the state of documentation in this department, this company and this planet). Anyway, if he has any sort of problem or inconvenience with this thing, he can call you any time.

The next 8 API users copy your sample code themselves, some of them without you being aware that they use or even need this API. Congratulations! Your high personal quality standards and your user-centric approach have won you a near-monopoly position in the rapidly expanding local sparse RAM API market.

Then some time later you stumble upon the following code:

add_range(0x100000,256)
add_range(0x200000,1024)
add_range(0x300000,1024)
...
add_range(0xb00000,128)
...
add_range(0x2c00000,1024)
...

Waitaminnit.

You knew the API was a bit too low-level for the quite common case where you need to allocate a whole lot of objects, doesn’t matter where. In that case, something like base=allocate_range(size) would be better than add_range(base,size) - that way users don’t have to invent addresses they don’t care about. But it wasn’t immediately obvious how this should work (Nth call to allocate_range() appends a range to the last allocated address, but where should the first call to allocate_range() put things? What about mixing add_range() and allocate_range()? etc.)

So you figured you’d have add_range(), and then whoever needed to allocate lots of objects, doesn’t matter where, could just write a 5-line allocate_range() function good enough for him, though not good enough for a public API.

But none of them did. Why? Isn’t it trivial to write such a function? Isn’t it ugly to hard-code arbitrary addresses? Doesn’t it feel silly to invent arbitrary addresses? Isn’t it actually hard to invent constant addresses when you put variable-sized data there, having to think about possible overlaps between ranges? Perhaps they don’t understand what a sparse RAM is? Very unlikely, that, considering their education and experience.

Somehow, something makes it very easy for them to copy sample code, but very hard to stray from that sample code in any syntactically substantial way. To them, it isn’t a sparse RAM you add ranges to. Rather, they think of it as a bunch of add_range() calls with hexadecimal parameters.

And add_range() with hex params they promptly will, just as it’s done in the sample. And they’ll complain about how this API is a bit awkward, with all these hex values and what-not.

API wrappers

If there’s someone who can see right through syntax deep into semantics, it’s the tenth user of your API, or more accurately, its first wrapper. The wrapper never actually uses an API directly in his “application code” as implied by the abbreviation, standing for “Application Programming Interface”. Rather, he wraps it with another (massive) layer of code, and has his application code use that layer.

The wrapper first comes to talk to you, either being forced to use your API because everybody else already does, or because he doesn’t like to touch something as low-level as “RAM” so if there’s already some API above it he prefers to go through that.

In your conversation, or more accurately, his monologue, he points out some admittedly interesting, though hardly pressing issues:

• It’s important to be able to trick a program using the sparse RAM API into allocating its data in specific address ranges, so that the resulting memory map is usable on certain hardware configurations and not just in simulations.
• In particular, it is important to be able to extract the memory map from the section headers of executables in the ELF and COFF format.
• Since add_range() calls are costly, and memory map formats such as the S-Record effectively specify a lot of small, adjacent ranges, there is a need for a layer joining many such ranges.
• An extensible API for the parsers of the various memory map formats is needed.
• …

When you manage to terminate the monologuish conversation, he walks off to implement his sparse RAM API on top of yours. He calls it SParser (layer lovers, having to invent many names, frequently deteriorate into amateur copywriters).

When he’s done (which is never; let’s say “when he has something out there”), nobody uses SParser but him, though he markets it heavily. Users won’t rely on the author who cares about The Right Thing but not about their problems. Other wrappers never use his extra layers because they write their own extra layers.

However, even with one person using it, SParser is your biggest headache in the sparse RAM department.

For example, your original implementation used a list of ranges you (slowly) scanned through to find the range containing a given address. Now you want to replace this with a page table, so that, given an address, you simply index into a page array with its high bits and either find a page with the data or report a bad address error.

But this precludes “shadowing”, where you have overlapping segments, one hiding the other’s data. You thought of that as a bug in the user code your original implementation didn’t detect. The wrapper thought it was a feature, and SParser uses it all over to have data used at some point and then “hidden” later in the program.

So you can’t deploy your new implementation, speeding up the code of innocent users, without breaking the code of this wrapper.

What to do

Add an allocate_range() API ASAP, update the tutorial, walk over to your users to help replace their hex constants with allocate_range() calls. Deploy the implementation with the page table, and send the complaining wrapper to complain upwards along the chain of command.

Why

Your users will switch to allocate_range() and be happy, more so when they get a speed-up from the switch to page tables. The wrapper, constituting the unhappy 10% of the stakeholders, will have no choice but fix his code.

Ivan drank half a bottle of vodka and woke up with a headache. Boris drank a full bottle of vodka and woke up with a headache. Why drink less?

Users are many, they follow a predictable path (copy sample code) and are easily satisfied (just make it convenient for them to follow that path). Wrappers are few, they never fail to surprise (you wouldn’t guess what and especially why their layers do), and always fail to be satisfied (they never use APIs and always wrap them). Why worry about the few?

The only reason this point is worth discussing at all is that users offend programmers while wrappers sweet-talk them, thus obscuring the obvious. It is natural to feel outrage when you give someone an add_range() function and a silly sample with hex in it, and not only do they mindlessly multiply hex numbers in their code, but they blame you for the inconvenience of “your API with all the hex in it”. It is equally natural to be flattered when someone spends time to discuss your work with you, at a level of true understanding (”sparse RAM”) rather than superficial syntactic pattern matching (”add_range(hex)”).

He who sees through this optical illusion will focus on the satisfaction of the happy many who couldn’t care less, securing the option to ignore the miserable few who think too much.

Ever noticed how academic asses are analog and industrial asses are digital? It’s legitimate to not know whether P equals NP, or to not know what x is if x*2=y but we don’t know y, for that matter. But it isn’t legitimate to not know how many cycles, megabytes or - the king of them all - man-months it will take, so numbers have to be pulled out of one’s ass.

The interesting thing is that the ass adapts, that the numbers pulled out of this unconventional digital device aren’t pure noise. Is it because digital asses know to synchronize? Your off-by-2-months estimation is fine as long as other estimations are off by 5. But it’s not just that, there must be something else, a mystery waiting to be discovered. We need a theory of computational proctology.

Ever noticed how painful the act of anal estimation is for the untrained, um, mind, but then eventually people actually get addicted to it? Much like managers who learn that problems can be made to go away by means such as saying a firm “No”, without the much harder process of understanding the problem, not to mention solving it? Anal prophecy is to the technical “expert” the same raw enjoyment that the triumph of power over knowledge is to the manager. “Your powers are nothing compared to mine!”

There once was a company called ArsDigita (I warmly recommend the founder’s blog and have his Tenth Rule tattooed all over my psyche), a name I tend to misread as “ArseDigital” - a tribute to an important method of numerical analysis and estimation in the computing industry.

I never managed a group larger than 5 people, luckily for the people in the group (perhaps more so for those remaining outside). Good managers are hard to find, which is the basis of my self-motivating motto: “This job could have been done worse”. Such is the background for the hereby presented pearls of wisdom assortment. As to “The Virtue of a Manager” title, it’s a ripoff of Paul Krugman’s exquisite title “The Conscience of a Liberal“. “The Private Part of a Self-Important Self-Description” is a great template.

***

A prime virtue of a manager is the ability to take pride in someone else’s work.

No, seriously. We’ve recently deployed a debugger internally and an algorithm developer had a look at it. I knew it was good, but it’s used to debug the sort of thing algo devs hate: code with an anal-retentive performance focus. So the last thing I expected was praise, but praise it the guy did.

Now, I had previously known proud moments from having done things myself, and here I had this proud moment with 90% of the work done by someone else. And I’m telling you, it was just like the real thing.

***

The defining trait of a manager is the distinctly wide gap between responsibility and understanding.

By far the funniest spot to have a gap at, hence the easiest target for a low blow: try to make jokes about a gap between one’s teeth and you’ll soon be exhausted, but this here is gold. This is mean-spirited though. Imagine living with a gap between your responsibility and your understanding and everybody laughing at you - how would that make you feel? Show compassion.

***

One can have the title of a manager or nominal reports for any of a number of reasons:

• An HR system with per-title wage ceilings: can’t give someone a raise without faking a title.
• A diametrically opposed case: some forms of brain damage cause people to accept lower paychecks given more impressive titles, larger rooms, etc.
• Someone is too senior to report to a team leader but doesn’t want a team to report to him, either.
• …

In a roomful of managers, how do you find the real ones among this variety - not “real” as opposed to incompetent or unimportant, but “real” as opposed to fake?

There are several cues, for example, only real managers can have other managers report to them. But the perfect, if-and-only-if discriminator is that real managers don’t write code. (The precise rule is that they can spend up to 2% of their time on a favorite piece of code without getting disqualified.)

***

The principal function of a manager is being the responsible adult.

Some managers occasionally point this out in frustration, both mourning their technical skills which dry up during their current gig where they only get to exercise adulthood, and because being the adult means getting tired of the annoying kids. A gal who both managed and met literally hundreds of managers during her career in some consulting agency said “Now I really understand management” when she got to babysit.

This is why I have hard time believing management can be taught - you can’t teach adulthood, it can only result from people growing up by themselves. I’m not sure if this feeling is fully aligned with reality, but quite some very successful managers never went to a management school (at least one of those is somewhat critical of MBAs), and some of those who went say it was worthless in terms of useful things learned.

The opposite is also true: childishness is fitting for a programmer. We were two fake code-writing managers in a meeting with one real one, and at one point the real one said: “Let’s not be childish about this”. The technically correct reply to her would have been “I’M NOT CHILDISH ABOUT THIS, HE IS!”, but I suppressed it for tactical reasons. Some time later I told her: “You don’t want us to stop being childish about this, not as long as you’re interested in our output as programmers. Recall: the reason you aren’t still programming is because of not being childish enough to truly enjoy this sort of game.”

And in fact since she started managing 20 programmers, she’s been talking about her work all the time, which she didn’t when she was programming. Well, some people like to play and some prefer to babysit. (I’m not sure where this leaves the quasi-managers who write code; presumably some are the elder and most responsible kid while others are the most restless who invent games for the gang.)

***

I’ve recently got a driving license. One thing I learned was that someone pushing his (presumably broken) car along the road is a “driver” as far as the law is concerned. I find this counter-intuitive, probably because pushing a car is not categorized in my head as “driving experience”, but, at least in Israel, that’s the law.

Likewise, doing the work of three people is not what most of us associate with “managerial responsibility”. However, if you’re given two reports without a drive of their own to work, that’s what your responsibility will be.

***

A manager will have favorite words. For example: acute (critical), priorities, agenda, rationale, integrity (shoot this manager first), responsibility (ownership), stakeholder.

Keep laughing at them. Once you become a manager, you’ll have favorite words whether you want it or not - it is useless to resist the dynamics inherent to your situation. My favorite word is “dynamics”. Its connotations are deep and its applicability wide - heartily recommended.

***

Managers get to do a lot of knowledge-free decision making, which necessarily drives them insane. Here’s how the manager’s bipolar disorder works.

During maniacal periods, the manager is the only one who can do anything around here. This frequently happens when the manager is under external pressure, and he feels that control is slipping out of his hands. He’s trying to compensate for his lack of knowledge by immense concentration and willpower. (Managers always have ample emergency supplies of both.) “Concentration” translates to an ability to derive general and far-reaching conclusions from insignificant details, then “willpower” translates to aggression.

Then depression follows: “Don’t bother me with details”. This results partly from exhaustion quickly arrived at during the mania (especially if reports were wise enough to not argue with the manager, letting his efforts defeat their own purpose.) The manager has delivered his trademark concentration and willpower, so he no longer feels guilty on that front. However, he’s overwhelmed by information and (rightly) feels that he doesn’t know what’s going on. He decides it is none of his business and concentrates on the Big Picture (does nothing). Usually, the cycle repeats upon a new wave of external pressure.

Awareness of the management cycle on behalf of the manager himself can help soften the cycle but not eliminate it. It is up to reports to apply counter-cycle measures by scheduling most work into depression periods when it is least disrupted. Special attention must be given to long-term projects, frequently characterized by a prolonged depressive apathy period at the beginning followed by a period of maniacal frenzy lasting until the end.

***

There’s a naive brain model in the spirit of “the brain has a reptilian part, a mammal part and a human part”. For example, if a student fails to answer a question in an oral exam with his human brain, the mammal brain feels bad about it and complains to the reptilian brain. The reptilian brain then cheerfully replies, “Who’s causing the trouble? Oh, that little guy behind the table? Not to worry - I’ll kill him”. The higher brains then supposedly suppress this - “What do you think this is, reptile - Jurassic Park?”, and the tension is translated into sweating.

The manager is the team’s reptilian brain; he doesn’t know enough to do real thinking, but he’s good at “taking responsibility”, bargaining, fighting, socializing, etc. A manager doesn’t know how to implement the feature, except for suspecting, based on experience, that it will conflict with a couple other features and it will take a week or three for the whole thing to stabilize (with him taking the heat when things break during those weeks). Therefore, instead of technical advice (which he might be otherwise qualified to give), he’ll propose something which solves the problem at his favorite social plane:

• Prioritize the feature away, delaying the implementation until forever
• Negotiate the feature away, by talking to whoever wants it out of it for something in return
• Redefine the feature away, by reducing the scope to the few scenarios which absolutely can’t be ignored
• …

Do not drag management into anything you actually want solved. Presented with a question, the manager will answer it by killing the little guy behind the table, so only go to him if you really want that. And once awakened, he might take a lot of sweat to suppress. (If he’s really a programmer posing as a quasi-manager, the chances for an actual solution can actually be worse: he’s more likely to feel guilty about his managerial ability and use the opportunity to exercise and develop that ability, instead of using his technical ability to think about the issue.)

***

There’s this quote from The Mythical Man-Month, supposedly by a pessimistic manager:

All programmers are optimists. Perhaps this modern sorcery especially attracts those who believe in happy endings and fairy god-mothers. Perhaps the hundreds of nitty frustrations drive away all but those who habitually focus on the end goal. Perhaps it is merely that computers are young, programmers are younger, and the young are always optimists. But however the selection process works, the result is indisputable: “This time it will surely run,” or “I just found the last bug.”

This is backwards. In reality, programmers are the more pessimistic people. Perhaps it’s because experience teaches programmers that programs always have bugs while teaching managers that programs always ship. Perhaps it’s because the programmer is the one with the actual knowledge, and the ignorant are always optimists. But however the selection process works, how many programmers have you seen saying “it will never work” and how many managers?

A programmer might be more optimistic locally, hoping in vain to have fixed this one piece of code where he has the illusion of complete understanding. However, it is invariably the manager who believes that everything will work out. A programmer can’t really believe that because there are so many things nobody even understands that are yet to be faced.

But the manager is used to knowing little and understanding less, and thus has learned to translate uncertainty to optimism. In fact a programmer can learn it, too, in the areas which are of little interest to him. I know a programmer who doesn’t care about optimization and who consequently describes others’ efforts to fit a program into a given performance budget as doomed to success: “It runs at the word of command” - a programmer’s expression of the managerial worldview worthy of a seasoned manager.

***

We don’t know how to test for programming ability. The best tech companies spend 5 to 10 interviews to solidly confirm that the candidate knows what is taught during the first 1.5 years of an undergraduate CS curriculum. Other processes measure less accurately by asking less relevant questions; the inaccuracy is somewhat ameliorated by the lack of precision - the non-uniform quirks of interviewers and general randomness of the process eliminate biases, causing all kinds of good candidates to sneak through the gates.

It is well known that we can’t find out during the interview what we inevitably find out once someone gets the job, but what are the corollaries? Here’s one I’ve heard a lot: trust recommendations more than interviews. Here’s another I haven’t: let others interview and get the new hires, then steal the best.

(Objection: the first recommendation is good for the company while the other is only good for the manager following it. Well, “competition between managers over team members isn’t a zero-sum game - it improves teamwork across the company”, this one we weasel out of in a snap.)

***

We have a VIP club at work called Bottleneck, its principal activity being the collective purchasing and consumption of alcoholic beverages. The club operates during work hours (regular meetings held on Thursdays, emergency meetings scheduled upon arrival of packages from abroad). Our room being the headquarters, I’m naturally a member. By now the club has shifted to high-end liquors at prices causing the consumption to contract to a sip per cup of coffee, but originally it was affordable to actually drink.

I noticed that minor alcoholic intoxication has a notable impact on my programming ability. I can still lay my hands on the right variable, but by the time I do I forget what you do next with these things. There’s that handy member somewhere in it, dot something, but dot what?

However, managerial ability is not affected. Things I can do just fine following a meeting of the Bottleneck club include progress monitoring, planning, risk assessment, general technical advice, and requirement negotiation. Now that I think of it, perhaps the managerial functions are affected for the better.

There’s this broad metaphor I have, with no conclusions attached - just an attempt to describe dynamics. I’ve recently shared it with the commie ex-VP school teacher and he liked it, so I thought it could fit well with the other borderline stuff I host at Proper Fixation under the “wetware” category.

So. One recurring theme in the history of civilization is the conflict between nomadic and settled people. Nomads think that land is for feeding cattle and you move elsewhere once there’s nothing left to graze. Villagers figure that land is for growing food, so you settle on it and fertilize it and irrigate it and stuff. Initially, nomads typically dominate the landscape, periodically attacking the settled villagers and taking their crops. However, the settled people eventually accumulate enough surplus to support cities, nation states and standing armies, extending their control to more and more lands and eventually exterminating the nomadic lifestyle altogether.

The way I painted this picture, I tend to side with the hard-working settled folk, the nomads being the parasitic losers I’ve depicted, and I think most of us civilized humans share similar sentiments. However, in my metaphor I side with the nomadic programmer, at least to a large extent, and I do so because of the meaning my metaphor assigns to “land”.

The thing I find analogous to land in programming is problems, because that’s where programmers live. Programmers live on (in?) problems in the sense of dealing with broken things most of the time - once something starts working, you move on to something that doesn’t. In another sense, large problems or problem areas a programmer deals with define that programmer’s territory. The programmer is in immediate demand to the extent that solutions to “his” problems are in demand; problems feed programmers. Strong programmers seek, in one way or another, to expand their responsibility to encompass more problems, and to preserve their existing responsibilities. And so on.

Now if we restate the respective worldviews of nomads and settlers in the terms of this metaphor, we’ll get this. Nomads think that problems exist for solving them and you move elsewhere once there’s nothing left to graze. Settlers think that problems exist for growing them, so they settle on them and fertilize them and irrigate them and stuff.

And now you can see why I’m inclined to sympathize with the nomadic programmer. Two other things fueling this sympathy are issues of personality to be discussed soon, and the fate of the nomad to be discussed immediately. And while the nomad is no longer the parasite, rest assured that he’s still, in the long run, the loser.

Initially - in a young and small organization - nomadic programmers tend to dominate the landscape. There are more problems than people around. The nomadic programmer travels from one urgent problem to another, grazing through them as fast as he can. Occasionally he stumbles upon a settler who has settled on a problem near the nomad’s territory and grown crops of code there. Well, if the problem occupied by the settler becomes urgent, or if the crops stand in the way of solving the nomad’s adjacent urgent problem, the nomad will go ahead and brutally solve the settler’s problem, wiping out his crops. The politics of the invasion will be trivial - a promise to deliver by the nomad carries lots of weight at this stage and the settler will not issue a counter-promise (to deliver in his own way) because he’s a peaceful code-growing villager who isn’t into stress which necessarily comes with delivering quickly.

However, the time goes by and sure enough, the settled people accumulate quite some surplus. What you grow on land is surplus wheat; what you grow on problems is surplus code. Code that wouldn’t naturally grow on a problem - but now that the problem was fertilized by the original settlers, they’ve grown enough code on it to support whole cities, a nation state, and a standing army of programmers, all making a living by fiddling with this code.

The nomad starts running out of pasture. Sure enough, there are lots of problems just like there used to be. But you can no longer solve them because (1) now it’s the majority and not a minority of problems that are already owned by someone (growing them rather than solving them) and (2) in most cases invasion is no longer an option. Now that the problem is owned by a nation state, responsible for lots of code and with lots of people working on that code, the nomad’s promise to deliver quickly carries very little weight compared to the danger of irritating the sovereign. While it is quite likely still true that a nomad will probably deliver more quickly than the whole nation state team, the nomad will not be able to take over the entire responsibility of the team. (It is possible that the single reason for the latter is the problems grown by the team itself and that a few nomads could in fact handle the original problem. But it is irrelevant since problems that could have been avoided are no less real than problems that couldn’t.)

So if the organization, by some decision making mechanism, lets the nomad invade the territory of the settled team and solve the stupid problem, and then the offended team, by some decision making mechanism, fights back by effectively going on strike, there is nothing the nomad will be able to offer the organization at this point. Of course it doesn’t have to come to this, just like political conflicts don’t have to come to full-scale wars, or personal conflicts to fist fights or court hearings. It’s enough for the worst case scenario to be likely to work out in favor of A rather than B to shift the balance decisively in favor of A. Even if neither A nor B nor anyone making decisions affecting A and B actually think in terms of this scenario, things tend to evolve and adapt such that decisions are made in favor of A. And in our case, the nomadic programmer is B.

Solving problems just isn’t the big thing in this organization anymore, just like the quality of life experienced by the inhabitants of some territory isn’t the main theme in international politics. Perhaps there are ways to improve the quality of life in Siberia, however this is not nearly as important politically as the fact that there’s already a guy exclusively responsible for the quality of life in Siberia. Perhaps Socialism with Chinese Characteristics could yield improvements in the lives of Siberians that Managed Democracy could not, however, if the Chinese try to act on this assumption, there will be a nuclear war. If what remains of a nomadic tribe somewhere in the region makes a similar attempt, then it will remain no more.

The disgruntled nomadic programmer reduces his ambition to merely being left alone to wander the remaining wilderness. However, this option is no more real for him now than the option of being left alone was available to the settler in the old days. Back then, the settlers were never safe since a nomad could always bump into them in an attempt to solve a related problem, and if their stuff got in the way, he’d rewrite or delete/disable their stuff. Now it is the nomad who is never safe since the nation states keep expanding their responsibilities into neighboring problems - having enough people to have some of them free for that some of the time.

(Actually having even partially idle workers on a team leaves few satisfying alternatives to an attempt at expanding the team’s responsibilities since other teams are always happy to seize an idle worker. Likewise, back in the old days the nomadic programmer had few satisfying alternatives to invading and solving others’ problems since otherwise he couldn’t keep his promises to deliver. It’s not (just) the intentions that fuel wars, it’s (also) the situation.)

The nation states seeking to expand won’t fight each other since the nomad is a much easier target, not having resources (time and reports) to look over his entire territory. Once a nation state team managed to take over some of that ever-shrinking territory, the nomad will never gain it back. Increasingly, the nomad has to reach compromises with neighboring nation states whenever his work is related to their work. Then it turns out that in order to be able to work on what he wants at all, he has to do it the way a chief commander or an officer of a nation state team wants him to do it - and then that in order to work on anything at all, he has to report to such a manager.

At this point the nomadic programmer can use his reputation and seniority to get pseudo-promoted to a non-productive position. Alternatively, he can actually become a report of a nation state team manager with whom the relationship is likely already strained - and his seniority, reputation and ambitions won’t make the transition into this particular position of a report any smoother. Alternatively he can quit. His failure is now complete.

(It may sound like a natural thing for a nomad to change jobs fairly frequently - part of a lifestyle rather than the failure of that lifestyle. However, nomadic programmers are those who like to travel from problem to problem - not from job to job; some like the latter but some don’t. A new job at a new place means a temporary, but possibly significant loss of confidence and efficiency. An African nomad won’t necessarily welcome a relocation to Alaska.)

As I’ve said above, I have reasons having to do with my personality to side with the nomadic programmer, especially at the stage of mounting pressure from nation state teams. The people I tend to relate to most easily seem to be those who prefer freedom to power. A talented freedom-seeker with a strong sense of responsibility will accumulate, well, responsibilities much more quickly than reports - a lot of territory to wander, and no standing army to protect it. (The problem with reports is that you take their freedom by telling them what to do and they take your freedom through your responsibility for their actions; who wants reports?) Since many freedom-lovers disdain politics, they won’t respect international borders - a problem should be solved, dammit; hence they’re likely to initiate invasions.

However, while this means that I personally will tend to find myself sympathizing with particular nomadic programmers, this does not mean that theirs is the right way or something. For example, it is unclear which share of programming problems out there can really be “solved” - grazed through and left alone - and which problems actually require continuous care and gardening that a true nomad is not likely to supply. Also, whether there’s a “solution” you only need to “maintain” or an “infrastructure” you want to “extend”, the code needs a permanent owner. I don’t believe in collective code ownership any more than in collective ownership of anything else - what it usually means is that everybody collectively fights over something. Therefore I think that ownership should generally be respected, and so a compromise which is, from a technical viewpoint, quite moronic, can otherwise be a great thing - a belief outside the nomad’s way.

So while I know where my sympathies lie, I don’t know which camp I’m in and this is why this metaphor doesn’t come with any conclusions, just the dynamics. In fact I’d rather leave it without conclusions but I wouldn’t mind expanding more on the dynamics. For example, some - but not all - settled civilizations were actually started by nomads enslaving argicultural villagers and settling among them. Apparently a similar distinction can be made between nation state teams of programmers; it is then interesting whether differences in their behavior can be traced to their different origins. Perhaps a person more entertained than appalled by the sort of perspective on the adventurous lives of programmers here presented is also the kind of person more entertained than appaled by the history of mankind in general and so could help develop this line of thought based on his knowledge of history. Could be fun.

Update (2009-08-18) - Chuck Moore: “I’ve met too many people who want to make a career out of a project instead of completing it” - the nomad’s view of the settlers. Nomadism is apparent in other writing by Chuck Moore - his disdain for “complexity” (which implies dependency on large teams of people you ought to manage, annoying constraints imposed by systems made by someone else and other things nomads don’t like), his firm opinion that distinct projects should have distinct code bases (customizability and “reuse” imply complexity and otherwise reduce the chances to “hermetically close” and truly complete a project), etc.

Proper Fixation always had more unfinished drafts than posts, but recently it’s getting ridiculous. I do have a couple of drafts I seriously intend to finish (usually the drafts which don’t make it to posthood during the first 4 hours or so go to the eternal drafthood land.) Until I’m able to think this stuff out to the point where I can share the results of my thinking, I figured I could share the far less scarce resource of Wisdom with ya.

***

Since I’ve violated the Golden Rule of Helping Friends with their PC Problems and attempted to help a friend with his PC problem, expectedly wiping out his hard drive in vain, I had many opportunities to explain the Programmer Paradox: how can a programmer fail to make a computer do as he wishes? While the difficulty of debugging a program without the source proved hard to explain to laymen, I think I’ve found a metaphor that does a good job. A programmer is to the blue screen of death what Mikhail Kalashnikov is to a loaded AK-47: just as helpless a victim as any other mortal, except for having a profound understanding of the mechanisms of his execution.

***

I would like to get some statistics on file encryption. For example, of all the files on the planet, X% are encrypted. Of all those files, Y% will never be read by someone due to encryption. Of all those files, Z% will never be read by malicious intruders. If I could lay my hands on the value of just one of these unknowns, I’d pick Z, because at least 100-Z% of the files will never be read by their owners. I would bet on Z lying somewhere between 0 and 1.

***

One of the key traits of good code is the ease at which it can be modified. One of the key traits of bad code is the high cost of modifying it. So good code is likely to deteriorate until it’s bad enough to become hard to change, and bad code is likely to stay bad. In short, code has a strong tendency to end up bad.

This can sound worthlessly pessimistic, similarly, for example, to “It is easier to break a leg than it is to cure it, therefore, most legs end up broken.” However, I think it’s more analogous to aging - the accumulation of changes in an organism, observably causing most animals to end up dead. Similarly, code that is used will be changed, code that is changed will degrade, and code that degrades beyond a certain point will die.

***

Health tends to be simpler than disease. For example, everybody can brush their teeth but few people can treat cavities. Similarly, it’s not very hard to maintain a sane development environment, but pretty hard to deal with the tide of bugs and of long-living branches resulting from a failure to do so. However, I’m generally optimistic about the chances of such cavities to be treated, and as usual, the optimism is based on the pain they cause - a strong incentive to seek and reward treatment.

***

There’s this evolution vs Intelligent Design debate. Well, I don’t know about life on Earth, but I sure have hard time believing in Intelligent Design in software. Code has to repeatedly survive exposure to users upon whom its fate depends. Yes, “users” can be a set containing just the author, but only if it’s honest-to-God USAGE, that is, the author has to pay a price when the program is hard to use - like not getting important things done properly. Show me a program that someone finds useful and that wasn’t subject to such evolutionary pressure, but rather was Intelligently Designed as useful.

I think that my intense hatred of the word “design” has to do with its prominent place in the speech of software creationists. These people are likely to constantly complain about not having enough resources to do The Right Thing in the ugly real world. They are also likely to give you software that you hate enough to wish to kill them, and be articulate enough to convince you that the problem is at your end, and fail to notice how this latter ability quadruples your desire to slash their body into square millimeter pieces.

***

I’ll conclude with an off-topic request: if you know a good text advocating a collectivistic or other kind of heterodox approach to economics, I’d be very grateful for a reference. By “advocacy”, I mean a text for laymen expressing support for a certain set of policies (as opposed to merely criticizing the effects of existing policies) - like Milton Friedman’s “Capitalism and Freedom”, for example.

This is going to be a bit atypical, because I’m going to talk, like, about organizing large teams of programmers. Which I rarely do, for the simple reason that it’s not my problem. I’m not a manager, I don’t think I’m likely to do a particularly good job as a manager in the near future, and I don’t want to be a manager. As far as I’m concerned - if your problem is organizing lots of people, you brought it upon yourself. So this “internal free market” thing, which tends to work well according to my observations, is an exception to my general rule of not making or thinking much about “organizational” observations.

So, free markets. Basically a way to create incentives (because you have to compete) at the cost of redundancy (because of duplicated efforts by many competitors). A redundancy vs dependencies issue, if you like - several competitors means less dependence on each - and since I generally think redundancy is a fair price to pay for removing dependencies, you can guess that I’m leaning towards free market fundamentalism.

At this stage I’m skipping the detours where I’m dragging exciting pseudo-related stuff into this, like the subprime mortgage crisis and the enigmatically overwhelming support for Barack Obama by top programmers and tech bloggers. I’m skipping that to get to the simple point of there being no good way for an employer to create incentives for programmers with money.

How exactly this trick fails to work, and what kind of LOCs you get when you pay by the LOC is out of my scope. If you’re sincerely surprised, there’s lots of material for you to browse - Econ 101 Management being as good a place to start as any. The single thing I have to say about the “financial incentive” method here is that its failure isn’t at all surprising to the free market fundamentalist.

In a free market, people solve their own problems, and (pay to) use the output of other people only when it helps them solve their problems, in a completely distributed way. Setting prices for things and subsidizing them is the trademark of a centralized, controlled economy. Now, does an employer paying by the LOC or by 1/#defects or whatever create an “internal free market”, or an “internal government bureaucracy”? Without looking at soft stuff, like a hypothetical offense to sacred engineering values carried by the act of creating incentives, we can see that subsidizing LOCs will create surplus LOCs, just the way it works with agricultural surplus and everywhere else.

What would we do if we wanted a real internal free market? “Free market” means that we want to have people solving their problems, and let them “pay” each other in some way to solve them, without us controlling the latter process. In our specific context:

1. “People” means “strong programmers” (or at least decent - or else why did “we”, the employer, hire them, dammit?) Folks who, at the very least, like to be productive and have their stuff used. Maybe they also “like to solve puzzles”, but not all do. For example, I hate puzzles, and have a strong preference for Alexandrian solutions. But you should still hire me.
2. For those people, “solving their problems” means delivering user-visible features. This is the basic responsibility of developers towards the organization, this is what the organization is capable of judging (it better be), and this is what links the whole operation to reality through the external market forces.

The only question is, what does it mean for a developer to “pay” another developer? Paying with money makes no sense, not with our definition of “people”. People who’re into those transactions tend to be self-employed. However, developers do have their own currency, karma points or whatever term you prefer to use for it (they are all irksome; economics is to life what proctology is to anatomy - it’s ugly because it’s true). I know two kinds:

1. Time. You can “pay” to developers, teams and the whole organization by volunteering to do particularly unsatisfying but important/urgent work.
2. Code. When someone uses your code, they are paying you (I repeat, it’s not you who are giving something to them; do not make this error or they’ll stop using your code.)

Time and code are not unlike gold and printed money, because you can’t make more time but you can make more code. However, proceeding with this analogy and trying to scale it to include inflation and such will expose my economical illiteracy and general lunacy to an extent making me want to stop now.

What we’ll do now is examine how “trading” time and code works in programming, and how it creates incentives to invest efforts into the most needed things similarly to the way prices do in free market economies. We’ll start with “trading code” - the less intuitive but the more fundamental kind of transaction.

You have to deliver something user-visible. The user couldn’t care less about the guts making up your program - how you parse things, what your communication protocols are, which optimized libraries you use for processing bottlenecks, etc. However, you do care about these things, because they are really needed for things to work. With all these things, you could reuse “infrastructure” others work on (for example, me), or you could roll your own (let’s ignore “international trade” where you use a third-party library for the moment).

To you, depending on me is a risk - who knows how many bugs I have, how well my stuff maps to your needs, etc. To me, on the other hand, having you reuse my code is the best thing that can happen to me during work hours. After work, better things can happen, in particular, those having to do with spending the money earned during work hours. But at work, the best thing that I can do is be productive and have others use my code. Lots of users is the workplace fortune equivalent to being rich in the real world. Do you see who pays who here?

What can an organization do to manage these infrastructure transactions?

1. The “economical”, “capitalist” solution: leave them alone except for securing them. “Leaving them alone” means not controlling them - not mandating the development and reuse of infrastructure and not assigning workforce to it. This means that by making my modules reusable, I’m only trying to please my internal users, so I’m likely to (try to) invest most effort into what they find important and helpful for doing their ultimate job. “Securing transactions” means something similar to the way public companies are forced to expose their accounting. If something becomes reusable code, it ought to have proper documentation, versioning, etc., and the organization must make sure it does.
2. The “political”, “socialist” solution: assign the task of developing a parser, an optimized library, etc. to a person/team - subsidize the parser (the price to a user is now lower - even if the parser is all buggy, a person is officially assigned to fix the bugs, and the responsibility for failures moves to that person and not to the one who decided to reuse the code). This means that the parser will be created even if in a “free company” nobody would want to develop and maintain it, knowing that most people wouldn’t take the risk of using it for the benefits it provides. Leading to surplus crops of parsers.
3. A further improvement on 2, the “communist” solution: force everyone to use The Parser. This means there are no “economical” means to punish the author whatsoever - where “punishing” means “not paying” and “not paying” means “stop using the code”. However, there’s still hope: you have political means to punish the author. For example, poke fun at the goddamn nightmare infrastructure, yell at the author, yell at his manager, ask your manager to yell at his manager’s manager - a whole slew of counter-infrastructure measures. Victims of infrastructural communism use them all the time.

So this is how “trading code” is (more accurately, “can be”) a better way of evolving reusable “infrastructure” than centralized planning. In general, the only thing I’m discussing is the reusable stuff - that’s what organizations can optimize (or pessimize, creating useless “reusable” modules and not creating the actually needed ones). Nothing can be done about things which aren’t reusable by definition, belonging to a single “feature”/”project” - those will have to be written once and only once no matter what.

What’s wrong with this picture? Could be many things, but one thing I’ll talk about (because I have a good answer for it) is the problem of “instant gratification”/”disruptive changes”/”local optimums”/etc. There are grand things that just can’t be done by small incremental changes, the by-products of work on “specific features”. You really need a person/team assigned to these things. This is somewhat similar to economies of scale which can be achieved by purchasing expensive machinery. How are many small farmers/shoe makers/etc. going to raise money for that machinery without central planning, if they’re all busy with their small short-term profits?

This is where entrepreneurs come into play. Entrepreneurs are people with fire up their asses. Normal people want enough money to get by, enough money to not worry about money, or enough money to not have to work for money. Entrepreneurs want more money than they can sensibly spend during the decades of their lifetime. And they want it because they desperately need that money to feed the fire raging up their asses. When they see a potential for making truckloads of money, many of them are willing to put their own savings on the line to chase that chance.

This psychological profile is a speculation of mine - my best attempt to comprehend the inexplicable behavior of making efforts and burning nerve endings to make more money than you could possibly need. However, I do have motivation which is quite similar in the context of our “economics of programming” analogy. I’m a “programming entrepreneur”, or at least I have, um, the same trademark proctological fireworks. I’m thrilled by opportunities to make stuff that, like, everybody will use, everything will depend on, …and everyone will want a piece of me when it breaks - so? It’s still worth it.

I can’t make such stuff as a by-product of working on something reasonably user-visible. I need to be assigned to it. What are the savings that I can put on the line? Time invested into doing unsatisfying, but important work. I call my own way of making these deals “buying development with debugging”. I’m usually willing to debug the weirder of the urgent problems, although it’s not much fun by itself, because it translates to a lot of karma points. I can then spend those karma points by working on what I want 80% of the time, 20% being the continuous urgent debugging tax.

Again, there’s more than one way for that kind of “entrepreneur” to start a programming venture:

1. The “economical” way - spend my own time implementing my ideas. Like a “real” entrepreneur putting his savings on the line and forced to look at his company bleeding that money if it doesn’t take off, I will want to stop as soon as possible when I realize that I’m failing. Those so-called “organizational karma points” you gain in the trenches have better uses than wasting on the development of worthless stuff nobody will use.
2. The “political” way - convincing “the government” (a manager) that my idea is worth implementing, and have someone assigned to it. Now nobody wants to admit the failure early on. I’m not losing anything when someone else struggles with the implementation - “I could do it better”. The person working on the thing isn’t really held responsible for the failure, either - not his idea, so why not keep trying to make it work? Everybody wants to make his stuff work and be used, after all. And the manager won’t want to admit the failure because of all people, he’ll get most of the blame. Therefore, the worthless effort will not be stopped for a lot of time.

Free market supporters are sometimes blamed for disrespecting people and reducing human nature to primitive egoism. Well, the only thing I can say is that I sure am a Good Person (how could it be different?), I respect myself lots, I successfully “launched” more than one “programming venture” both ways - “economical” (DIY) and “political” (persuasion), and of each of these two kinds, some succeeded and some failed.

And believe you me, deep down I refuse to take responsibility for the failing “politically launched” projects even now when we talk about it. On the other hand, the “economically launched” failures are - seriously - the best thing that happened to me in my professional life. I attribute most of my occasional successes - or, more accurately, non-failures - to lessons learned from the DIY failures, which I had no choice but admit responsibility for. (Damn, that was painful. To the extent that wasn’t on my job description.)

Now, I’m not an “internal free market fundamentalist”, simply because I know much more about programming than I do about economics, and obnoxious/oversimplified opinions usually correlate with ignorance. However, my experience seems to show that “internal free markets” are healthy enough to sustain continuous improvements on many scales, and eventually punish both “greedy” “instant gratification” techniques of pleasing managers/customers and architectural masturbation, promoting solid work.

And if you’re not a manager (I mostly care about non-managers, guys and gals like me, you know), I think this quasi-economical angle can contribute to your ability to look at some young initiative around you and say “Hm, this might work out” and conversely “Epic fail on the way, I’m not going to touch this with a laser pointer, man”. So, FYI.

I’m depressed. What I’ll do is I’ll tell you the 3 cardinal programming jokes. And if it helps cheer me up, I’ll consider my job well done.

I must warn you about those jokes. Firstly, they are translated from Russian and Hebrew by yours truly, which may cause them to lose some of their charm. Secondly, I’m not sure they came with that much charm to begin with, because my taste in jokes (or otherwise) can be politely characterized as “lowbrow”. In particular, all 3 jokes are based on the sewer/plumber metaphor. I didn’t consciously collect them based on this criterion, it just turns out that I can’t think of a better metaphor for programming.

By the way, I was recently told by a very strong programmer that of all things, he wanted to become a plumber as a kid. ‘Cause it was very interesting to him, the tools, the pipes, how you make the whole thing work. And then he felt he understood enough of it, so he figured he’d become a programmer instead. And now he is, and he has enough (virtual) pipes full of (virtual) shit to keep him curious about how to make it work for the rest of his life. By which I mean to say, hey, it’s not just my bad taste, it is a good metaphor, see?

So, the jokes. Lowbrow, depressing stuff. You have been warned.

Expanding your skill set

A very important thing. You should be learning stuff. Yada yada.

With many things though, people have this strange tendency to avoid knowing them, and instead ask someone else unfortunate enough to already know them. Say, Makefiles. Is it just my experience or do people worldwide pretend to be incapable of dealing with a hairy Makefile, and leave its regularly scheduled tweaking to a small set of knowledgeable victims?

Or debugging of the lowest kind, with race conditions and creative memory corruption. People like to give up on that, as long as someone else can take over. “I just don’t know how to proceed”. Right.

Sometimes I wish I could put this claim to a test. Check if they’d say this at gunpoint. Or, more humanely and therefore much less cheaply, propose them $1M if they do know how to proceed. I bet they’d think a bit harder. If you’re working on AI, specifically on preparing it to the Turing test, don’t forget to teach it this principle, or else it has no chance of passing for a human.

I find that the following describes the double-edged sword that is skill set expansion quite well:

A plumber and his apprentice pay a visit to a manhole requiring their attention. The plumber goes down the manhole, and the apprentice stays above with the toolbox. The plumber asks for wrench #3, and the apprentice puts the wrench into his hand. 2 minutes pass. “Wrench #5!” The apprentice finds the wrench and passes it to the plumber. 5 more minutes. “Wrench #6!” The plumber is given that, takes a couple more minutes and finally comes out.

The next scene should really be a small piece of pantomime, but I’ll have to get by with words alone. Not unexpectedly for this type of joke, the plumber comes out with his arms covered with excrement. He slowly sweeps his right hand over his left arm, then the left hand over the right arm, shakes his hands and reaches for something to wipe them with. And to the apprentice he says:

“Watch and learn, son, or you’ll be passing wrenches for the rest of your life”.

Really, you should learn things. Expand your skill set. Who wants to be passing wrenches?

Layers of abstraction

Abstraction is good. Or should I say legitimate. Or should I say inevitable. I mean, you have to count on something. Something has to work, because you can’t build things on top of nothing.

Except it won’t work. That something you build things on top of won’t work.

What’s that? “Whining”? Yep, definitely. This here is whining.

Whining is good. Or should I say legitimate. Or should I say inevitable. Because if you aren’t allowed to whine about frigging data channels which drop chunks of data and duplicate chunks of data because some fucking hardware subcontructor couldn’t be bothered to implement arbitration for shared data access, if you aren’t allowed to whine about that…

If you aren’t allowed to whine about that, you should be allowed to whine about memory, which flips bits, and zeros bytes, and it does so once per hour for some weird sequence of accesses having nothing to do with the address where data actually changes. Fuck that, OK? Fuck DDR2. Fuck its controllers and the zillions of their configuration parameters.

A plumber climbs out of a manhole, this time without a preamble, and his arms are covered with - guess what? - excrement! A beautiful little girl in a beautiful white dress happens to pass by. The plumber seizes the opportunity and (another piece of pantomime) quickly, but firmly sweeps his hands over the girl’s white dress.

Little girl (appalled): AAAH!!

Plumber (outraged): Oh yeah? I bet you love to take a shit though.

Yep. You love to allocate objects in memory, don’t you? Megabytes of them. And then a board designer decides to wipe his filthy hands with your beautiful white huge software system. Debug that, you perverted memory-addicted individual.

Taking pride in your work

And still, I actually like my work, on a level. Why? It feels inherently cool to design stuff that becomes this bunch of tiny parts, transistors and all, switching hundreds of millions of times a second, and then to write code that manages all the flying circus.

I know people who feel the same about computer vision. People for whom it’s a personal priority to work on computer vision, where they are given images and they look for stuff in them. Who wants to be doing that? Who wants to be responsible for the solution of a problem that can’t even be precisely defined? Me, I wanna be doing hardware.

What do I actually do most of the time though? I eat hexadecimal. I sit near a debugger, and I keep hitting Page Up in a memory view window, to find the beginning of the array that overwrote this piece of data (I guess the element size from the repetitive patterns and such), and along comes a computer vision geek and he says, “damn it, man, you got out of the Matrix!”

Well, I dunno, I find it much easier to guess what buggy code did to my memory than to find out “why” an algorithm thinks this here is a person when in fact it’s a shade of a tree. Because if you look closely at the pixels, the shade kinda looks like a person, but of course we could reject it based on its motion, but of course that would mean we’d approve these reflections over here based on their motion, but, but, but…

What my bogus example is saying is that you have lots and lots of cues but each can work both for you and against you, and now how do you weigh all that, without even a formal spec? I’d rather eat hexadecimal, thank you very much.

And we look at each other, and sincerely think that our jobs are pretty nifty, but the other guy’s job is awful and how can he be doing it. And I suspect that if one looks at this from aside, one might wonder where the actual fun is, because there is actual fun in here, or so all the participants testify. And I think I know the answer.

An airplane lands, and passengers come out. One of them notices a guy underneath the airplane. As you’d guess, the guy is a plumber. The plumber touches some lock, and immediately gets covered by excrement streaming from an opening at the bottom of the plane.

The pantomime cleanup routine follows, and then comes the turn of the dialog.

Passenger (appalled): What on Earth makes you keep this job?

Plumber (proudly): Hey, I’m in the aerospace business!

The aerospace effect happens to different people with different things. With some, it’s “Hey, I’m making real hardware!” With others, it’s “Hey, I’m finding real objects in real images!” It’s a good thing people are different, because so are the currents of excrement, and someone ought to swim in each. We can’t all be passing wrenches.