The nomadic programmer

There's this broad metaphor I have, with no conclusions attached - just an attempt to describe dynamics. I've recently shared it with the commie ex-VP school teacher and he liked it, so I thought it could fit well with the other borderline stuff I host at Proper Fixation under the "wetware" category.

So. One recurring theme in the history of civilization is the conflict between nomadic and settled people. Nomads think that land is for feeding cattle and you move elsewhere once there's nothing left to graze. Villagers figure that land is for growing food, so you settle on it and fertilize it and irrigate it and stuff. Initially, nomads typically dominate the landscape, periodically attacking the settled villagers and taking their crops. However, the settled people eventually accumulate enough surplus to support cities, nation states and standing armies, extending their control to more and more lands and eventually exterminating the nomadic lifestyle altogether.

The way I painted this picture, I tend to side with the hard-working settled folk, the nomads being the parasitic losers I've depicted, and I think most of us civilized humans share similar sentiments. However, in my metaphor I side with the nomadic programmer, at least to a large extent, and I do so because of the meaning my metaphor assigns to "land".

The thing I find analogous to land in programming is problems, because that's where programmers live. Programmers live on (in?) problems in the sense of dealing with broken things most of the time - once something starts working, you move on to something that doesn't. In another sense, large problems or problem areas a programmer deals with define that programmer's territory. The programmer is in immediate demand to the extent that solutions to "his" problems are in demand; problems feed programmers. Strong programmers seek, in one way or another, to expand their responsibility to encompass more problems, and to preserve their existing responsibilities. And so on.

Now if we restate the respective worldviews of nomads and settlers in the terms of this metaphor, we'll get this. Nomads think that problems exist for solving them and you move elsewhere once there's nothing left to graze. Settlers think that problems exist for growing them, so they settle on them and fertilize them and irrigate them and stuff.

And now you can see why I'm inclined to sympathize with the nomadic programmer. Two other things fueling this sympathy are issues of personality to be discussed soon, and the fate of the nomad to be discussed immediately. And while the nomad is no longer the parasite, rest assured that he's still, in the long run, the loser.

Initially - in a young and small organization - nomadic programmers tend to dominate the landscape. There are more problems than people around. The nomadic programmer travels from one urgent problem to another, grazing through them as fast as he can. Occasionally he stumbles upon a settler who has settled on a problem near the nomad's territory and grown crops of code there. Well, if the problem occupied by the settler becomes urgent, or if the crops stand in the way of solving the nomad's adjacent urgent problem, the nomad will go ahead and brutally solve the settler's problem, wiping out his crops. The politics of the invasion will be trivial - a promise to deliver by the nomad carries lots of weight at this stage and the settler will not issue a counter-promise (to deliver in his own way) because he's a peaceful code-growing villager who isn't into stress which necessarily comes with delivering quickly.

However, the time goes by and sure enough, the settled people accumulate quite some surplus. What you grow on land is surplus wheat; what you grow on problems is surplus code. Code that wouldn't naturally grow on a problem - but now that the problem was fertilized by the original settlers, they've grown enough code on it to support whole cities, a nation state, and a standing army of programmers, all making a living by fiddling with this code.

The nomad starts running out of pasture. Sure enough, there are lots of problems just like there used to be. But you can no longer solve them because (1) now it's the majority and not a minority of problems that are already owned by someone (growing them rather than solving them) and (2) in most cases invasion is no longer an option. Now that the problem is owned by a nation state, responsible for lots of code and with lots of people working on that code, the nomad's promise to deliver quickly carries very little weight compared to the danger of irritating the sovereign. While it is quite likely still true that a nomad will probably deliver more quickly than the whole nation state team, the nomad will not be able to take over the entire responsibility of the team. (It is possible that the single reason for the latter is the problems grown by the team itself and that a few nomads could in fact handle the original problem. But it is irrelevant since problems that could have been avoided are no less real than problems that couldn't.)

So if the organization, by some decision making mechanism, lets the nomad invade the territory of the settled team and solve the stupid problem, and then the offended team, by some decision making mechanism, fights back by effectively going on strike, there is nothing the nomad will be able to offer the organization at this point. Of course it doesn't have to come to this, just like political conflicts don't have to come to full-scale wars, or personal conflicts to fist fights or court hearings. It's enough for the worst case scenario to be likely to work out in favor of A rather than B to shift the balance decisively in favor of A. Even if neither A nor B nor anyone making decisions affecting A and B actually think in terms of this scenario, things tend to evolve and adapt such that decisions are made in favor of A. And in our case, the nomadic programmer is B.

Solving problems just isn't the big thing in this organization anymore, just like the quality of life experienced by the inhabitants of some territory isn't the main theme in international politics. Perhaps there are ways to improve the quality of life in Siberia, however this is not nearly as important politically as the fact that there's already a guy exclusively responsible for the quality of life in Siberia. Perhaps Socialism with Chinese Characteristics could yield improvements in the lives of Siberians that Managed Democracy could not, however, if the Chinese try to act on this assumption, there will be a nuclear war. If what remains of a nomadic tribe somewhere in the region makes a similar attempt, then it will remain no more.

The disgruntled nomadic programmer reduces his ambition to merely being left alone to wander the remaining wilderness. However, this option is no more real for him now than the option of being left alone was available to the settler in the old days. Back then, the settlers were never safe since a nomad could always bump into them in an attempt to solve a related problem, and if their stuff got in the way, he'd rewrite or delete/disable their stuff. Now it is the nomad who is never safe since the nation states keep expanding their responsibilities into neighboring problems - having enough people to have some of them free for that some of the time.

(Actually having even partially idle workers on a team leaves few satisfying alternatives to an attempt at expanding the team's responsibilities since other teams are always happy to seize an idle worker. Likewise, back in the old days the nomadic programmer had few satisfying alternatives to invading and solving others' problems since otherwise he couldn't keep his promises to deliver. It's not (just) the intentions that fuel wars, it's (also) the situation.)

The nation states seeking to expand won't fight each other since the nomad is a much easier target, not having resources (time and reports) to look over his entire territory. Once a nation state team managed to take over some of that ever-shrinking territory, the nomad will never gain it back. Increasingly, the nomad has to reach compromises with neighboring nation states whenever his work is related to their work. Then it turns out that in order to be able to work on what he wants at all, he has to do it the way a chief commander or an officer of a nation state team wants him to do it - and then that in order to work on anything at all, he has to report to such a manager.

At this point the nomadic programmer can use his reputation and seniority to get pseudo-promoted to a non-productive position. Alternatively, he can actually become a report of a nation state team manager with whom the relationship is likely already strained - and his seniority, reputation and ambitions won't make the transition into this particular position of a report any smoother. Alternatively he can quit. His failure is now complete.

(It may sound like a natural thing for a nomad to change jobs fairly frequently - part of a lifestyle rather than the failure of that lifestyle. However, nomadic programmers are those who like to travel from problem to problem - not from job to job; some like the latter but some don't. A new job at a new place means a temporary, but possibly significant loss of confidence and efficiency. An African nomad won't necessarily welcome a relocation to Alaska.)

As I've said above, I have reasons having to do with my personality to side with the nomadic programmer, especially at the stage of mounting pressure from nation state teams. The people I tend to relate to most easily seem to be those who prefer freedom to power. A talented freedom-seeker with a strong sense of responsibility will accumulate, well, responsibilities much more quickly than reports - a lot of territory to wander, and no standing army to protect it. (The problem with reports is that you take their freedom by telling them what to do and they take your freedom through your responsibility for their actions; who wants reports?) Since many freedom-lovers disdain politics, they won't respect international borders - a problem should be solved, dammit; hence they're likely to initiate invasions.

However, while this means that I personally will tend to find myself sympathizing with particular nomadic programmers, this does not mean that theirs is the right way or something. For example, it is unclear which share of programming problems out there can really be "solved" - grazed through and left alone - and which problems actually require continuous care and gardening that a true nomad is not likely to supply. Also, whether there's a "solution" you only need to "maintain" or an "infrastructure" you want to "extend", the code needs a permanent owner. I don't believe in collective code ownership any more than in collective ownership of anything else - what it usually means is that everybody collectively fights over something. Therefore I think that ownership should generally be respected, and so a compromise which is, from a technical viewpoint, quite moronic, can otherwise be a great thing - a belief outside the nomad's way.

So while I know where my sympathies lie, I don't know which camp I'm in and this is why this metaphor doesn't come with any conclusions, just the dynamics. In fact I'd rather leave it without conclusions but I wouldn't mind expanding more on the dynamics. For example, some - but not all - settled civilizations were actually started by nomads enslaving argicultural villagers and settling among them. Apparently a similar distinction can be made between nation state teams of programmers; it is then interesting whether differences in their behavior can be traced to their different origins. Perhaps a person more entertained than appalled by the sort of perspective on the adventurous lives of programmers here presented is also the kind of person more entertained than appaled by the history of mankind in general and so could help develop this line of thought based on his knowledge of history. Could be fun.

Update (2009-08-18) - Chuck Moore: "I’ve met too many people who want to make a career out of a project instead of completing it" - the nomad's view of the settlers. Nomadism is apparent in other writing by Chuck Moore - his disdain for "complexity" (which implies dependency on large teams of people you ought to manage, annoying constraints imposed by systems made by someone else and other things nomads don't like), his firm opinion that distinct projects should have distinct code bases (customizability and "reuse" imply complexity and otherwise reduce the chances to "hermetically close" and truly complete a project), etc.

11 comments ↓

#1 WesleyD on 08.06.09 at 1:58 pm

Beautiful essay.

How could anybody *not* identify with the nomad though?

#2 Jonathan on 08.06.09 at 3:48 pm

This reminds me of "mappers vs. packers" - http://www.c2.com/cgi/wiki?MappersVsPackers - like anyone is going to say "YES! That's me. I'm a PACKER."

#3 arakyd on 08.06.09 at 5:45 pm

Interesting metaphor. Reminds me of a comment I wrote on Hacker News once in defense of Mel of Usenet legend. Everyone knows the part of the story where he does weird stuff that no one can maintain, but most people overlook the part where his unmaintained code continues to work for years after he leaves the company. Definitely a nomad.

I think that, at scales larger than an organization, there are very few problems that can be solved and left alone for very long; most of the real nomads are doing math or logic or maybe protocol design (not that all of their output is useful, or even most of it). Linus Torvalds, for example, has lots of reports, and when he goes off and bangs out a solution like git someone still has to maintain it. Even someone like Chuck Moore, who would seem to be a nomad if there ever was one, is cultivating a (relatively small and slow to change) set of tools, and even he has some reports. He is doing his own thing, but that thing is pretty much to iterate his tools. (He's said that he is getting bored of doing hardware, but so far that's still what he's doing). So that's not exactly being a nomad. I think that if you are ambitious, and you don't want to be an academic, then you can't really be a nomad.

Long lasting solutions can only be found/created in stable environments. Mel was writing for hardware that wasn't going to change quickly. Maybe most people are relative nomads in the embedded space. It's all relative. In most companies, nomads are working on top of several layers of settled civilizations which will eventually erode his solutions unless they are good enough and/or early enough that the company civilization will build around them; even then, the solution will go away when the company dies.

Going back "up" the ladder, the same sort of pattern seems to hold in academia. Most people start as nomads, and the many transition to becoming settlers right about the time they get tenure.

Its hard to tell ahead of time what solutions will really last a long time. So far it looks to me like pretty much all long lasting code has to be cultivated by settlers to one extent or another. It would be interesting to do a survey of long lasting projects and companies to see what the oldest code looks like and find out how well it works.

#4 Yossi Kreinin on 08.07.09 at 7:13 am

@WesleyD: I guess there can be plenty of reasons. For example, you could claim that the nomadic lifestyle doesn't scale to handle problems where a lot of people have to cooperate and that such people get in the way of the more responsible people trying to build an organization that can deliver consistently or something. That would be debatable but you could argue along these lines and if your character is that of an empire-building hierarchy dweller you would definitely like arguing along these lines, or in any other way making you look good and you'd be entitled to your opinion; it's not like this personality type is inherently evil or something, and in fact they can get great things done under the right circumstances.

@Jonathan: The c2.com page gives the impression that "Mappers vs Packers" is supposed to be polarizing but somehow I don't feel that; in programming there's plenty to "map" (every PL has function calls) and plenty to "pack" (Python keyword arguments declared with **args land into a dictionary called args which CL keyword arguments don't). What I think programmers should be doing well is "driving" through their mental maps; a driver doesn't have to know a city but he should see everything happening in a junction - which turns could be taken and where other cars could come from and what should happen then, likewise a programmer should know the territory not in the sense of thinking of all details simultaneously, but in the sense of being able to see everything related to a particular detail simultaneously. This isn't precisely "mapping" in the sense that loads of things just won't compress away but also isn't precisely "packing" because there's structure, a map, right? But some "packing" seems to be going into this. So, I dunno, perhaps it comes from lack of understanding, but I have no trouble to say, "That's me, I'm a packer".

arakyd: "I think that if you are ambitious, and you don’t want to be an academic, then you can’t really be a nomad" - spot-on.

(A minute of silence to honor the tragedy of those trying.)

Mel wasn't (just) a nomad though; he was a bastard, and the extent to which his code "worked" is debatable - a program that can't be updated or configured to deal better cards to the human player than it gives itself sucks, doubly so if it's one of the requirements. Using a lower level of abstraction than needed to entertain yourself and encrypt your code at the expense of your employer is evil, likewise general hostility to coworkers is a trait nomads needn't have, in fact plenty of them are very helpful and expand the territory they wander on during their expeditions to help someone or other.

Regarding the lifespan of code: I think the average is 7 years or so, and I'm perfectly happy with that (some of the things I do end up as embedded chips and their lifespan - the years through which new projects are targeted at them, after which they are only manufactured when more units of a finished product are needed - is 5 to 7 years and Moore's law guarantees it won't be larger). I don't dwell on the projections of immortality of a few code snippets from System V surviving in the Linux kernel which is supposedly forever; the mortality of my code is the way of life just like the mortality of me. At this lower level of ambition though, the trouble of a nomad isn't that his stuff would eventually erode away but that it wouldn't be made/used in the first place because the problem space is all occupied by settlers.

#5 Vladimir Levin on 08.09.09 at 5:49 am

I do like this colorful analogy, but I wonder if it comes down to the simpler, and I believe relatively well-understood, generalist vs. specialist dialectic.

#6 Yossi Kreinin on 08.09.09 at 9:11 am

@Vladimir: If settled means specialist and nomadic means generalist - I think not. Just like a nation state, a settled team can seize territory with very different landscape and climate, while a nomad can actually prefer problems from the same domain. Or vice versa.

#7 Kragen Javier Sitaker on 08.23.09 at 11:47 pm

This is one of the most insightful essays I've seen on programming in a long time. I think it explains an enormous amount about the relative popularity of Scheme and C++, for example.

One point: it is often the case that the settlers eventually produce better software than the nomads did, simply because they are putting two or three orders of magnitude more person-hours into a given problem than the nomads did. And that is why we aren't all programming in ColorForth and Oberon.

#8 Yossi Kreinin on 08.25.09 at 9:23 am

@Kragen: Yeah, the trouble of 10 programmers, each 10x more productive than a competitor's programmer, is that it's the competitor who is 10x more productive than them by virtue of employing 1000 programmers.

#9 Ofer on 09.22.09 at 11:14 am

The start you refer to is quite late actually, when I think of start I imagine nothing, I mean real nothing - so you need some villagers - which is an additional type. The villagers plant the seeds and help them grow. They don't have time to build big cities, and usually this is not what they are looking for (if they would they'd probably wouldn't come to this desert) instead the find new places to plant their seeds and there is a lot of place to do that.
The era of the villagers ends when more people come to their fertile village, people who don't find place to seed plants or don't want to - they either want to establish themselves, these are the settlers.
The nomad usually afraid of taking responsibility, not in new fields or in an existing one - he just arrive do whatever he think is right in order to reach a very short term target.

Bottom line - yes, there are settlers and nomadic programmers and both are bad for the organization since their motivation are political - settlers want to be needed as they control a territory and nomadic want to be the hero of the day.
The people who actually good for he organization are the villagers who really solve the problems the organization is facing in a productive way.

#10 Yossi Kreinin on 09.22.09 at 10:58 pm

That you think there's a third way which is unquestionably good while the other ways are unquestionably bad doesn't surprise me much, likewise you wouldn't be surprised by my disagreement… For one thing, I'm positive that the early villager and the seasoned problem fertilizer are the same people at different points of their lives. As to the presumed short-sightedness of the nomadic approach - not necessarily so, I find; they can alternatively aim at a closed solution that can last long, it's just the period when they work on it that is short. The C/Unix design style vs the C++ design style or at their simplistic projections along one possible dimension exemplify the schism to me (Unix gives you closed programs in C you can use, C++ gives you open frameworks that tend to grow as time goes by and that you can yourself extend).

#11 Ofer on 09.23.09 at 12:48 am

Nomadic programmers and settlers can do good thing but they'll do that only if they'll forget their type.
If a nomadic programmer put some time to make his solution to last longer than the current problem he need to solve - he is less nomadic at this point.
If a settler programmer solves a required problem in a simple and fast manner while bending the current software structure - he is less settler.

Leave a Comment