Engineers vs managers: economics vs business

When chased by a bear, engineers want to run faster than the bear, managers want to run faster than you. This is known as "the best vs the good enough", and is a very common theme.

For instance, company A releases a good enough technology, company B releases the best technology on the market. B fails and A succeeds, because A releases earlier, or because A's technology is more compatible with the status quo, etc. Engineers will commonly feel sympathy for B, managers will applaud the shrewdness of A.

It's a common story and an interesting angle, but the "best vs good enough" formulation misses something. It sounds as if there's a road towards "the best" - towards the 100%. Engineers want to keep going until they actually reach 100%. And managers force them to quit at 70%:

There comes a time in the life of every project where the right thing to do is shoot the engineers and ship the fucker.

However, frequently the road towards "the best" looks completely different from the road to "the good enough" from the very beginning. The different goals of engineers and managers make their thinking work in different directions. A simple example will illustrate this difference.

Suppose there's a bunch of computers where people can run stuff. Some system is needed to decide who runs what, when and where. What to do?

  • An engineer will want to keep as many computers occupied at every moment as possible - otherwise they're wasted.
  • A manager will want to give each team as few computers as absolutely necessary - otherwise they're wasted.

These directions aren't just opposite - "as many as possible" vs "as few as necessary". They focus on different things. The engineer imagines idle machines longing for work, and he wants to feed them with tasks. The manager thinks of irate users longing for machines, and he wants to feed them with enough machines to be quiet. Their definitions of "success" are barely related, as are their definitions of "waste".

An engineer's solution is to have everybody submit their jobs to a queue managed by a central server. The Condor software is an example implementation; there are many others, with many subtle issues and differences - or you can roll your own. The upshot is that once a machine becomes idle, it can immediately yank a next job from the server's queue. As long as there's anything left to do, no machine is idle. This is the best possible situation.

A manager's solution is to give the ASIC team 12 servers: asic01, asic02, … asic12. QA gets 20 machines, qa01 … qa20, and so on. If you want to work on a machine, you ssh to it and you work. If you're an ASIC engineer and you want to use a QA machine, you can't log in. If users fight over their team's machines, they can go to their manager. If their manager decides the team needs more machines, he goes to upper management. Good enough.

The "good enough" is not 70% of "the best" - it's not even in the same direction. In fact, it's more like -20%: once the "good enough" solution is deployed, the road towards "the best" gets harder. You restrict access to machines, and you get people used to the ssh session interface, which "the best" solution will not provide.

Which solution is actually better? Tough question.

  • The manager's solution requires no programming or installation, and trivial administration.
  • The manager's solution requires no changes of habits (ssh is the standard).
  • The engineer's solution yields ~100% utilization, the manager's 50%, 30%, or 10%, depending.
  • The manager's solution will cause less headache to the managers, on average. Once in a while, you buy a team some more machines, and they shut up. The engineer's solution requires to set priorities at times of overload, and people will constantly argue about priority with managers.
  • The engineer's solution doesn't run things on machines that are already busy anyway. Users armed with ssh will tend to do this, possibly with horrible slowdowns due to swapping, etc.
  • The engineer's solution can provide the lowest latencies.
  • The engineer's solution makes it trivial to utilize new machines - no need to set up sessions, just submit jobs as previously. This is good - and bad: people will demand new machines instead of reducing the load.

So there are many conflicting considerations. Their importance varies between cases, and is very hard to measure.

"The best" solution is not necessarily the best - rather, it's what the mindset of looking for the best yields. Likewise, the "good enough" solution is not necessarily good enough - it's just what you come up with when you look for a good enough approach.

Obviously, portraying "engineers" and "managers" this way is a gross oversimplification, and most real people can look at things from both angles. I like this oversimplification for two reasons:

  1. It does help to understand and predict many common arguments and reactions.
  2. A person's perspective frequently does coincide with the title: engineers aim at the best, managers look for the good enough. Just the title is thus a good basis for prediction.

How does title affect judgement? There are arguments to the effect of "aiming at the good enough is wiser, and managers are in a better position to achieve wisdom", or vice versa. However, I don't believe either viewpoint is more correct than the other. Rather, they are biases. People in different positions acquire different biases, because they have different incentives and constraints.

The difference in incentives is that engineers are rewarded for success, while managers are rewarded for non-failure.

An exceptional engineer is unique and valuable, like a great chef - being an OK engineer is more like cooking for McDonald's. An engineer needs unusual success to be noticed. One reason such success is achievable is because he works within his area of expertise, on things that he understands. There's rarely a formal quality metric, but his deep understanding creates in him a sense of beauty that serves as his metric.

A manager is fine as long as the project doesn't fail. Most projects fail - if yours didn't, it's quite an achievement. One reason they fail is that there are a lot of ways to fail. Forget just one item in a lengthy checklist, and it won't matter how well the other 99 items are handled. People may not buy a car because there's no convenient place for a resting arm. A manager looks after a whole lot of items, usually without being able to understand most of them in any depth. He's inclined to look for simple pass/fail criteria.

If your success function is a continuous metric, you'll aim at the best. If it's a long list of Boolean values - V or X - you'll want "good enough" (V) at every entry. "The best" is just a costlier way to get a V, at a higher risk to end up with an X. The manager's outlook leads to binarization.

Another difference is that engineers and managers control different things - "One man's constant is another man's variable". In our example, the engineer doesn't decide how many machines to purchase or how to allocate them. If the number of machines is constant, what remains is to make the most of them. The manager, on the other hand, doesn't directly control utilization, and frequently doesn't know how to improve it. If utilization is constant, what remains is to properly ration it.

More generally, engineers tend optimize within constraints set by management, in part because they can't change those constraints. As to managers, they tend to settle for "good enough", in part because they can't optimize.

Which biases lead to more sensible solutions depends on the situation. However, there's a subtle reason to like the engineer's devotion to excellence, despite the manager's pressure for mediocrity. It's precisely when the manager is right in that excellence will not contribute to sales when the engineer contributes the most to users.

If an engineer's job is done so poorly as to make the product non-marketable, the product will fail and won't be used. When the quality along all dimensions passes the adoption threshold, every improvement along any dimension is effectively a gift to the users - who'd be using the product anyway. This is false where improvements contribute significantly to popularity, but true where they don't.

The improvements are inconsequential for business, but consequential economically. Economically, anything that helps people is good. For business, anything that creates profit is good. Often these definitions of "good" coincide, but sometimes they don't. In these cases, people who aim at excellence for excellence's sake help bridge the gap.

The manager's checklist approach - the business angle - ensures that the product is marketable, which is great because otherwise it won't get used. But the engineer's urge to optimize is closer to the really important goal: making the best of what we've got; this is the economics angle. So while it can lead to problems just like any other bias, it's likeable that way.

27 comments ↓

#1 Z.T. on 11.19.11 at 12:55 pm

"Engineers will commonly feel sympathy for A, managers will applaud the shrewdness of B." - you clearly meant the opposite

"asic01..asic12, qa01-qa20" - sometimes, the manager's solution is implemented on top of the engineer's solution (the user visible machines are actually VMs running on top of a cluster of real servers) and sometimes the engineer's solution is implemented on top of the manager's solution (each department's machine runs a distributed work client the queue for which is accessible to other departments). This makes things more complicated, both socially and technically.

"One reason they fail is that there are a lot of ways to fail." - "Happy families are all alike; every unhappy family is unhappy in its own way."

"engineers tend optimize within constraints set by management, in part because they can’t change those constraints" - necessity is the mother of invention. See Unix and C on the PDP-6. All art forms require constraints.

"When the quality along all dimensions passes the adoption threshold, every improvement along any dimension is effectively a gift to the users - who’d be using the product anyway." - companies frequently won't spend money on improving their product if they already have the best in the market. They will even buy patents for a better product and sit on them instead of building it, because that's the most profitable thing they can do. A corporation's job is not to build the best product - it's to make money. Only fear of a competitor who does not agree to hold back and make money will lead a corporation to invest heavily in R&D. Markets naturally devolve into oligopolies that collude (with each other and government) to maximize profit and destroy newcomers who wish the destabilize the status quo (through technology or simply by reducing margins).

"Economically, anything that helps people is good." - what is this definition of "economy"?

The best products possible, disregarding profit, can be made as public works, using public funds or donations by artisans chosen in a contest. Remove profit motive and leave only reputation on the line to ensure quality work. Unfortunately I don't know how to avoid corruption in the panning committee.

#2 Yossi Kreinin on 11.19.11 at 11:27 pm

Regarding A and B - thanks, fixed, I probably wrote it that way because it sounded naturally with first A, then B. It's part of the surprisingly hard problem of assigning letters such as A and B to one's imaginary entities…

Regarding one solution on top of the other - yeah, quite so.

Regarding companies not spending money on R&D - well, in that case I meant to whoever worked on features that are non-essential, sales-wise, before R&D spending was completely stopped. The extent to which desirable improvements won't happen for business reasons is debatable, and my guess would be much more optimistic than yours (I'd estimate a smaller extent of the problem), but almost everyone agrees it's non-zero.

Regarding the definition of economics - well, one definition of the subject of economics is "allocation of scarce resources to alternative uses", and "helping people" is a vague way to describe the stated goal of a "good" allocation. To define this goal, you need a moral framework; in the Anglo-Saxon world, two popular frameworks are utilitarianism (the greatest good for the greatest number) and Rawlsianism (favor the worst-off). Utilitarianism sort of leads to "pro-growth"/"free-market" policies and Rawlsianism sort of leads to "socialist" policies; "anything that helps people" sort of captures the common in their stated goals.

Regarding removing the profit motive - corruption aside, earning reputation is not fundamentally different from earning money, in the sense that there are still cases where something can be done that helps but will not significantly affect your reputation, so the public interest and the private interest don't coincide; in that way it's qualitatively a similar situation. Whether it's quantitatively better I don't know.

#3 anon e. mouse on 11.21.11 at 7:38 am

I think this is underestimating what engineers do.

Most try to resolve the tension between economics and technical perfection.

#4 noone on 11.21.11 at 8:08 am

Managers don't ensure their products are marketable. Most will lie directly to the rest of management about what they have achieved while subverting any real value generation at all by engineers. When you consider this propensity to lie and deceive, good enough is really just good enough to fool all the other managers, many of whom have no technical basis to even understand the products being developed.

In your example, the engineer's solution is the best in the long term. The 'good enough with lies' approach eventually leads to collapsed companies and failed products. However, by the time they collapse, the managers will be long gone.

Additionally it is unfair to compare the manager's solution to the engineers simply on the basis of the manager having unlimited ability to add more resources. This is just an example of 'might makes right'.

At the end of the day, managers can't ignore basic economics. At some point all the scheming, lying, and shortcutting always causes companies or products to collapse just as a gambler eventually goes bankrupt.

#5 Yossi Kreinin on 11.21.11 at 9:30 am

@anon: well, it equally underestimates what managers do, since they try to resolve this same tension, too (though I'd put it in slightly different terms); it's just the bare bones bias of the respective positions, at least the way I see it.

#6 Yossi Kreinin on 11.21.11 at 9:34 am

@noone: I don't think it's that bad, really.

Regarding "best solution for the long term" - as was mentioned above, there are also hybrid solutions; one reason is that both solutions I mentioned have their drawbacks in their straightforward forms.

Regarding "might makes right" - the manager in my example is trying to buy less machines just as much as the engineer, the question is who succeeds more in that and in helping people getting work done.

#7 so on 11.21.11 at 2:15 pm

Technical debt gets solved how?

#8 ghettoimp on 11.21.11 at 6:44 pm

Thanks, nice.

#9 Freek on 11.22.11 at 12:41 am

Question! Who builds this company’s products? You do. Engineers do. Not managers. They should be answering to you, not you to them! Who else has an idea like this man’s Coke machine? All right! Tell me. Better yet… can anyone show me?

#10 hb on 11.22.11 at 1:06 am

You forgot one thing: a manager's 70% solution will come back to bite him in the ass one year later when he wants to quickly extend/scale/improve the product and fails.

#11 Yossi Kreinin on 11.22.11 at 2:20 am

There's quite some anti-manager bias in the comments. Something to do with the 99% being 99x more people than the 1%?

#12 GJR on 11.22.11 at 5:15 am

Trite article. As an example: "Suppose there’s a bunch of computers where people can run stuff. ". Where? At your site, the customers, Amazon?
Engineers solve problems; financial as well as technical. Managers should help them do this.

#13 Scott Brickey on 11.22.11 at 6:34 am

I don't exactly see this fitting Apple's model… everything I've read indicates that they strive for excellent, from the management's acceptance criteria down to the engineer… exceptions to this goal may very well have meant that their products were NOT the success that they became.

#14 MHohlios on 11.22.11 at 8:25 am

@Scott

I think you might be passing up some important information about Apple. While they may appear to be polished on the outside, really look at their products and you will find they almost always launch a 70% version first. (i.e. no front camera, ipad no camera, no cut and paste, no multitasking). They always come out with a version 2 or 3 or 4 to improve on the last 30% but the market continues to move and they stay behind. It is the simplicity offered by not doing the full 100% that gains them users. Very few users need the full 100% and 70% seems to work just fine.

Apple seems to fit nicely into this 70% structure.

#15 gus3 on 11.23.11 at 1:11 pm

Managers may not understand, but it's their job to get good, knowledgeable people to advise them. Think Captain Kirk: he wasn't an engineer, he wasn't a navigator, he wasn't a radio operator. He knew enough of each of those to be passable, but there were clearly parts of each that were beyond his expertise. However, he could see the big-picture elephant, not just little parts of the elephant like the blind men in the poem.

That said, I know from experience that having a manager who does understand the product can be a blessing. I've worked under a couple such managers, and also under managers who were ignorant asses. One such ignorant ass told his subordinates that he had promised delivery of QA tests within two weeks–which would have taken the staff more man-hours to develop than they would have if they worked 24/7. They went to their boss's boss, the division VP, and explained the reality of the situation. The ignorant ass didn't last long after that, but it didn't matter. The old COBOL code just never worked right on Windows, and the company was bought out and ransacked a year later.

Oddly, the same thing happened where I worked with an excellent manager, but that was a strategy to get the product some marketing that it desperately needed.

#16 Yossi Kreinin on 11.23.11 at 11:07 pm

Seriously? You worked on COBOL code that shipped on Windows?

#17 Thomas Rushton on 11.24.11 at 12:54 am

COBOL for Windows? Hell yes. Haven't you heard of COBOL.NET? http://www.netcobol.com/cobol-compiler-for-net/

#18 Yossi Kreinin on 11.24.11 at 3:41 am

I've heard of it, but never met anyone who worked with it.

#19 Sud on 11.25.11 at 7:34 pm

Well said!

#20 gumaling, riza on 02.22.12 at 10:07 pm

this page is really good!
this one one made me realize the differences between managers and engineers..!
this one is funny!

#21 Kragen Javier Sitaker on 08.24.12 at 4:14 pm

You've nver met anyone who worked with COBOL on Windows? In my second "real job", in 1997, I worked for Keane on a Y2K remediation project. For sniffing out date-handling logic, we had a thing called the "Keane Assessment Tool", which was basically fgrep with a GUI, written in Micro Focus COBOL and only runnable on Windows. I quickly hacked together a replacement in Perl that I could run on the Suns, which let me do more code scanning in an hour than I could have done with KAT in a week. Unfortunately, my people skills (and UI design skills, I suppose) were not up to the challenge of getting other people to use it, particularly since that would probably mean billing the client for less hours.

#22 Yossi Kreinin on 08.25.12 at 1:50 am

How billing for hours works is something that generally escapes me, because clearly the client is asking for trouble by encouraging you to work for more hours than needed. I think one theme there is that clients know that something should take at most N hours and will get annoyed if you spend more time on it; and, if you're done faster, and it's not a one-off project, you're more likely to get the next one. But I'm really not sure how this type of arrangement should be made to work.

#23 shubham on 04.21.13 at 7:55 pm

Wow! Really really good article. The "good enough" vs "best" had me. I appreciate your approach of "gross oversimplification"

#24 Yossi Kreinin on 04.21.13 at 11:07 pm

Thanks

#25 iame on 05.02.13 at 8:36 am

I am a undergrad industrial engineer and what I've heard from most professors in both business and engineering is that as an IE my job is to present my employer with the best possible alternative wich is often the most expensive route and let those on the business side make adjustments that are then sent back to me to sign off on based upon project feasability.

#26 iame on 05.02.13 at 8:36 am

I am a undergrad industrial engineer and what I've heard from most professors in both business and engineering is that my job is to present my employer with the best possible alternative wich is often the most expensive route and let those on the business side make adjustments that are then sent back to me to sign off on based upon project feasability.

#27 Yossi Kreinin on 05.02.13 at 11:21 am

Well, you're presenting it from the angle of "whose duty it is to do what", which deserves a separate discussion and I guess depends a lot on the organization. I was mostly looking at how people tend to think, not how they set the boundaries of their responsibilities; here both engineers and managers can see their responsibility as very broad or very narrow, and things will accordingly work very differently.

Leave a Comment