January/February 2018 issue of acmqueue

The January/February issue of acmqueue is out now

The Bike Shed

Web Services

  Download PDF version of this article PDF

Center Wheel for Success

"Not invented here" syndrome is not unique to the IT world.

Poul-Henning Kamp

When I first read the claim that HealthCare.gov, the Web site initiated by the Affordable Care Act, had cost $500 million to create,4 I didn't believe the number. There is no way to make a Web site cost that much. But the actual number seems not to be an order-of-magnitude lower, and as I understand the reports, the Web site doesn't have much to show for the high cost in term of performance, features, or quality in general.

This is hardly a unique experience in the IT world. In fact, it seems more the rule than the exception.

Here in Denmark we are in no way immune: POLSAG, a new case-management system for the Danish police force, was scrapped after running up a tab of US$100 million and having nothing usable to show for it. We are quick to dismiss these types of failures as politicians asking for the wrong systems and incompetent and/or greedy companies being happy to oblige. While that may be part of the explanation, it is hardly sufficient.

The traditional response from the IT world is that the Next Big Thing will fix this, where the Next Big Thing has been a seemingly infinite sequence of concepts such as high-level languages, structured programming, relational databases, SQL, fourth-generation languages, object-oriented programming, agile methodologies, and so on ad nauseam. I think it is fair to say that none of these technologies has made any significant difference in the success/failure ratio of IT projects. Clearly they allow us to make much bigger projects, but the actual success/failure rate seems to be pretty much the same.

At the same time, there are all these amazing success stories, where a couple of college kids change the way we think about information retrieval with their Google information-scoring algorithm, or a bunch of friends change the way we communicate with their Twitter information-distribution system.

Why, despite politicians' lofty speeches, does that never happen in government IT applications? There is clearly something we are missing here, something we're doing wrong, without even thinking about it. That particular mistake is far more common that it should be in a (so-called) "knowledge economy."

Lessons from Wheelbarrows

Growing up in the countryside, I spent a good portion of my youth operating a wheelbarrow. The European wheelbarrow is a rationalization of the handbarrow, which was basically two planks, two feet apart, with boards nailed or tied between them. One person grabs the two planks at the front, one in each hand, another grabs them at the back, and then they trudge away with their load.

Sometime back in the early one thousands, a productivity consultant must have pointed out that if you replaced the person in front with a wheel, then you could get twice as many wheelbarrows moving with the same number of workers. (This industrial application of technology undoubtedly earned the consultant a hefty fee.)

And that's it! That is the very same contraption I lugged around as a kid and the same one I used just a few hours ago for gardening. As anybody knows, using a wheelbarrow is easier than carrying things, but it is still quite heavy work. You lift roughly half the load yourself, you provide the energy for motion, and you must steer it in the right direction, which is difficult on account of the first two expenditures of energy.

While a vast improvement over the handbarrow, the European wheelbarrow is stupidly inefficient, at least compared with the Chinese version.2 Somebody in China was smarter than the Medieval European downsizer and moved the wheel to the middle of the wheelbarrow, so that the entire weight of the load is carried by the wheel. The Chinese wheelbarrow will readily transport two or three times the load of a European wheelbarrow, with the operator hardly breaking a sweat, just pushing and steering, with barely any lifting.

From a management perspective, the Chinese wheelbarrow is identical to the European one: one wheel, two handles, one operator. Looking at it that way, however, we blind ourselves to how differently they work, and we miss the full productivity improvement of the wheel.

In Europe we have known about the Chinese wheelbarrow since at least 1797,2 yet, to this day, we still sweat while lifting half the load carried on our nonoptimized wheelbarrows.

The "not invented here" syndrome is not unique to the IT world.

I'm beginning to think that the reason our big IT projects sink is that we make the same kind of mistake: mindlessly replacing human labor with technology instead of solving the actual problem.

Many human jobs can be replaced directly with computers. E-mail replaced the old telegraph system, delivering the exact same conceptual service: delivering a text message quickly while using hardly any manpower. But delivering text messages was the least e-mail could do—once we got to know it better. First there were programs answering e-mails, sending source code, or looking up things in databases. Next came programs sending e-mails to other programs, to keep databases synchronized, and then e-mails containing pictures, sound, and vice presidents.1

The e-mail system we know today, as envisioned by Ray Tomlinson, was not the only such system somebody came up with, however. The state-sanctioned post and telegraph monopolies attempted to standardize e-mail—or "telematic services" as they called it—in CCITT (International Telegraph and Telephone Consultative Committee) recommendations X.400-X.599,3 as part of the grand vision of "The Intelligent Network."

They started approximately 15 years before Tomlinson. They spent uncountable millions of all sorts of currencies. They had legislators mandating that their way be the one and only legal way forward. And they failed utterly, miserably, and definitively.

Why is it that in IT one person can often do what thousands cannot?

It is tempting to speculate that HealthCare.gov would have worked much better had they given the task to a 10-person company rather than a conglomerate with 69,000 employees all over the globe. I'm sure that is a necessary part of the solution, but again, it is hardly a sufficient condition for success.

For one thing, while there are "only" 380,000 words in the Affordable Care Act (also known as Obamacare), the regulations floating from the law amount to 12 million words (and counting). No 10-person company would even be able to read all that verbiage before the delivery deadline had whooshed past.

Interestingly, The New York Times reports that HealthCare.gov contains an estimated 500 million lines of code.4 That's no more likely to be true than the $500 million price tag.

I looked at one of the actual laws that make up Obamacare, the PPACA (Patient Protection and Affordable Care Act),5 and since I was not going to read all 906 pages, I started in the middle, on page 403. After a few pages I ran into this definition of patient decision aid:

"(1) PATIENT DECISION AID—The term 'patient decision aid' means an educational tool that helps patients, caregivers, or authorized representatives understand and communicate their beliefs and preferences related to their treatment options, and to decide with their health care provider what treatments are best for them based on their treatment options, scientific evidence, circumstances, beliefs, and preferences."

Reading on, I found the requirements:

"(2) REQUIREMENTS FOR PATIENT DECISION AIDS—Patient decision aids developed and produced pursuant to a grant or contract under paragraph (1)—
"(A) shall be designed to engage patients, caregivers, and authorized representatives in informed decision making with health care providers;
"(B) shall present up-to-date clinical evidence about the risks and benefits of treatment options in a form and manner that is age-appropriate and can be adapted for patients, caregivers, and authorized representatives from a variety of cultural and educational backgrounds to reflect the varying needs of consumers and diverse levels of health literacy;

"(C) shall, where appropriate, explain why there is a lack of evidence to support one treatment option over another; and
"(D) shall address health care decisions across the age span, including those affecting vulnerable populations including children."

Unless Congress thinks of teachers as "educational tools," I think we can take it as written here that they expect this to be some kind of computer program. But read it again and pay attention to the language. When was the last time you saw a computer program that "engaged," "explained," or "addressed decisions?" Or, for that matter, when have you seen a program that "adapted for [...] a variety of cultural and educational backgrounds to reflect the varying needs of consumers and diverse levels of health literacy"?

These paragraphs legislate that Obamacare will fund research in heavy-duty state-of-the-art artificial intelligence—I somehow doubt that is what Congress intended it to say. I posit that Congress worried about having enough doctors and nurses for this new health care, so they wanted to use computers to cut down the talking and explaining. In other words, they want to save manpower—by replacing the front man on the handbarrow with a wheel.

I have used a handbarrow once, in an emergency. My fellow campers and I constructed it from two young pine trees, wrapping the sail from our tent around them. Compared with a wheelbarrow, it was both easier and faster, because the front man didn't get stuck in any holes or hit any rocks, and he helped with all of navigation, lifting, locomotion, and steering. When we met the first responders, they gently lifted our friend with his injured leg from our makeshift version to their professional handbarrow and carried him the rest of the way to their ambulance on a hi-tech aluminum stretcher.

I am absolutely sure that Congress would never replace the front man on an ambulance stretcher with a wheel to save manpower—yet, in a way, they did just that. I won't claim to know the correct way to optimize a health care consultation with computers—there may be one, but more importantly, there may not.

Blindly deciding that information technology will be substituted for humans is unenlightened. IT is not a magic potion that makes unpleasant or inconvenient things disappear. The right thing to do is to ask, as a Chinese engineer did 2,000 years ago, "If we're going to put a wheel on this thing, where is the best place to put it?"

And to realize that two questions were asked.


1. Borenstein, M., Linimon, M. 1993. The extension of MIME content-types to a new medium. RFC 1437; http://www.rfc-editor.org/rfc/rfc1437.txt.

2. De Decker, K. 2011. How to downsize a transport network: the Chinese wheelbarrow. Low-tech Magazine; http://www.lowtechmagazine.com/2011/12/the-chinese-wheelbarrow.html.

3. International Telecommunication Union. ITU-T recommendations; http://www.itu.int/ITU-T/recommendations/index.aspx?ser=X.

4. LaFraniere, S., Austen, I. Pear, R. 2013. Contractors see weeks of work on health site. The New York Times (October 20); http://www.nytimes.com/2013/10/21/us/insurance-site-seen-needing-weeks-to-fix.html.

5. Patient Protection and Affordable Care Act. 2010; http://www.gpo.gov/fdsys/pkg/PLAW-111publ148/content-detail.html.


[email protected]

Poul-Henning Kamp ([email protected]) is one of the primary developers of the FreeBSD operating system, which he has worked on from the very beginning. He is widely unknown for his MD5-based password scrambler, which protects the passwords on Cisco routers, Juniper routers, and Linux and BSD systems. Some people have noticed that he wrote a memory allocator, a device file system, and a disk encryption method that is actually usable. Kamp lives in Denmark with his wife, son, daughter, about a dozen FreeBSD computers, and one of the world's most precise NTP (Network Time Protocol) clocks. He makes a living as an independent contractor doing all sorts of stuff with computers and networks.

© 2013 ACM 1542-7730/13/1200 $10.00


Originally published in Queue vol. 11, no. 12
see this item in the ACM Digital Library



Benjamin Treynor Sloss, Mike Dahlin, Vivek Rau, Betsy Beyer - The Calculus of Service Availability
You're only as available as the sum of your dependencies.

Tom Killalea - The Hidden Dividends of Microservices
Microservices aren't for every company, and the journey isn't easy.

Ben Maurer - Fail at Scale
Reliability in the face of rapid change

Aiman Erbad, Charles Krasic - Sender-side Buffers and the Case for Multimedia Adaptation
A proposal to improve the performance and availability of streaming video and other time-sensitive media


(newest first)

Displaying 10 most recent comments. Read the full list here

rembal | Fri, 26 Dec 2014 20:28:49 UTC

The email case is not one person against a group of experts. It's a case of group of experts vs group of experts, but working in different ways.

One group gathers together and quarells over a standard that will be based on the compromise they reach. It often ends up as a medley of their ideas, as everybody wants to have a say, and the hierarchy of the gruoup is based more on their esteem then their input.

Members of the second group works individually, creating a set of standards. Good standards attract more colaborators, creating a positive feedback loop and weak standards gradually die out. Hierarchy is created ad-hoc.

The second approach is more evolutionary. It makes me wonder how would a living organism look like if it was created by a comitee. That might explain plytapus...

Fred Mellender | Fri, 17 Jan 2014 22:52:00 UTC

The European wheelbarrow used for gardening has a small wheel in front to make the carrying bed low. This means the cargo does not have to be lifted very high to load the barrow. A large center wheel would raise the carrying bed. A small wheel in the center would not work because the front of the barrow would hit the ground when the handles were lifted to waist height. The Chinese barrow was made for long hauls where the effort to lift the cargo is outweighed by the easy in pushing over the distance. The short haul, ease of loading European barrow is ideal for gardening.

Moody | Sat, 11 Jan 2014 05:00:56 UTC

Interesting perspectives in your article. There are many models / metaphors that can be applied in this context. Each one highlights different aspects of the the underlying structural dynamics: 1. Larger organizations typically don't have processes to match skill to task. People with design skills may end up doing testing. Architects may end up in analysis. The metaphor here is of asking the right hand to do a foot's work. Overall architecture and design are critical to the success / failure of the project. These are often relegated to people who are not matched to that task. 2. Missing feedback loops at all levels - a project is a learning process on several fronts - requirements, architecture, design, implementation, testing, deployment. There are missing feedback loops between these stages to change requirements, architecture, design, etc. as the project learns. Tasks tend to be siloed as a way to manage complexity. And those silo walls in complex, dynamic leads to the high failure rates we see in such projects. The metaphor here is driving while looking in the rearview mirror. 3. Project decisions for funding, resources, schedules are often made by people who don't have a good grasp of the dynamic complexity of such projects. And these decisions become rigid milestones where project / process learning is not fed back to change the milestones. The metaphor here is sailing with the sail in a position fixed at the shoreline and not adjusted with major shifts in the wind direction. 4. IT culture tends to be monolithic silo oriented, i.e., individual / group / small team oriented with such entities having an adverserial mindset against each other rather than a cooperative project mindset. The metaphor here is that we lots of lone rangers wanting to do their own hi ho silver away.

This list could go on a bit longer as there are other dynamics at play in this context. Some thoughts for you to consider. Enjoyed reading your article and your insights into IT projects.

Torben Mogensen | Tue, 24 Dec 2013 17:16:51 UTC

One reason a single person (or a small team) can succeed where a large corporation will fail is that the single person will realise that he can't ever complete a complex solution, so he simplifies, challenges assumptions and maybe solves a slightly different (but equally relevant) problem than the one presented to him in the first place. The large corporation will blindly try to implement the complex solutions, creating more complexity on the way, and nobody will dare question the assumptions or solve slightly different problems, because the best way to keep their jobs is to do exactly as they are told. And because of the multitude of people involved, miscommunication will go undetected for a long time, so different parts of the system that don't work together will get layers of patching put on top.

Nearly all successful large systems have started out as small well-functioning systems that have evolved slowly over time. While that can also lead to undue complexity and "baggage", it is incredibly difficult to design large systems from scratch, and in all too many cases such attempts fail miserably. We really only hear about failures of public systems, but there are equally many (and maybe more) failed colossal IT systems in industry.

Claus Gravesen | Mon, 23 Dec 2013 10:59:32 UTC

"The point here is that politicians point to the 1-man wonders and then legislate 10.000-man fiaskos into existence."

So incredibly well put..!

Happy holidays one and all :-)

Mike Small | Sun, 22 Dec 2013 20:48:41 UTC

I think because you don't have friends or colleagues working in that area, perhaps, you're letting a bit of jargon from another industry trip you up. (The use of the word tool maybe is especially confusing.) Brochures, pamphlets, videos, static html etc. is the form that decision aids take from what I've seen of my friend's work. I suppose an expert system could function as a decision aid as well, but I've never heard of one of the foundations who produce them creating such a thing (on their budgets it would not be realistic I expect).

Here's another example of an organization, this one in Canada, who creates such things and has the term in their name: http://decisionaid.ohri.ca/about.html

Here is a sample of one of their decision aids: http://decisionaid.ohri.ca/AZsumm.php?ID=1648

Thomas L. Kjeldsen | Sun, 22 Dec 2013 20:24:01 UTC

Does complying with regulations slow down development?

In Utopia someone will read all relevant laws and regulations and can work as a Verifier during system development, to ensure that the current solution and future plans comply with regulations. Ideally, the Verifier can decide in constant time, but this is most likely not the case. This can slow down development speed because, ideally, every step would have to be verified by the Verifier. With frequent changes to laws and regulations the Verifier becomes even slower. As a consequence, development speed slows down, the overhead gets bigger and the cost increases.

Another approach is to not care about regulation or even break it intentionally. This way you can move fast, but I doubt you'll win any big governmental contracts.

Poul-Henning Kamp | Sun, 22 Dec 2013 20:15:45 UTC

@Mike Small:

To paraphrase Justice Antonin Scalia: "I trust Congress know how to write 'brochure' if they intend to write 'brochure'."

Mike Small | Sun, 22 Dec 2013 18:09:40 UTC

That description doesn't at all imply a computer program to me. Rather it could describes a brochure (online or otherwise) or similar document with writing and pictures. It engages the way your article engages. It could be produced by the likes of organizations like the following with no programmers or computer scientists on staff: http://www.informedmedicaldecisions.org/

Poul-Henning Kamp | Sun, 22 Dec 2013 17:43:11 UTC

@J Story:

Correct, it's obviously inconvenient volumetrically to have the wheel in the middle of things, but not as a matter of mass. But read the article in Low-Tech Magazine (ref #2) and you'll see that it is far more interesting than that. (Sails on a wheelbarrow ??)


The point here is that politicians point to the 1-man wonders and then legislate 10.000-man fiaskos into existence.

Displaying 10 most recent comments. Read the full list here
Leave this field empty

Post a Comment:

© 2018 ACM, Inc. All Rights Reserved.