Columns > The Bikeshed

      view issue

Please Put OpenSSL Out of Its Misery
Download PDF version of this article

by Poul-Henning Kamp | April 12, 2014

Topic: Security

  • View Comments
  • Print

Please Put OpenSSL Out of Its Misery

OpenSSL must die, for it will never get any better.

Poul-Henning Kamp

The OpenSSL software package is around 300,000 lines of code, which means there are probably around 299 bugs still there, now that the Heartbleed bug — which allowed pretty much anybody to retrieve internal state to which they should normally not have access — has been fixed.

That's really all you need to know, but you also know that won't stop me, right?

Securing a computer network connection is not really hard in theory. First you let exceptionally skilled cryptographers design some cryptographic building blocks. You will need a good hash-function, a good symmetric block cipher, and a good asymmetric cipher. Next you get exceptionally skilled crypto-protocol designers to define how these building blocks should be tied together in a blow-by-blow fashion. Then an exceptionally skilled API designer defines how applications get access to the protocol, via a well-thought-out and error-resistant API with well-chosen and reliable default values and a good error reporting mechanism. Then exceptionally skilled programmers implement the algorithms and protocols according to the API in high-quality, fully-audited and analyzed library source code. And after that the application programmer — who's usually anything but exceptionally skilled — finally gets to write code to open a secure connection.

But we're not quite done.

We need to ensure that the compiler correctly translates the high-level language to machine instructions. And only exceptionally skilled compiler programmers will do! We also need to make sure that the computing environment is trustworthy — there can be no bugs, mistakes, backdoors, or malware in system libraries or in the operating system kernel. And written by? You got it , exceptionally skilled kernel programmers obviously! Of course this is all in vain if the CPU does not execute instructions faithfully, but exceptionally skilled CPU engineers will no doubt see to that.

Now — finally — we can securely transmit a picture of a cat from one computer to another.

That wasn't so bad, was it?

Specifying a dream team as I did is not without pitfalls, as OpenSSL contains this code:

* The aim of right-shifting md_size is so that the compiler
* doesn't figure out that it can remove div_spoiler as that
* would require it to prove that md_size is always even,
* which I hope is beyond it.
div_spoiler = md_size >> 1;
div_spoiler <<= (sizeof(div_spoiler)-1)*8;
rotate_offset =
   (div_spoiler + mac_start - scan_start) % md_size;

Be honest — would you have considered a proven, correct, compiler improvement a security risk before I showed you that?

Of course you wouldn't! It was proven correct, wasn't it?

And that's where our security model, if you can call it that, breaks down. There is so much code involved, that no one has a clue if it all works or not.

On my computer the numbers are roughly:

Operating system Kernel:        2.0 million lines of code
Compiler:                       2.0 --//--
C language runtime library:     0.5 --//--
Cryptolibrary:                  0.3 --//--

Each additional programming language involved will add about a million lines. Using a graphical user interface roughly doubles the number, and a browser almost doubles it again. A reasonable estimate is that the computer on which you read this has at least 100, but more likely 1,000 bugs through which your security can be compromised. If those bugs were unique to your computer, that wouldn't be too bad. But that's not the case, as they are the exact same bugs on millions and millions of computers, and, therefore, every bug has pandemic potential. And — as if the task were not difficult enough — there are people actively trying to sabotage the result, in order to make "intelligence gathering" easier and less expensive. Apparently some of those exceptionally skilled engineers and scientists on our dream-team have hidden agendas — otherwise the leadership of the NSA would be deeply incompetent.

Then there are Certificate Authorities. Their job is to act as "trust-anchors" so that once you have established a secure connection you also know who you are talking to.

CA's trustworthiness is a joke.

Do you trust:


to authenticate a connection to your bank?

You don't even know what "GÜVENLİĞİ HİZMETLERİ" means, right?

I certainly don't, and I don't trust them either!

In August 2012 they issued bogus certificates with which the holder could claim to be Google. TÜRKTRUST, of course, maintains that it was all "a one-time mistake," that "no foul play was involved" and that "it can never happen again."

Nobody believes a word of it.

Yet, your browser still trusts them by default, just as it trusts hundreds of other more or less suspect organizations to tell the truth. So we're connecting using bloated buggy software, trusting shady government-infiltrated CAs to assure us who we talk to?

There must be a better way, right?

Well there isn't.

We have never found a simpler way for two parties with no prior contact, to establish a secure telecommunications channel. Until the development of asymmetric cryptography (1973-1977) it was thought to be impossible. And if you want to know who is at the other end, the only way is to trust some third party who claims to know. No mathematical or cryptographic breakthrough will ever change that, given that our identities are social constructs, not physical realities.

So if we want e-commerce to work, we have to make the buggy code work and we should implement something more trustworthy than the "CA-Mafia."

And that brings me back to OpenSSL — which sucks. The code is a mess, the documentation is misleading, and the defaults are deceptive. Plus it is 300,000 lines of code that suffer from just about every software engineering ailment you can imagine:

  • No central architectural authority
  • 6,740 goto statements
  • Inline assembly code
  • Multiple different coding styles
  • Obscure use of macro preprocessors
  • Inconsistent naming conventions
  • Far too many selections and options
  • Unexplained dead code
  • Misleading and incoherent comments

and so on and so on.

And it's nobody's fault.

No one was ever truly in charge of OpenSSL, it just sort of became the default landfill for prototypes of cryptographic inventions, and since it had everything cryptographic under the sun (somewhere , if you could find out how to use it), it also became the default source of cryptographic functionality.

I'm sure more than one person has thought "Nobody ever got fired for using OpenSSL".

And that is why everybody is panicking on the Internet as I write this.

This bug was pretty bad, even as bugs in OpenSSL go, but my co-columnist at ACM Queue, Kode Vicious, managed to find a silver lining: "Because they used a 'short' integer, only 64 kilobytes worth of secrets are exposed."

And that is not the first nor will it be the last serious bug in OpenSSL, and, therefore, OpenSSL must die, for it will never get any better.

We need a well-designed API, as simple as possible to make it hard for people to use it incorrectly. And we need multiple independent quality implementations of that API, so that if one turns out to be crap, people can switch to a better one in a matter of hours.



Poul-Henning Kamp ( is one of the primary developers of the FreeBSD operating system, which he has worked on from the very beginning. He is widely unknown for his MD5-based password scrambler, which protects the passwords on Cisco routers, Juniper routers, and Linux and BSD systems. Some people have noticed that he wrote a memory allocator, a device file system, and a disk encryption method that is actually usable. Kamp lives in Denmark with his wife, son, daughter, about a dozen FreeBSD computers, and one of the world's most precise NTP (Network Time Protocol) clocks. He makes a living as an independent contractor doing all sorts of stuff with computers and networks.

© 2014 ACM 1542-7730/14/0400 $10.00


Originally published in Queue vol. 12, no. 3
see this item in the ACM Digital Library

Back to top


Displaying 10 most recent comments. Read the full list here
  • Benni | Tue, 15 Apr 2014 18:58:03 UTC

    @Willy: Well, if you do contractor work, and your contractor is the BND, then certainly, they expect a different quality of the code. For example, readable code is, in the eyes of BND, certainly not "good code" if an open source project is concerned where your objective is to "introduce features" that deliberately weaken the security. If the code would be written in a documented and understandable manner, then you could not introduce the "features" that the BND likes without it getting noticed. Either the people on openssl are just plainly incompetent, or their main contractor is in Pullach, wo wants such code styles. 
  • Tom Limoncelli | Wed, 16 Apr 2014 01:41:35 UTC

    The patch is a good example of how to write bad C code:
    Statements like this:
    if (1 + 2 + payload + 16 > s->s3->rrec.length)
    More clear and "defensive" would be:
    if ((1 + 2 + payload + 16) > s->s3->rrec.length)
    As published, the next person to modify the code is likely to mess it up.  Don't write code for yourself, write it for the next person that has to read it.
    If you think "but that's a waste of parenthesis" then tell me what this code does:
    x = 1 << 16 + 1
    Does it evaluate to 17 or 32?  The answer will surprise you.
  • Javier | Wed, 16 Apr 2014 04:01:52 UTC

    Very nice article
    Aside from DANE, Convergence(  is another alternative to CA.
    OpenBSD is cleaning OpenSSL(
  • Kevin L | Wed, 16 Apr 2014 11:32:12 UTC

    As an experienced engineer in both the process industries (MS in ChemE) and software (BS in CompSci), I think that data breaches at the SSL/TLS layer need to be treated in much the same way as pressure breaches in physical systems: you use multiple independent systems to verify each other **while online**, and if any one of those systems sees a problem you interrupt the flow (terminate the connection).
    Applied to this case, that would be two or more SSL libraries executing in separate processes that would each perform the SSL/TLS protocol independently, with the HTTP process comparing their outputs and only sending bytes to the other side when the outputs fully agree.  If they disagree, the HTTP daemon would log an error and terminate the connection.  In this fashion each library would expose the bugs in the other, while protecting the users.  Eventually both libraries would get better, only failing simultaneously for errors in the protocol design or the underlying crypto math.  (This is also the approach used in aviation: 3 computers, 2 to check for hardware faults and a third in an independent implementation to check for logic faults.)
    Outside of generating nonces and random numbers that would have to be passed between the libraries and the HTTP process, this doesn't seem like it would be too hard to do.
  • Peter Kriens | Wed, 16 Apr 2014 14:39:18 UTC

    Isn't this also a reason to not program this kind of code in C(++) instead of a managed language?
  • Morten Jensen | Wed, 16 Apr 2014 20:37:41 UTC

    Great article, was referred here from the debate:
    > /* 
    > * The aim of right-shifting md_size is so that the compiler 
    > * doesn't figure out that it can remove div_spoiler as that 
    > * would require it to prove that md_size is always even, 
    > * which I hope is beyond it. 
    > */ 
    > div_spoiler = md_size >> 1; 
    > div_spoiler  rotate_offset = 
    >    (div_spoiler + mac_start - scan_start) % md_size;
    > Be honest  would you have considered a proven, correct, compiler improvement a security risk > before I showed you that?
    > Of course you wouldn't! It was proven correct, wasn't it?
    I do embedded development, and I sometimes experience that I have to qualify a variable volatile, to get the compiler to stop optimizing the code in ways that spoils the intended behavior. 
    I would imagine the OpenSSL code could do the operation on a volatile-qualified variable and achieve the same effect without having to resort to nasty tricks ?
    From my K&R on the volatile qualifier: "The purpose of volatile is to force an implementation to suppress optimization that could otherwise occur."
  • Sebastiaan | Thu, 17 Apr 2014 22:21:07 UTC

    @willy just picked two random commits and yup, it's scary!
    A new go to fail in the making? I'm no C programmer, but these two commits removed curly brackets and subsequently converted a single line of code inside an 'if' statement to two lines... Sure this code is correct, but *visually* harder a lot harder to spot?;a=commitdiff;h=34e43b909f02de444678d937ed3dc347ce13ba1a;a=commitdiff;h=833a896681b3287e5ab9c01f4f0234691f4076a8
  • Sebastiaan | Thu, 17 Apr 2014 22:45:53 UTC

    Must be because English is not my native language, but in my book these commits don't fall in the category 'typo'...;a=commitdiff;h=79dabcc1373be17283d736901ddf9194609ba701;a=commitdiff;h=82f42a1d2e9271359b60d16249c26baadae788db;a=commitdiff;h=24f599af21dfd8e1de693fd84fb612269c28de0c
  • Martin Leiser | Fri, 25 Apr 2014 21:19:11 UTC

    hmmm... I think it is all about Architecture.
    Software has bug. Period.
    On the other side:
    We know for 50 years how a MMU works.
    Hardware has bugs too, but much less.
    Why is to web server, openssl, the private key in one adress space?
    Use stunnel with PKCS11 to off load the private key.
    And the web server in a different process.
    Two simpl rules:
    Divide and conquer.
    Least privilege.
    Add a third rule: CPUs are cheap.. switch off optimizations for security relevant code.
    But all that is not the trend of today is multithreading, throwing 100.000.000
    Lines of code in one application server.....
    Dont blame the openssl programmers, it is their bug,
    but our fault.
  • Dan Cross | Fri, 09 May 2014 16:27:23 UTC

    tal: it evaluates to neither.  :-)  It's 2^17 (arithmetic shift has lower precedence than addition).
    Something I learned long ago is that if one is to successfully program in some language, one must write idiomatic code in that language.  Unfortunately, I think that one of the problems with writing C code is that there is a lack of idiom around things like parenthesizing expressions that can lead to surprising problems if one is not familiar with the precedence rules of the language.
Displaying 10 most recent comments. Read the full list here
Leave this field empty

Post a Comment:

(Required - 4,000 character limit - HTML syntax is not allowed and will be removed)