Software Drift

Open source forking

Dear KV,

I've been porting some code between two related—but divergent—open source projects. The project I'm porting to was forked from another project many years ago, and it seems that the names of the APIs have changed and no one is tracking what's going on in the other project. This has led to APIs for exactly the same type of function having different names, even if what they do is the same. As you can imagine, this is maddening and means that the first part of "porting" is more like mechanical translation from one name to another. I could see this happening with proprietary software that is developed in secret, but both of these projects are developed publicly with long-term mailing lists and public repositories. Surely, there has to be a better way.

Porting Parted Paths

Dear PPP,

I see you have fallen into the common trap we all fall into when building software: We assume that people are not the problem and that everything has a nice, neat, technical solution. Alas, that is not the case. In fact, in open source—even more than in corporate development, where one can use the carrot of cash and the stick of firing people out of cannons (KV's favorite way of firing them)—people can just say, "Fork you," and go their own way.

A fork often develops in an open source project because of personal disagreements, which are, of course, always couched in technical terms. Alice believes she is designing or implementing the software in a better, stronger, faster way than Bob, and, of course, Bob thinks he's doing the better job. And so they part ways, and fork.

Now, sometimes forks, which are a lot like divorces, are messy and sometimes they are amicable, but as anyone who has ever gone through a breakup knows, even the most amicable among them usually go through a cooling-off period. Even though you've agreed "to be friends," you don't contact or see the other person for a while. So too, with forks in software.

Thus, we get software drift. Alice adds a feature but doesn't tell Bob, and why should she? Bob can read the mailing list as well as any other Tom, Dick, or Mary, and the repo is open. Why should Alice tell Bob anything? Bob, of course, feels the same way. So now the codebases drift apart, slowly or quickly but inevitably and inextricably.

Since the systems have a common parent, they probably work in the same technical domain, and therefore the features and fixes that are going to be added are probably similar. KV happens to have an example case at hand: two operating systems that diverged before they added SMP (symmetric multiprocessing) support. When an operating system adds SMP to an existing kernel, the first thing we think of is locks, those handy-dandy little performance killers that we've all been sprinkling around our code since the end of Dennard scaling.

There are many types of locks—just look in any operating system textbook—but the most basic of them is the mutex. You would think that this very simple type of lock would have a very simple API that one system could easily copy from the other. Why make the APIs different when what they express is the same? I don't know. Ask Bob and Alice, because you can see what they did here:

 Alice's Mutex
  void
 mtx_init(struct mtx *mutex, const char *name, const char *type, int opts); 
 
 void
 mtx_destroy(struct mtx *mutex); 
 
 void 
 mtx_lock(struct mtx *mutex); 
 
 void
 mtx_unlock(struct mtx *mutex);
 
 
 Bob's Mutex
  void
 mutex_init(kmutex_t *mtx, kmutex_type_t type, int ipl);
 
 void
 mutex_destroy(kmutex_t *mtx);
 
 void
 mutex_enter(kmutex_t *mtx);
 
 void
 mutex_exit(kmutex_t *mtx);

I won't even get into the "Who got there first?" question, but whoever did, maybe, just maybe, it might have been a good idea for Alice and Bob to cooperate and use similar names for the APIs. What's ironic—wait, not ironic, angering—about all this is that even the argument lists of three of the four APIs listed here are the same.

Now, just because we're having so much fun, let's compare what the respective manual pages say about these functions:

 The mtx_lock() function acquires an MTX_DEF mutual exclusion lock on behalf of the currently running kernel thread. If another kernel thread is holding the mutex, the caller will be disconnected from the CPU until the mutex is available (i.e., it will block).
 Acquire a mutex. If the mutex is already held, the caller will block and not return until the mutex is acquired.

I was going to reorder these statements to make it a bit harder to guess which line was from which system, but I'm feeling slightly kind tonight, having had a double Vesper, and left them in matching order.

"Ah, but wait!" I hear you cry. "Maybe Alice's mutex is better than Bob's!" or vice versa. The underlying performance characteristics actually do not matter here. We are talking about the semantic load of remembering two names for the same thing, and who wants that extra bit of cognitive load sprinkled throughout the codebase? Not KV! If a rose by any other name would still smell just as sweet, then a mutex by another name is still a basic locking primitive that doesn't need to be named twice.

To top all this off, there is a third fork of the same codebase that also has its own, uniquely named set of mutex functions, but it's late and I'm not going to copy and paste more annoying code into this buffer. I will simply commend all readers to look before they code and not splatter the world with nine billion names for mutexes or any other API.

George V. Neville-Neil works on networking and operating-system code for fun and profit. He also teaches courses on various subjects related to programming. His areas of interest are computer security, operating systems, networking, time protocols, and the care and feeding of large codebases. He is the author of The Kollected Kode Vicious and coauthor with Marshall Kirk McKusick and Robert N. M. Watson of The Design and Implementation of the FreeBSD Operating System. For nearly 20 years, he has been the columnist better known as Kode Vicious. Since 2014, he has been an industrial visitor at the University of Cambridge, where he is involved in several projects relating to computer security. He earned his bachelor's degree in computer science at Northeastern University in Boston, Massachusetts, and is a member of ACM, the Usenix Association, and IEEE. His software not only runs on Earth, but also has been deployed as part of VxWorks in NASA's missions to Mars. He is an avid bicyclist and traveler who currently lives in New York City.

Originally published in Queue vol. 22, no. 1—
Comment on this article in the ACM Digital Library

More related articles:

Amanda Casari, Julia Ferraioli, Juniper Lovato - Beyond the Repository
Much of the existing research about open source elects to study software repositories instead of ecosystems. An open source repository most often refers to the artifacts recorded in a version control system and occasionally includes interactions around the repository itself. An open source ecosystem refers to a collection of repositories, the community, their interactions, incentives, behavioral norms, and culture. The decentralized nature of open source makes holistic analysis of the ecosystem an arduous task, with communities and identities intersecting in organic and evolving ways. Despite these complexities, the increased scrutiny on software security and supply chains makes it of the utmost importance to take an ecosystem-based approach when performing research about open source.

Guenever Aldrich, Danny Tsang, Jason McKenney - Three-part Harmony for Program Managers Who Just Don't Get It, Yet
This article examines three tools in the system acquisitions toolbox that can work to expedite development and procurement while mitigating programmatic risk: OSS, open standards, and the Agile/Scrum software development processes are all powerful additions to the DoD acquisition program management toolbox.

Jessie Frazelle - Open-source Firmware
Open-source firmware can help bring computing to a more secure place by making the actions of firmware more visible and less likely to do harm. This article’s goal is to make readers feel empowered to demand more from vendors who can help drive this change.

Marshall Kirk McKusick, George V. Neville-Neil - Thread Scheduling in FreeBSD 5.2
A busy system makes thousands of scheduling decisions per second, so the speed with which scheduling decisions are made is critical to the performance of the system as a whole. This article - excerpted from the forthcoming book, “The Design and Implementation of the FreeBSD Operating System“ - uses the example of the open source FreeBSD system to help us understand thread scheduling. The original FreeBSD scheduler was designed in the 1980s for large uniprocessor systems. Although it continues to work well in that environment today, the new ULE scheduler was designed specifically to optimize multiprocessor and multithread environments. This article first studies the original FreeBSD scheduler, then describes the new ULE scheduler.