The Woes of IDEs:
Programming speed and quality are hindered by poor integrated development environment interfaces.
Preaching emanating from the ranks and gurus of the human interface world is slowly convincing management, software designers--and even programmers--that better human-machine interfaces can increase productivity by speeding the work, decreasing learning time, lowering the burden on human memory, and easing users’ physical and mental stress.
User Interface Designers, Slaves of Fashion:
The status quo prevails in interface design, and the flawed concept of cut-and-paste is a perfect example.
The discipline, science, and art of interface design has gone stagnant. The most widely read books on the subject are primarily compendia of how to make the best of received widgets. The status quo is mistaken for necessity. Constrained in this chamber pot, designers wander around giving the users of their products little comfort or fresh air.
Stand and Deliver: Why I Hate Stand-Up Meetings:
Stand-up meetings are an important component of the ’whole team’, which is one of the fundamental practices of extreme programming (XP).
According to the Extreme Programming Web site, the stand-up meeting is one part of the rules and practices of extreme programming: “Communication among the entire team is the purpose of the stand-up meeting. They should take place every morning in order to communicate problems, solutions, and promote team focus. The idea is that everyone stands up in a circle in order to avoid long discussions. It is more efficient to have one short meeting that everyone is required to attend than many meetings with a few developers each.”
The Big Bang Theory of IDEs:
Pondering the vastness of the ever-expanding universe of IDEs, you might wonder whether a usable IDE is too much to ask for.
Remember the halcyon days when development required only a text editor, a compiler, and some sort of debugger (in cases where the odd printf() or two alone didn’t serve)? During the early days of computing, these were independent tools used iteratively in development’s golden circle. Somewhere along the way we realized that a closer integration of these tools could expedite the development process. Thus was born the integrated development environment (IDE), a framework and user environment for software development that’s actually a toolkit of instruments essential to software creation. At first, IDEs simply connected the big three (editor, compiler, and debugger), but nowadays most go well beyond those minimum requirements.
The Demise of the Waterfall Model Is Imminent, and Other Urban Myths:
Rumors of the demise of the Waterfall Life-cycle Model are greatly exaggerated.
We discovered this and other disappointing indicators about current software engineering practices in a recent survey of almost 200 software professionals. These discoveries raise questions about perception versus reality with respect to the nature of software engineers, software engineering practice, and the industry.
Death by UML Fever:
Self-diagnosis and early treatment are crucial in the fight against UML Fever.
A potentially deadly illness, clinically referred to as UML (Unified Modeling Language) fever, is plaguing many software-engineering efforts today. This fever has many different strains that vary in levels of lethality and contagion. A number of these strains are symptomatically related, however. Rigorous laboratory analysis has revealed that each is unique in origin and makeup. A particularly insidious characteristic of UML fever, common to most of its assorted strains, is the difficulty individuals and organizations have in self-diagnosing the affliction. A consequence is that many cases of the fever go untreated and often evolve into more complex and lethal strains.
Opinion: Buffer Overrun Madness:
Why do good programmers follow bad practices?
In January 2003, the Slammer worm was reported to be the fastest spreading ever. Slammer gets access by exploiting a buffer overrun. If you peruse CERT advisories or security upgrade releases, you will see that the majority of computer security holes are buffer overruns. These would be minor irritations but for the world’s addiction to the weakly typed programming languages C and its derivative C++. Buffer overruns are a kind of array bounds error. There are many variations on how one might actually happen, but here is a typical scenario. Function F calls function G, and G returns a string. F allocates a buffer to hold the result and passes a pointer to its zeroth character.
First, Do No Harm: A Hippocratic Oath for Software Developers?:
What’s wrong with taking our profession a little more seriously?
When asked about the Hippocratic Oath, most people are likely to recall the phrase, “First, do no harm.” It’s a logical response, as even those unfamiliar with the oath could figure out that avoiding additional injury in the course of treatment is critical. In fact, it’s natural to strive in any endeavor not to break something further in the course of repair. In software engineering, as in medicine, doing no harm starts with a deep understanding of the tools and techniques available. Using this theme and some medical metaphors, I offer some observations on the practice of software engineering. Whatever we do, first, do no harm.
From This Moment On:
Divining the future of computers with computers
Science fiction seems to have spawned two divergent subgenres. One, which is out of favor, paints a bright future for us, assuming an optimistic, Darwinian "perfectability." These scenarios project an ever-expanding (or rather, a never-imploding) cosmos with ample time for utopian evolutions.
Opinion: For Want of a Comma, the Meaning Was Lost:
What does punctuation have to do with software development?
Odder things have happened in publishing, but not by much. The bestseller list on Amazon (as I write this) is topped by Lynne Truss’s book on... punctuation. It’s outselling Harry Potter. Starting from the title, ’Eats, Shoots and Leaves’ (Gotham Books, 2004), which changes meaning drastically when the comma is omitted, the book is an entertaining romp through the advantages of writing correctly and clearly with the proper use of punctuation.
The Burning Bag of Dung and Other Environmental Antipatterns:
And you think you have problems?
In my youth a favorite prank of the local delinquents was to place a paper bag full of doggy doo on a neighbor’s porch, light it on fire, ring the doorbell, and then flee. The home-owner, upon answering the door, had no choice but to stomp out the incendiary feces, getting their shoes dirty in the process. Why this scatological anecdote? Because it is a metaphor for work situations in which things have gotten so bad that the only way to “put the fire out” is to step into it. I call this the “burning bag of dung” antipattern.
Kode Vicious Strikes Again:
Kall us krazy, but we’re making Kode Vicious a regular.
Dear Kode Vicious, I have this problem. I can never seem to find bits of code I know I wrote. This isn’t so much work code--that’s on our source server--but you know, those bits of test code I wrote last month, I can never find them. How do you deal with this?
Programming in Franglais:
Six of one, half a dozen of d’autre
When I was studying French in high school, we students often spoke “Franglais”: French grammar and words where we knew them, English inserted where our command of French failed us. It was pretty awful, and the teacher did not think highly of it. But we could communicate haltingly because we all had about the same levels of knowledge of the respective languages. Today, there is a kind of programmer’s Franglais that is all too pervasive. Those who are old enough will remember the pitched controversy in the late 1960s and early 1970s over whether compilers, operating systems, and other systems programs should be written in assembly code or a high-level language.
Kode Vicious: The Return:
A koder with attitude, KV answers your questions. Miss Manners he ain’t.
Dear KV, Whenever my team reviews my code, they always complain that I don’t check for return values from system calls. I can see having to check a regular function call, because I don’t trust my co-workers, but system calls are written by people who know what they’re doing--and, besides, if a system call fails, there isn’t much I can do to recover. Why bother?
Kode Vicious Unleashed:
Koding konundrums driving you nuts? Ko-workers making you krazy? Not to worry, Kode Vicious has you covered.
Dear KV, My officemate writes methods that are 1,000 lines long and claims they are easier to understand than if they were broken down into a smaller set of methods. How can we convince him his code is a maintenance nightmare?
Kode Vicious Reloaded:
A koder with attitude, KV answers your questions. Miss Manners he ain’t.
The program should be a small project, but every time I start specifying the objects and methods it seems to grow to a huge size, both in the number of lines and the size of the final program.
What’s on Your Hard Drive?:
What’s on Your Hard Drive?
Tool I love! GCC. I enjoy GCC because each of its compilers does exactly what it is told-nothing more, nothing less. If you can’t write a program in your favorite language without the use of an IDE, you are not a programmer.
Kode Vicious Battles On:
Kode Vicious is at it again, dragging you out of your koding quagmires and kombating the enemies of kommon sense.
Dear KV, I’m maintaining some C code at work that is driving me right out of my mind. It seems I cannot go more than three lines in any file without coming across a chunk of code that is conditionally compiled.
Kode Vicious vs. Mothra:
A koder with attitude, KV answers your questions. Miss Manners he ain’t.
Dear KV, My co-workers keep doing really bad things in the code, such as writing C++ code with macros that have gotos that jump out of them, and using assert in lower-level functions as an error-handling facility. I keep trying to get them to stop doing these things, but the standard response I get is, “Yeah, it’s not pretty, but it works.” How can I get them to start asking, “Is there a better way to do this?” They listen to my arguments but don’t seem convinced. In some cases they even insist they are following good practices.
Mal Managerium: A Field Guide:
I have seen the enemy, and he is me.
Please allow me the pleasure of leading you on an “office safari,” so to speak. On today’s journey we’ll travel the corridors of computerdom in search of the widespread but elusive mal managerium, or bad manager, in common parlance. They will be difficult to spot because we will be in a sense looking for that most elusive creature of all: ourselves. That is to say, it’s quite possible that many of us will share some of the qualities with the various types of bad managers we shall encounter. Qualities that we are loath to admit we possess, I might add.
What’s on Your Hard Drive?:
What’s on Your Hard Drive?
Tool I love! DbVisualizer. This tool flatters my intuition. Whenever I’m working with a database and think, "There really ought to be a tool to ___________," DbVisualizer usually has what I need.
Kode Vicious Cycles On:
A koder with attitude, KV answers your questions. Miss Manners he ain’t.
Not only does California give you plenty of sun, it also apparently has employers that give you plenty of time to play around with the smaller problems that you like, in a programming language that’s irrelevant to the later implementation.
Call That Gibberish?:
Detecting the real from the fake is getting harder.
The Ninth World Multiconference SCI (Systematics, Cybernetics, and Informatics) 2005 has attracted more attention than its vaporific title usually merits by accepting a spoof paper from three MIT graduate students. The Times (of London, by default, of course) ran the eye-catching headline, “How gibberish put scientists to shame” (April 6, 2005). One of the students, Jeremy Stribling, explains how they had developed a computer program to generate random sequences of technobabble in order to confirm their suspicions that papers of dubious academicity were bypassing serious, or indeed, any scrutiny. In fact, the students claim ulterior, financial motives behind this lack of proper peer review.
What’s on Your Hard Drive?:
What’s on Your Hard Drive?
Tool I love! Python. I?m still a newbie to Python but I?m quite impressed with it thus far. As a scripting language, it can quickly test an idea or an algorithm, even if the project I?m working on doesn?t use Python. Also, with free tools such as wxPython and py2exe, a Python script can easily become a full-blown distributable application with a robust UI.
A Conversation with Roger Sessions and Terry Coatta:
The difference between objects and components? That’s debatable.
In the December/January 2004-2005 issue of Queue, Roger Sessions set off some fireworks with his article about objects, components, and Web services and which should be used when (“Fuzzy Boundaries,” 40-47). Sessions is on the board of directors of the International Association of Software Architects, the author of six books, writes the Architect Technology Advisory, and is CEO of ObjectWatch. He has a very object-oriented viewpoint, not necessarily shared by Queue editorial board member Terry Coatta, who disagreed with much of what Sessions had to say in his article. Coatta is an active developer who has worked extensively with component frameworks.
Kode Vicious Unscripted:
The problem? Computers make it too easy to copy data.
Some months, when he’s feeling ambitious, Kode Vicious reads through all of your letters carefully, agonizing for days over which to respond to. Most of the time, though, he takes a less measured approach. This usually involves printing the letters out, throwing them up in the air, and seeing which land face up, repeating the process until only two remain. And occasionally, KV dispenses with reader feedback altogether, as is the case this month.
Kode Vicious:
The Doctor is In
KV is back on duty and ready to treat another koding illness: bad APIs. This is one of the most widespread pathologies affecting, and sometimes infecting, us all. But whether we write APIs or simply use APIs (or both), we would all do well to read on and heed the vicious one’s advice.
A Conversation with Ray Ozzie:
Cooperate, Communicate, Collaborate
There are not many names bigger than Ray Ozzie’s in computer programming. An industry visionary and pioneer in computer-supported cooperative work, he began his career as an electrical engineer but fairly quickly got into computer science and programming. He is the creator of IBM’s Lotus Notes and is now chief technical officer of Microsoft, reporting to chief software architect Bill Gates. Recently, Ozzie’s role as chief technical officer expanded as he assumed responsibility for the company’s software-based services strategy across its three major divisions.
Stop Whining about Outsourcing!:
I’m sick of hearing all the whining about how outsourcing is going to migrate all IT jobs to the country with the lowest wages.
The paranoia inspired by this domino theory of job migration causes American and West European programmers to worry about India, Indian programmers to worry about China, Chinese programmers to worry about the Czech Republic, and so on. Domino theorists must think all IT jobs will go to the Republic of Elbonia, the extremely poor, fourth-world, Eastern European country featured in the Dilbert comic strip.
Gettin’ Your Kode On:
Dear KV, Simple question: When is the right time to call the c_str() method on a string to get the actual pointer?
Another year is upon us and we are happy to have Kode Vicious still ranting against the ills of insecure programming, insufficient commenting, and numerous other forms of koding malpractice. Yet despite his best efforts, the bittersweet truth is that these problems are not going away anytime soon, and therefore should continue to provide ample fodder for future KV columns. Oh, to live in a world that doesn’t need KV’s advice or doctors, for that matter.
Evolution or Revolution?:
Where is the High in High Tech?
We work in an industry that prides itself on "changing the world," one that chants a constant mantra of innovation and where new products could aptly be described as "this year’s breakthrough of the century." While there are some genuine revolutions in the technology industry, including cellphones, GPS (global positioning system), quantum computing, encryption, and global access to content, the vast majority of new product introductions are evolutionary, not revolutionary. Real technical breakthroughs are few and far between. Most new products are just a recycling of an earlier idea.
Logging on with KV:
A koder with attitude, KV answers your questions. Miss Manners he ain’t.
Dear KV, I’ve been stuck with writing the logging system for a new payment processing system at work. As you might imagine, this requires logging a lot of data because we have to be able to reconcile the data in our logs with our customers and other users, such as credit card companies, at the end of each billing cycle, and we have to be prepared if there is any argument over the bill itself. I’ve been given the job for two reasons: because I’m the newest person in the group and because no one thinks writing yet another logging system is very interesting.
Rationalizing a Home Terabyte Server:
Self-indulgent, or a view of the future?
With 1 TB of RAID 5 storage, most of my friends believe I have really gone off the deep end with my home server. They may be right, but as in most things in life, I have gotten to this point through a rational set of individual upgrades all perfectly reasonable at the time. Rather than being overly indulgent to my inner geek, am I an early adopter of what will be the inevitable standard for home IT infrastructure? Here is my story; you be the judge.
The Heart of Eclipse:
A look inside an extensible plug-in architecture
ECLIPSE is both an open, extensible development environment for building software and an open, extensible application framework upon which software can be built. Considered the most popular Java IDE, it provides a common UI model for working with tools and promotes rapid development of modular features based on a plug-in component model. The Eclipse Foundation designed the platform to run natively on multiple operating systems, including Macintosh, Windows, and Linux, providing robust integration with each and providing rich clients that support the GUI interactions everyone is familiar with: drag and drop, cut and paste (clipboard), navigation, and customization.
Breaking the Major Release Habit:
Can agile development make your team more productive?
Keeping up with the rapid pace of change can be a daunting task. Just as you finally get your software working with a new technology to meet yesterday’s requirements, a newer technology is introduced or a new business trend comes along to upset the apple cart. Whether your new challenge is Web services, SOA (service-oriented architecture), ESB (enterprise service bus), AJAX, Linux, the Sarbanes-Oxley Act, distributed development, outsourcing, or competitive pressure, there is an increasing need for development methodologies that help to shorten the development cycle time, respond to user needs faster, and increase quality all at the same time.
You Can Look It Up: or Maybe Not:
Chasing citations through endless, mislabeled nodes
Many are said to have said, "If I can’t take it with me, I’m not going!" I’ve just said it, but that hardly counts. Who, we demand, said or wrote it first? It’s what I call (and claim first rights on) a FUQ (frequently unanswerable question, pronounced fook to avoid ambiguity and altercation). Yogi Berra’s famous advice was "You can look it up," meaning, in fact, "Take my word on this." He knew quite well that few had the means or patience to wade through the records. Nowadays, of course, as we quip in Unix, it’s easier done than sed.
As Big as a Barn?:
Taking measure of measurement
The Texas rancher is trying to impress the English farmer with the size of his property. "I can drive out at dawn across my land, and by sundown I still haven’t reached my ranch’s borders." The Englishman nods sympathetically and says, "Yes, yes, I know what you mean. I have a car like that, too."
Alloneword:
Errors, deceptions, and abmiguity
Three years ago, to the very tick, my first Curmudgeon column appeared in ACM Queue to the rapturous, one-handed claps of the silent majority. Since then my essays have alternated intermittently with those of other grumpy contributors. With this issue (muffled drumroll), I’m proud to announce a Gore-like climate change in the regime that will redefine the shallow roots of ACJ (agile computer journalism, of which more anon). The astute ACM Queue Management (yes, there is such - you really must read the opening pages of this magazine!) has offered me the chance to go solo. For the next few Queues, at least, I am crowned King Curmudgeon, the Idi Amin of Moaners, nay, Supreme General Secretary of the Complaining Party!
Gettin’ Your Head Straight:
Kode Vicious is hungry. He sustains himself on your questions from the software development trenches (and lots of beer). Without your monthly missives, KV is like a fish out of water, or a scientist without a problem to solve. So please, do you part to keep him sane (or at least free from psychotic episodes), occupied, and useful.
Dear KV, One of the biggest problems I have is memory. Not the RAM in my computer, but the wet squishy stuff in my head. It seems that no matter how many signs I put up around my cube, nor how often I turn off all the annoying instant messaging clients I need to use for work, I can’t get through more than 15 minutes of work without someone interrupting me, and then I lose my train of thought. If this happens when I’m reading e-mail, that’s not a problem, but when working on code, in particular when debugging a difficult problem in code, this makes my life very difficult.
Five Steps to a Better Vista Installation - Transcript:
Unravel the mysteries and learn the best practices associated with mastering the new application installation routines for Vista applications. In this Premium Queuecast hosted by Michael Vizard, Bob Corrigan, senior manager for global product marketing at Macrovision, and Robert Dickau, principal trainer, reveal the five most crucial things you need to know about Vista application installations.
Joining me today is Bob Corrigan, Senior Manager for Global Product Marketing at Macrovision; and Robert Dickau, Principal Trainer for Macrovision. Today’s topic is Vista, and the installation routines around Vista and the opportunities that bring developers.
Software Operations’ Profit Potential:
Today’s software producer faces many challenges in building and keeping a satisfied customer base. In this ACM Premium Queuecast, Macrovision FLEXnet Publisher Product Manager Mitesh Pancholy discusses how companies can solve their license management challenges and turn their software operations into a profit center.
Joining me today is Mitesh Pancholy, Product Manager for Macrovision, and Abby Domini, also with Macrovision. We’re going to talk about license management today.
Managing Collaboration:
Jeff Johnstone of TechExcel explains why there is a need for a new approach to application lifecycle management that better reflects the business requirements and challenges facing development teams.
I think that fundamentally development is thought of, has become more of a business process than simply a set of tools. In the past, like you said, developers and development organizations were kind of on their own. They were fairly autonomous and they would do things that were appropriate for each piece of the process and they would adopt technologies that were appropriate at a technology and tool level, but they didn’t really think of themselves as an integral part of any higher business process.
The Yin and Yang of Software Development:
How infrastructure elements allow development teams to increase productivity without restricting creativity
The C/C++ Solution Manager at Parasoft explains how infrastructure elements allow development teams to increase productivity without restricting creativity.
There’s a Lot of It About:
And everybody’s doing it.
A lot of what, and about where? I hear you cry. One question at a time, I reply. First, there’s too much of everything these days, and, second, it’s happening all over. Furthermore, everybody’s doing it. As a contemporary Wordsworth might say: "The Web is too much with us, late and soon, getting and browsing we lay waste our powers." There is a glut of unfiltered information proving more dangerous than Alexander Pope’s "A Little Learning" where "shallow draughts intoxicate the brain."
A Conversation with Steve Bourne, Eric Allman, and Bryan Cantrill:
In part one of a two-part series, three Queue editorial board members discuss the practice of software engineering.
In part one of a two-part series, three Queue editorial board members discuss the practice of software engineering. In their quest to solve the next big computing problem or develop the next disruptive technology, software engineers rarely take the time to look back at the history of their profession. What’s changed? What hasn’t changed? In an effort to shed light on these questions, we invited three members of ACM Queue’s editorial advisory board to sit down and offer their perspectives on the continuously evolving practice of software engineering.
The Fabrication of Reality:
Is there an "out there" out there?
There are always anniversaries, real or concocted, to loosen the columnist’s writer’s block and/or justify the intake of alcohol. I’ll drink to that to the fact that we are blessed with a reasonably regular solar system providing a timeline of annual increments against which we can enumerate and toast past events. Hic semper hic. When the drinking occurs in sporadic and excessive bursts, it becomes known, disapprovingly, as "bingeing." I’m tempted to claim that this colorful Lincolnshire dialect word binge, meaning soak, was first used in the boozing-bout sense exactly 200 years ago. And that, shurely, calls for a schelebration.
A Conversation with Steve Bourne, Eric Allman, and Bryan Cantrill:
In part two of their discussion, our editorial board members consider XP and Agile.
In the July/August 2008 issue of ACM Queue we published part one of a two-part discussion about the practice of software engineering. The goal was to gain some perspective on the tools, techniques, and methodologies that software engineers use in their daily lives. Three members of Queue’s editorial advisory board participated: Steve Bourne, Eric Allman, and Bryan Cantrill, each of whom has made significant and lasting real-world contributions to the field (for more information on each of the participants, see part one). In part two we rejoin their conversation as they discuss XP (Extreme Programming) and Agile.
Don’t be Typecast as a Software Developer:
Kode Vicious’s temper obviously suffers from having to clean up after the mistakes of his peers. What would he have them learn now so that he can look forward to a graceful and mellow old age?
I would like to think that learning more will help me in my everyday job of writing glue and customization code at a systems integrator. But the obvious applicable knowledge is specific to tools and packages that may become obsolete or discontinued even within the lifetime of the project, and in some cases have already reached this destination.
One Peut-Être, Two Peut-Être, Three Peut-Être, More:
Puns and allusions
One is always loath to explain a joke. In face-to-face badinage, the joker can judge the comprehension of the jokes from their immediate reactions. Failing to win the approving smiles, chuckles, or belly laughs, the raconteur has a choice of remedies including the Quick Exit Strategy ("What a dumb crowd. I’m out of here!"). The modest teller will accept the blame ("Oh, I forgot to mention that the mother-in-law was a blond Republican Fortran programmer!") and order drinks all around.
Words Fail Them:
Dedesignating and other linguistic hazards
A recent announcement on the closing of an English nudist beach (have I captured your attention so early?) concluded with an apology to "all the naturalists" affected. This upset the "bird watchers," both naturalists and naturists (nudge, nudge), as well as those "word watchers" devoted to gooder English. Miffed and bemused letters appeared in Sally Baker’s London Times Feedback column, the traditional sounding board for disgruntled pop grammarians.
You Don’t Know Jack About Software Maintenance:
Long considered an afterthought, software maintenance is easiest and most effective when built into a system from the ground up.
Everyone knows maintenance is hard and boring, and avoids doing it. Besides, their pointy-haired bosses say things like: "No one needs to do maintenance - that’s a waste of time."
Merge Early, Merge Often:
Integrating changes in branched development
When doing merged development, how often should you merge? It’s obvious that if I wait too long, then I spend days in merge hell, where nothing seems to work and where I wind up using the revert command more often than commit; but the whole point of branched development is to be able to protect the main branch of development from unstable changes. Is there a happy middle ground?
Some Rules and Restrictions May Apply:
An inquiry into contracts and the Next Big Thing
In many of our interactions with the outside world (solipsists can stop reading now, if indeed they ever started) we enter into contracts with diverse entities, some up front, some lurking below the surface. The commonly construed contractual theme is a mutual agreement where each party accepts certain costs and responsibilities, and in return can rely on certain benefits and rewards.
Commitment Issues:
When is the right time to commit changes?
One of the other people on my project insists on checking in unrelated changes in large batches. When I say unrelated, what I mean is he will fix several unrelated bugs and then make a few minor changes to spacing and indentation across the entire source tree. He will then commit all of these changes at once, usually with a short commit message that lists only the bugs he claims to have fixed. Do you think I’m being too picky in wanting each checkin to address only one issue or problem?
Broken Builds:
Frequent broken builds could be symptomatic of deeper problems within a development project.
Is there anything more aggravating to programmers than fellow team members checking in code that breaks a build? I find myself constantly tracking down minor mistakes in other people’s code simply because they didn’t check that their changes didn’t break the build. The worst part is when someone has broken the build and they get indignant about my pointing it out. Are there any better ways to protect against these types of problems?
Simplicity Betrayed:
Emulating a video system shows how even a simple interface can be more complex—and capable—than it appears.
An emulator is a program that runs programs built for different computer architectures from the host platform that supports the emulator. Approaches differ, but most emulators simulate the original hardware in some way. At a minimum the emulator interprets the original CPU instructions and provides simulated hardware-level devices for input and output. For example, keyboard input is taken from the host platform and translated into the original hardware format, resulting in the emulated program "seeing" the same sequence of keystrokes. Conversely, the emulator will translate the original hardware screen format into an equivalent form on the host machine.
Avoiding Obsolescence:
Overspecialization can be the kiss of death for sysadmins.
Dear KV, What is the biggest threat to systems administrators? Not the technical threat (security, outages, etc.), but the biggest threat to systems administrators as a profession?
Collecting Counters:
Gathering statistics is important, but so is making them available to others.
Over the past month I’ve been trying to figure out a problem that occurs on our systems when the network is under heavy load. After about two weeks I was able to narrow down the problem from "the network is broken" (a phrase that my coworkers use mostly to annoy me), to being something that is going wrong on the network interfaces in our systems.
Tackling Architectural Complexity with Modeling:
Component models can help diagnose architectural problems in both new and existing systems.
The ever-increasing might of modern computers has made it possible to solve problems once thought too difficult to tackle. Far too often, however, the systems for these functionally complex problem spaces have overly complicated architectures. In this article I use the term architecture to refer to the overall macro design of a system rather than the details of how the individual parts are implemented. The system architecture is what is behind the scenes of usable functionality, including internal and external communication mechanisms, component boundaries and coupling, and how the system will make use of any underlying infrastructure (databases, networks, etc.) .
Gardening Tips:
A good library is like a garden.
I’ve been maintaining a set of libraries for my company for the past year. The libraries are used to interface to some special hardware that we sell, and all of the code we sell to our end users runs on top of the libraries, which talk, pretty much directly, to our hardware. The one problem I keep having is that the application programmers continually reach around the library to talk directly to the hardware, and this causes bugs in our systems because the library code maintains state about the hardware. If I make the library stateless, then every library call will have to talk to the hardware, which will slow down the library and all of the code that uses it.
B.Y.O.C. (1,342 Times and Counting):
Why can’t we all use standard libraries for commonly needed algorithms?
Although seldom articulated clearly, or even at all, one of the bedrock ideas of good software engineering is reuse of code libraries holding easily accessible implementations of common algorithms and facilities. The reason for this reticence is probably because there is no way to state it succinctly, without sounding like a cheap parody of Occam’s razor: It is pointless to do with several where few will suffice.
The One-second War (What Time Will You Die?):
As more and more systems care about time at the second and sub-second level, finding a lasting solution to the leap seconds problem is becoming increasingly urgent.
Thanks to a secretive conspiracy working mostly below the public radar, your time of death may be a minute later than presently expected. But don’t expect to live any longer, unless you happen to be responsible for time synchronization in a large network of computers, in which case this coup will lower your stress level a bit every other year or so. We’re talking about the abolishment of leap seconds, a crude hack added 40 years ago, to paper over the fact that planets make lousy clocks compared with quantum mechanical phenomena.
The Most Expensive One-byte Mistake:
Did Ken, Dennis, and Brian choose wrong with NUL-terminated text strings?
IT both drives and implements the modern Western-style economy. Thus, we regularly see headlines about staggeringly large amounts of money connected with IT mistakes. Which IT or CS decision has resulted in the most expensive mistake?
Managing Technical Debt:
Shortcuts that save money and time today can cost you down the road.
In 1992, Ward Cunningham published a report at OOPSLA (Object-oriented Programming, Systems, Languages, and Applications) in which he proposed the concept of technical debt. He defines it in terms of immature code: "Shipping first-time code is like going into debt." Technical debt isn’t limited to first-time code, however. There are many ways and reasons (not all bad) to take on technical debt.
A Generation Lost in the Bazaar:
Quality happens only when someone is responsible for it.
Thirteen years ago, Eric Raymond’s book "The Cathedral and the Bazaar" (O’Reilly Media, 2001) redefined our vocabulary and all but promised an end to the waterfall model and big software companies, thanks to the new grass-roots open source software development movement. I found the book thought provoking, but it did not convince me. On the other hand, being deeply involved in open source, I couldn’t help but think that it would be nice if he was right.
The Essence of Software Engineering: The SEMAT Kernel:
A thinking framework in the form of an actionable kernel
Everyone who develops software knows that it is a complex and risky business, and its participants are always on the lookout for new ideas that will lead to better software. Fortunately, software engineering is still a young and growing profession that sees innovations and improvements in best practices every year. Just look, for example, at the improvements and benefits that lean and agile thinking have brought to software-development teams.
Divided by Division:
Is there a best used-by date for software?
Do you know of any rule of thumb for how often a piece of software should need maintenance? I’m not thinking about bug fixes, since bugs are there from the moment the code is written, but about the constant refactoring that seems to go on in code. Sometimes I feel as if programmers use refactoring as a way of keeping their jobs, rather than offering any real improvement.
Swamped by Automation:
Whenever someone asks you to trust them, don’t.
So your group fell for the "just install this software and things will be great" ploy. It’s an old trick that continues to snag sysadmins and others who have supporting roles around developers. Whenever someone asks you to trust them, don’t. Cynical as that might be, it’s better than being suckered.
Cherry-picking and the Scientific Method:
Software is supposed be a part of computer science, and science demands proof.
So while haggling with the cherry seller, it became obvious that buying a whole flat of cherries would be a better deal than buying a single basket, even though that was all we really wanted. Not wanting to pass up a deal, however, my friend bought the entire flat and off we went, eating and talking. It took another 45 minutes to get home, and during that time we had eaten more than half the flat of cherries.
The Naming of Hosts is a Difficult Matter:
Also, the perils of premature rebooting
The naming of hosts is a difficult matter that ranks with coding style, editor choice, and language preference in the pantheon of things computer people fight about that don’t matter to anyone else in the whole world.
Online Algorithms in High-frequency Trading:
The challenges faced by competing HFT algorithms
HFT (high-frequency trading) has emerged as a powerful force in modern financial markets. Only 20 years ago, most of the trading volume occurred in exchanges such as the New York Stock Exchange, where humans dressed in brightly colored outfits would gesticulate and scream their trading intentions. Nowadays, trading occurs mostly in electronic servers in data centers, where computers communicate their trading intentions through network messages. This transition from physical exchanges to electronic platforms has been particularly profitable for HFT firms, which invested heavily in the infrastructure of this new environment.
Barbarians at the Gateways:
High-frequency Trading and Exchange Technology
I am a former high-frequency trader. For a few wonderful years I led a group of brilliant engineers and mathematicians, and together we traded in the electronic marketplaces and pushed systems to the edge of their capability.
Agile and SEMAT - Perfect Partners:
Combining agile and SEMAT yields more advantages than either one alone
Today, as always, many different initiatives are under way to improve the ways in which software is developed. The most popular and prevalent of these is the agile movement. One of the newer kids on the block is the SEMAT (Software Engineering Method and Theory) initiative. As with any new initiative, people are struggling to see how it fits into the world and relates to all the other things going on. For example, does it improve or replace their current ways of working?
Bugs and Bragging Rights:
It’s not always size that matters.
Dear KV, I’ve been dealing with a large program written in Java that seems to spend most of its time asking me to restart it because it has run out of memory.
The Software Inferno:
Dante’s tale, as experienced by a software architect
The Software Inferno is a tale that parallels The Inferno, Part One of The Divine Comedy written by Dante Alighieri in the early 1300s. That literary masterpiece describes the condemnation and punishment faced by a variety of sinners in their hell-spent afterlives as recompense for atrocities committed during their earthly existences. The Software Inferno is a similar account, describing a journey where "sinners against software" are encountered amidst their torment, within their assigned areas of eternal condemnation, and paying their penance.
This is the Foo Field:
The meaning of bits and avoiding upgrade bog downs
When will someone write documentation that tells you what the bits mean rather than what they set? I’ve been working to integrate a library into our system, and every time I try to figure out what it wants from my code, all it tells me is what a part of it is: "This is the foo field." The problem is that it doesn’t tell me what happens when I set foo. It’s as if I’m supposed to know that already.
The Logic of Logging:
And the illogic of PDF
I work in a pretty open environment, and by open I mean that many people have the ability to become the root user on our servers so that they can fix things as they break.
Major-league SEMAT: Why Should an Executive Care?:
Becoming better, faster, cheaper, and happier
In today’s ever more competitive world, boards of directors and executives demand that CIOs and their teams deliver "more with less." Studies show, without any real surprise, that there is no one-size-fits-all method to suit all software initiatives, and that a practice-based approach with some light but effective degree of order and governance is the goal of most software-development departments.
Forked Over:
Shortchanged by open source
How can one make reasonable packages based on open-source software when most open-source projects simply advise you to take the latest bits on GitHub or SourceForge? We could fork the code, as GitHub encourages us to do, and then make our own releases, but that puts the release-engineering work that we would expect from the project onto us.
ACM and the Professional Programmer:
How do you, the reader, stay informed about research that influences your work?
In the very early days of computing, professional programming was nearly synonymous with academic research because computers tended to be devices that existed only or largely in academic settings. As computers became commercially available, they began to be found in private-sector, business environments. The 1950s and 1960s brought computing in the form of automation and data processing to the private sector and, along with this came a growing community of professionals whose focus on computing was pragmatic and production oriented. Computing was (and still is) evolving, and the academic community continued to explore new software and hardware concepts and constructs. New languages were invented (and are still being invented) to try new ideas in the formulation of programs.
The Responsive Enterprise: Embracing the Hacker Way:
Soon every company will be a software company.
As of July 2014, Facebook, founded in 2004, is in the top 20 of the most valuable companies in the S&P 500, putting the 10-year-old software company in the same league as IBM, Oracle, and Coca-Cola. Of the top five fastest-growing companies with regard to market capitalization in 2014 (table 1), three are software companies: Apple, Google, and Microsoft (in fact, one could argue that Intel is also driven by software, making it four out of five).
A New Software Engineering:
What happened to the promise of rigorous, disciplined, professional practices for software development?
What happened to software engineering? What happened to the promise of rigorous, disciplined, professional practices for software development, like those observed in other engineering disciplines? What has been adopted under the rubric of "software engineering" is a set of practices largely adapted from other engineering disciplines: project management, design and blueprinting, process control, and so forth. The basic analogy was to treat software as a manufactured product, with all the real "engineering" going on upstream of that - in requirements analysis, design, modeling, etc.
Raw Networking:
Relevance and repeatability
Dear KV, The company I work for has decided to use a wireless network link to reduce latency, at least when the weather between the stations is good. It seems to me that for transmission over lossy wireless links we’ll want our own transport protocol that sits directly on top of whatever the radio provides, instead of wasting bits on IP and TCP or UDP headers, which, for a point-to-point network, aren’t really useful.
Hickory Dickory Doc:
On null encryption and automated documentation
Dear KV, While reviewing some encryption code in our product, I came across an option that allowed for null encryption. This means the encryption could be turned on, but the data would never be encrypted or decrypted. It would always be stored "in the clear." I removed the option from our latest source tree because I figured we didn’t want an unsuspecting user to turn on encryption but still have data stored in the clear. One of the other programmers on my team reviewed the potential change and blocked me from committing it, saying that the null code could be used for testing.
Still Finding the Right Questions:
Branching out and changing with the times at acmqueue
Welcome to the newest incarnation of acmqueue. When we started putting together the first edition of ACM Queue in early 2003, it was a completely new experiment in publishing for ACM. Targeting a practitioner audience meant that much of what we did would differ from academic publishing. We created a new editorial board whose role was not only to vet articles, but also to identify topics and authors that would be of interest to both practitioners and academics. The board created the concept of guest experts who would take on an issue of the magazine and help the board acquire content and sometimes write the overarching piece that tied it all together.
Lean Software Development - Building and Shipping Two Versions:
Catering to developers’ strengths while still meeting team objectives
Once I was managing a software team and we were working on several initiatives. Projects were assigned based on who was available, their skillsets, and their development goals. This resulted in two developers, Mary and Melissa, being assigned to the same project.
Automation Should Be Like Iron Man, Not Ultron:
The "Leftover Principle" Requires Increasingly More Highly-skilled Humans.
A few years ago we automated a major process in our system administration team. Now the system is impossible to debug. Nobody remembers the old manual process and the automation is beyond what any of us can understand. We feel like we’ve painted ourselves into a corner. Is all operations automation doomed to be this way?
It Probably Works:
Probabilistic algorithms are all around us--not only are they acceptable, but some programmers actually seek out chances to use them.
Probabilistic algorithms exist to solve problems that are either impossible or unrealistic (too expensive, too time-consuming, etc.) to solve precisely. In an ideal world, you would never actually need to use probabilistic algorithms. To programmers who are not familiar with them, the idea can be positively nervewracking: "How do I know that it will actually work? What if it’s inexplicably wrong? How can I debug it? Maybe we should just punt on this problem, or buy a whole lot more servers..."
Pickled Patches:
On repositories of patches and tension between security professionals and in-house developers
I recently came upon a software repository that was not a repo of code, but a repo of patches. The project seemed to build itself out of several other components and then had complicated scripts that applied the patches in a particular order. I had to look at this repo because I wanted to fix a bug in the system, but trying to figure out what the code actually looked like at any particular point in time was baffling. Are there tools that would help in working like this?
The Paradox of Autonomy and Recognition:
Thoughts on trust and merit in software team culture
Who doesn’t want recognition for their hard work and contributions? Early in my career I wanted to believe that if you worked hard, and added value, you would be rewarded. I wanted to believe in the utopian ideal that hard work, discipline, and contributions were the fuel that propelled you up the corporate ladder. Boy, was I wrong.
GNL is Not Linux:
What’s in a Name?
What, indeed, is in a name? As you’ve already seen, this quasi-technical topic continues to cause a bit of heat in the software community, particularly in the open-source world. You can find the narrative from the GNU side by clicking on the link provided in the postscript to this article, but KV finds that narrative lacking, and so, against my better judgment about pigs and dancing, I will weigh in with a few comments.
Use-Case 2.0:
The Hub of Software Development
Use cases have been around for almost 30 years as a requirements approach and have been part of the inspiration for more-recent techniques such as user stories. Now the inspiration has flown in the other direction. Use-Case 2.0 is the new generation of use-case-driven development-light, agile, and lean-inspired by user stories and the agile methodologies Scrum and Kanban.
Delegation as Art:
Be someone who makes everyone else better.
When I started my career as a junior engineer, I couldn’t wait to be senior. I would regularly review our promotion guidelines and assess my progress and contributions against them. Of course, at the time I didn’t really understand what being senior meant. Being a senior engineer means having strong technical skills, the ability to communicate well and navigate ambiguous situations, and most important of all, the ability to grow and lead other people. Leadership isn’t just for managers anymore.
What Are You Trying to Pull?:
A single cache miss is more expensive than many instructions.
Saving instructions - how very 1990s of him. It’s always nice when people pay attention to details, but sometimes they simply don’t pay attention to the right ones. While KV would never encourage developers to waste instructions, given the state of modern software, it does seem like someone already has. KV would, as you did, come out on the side of legibility over the saving of a few instructions.
Nine Things I Didn’t Know I Would Learn Being an Engineer Manager:
Many of the skills aren’t technical at all.
When I moved from being an engineer to being a dev lead, I knew I had a lot to learn. My initial thinking was that I had to be able to do thorough code reviews, design and architect websites, see problems before they happened, and ask insightful technical questions. To me that meant learning the technology and becoming a better engineer. When I actually got into the role (and after doing it almost 15 years), the things I have learned--and that have mattered the most--weren’t those technical details.
Introducing Research for Practice:
Expert-curated guides to the best of CS research
Reading a great research paper is a joy. A team of experts deftly guides you, the reader, through the often complicated research landscape, noting the prior art, the current trends, the pressing issues at hand--and then, sometimes artfully, sometimes through seeming sheer force of will, expands the body of knowledge in a fell swoop of 12 or so pages of prose. A great paper contains a puzzle and a solution; these can be useful, enlightening, or both.
Chilling the Messenger:
Keeping ego out of software-design review
Trying to correct someone who has just done a lot of work, even if, ultimately, that work is not the right work, is a daunting task. The person in question no doubt believes that he has worked very hard to produce something of value to the rest of the team, and walking in and spitting on it, literally or metaphorically, probably crosses your "offense" line--at least I think it does. I’m a bit surprised that since this is the first sprint and there is already so much code written, shouldn’t the software have shown up after the sprints established what was needed, who the stakeholders were, etc.?
Dynamics of Change: Why Reactivity Matters:
Tame the dynamics of change by centralizing each concern in its own module.
Professional programming is about dealing with software at scale. Everything is trivial when the problem is small and contained: it can be elegantly solved with imperative programming or functional programming or any other paradigm. Real-world challenges arise when programmers have to deal with large amounts of data, network requests, or intertwined entities, as in UI (user interface) programming.
Cloud Calipers:
Naming the next generation and remembering that the cloud is just other people’s computers
For the time being, we are likely to continue to have programmers who version their functions as a result of the limitations of their languages, but let’s hope we can stop them naming their next generations after the next generation.
Fresh Starts:
Just because you have been doing it the same way doesn’t mean you are doing it the right way.
I love fresh starts. Growing up, one of my favorite things was starting a new school year. From the fresh school supplies to the promise of a new class of students, teachers, and lessons, I couldn’t wait for summer to be over and to go back to school. The same thing happens with new jobs. They reinvigorate you, excite you, and get you going.
Industrial Scale Agile - from Craft to Engineering:
Essence is instrumental in moving software development toward a true engineering discipline.
There are many, many ways to illustrate how fragile IT investments can be. You just have to look at the way that, even after huge investments in education and coaching, many organizations are struggling to broaden their agile adoption to the whole of their organization - or at the way other organizations are struggling to maintain the momentum of their agile adoptions as their teams change and their systems mature.
Resolving Conflict:
Don’t "win." Resolve.
I am conflicted about conflict. On one hand, I hate it. Hearing people disagree, even about minor things, makes me want to run through the nearest wall and curl up under my bed until it’s over. On the other hand, when it happens, I always want to get into it.
Does Anybody Listen to You?:
How do you step up from mere contributor to real change-maker?
An idea on its own is not worth much. Just because you think you know a better way to do something, even if you’re right, no one is required to care. Making great things happen at work is about more than just being smart. Good ideas succeed or fail depending on your ability to communicate them correctly to the people who have the power to make them happen. When you are navigating an organization, it pays to know whom to talk to and how to reach them. Here is a simple guide to sending your ideas up the chain and actually making them stick.
Side Effects, Front and Center!:
One System’s Side Effect is Another’s Meat and Potatoes.
We think of computation in terms of its consequences. The big MapReduce job returns a large result. Web interactions display information. Enterprise applications update the database and return an answer. These are the reasons we do our work. What we rarely discuss are the side effects of doing the work we intend. Side effects may be unwanted, or they may actually cause desired behavior at different layers of the system. This column points out some fun patterns to keep in mind as we build and use our systems.
Research for Practice: Technology for UnderservedCommunities; Personal Fabrication:
Expert-curated Guides to the Best of CS Research
This installment of Research for Practice provides curated reading guides to technology for underserved communities and to new developments in personal fabrication. First, Tawanna Dillahunt describes design considerations and technology for underserved and impoverished communities. Designing for the more than 1.6 billion impoverished individuals worldwide requires special consideration of community needs, constraints, and context. Tawanna’s selections span protocols for poor-quality communication networks, community-driven content generation, and resource and public service discovery. Second, Stefanie Mueller and Patrick Baudisch provide an overview of recent advances in personal fabrication (e.g., 3D printers). Their selection covers new techniques for fabricating (and emulating) complex materials (e.g., by manipulating the internal structure of an object), for more easily specifying object shape and behavior, and for human-in-the-loop rapid prototyping.
Is There a Single Method for the Internet of Things?:
Essence can keep software development for the IoT from becoming unwieldy.
The Industrial Internet Consortium predicts the IoT (Internet of Things) will become the third technological revolution after the Industrial Revolution and the Internet Revolution. Its impact across all industries and businesses can hardly be imagined. Existing software (business, telecom, aerospace, defense, etc.) is expected to be modified or redesigned, and a huge amount of new software, solving new problems, will have to be developed. As a consequence, the software industry should welcome new and better methods.
Operational Excellence in April Fools’ Pranks:
Being funny is serious work.
Successful pranks require care and planning. Write a design proposal and a project plan. Involve operations early. If this is a technical change to your website, perform load testing, preferably including a "dark launch" or hidden launch test. Hide the prank behind a feature flag rather than requiring a new software release. Perform a retrospective and publish the results widely. Remember that some of the best pranks require little or no technical changes at all. For example, one could simply summarize the best practices for launching any new feature but write it under the guise of how to launch an April Fools’ prank.
Popping Kernels:
Choosing between programming in the kernel or in user space
In a world in which high-performance code continues to be written in a fancy assembler, a.k.a. C, with no memory safety and plenty of other risks, the only recourse is to stick to software engineering basics. Reduce the amount of code in harm’s way, keep coupling between subsystems efficient and explicit, and work to provide better tools for the job, such as static code checkers and large suites of runtime tests.
DevOps Metrics:
Your biggest mistake might be collecting the wrong data.
Delivering value to the business through software requires processes and coordination that often span multiple teams across complex systems, and involves developing and delivering software with both quality and resiliency. As practitioners and professionals, we know that software development and delivery is an increasingly difficult art and practice, and that managing and improving any process or system requires insights into that system. Therefore, measurement is paramount to creating an effective software value stream. Yet accurate measurement is no easy feat.
Continuous Delivery Sounds Great, but Will It Work Here?:
It’s not magic, it just requires continuous, daily improvement at all levels.
Continuous delivery is a set of principles, patterns, and practices designed to make deployments predictable, routine affairs that can be performed on demand at any time. This article introduces continuous delivery, presents both common objections and actual obstacles to implementing it, and describes how to overcome them using real-life examples. Continuous delivery is not magic. It’s about continuous, daily improvement at all levels of the organization.
Manual Work is a Bug:
A.B.A: always be automating
Every IT team should have a culture of constant improvement - or movement along the path toward the goal of automating whatever the team feels confident in automating, in ways that are easy to change as conditions change. As the needle moves to the right, the team learns from each other’s experiences, and the system becomes easier to create and safer to operate. A good team has a structure in place that makes the process frictionless and collaborative
GitOps: A Path to More Self-service IT:
IaC + PR = GitOps
GitOps lowers the bar for creating self-service versions of common IT processes, making it easier to meet the return in the ROI calculation. GitOps not only achieves this, but also encourages desired behaviors in IT systems: better testing, reduction of bus factor, reduced wait time, more infrastructure logic being handled programmatically with IaC, and directing time away from manual toil toward creating and maintaining automation.
Tear Down the Method Prisons! Set Free the Practices!:
Essence: a new way of thinking that promises to liberate the practices and enable true learning organizations
This article explains why we need to break out of this repetitive dysfunctional behavior, and it introduces Essence, a new way of thinking that promises to free the practices from their method prisons and thus enable true learning organizations.
Know Your Algorithms:
Stop using hardware to solve software problems.
Knowing that your CPU is in use 100 percent of the time doesn’t tell you much about the overall system other than it’s busy, but busy with what? Maybe it’s sitting in a tight loop, or some clown added a bunch of delay loops during testing that are no longer necessary. Until you profile your system, you have no idea why the CPU is busy. All systems provide some form of profiling so that you can track down where the bottlenecks are, and it’s your responsibility to apply these tools before you spend money on brand new hardware.
SageDB and NetAccel:
Learned models within the database system; network-accelerated query processing
The CIDR (Conference on Innovative Data Systems Research) runs once every two years, and luckily for us 2019 is one of those years. I’ve selected two papers from this year’s conference that highlight bold and exciting directions for data systems.
The Worst Idea of All Time:
Revelations at 100!
In February 2004, with the other members of the Queue editorial board, I was at our monthly in-person dinner meeting, where we gather to come up with interesting discussion topics that will result in practitioner-oriented articles (and the best authors to write them) for publication in Queue. It was only our second year in business, and although we had published some successful and widely read articles, Queue still had no regular columnists. I was initially invited to board meetings by another editorial board member, Eric Allman, and had written a couple of articles for the publication. I was also co-authoring my first book but had never been a columnist.
Industry-scale Knowledge Graphs: Lessons and Challenges:
Five diverse technology companies show how it’s done
This article looks at the knowledge graphs of five diverse tech companies, comparing the similarities and differences in their respective experiences of building and using the graphs, and discussing the challenges that all knowledge-driven enterprises face today. The collection of knowledge graphs discussed here covers the breadth of applications, from search, to product descriptions, to social networks.
The DevOps Phenomenon:
An executive crash course
Stressful emergency releases are a thing of the past for companies that subscribe to the DevOps method of software development and delivery. New releases are frequent. Bugs are fixed rapidly. New business opportunities are sought with gusto and confidence. New features are released, revised, and improved with rapid iterations. DevOps presents a strategic advantage for organizations when compared with traditional software-development methods. Leadership plays an important role during that transformation. DevOps is about providing guidelines for faster time to market of new software features and achieving a higher level of stability. Implementing cross-functional, product-oriented teams helps bridge the gaps between software development and operations.
MUST and MUST NOT:
On writing documentation
Pronouncements without background or explanatory material are useless to those who are not also deeply steeped in the art and science of computer security or security in general. It takes a particular bend of mind to think like an attacker and a defender all at once, and most people are incapable of doing this; so, if you want the people reading the document to follow your guidance, then you must take them on a journey from ignorance to knowledge.
Surviving Software Dependencies:
Software reuse is finally here but comes with risks.
Software reuse is finally here, and its benefits should not be understated, but we’ve accepted this transformation without completely thinking through the potential consequences. The Copay and Equifax attacks are clear warnings of real problems in the way software dependencies are consumed today. There’s a lot of good software out there. Let’s work together to find out how to reuse it safely.
Velocity in Software Engineering:
From tectonic plate to F-16
Software engineering occupies an increasingly critical role in companies across all sectors, but too many software initiatives end up both off target and over budget. A surer path is optimized for speed, open to experimentation and learning, agile, and subject to regular course correcting. Good ideas tend to be abundant, though execution at high velocity is elusive. The good news is that velocity is controllable; companies can invest systematically to increase it.
Revealing the Critical Role of Human Performance in Software:
It’s time to revise our appreciation of the human side of Internet-facing software systems.
Understanding, supporting, and sustaining the capabilities above the line of representation require all stakeholders to be able to continuously update and revise their models of how the system is messy and yet usually manages to work. This kind of openness to continually reexamine how the system really works requires expanding the efforts to learn from incidents.
Above the Line, Below the Line:
The resilience of Internet-facing systems relies on what is above the line of representation.
Knowledge and understanding of below-the-line structure and function are continuously in flux. Near-constant effort is required to calibrate and refresh the understanding of the workings, dependencies, limitations, and capabilities of what is present there. In this dynamic situation no individual or group can ever know the system state. Instead, individuals and groups must be content with partial, fragmented mental models that require more or less constant updating and adjustment if they are to be useful.
Cognitive Work of Hypothesis Exploration During Anomaly Response:
A look at how we respond to the unexpected
Four incidents from web-based software companies reveal important aspects of anomaly response processes when incidents arise in web operations, two of which are discussed in this article. One particular cognitive function examined in detail is hypothesis generation and exploration, given the impact of obscure automation on engineers’ development of coherent models of the systems they manage. Each case was analyzed using the techniques and concepts of cognitive systems engineering. The set of cases provides a window into the cognitive work "above the line" in incident management of complex web-operation systems.
Managing the Hidden Costs of Coordination:
Controlling coordination costs when multiple, distributed perspectives are essential
Some initial considerations to control cognitive costs for incident responders include: (1) assessing coordination strategies relative to the cognitive demands of the incident; (2) recognizing when adaptations represent a tension between multiple competing demands (coordination and cognitive work) and seeking to understand them better rather than unilaterally eliminating them; (3) widening the lens to study the joint cognition system (integration of human-machine capabilities) as the unit of analysis; and (4) viewing joint activity as an opportunity for enabling reciprocity across inter- and intra-organizational boundaries.
Beyond the Fix-it Treadmill:
The Use of Post-Incident Artifacts in High-Performing Organizations
Given that humanity’s study of the sociological factors in safety is almost a century old, the technology industry’s post-incident analysis practices and how we create and use the artifacts those practices produce are all still in their infancy. So don’t be surprised that many of these practices are so similar, that the cognitive and social models used to parse apart and understand incidents and outages are few and cemented in the operational ethos, and that the byproducts sought from post-incident analyses are far-and-away focused on remediation items and prevention.
How Do Committees Invent? and Ironies of Automation:
The formulation of Conway’s law and the counterintuitive consequences of increasing levels of automation
The Lindy effect tells us that if a paper has been highly relevant for a long time, it’s likely to continue being so for a long time to come as well. My first choice is "How Do Committees Invent?" Author Melvin E. Conway provides a lot of great material that led up to the formulation of the law that bears his name. My second choice is Lisanne Bainbridge’s "Ironies of Automation." It’s a classic treatise on the counterintuitive consequences of increasing levels of automation.
Removing Kode:
Dead functions and dead features
Removing dead code from systems is one of KV's favorite koding pastimes because there is nothing quite like that feeling you get when you get rid of something you know wasn't being used. Code removal is like cleaning house, only sometimes you clean house with a flame thrower, which, honestly, is very satisfying. Since you're using a version-control system (you had better be using a VCS!), it's very easy to remove code without worry. If you ever need the code you removed, you can retrieve it from the VCS at will.
Out-of-this-World Additive Manufacturing:
From thingamabobs to rockets, 3D printing takes many forms.
Popular culture uses the term 3D printing as a synonym for additive manufacturing processes. In 2010, the American Society for Testing and Materials group came up with a set of standards to classify additive manufacturing processes into seven categories. Each process uses different materials and machine technology, which affects the use cases and applications, as well as the economics. I went down a rabbit hole researching the various processes in my hunt to buy the best 3D printer.
Best Practice: Application Frameworks:
While powerful, frameworks are not for everyone.
While frameworks can be a powerful tool, they have some disadvantages and may not make sense for all organizations. Framework maintainers need to provide standardization and well-defined behavior while not being overly prescriptive. When frameworks strike the right balance, however, they can offer large developer productivity gains. The consistency provided by widespread use of frameworks is a boon for other teams such as SRE and security that have a vested interest in the quality of applications. Additionally, the structure of frameworks provides a foundation for building higher-level abstractions such as microservices platforms, which unlock new opportunities for system architecture and automation.
Offline Algorithms in Low-Frequency Trading:
Clearing Combinatorial Auctions
Expectations run high for software that makes real-world decisions, particularly when money hangs in the balance. This third episode of the Drill Bits column shows how well-designed software can effectively create wealth by optimizing gains from trade in combinatorial auctions. We'll unveil a deep connection between auctions and a classic textbook problem, we'll see that clearing an auction resembles a high-stakes mutant Tetris, we'll learn to stop worrying and love an NP-hard problem that's far from intractable in practice, and we'll contrast the deliberative business of combinatorial auctions with the near-real-time hustle of high-frequency trading.
The SPACE of Developer Productivity:
There's more to it than you think.
Developer productivity is about more than an individual's activity levels or the efficiency of the engineering systems relied on to ship software, and it cannot be measured by a single metric or dimension. The SPACE framework captures different dimensions of productivity, and here we demonstrate how this framework can be used to understand productivity in practice and why using it will help teams better understand developer productivity and create better measures to inform their work and teams.
Aversion to Versions:
Resolving code-dependency issues
One should never hardcode a version or a path inside the code itself. Code needs to be flexible so that it can be installed anywhere and run anywhere so long as the necessary dependencies can be resolved, either at build time for statically compiled code or at runtime for interpreted code or code with dynamically linked libraries. There are current, good ways to get this right, so it's a shame that so many people continue to get it wrong.
Software Development in Disruptive Times:
Creating a software solution with fast decision capability, agile project management, and extreme low-code technology
In this project, the challenge was to "deploy software faster than the coronavirus spread." In a project with such peculiar characteristics, several factors can influence success, but some clearly stand out: top management support, agility, understanding and commitment of the project team, and the technology used. Conventional development approaches and technologies would simply not be able to meet the requirements promptly.
Schrödinger's Code:
Undefined behavior in theory and practice
Undefined behavior ranks among the most baffling and perilous aspects of popular programming languages. This installment of Drill Bits clears up widespread misconceptions and presents practical techniques to banish undefined behavior from your own code and pinpoint meaningless operations in any software—techniques that reveal alarming faults in software supporting business-critical applications at Fortune 500 companies.
In Praise of the Disassembler:
There's much to be learned from the lower-level details of hardware.
When you're starting out you want to be able to hold the entire program in your head if at all possible. Once you're conversant with your first, simple assembly language and the machine architecture you're working with, it will be completely possible to look at a page or two of your assembly and know not only what it is supposed to do but also what the machine will do for you step by step. When you look at a high-level language, you should be able to understand what you mean it to do, but often you have no idea just how your intent will be translated into action.
Static Analysis: An Introduction:
The fundamental challenge of software engineering is one of complexity.
Modern static-analysis tools provide powerful and specific insights into codebases. The Linux kernel team, for example, developed Coccinelle, a powerful tool for searching, analyzing, and rewriting C source code; because the Linux kernel contains more than 27 million lines of code, a static-analysis tool is essential both for finding bugs and for making automated changes across its many libraries and modules. Another tool targeted at the C family of languages is Clang scan-build, which comes with many useful analyses and provides an API for programmers to write their own analyses. Like so many things in computer science, the utility of static analysis is self-referential: To write reliable programs, we must also write programs for our programs.
Static Analysis at GitHub:
An experience report
The Semantic Code team at GitHub builds and operates a suite of technologies that power symbolic code navigation on github.com. We learned that scale is about adoption, user behavior, incremental improvement, and utility. Static analysis in particular is difficult to scale with respect to human behavior; we often think of complex analysis tools working to find potentially problematic patterns in code and then trying to convince the humans to fix them.
Human-Centered Approach to Static-Analysis-Driven Developer Tools:
The future depends on good HCI
Complex and opaque systems do not scale easily. A human-centered approach for evolving tools and practices is essential to ensuring that software is scaled safely and securely. Static analysis can unveil information about program behavior, but the goal of deriving this information should not be to accumulate hairsplitting detail. HCI can help direct static-analysis techniques into developer-facing systems that structure information and embody relationships in representations that closely mirror a programmer's thought. The survival of great software depends on programming languages that support, rather than inhibit, communicating, reasoning, and abstract thinking.
Designing UIs for Static Analysis Tools:
Evaluating tool design guidelines with SWAN
Static-analysis tools suffer from usability issues such as a high rate of false positives, lack of responsiveness, and unclear warning descriptions and classifications. Here, we explore the effect of applying user-centered approach and design guidelines to SWAN, a security-focused static-analysis tool for the Swift programming language. SWAN is an interesting case study for exploring static-analysis tool usability because of its large target audience, its potential to integrate easily into developers' workflows, and its independence from existing analysis platforms.
Meaning and Context in Computer Programs:
Sharing domain knowledge among programmers using the source code as the medium
When you look at a function program's source code, how do you know what it means? Is the meaning found in the return values of the function, or is it located inside the function body? What about the function name? Answering these questions is important to understanding how to share domain knowledge among programmers using the source code as the medium. The program is the medium of communication among programmers to share their solutions.
Middleware 101:
What to know now and for the future
Whether segregating a sophisticated software component into smaller services, transferring data between computers, or creating a general gateway for seamless communication, you can rely on middleware to achieve communication between different devices, applications, and software layers. Following the increasing agile movement, the tech industry has adopted the use of fast waterfall models to create stacks of layers for each structural need, including integration, communication, data, and security. Given this scope, emphasis must now be on endpoint connection and agile development. This means that middleware should not serve solely as an object-oriented solution to execute simple request-response commands.
Long Live Software Easter Eggs!
It's a period of unrest. Rebel developers, striking from continuous deployment servers, have won their first victory. During the battle, rebel spies managed to push an epic commit in the HTML code of https://pro.sony. Pursued by sinister agents, the rebels are hiding in commits, buttons, tooltips, API, HTTP headers, and configuration screens.
Linear Address Spaces:
Unsafe at any speed
The linear address space as a concept is unsafe at any speed, and it badly needs mandatory CHERI seat belts. But even better would be to get rid of linear address spaces entirely and go back to the future, as successfully implemented in the Rational R1000 computer 30-plus years ago.
The Challenges of IoT, TLS, and Random Number Generators in the Real World:
Bad random numbers are still with us and are proliferating in modern systems.
Many in the cryptographic community scoff at the mistakes made in implementing RNGs. Many cryptographers and members of the IETF resist the call to make TLS more resilient to this class of failures. This article discusses the history, current state, and fragility of the TLS protocol, and it closes with an example of how to improve the protocol. The goal is not to suggest a solution but to start a dialog to make TLS more resilient by proving that the security of TLS without the assumption of perfect random numbers is possible.
The Four Horsemen of an Ailing Software Project:
Don't let the pale rider catch you with an exception.
KV has talked about various measures of software quality in past columns, but perhaps falling software quality is one of the most objective measures that a team is failing. This Pestilence, brought about by the low morale engendered in the team by War and Famine, is a clear sign that something is wrong. In the real world, a diseased animal can be culled so that disease does not spread and become a pestilence over the land. Increasing bug counts, especially in the absence of increased functionality, is a sure sign of a coming project apocalypse.
OCCAM-v2: Combining Static and Dynamic Analysis for Effective and Efficient Whole-program Specialization:
Leveraging scalable pointer analysis, value analysis, and dynamic analysis
OCCAM-v2 leverages scalable pointer analysis, value analysis, and dynamic analysis to create an effective and efficient tool for specializing LLVM bitcode. The extent of the code-size reduction achieved depends on the specific deployment configuration. Each application that is to be specialized is accompanied by a manifest that specifies concrete arguments that are known a priori, as well as a count of residual arguments that will be provided at runtime. The best case for partial evaluation occurs when the arguments are completely concretely specified. OCCAM-v2 uses a pointer analysis to devirtualize calls, allowing it to eliminate the entire body of functions that are not reachable by any direct calls.
Use Cases are Essential:
Use cases provide a proven method to capture and explain the requirements of a system in a concise and easily understood format.
While the software industry is a fast-paced and exciting world in which new tools, technologies, and techniques are constantly being developed to serve business and society, it is also forgetful. In its haste for fast-forward motion, it is subject to the whims of fashion and can forget or ignore proven solutions to some of the eternal problems that it faces. Use cases, first introduced in 1986 and popularized later, are one of those proven solutions.
Low-code Development Productivity:
"Is winter coming" for code-based technologies?
This article aims to provide new insights on the subject by presenting the results of laboratory experiments carried out with code-based, low-code, and extreme low-code technologies to study differences in productivity. Low-code technologies have clearly shown higher levels of productivity, providing strong arguments for low-code to dominate the software development mainstream in the short/medium term. The article reports the procedure and protocols, results, limitations, and opportunities for future research.
Dear Diary:
On keeping a laboratory notebook
While a debug log is helpful, it's not the same thing as a laboratory notebook. If more computer scientists acted like scientists, we wouldn't have to fight over whether computing is an art or a science.
DevEx in Action:
A study of its tangible impacts
DevEx (developer experience) is garnering increased attention at many software organizations as leaders seek to optimize software delivery amid the backdrop of fiscal tightening and transformational technologies such as AI. Intuitively, there is acceptance among technical leaders that good developer experience enables more effective software delivery and developer happiness. Yet, at many organizations, proposed initiatives and investments to improve DevEx struggle to get buy-in as business stakeholders question the value proposition of improvements.
Challenges in Adopting and Sustaining Microservice-based Software Development:
Organizational challenges can be more difficult than technical ones.
MS (microservice) has become the latest buzzword in software development. The MS approach to software development offers an alternative to the conventional monolith style. While benefits of MS-based development over monolith style are clear, industry experts agree that neither style provides an absolute advantage in all situations. Proponents contend that an MS approach to software development more readily facilitates mapping organizational changes manifesting from a more dynamic business environment to corresponding IT/IS (information technology/information systems) changes. This article identifies key challenges from the initial decision to adopt MSs to the ongoing task of sustaining the new paradigm over the long haul.
Make Two Trips:
Larry David's New Year's resolution works for IT too.
Whether your project is as simple as carrying groceries into the house or as complex as a multiyear engineering project, "make two trips" can simplify the project, reduce the chance of error, improve the probability of success, and lead to easier explanations.
Structuring Success:
The problem with software structure is people don't really learn it until they really need it.
The problem with software structure is people don't really learn it until they really need it, and your homework assignments will have to force them to look closely at how they are solving problems, not only through algorithms but through structure.
Zero Tolerance for Bias
From gambling to military conscription, randomization makes crucial real-world decisions. With blood and treasure at stake, fairness is not negotiable. Unfortunately, bad advice and biased methods abound. We'll learn how to navigate around misinformation, develop sound methods, and compile checklists for design and code reviews.
Working Models for Tackling Tech Debt:
Understand the options to tailor an approach that suits your needs
Remember that not all debt is bad, and sometimes, in fact, strategic tech debt can even be used as a valuable tool to achieve certain business goals?just as financial debt can be taken on to obtain capital that can be invested in other profitable ventures. For example, taking a shortcut to get a product to market quickly could prove to be a wise decision if it allows the company to learn from customer feedback and then iterate accordingly on the product.
Questioning the Criteria for Evaluating Non-cryptographic Hash Functions:
Maybe we need to think more about non-cryptographic hash functions.
Although cryptographic and non-cryptographic hash functions are everywhere, there seems to be a gap in how they are designed. Lots of criteria exist for cryptographic hashes motivated by various security requirements, but on the non-cryptographic side there is a certain amount of folklore that, despite the long history of hash functions, has not been fully explored. While targeting a uniform distribution makes a lot of sense for real-world datasets, it can be a challenge when confronted by a dataset with particular patterns.
Unwanted Surprises:
When that joke of an API is on you
There is the higher-order question of whether loosely typed languages with coercion are really a good idea in the first place. If you don't know what you're operating on, or what the expected output range might be, then maybe you ought not to be operating on that data in the first place. But now these languages have gotten into the wild and we'll never be able to hunt them down and kill them soon enough for my liking, or for the greater good.
Retrofitting: Principles and Practice
Retrofitting radically new functionality onto production software tests every skill of the programmers craft. A practical case study illuminates principles for bolting new tricks onto old dogs.
String Matching at Scale:
A call for interdisciplinary collaboration and better-directed resources
String matching can't be that difficult. But what are we matching on? What is the intrinsic identity of a software component? Does it change when developers copy and paste the source code instead of fetching it from a package manager? Is every package-manager request fetching the same artifact from the same upstream repository mirror? Can we trust that the source code published along with the artifact is indeed what's built into the release executable? Is the tool chain kosher?
A Clean Approach to Process Optimization:
What I learned from my dishwasher about automating processes
My soap-loading technique isn't revolutionary, but it does demonstrate a point about process design: You can eliminate delays in starting a process by front-loading tasks whenever possible. Front-loading changes when you do tasks but not their order. The process still involves a loop: load dishes, add soap, press start button, empty dishes repeat. You've only changed your mental model of where the loop starts.
Can't We Have Nice Things?
We build apparatus in order to show some effect we're trying to discover or measure. A good example is Faraday's motor experiment, which showed the interaction between electricity and magnetism. The apparatus has several components, but the main feature is that it makes visible an invisible force: electromagnetism. Faraday clearly had a hypothesis about the interaction between electricity and magnetism, and all science starts from a hypothesis. The next step was to show, through experiment, an effect that proved or disproved the hypothesis. This is how empiricists operate.
Can't We Have Nice Things?
We build apparatus in order to show some effect we're trying to discover or measure. A good example is Faraday's motor experiment, which showed the interaction between electricity and magnetism. The apparatus has several components, but the main feature is that it makes visible an invisible force: electromagnetism. Faraday clearly had a hypothesis about the interaction between electricity and magnetism, and all science starts from a hypothesis. The next step was to show, through experiment, an effect that proved or disproved the hypothesis. This is how empiricists operate.