Information about the acmqueue app and browser-based digital edition.

Subscribers and ACM Professional members login here

Features

Debugging Incidents in Google's Distributed Systems
There are lessons to be learned in the way Google engineers debug production issues in their complex systems. Their tools, strategies, and tasks can be applied to smaller systems.
- Charisma Chan and Beth Cooper

Is Persistent Memory Persistent?
Job number one of a computer system is protecting its data. But stuff happens. Even when every precaution has been taken, a power failure can destroy data. A simple inexpensive testbed can help prepare for such a catastrophe.
- Terence Kelly

Dark Patterns: Past, Present, and Future
Deceptive sales practices are as old as time, but online services have found new ways to manipulate their customers through user interfaces. The industry needs to shine a light on these so-called dark patterns.
- Arvind Narayanan, Arunesh Mathur, Marshini Chetty, and Mihir Kshirsagar

Columns / Departments

KODE VICIOUS
Broken Hearts and Coffee Mugs
No matter how you approach it, a security review is difficult. KV describes the process, step by step, so you know how to prepare for the ordeal.
- George Neville-Neil

COMMIT TO MEMORY
Power to the People
Reducing the carbon footprints of data centers is a lofty but necessary goal. The large operators are in varying stages of achieving the goal. Each has a slightly different approach.
- Jessie Frazelle

Subscribers and ACM Professional members login here

Features

Demystifying Stablecoins
Volatility has marked the emergence of cryptocurrencies. In response, there has been a flood of proposals for new designs that purport to provide a stable exchange rate. These so-called stablecoins may face numerous regulatory hurdles, but they are probably here to stay.
- Jeremy Clark, Didem Demirag, and Seyedehmahsa Moosavi, Concordia University

Case Study

To Catch a Failure: The Record-and-Replay Approach to Debugging
Mozilla developed the record-and-replay debugging tool, called rr, for internal use, but unexpectedly, it came to be widely used outside of Mozilla. Its developers discuss the challenges of debugging and the reason for the success of rr.
- A discussion with Robert O'Callahan, Kyle Huey, Devon O'Dell, and Terry Coatta

Columns / Departments

COMMIT TO MEMORY
Chipping Away at Moore's Law
The shrinkage of transistors predicted by Moore's law has slowed down in recent years. The technology exists for chips as small as 7nm, but the high cost of manufacturing them has prevented their widespread use.
- Jessie Frazelle

KODE VICIOUS
Kode Vicious Plays in Traffic
The use of software to control moving vehicles raises plenty of safety issues. But there are design principles to follow that may help ameliorate the risk.
- George Neville-Neil

ESCAPING THE SINGULARITY...
The Best Place to Build a Subway
Developing software can be compared with building a city's infrastructure. While it may be easier to start with a clean slate, the reality is that you must work around existing structures. The same holds true for computer systems.
- Pat Helland

EVERYTHING SYSADMIN
Communicate Using the Numbers 1, 2, 3, and More
The length of a list has a bearing on how listeners perceive the content. Knowing this will make you a better communicator.
- Thomas A. Limoncelli

THE MORNING PAPER
How do Committee's Invent? And Ironies of Automation
Two papers that have withstood the test of time hold true today. The first revisits Conway's law and the effect of organizational forces on design and development. The second looks at the idea that the more we automate, the more we depend on human skill.
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

Revealing the Critical Role of Human Performance in Software
Knowing how people detect problems and work together to resolve them is essential for successful Internet-facing systems. The articles in this issue examine the role of human performance in achieving more resilient systems.
- David D. Woods and John Allspaw

Above the Line, Below the Line
Imagine a line running horizontally across an Internet-facing system. Above the line are the forces that shape, direct, and restore the technical artifacts that lie below that line. Understanding what goes on underneath is key to the system's resilience.
- Richard I. Cook, M.D.

Cognitive Work of Hypothesis Exploration during Anomaly Response
A primary function above the line of representation is coordination across multiple roles during service outages, particularly how people explore various hypotheses when responding to an incident.
- Marisa R. Grayson

Managing the Hidden Costs of Coordination
A service outage usually initiates an all-hands-on-deck approach, but smooth coordination of these multiple perspectives is important. Without it, cognitive costs can take a toll.
- Laura M.D. Maguire

Beyond the "Fix-it" Treadmill
Something goes wrong with your software. Chaos ensues until it is fixed. Then comes the postmortem focusing on what went wrong this time. There could be a more holistic approach that will prevent failures down the line.
- J. Paul Reed

Columns / Departments

COMMIT TO MEMORY
Securing the Boot Process
Like a relay race where one team member hands a baton to another team member, the boot sequence of a computer works better when all the parts know and trust each other. Standards and system transparency help achieve that trust.
- Jessie Frazelle

KODE VICIOUS
Master of Tickets
In the course of a day, Alice writes 1,000 lines of buggy code and Carol writes 100 lines of well-crafted code. Which should be rewarded? Quality or quantity? KV's answer won't surprise anyone.
- George Neville-Neil

THE MORNING PAPER
The Way We Think About Data
Consider these ideas when it comes to data: First, interpretable models may offer benefits not found in a black-box model. Second, local-first software may combine the advantages of cloud apps with native, installed apps.
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

Blockchain Technology: What is it Good for?
Many industries want to know how blockchain technology can be applied to their particular universe. Here's a way to judge blockchain's potential across many use cases.
- Scott Ruoti, Ben Kaiser, Arkady Yerukhimovich, Jeremy Clark, and Robert Cunningham

The Reliability of Enterprise Applications
Engineering for enterprise reliability can mean crossing a complex landscape of third-party software that often compromises the very goal it seeks to achieve. With the right set of principles and methodologies, enterprises can avoid the pitfalls and maintain reliability.
- Sanjay Sha

Optimizations in C++ Compilers
In their constant search for perfect algorithms, programmers often overlook the magic of compilers. But after looking at the many types of optimizations your compiler can do, you may just learn to love assembly output.
- Matt Godbolt

Columns / Departments

KODE VICIOUS
Numbers Are for Computers, Strings Are for Humans
Computers like to work with numbers, and they are good at it. Trying to communicate a concept such as time as a string can only cause problems. Let your computer do the math.
- George Neville-Neil

ESCAPING THE SINGULARITY...
It's Not Your Grandmother's Database Anymore
Space Time Discontinuum
You can do so much more with big distributed systems, but they present a dilemma when your calculations are coming from many different sources: Wait for the perfect answer? Or go with a faster but incomplete answer?
- Pat Helland

EVERYTHING SYSADMIN
API Practices If You Hate Your Customers
It's easy to drive customers away by making your external APIs costly and difficult to use. But it's not all that hard to keep customers by following a few best practices in building your APIs.
- Thomas A. Limoncelli

COMMIT TO MEMORY
Opening Up the Baseboard Management Controller
The open sourcing of the data center has led to a number of innovative projects involving the BMC, which monitors and manages the physical state of the hardware.
- Jessie Frazelle

THE MORNING PAPER
Back Under a SQL Umbrella
For data processing at YouTube, Google uses Procella, which the first paper presented here describes as unifying a large-scale data platform with a SQL query engine to address all of the data needs of an organization. Save one: machine learning; a second paper advocates using an RDMS as an ML platform. Do all roads lead to SQL?
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

The Effects of Mixing Machine Learning and Human Judgment
What's the best approach to effective decision making? Solo human, solo computer, or both working in tandem? Two experiments show that collaboration between humans and machines does not necessarily lead to better outcomes.
- Michelle Vaccaro and Jim Waldo

Hack for Hire
An emerging black market offers hack-for-hire services to break into targeted email accounts. A covert investigation reveals this is a niche market and that security keys can be effective in protecting against these attacks—at least for now.
- Ariana Mirian

Columns / Departments

KODE VICIOUS
Koding Academies
So-called coding academies are a quick fix to train front-end plumbers, but they fall short in preparing students for careers in software engineering or computer science.
- George Neville-Neil

ESCAPING THE SINGULARITY...
Write Amplification Versus Read Perspiration
To read, or to write? In computing, writing may create more work—reorganizing, merging, reindexing. Otherwise, you must search or do other work to support future reads. There are tradeoffs to consider between the two.
- Pat Helland

THE MORNING PAPER
Putting Machine Learning into Production Systems
Data validation and software engineering are two different approaches to incorporating machine learning into development projects. Google takes the former approach, while Microsoft chooses the latter. Here are case studies of both.
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

Open-source Firmware
By making the actions of firmware more visible, computing becomes more secure. The key to making this happen is to support platforms that are moving away from proprietary firmware and adopting open-source solutions.
- Jessie Frazelle

The Velocity of Software Engineering
Is your software-engineering team more like a tectonic plate or an F-16? High-velocity teams get the best results. They are optimized for speed, open to experimentation, agile, and subject to regular course correcting.
- Tom Killalea

Columns / Departments

CASE STUDY
DAML: The Contract Language of Distributed Ledgers
DAML (Digital Asset Modeling Language) puts blockchain technology to work for the financial services industry, particularly in distributed ledgers. It was an eventful journey to get there, with many lessons learned along the way.
A discussion with Shaul Kfir and Camille Fournier

KODE VICIOUS
What is a CSO Good for?
A chief security officer, not to be confused with a security engineer, needs to do more than buy a third-party security product. There has to be an understanding of the potential threats and a clear plan to deal with them.
- George Neville-Neil

THE SOFT SIDE
The Evolution of Management
With each step up the ladder—from individual contributor to manager to organization leader—you will face transitions that require a shift in mindset and a focus on new skills.
- Kate Matsudaira

EVERYTHING SYSADMIN
Demo Data as Code
Generating demo data is common practice in many organizations. While it may seem like a one-time task to be over and done with, automating the process will save pain down the road.
- Thomas A. Limoncelli

THE MORNING PAPER
Time Protection in Operating Systems and Speaker Legitimacy Detection
Two diverse security topics for this edition of The Morning Paper: Timing-based attacks are going to require attention from future operating systems that need to offer time protection. Second, voice-imitation software is becoming so good that it will require its own brand of protection from social-engineering attacks.
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

Surviving Software Dependencies
The shift to software reuse has happened so quickly that the risks are not yet fully understood.
- Russ Cox

Industry-scale Knowledge Graphs: Lessons and Challenges
Google, Microsoft, IBM Watson, Facebook, and eBay must each provide structured data that makes their products more intelligent and "magical." Each company serves as an example of how best to build and deploy the knowledge graphs that make this possible.
- Natasha Noy, Yuqing Gao, Anshu Jain, Anant Narayanan, Alan Patterson, and Jamie Taylor

Access Control and Health Care Records: Who Owns the Data?
Data management in health care is burdened by legacy systems and fragmented to the point of no return. A Toronto-based startup called HealthChain set out to bring some sense of control to the field. Here's what the company has learned.
- A discussion with David Evans, Richard McDonald, and Terry Coatta

Columns / Departments

KODE VICIOUS
MUST and MUST NOT
KV has some tips for putting your processes into understandable prose. He draws on wisdom from a more than 20-year-old document known as RFC 2119.
- George Neville-Neil

THE SOFT SIDE OF SOFTWARE
Overly Attached
Becoming emotionally attached to your work can lead to compromised decision-making. A good leader must be able to see when this is happening and insert an objective voice.
- Kate Matsudaira

ESCAPING THE SINGULARITY
Extract, Shoehorn, and Load
An increasing part of the computer landscape is moving data from system to system, but it doesn't always fit. Sometimes it has to be shoehorned into a shape that will allow it to be understood.
- Pat Helland

RESEARCH FOR PRACTICE
The DevOps Phenomenon
DevOps holds a lot of promise for companies hoping to capture market share and deliver value faster, but switching from traditional methods has its challenges. A number of papers have addressed these early implementation hurdles.
- Anna Wiedemann, Nicole Forsgren, Manuel Wiesche, Heiko Gewald, and Helmut Krcmar

THE MORNING PAPER
GAN Dissection and Datacenter RPCs
This edition of "The Morning Paper" tackles two diverse topics. The first paper dissects what happens inside a GAN (generative adversarial network) as it is generating an image. The second argues that in certain cases, RPCs (remote procedure calls) can still do the job in datacenters.
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

Net Neutrality: Unexpected Solution to Blockchain Scaling
Blockchains do not scale well, but cloud-delivery networks may lead to a solution to this problem. The trick is in establishing a provably neutral network design.
- Aleksandar Kuzmanovic

Garbage Collection as a Joint Venture
Google Chrome uses an approach called cross-component tracing to solve the problem of memory management across component boundaries. The result is a safe and efficient reclamation of memory.
- Ulan Degenbaev, Michael Lippautz, and Hannes Payer

Online Event Processing
Online event processing is finding a foothold in data management, particularly in cases where transaction processing has not been up to the task. OLEP allows an application to guarantee strong consistency properties across heterogeneous data systems.
- Martin Kleppmann, Alastair R. Beresford, and Boerge Svingen

Columns / Departments

KODE VICIOUS
The Worst Idea of All Time
KV has been ranting and raving now for 100 columns. From ignoble beginnings, KV, nevertheless, has persisted—and will continue to wrangle with anger management for many columns to come.
- George Neville-Neil

THE SOFT SIDE OF SOFTWARE
How to Create a Great Team Culture (and Why It Matters)
Good teams don't just happen. They spring from a leader who takes the time to build a culture where the team is greater than the sum of its individuals. When that happens, it's electric.
- Kate Matsudaira

EVERYTHING SYSADMIN
Tom's Top Ten Things Executives Should Know about Software
If you're in a company whose executives doesn't understand how software is changing business, beware. If you are one of these execs, it's time to learn about this new world. If you work for these execs, start teaching them.
- Tom Limoncelli

RESEARCH FOR PRACTICE
Troubling Trends in Machine-learning Scholarship
Many research papers in the field of machine learning suffer from flaws that could mislead the public and stymie future research. Becoming aware of these problems could help avoid them in future endeavors.
- Zachary C. Lipton and Jacob Steinhardt

THE MORNING PAPER
SageDB and NetAccel
A new addition to acmqueue examines two papers from the 2019 Conference on Innovative Data Systems Research. Each addresses a bold new direction for data-systems research—one looks at the role of learned models and the other examines network-accelerated query processing.
- Adrian Colyer

Subscribers and ACM Professional members login here

Features

A Hitchhiker's Guide to the Blockchain Universe
The blockchain is purported to be the answer to just about everything, yet not many people can explain exactly what it is. Here is a look at the basics: the individual components, how they fit together, and the problems with blockchain technology.
- James Waldo

Achieving Digital Permanence
Once upon a time, giving permanent life to data meant storing it on a clay tablet, paper scroll or punch card. Then came the Information Age. Now every piece of data is stored on digital media. The challenge is making it last and keeping it accurate.
- Raymond Blum, with Betsy Bower

Identity by Any Other Name
Notions of identity underlie the basic mechanisms of distributed systems, including interchangeability, idempotence, and immutability. Identifiers hold these intertwined systems all together.
- Pat Helland

Metrics That Matter
The Google SRE team has identified some service metrics that are critical to product reliability. Choosing the right metrics is the difference between delighting users or driving them away.
- Ben Treynor, Shylaja Nukala, and Vivek Rau

Columns / Departments

KODE VICIOUS
Know Your Algorithms
Buying newer, faster hardware is not always the answer to your computing problems. It's very often the underlying software that holds the clues to what you are seeking.
- George Neville-Neil

THE SOFT SIDE
Design Patterns for Managing Up
You are always going to encounter situations where you are not at your best—you don't know an answer, you disagree with a decision, you make a mistake. You can recognize these patterns and be prepared to deal with them.
- Kate Matsudaira

RESEARCH FOR PRATICE
Edge Computing
Some applications are better served by moving out of the cloud and into everyday lives. Here is a review of some of the research guiding this trend toward edge computing.
- Nitesh Mor

Subscribers and ACM Professional members login here

Features

Tear Down the Method Prisons! Set Free the Practices!
The professional practices that software developers need in order to work in the most efficient and collaborative way possible are all too often imprisoned within proprietary method jails. Essence may hold the key to liberation.
- Ivar Jacobson and Roly Stimson

Understanding Database Reconstruction Attacks on Public Data
The U.S. Census Bureau is especially vulnerable to database reconstruction attacks and is evaluating ways to lessen the threat. This has implications for the upcoming decennial census and protecting the privacy of millions of U.S. citizens.
- Simson Garfinkel, John Abowd, and Christian Martindale

Benchmarking "Hello, World!"
Tools for observing today's large-scale systems are often lacking. Exploring a very simple environment can help reveal the shortcomings of the tools you are using and direct tool designers and users to fill in the gaps.
- Richard L. Sites

CASE STUDY

Codeflow: Improving the Code Review Process at Microsoft
Large numbers of well-compensated people spend a lot of time on code review. When you're talking about a shop the size of Microsoft, the costs are impressive. That's why a team at the company set out to study the process, making it better for everyone.
A discussion with Jacek Czerwonka, Michaela Greiler, Christian Bird, Lucas Panjer, and Terry Coatta

Columns / Departments

KODE VICIOUS
Writing a Test Plan
How do you create a useful document without turning it into a massive time sink and distraction? Draw on those old lab reports from high school science class.
- George Neville-Neil

EVERYTHING SYSADMIN
Making DevOps Work with SQL
Don't discount DevOps just because you use a SQL database. It may be a difficult transition, but once done, life will be much easier.
- Thomas A. Limoncelli

THE SOFT SIDE
The Importance of a Great Finish
Projects tend to start with a bang and end with a whimper. Don't let this happen. You have to keep your momentum up and finish strong, every time. This is what will get you noticed.
- Kate Matsudaira

RESEARCH FOR PRATICE
Security for the Modern Age
Securely running processes that require the entire syscall interface creates a need for new methods. Solving this problem has led to research into different ways to automate security and defend against attacks in the modern age.
- Jessie Frazelle

Features

How to Live in a Post-Meltdown and -Spectre World
- Rich Bennett, Craig Callahan, Stacy Jones, Matt Levine, Merrill Miller, and Andy Ozment
Major vulnerabilities and their variants create a risk landscape that demands immediate defenses and some tradeoffs. A strong vulnerability management foundation is essential to fight these current and future battles.

Using Remote Cache Service for Bazel
- Alpha Lam
Bazel is an open-source system that provides the power to run build tasks remotely and massively parallel. Its new remote cache feature can help software developers reduce build time by almost an order of magnitude.

Tracking and Controlling Microservice Dependencies
- Silvia Esparrachiari, Tanya Reilly, and Ashleigh Rentz
Dependency cycles are a critical detail of system design that can easily go unnoticed until it's too late. Managing these dependencies is essential to system reliability, especially with the growth of today's massive interdependent software systems.

Why SRE Documents Matter
- Shylaja Nukala and Vivek Rau
Documentation is key to a successful SRE team. Without it, SREs will waste valuable time trying to solve problems through trial and error. With it, teams can scale up and take a principled approach to managing new and unfamiliar services.

Columns / Departments

KODE VICIOUS
A Chance Gardener
- George Neville-Neil
Open source is like a garden with many different species of plants, some of which are beneficial and others of which are toxic. How does a company weed its way through this garden?

THE SOFT SIDE OF SOFTWARE
How to Get Things Done When You Don't Feel Like It
- Kate Matsudaira
Face it, there are days when you're just not feeling it. But there are strategies for dealing with those bad days and turning out good work in spite of it all.

Features

The Mythos of Machine-learning Model Interpretability
- Zachary C. Lipton
The rapid progress of machine learning is making automated decision-making possible, but an important yet misunderstood component is the ability to interpret the models used in the process. What exactly is interpretability and why is it important?

Corp to Cloud: Google's Virtual Desktops
- Matt Fata, Philippe-Joseph Arida, Patrick Hahn, and Betsy Beyer
GDesktop, the internal virtual desktop used at Google, was, until recently, hosted on commercially available hardware on the company's corporate network. Today it runs on the Google Compute Platform. Here's the step-by-step migration to the cloud.

Mind Your State for Your State of Mind
- Pat Helland
Storage and applications have struggled side by side through changing requirements and environments. These changes have implications for durable state and session state, which must be kept in mind when developing applications.

Columns / Departments

KODE VICIOUS
The Obscene Coupling Known as Spaghetti Code
- George Neville-Neil
Needlessly chaining functions, silently mixing and transparently passing data through multiple layers of interfaces, and, most painfully, burying important error output-will it ever stop?

THE SOFT SIDE OF SOFTWARE
The Secret Formula for Choosing the Right Next Role
- Kate Matsudaira
Fancy titles and prestige projects will get you only so far. Better to focus on long-term goals and compatible colleagues when deciding where to go next in your career.

EVERYTHING SYSADMIN
GitOps: A Path to More Self-service IT
- Thomas A. Limoncelli
GitOps is a workflow that empowers users to do their own IT through pull requests. For systems where it is appropriate, GitOps reduces wait times, improves system safety, and increases return on investment.

RESEARCH FOR PRACTICE
Knowledge Base Construction in the Machine-learning Era
- Alex Ratner and Chris Ré
KBC, once reserved for well-funded groups in academia, industry or government, is becoming more accessible thanks to easy-to-use deep-learning models. Three papers summarized here address critical design choices that must be made for these approaches to work.

Features

C is Not a Low-level Language
- David Chisnall
Given the prevalence of parallel hardware today, from multicore CPUs to manycore GPUs, implementations of C have had to become increasingly complex to maintain the illusion that C maps easily to the underlying hardware. This has led to vulnerabilities.

Algorithms Behind Modern Storage Systems
- Alex Petrov
As database systems grow, scaling storage systems becomes more challenging. It's important to know the tradeoffs of the two design approaches used for storage systems in most database systems: read-optimized B-trees and write-optimized LSM-trees.

General Equation and Empirical Verification of Workload Frequency Scaling
- Noor Mubeen
Measuring the productive performance, or scale factor, of workloads requires some workload-scaling equations, presented here. The validation of these equations turns out to be tricky, but a histogram ridge trace can help establish significantly accurate estimates.

Columns / Departments

KODE VICIOUS
Every Silver Lining Has a Cloud
- George Neville-Neil
Moving an application to the cloud may sound good to budget-conscious managers, but suddenly you are fighting for resources with many other cloud tenants. Performance may suffer.

ESCAPING THE SINGULARITY
Consistently Eventual
- Pat Helland
The idea of eventual consistency is that at some point work on a data item will stop and its value will be known. For many items, the work never fully settles on a value. They are more accurately described as consistently eventual.

RESEARCH FOR PRACTICE
FPGAs in Data Centers
- Gustavo Alonso
Field-programmable gate arrays have started to appear in commercial data centers because they are resource- and energy-efficient. Four papers presented here show how FPGA designs are quickly evolving and detail some of the ongoing debates around FPGAs.

Features

Canary Analysis Service
- Štěpàn Davidovič
Google's centralized CAS takes the canary-in- the-coalmine approach to determining whether a partial and time-limited change in service is safe and effective. In the end, CAS quickens development, improves production safety, and helps prevent outages.

Thou Shalt Not Depend on Me
- Tobias Lauinger, Abdelberi Chaabane, and Christo B. Wilson
Websites that use third-party components such as JavaScript libraries are vulnerable to any security issues that these libraries may contain. The first step is understanding the scope of the problem, then determining what can be done to improve the situation.

Designing cluster schedulers for Internet-scale services
- Diptanu Gon Choudhury and Timothy Perrett
Cluster schedulers for distributed systems have become ubiquitous but implementing them is tricky. They must be designed for failure in order to assure system stability and reliability.

Columns / Departments

EVERYTHING SYSADMIN
Manual Work is a Bug
- Thomas A. Limoncelli
A tale of two sysadmins—one who took time to write a lot of code to automate tasks; one who did not. One was successful; one not so much.

KODE VICIOUS
Watchdogs vs. Snowflakes
- George Neville-Neil
How to deal with a distributed system that randomly jams up? There may be no easy answers—only wild-ass guesses.

THE SOFT SIDE OF SOFTWARE
How to Come Up with Great Ideas
- Kate Matsudaira
Train yourself to have the mindset of an entrepreneur, and you will have no shortage of ideas. Here are a few techniques to start thinking in those terms.

RESEARCH FOR PRACTICE
Prediction-serving Systems
- Dan Crankshaw and Joseph Gonzalez
Machine learning involves training a model that can then be used to make predictions based on new inputs. Most research in this area has focused on designing and training these models. Here we look at papers that address the often-overlooked area of managing these models once they are trained.

Features

Continuous Delivery Sounds Great, But Will It Work Here?
- Jez Humble
The ability to get new features, changes, and bug fixes into the hands of users safely and quickly is a competitive advantage. Continuous delivery achieves that, but many people erroneously assume the obstacles to its implementation are too great.

Containers Will Not Fix Your Broken Culture
(and Other Hard Truths)
- Bridget Kromhout
The right DevOps practices can solve a lot of problems, but they go only so far without the right work culture. For these tools to do their job, they must be used in an environment where people can work together in building the future.

Monitoring in a DevOps World
- Theo Schlossnagle
With the emergence of DevOps, the ground has shifted under traditional monitoring paradigms. The new world is fluid and fast, and to keep up, monitoring must be thought about in a completely different way.

DevOps Metrics:
Your Biggest Mistake Might Be Collecting the Wrong Data
- Nicole Forsgren, Ph.D., and Mik Kersten, Ph.D.
Measurement is paramount to the success of DevOps transformations, but determining exactly what to measure can be daunting. Here is a guide to the types of data an organization should collect to ensure effective software development and delivery.

Columns / Departments

KODE VICIOUS
Popping Kernels
- George Neville-Neil
Whether to program in the operating-system kernel or in user space depends on the circumstances. What really matters is following programming best practices, no matter the venue.

THE SOFT SIDE OF SOFTWARE
How Is Your Week Going So Far?
- Kate Matsudaira
Easy to forget, but "Good job!" goes a long way toward inspiring motivation and productivity. To be most effective, praise must have three elements: it must be frequent, specific, and strategic.

RESEARCH FOR PRACTICE
Toward a Network of Connected Things
- Deepak Vasisht
The IoT (Internet of things) is quickly moving from theory to reality. This installment of RfP presents research papers that may help accelerate and capitalize on the trend.

Features

Bitcoin's Underlying Incentives
- Yonatan Sompolinsky and Aviv Zohar

Built into the bitcoin protocol are economic incentives that govern the cryptocurrency's capabilities, security guarantees, and future development. Stronger incentives are crucial to the survival of all cryptocurrencies.

Titus: Introducing Containers to the Netflix Cloud
- Andrew Leung, Andrew Spyker, and Tim Bozarth

While Netflix already went all in on cloud migration, it is now investing in container technology. To do so, it developed Titus, a container-management system, which facilitated container adoption in its existing infrastructure.

Abstracting the Geniuses Away from Failure Testing
- Peter Alvaro and Severine Tymon

Failure testing has long relied on super-users to supply the appropriate faults to inject to expose design flaws, but this approach has limitations. What is needed is a way to automate the super-user's process.

Columns / Departments

KODE VICIOUS
Reducing the Attack Surface
- George Neville-Neil

Be careful what code you supply your development teams—it may be dangerous.

THE SOFT SIDE OF SOFTWARE
Views from the Top
- Kate Matsudaira

The perspective can be quite different, depending on which side of the org chart you fall.

EVERYTHING SYSADMIN
Operational Excellence in April Fools' Pranks
- Thomas A. Limoncelli

The best AFPs are topical and absurdist. They don't get in the way of customers or business. And they don't just happen—they require careful planning and execution.

RESEARCH FOR PRACTICE
Cluster Scheduling for Data Centers
- Malte Schwarzkopf

This curated selection of research papers will help readers understand how to use cluster managers to schedule workloads efficiently and how to scale these managers and their schedulers.

Features

Bitcoin's Academic Pedigree
- Arvind Narayanan and Jeremy Clark

Cryptocurrency is not a 21st century concept but originated in the academic literature of the 1980s and 1990s. Building on this research, digital cash will continue to find commercial success, aided by collaboration among academia, outside researchers, and practitioners.

Cache Me If You Can
- Jacob Loveless

Building a better, decentralized Internet is the goal, but how to get there? The tools and technology are available. The first step is defining the problem as completely as possible.

Network Applications Are Interactive
- Antony Alappatt

The network era brings opportunity for new applications, but their development requires a move away from the old sequential model centered on algorithms to new models with interactions at their core.

Columns / Departments

Kode Vicious
Cold, Hard Cache
- George Neville-Neil

KV reveals secrets to implementing and maintaining a well-organized cache.

The Soft Side of Software
Breadth and Depth
- Kate Matsudaira

When it comes to growing your career, is it better to go wide and learn a lot of different things, or go deep and learn a few things really well?

Escaping the Singularity
XML and JSON Are Like Cardboard
- Pat Helland

Like the sturdy packaging around your new purchases, XML and JSON offer savings and efficiencies in protecting your data that more than make up for the overhead.

Research for Practice
Private Online Communication; Highlights in Systems Verification

Albert Kwon reviews several papers that take on the increasingly important topic of ensuring privacy in our daily online communication. Then James R. Wilcox looks at research into systems verification techniques, aiming to eliminate entire classes of bugs.

Features

Is There a Single Method for the Internet of Things?
- Ivar Jacobson, Ian Spence, Pan-Wei Ng

The IoT will demand new and better methods of developing the vast amount of new software that will be required. The OMG standard Essence can help keep it all from becoming unwieldy.

Metaphors We Compute By
- Alvaro Videla

Programmers must be able to tell a story with their code. Like writers, they must know their metaphors. Thus we get concepts such as queues, nodes, traffic, congestion—all shaping the way people understand what's happening in a program.

Hootsuite: In Pursuit of Reactive Systems
- A discussion with Edward Steel, Yanik Berube, Jonas Bonér, Ken Britton, and Terry Coatta

In a case study of this most widely used SaaS platform for managing social media, participants discuss Hootsuite's transition from a LAMP platform to a microservices architecture, aided by Scala and Lightbend.

Columns / Departments

Kode Vicious
IoT: The Internet of Terror
- George Neville-Neil

Don't ever, ever underestimate the need for encryption in IoT software, no matter the cost in battery life or anything else.

The Soft Side of Software
10 Ways to be a Better Interviewer
- Kate Matsudaira

You have an hour in a room with two chairs and a whiteboard to determine if a job candidate is the right fit. Preparation is key to choosing the best person for the job.

Everything Sysadmin
Four Ways to Make CS and IT Curricula More Immersive
- Thomas A. Limoncelli

Computer science and IT curricula in today's universities should be immersive, more reliably reflecting the real world. Better to start off experiencing a well-run system than figuring out how to fix a badly run one.

Research for Practice
Vigorous Public Debates in Academic Computer Science
- John Regehr

As with any academic discipline, computer science has inspired energetic public debates over what works and what doesn't—from GoTo statements to security protocols.

Features

The IDAR Graph
- Mark A. Overton

UML comes up short in conveying the hierarchy of a design, making it difficult for humans to understand exactly how it's put together. The IDAR graph is a clearer alternative for representing object-oriented designs, resulting in cleaner software with fewer bugs.

The Calculus of Service Availability
- Ben Treynor, Mike Dahlin, Vivek Rau, Betsy Beyer

Most software services and systems should aim for almost-perfect reliability. Google maintains an aggressive SLO of 99.99 percent availability (the "four nines") on its products. Here's how it happens.

Data Sketching
- Graham Cormode

The vast scale of information today means making some compromises—for example, maintaining a summary, or sketch, of data rather than every last bit. Here we look at four innovative algorithmic ideas behind data sketching.

Columns / Departments

Kode Vicious
The Observer Effect
- George Neville-Neil

Frequently polling a system for information can easily overtax it. For it to be useful, you need to find just the right frequency, depending on the task at hand.

The Soft Side of Software
Conversations with Technology Leaders: Erik Meijer
- Kate Matsudaira

Renowned for his innovative work in software development, Meijer has many engineering and leadership lessons to share. Great leaders in this field share a desire for constant learning and a passion for technology.

Escaping the Singularity
Side Effects, Front and Center!
- Pat Helland

The work that software developers do often results in side effects. These unintended consequences may be just too much information—but they may also be relevant.

Research for Practice
- Technology for Underserved Communities
- Personal Fabrication
- Peter Bailis

Finding ways to support the technology needs of underserved communities is a worthy goal. Tawanna Dillahunt looks at three efforts to do so. Then Stefanie Mueller and Patrick Baudisch examine the future of 3D printers, especially their potential for innovation among nontechnical users.

Features

Making Money from Math
- Erik Meijer

Machine-learned models differ from human coding in that they acknowledge uncertainty in their code rather than assuming all computations are precise. Probabilistic programming is a way to bridge the gap between the two.

MongoDB's JavaScript Fuzzer
- Robert Guo

A home-grown JavaScript fuzzer has become MongoDB's most prolific bug-finding tool. It will find the inevitable edge case that defies other code-testing methods.

Too Big NOT to Fail
- Pat Helland, Simon Weaver, and Ed Harris

In web-scale computing, running hundreds of thousands of servers, everything must be simple, predictable, and designed to expect and embrace failure—a fundamentally different approach from smaller environments.

The Debugging Mindset
- Devon H. O'Dell

Applying the psychology of problem solving to the science of debugging can only help programmers become more efficient and effective at their jobs.

Columns / Departments

Kode Vicious
Forced Exception Handling
- George Neville-Neil

Why do nonfatal errors often lead to catastrophic failures? The answer most likely has to do with human nature.

The Soft Side of Software
Does Anybody Listen to You?
- Kate Matsudaira

Knowing the right people, the right time, and the right way to introduce your ideas at work will give them life.

Research for Practice
- Tracing and Debugging Distributed Systems
- Programming by Examples
- Peter Bailis

To bring us up to speed on the state of the art in debugging distributed systems, Peter Alvaro looks at techniques for dealing with the challenges of large-scale tracing systems. Then Sumit Gulwani looks at recent advances in PBE—programming - examples. Why write a program when you can synthesize one from existing examples?

Features

Time, but Faster
- Theo Schlossnagle

Time may be an illusion, but in computing it is a necessary one. As computers get faster, system performance can improve, but only if it can be measured against some concept of time.

Heterogeneous Computing: Here to Stay
- Mohamed Zahran

Heterogeneous computing is quickly becoming the norm. Making the best use of it will require revisiting practices and methods on both the hardware and software sides.

Uninitialized Reads
- Robert C. Seacord

A look at object initialization, indeterminate values, and trap representations demonstrates the need for addressing uninitialized reads in the proposed revision to the current C standard.

Pervasive, Dynamic Authentication of Physical Items
- Meng-Day (Mandel) Yu and Srinivas Devadas

Silicon PUFs (physical unclonable functions) offer advantages in the authentication and security of physical items over other more common approaches such as bar codes, QR codes, holograms, and RFID tags.

Columns / Departments

Research for Practice:
- Cryptocurrencies, Blockchains, and Smart Contracts
- Hardware for Deep Learning
In this issue RfP turns to Arvind Narayanan and Andrew Miller to present research into both the promise and dangers of using and applying cryptocurrencies. Song Han looks at ways of deploying deep neural networks using specialized hardware.

Everything SysAdmin:
Are You Load Balancing Wrong?
The use of load balancers to increase capacity and improve resiliency is widespread in today's web-centric, service-centric environments. Most people, however, are doing it all wrong.

Kode Vicious:
The Chess Player who Couldn't Pass the Salt
Computer scientists can get their AI machines to play chess really well—but is this really intelligence, or is it just a way of applying statistics to very large data sets?

Features

FAUCET: Deploying SDN in the Enterprise
-Josh Bailey

With Faucet and OpenFlow 1.3 hardware, network operators can migrate toward SDN, taking advantage of DevOps practices to deploy features rapidly in the enterprise.

BBR: Congestion-Based Congestion Control
- Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Van Jacobson, and Soheil Yeganeh

TCP's loss-based congestion control is the culprit in slowing down data on the Internet. The solution could be BBR, Google's new approach to congestion control based on bottleneck bandwidth and round-trip propagation time.

Life Beyond Distributed Transactions
- Pat Helland

Programmers who simply want to solve business problems in today's world are distracted by problems of scale. The potential exists for using patterns to build successful scalable enterprise applications.

Industrial Scale Agile: from Craft to Engineering
- Ivar Jacobson, Ian Spence, and Ed Seidewitz

With software becoming more essential to the world's activities, it's time for software development to become more of an engineering discipline than a craft. Essence provides a language and kernel to help achieve this shift.

Columns / Departments

Research for Practice:
- Practical Information Flow for Web Security
- Distributed Transactions and Networks as Physical Sensors
RfP continues with Irene Zhang describing three papers that dispute the perception of distributed transactions being prohibitively expensive. Then Fadel Adib delves into the previously fantastical world of using computer networks for such tasks as seeing through walls.

The Soft Side of Software:
Resolving Conflict
It's not all about winning. Conflict can be productive if you negotiate and manage it well.

Kode Vicious:
The Unholy Trinity of Software Development
Will your project self-destruct if you combine tests, documentation, and code into one file?

Features

• Marius Eriksen
Functional at Scale

• Adam Morrison
Scaling Synchronization in Multi-Core Programs


• Case Study
React: Facebook's Functional Turn on Writing JavaScript

Columns / Departments

• Research for Practice:
- Practical Information Flow for Web Security
- The Red Future of Mobile Web Computing

• Escaping the Singularity:
The Power of Babble

• Everything Sysadmin:
10 Optimizations on Linear Search

• The Soft Side of Software:
Fresh Starts

• Kode Vicious:
Cloud Calipers

Features

• The Hidden Dividends of Microservices
• Idle-Time Garbage-Collection Scheduling
• Dynamics of Change: Why Reactivity Matters
• Cluster-Level Logging of Containers with Containers

Columns / Departments

• Research for Practice:
Distributed Consensus and Implications of NVM on Database Management Systems
• Escaping the Singularity:
  The Singular Success of SQL
• The Soft Side of Software:
  Bad Software Architecture is a People Problem
• Kode Vicious:
  Chilling the Messenger

Features

• Debugging Distributed Systems
• Flame Graphs
• Should I Upload or Ship My Big Data to the Cloud?

Columns / Departments

• Introducing RfP - Research for Practice
• Escaping the Singularity:
  Standing on Giant Distributed Shoulders
• The Soft Side of Software:
  Nine Things I Didn't Know I Would Learn Being an Engineer Manager
• Everything Sysadmin:
  The Small Batches Principle
• Kode Vicious:
  What Are You Trying to Pull?

Features

• Statistics for Engineers
• Why Logical Clocks are Easy
• Borg, Omega, and Kubernetes
• Use-Case 2.0

Columns / Departments

• The Bikeshed:
  More Encryption Means Less Privacy
• The Soft Side of Software:
  Delegation as Art
• Kode Vicious:
  GNL is Not Linux

Features

• Non-volatile Storage
• Time is an Illusion
• Schema.org: Evolution of Structured Data on the Web
• Immutability Changes Everything
• Accountability in Algorithmic Decision-Making
• The Verification of a Distributed System

Columns / Departments

• The Soft Side of Software:
  The Paradox of Autonomy and Recognition
• Everything Sysadmin:
  How Sysadmins Devalue Themselves
• Kode Vicious:
  Code Hoarding

Features

• How to De-identify Your Data
• Fail at Scale: Reliability in the Face of Rapid Change
• Optimizing NUMA Systems Applications with Carrefour
• Componentizing the Web
• It Probably Works

Case Study

• A Purpose-built Global Network: Google's Move to SDN

Columns / Departments

• Version Status:
  Still Finding the Right Questions
• The Soft Side of Software:
  Lean Software Development—Building and Shipping Two Versions
• Everything Sysadmin:
  Automation Should Be Like Iron Man, Not Ultron
• Kode Vicious:
  Pick a Peck of Pickled Patches and Zapped