Articles

20 Obstacles to Scalability

Watch out for these pitfalls that can prevent Web application scaling.

A Call to Arms

Long anticipated, the arrival of radically restructured database architectures is now finally at hand.

A Decade of OS Access-control Extensibility

Open source security foundations for mobile and embedded devices

A File System All Its Own

Flash memory has come a long way. Now it's time for software to catch up.

A Guided Tour through Data-center Networking

A good user experience depends on predictable performance within the data-center network.

A High-Performance Team

From design to production, performance should be part of the process.

A Hitchhiker's Guide to the Blockchain Universe

Blockchain remains a mystery, despite its growing acceptance.

A New Objective-C Runtime: from Research to Production

Backward compatibility always trumps new features.

A New Software Engineering

What happened to the promise of rigorous, disciplined, professional practices for software development?

A Passage to India

Most American IT employees take a dim view of offshore outsourcing. It's considered unpatriotic and it drains valuable intellectual capital and jobs from the United States to destinations such as India or China. Online discussion forums on sites such as isyourjobgoingoffshore.com are headlined with titles such as "How will you cope?" and "Is your career in danger?" A cover story in BusinessWeek magazine a couple of years ago summed up the angst most people suffer when faced with offshoring: "Is your job next?"

A Pioneer's Flash of Insight

Jim Gray's vision of flash-based storage anchors this issue's theme.

A Plea to Software Vendors from Sysadmins - 10 Do's and Don'ts

What can software vendors do to make the lives of sysadmins a little easier?

A Primer on Provenance

Better understanding of data requires tracking its history and context.

A Purpose-built Global Network: Google's Move to SDN

A discussion with Amin Vahdat, David Clark, and Jennifer Rexford

A Requirements Primer

Many software engineers and architects are exposed to compliance through the growing number of rules, regulations, and standards with which their employers must comply. Some of these requirements, such as HIPAA (Health Insurance Portabililty and Accountability Act), focus primarily on one industry, whereas others, such as SOX (Sarbanes-Oxley Act), span many industries. Some apply to only one country, while others cross national boundaries. To help navigate this often confusing world, Queue has assembled a short primer that provides background on four of the most important compliance challenges that organizations face today.

A Threat Analysis of RFID Passports

Do RFID passports make us vulnerable to identity theft?

A Time and a Place for Standards

History shows how abuses of the standards process have impeded progress. Over the next decade, we will encounter at least three major opportunities where success will hinge largely on our ability to define appropriate standards. That's because intelligently crafted standards that surface at just the right time can do much to nurture nascent industries and encourage product development simply by creating a trusted and reliable basis for interoperability. From where I stand, the three specific areas I see as particularly promising are: (1) all telecommunications and computing capabilities that work together to facilitate collaborative work; (2) hybrid computing/home entertainment products providing for the online distribution of audio and/or video content; and (3) wireless sensor and network platforms (the sort that some hope the 802.15.4 and ZigBee Alliance standards will ultimately enable).

A Tour through the Visualization Zoo

A survey of powerful visualization techniques, from the obvious to the obscure

A Tribute to Jim Gray

Computer science attracts many very smart people, but a few stand out above the others, somehow blessed with a kind of creativity that most of us are denied. Names such as Alan Turing, Edsger Dijkstra, and John Backus come to mind. Jim Gray is another.

A co-Relational Model of Data for Large Shared Data Banks

Contrary to popular belief, SQL and noSQL are really just two sides of the same coin.

AI Gets a Brain

New technology allows software to tap real human intelligence.

AI in Computer Games

If you've been following the game development scene, you've probably heard many remarks such as: "The main role of graphics in computer games will soon be over; artificial intelligence is the next big thing!" Although you should hardly buy into such statements, there is some truth in them. The quality of AI (artificial intelligence) is a high-ranking feature for game fans in making their purchase decisions and an area with incredible potential to increase players' immersion and fun.

API Design Matters

Why changing APIs might become a criminal offense. After more than 25 years as a software engineer, I still find myself underestimating the time it will take to complete a particular programming task. Sometimes, the resulting schedule slip is caused by my own shortcomings: as I dig into a problem, I simply discover that it is a lot harder than I initially thought, so the problem takes longer to solvesuch is life as a programmer. Just as often I know exactly what I want to achieve and how to achieve it, but it still takes far longer than anticipated. When that happens, it is usually because I am struggling with an API that seems to do its level best to throw rocks in my path and make my life difficult.

ASPs:
The Integration Challenge

The promise of software as a service is becoming a reality with many ASPs (application service providers). Organizations using ASPs and third-party vendors that provide value-added products to ASPs need to integrate with them. ASPs enable this integration by providing Web service-based APIs. There are significant differences between integrating with ASPs over the Internet and integrating with a local application. When integrating with ASPs, users have to consider a number of issues, including latency, unavailability, upgrades, performance, load limiting, and lack of transaction support.

Above the Line, Below the Line

The resilience of Internet-facing systems relies on what is below the line of representation.

Abstracting the Geniuses Away from Failure Testing

Ordinary users need tools that automate the selection of custom-tailored faults to inject.

Abstraction in Hardware System Design

Applying lessons from software languages to hardware languages using Bluespec SystemVerilog

Accountability in Algorithmic Decision-making

A view from computational journalism

Achieving Digital Permanence

The many challenges to maintaining stored information and ways to overcome them

Adopting DevOps Practices in Quality Assurance

Merging the art and science of software development

Advances and Challenges in Log Analysis

Logs contain a wealth of information for help in managing systems.

Agile and SEMAT - Perfect Partners

Combining agile and SEMAT yields more advantages than either one alone

Algorithms Behind Modern Storage Systems

Different uses for read-optimized B-trees and write-optimized LSM-trees

All Your Database Are Belong to Us

In the big open world of the cloud, highly available distributed objects will rule.

An Open Web Services Architecture

The name of the game is web services.

Anatomy of a Solid-state Drive

While the ubiquitous SSD shares many features with the hard-disk drive, under the surface they are completely different.

Another Day, Another Bug

We asked our readers which tools they use to squash bugs. Here's what they said.

Arm Your Applications for Bulletproof Deployment: A Conversation with Tom Spalthoff

Companies can achieve a reliable desktop environment while reducing the time and cost spent preparing high-quality application packages.

Arrogance in Business Planning

Technology business plans that assume no competition (ever)

Attack Trends:
2004 and 2005

Counterpane Internet Security Inc. monitors more than 450 networks in 35 countries, in every time zone. In 2004 we saw 523 billion network events, and our analysts investigated 648,000 security “tickets.” What follows is an overview of what’s happening on the Internet right now, and what we expect to happen in the coming months.

Automated QA Testing at EA: Driven by Events

A discussion with Michael Donat, Jafar Husain, and Terry Coatta

Automating Software Failure Reporting

We can only fix those bugs we know about.

BASE: An Acid Alternative

In partitioned databases, trading some consistency for availability can lead to dramatic improvements in scalability.

BBR: Congestion-Based Congestion Control

Measuring bottleneck bandwidth and round-trip propagation time

BPM: The Promise and the Challenge

Over the last decade, businesses and governments have been giving increasing attention to business processes - to their description, automation, and management. This interest grows out of the need to streamline business operations, consolidate organizations, and save costs, reflecting the fact that the process is the basic unit of business value within an organization.

Barbarians at the Gateways

High-frequency Trading and Exchange Technology

Benchmarking "Hello, World!"

Best Practice (BPM)

In business process management, finding the right tool suite is just the beginning.

Best Practices on the Move: Building Web Apps for Mobile Devices

Which practices should be modified or avoided altogether by developers for the mobile Web?

Better Scripts, Better Games

Smarter, more powerful scripting languages will improve game performance while making gameplay development more efficient.

Better, Faster, More Secure

Since I started a stint as chair of the IETF (Internet Engineering Task Force) in March 2005, I have frequently been asked, “What’s coming next?” but I have usually declined to answer. Nobody is in charge of the Internet, which is a good thing, but it makes predictions difficult (and explains why this article starts with a disclaimer: It represents my views alone and not those of my colleagues at either IBM or the IETF).

Beyond Beowulf Clusters

In the early ’90s, the Berkeley NOW (Network of Workstations) Project under David Culler posited that groups of less capable machines (running SunOS) could be used to solve scientific and other computing problems at a fraction of the cost of larger computers. In 1994, Donald Becker and Thomas Sterling worked to drive the costs even lower by adopting the then-fledgling Linux operating system to build Beowulf clusters at NASA’s Goddard Space Flight Center. By tying desktop machines together with open source tools such as PVM (Parallel Virtual Machine), MPI (Message Passing Interface), and PBS (Portable Batch System), early clusters—which were often PC towers stacked on metal shelves with a nest of wires interconnecting them—fundamentally altered the balance of scientific computing.

Beyond Instant Messaging

Platforms and standards for these services must anticipate and accommodate future developments.

Beyond Page Objects: Testing Web Applications with State Objects

Use states to drive your tests

Beyond Relational Databases

There is more to data access than SQL.

Beyond Server Consolidation

Virtualization technology was developed in the late 1960s to make more efficient use of hardware. Hardware was expensive, and there was not that much available.

Beyond the Fix-it Treadmill

The Use of Post-Incident Artifacts in High-Performing Organizations

Big Games, Small Screens

One thing that becomes immediately apparent when creating and distributing mobile 3D games is that there are fundamental differences between the cellphone market and the more traditional games markets, such as consoles and handheld gaming devices. The most striking of these are the number of delivery platforms; the severe constraints of the devices, including small screens whose orientation can be changed; limited input controls; the need to deal with other tasks; the nonphysical delivery mechanism; and the variations in handset performance and input capability.

Bitcoin's Academic Pedigree

The concept of cryptocurrencies is built from forgotten ideas in research literature.

Bitcoin's Underlying Incentives

The unseen economic forces that govern the Bitcoin protocol

Black Box Debugging

It's all about what takes place at the boundary of an application.

Blaster Revisited

What lessons can we learn from the carnage the Blaster worm created? The following tale is based upon actual circumstances from corporate enterprises that were faced with confronting and eradicating the Blaster worm, which hit in August 2003. The story provides views from many perspectives, illustrating the complexity and sophistication needed to combat new blended threats.

Blockchain Technology: What Is It Good for?

Industry's dreams and fears for this new technology

Blurring Lines Between Hardware and Software

Software development for embedded systems clearly transcends traditional "programming" and requires intimate knowledge of hardware, as well as deep understanding of the underlying application that is to be implemented.

Borg, Omega, and Kubernetes

Lessons learned from three container-management systems over a decade

Box Their SOXes Off

Data is a precious resource for any large organization. The larger the organization, the more likely it will rely to some degree on third-party vendors and partners to help it manage and monitor its mission-critical data. In the wake of new regulations for public companies, such as Section 404 of SOX (Sarbanes-Oxley Act of 2002), the folks who run IT departments for Fortune 1000 companies have an ever-increasing need to know that when it comes to the 24/7/365 monitoring of their critical data transactions, they have business partners with well-planned and well-documented procedures.

Breaking the Major Release Habit

Can agile development make your team more productive?

Bridging the Object-Relational Divide

ORM technologies can simplify data access, but be aware of the challenges that come with introducing this new layer of abstraction.

Bringing Arbitrary Compute to Authoritative Data

Many disparate use cases can be satisfied with a single storage system.

Broadcast Messaging:
Messaging to the Masses

This powerful form of communication has social implications as well as technical challenges.

Browser Security:
Lessons from Google Chrome

Google Chrome developers focused on three key problems to shield the browser from attacks.

Bufferbloat: Dark Buffers in the Internet

Networks without effective AQM may again be vulnerable to congestion collapse.

Building Collaboration into IDEs

Edit>Compile>Run>Debug>Collaborate?

Building Nutch:
Open Source Search

Search engines are as critical to Internet use as any other part of the network infrastructure, but they differ from other components in two important ways. First, their internal workings are secret, unlike, say, the workings of the DNS (domain name system). Second, they hold political and cultural power, as users increasingly rely on them to navigate online content.

Building Scalable Web Services

Build only what you really need.

Building Secure Web Applications

In these days of phishing and near-daily announcements of identity theft via large-scale data losses, it seems almost ridiculous to talk about securing the Web. At this point most people seem ready to throw up their hands at the idea or to lock down one small component that they can control in order to keep the perceived chaos at bay. 

Building Systems to Be Shared, Securely

The history of computing has been characterized by continuous transformation resulting from the dramatic increases in performance and drops in price described by Moore's law. Computing "power" has migrated from centralized mainframes/servers to distributed systems and the commodity desktop. Despite these changes, system sharing remains an important tool for computing. From the multitasking, file-sharing, and virtual machines of the desktop environment to the large-scale sharing of server-class ISP hardware in collocation centers, safely sharing hardware between mutually untrusting parties requires addressing critical concerns of accidental and malicious damage.

C Is Not a Low-level Language

Your computer is not a fast PDP-11.

CPU DB: Recording Microprocessor History

With this open database, you can mine microprocessor trends over the past 40 years.

Cache Me If You Can

Building a decentralized web-delivery model

Caching XML Web Services for Mobility

In the face of unreliable connections and low bandwidth, caching may offer reliable wireless access to Web services.

Canary Analysis Service

Automated canarying quickens development, improves production safety, and helps prevent outages.

Certificate Transparency

Public, verifiable, append-only logs

Challenges of Memory Management on Modern NUMA System

Optimizing NUMA systems applications with Carrefour

Closed Source Fights Back

SCO vs. The World-What Were They Thinking?

Cluster-level Logging of Containers with Containers

Logging Challenges of Container-Based Cloud Deployments

Code Spelunking:
Exploring Cavernous Code Bases

Code diving through unfamiliar source bases is something we do far more often than write new code from scratch--make sure you have the right gear for the job.

Code Spelunking Redux

Is this subject important enough to warrant two articles in five years? I believe it is.

Coding Guidelines:
Finding the Art in the Science

What separates good code from great code?

Coding Smart: People vs. Tools

Tools can help developers be more productive, but they're no replacement for thinking.

Coding for the Code

Can models provide the DNA for software development?

Cognitive Work of Hypothesis Exploration During Anomaly Response

A look at how we respond to the unexpected

Collaboration in System Administration

For sysadmins, solving problems usually involves collaborating with others. How can we make it more effective?

Commercializing Open Source Software

Many have tried, a few are succeeding, but challenges abound.

Communications Surveillance:
Privacy and Security at Risk

As the sophistication of wiretapping technology grows, so too do the risks it poses to our privacy and security.

Compliance Deconstructed

The topic of compliance becomes increasingly complex each year. Dozens of regulatory requirements can affect a company’s business processes. Moreover, these requirements are often vague and confusing. When those in charge of compliance are asked if their business processes are in compliance, it is understandably difficult for them to respond succinctly and with confidence. This article looks at how companies can deconstruct compliance, dealing with it in a systematic fashion and applying technology to automate compliance-related business processes. It also looks specifically at how Microsoft approaches compliance to SOX (Sarbanes-Oxley Act of 2002).

Complying with Compliance

“Hey, compliance is boring. Really, really boring. And besides, I work neither in the financial industry nor in health care. Why should I care about SOX and HIPAA?” Yep, you’re absolutely right. You write payroll applications, or operating systems, or user interfaces, or (heaven forbid) e-mail servers. Why should you worry about compliance issues?

Componentizing the Web

We may be on the cusp of a new revolution in web development.

Computers in Patient Care: The Promise and the Challenge

Information technology has the potential to radically transform health care. Why has progress been so slow?

Computing without Processors

Heterogeneous systems allow us to target our programming to the appropriate environment.

Condos and Clouds

Constraints in an environment empower the services.

Containers Will Not Fix Your Broken Culture (and Other Hard Truths)

Complex socio-technical systems are hard; film at 11.

Continuous Delivery Sounds Great, but Will It Work Here?

It's not magic, it just requires continuous, daily improvement at all levels.

Controlling Queue Delay

A modern AQM is just one piece of the solution to bufferbloat.

Cooling the Data Center

What can be done to make cooling systems in data centers more energy efficient?

Corp to Cloud: Google's Virtual Desktops

How Google moved its virtual desktops to the cloud

Crash Consistency

Rethinking the Fundamental Abstractions of the File System

Creating Languages in Racket

Sometimes you just have to make a better mousetrap.

Criminal Code:
The Making of a Cybercriminal

NOTE: This is a fictional account of malware creators and their experiences. Although the characters are made up, the techniques and events are patterned on real activities of many different groups developing malicious software.

Culture Surprises in Remote Software Development Teams

"When in Rome" doesn't help when your team crosses time zones, and your deadline doesn't.

Cybercrime:
An Epidemic

Painted in the broadest of strokes, cybercrime essentially is the leveraging of information systems and technology to commit larceny, extortion, identity theft, fraud, and, in some cases, corporate espionage. Who are the miscreants who commit these crimes, and what are their motivations? One might imagine they are not the same individuals committing crimes in the physical world. Bank robbers and scam artists garner a certain public notoriety after only a few occurrences of their crimes, yet cybercriminals largely remain invisible and unheralded. Based on sketchy news accounts and a few public arrests, such as Mafiaboy, accused of paralyzing Amazon, CNN, and other Web sites, the public may infer these miscreants are merely a subculture of teenagers.

Cybercrime 2.0: When the Cloud Turns Dark

Web-based malware attacks are more insidious than ever. What can be done to stem the tide?

DAFS:
A New High-Performance Networked File System

This emerging file-access protocol dramatically enhances the flow of data over a network, making life easier in the data center.

DNS Complexity

DNS (domain name system) is a distributed, coherent, reliable, autonomous, hierarchical database, the first and only one of its kind. Created in the 1980s when the Internet was still young but overrunning its original system for translating host names into IP addresses, DNS is one of the foundation technologies that made the worldwide Internet (and the World Wide Web) possible. Yet this did not all happen smoothly, and DNS technology has been periodically refreshed and refined. Though it’s still possible to describe DNS in simple terms, the underlying details are by now quite sublime.

DSL for the Uninitiated

Domain-specific languages bridge the semantic gap in programming.

DSPs: Back to the Future

From the dawn of the DSP (digital signal processor), an old quote still echoes: "Oh, no! We'll have to use state-of-the-art 5µm NMOS!" The speaker's name is lost in the fog of history, as are many things from the ancient days of 5µm chip design. This quote refers to the first Bell Labs DSP whose mask set in fact underwent a 10 percent linear lithographic shrink to 4.5µm NMOS (N-channel metal oxide semiconductor) channel length and taped out in late 1979 with an aggressive full-custom circuit design.

Data Sketching

The approximate approach is often faster and more efficient.

Data in Flight

How streaming SQL technology can help solve the Web 2.0 data crunch.

Data-Parallel Computing

Users always care about performance. Although often it's just a matter of making sure the software is doing only what it should, there are many cases where it is vital to get down to the metal and leverage the fundamental characteristics of the processor.

Databases of Discovery

Open-ended database ecosystems promote new discoveries in biotech. Can they help your organization, too?

Death by UML Fever

A potentially deadly illness, clinically referred to as UML (Unified Modeling Language) fever, is plaguing many software-engineering efforts today. This fever has many different strains that vary in levels of lethality and contagion. A number of these strains are symptomatically related, however. Rigorous laboratory analysis has revealed that each is unique in origin and makeup. A particularly insidious characteristic of UML fever, common to most of its assorted strains, is the difficulty individuals and organizations have in self-diagnosing the affliction. A consequence is that many cases of the fever go untreated and often evolve into more complex and lethal strains.

Debugging Distributed Systems

Challenges and options for validation and debugging

Debugging in an Asynchronous World

Hard-to-track bugs can emerge when you can't guarantee sequential execution. The right tools and the right techniques can help.

Decentralizing SIP

If you're looking for a low-maintenance IP communications network, peer-to-peer SIP might be just the thing.

Demystifying Stablecoins

Cryptography meets monetary policy

Describing the Elephant:
The Different Faces of IT as Service

In a well-known fable, a group of blind men are asked to describe an elephant. Each encounters a different part of the animal and, not surprisingly, provides a different description. We see a similar degree of confusion in the IT industry today, as terms such as service-oriented architecture, grid, utility computing, on-demand, adaptive enterprise, data center automation, and virtualization are bandied about. As when listening to the blind men, it can be difficult to know what reality lies behind the words, whether and how the different pieces fit together, and what we should be doing about the animal(s) that are being described.

Design Exploration through Code-generating DSLs

High-level DSLs for low-level programming

Designing Cluster Schedulers for Internet-Scale Services

Embracing failures for improving availability

Designing Portable Collaborative Networks

A middleware solution to keep pace with the ever-changing ways in which mobile workers collaborate.

Desktop Linux: Where Art Thou?

Linux on the desktop has come a long way - and it's been a roller-coaster ride. At the height of the dot-com boom, around the time of Red Hat's initial public offering, people expected Linux to take off on the desktop in short order. A few years later, after the stock market crash and the failure of a couple of high-profile Linux companies, pundits were quick to proclaim the stillborn death of Linux on the desktop.

DevOps Metrics

Your biggest mistake might be collecting the wrong data.

Digitally Assisted Analog Integrated Circuits

In past decades, "Moore's law" has governed the revolution in microelectronics. Through continuous advancements in device and fabrication technology, the industry has maintained exponential progress rates in transistor miniaturization and integration density. As a result, microchips have become cheaper, faster, more complex, and more power efficient.

Disambiguating Databases

Use the database built for your access model.

Discrimination in Online Ad Delivery

Google ads, black names and white names, racial discrimination, and click advertising

Disks from the Perspective of a File System

Disks lie. And the controllers that run them are partners in crime.

Dismantling the Barriers to Entry

We have to choose to build a web that is accessible to everyone.

Distributed Computing Economics

Computing economics are changing. Today there is rough price parity between: (1) one database access; (2) 10 bytes of network traffic; (3) 100,000 instructions; (4) 10 bytes of disk storage; and (5) a megabyte of disk bandwidth. This has implications for how one structures Internet-scale distributed computing: one puts computing as close to the data as possible in order to avoid expensive network traffic.

Distributed Development:
Lessons Learned

Why repeat the mistakes of the past if you don't have to?

Division of Labor in Embedded Systems

You can choose among several strategies for partitioning an embedded application over incoherent processor cores. Here's practical advice on the advantages and pitfalls of each.

Document & Media Exploitation

A computer used by Al Qaeda ends up in the hands of a Wall Street Journal reporter. A laptop from Iran is discovered that contains details of that country's nuclear weapons program. Photographs and videos are downloaded from terrorist Web sites.

Domain-specific Languages and Code Synthesis Using Haskell

Looking at embedded DSLs

Don't Settle for Eventual Consistency

Stronger properties for low-latency geo-replicated storage

Dynamics of Change: Why Reactivity Matters

Tame the dynamics of change by centralizing each concern in its own module.

E-mail Authentication:
What, Why, How?

Internet e-mail was conceived in a different world than we live in today. It was a small, tightly knit community, and we didn’t really have to worry too much about miscreants. Generally, if someone did something wrong, the problem could be dealt with through social means; “shunning” is very effective in small communities.

Energy Management on Handheld Devices

Handheld devices are becoming ubiquitous and as their capabilities increase, they are starting to displace laptop computers - much as laptop computers have displaced desktop computers in many roles. Handheld devices are evolving from today's PDAs, organizers, cellular phones, and game machines into a variety of new forms. Although partially offset by improvements in low-power electronics, this increased functionality carries a corresponding increase in energy consumption. Second, as a consequence of displacing other pieces of equipment, handheld devices are seeing more use between battery charges. Finally, battery technology is not improving at the same pace as the energy requirements of handheld electronics.

Enhanced Debugging with Traces

An essential technique used in emulator development is a useful addition to any programmer's toolbox.

Enterprise Grid Computing

I have to admit a great measure of sympathy for the IT populace at large, when it is confronted by the barrage of hype around grid technology, particularly within the enterprise. Individual vendors have attempted to plant their flags in the notionally virgin technological territory and proclaim it as their own, using terms such as grid, autonomic, self-healing, self-managing, adaptive, utility, and so forth. Analysts, well, analyze and try to make sense of it all, and in the process each independently creates his or her own map of this terra incognita, naming it policy-based computing, organic computing, and so on.

Enterprise SSDs

Solid-state drives are finally ready for the enterprise. But beware, not all SSDs are created alike.

Enterprise Search: Tough Stuff

The last decade has witnessed the growth of information retrieval from a boutique discipline in information and library science to an everyday experience for billions of people around the world. This revolution has been driven in large measure by the Internet, with vendors focused on search and navigation of Web resources and Web content management. Simultaneously, enterprises have invested in networking all of their information together to the point where it is increasingly possible for employees to have a single window into the enterprise.

Enterprise Software as Service

While the practice of outsourcing business functions such as payroll has been around for decades, its realization as online software services has only recently become popular. In the online service model, a provider develops an application and operates the servers that host it. Customers access the application over the Internet using industry-standard browsers or Web services clients. A wide range of online applications, including e-mail, human resources, business analytics, CRM (customer relationship management), and ERP (enterprise resource planning), are available.

Enterprise-Grade Wireless

We have been working in the wireless space in one form or another in excess of 10 years and have participated in every phase of its maturation process. We saw wireless progress from a toy technology before the dot-com boom, to something truly promising during the boom, only to be left wanting after the bubble when the technology was found to be not ready for prime time. Fortunately, it appears that we have finally reached the point where the technology and the enterprise's expectations have finally converged.

Erlang for Concurrent Programming

What role can programming languages play in dealing with concurrency? One answer can be found in Erlang, a language designed for concurrency from the ground up.

Error Messages:
What's the Problem?

Computer users spend a lot of time chasing down errors - following the trail of clues that starts with an error message and that sometimes leads to a solution and sometimes to frustration. Problems with error messages are particularly acute for system administrators (sysadmins) - those who configure, install, manage, and maintain the computational infrastructure of the modern world - as they spend a lot of effort to keep computers running amid errors and failures.

Eventual Consistency Today: Limitations, Extensions, and Beyond

How can applications be built on eventually consistent infrastructure given no guarantee of safety?

Eventually Consistent

Building reliable distributed systems at a worldwide scale demands trade-offs?between consistency and availability.

Eventually Consistent: Not What You Were Expecting?

Methods of quantifying consistency (or lack thereof) in eventually consistent storage systems

Evolution and Practice: Low-latency Distributed Applications in Finance

The finance industry has unique demands for low-latency distributed systems.

Evolution of the Product Manager

Better education needed to develop the discipline

Exposing the ORM Cache

Familiarity with ORM caching issues can help prevent performance problems and bugs.

Extending the Semantics of Scheduling Priorities

Increasing parallelism demands new paradigms.

Extensible Programming for the 21st Century

Is an open, more flexible programming environment just around the corner?

Extreme Software Scaling

The advent of SMP (symmetric multiprocessing) added a new degree of scalability to computer systems. Rather than deriving additional performance from an incrementally faster microprocessor, an SMP system leverages multiple processors to obtain large gains in total system performance. Parallelism in software allows multiple jobs to execute concurrently on the system, increasing system throughput accordingly. Given sufficient software parallelism, these systems have proved to scale to several hundred processors.

FPGA Programming for the Masses

The programmability of FPGAs must improve if they are to be part of mainstream computing.

Fail at Scale

Reliability in the face of rapid change

Faucet: Deploying SDN in the Enterprise

Using OpenFlow and DevOps for rapid development

Fault Injection in Production

Making the case for resilience testing

Fighting Physics: A Tough Battle

Thinking of doing IPC over the long haul? Think again. The laws of physics say you're hosed.

Fighting Spam with Reputation Systems

User-submitted spam fingerprints

Finding More Than One Worm in the Apple

If you see something, say something.

Finding Usability Bugs with Automated Tests

Automated usability tests can be valuable companions to in-person tests.

Flash Disk Opportunity for Server Applications

Future flash-based disks could provide breakthroughs in IOPS, power, reliability, and volumetric capacity when compared with conventional disks.

Flash Storage Today

Can flash memory become the foundation for a new tier in the storage hierarchy?

Four Billion Little Brothers?:
Privacy, mobile phones, and ubiquitous data collection

Participatory sensing technologies could improve our lives and our communities, but at what cost to our privacy?

From COM to Common

Ten years ago, the term component software meant something relatively specific and concrete. A small number of software component frameworks more or less defined the concept for most people. Today, few terms in the software industry are less precise than component software. There are now many different forms of software componentry for many different purposes. The technologies and methodologies of 10 years ago have evolved in fundamental ways and have been joined by an explosion of new technologies and approaches that have redefined our previously held notions of component software.

From IR to Search, and Beyond

Searching has come a long way since the 60s, but have we only just begun?

From Liability to Advantage: A Conversation with John Graham-Cumming and John Ousterhout

Software production has become a bottleneck in many development organizations.

From Server Room to Living Room

How open source and TiVo became a perfect match

From the EDVAC to WEBVACs

Cloud computing for computer scientists

Fun and Games:
Multi-Language Development

Computer games (or "electronic games" if you encompass those games played on console-class hardware) comprise one of the fastest-growing application markets in the world. Within the development community that creates these entertaining marvels, multi-language development is becoming more commonplace as games become more and more complex. Today, asking a development team to construct a database-enabled Web site with the requirement that it be written entirely in C++ would earn scornful looks and rolled eyes, but not long ago the idea that multiple languages were needed to accomplish a given task was scoffed at.

Functional at Scale

Applying functional programming principles to distributed computing projects

Future Graphics Architectures

Graphics architectures are in the midst of a major transition. In the past, these were specialized architectures designed to support a single rendering algorithm: the standard Z buffer. Realtime 3D graphics has now advanced to the point where the Z-buffer algorithm has serious shortcomings for generating the next generation of higher-quality visual effects demanded by games and other interactive 3D applications. There is also a desire to use the high computational capability of graphics architectures to support collision detection, approximate physics simulations, scene management, and simple artificial intelligence.

Fuzzy Boundaries:
Objects, Components, and Web Services

It's easy to transform objects into components and Web services, but how do we know which is right for the job?

GPUs: A Closer Look

A gamer wanders through a virtual world rendered in near- cinematic detail. Seconds later, the screen fills with a 3D explosion, the result of unseen enemies hiding in physically accurate shadows. Disappointed, the user exits the game and returns to a computer desktop that exhibits the stylish 3D look-and-feel of a modern window manager. Both of these visual experiences require hundreds of gigaflops of computing performance, a demand met by the GPU (graphics processing unit) present in every consumer PC.

Game Development:
Harder Than You Think

The hardest part of making a game has always been the engineering. In times past, game engineering was mainly about low-level optimization - writing code that would run quickly on the target computer, leveraging clever little tricks whenever possible.

Gaming Graphics:
The Road to Revolution

It has been a long journey from the days of multicolored sprites on tiled block backgrounds to the immersive 3D environments of modern games. What used to be a job for a single game creator is now a multifaceted production involving staff from every creative discipline. The next generation of console and home computer hardware is going to bring a revolutionary leap in available computing power; a teraflop (trillion floating-point operations per second) or more will be on tap from commodity hardware.

Garbage Collection as a Joint Venture

A collaborative approach to reclaiming memory in heterogeneous software systems

Getting Bigger Reach Through Speech

Mark Ericson, vice president of product strategy for BlueNote Networks argues that in order to take advantage of new voice technologies you have to have a plan for integrating that capability directly into the applications that drive your existing business processes.

Getting Gigascale Chips:
Challenges and Opportunities in Continuing Moore's Law

Processor performance has increased by five orders of magnitude in the last three decades, made possible by following Moore's law - that is, continued technology scaling, improved transistor performance to increase frequency, additional (to avoid repetition) integration capacity to realize complex architectures, and reduced energy consumed per logic operation to keep power dissipation within limits. Advances in software technology, such as rich multimedia applications and runtime systems, exploited this performance explosion, delivering to end users higher productivity, seamless Internet connectivity, and even multimedia and entertainment.

Getting What You Measure

Four common pitfalls in using software metrics for project management

Go Static or Go Home

In the end, dynamic systems are simply less secure.

Going with the Flow

Workflow systems can provide value beyond automating business processes.

Hack for Hire

Investigating the emerging black market of retail email account hacking services

Hadoop Superlinear Scalability

The perpetual motion of parallel performance

Hard Disk Drives:
The Good, the Bad and the Ugly!

HDDs (hard-disk drives) are like the bread in a peanut butter and jelly sandwich—sort of an unexciting piece of hardware necessary to hold the “software.” They are simply a means to an end. HDD reliability, however, has always been a significant weak link, perhaps the weak link, in data storage. In the late 1980s people recognized that HDD reliability was inadequate for large data storage systems so redundancy was added at the system level with some brilliant software algorithms, and RAID (redundant array of inexpensive disks) became a reality. RAID moved the reliability requirements from the HDD itself to the system of data disks.

Hazy: Making it Easier to Build and Maintain Big-data Analytics

Racing to unleash the full potential of big data with the latest statistical and machine-learning techniques.

Heterogeneous Computing: Here to Stay

Hardware and Software Perspectives

Hidden in Plain Sight

Improvements in the observability of software can help you diagnose your most crippling performance problems.

High Performance Web Sites

Want to make your Web site fly? Focus on front-end performance.

How Do I Model State? Let Me Count the Ways

A study of the technology and sociology of Web services specifications

How Fast is Your Web Site?

Web site performance data has never been more readily available.

How Not to Write Fortran in Any Language

There are characteristics of good coding that transcend all programming languages.

How OSGi Changed My Life

In the early 1980s I discovered OOP (object-oriented programming) and fell in love with it, head over heels. As usual, this kind of love meant convincing management to invest in this new technology, and most important of all, send me to cool conferences. So I pitched the technology to my manager. I sketched him the rosy future, how one day we would create applications from ready-made classes. We would get those classes from a repository, put them together, and voila, a new application would be born.

How Will Astronomy Archives Survive the Data Tsunami?

Astronomers are collecting more data than ever. What practices can keep them ahead of the flood?

How to De-identify Your Data

Balancing statistical accuracy and subject privacy in large social-science data sets

How to Live in a Post-Meltdown and -Spectre World

Learn from the past to prepare for the next battle.

I/O Virtualization

Decoupling a logical device from its physical implementation offers many compelling advantages.

Idempotence Is Not a Medical Condition

An essential property for reliable systems

Identity by Any Other Name

The complex cacophony of intertwined systems

Idle-Time Garbage-Collection Scheduling

Taking advantage of idleness to reduce dropped frames and memory consumption

If You Have Too Much Data, then "Good Enough" Is Good Enough

In today's humongous database systems, clarity may be relaxed, but business needs can still be met.

Immutability Changes Everything

We need it, we can afford it, and the time is now.

Improving Performance on the Internet

Given the Internet's bottlenecks, how can we build fast, scalable content-delivery systems?

Industrial Scale Agile - from Craft to Engineering

Essence is instrumental in moving software development toward a true engineering discipline.

Industry-scale Knowledge Graphs: Lessons and Challenges

Five diverse technology companies show how it's done

Information Extraction:
Distilling Structured Data from Unstructured Text

Distilling structured data from unstructured text

Injecting Errors for Fun and Profit

Error-detection and correction features are only as good as our ability to test them.

Instant Messaging or Instant Headache?

It's a reality. You have IM (instant messaging) clients in your environment. You have already recognized that it is eating up more and more of your network bandwidth and with Microsoft building IM capability into its XP operating system and applications, you know this will only get worse. Management is also voicing concerns over the lost user productivity caused by personal conversations over this medium. You have tried blocking these conduits for conversation, but it is a constant battle.

Integrating RFID

RFID (radio frequency identification) has received a great deal of attention in the commercial world over the past couple of years. The excitement stems from a confluence of events. First, through the efforts of the former Auto-ID Center and its sponsor companies, the prospects of low-cost RFID tags and a networked supply chain have come within reach of a number of companies. Second, several commercial companies and government bodies, such as Wal-Mart and Target in the United States, Tesco in Europe, and the U.S. Department of Defense, have announced RFID initiatives in response to technology improvements.

Intellectual Property and Software Piracy:
The Power of IP Protection and Software Licensing, an interview with Aladdin vice president Gregg Gronowski

The Power of IP Protection and Software Licensing, an interview with Aladdin vice president Gregg Gronowski

Interactive Dynamics for Visual Analysis

A taxonomy of tools that support the fluent and flexible use of visualizations

Intermediate Representation

The increasing significance of intermediate representations in compilers

Internal Access Controls

Trust, but Verify

Is Open Source Right for You?:
A Fictional Case Study of Open Source in a Commercial Software Shop

The media often present open source software as a direct competitor to commercial software. This depiction, usually pitting David (Linux) against Goliath (Microsoft), makes for fun reading in the weekend paper. However, it mostly misses the point of what open source means to a development organization. In this article, I use the experiences of GizmoSoft (a fictitious software company) to present some perspectives on the impact of open source software usage in a software development shop.

Is There a Single Method for the Internet of Things?

Essence can keep software development for the IoT from becoming unwieldy.

It Probably Works

Probabilistic algorithms are all around us--not only are they acceptable, but some programmers actually seek out chances to use them.

Java Security Architecture Revisited

Hard technical problems and tough business challenges

Java in a Teacup

Programming Bluetooth-enabled devices using J2ME

JavaScript and the Netflix User Interface

Conditional dependency resolution

Keeping Bits Safe:
How Hard Can It Be?

As storage systems grow larger and larger, protecting their data for long-term storage is becoming more and more challenging.

Keeping Score in the IT Compliance Game

Achieving developer acceptance of standardized procedures for managing applications from development to release is one of the largest hurdles facing organizations today. Establishing a standardized development-to-release workflow, often referred to as the ALM (application lifecycle management) process, is particularly critical for organizations in their efforts to meet tough IT compliance mandates. This is much easier said than done, as different development teams have created their own unique procedures that are undocumented, unclear, and nontraceable.

Lack of Priority Queuing Considered Harmful

Most modern routers consist of several line cards that perform packet lookup and forwarding, all controlled by a control plane that acts as the brain of the router, performing essential tasks such as management functions, error reporting, control functions including route calculations, and adjacency maintenance. This control plane has many names; in this article it is the route processor, or RP. The route processor calculates the forwarding table and downloads it to the line cards using a control-plane bus. The line cards perform the actual packet lookup and forwarding.

Languages, Levels, Libraries, and Longevity

New programming languages are born every day. Why do some succeed and some fail? In 50 years, we've already seen numerous programming systems come and (mostly) go, although some have remained a long time and will probably do so for: decades? centuries? millennia? The questions about language designs, levels of abstraction, libraries, and resulting longevity are numerous. Why do new languages arise? Why is it sometimes easier to write new software than to adapt old software that works? How many different levels of languages make sense? Why do some languages last in the face of "better" ones?

Leaking Space

Eliminating memory hogs

Learning from the Web

In the past decade we have seen a revolution in computing that transcends anything seen to date in terms of scope and reach, but also in terms of how we think about what makes up “good” and “bad” computing. The Web taught us several unintuitive lessons:

Lessons from the Floor

The manufacturing industry can teach us a lot about measuring performance in large-scale Internet services.

Lessons from the Letter

Security flaws in a large organization

Leveraging Application Frameworks

In today's competitive, fast-paced computing industry, successful software must increasingly be: (1) extensible to support successions of quick updates and additions to address new requirements and take advantage of emerging markets; (2) flexible to support a growing range of multimedia data types, traffic flows, and end-to-end QoS (quality of service) requirements; (3) portable to reduce the effort required to support applications on heterogeneous operating-system platforms and compilers; (4) reliable to ensure that applications are robust and tolerant to faults; (5) scalable to enable applications to handle larger numbers of clients simultaneously; and (6) affordable to ensure that the total ownership costs of software acquisition and evolution are not prohibitively high.

Life Beyond Distributed Transactions

An apostate's opinion

META II: Digital Vellum in the Digital Scriptorium

Revisiting Schorre's 1962 compiler-compiler

Major-league SEMAT: Why Should an Executive Care?

Becoming better, faster, cheaper, and happier

Making Money Using Math

Modern applications are increasingly using probabilistic machine-learned models.

Making SIP Make Cents

P2P payments using SIP could enable new classes of micropayment applications and business models.

Making Sense of Revision-control Systems

Whether distributed or centralized, all revision-control systems come with complicated sets of tradeoffs. How do you find the best match between tool and team?

Making a Case for Efficient Supercomputing

A supercomputer evokes images of "big iron" and speed; it is the Formula 1 racecar of computing. As we venture forth into the new millennium, however, I argue that efficiency, reliability, and availability will become the dominant issues by the end of this decade, not only for supercomputing, but also for computing in general.

Making the Mobile Web Faster

Mobile performance issues? Fix the back end, not just the client.

Making the Web Faster with HTTP 2.0

HTTP continues to evolve

Managing Collaboration

Jeff Johnstone of TechExcel explains why there is a need for a new approach to application lifecycle management that better reflects the business requirements and challenges facing development teams.

Managing Contention for Shared Resources on Multicore Processors

Contention for caches, memory controllers, and interconnects can be alleviated by contention-aware scheduling algorithms.

Managing Semi-Structured Data

I vividly remember during my first college class my fascination with the relational database—an information oasis that guaranteed a constant flow of correct, complete, and consistent information at our disposal. In that class I learned how to build a schema for my information, and I learned that to obtain an accurate schema there must be a priori knowledge of the structure and properties of the information to be modeled.

Managing Technical Debt

Shortcuts that save money and time today can cost you down the road.

Managing the Hidden Costs of Coordination

Controlling coordination costs when multiple, distributed perspectives are essential

Massively Multiplayer Middleware

Wish is a multiplayer, online, fantasy role-playing game being developed by Mutable Realms. It differs from similar online games in that it allows tens of thousands of players to participate in a single game world (instead of the few hundred players supported by other games). Allowing such a large number of players requires distributing the processing load over a number of machines and raises the problem of choosing an appropriate distribution technology.

Maximizing Power Efficiency with Asymmetric Multicore Systems

Asymmetric multicore systems promise to use a lot less energy than conventional symmetric processors. How can we develop software that makes the most out of this potential?

Meet the Virts

When you dig into the details of supposedly overnight success stories, you frequently discover that they've actually been years in the making. Virtualization has been around for more than 30 years since the days when some of you were feeding stacks of punch cards into very physical machines yet in 2007 it tipped. VMware was the IPO sensation of the year; in November 2007 no fewer than four major operating system vendors (Microsoft, Oracle, Red Hat, and Sun) announced significant new virtualization capabilities; and among fashionable technologists it seems virtual has become the new black.

Metamorphosis: the Coming Transformation of Translational Systems Biology

In the future computers will mine patient data to deliver faster, cheaper healthcare, but how will we design them to give informative causal explanations? Ideas from philosophy, model checking, and statistical testing can pave the way for the needed translational systems biology.

Metaphors We Compute By

Code is a story that explains how to solve a particular problem.

Metrics That Matter

Critical but oft-neglected service metrics that every SRE and product owner should care about

Mind Your State for Your State of Mind

The interactions between storage and applications can be complex and subtle.

Mobile Application Development: Web vs. Native

Web apps are cheaper to develop and deploy than native apps, but can they match the native user experience?

Mobile Media:
Making It a Reality

Many future mobile applications are predicated on the existence of rich, interactive media services. The promise and challenge of such services is to provide applications under the most hostile conditions - and at low cost to a user community that has high expectations. Context-aware services require information about who, where, when, and what a user is doing and must be delivered in a timely manner with minimum latency. This article reveals some of the current state-of-the-art "magic" and the research challenges.

Model-based Testing: Where Does It Stand?

MBT has positive effects on efficiency and effectiveness, even if it only partially fulfills high expectations.

Modeling People and Places with Internet Photo Collections

Understanding the world from the sea of online photos

Modern Performance Monitoring

Today's diverse and decentralized computer world demands new thinking about performance monitoring and analysis.

Modern System Power Management

The Advanced Configuration and Power Interface (ACPI) is the most widely used power and configuration interface for laptops, desktops, and server systems. It is also very complex, and its current specification weighs in at more than 500 pages. Needless to say, operating systems that choose to support ACPI require significant additional software support, up to and including fundamental OS architecture changes. The effort that ACPI's definition and implementation has entailed is worth the trouble because of how much flexibility it gives to the OS (and ultimately the user) to control power management policy and implementation.

MongoDB's JavaScript Fuzzer

The fuzzer is for those edge cases that your testing didn't catch.

Monitoring and Control of Large Systems with MonALISA

MonALISA developers describe how it works, the key design principles behind it, and the biggest technical challenges in building it.

Monitoring in a DevOps World

Perfect should never be the enemy of better.

Monitoring, at Your Service

Automated monitoring can increase the reliability and scalability of today's online software services.

Multipath TCP

Decoupled from IP, TCP is at last able to support multihomed hosts.

Multitier Programming in Hop

A first step toward programming 21st-century applications

NUMA (Non-Uniform Memory Access): An Overview

NUMA becomes more common because memory controllers get close to execution units on microprocessors.

National Internet Defense - Small States on the Skirmish Line

Attacks in Estonia and Georgia highlight key vulnerabilities in national Internet infrastructure.

Natural Language Translation at the Intersection of AI and HCI

Old questions being answered with both AI and HCI

Net Neutrality: Unexpected Solution to Blockchain Scaling

Cloud-delivery networks could dramatically improve blockchains' scalability, but clouds must be provably neutral first.

Network Applications Are Interactive

The network era requires new models, with interactions instead of algorithms.

Network Forensics

The dictionary defines forensics as "the use of science and technology to investigate and establish facts in criminal or civil courts of law." I am more interested, however, in the usage common in the computer world: using evidence remaining after an attack on a computer to determine how the attack was carried out and what the attacker did.

Network Front-end Processors, Yet Again

The history of NFE processors sheds light on the tradeoffs involved in designing network stack software.

Network Virtualization:
Breaking the Performance Barrier

The recent resurgence in popularity of virtualization has led to its use in a growing number of contexts, many of which require high-performance networking. Consider server consolidation, for example. The efficiency of network virtualization directly impacts the number of network servers that can effectively be consolidated onto a single physical machine. Unfortunately, modern network virtualization techniques incur significant overhead, which limits the achievable network performance. We need new network virtualization techniques to realize the full benefits of virtualization in network-intensive domains.

Nine IM Accounts and Counting

The key word with instant messaging today is interoperability. Various standards are in contention.

No Source Code? No Problem!

What if you have to port a program, but all you have is a binary?

Non-volatile Storage

Implications of the Datacenter's Shifting Center

Nonblocking Algorithms and Scalable Multicore Programming

Exploring some alternatives to lock-based synchronization

Not Your Father's PBX?

Perhaps no piece of office equipment is more taken for granted than the common business telephone. The technology behind this basic communication device, however, is in the midst of a major transformation. Businesses are now converging their voice and data networks in order to simplify their network operations and take advantage of the new functional benefits and capabilities that a converged network delivers from greater productivity and cost savings to enhanced mobility.

OCaml for the Masses

Why the next language you learn should be functional

ORM in Dynamic Languages

O/R mapping frameworks for dynamic languages such as Groovy provide a different flavor of ORM that can greatly simplify application code.

Ode to a Sailor

sailor, fleeting mood image of you; all sailor in bear grace, rough hands and poetic dream;

Of Processors and Processing

Digital signal processing is a stealth technology. It is the core enabling technology in everything from your cellphone to the Mars Rover. It goes much further than just enabling a one-time breakthrough product. It provides ever-increasing capability; compare the performance gains made by dial-up modems with the recent performance gains of DSL and cable modems. Remarkably, digital signal processing has become ubiquitous with little fanfare, and most of its users are not even aware of what it is.

On Mapping Alogrithms to DSP Architectures

Our complex world is characterized by representation, transmission, and storage of information - and information is mostly processed in digital form. With the advent of DSPs (digital signal processors), engineers are able to implement complex algorithms with relative ease. Today we find DSPs all around us - in cars, digital cameras, MP3 and DVD players, modems, and so forth. Their widespread use and deployment in complex systems has triggered a revolution in DSP architectures, which in turn has enabled engineers to implement algorithms of ever-increasing complexity.

On Plug-ins and Extensible Architectures

Extensible application architectures such as Eclipse offer many advantages, but one must be careful to avoid "plug-in hell."

One Step Ahead

Security vulnerabilities abound, but a few simple steps can minimize your risk.

Online Algorithms in High-frequency Trading

The challenges faced by competing HFT algorithms

Online Event Processing

Achieving consistency where distributed transactions have failed

Oops! Coping with Human Error in IT Systems

Errors Happen. How to Deal.

Open Source to the Core

The open source development model is not exactly new. Individual engineers have been using open source as a collaborative development methodology for decades. Now that it has come to the attention of upper and middle management, however, it's finally being openly acknowledged as a commercial engineering force-multiplier and important option for avoiding significant software development costs.

Open Spectrum:
A Path to Ubiquitous Connectivity

Path to Ubiquitous Connectivity

Open vs. Closed:
Which Source is More Secure?

Which source is more secure?

Open-source Firmware

Step into the world behind the kernel.

OpenFlow: A Radical New Idea in Networking

An open standard that enables software-defined networking

Optimizations in C++ Compilers

A practical journey

Orchestrating an Automated Test Lab

Networking and the Internet are encouraging increasing levels of interaction and collaboration between people and their software. Whether users are playing games or composing legal documents, their applications need to manage the complex interleaving of actions from multiple machines over potentially unreliable connections. As an example, Silicon Chalk is a distributed application designed to enhance the in-class experience of instructors and students. Its distributed nature requires that we test with multiple machines. Manual testing is too tedious, expensive, and inconsistent to be effective. While automating our testing, however, we have found it very labor intensive to maintain a set of scripts describing each machine's portion of a given test.

Order from Chaos

There is probably little argument that the past decade has brought the “big bang” in the amount of online information available for processing by humans and machines. Two of the trends that it spurred (among many others) are: first, there has been a move to more flexible and fluid (semi-structured) models than the traditional centralized relational databases that stored most of the electronic data before; second, today there is simply too much information available to be processed by humans, and we really need help from machines.

Other People's Data

Companies have access to more types of external data than ever before. How can they integrate it most effectively?

Outsourcing: Devising a Game Plan

Your CIO just summoned you to duty by handing off the decision-making power about whether to outsource next years big development project to rewrite the internal billing system. That's quite a daunting task! How can you possibly begin to decide if outsourcing is the right option for your company? There are a few strategies that you can follow to help you avoid the pitfalls of outsourcing and make informed decisions. Outsourcing is not exclusively a technical issue, but it is a decision that architects or development managers are often best qualified to make because they are in the best position to know what technologies make sense to keep in-house.

Parallel Processing with Promises

A simple method of writing a collaborative system

Parallel Programming with Transactional Memory

While sometimes even writing regular, single-threaded programs can be quite challenging, trying to split a program into multiple pieces that can be executed in parallel adds a whole dimension of additional problems. Drawing upon the transaction concept familiar to most programmers, transactional memory was designed to solve some of these problems and make parallel programming easier. Ulrich Drepper from Red Hat shows us how it's done.

Passing a Language through the Eye of a Needle

How the embeddability of Lua impacted its design

Passively Measuring TCP Round-trip Times

A close look at RTT measurements with TCP

Patching the Enterprise

Organizations of all sizes are spending considerable efforts on getting patch management right - their businesses depend on it.

People and Process

Minimizing the pain of business process change

People in Our Software

People are not well represented in today's software. With the exception of IM (instant messaging) clients, today's applications offer few clues that people are actually living beings. Static strings depict things associated with people like e-mail addresses, phone numbers, and home-page URLs. Applications also tend to show the same information about a person, no matter who is viewing it.

Perfect Storm:
The Insider, Naivety, and Hostility

Every year corporations and government installations spend millions of dollars fortifying their network infrastructures. Firewalls, intrusion detection systems, and antivirus products stand guard at network boundaries, and individuals monitor countless logs and sensors for even the subtlest hints of network penetration. Vendors and IT managers have focused on keeping the wily hacker outside the network perimeter, but very few technological measures exist to guard against insiders - those entities that operate inside the fortified network boundary. The 2002 CSI/FBI survey estimates that 70 percent of successful attacks come from the inside. Several other estimates place those numbers even higher.

Performance Anti-Patterns

Want your apps to run faster? Here's what not to do.

Persistent Memory Programming on Conventional Hardware

The persistent memory style of programming can dramatically simplify application software.

Pervasive, Dynamic Authentication of Physical Items

The use of silicon PUF circuits

Phishing Forbidden

Phishing is a significant risk facing Internet users today.1,2 Through e-mails or instant messages, users are led to counterfeit Web sites designed to trick them into divulging usernames, passwords, account numbers, and personal information. It is up to the user to ensure the authenticity of the Web site.

Playing for Keeps

Inflection points come at you without warning and quickly recede out of reach. We may be nearing one now. If so, we are now about to play for keeps, and “we” doesn’t mean just us security geeks. If anything, it’s because we security geeks have not worked the necessary miracles already that an inflection point seems to be approaching at high velocity.

Postmortem Debugging in Dynamic Environments

Modern dynamic languages lack tools for understanding software failures.

Power-Efficient Software

Power-manageable hardware can help save energy, but what can software developers do to address the problem?

Powering Down

Power management - from laptops to rooms full of servers - is a topic of interest to everyone. In the beginning there was the desktop computer. It ran at a fixed speed and consumed less power than the monitor it was plugged into. Where computers were portable, their sheer size and weight meant that you were more likely to be limited by physical strength than battery life. It was not a great time for power management.

Principles of Robust Timing over the Internet

The key to synchronizing clocks over networks is taming delay variability.

Privacy, Anonymity, and Big Data in the Social Sciences

Quality social science research and the privacy of human subjects requires trust.

Probing Biomolecular Machines with Graphics Processors

The evolution of GPU processors and programming tools is making advanced simulation and analysis techniques accessible to a growing community of biomedical scientists.

Productivity in Parallel Programming: A Decade of Progress

Looking at the design and benefits of X10

Programmers Are People, too

I would like to start out this article with an odd, yet surprisingly uncontroversial assertion, which is this: programmers are human. I wish to use this as a premise to explore how to improve the programmer’s lot. So, please, no matter your opinion on the subject, grant me this assumption for the sake of argument.

Programming Without a Net

Embedded systems programming presents special challenges to engineers unfamiliar with that environment.

Provenance in Sensor Data Management

A cohesive, independent solution for bringing provenance to scientific research

Proving the Correctness of Nonblocking Data Structures

So you've decided to use a nonblocking data structure, and now you need to be certain of its correctness. How can this be achieved? When a multithreaded program is too slow because of a frequently acquired mutex, the programmer's typical reaction is to question whether this mutual exclusion is indeed required. This doubt becomes even more pronounced if the mutex protects accesses to only a single variable performed using a single instruction at every site. Removing synchronization improves performance, but can it be done without impairing program correctness?

Purpose-Built Languages

While often breaking the rules of traditional language design, the growing ecosystem of purpose-built "little" languages is an essential part of systems development.

Putting It All Together

Component integration is one of the tough challenges in embedded system design. Designers search for conservative design styles and reliable techniques for interfacing and verification.

Quality Assurance:
Much More than Testing

Quality assurance isn't just testing, or analysis, or wishful thinking. Although it can be boring, difficult, and tedious, QA is nonetheless essential.

Rate-limiting State

The edge of the Internet is an unruly place

Reading, Writing, and Code

Forty years ago, when computer programming was an individual experience, the need for easily readable code wasn't on any priority list. Today, however, programming usually is a team-based activity, and writing code that others can easily decipher has become a necessity. Creating and developing readable code is not as easy as it sounds.

Real-World Concurrency

In this look at how concurrency affects practitioners in the real world, Cantrill and Bonwick argue that much of the anxiety over concurrency is unwarranted.

Realtime Computer Vision with OpenCV

Mobile computer-vision technology will soon become as ubiquitous as touch interfaces.

Realtime GPU Audio

Finite difference-based sound synthesis using graphics processors

Realtime Garbage Collection

It's now possible to develop realtime systems using Java.

Reconfigurable Future

The Ability to Produce Cheaper, More Compact Chips is a Double-edged Sword.

Reliable Cron across the Planet

...or How I stopped worrying and learned to love time

Rethinking Passwords

Our authentication system is lacking. Is improvement possible?

Returning Control to the Programmer:
SIMD Intrinsics for Virtual Machines

Exposing SIMD units within interpreted languages could simplify programs and unleash floods of untapped processor power.

Revealing the Critical Role of Human Performance in Software

It's time to revise our appreciation of the human side of Internet-facing software systems.

Revisiting Network I/O APIs: The netmap Framework

It is possible to achieve huge performance improvements in the way packet processing is done on modern operating systems.

Rules for Mobile Performance Optimization

An overview of techniques to speed page loading

SAGE: Whitebox Fuzzing for Security Testing

SAGE has had a remarkable impact at Microsoft.

SIP:
Basics and Beyond

More than just a simple telephony application protocol, SIP is a framework for developing communications systems.

Scalability Techniques for Practical Synchronization Primitives

Designing locking primitives with performance in mind

Scalable Parallel Programming with CUDA

The advent of multicore CPUs and manycore GPUs means that mainstream processor chips are now parallel systems. Furthermore, their parallelism continues to scale with Moore's law. The challenge is to develop mainstream application software that transparently scales its parallelism to leverage the increasing number of processor cores, much as 3D graphics applications transparently scale their parallelism to manycore GPUs with widely varying numbers of cores.

Scalable SQL

How do large-scale sites and applications remain SQL-based?

Scaling Existing Lock-based Applications with Lock Elision

Lock elision enables existing lock-based programs to achieve the performance benefits of nonblocking synchronization and fine-grain locking with minor software engineering effort.

Scaling Synchronization in Multicore Programs

Advanced synchronization methods can boost the performance of multicore software.

Scaling in Games & Virtual Worlds

Online games and virtual worlds have familiar scaling requirements, but don't be fooled: everything you know is wrong.

Schema.org: Evolution of Structured Data on the Web

Big data makes common schemas even more necessary.

Search Considered Integral

A combination of tagging, categorization, and navigation can help end-users leverage the power of enterprise search.

Searching vs. Finding

Finding information and organizing it so that it can be found are two key aspects of any company's knowledge management strategy. Nearly everyone is familiar with the experience of searching with a Web search engine and using a search interface to search a particular Web site once you get there. (You may have even noticed that the latter often doesn't work as well as the former.) After you have a list of hits, you typically spend a significant amount of time following links, waiting for pages to download, reading through a page to see if it has what you want, deciding that it doesn't, backing up to try another link, deciding to try another way to phrase your request, et cetera.

Securing Elasticity in the Cloud

Elastic computing has great potential, but many security challenges remain.

Securing the Network Time Protocol

Crackers discover how to use NTP as a weapon for abuse.

Securing the Tangled Web

Preventing script injection vulnerabilities through software design

Security - Problem Solved?

There are plenty of security problems that have solutions. Yet, our security problems don’t seem to be going away. What’s wrong here? Are consumers being offered snake oil and rejecting it? Are they not adopting solutions they should be adopting? Or, is there something else at work, entirely? We’ll look at a few places where the world could easily be a better place, but isn’t, and build some insight as to why.

Security Collapse in the HTTPS Market

Assessing legal and technical solutions to secure HTTPS

Security in the Browser

Web browsers leave users vulnerable to an ever-growing number of attacks. Can we make them secure while preserving their usability?

Security is Harder than You Think

Many developers see buffer overflows as the biggest security threat to software and believe that there is a simple two-step process to secure software: switch from C or C++ to Java, then start using SSL (Secure Sockets Layer) to protect data communications. It turns out that this naïve tactic isn't sufficient. In this article, we explore why software security is harder than people expect, focusing on the example of SSL.

Security: The Root of the Problem

Security bug? My programming language made me do it! It doesn't seem that a day goes by without someone announcing a critical flaw in some crucial piece of software or other. Is software that bad? Are programmers so inept? What the heck is going on, and why is the problem getting worse instead of better? One distressing aspect of software security is that we fundamentally don't seem to "get it."

Self-Healing Networks

Wireless networks that fix their own broken communication links may speed up their widespread acceptance.

Self-Healing in Modern Operating Systems

A few early steps show there's a long (and bumpy) road ahead.

Sender-side Buffers and the Case for Multimedia Adaptation

A proposal to improve the performance and availability of streaming video and other time-sensitive media

Sensible Authentication

The problem with securing assets and their functionality is that, by definition, you don't want to protect them from everybody. It makes no sense to protect assets from their owners, or from other authorized individuals (including the trusted personnel who maintain the security system). In effect, then, all security systems need to allow people in, even as they keep people out. Designing a security system that accurately identifies, authenticates, and authorizes trusted individuals is highly complex and filled with nuance, but critical to security.

Sentient Data Access via a Diverse Society of Devices

Today's ubiquitous computing environment cannot benefit from the traditional understanding of a hierarchical file system.

Should You Upload or Ship Big Data to the Cloud?

The accepted wisdom does not always hold true.

Sifting Through the Software Sandbox:
SCM Meets QA

Thanks to modern SCM (software configuration management) systems, when developers work on a codeline they leave behind a trail of clues that can reveal what parts of the code have been modified, when, how, and by whom. From the perspective of QA (quality assurance) and test engineers, is this all just "data," or is there useful information that can improve the test coverage and overall quality of a product?

Simplicity Betrayed

Emulating a video system shows how even a simple interface can be more complex—and capable—than it appears.

Simulators:
Virtual Machines of the Past (and Future)

Simulators are a form of "virtual machine" intended to address a simple problem: the absence of real hardware. Simulators for past systems address the loss of real hardware and preserve the usability of software after real hardware has vanished. Simulators for future systems address the variability of future hardware designs and facilitate the development of software before real hardware exists.

Sink or Swim:
Know When It's Time to Bail

A diagnostic to help you measure organizational dysfunction and take action

SoC: Software, Hardware, Nightmare, Bliss

System-on-a-chip design offers great promise by shrinking an entire computer to a single chip. But with the promise come challenges that need to be overcome before SoC reaches its full potential.

Social Bookmarking in the Enterprise

Can your organization benefit from social bookmarking tools?

Social Perception

Modeling human interaction for the next generation of communication services

Software Development with Code Maps

Could those ubiquitous hand-drawn code diagrams become a thing of the past?

Software Needs Seatbelts and Airbags

Finding and fixing bugs in deployed software is difficult and time-consuming. Here are some alternatives.

Software Transactional Memory: Why Is It Only a Research Toy?

The promise of STM may likely be undermined by its overheads and workload applicabilities.

Software and the Concurrency Revolution

Leveraging the full power of multicore processors demands new tools and new thinking from the software industry. Concurrency has long been touted as the "next big thing" and "the way of the future," but for the past 30 years, mainstream software development has been able to ignore it. Our parallel future has finally arrived: new machines will be parallel machines, and this will require major changes in the way we develop software. The introductory article in this issue ("The Future of Microprocessors" by Kunle Olukotun and Lance Hammond) describes the hardware imperatives behind this shift in computer architecture from uniprocessors to multicore processors, also known as CMPs (chip multiprocessors).

Spam, Spam, Spam, Spam, Spam, the FTC, and Spam

A forum sponsored by the FTC highlights just how bad spam is, and how it's only going to get worse without some intervention.

Spicing Up Dart with Side Effects

A set of extensions to the Dart programming language, designed to support asynchrony and generator functions

Splinternet Behind the Great Firewall of China

Once China opened its door to the world, it could not close it again.

Standardizing Storage Clusters

Data-intensive applications such as data mining, movie animation, oil and gas exploration, and weather modeling generate and process huge amounts of data. File-data access throughput is critical for good performance. To scale well, these HPC (high-performance computing) applications distribute their computation among numerous client machines. HPC clusters can range from hundreds to thousands of clients with aggregate I/O demands ranging into the tens of gigabytes per second.

Statistics for Engineers

Applying statistical techniques to operations data

Storage Systems:
Not Just a Bunch of Disks Anymore

The sheer size and scope of data available today puts tremendous pressure on storage systems to perform in ways never imagined.

Storage Virtualization Gets Smart

Over the past 20 years we have seen the transformation of storage from a dumb resource with fixed reliability, performance, and capacity to a much smarter resource that can actually play a role in how data is managed. In spite of the increasing capabilities of storage systems, however, traditional storage management models have made it hard to leverage these data management capabilities effectively. The net result has been overprovisioning and underutilization. In short, although the promise was that smart shared storage would simplify data management, the reality has been different.

Stream Processors: Progammability and Efficiency

Many signal processing applications require both efficiency and programmability. Baseband signal processing in 3G cellular base stations, for example, requires hundreds of GOPS (giga, or billions, of operations per second) with a power budget of a few watts, an efficiency of about 100 GOPS/W (GOPS per watt), or 10 pJ/op (picoJoules per operation). At the same time programmability is needed to follow evolving standards, to support multiple air interfaces, and to dynamically provision processing resources over different air interfaces. Digital television, surveillance video processing, automated optical inspection, and mobile cameras, camcorders, and 3G cellular handsets have similar needs.

Streams and Standards:
Delivering Mobile Video

Don’t believe me? Follow along… Mobile phones are everywhere. Everybody has one. Think about the last time you were on an airplane and the flight was delayed on the ground. Immediately after the dreaded announcement, you heard everyone reach for their phones and start dialing.

Structured Deferral: Synchronization via Procrastination

We simply do not have a synchronization mechanism that can enforce mutual exclusion.

Successful Strategies for IPv6 Rollouts. Really.

Knowing where to begin is half the battle.

Surviving Software Dependencies

Software reuse is finally here but comes with risks.

System Administration Soft Skills

How can system administrators reduce stress and conflict in the workplace?

TCP Offload to the Rescue

In recent years, TCP/IP offload engines, known as TOEs, have attracted a good deal of industry attention and a sizable share of venture capital dollars. A TOE is a specialized network device that implements a significant portion of the TCP/IP protocol in hardware, thereby offloading TCP/IP processing from software running on a general-purpose CPU. This article examines the reasons behind the interest in TOEs and looks at challenges involved in their implementation and deployment.

Tackling Architectural Complexity with Modeling

Component models can help diagnose architectural problems in both new and existing systems.

Tear Down the Method Prisons! Set Free the Practices!

Essence: a new way of thinking that promises to liberate the practices and enable true learning organizations

Testable System Administration

Models of indeterminism are changing IT management.

Testing a Distributed System

Testing a distributed system can be trying even under the best of circumstances.

The (not so) Hidden Computer

The growing complexity of purpose-built systems is making it difficult to conceal the computers within.

The API Performance Contract

How can the expected interactions between caller and implementation be guaranteed?

The Age of Corporate Open Source Enlightenment

Like it or not, zealots and heretics are finding common ground in the open source holy war.

The Answer is 42 of Course

Why is security so hard? As a security consultant, I’m glad that people feel that way, because that perception pays my mortgage. But is it really so difficult to build systems that are impenetrable to the bad guys?

The Antifragile Organization

Embracing Failure to Improve Resilience and Maximize Availability

The Balancing Act of Choosing Nonblocking Features

Design requirements of nonblocking systems

The Big Bang Theory of IDEs

Remember the halcyon days when development required only a text editor, a compiler, and some sort of debugger (in cases where the odd printf() or two alone didn't serve)? During the early days of computing, these were independent tools used iteratively in development's golden circle. Somewhere along the way we realized that a closer integration of these tools could expedite the development process. Thus was born the integrated development environment (IDE), a framework and user environment for software development that's actually a toolkit of instruments essential to software creation. At first, IDEs simply connected the big three (editor, compiler, and debugger), but nowadays most go well beyond those minimum requirements.

The Calculus of Service Availability

You're only as available as the sum of your dependencies.

The Case Against Data Lock-in

Want to keep your users? Just make it easy for them to leave.

The Challenge of Cross-language Interoperability

Interfacing between languages is increasingly important.

The Cost of Virtualization

Virtualization can be implemented in many different ways. It can be done with and without hardware support. The virtualized operating system can be expected to be changed in preparation for virtualization, or it can be expected to work unchanged. Regardless, software developers must strive to meet the three goals of virtualization spelled out by Gerald Popek and Robert Goldberg: fidelity, performance, and safety.

The Curse of the Excluded Middle

"Mostly functional" programming does not work.

The Debugging Mindset

Understanding the psychology of learning strategies leads to effective problem-solving skills.

The Deliberate Revolution

Transforming Integration With XML Web Services

The Effects of Mixing Machine Learning and Human Judgment

Collaboration between humans and machines does not necessarily lead to better outcomes.

The Emergence of iSCSI

Modern SCSI, as defined by the SCSI-3 Architecture Model, or SAM, really considers the cable and physical interconnections to storage as only one level in a larger hierarchy.

The Essence of Software Engineering: The SEMAT Kernel

A thinking framework in the form of an actionable kernel

The Evolution of Security

Security people are never in charge unless an acute embarrassment has occurred. Otherwise, their advice is tempered by “economic reality,” which is to say that security is a means, not an end. This is as it should be. Since means are about tradeoffs, security is about trade-offs, but you knew all that.

The Evolution of Web Development for Mobile Devices

Building Web sites that perform well on mobile devices remains a challenge.

The Family Dynamics of 802.11

The 802.11 family of standards is helping to move wireless LANs into promising new territory.

The Five-Minute Rule 20 Years Later:
and How Flash Memory Changes the Rules

The old rule continues to evolve, while flash memory adds two new rules.

The Flame Graph

This visualization of software execution is a new necessity for performance profiling and debugging.

The Future of Human-Computer Interaction

Personal computing launched with the IBM PC. But popular computing launched with the modern WIMP (windows, icons, mouse, pointer) interface, which made computers usable by ordinary people.

The Future of Microprocessors

The performance of microprocessors that power modern computers has continued to increase exponentially over the years for two main reasons. First, the transistors that are the heart of the circuits in all processors and memory chips have simply become faster over time on a course described by Moore’s law,1 and this directly affects the performance of processors built with those transistors. Moreover, actual processor performance has increased faster than Moore’s law would predict,2 because processor designers have been able to harness the increasing numbers of transistors available on modern chips to extract more parallelism from software.

The Future of WLAN

Overcoming the Top Ten Challenges in wireless networking--will it allow wide-area mesh networks to become ubiquitous?

The Heart of Eclipse

A look inside and extensible plug-in architecture ECLIPSE is both an open, extensible development environment for building software and an open, extensible application framework upon which software can be built. Considered the most popular Java IDE, it provides a common UI model for working with tools and promotes rapid development of modular features based on a plug-in component model. The Eclipse Foundation designed the platform to run natively on multiple operating systems, including Macintosh, Windows, and Linux, providing robust integration with each and providing rich clients that support the GUI interactions everyone is familiar with: drag and drop, cut and paste (clipboard), navigation, and customization.

The Hidden Dividends of Microservices

Microservices aren't for every company, and the journey isn't easy.

The Hitchhiker's Guide to Biomorphic Software

The natural world may be the inspiration we need for solving our computer problems. While it is certainly true that "the map is not the territory," most visitors to a foreign country do prefer to take with them at least a guidebook to help locate themselves as they begin their explorations. That is the intent of this article. Although there will not be enough time to visit all the major tourist sites, with a little effort and using the information in the article as signposts, the intrepid explorer can easily find numerous other, interesting paths to explore.

The IDAR Graph

An improvement over UML

The Ideal HPC Programming Language

Maybe it's Fortran. Or maybe it just doesn't matter.

The Inevitability of Reconfigurable Systems

The introduction of the microprocessor in 1971 marked the beginning of a 30-year stall in design methods for electronic systems. The industry is coming out of the stall by shifting from programmed to reconfigurable systems. In programmed systems, a linear sequence of configuration bits, organized into blocks called instructions, configures fixed hardware to mimic custom hardware. In reconfigurable systems, the physical connections among logic elements change with time to mimic custom hardware. The transition to reconfigurable systems will be wrenching, but this is inevitable as the design emphasis shifts from cost performance to cost performance per watt. Here's the story.

The Invisible Assistant

One lab's experiment with ubiquitous computing

The Long Road to 64 Bits

"Double, double, toil and trouble"... Shakespeare's words (Macbeth, Act 4, Scene 1) often cover circumstances beyond his wildest dreams. Toil and trouble accompany major computing transitions, even when people plan ahead. To calibrate "tomorrow's legacy today," we should study "tomorrow's legacy yesterday." Much of tomorrow's software will still be driven by decades-old decisions. Past decisions have unanticipated side effects that last decades and can be difficult to undo.

The Magic of RFID

Many modern technologies give the impression they work by magic, particularly when they operate automatically and their mechanisms are invisible. A technology called RFID (radio frequency identification), which is relatively new to the mass market, has exactly this characteristic and for many people seems a lot like magic. RFID is an electronic tagging technology that allows an object, place, or person to be automatically identified at a distance without a direct line-of-sight, using an electromagnetic challenge/response exchange.

The Mythos of Model Interpretability

In machine learning, the concept of interpretability is both important and slippery.

The NSA and Snowden: Securing the All-Seeing Eye

How good security at the NSA could have stopped him

The Network is Reliable

An informal survey of real-world communications failures

The Network's New Role

Application-oriented networks can help bridge the gap between enterprises.

The Obama Campaign:
A Programmer's Perspective

The Obama campaign has been praised for its innovative use of technology. What was the key to its success?

The Pain of Implementing LINQ Providers

It's no easy task for NoSQL

The Pathologies of Big Data

Scale up your datasets enough and all your apps will come undone. What are the typical problems and where do the bottlenecks generally surface?

The Price of Performance

In the late 1990s, our research group at DEC was one of a growing number of teams advocating the CMP (chip multiprocessor) as an alternative to highly complex single-threaded CPUs. We were designing the Piranha system,1 which was a radical point in the CMP design space in that we used very simple cores (similar to the early RISC designs of the late ’80s) to provide a higher level of thread-level parallelism. Our main goal was to achieve the best commercial workload performance for a given silicon budget.

The Reincarnation of Virtual Machines

Virtualization makes a comeback.

The Reliability of Enterprise Applications

Understanding enterprise reliability

The Responsive Enterprise: Embracing the Hacker Way

Soon every company will be a software company.

The Rise and Fall of CORBA

Depending on exactly when one starts counting, CORBA is about 10-15 years old. During its lifetime, CORBA has moved from being a bleeding-edge technology for early adopters, to being a popular middleware, to being a niche technology that exists in relative obscurity. It is instructive to examine why CORBA—despite once being heralded as the “next-generation technology for e-commerce”—suffered this fate. CORBA’s history is one that the computing industry has seen many times, and it seems likely that current middleware efforts, specifically Web services, will reenact a similar history.

The Road to SDN

An intellectual history of programmable networks

The Robustness Principle Reconsidered

Seeking a middle ground

The Scalability Problem

Back in the mid-1990s, I worked for a company that developed multimedia kiosk demos. Our biggest client was Intel, and we often created demos that appeared in new PCs on the end-caps of major computer retailers such as CompUSA. At that time, performance was in demand for all application classes from business to consumer. We created demos that showed, for example, how much faster a spreadsheet would recalculate (you had to do that manually back then) on a new processor as compared with the previous year's processor. The differences were immediately noticeable to even a casual observer - and it mattered.

The Science of Managing Data Science

Lessons learned managing a data science research team

The Seven Deadly Sins of Linux Security

Avoid these common security risks like the devil.

The Software Inferno

Dante's tale, as experienced by a software architect

The Story of the Teapot in DHTML

It's easy to do amazing things, such as rendering the classic teapot in HTML and CSS.

The Sun Never Sits on Distributed Development

People around the world can work around the clock on a distributed project, but the real challenge lies in taming the social dynamics.

The Verification of a Distributed System

A practitioner's guide to increasing confidence in system correctness

The Virtualization Reality

A number of important challenges are associated with the deployment and configuration of contemporary computing infrastructure. Given the variety of operating systems and their many versions—including the often-specific configurations required to accommodate the wide range of popular applications—it has become quite a conundrum to establish and manage such systems.

The Web Won't Be Safe or Secure until We Break It

Unless you've taken very particular precautions, assume every Web site you visit knows exactly who you are.

The World According to LINQ

Big data is about more than size, and LINQ is more than up to the task.

The Yin and Yang of Software Development

The C/C++ Solution Manager at Parasoft explains how infrastructure elements allow development teams to increase productivity without restricting creativity.

There is No Now

Problems with simultaneity in distributed systems

There's Just No Getting around It: You're Building a Distributed System

Building a distributed system requires a methodical approach to requirements.

There's No Such Thing as a Free (Software) Lunch

"The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software to make sure the software is free for all its users." So begins the GNU General Public License, or GPL, which has become the most widely used of open source software licenses. Freedom is the watchword; it's no coincidence that the organization that wrote the GPL is called the Free Software Foundation and that open source developers everywhere proclaim, "Information wants to be free."

There's No Such Thing as a General-purpose Processor

And the belief in such a device is harmful

Thinking Clearly about Performance

Improving the performance of complex software is difficult, but understanding some fundamental principles can make it easier.

Thinking Methodically about Performance

The USE method addresses shortcomings in other commonly used methodologies.

Thou Shalt Not Depend on Me

A look at JavaScript libraries in the wild

Thread Scheduling in FreeBSD 5.2

A busy system makes thousands of scheduling decisions per second, so the speed with which scheduling decisions are made is critical to the performance of the system as a whole. This article - excerpted from the forthcoming book, "The Design and Implementation of the FreeBSD Operating System" - uses the example of the open source FreeBSD system to help us understand thread scheduling. The original FreeBSD scheduler was designed in the 1980s for large uniprocessor systems. Although it continues to work well in that environment today, the new ULE scheduler was designed specifically to optimize multiprocessor and multithread environments. This article first studies the original FreeBSD scheduler, then describes the new ULE scheduler.

Threads without the Pain

Multithreaded programming need not be so angst-ridden.

TiVo-lution

The challenges of delivering a reliable, easy-to-use DVR service to the masses

Time is an Illusion.

Lunchtime doubly so. - Ford Prefect to Arthur Dent in "The Hitchhiker's Guide to the Galaxy", by Douglas Adams

Time, but Faster

A computing adventure about time through the looking glass

Titus: Introducing Containers to the Netflix Cloud

Approaching container adoption in an already cloud-native infrastructure

Too Big NOT to Fail

Embrace failure so it doesn't embrace you.

Too Darned Big to Test

The increasing size and complexity of software, coupled with concurrency and distributed systems, has made apparent the ineffectiveness of using only handcrafted tests. The misuse of code coverage and avoidance of random testing has exacerbated the problem. We must start again, beginning with good design (including dependency analysis), good static checking (including model property checking), and good unit testing (including good input selection). Code coverage can help select and prioritize tests to make you more efficient, as can the all-pairs technique for controlling the number of configurations.

Too Much Information

Two applications reveal the key challenges in making context-aware computing a reality. As mobile computing devices and a variety of sensors become ubiquitous, new resources for applications and services - often collectively referred to under the rubric of context-aware computing - are becoming available to designers and developers. In this article, we consider the potential benefits and issues that arise from leveraging context awareness in new communication services that include the convergence of VoIP (voice over IP) and traditional information technology.

Toward Energy-Efficient Computing

What will it take to make server-side computing more energy efficient?

Toward Higher Precision

An introduction to PTP and its significance to NTP practitioners

Toward Software-defined SLAs

Enterprise computing in the public cloud

Toward a Commodity Enterprise Middleware

Can AMQP enable a new era in messaging middleware? AMQP (Advanced Message Queuing Protocol) was born out of my own experience and frustrations in developing front- and back-office processing systems at investment banks. It seemed to me that we were living in integration Groundhog Day - the same problems of connecting systems together would crop up with depressing regularity. Each time the same discussions about which products to use would happen, and each time the architecture of some system would be curtailed to allow for the fact that the chosen middleware was reassuringly expensive.

Tracking and Controlling Microservice Dependencies

Dependency management is a crucial part of system and software design.

Trials and Tribulations of Debugging Concurrency

We now sit firmly in the 21st century where the grand challenge to the modern-day programmer is neither memory leaks nor type issues (both of those problems are now effectively solved), but rather issues of concurrency. How does one write increasingly complex programs where concurrency is a first-class concern. Or even more treacherous, how does one debug such a beast? These questions bring fear into the hearts of even the best programmers.

Triple-Parity RAID and Beyond

As hard-drive capacities continue to outpace their throughput, the time has come for a new level of RAID.

UML Fever:
Diagnosis and Recovery

Acknowledgment is only the first step toward recovery from this potentially devastating affliction. The Institute of Infectious Diseases has recently published research confirming that the many and varied strains of UML Fever continue to spread worldwide, indiscriminately infecting software analysts, engineers, and managers alike. One of the fevers most serious side effects has been observed to be a significant increase in both the cost and duration of developing software products. This increase is largely attributable to a decrease in productivity resulting from fever-stricken individuals investing time and effort in activities that are of little or no value to producing deliverable products.

Under New Management

Autonomic computing is revolutionizing the way we manage complex systems.

Undergraduate Software Engineering: Addressing the Needs of Professional Software Development

Addressing the Needs of Professional Software Development

Understanding DRM

The explosive growth of the Internet and digital media has created both tremendous opportunities and new threats for content creators. Advances in digital technology offer new ways of marketing, disseminating, interacting with, and monetizing creative works, giving rise to expanding markets that did not exist just a few years ago. At the same time, however, the technologies have created major challenges for copyright holders seeking to control the distribution of their works and protect against piracy.

Understanding Database Reconstruction Attacks on Public Data

These attacks on statistical databases are no longer a theoretical danger.

Understanding Software Patching

Developing and deploying patches is an increasingly important part of the software development process.

Unified Communications with SIP

SIP can provide realtime communications as a network service.

Unifying Biological Image Formats with HDF5

The biosciences need an image format capable of high performance and long-term maintenance. Is HDF5 the answer?

Unikernels: Rise of the Virtual Library Operating System

What if all the software layers in a virtual appliance were compiled within the same safe, high-level language framework?