20 Obstacles to Scalability
Watch out for these pitfalls that can prevent Web application scaling.
A Call to Arms
Long anticipated, the arrival of radically restructured database architectures is now finally at hand.
A Decade of OS Access-control Extensibility
Open source security foundations for mobile and embedded devices
A File System All Its Own
Flash memory has come a long way. Now it's time for software to catch up.
A Guided Tour through Data-center Networking
A good user experience depends on predictable performance within the data-center network.
A High-Performance Team
From design to production, performance should be part of the process.
A New Objective-C Runtime: from Research to Production
Backward compatibility always trumps new features.
A New Software Engineering
What happened to the promise of rigorous, disciplined, professional practices for software development?
A Passage to India
Most American IT employees take a dim view of offshore outsourcing. It's considered unpatriotic and it drains valuable intellectual capital and jobs from the United States to destinations such as India or China. Online discussion forums on sites such as isyourjobgoingoffshore.com are headlined with titles such as "How will you cope?" and "Is your career in danger?" A cover story in BusinessWeek magazine a couple of years ago summed up the angst most people suffer when faced with offshoring: "Is your job next?"
A Pioneer's Flash of Insight
Jim Gray's vision of flash-based storage anchors this issue's theme.
A Plea to Software Vendors from Sysadmins - 10 Do's and Don'ts
What can software vendors do to make the lives of sysadmins a little easier?
A Primer on Provenance
Better understanding of data requires tracking its history and context.
A Purpose-built Global Network: Google's Move to SDN
A discussion with Amin Vahdat, David Clark, and Jennifer Rexford
A Requirements Primer
Many software engineers and architects are exposed to compliance through the growing number of rules, regulations, and standards with which their employers must comply. Some of these requirements, such as HIPAA (Health Insurance Portabililty and Accountability Act), focus primarily on one industry, whereas others, such as SOX (Sarbanes-Oxley Act), span many industries. Some apply to only one country, while others cross national boundaries. To help navigate this often confusing world, Queue has assembled a short primer that provides background on four of the most important compliance challenges that organizations face today.
A Threat Analysis of RFID Passports
Do RFID passports make us vulnerable to identity theft?
A Time and a Place for Standards
History shows how abuses of the standards process have impeded progress. Over the next decade, we will encounter at least three major opportunities where success will hinge largely on our ability to define appropriate standards. That's because intelligently crafted standards that surface at just the right time can do much to nurture nascent industries and encourage product development simply by creating a trusted and reliable basis for interoperability. From where I stand, the three specific areas I see as particularly promising are: (1) all telecommunications and computing capabilities that work together to facilitate collaborative work; (2) hybrid computing/home entertainment products providing for the online distribution of audio and/or video content; and (3) wireless sensor and network platforms (the sort that some hope the 802.15.4 and ZigBee Alliance standards will ultimately enable).
A Tour through the Visualization Zoo
A survey of powerful visualization techniques, from the obvious to the obscure
A Tribute to Jim Gray
Computer science attracts many very smart people, but a few stand out above the others, somehow blessed with a kind of creativity that most of us are denied. Names such as Alan Turing, Edsger Dijkstra, and John Backus come to mind. Jim Gray is another.
A co-Relational Model of Data for Large Shared Data Banks
Contrary to popular belief, SQL and noSQL are really just two sides of the same coin.
AI Gets a Brain
New technology allows software to tap real human intelligence.
AI in Computer Games
If you've been following the game development scene, you've probably heard many remarks such as: "The main role of graphics in computer games will soon be over; artificial intelligence is the next big thing!" Although you should hardly buy into such statements, there is some truth in them. The quality of AI (artificial intelligence) is a high-ranking feature for game fans in making their purchase decisions and an area with incredible potential to increase players' immersion and fun.
API Design Matters
Why changing APIs might become a criminal offense. After more than 25 years as a software engineer, I still find myself underestimating the time it will take to complete a particular programming task. Sometimes, the resulting schedule slip is caused by my own shortcomings: as I dig into a problem, I simply discover that it is a lot harder than I initially thought, so the problem takes longer to solvesuch is life as a programmer. Just as often I know exactly what I want to achieve and how to achieve it, but it still takes far longer than anticipated. When that happens, it is usually because I am struggling with an API that seems to do its level best to throw rocks in my path and make my life difficult.
ASPs:
The Integration Challenge
The promise of software as a service is becoming a reality with many ASPs (application service providers). Organizations using ASPs and third-party vendors that provide value-added products to ASPs need to integrate with them. ASPs enable this integration by providing Web service-based APIs. There are significant differences between integrating with ASPs over the Internet and integrating with a local application. When integrating with ASPs, users have to consider a number of issues, including latency, unavailability, upgrades, performance, load limiting, and lack of transaction support.
Abstracting the Geniuses Away from Failure Testing
Ordinary users need tools that automate the selection of custom-tailored faults to inject.
Abstraction in Hardware System Design
Applying lessons from software languages to hardware languages using Bluespec SystemVerilog
Accountability in Algorithmic Decision-making
A view from computational journalism
Adopting DevOps Practices in Quality Assurance
Merging the art and science of software development
Advances and Challenges in Log Analysis
Logs contain a wealth of information for help in managing systems.
Agile and SEMAT - Perfect Partners
Combining agile and SEMAT yields more advantages than either one alone
Algorithms Behind Modern Storage Systems
Different uses for read-optimized B-trees and write-optimized LSM-trees
All Your Database Are Belong to Us
In the big open world of the cloud, highly available distributed objects will rule.
An Open Web Services Architecture
The name of the game is web services.
Anatomy of a Solid-state Drive
While the ubiquitous SSD shares many features with the hard-disk drive, under the surface they are completely different.
Another Day, Another Bug
We asked our readers which tools they use to squash bugs. Here's what they said.
Arm Your Applications for Bulletproof Deployment: A Conversation with Tom Spalthoff
Companies can achieve a reliable desktop environment while reducing the time and cost spent preparing high-quality application packages.
Arrogance in Business Planning
Technology business plans that assume no competition (ever)
Attack Trends:
2004 and 2005
Counterpane Internet Security Inc. monitors more than 450 networks in 35 countries, in every time zone. In 2004 we saw 523 billion network events, and our analysts investigated 648,000 security “tickets.” What follows is an overview of what’s happening on the Internet right now, and what we expect to happen in the coming months.
Automated QA Testing at EA: Driven by Events
A discussion with Michael Donat, Jafar Husain, and Terry Coatta
Automating Software Failure Reporting
We can only fix those bugs we know about.
BASE: An Acid Alternative
In partitioned databases, trading some consistency for availability can lead to dramatic improvements in scalability.
BBR: Congestion-Based Congestion Control
Measuring bottleneck bandwidth and round-trip propagation time
BPM: The Promise and the Challenge
Over the last decade, businesses and governments have been giving increasing attention to business processes - to their description, automation, and management. This interest grows out of the need to streamline business operations, consolidate organizations, and save costs, reflecting the fact that the process is the basic unit of business value within an organization.
Barbarians at the Gateways
High-frequency Trading and Exchange Technology
Best Practice (BPM)
In business process management, finding the right tool suite is just the beginning.
Best Practices on the Move: Building Web Apps for Mobile Devices
Which practices should be modified or avoided altogether by developers for the mobile Web?
Better Scripts, Better Games
The video game industry earned $8.85 billion in revenue in 2007, almost as much as movies made at the box office. Much of this revenue was generated by blockbuster titles created by large groups of people. Though large development teams are not unheard of in the software industry, game studios tend to have unique collections of developers. Software engineers make up a relatively small portion of the game development team, while the majority of the team consists of content creators such as artists, musicians, and designers.
Better, Faster, More Secure
Since I started a stint as chair of the IETF (Internet Engineering Task Force) in March 2005, I have frequently been asked, “What’s coming next?” but I have usually declined to answer. Nobody is in charge of the Internet, which is a good thing, but it makes predictions difficult (and explains why this article starts with a disclaimer: It represents my views alone and not those of my colleagues at either IBM or the IETF).
Beyond Beowulf Clusters
In the early ’90s, the Berkeley NOW (Network of Workstations) Project under David Culler posited that groups of less capable machines (running SunOS) could be used to solve scientific and other computing problems at a fraction of the cost of larger computers. In 1994, Donald Becker and Thomas Sterling worked to drive the costs even lower by adopting the then-fledgling Linux operating system to build Beowulf clusters at NASA’s Goddard Space Flight Center. By tying desktop machines together with open source tools such as PVM (Parallel Virtual Machine), MPI (Message Passing Interface), and PBS (Portable Batch System), early clusters—which were often PC towers stacked on metal shelves with a nest of wires interconnecting them—fundamentally altered the balance of scientific computing.
Beyond Instant Messaging
Platforms and standards for these services must anticipate and accommodate future developments.
Beyond Page Objects: Testing Web Applications with State Objects
Use states to drive your tests
Beyond Relational Databases
There is more to data access than SQL.
Beyond Server Consolidation
Virtualization technology was developed in the late 1960s to make more efficient use of hardware. Hardware was expensive, and there was not that much available.
Big Games, Small Screens
One thing that becomes immediately apparent when creating and distributing mobile 3D games is that there are fundamental differences between the cellphone market and the more traditional games markets, such as consoles and handheld gaming devices. The most striking of these are the number of delivery platforms; the severe constraints of the devices, including small screens whose orientation can be changed; limited input controls; the need to deal with other tasks; the nonphysical delivery mechanism; and the variations in handset performance and input capability.
Bitcoin's Academic Pedigree
The concept of cryptocurrencies is built from forgotten ideas in research literature.
Bitcoin's Underlying Incentives
The unseen economic forces that govern the Bitcoin protocol
Black Box Debugging
It's all about what takes place at the boundary of an application.
Blaster Revisited
What lessons can we learn from the carnage the Blaster worm created? The following tale is based upon actual circumstances from corporate enterprises that were faced with confronting and eradicating the Blaster worm, which hit in August 2003. The story provides views from many perspectives, illustrating the complexity and sophistication needed to combat new blended threats.
Blurring Lines Between Hardware and Software
Software development for embedded systems clearly transcends traditional "programming" and requires intimate knowledge of hardware, as well as deep understanding of the underlying application that is to be implemented.
Borg, Omega, and Kubernetes
Lessons learned from three container-management systems over a decade
Box Their SOXes Off
Data is a precious resource for any large organization. The larger the organization, the more likely it will rely to some degree on third-party vendors and partners to help it manage and monitor its mission-critical data. In the wake of new regulations for public companies, such as Section 404 of SOX (Sarbanes-Oxley Act of 2002), the folks who run IT departments for Fortune 1000 companies have an ever-increasing need to know that when it comes to the 24/7/365 monitoring of their critical data transactions, they have business partners with well-planned and well-documented procedures.
Breaking the Major Release Habit
Can agile development make your team more productive?
Bridging the Object-Relational Divide
ORM technologies can simplify data access, but be aware of the challenges that come with introducing this new layer of abstraction.
Bringing Arbitrary Compute to Authoritative Data
Many disparate use cases can be satisfied with a single storage system.
Broadcast Messaging:
Messaging to the Masses
This powerful form of communication has social implications as well as technical challenges.
Browser Security:
Lessons from Google Chrome
Google Chrome developers focused on three key problems to shield the browser from attacks.
Bufferbloat: Dark Buffers in the Internet
Networks without effective AQM may again be vulnerable to congestion collapse.
Building Collaboration into IDEs
Edit>Compile>Run>Debug>Collaborate?
Building Nutch:
Open Source Search
Search engines are as critical to Internet use as any other part of the network infrastructure, but they differ from other components in two important ways. First, their internal workings are secret, unlike, say, the workings of the DNS (domain name system). Second, they hold political and cultural power, as users increasingly rely on them to navigate online content.
Building Scalable Web Services
In the early days of the Web we severely lacked tools and frameworks, and in retrospect it seems noteworthy that those early Web services scaled at all. Nowadays, while the tools have progressed, so too have expectations with respect to richness of interaction, performance, and scalability. In view of these raised expectations it is advisable to build only what you really need, relying on other people's work where possible. Above all, be cautious in choosing when, what, and how to optimize.
Building Secure Web Applications
In these days of phishing and near-daily announcements of identity theft via large-scale data losses, it seems almost ridiculous to talk about securing the Web. At this point most people seem ready to throw up their hands at the idea or to lock down one small component that they can control in order to keep the perceived chaos at bay.
Building Systems to Be Shared, Securely
The history of computing has been characterized by continuous transformation resulting from the dramatic increases in performance and drops in price described by Moore's law. Computing "power" has migrated from centralized mainframes/servers to distributed systems and the commodity desktop. Despite these changes, system sharing remains an important tool for computing. From the multitasking, file-sharing, and virtual machines of the desktop environment to the large-scale sharing of server-class ISP hardware in collocation centers, safely sharing hardware between mutually untrusting parties requires addressing critical concerns of accidental and malicious damage.
C Is Not a Low-level Language
Your computer is not a fast PDP-11.
CPU DB: Recording Microprocessor History
With this open database, you can mine microprocessor trends over the past 40 years.
Cache Me If You Can
Building a decentralized web-delivery model
Caching XML Web Services for Mobility
In the face of unreliable connections and low bandwidth, caching may offer reliable wireless access to Web services.
Canary Analysis Service
Automated canarying quickens development, improves production safety, and helps prevent outages.
Certificate Transparency
Public, verifiable, append-only logs
Challenges of Memory Management on Modern NUMA System
Optimizing NUMA systems applications with Carrefour
Closed Source Fights Back
SCO vs. The WorldWhat Were They Thinking?
Cluster-level Logging of Containers with Containers
Logging Challenges of Container-Based Cloud Deployments
Code Spelunking:
Exploring Cavernous Code Bases
Code diving through unfamiliar source bases is something we do far more often than write new code from scratch--make sure you have the right gear for the job.
Code Spelunking Redux
It has been five years since I first wrote about code spelunking, and though systems continue to grow in size and scope, the tools we use to understand those systems are not growing at the same rate. In fact, I believe we are steadily losing ground. So why should we go over the same ground again? Is this subject important enough to warrant two articles in five years? I believe it is.
Coding Guidelines:
Finding the Art in the Science
What separates good code from great code?
Coding Smart: People vs. Tools
Tools can help developers be more productive, but they're no replacement for thinking.
Coding for the Code
Can models provide the DNA for software development?
Collaboration in System Administration
For sysadmins, solving problems usually involves collaborating with others. How can we make it more effective?
Commercializing Open Source Software
Many have tried, a few are succeeding, but challenges abound.
Communications Surveillance:
Privacy and Security at Risk
As the sophistication of wiretapping technology grows, so too do the risks it poses to our privacy and security.
Compliance Deconstructed
The topic of compliance becomes increasingly complex each year. Dozens of regulatory requirements can affect a company’s business processes. Moreover, these requirements are often vague and confusing. When those in charge of compliance are asked if their business processes are in compliance, it is understandably difficult for them to respond succinctly and with confidence. This article looks at how companies can deconstruct compliance, dealing with it in a systematic fashion and applying technology to automate compliance-related business processes. It also looks specifically at how Microsoft approaches compliance to SOX (Sarbanes-Oxley Act of 2002).
Complying with Compliance
“Hey, compliance is boring. Really, really boring. And besides, I work neither in the financial industry nor in health care. Why should I care about SOX and HIPAA?” Yep, you’re absolutely right. You write payroll applications, or operating systems, or user interfaces, or (heaven forbid) e-mail servers. Why should you worry about compliance issues?
Componentizing the Web
We may be on the cusp of a new revolution in web development.
Computers in Patient Care: The Promise and the Challenge
Information technology has the potential to radically transform health care. Why has progress been so slow?
Computing without Processors
Heterogeneous systems allow us to target our programming to the appropriate environment.
Condos and Clouds
Constraints in an environment empower the services.
Containers Will Not Fix Your Broken Culture (and Other Hard Truths)
Complex socio-technical systems are hard; film at 11.
Continuous Delivery Sounds Great, but Will It Work Here?
It's not magic, it just requires continuous, daily improvement at all levels.
Controlling Queue Delay
A modern AQM is just one piece of the solution to bufferbloat.
Cooling the Data Center
What can be done to make cooling systems in data centers more energy efficient?
Crash Consistency
Rethinking the Fundamental Abstractions of the File System
Creating Languages in Racket
Sometimes you just have to make a better mousetrap.
Criminal Code:
The Making of a Cybercriminal
NOTE: This is a fictional account of malware creators and their experiences. Although the characters are made up, the techniques and events are patterned on real activities of many different groups developing malicious software.
Culture Surprises in Remote Software Development Teams
"When in Rome" doesn't help when your team crosses time zones, and your deadline doesn't.
Cybercrime:
An Epidemic
Painted in the broadest of strokes, cybercrime essentially is the leveraging of information systems and technology to commit larceny, extortion, identity theft, fraud, and, in some cases, corporate espionage. Who are the miscreants who commit these crimes, and what are their motivations? One might imagine they are not the same individuals committing crimes in the physical world. Bank robbers and scam artists garner a certain public notoriety after only a few occurrences of their crimes, yet cybercriminals largely remain invisible and unheralded. Based on sketchy news accounts and a few public arrests, such as Mafiaboy, accused of paralyzing Amazon, CNN, and other Web sites, the public may infer these miscreants are merely a subculture of teenagers.
Cybercrime 2.0: When the Cloud Turns Dark
Web-based malware attacks are more insidious than ever. What can be done to stem the tide?
DAFS:
A New High-Performance Networked File System
This emerging file-access protocol dramatically enhances the flow of data over a network, making life easier in the data center.
DNS Complexity
DNS (domain name system) is a distributed, coherent, reliable, autonomous, hierarchical database, the first and only one of its kind. Created in the 1980s when the Internet was still young but overrunning its original system for translating host names into IP addresses, DNS is one of the foundation technologies that made the worldwide Internet (and the World Wide Web) possible. Yet this did not all happen smoothly, and DNS technology has been periodically refreshed and refined. Though it’s still possible to describe DNS in simple terms, the underlying details are by now quite sublime.
DSL for the Uninitiated
Domain-specific languages bridge the semantic gap in programming.
DSPs: Back to the Future
From the dawn of the DSP (digital signal processor), an old quote still echoes: "Oh, no! We'll have to use state-of-the-art 5µm NMOS!" The speaker's name is lost in the fog of history, as are many things from the ancient days of 5µm chip design. This quote refers to the first Bell Labs DSP whose mask set in fact underwent a 10 percent linear lithographic shrink to 4.5µm NMOS (N-channel metal oxide semiconductor) channel length and taped out in late 1979 with an aggressive full-custom circuit design.
Data Sketching
The approximate approach is often faster and more efficient.
Data in Flight
How streaming SQL technology can help solve the Web 2.0 data crunch.
Data-Parallel Computing
Users always care about performance. Although often it's just a matter of making sure the software is doing only what it should, there are many cases where it is vital to get down to the metal and leverage the fundamental characteristics of the processor.
Databases of Discovery
Open-ended database ecosystems promote new discoveries in biotech. Can they help your organization, too?
Death by UML Fever
A potentially deadly illness, clinically referred to as UML (Unified Modeling Language) fever, is plaguing many software-engineering efforts today. This fever has many different strains that vary in levels of lethality and contagion. A number of these strains are symptomatically related, however. Rigorous laboratory analysis has revealed that each is unique in origin and makeup. A particularly insidious characteristic of UML fever, common to most of its assorted strains, is the difficulty individuals and organizations have in self-diagnosing the affliction. A consequence is that many cases of the fever go untreated and often evolve into more complex and lethal strains.
Debugging Distributed Systems
Challenges and options for validation and debugging
Debugging in an Asynchronous World
Hard-to-track bugs can emerge when you can't guarantee sequential execution. The right tools and the right techniques can help.
Decentralizing SIP
If you're looking for a low-maintenance IP communications network, peer-to-peer SIP might be just the thing.
Describing the Elephant:
The Different Faces of IT as Service
In a well-known fable, a group of blind men are asked to describe an elephant. Each encounters a different part of the animal and, not surprisingly, provides a different description. We see a similar degree of confusion in the IT industry today, as terms such as service-oriented architecture, grid, utility computing, on-demand, adaptive enterprise, data center automation, and virtualization are bandied about. As when listening to the blind men, it can be difficult to know what reality lies behind the words, whether and how the different pieces fit together, and what we should be doing about the animal(s) that are being described.
Design Exploration through Code-generating DSLs
High-level DSLs for low-level programming
Designing Cluster Schedulers for Internet-Scale Services
Embracing failures for improving availability
Designing Portable Collaborative Networks
A middleware solution to keep pace with the ever-changing ways in which mobile workers collaborate.
Desktop Linux: Where Art Thou?
Linux on the desktop has come a long way - and it's been a roller-coaster ride. At the height of the dot-com boom, around the time of Red Hat's initial public offering, people expected Linux to take off on the desktop in short order. A few years later, after the stock market crash and the failure of a couple of high-profile Linux companies, pundits were quick to proclaim the stillborn death of Linux on the desktop.
DevOps Metrics
Your biggest mistake might be collecting the wrong data.
Digitally Assisted Analog Integrated Circuits
In past decades, "Moore's law" has governed the revolution in microelectronics. Through continuous advancements in device and fabrication technology, the industry has maintained exponential progress rates in transistor miniaturization and integration density. As a result, microchips have become cheaper, faster, more complex, and more power efficient.
Disambiguating Databases
Use the database built for your access model.
Discrimination in Online Ad Delivery
Google ads, black names and white names, racial discrimination, and click advertising
Disks from the Perspective of a File System
Disks lie. And the controllers that run them are partners in crime.
Dismantling the Barriers to Entry
We have to choose to build a web that is accessible to everyone.
Distributed Computing Economics
Computing economics are changing. Today there is rough price parity between: (1) one database access; (2) 10 bytes of network traffic; (3) 100,000 instructions; (4) 10 bytes of disk storage; and (5) a megabyte of disk bandwidth. This has implications for how one structures Internet-scale distributed computing: one puts computing as close to the data as possible in order to avoid expensive network traffic.
Distributed Development:
Lessons Learned
Why repeat the mistakes of the past if you don't have to?
Division of Labor in Embedded Systems
You can choose among several strategies for partitioning an embedded application over incoherent processor cores. Heres practical advice on the advantages and pitfalls of each.
Document & Media Exploitation
A computer used by Al Qaeda ends up in the hands of a Wall Street Journal reporter. A laptop from Iran is discovered that contains details of that country's nuclear weapons program. Photographs and videos are downloaded from terrorist Web sites.
Domain-specific Languages and Code Synthesis Using Haskell
Looking at embedded DSLs
Don't Settle for Eventual Consistency
Stronger properties for low-latency geo-replicated storage
Dynamics of Change: Why Reactivity Matters
Tame the dynamics of change by centralizing each concern in its own module.
E-mail Authentication:
What, Why, How?
Internet e-mail was conceived in a different world than we live in today. It was a small, tightly knit community, and we didn’t really have to worry too much about miscreants. Generally, if someone did something wrong, the problem could be dealt with through social means; “shunning” is very effective in small communities.
Energy Management on Handheld Devices
Handheld devices are becoming ubiquitous and as their capabilities increase, they are starting to displace laptop computers - much as laptop computers have displaced desktop computers in many roles. Handheld devices are evolving from today's PDAs, organizers, cellular phones, and game machines into a variety of new forms. Although partially offset by improvements in low-power electronics, this increased functionality carries a corresponding increase in energy consumption. Second, as a consequence of displacing other pieces of equipment, handheld devices are seeing more use between battery charges. Finally, battery technology is not improving at the same pace as the energy requirements of handheld electronics.
Enhanced Debugging with Traces
An essential technique used in emulator development is a useful addition to any programmer's toolbox.
Enterprise Grid Computing
I have to admit a great measure of sympathy for the IT populace at large, when it is confronted by the barrage of hype around grid technology, particularly within the enterprise. Individual vendors have attempted to plant their flags in the notionally virgin technological territory and proclaim it as their own, using terms such as grid, autonomic, self-healing, self-managing, adaptive, utility, and so forth. Analysts, well, analyze and try to make sense of it all, and in the process each independently creates his or her own map of this terra incognita, naming it policy-based computing, organic computing, and so on.
Enterprise SSDs
Solid-state drives are finally ready for the enterprise. But beware, not all SSDs are created alike.
Enterprise Search: Tough Stuff
The last decade has witnessed the growth of information retrieval from a boutique discipline in information and library science to an everyday experience for billions of people around the world. This revolution has been driven in large measure by the Internet, with vendors focused on search and navigation of Web resources and Web content management. Simultaneously, enterprises have invested in networking all of their information together to the point where it is increasingly possible for employees to have a single window into the enterprise.
Enterprise Software as Service
While the practice of outsourcing business functions such as payroll has been around for decades, its realization as online software services has only recently become popular. In the online service model, a provider develops an application and operates the servers that host it. Customers access the application over the Internet using industry-standard browsers or Web services clients. A wide range of online applications, including e-mail, human resources, business analytics, CRM (customer relationship management), and ERP (enterprise resource planning), are available.
Enterprise-Grade Wireless
We have been working in the wireless space in one form or another in excess of 10 years and have participated in every phase of its maturation process. We saw wireless progress from a toy technology before the dot-com boom, to something truly promising during the boom, only to be left wanting after the bubble when the technology was found to be not ready for prime time. Fortunately, it appears that we have finally reached the point where the technology and the enterprise's expectations have finally converged.
Erlang for Concurrent Programming
What role can programming languages play in dealing with concurrency? One answer can be found in Erlang, a language designed for concurrency from the ground up.
Error Messages:
What's the Problem?
Computer users spend a lot of time chasing down errors - following the trail of clues that starts with an error message and that sometimes leads to a solution and sometimes to frustration. Problems with error messages are particularly acute for system administrators (sysadmins) - those who configure, install, manage, and maintain the computational infrastructure of the modern world - as they spend a lot of effort to keep computers running amid errors and failures.
Eventual Consistency Today: Limitations, Extensions, and Beyond
How can applications be built on eventually consistent infrastructure given no guarantee of safety?
Eventually Consistent
At the foundation of Amazon's cloud computing are infrastructure services such as Amazon's S3 (Simple Storage Service), SimpleDB, and EC2 (Elastic Compute Cloud) that provide the resources for constructing Internet-scale computing platforms and a great variety of applications. The requirements placed on these infrastructure services are very strict; they need to score high marks in the areas of security, scalability, availability, performance, and cost effectiveness, and they need to meet these requirements while serving millions of customers around the globe, continuously.
Eventually Consistent: Not What You Were Expecting?
Methods of quantifying consistency (or lack thereof) in eventually consistent storage systems
Evolution and Practice: Low-latency Distributed Applications in Finance
The finance industry has unique demands for low-latency distributed systems.
Evolution of the Product Manager
Better education needed to develop the discipline
Exposing the ORM Cache
Familiarity with ORM caching issues can help prevent performance problems and bugs.
Extending the Semantics of Scheduling Priorities
Increasing parallelism demands new paradigms.
Extensible Programming for the 21st Century
Is an open, more flexible programming environment just around the corner?
Extreme Software Scaling
The advent of SMP (symmetric multiprocessing) added a new degree of scalability to computer systems. Rather than deriving additional performance from an incrementally faster microprocessor, an SMP system leverages multiple processors to obtain large gains in total system performance. Parallelism in software allows multiple jobs to execute concurrently on the system, increasing system throughput accordingly. Given sufficient software parallelism, these systems have proved to scale to several hundred processors.
FPGA Programming for the Masses
The programmability of FPGAs must improve if they are to be part of mainstream computing.
Fail at Scale
Reliability in the face of rapid change
Faucet: Deploying SDN in the Enterprise
Using OpenFlow and DevOps for rapid development
Fault Injection in Production
Making the case for resilience testing
Fighting Physics: A Tough Battle
Thinking of doing IPC over the long haul? Think again. The laws of physics say you're hosed.
Fighting Spam with Reputation Systems
User-submitted spam fingerprints
Finding More Than One Worm in the Apple
If you see something, say something.
Finding Usability Bugs with Automated Tests
Automated usability tests can be valuable companions to in-person tests.
Flash Disk Opportunity for Server Applications
Future flash-based disks could provide breakthroughs in IOPS, power, reliability, and volumetric capacity when compared with conventional disks.
Flash Storage Today
Can flash memory become the foundation for a new tier in the storage hierarchy?
Four Billion Little Brothers?:
Privacy, mobile phones, and ubiquitous data collection
Participatory sensing technologies could improve our lives and our communities, but at what cost to our privacy?
From COM to Common
Ten years ago, the term component software meant something relatively specific and concrete. A small number of software component frameworks more or less defined the concept for most people. Today, few terms in the software industry are less precise than component software. There are now many different forms of software componentry for many different purposes. The technologies and methodologies of 10 years ago have evolved in fundamental ways and have been joined by an explosion of new technologies and approaches that have redefined our previously held notions of component software.
From IR to Search, and Beyond
Searching has come a long way since the 60s, but have we only just begun?
From Liability to Advantage: A Conversation with John Graham-Cumming and John Ousterhout
Software production has become a bottleneck in many development organizations.
From Server Room to Living Room
How open source and TiVo became a perfect match
From the EDVAC to WEBVACs
Cloud computing for computer scientists
Fun and Games:
Multi-Language Development
Computer games (or "electronic games" if you encompass those games played on console-class hardware) comprise one of the fastest-growing application markets in the world. Within the development community that creates these entertaining marvels, multi-language development is becoming more commonplace as games become more and more complex. Today, asking a development team to construct a database-enabled Web site with the requirement that it be written entirely in C++ would earn scornful looks and rolled eyes, but not long ago the idea that multiple languages were needed to accomplish a given task was scoffed at.
Functional at Scale
Applying functional programming principles to distributed computing projects
Future Graphics Architectures
Graphics architectures are in the midst of a major transition. In the past, these were specialized architectures designed to support a single rendering algorithm: the standard Z buffer. Realtime 3D graphics has now advanced to the point where the Z-buffer algorithm has serious shortcomings for generating the next generation of higher-quality visual effects demanded by games and other interactive 3D applications. There is also a desire to use the high computational capability of graphics architectures to support collision detection, approximate physics simulations, scene management, and simple artificial intelligence.
Fuzzy Boundaries:
Objects, Components, and Web Services
It's easy to transform objects into components and Web services, but how do we know which is right for the job?
GPUs: A Closer Look
A gamer wanders through a virtual world rendered in near- cinematic detail. Seconds later, the screen fills with a 3D explosion, the result of unseen enemies hiding in physically accurate shadows. Disappointed, the user exits the game and returns to a computer desktop that exhibits the stylish 3D look-and-feel of a modern window manager. Both of these visual experiences require hundreds of gigaflops of computing performance, a demand met by the GPU (graphics processing unit) present in every consumer PC.
Game Development:
Harder Than You Think
The hardest part of making a game has always been the engineering. In times past, game engineering was mainly about low-level optimization - writing code that would run quickly on the target computer, leveraging clever little tricks whenever possible.
Gaming Graphics:
The Road to Revolution
It has been a long journey from the days of multicolored sprites on tiled block backgrounds to the immersive 3D environments of modern games. What used to be a job for a single game creator is now a multifaceted production involving staff from every creative discipline. The next generation of console and home computer hardware is going to bring a revolutionary leap in available computing power; a teraflop (trillion floating-point operations per second) or more will be on tap from commodity hardware.
Getting Bigger Reach Through Speech
Mark Ericson, vice president of product strategy for BlueNote Networks argues that in order to take advantage of new voice technologies you have to have a plan for integrating that capability directly into the applications that drive your existing business processes.
Getting Gigascale Chips:
Challenges and Opportunities in Continuing Moore's Law
Processor performance has increased by five orders of magnitude in the last three decades, made possible by following Moore's law - that is, continued technology scaling, improved transistor performance to increase frequency, additional (to avoid repetition) integration capacity to realize complex architectures, and reduced energy consumed per logic operation to keep power dissipation within limits. Advances in software technology, such as rich multimedia applications and runtime systems, exploited this performance explosion, delivering to end users higher productivity, seamless Internet connectivity, and even multimedia and entertainment.
Getting What You Measure
Four common pitfalls in using software metrics for project management
Go Static or Go Home
In the end, dynamic systems are simply less secure.
Going with the Flow
Workflow systems can provide value beyond automating business processes.
Hadoop Superlinear Scalability
The perpetual motion of parallel performance
Hard Disk Drives:
The Good, the Bad and the Ugly!
HDDs (hard-disk drives) are like the bread in a peanut butter and jelly sandwich—sort of an unexciting piece of hardware necessary to hold the “software.” They are simply a means to an end. HDD reliability, however, has always been a significant weak link, perhaps the weak link, in data storage. In the late 1980s people recognized that HDD reliability was inadequate for large data storage systems so redundancy was added at the system level with some brilliant software algorithms, and RAID (redundant array of inexpensive disks) became a reality. RAID moved the reliability requirements from the HDD itself to the system of data disks.
Hazy: Making it Easier to Build and Maintain Big-data Analytics
Racing to unleash the full potential of big data with the latest statistical and machine-learning techniques.
Heterogeneous Computing: Here to Stay
Hardware and Software Perspectives
Hidden in Plain Sight
Improvements in the observability of software can help you diagnose your most crippling performance problems.
High Performance Web Sites
Google Maps, Yahoo! Mail, Facebook, MySpace, YouTube, and Amazon are examples of Web sites built to scale. They access petabytes of data sending terabits per second to millions of users worldwide. The magnitude is awe-inspiring. Users view these large-scale Web sites from a narrower perspective. The typical user has megabytes of data that are downloaded at a few hundred kilobits per second. Users are not so interested in the massive number of requests per second being served; they care more about their individual requests. As they use these Web applications, they inevitably ask the same question: "Why is this site so slow?"
How Do I Model State? Let Me Count the Ways
A study of the technology and sociology of Web services specifications
How Fast is Your Web Site?
Web site performance data has never been more readily available.
How Not to Write Fortran in Any Language
There are characteristics of good coding that transcend all programming languages.
How OSGi Changed My Life
In the early 1980s I discovered OOP (object-oriented programming) and fell in love with it, head over heels. As usual, this kind of love meant convincing management to invest in this new technology, and most important of all, send me to cool conferences. So I pitched the technology to my manager. I sketched him the rosy future, how one day we would create applications from ready-made classes. We would get those classes from a repository, put them together, and voila, a new application would be born.
How Will Astronomy Archives Survive the Data Tsunami?
Astronomers are collecting more data than ever. What practices can keep them ahead of the flood?
How to De-identify Your Data
Balancing statistical accuracy and subject privacy in large social-science data sets
I/O Virtualization
Decoupling a logical device from its physical implementation offers many compelling advantages.
Idempotence Is Not a Medical Condition
An essential property for reliable systems
Idle-Time Garbage-Collection Scheduling
Taking advantage of idleness to reduce dropped frames and memory consumption
If You Have Too Much Data, then "Good Enough" Is Good Enough
In today's humongous database systems, clarity may be relaxed, but business needs can still be met.
Immutability Changes Everything
We need it, we can afford it, and the time is now.
Improving Performance on the Internet
When it comes to achieving performance, reliability, and scalability for commercial-grade Web applications, where is the biggest bottleneck? In many cases today, we see that the limiting bottleneck is the middle mile, or the time data spends traveling back and forth across the Internet, between origin server and end user.
Industrial Scale Agile - from Craft to Engineering
Essence is instrumental in moving software development toward a true engineering discipline.
Information Extraction:
Distilling Structured Data from Unstructured Text
Distilling structured data from unstructured text
Injecting Errors for Fun and Profit
Error-detection and correction features are only as good as our ability to test them.
Instant Messaging or Instant Headache?
It's a reality. You have IM (instant messaging) clients in your environment. You have already recognized that it is eating up more and more of your network bandwidth and with Microsoft building IM capability into its XP operating system and applications, you know this will only get worse. Management is also voicing concerns over the lost user productivity caused by personal conversations over this medium. You have tried blocking these conduits for conversation, but it is a constant battle.
Integrating RFID
RFID (radio frequency identification) has received a great deal of attention in the commercial world over the past couple of years. The excitement stems from a confluence of events. First, through the efforts of the former Auto-ID Center and its sponsor companies, the prospects of low-cost RFID tags and a networked supply chain have come within reach of a number of companies. Second, several commercial companies and government bodies, such as Wal-Mart and Target in the United States, Tesco in Europe, and the U.S. Department of Defense, have announced RFID initiatives in response to technology improvements.
Intellectual Property and Software Piracy:
The Power of IP Protection and Software Licensing, an interview with Aladdin vice president Gregg Gronowski
The Power of IP Protection and Software Licensing, an interview with Aladdin vice president Gregg Gronowski
Interactive Dynamics for Visual Analysis
A taxonomy of tools that support the fluent and flexible use of visualizations
Intermediate Representation
The increasing significance of intermediate representations in compilers
Internal Access Controls
Trust, but Verify
Is Open Source Right for You?:
A Fictional Case Study of Open Source in a Commercial Software Shop
The media often present open source software as a direct competitor to commercial software. This depiction, usually pitting David (Linux) against Goliath (Microsoft), makes for fun reading in the weekend paper. However, it mostly misses the point of what open source means to a development organization. In this article, I use the experiences of GizmoSoft (a fictitious software company) to present some perspectives on the impact of open source software usage in a software development shop.
Is There a Single Method for the Internet of Things?
Essence can keep software development for the IoT from becoming unwieldy.
It Probably Works
Probabilistic algorithms are all around us--not only are they acceptable, but some programmers actually seek out chances to use them.
Java Security Architecture Revisited
Hard technical problems and tough business challenges
Java in a Teacup
Programming Bluetooth-enabled devices using J2ME
JavaScript and the Netflix User Interface
Conditional dependency resolution
Keeping Bits Safe:
How Hard Can It Be?
As storage systems grow larger and larger, protecting their data for long-term storage is becoming more and more challenging.
Keeping Score in the IT Compliance Game
Achieving developer acceptance of standardized procedures for managing applications from development to release is one of the largest hurdles facing organizations today. Establishing a standardized development-to-release workflow, often referred to as the ALM (application lifecycle management) process, is particularly critical for organizations in their efforts to meet tough IT compliance mandates. This is much easier said than done, as different development teams have created their own unique procedures that are undocumented, unclear, and nontraceable.
Lack of Priority Queuing Considered Harmful
Most modern routers consist of several line cards that perform packet lookup and forwarding, all controlled by a control plane that acts as the brain of the router, performing essential tasks such as management functions, error reporting, control functions including route calculations, and adjacency maintenance. This control plane has many names; in this article it is the route processor, or RP. The route processor calculates the forwarding table and downloads it to the line cards using a control-plane bus. The line cards perform the actual packet lookup and forwarding.
Languages, Levels, Libraries, and Longevity
New programming languages are born every day. Why do some succeed and some fail? In 50 years, we've already seen numerous programming systems come and (mostly) go, although some have remained a long time and will probably do so for: decades? centuries? millennia? The questions about language designs, levels of abstraction, libraries, and resulting longevity are numerous. Why do new languages arise? Why is it sometimes easier to write new software than to adapt old software that works? How many different levels of languages make sense? Why do some languages last in the face of "better" ones?
Leaking Space
Eliminating memory hogs
Learning from the Web
In the past decade we have seen a revolution in computing that transcends anything seen to date in terms of scope and reach, but also in terms of how we think about what makes up “good” and “bad” computing. The Web taught us several unintuitive lessons:
Lessons from the Floor
The manufacturing industry can teach us a lot about measuring performance in large-scale Internet services.
Lessons from the Letter
Security flaws in a large organization
Leveraging Application Frameworks
In today's competitive, fast-paced computing industry, successful software must increasingly be: (1) extensible to support successions of quick updates and additions to address new requirements and take advantage of emerging markets; (2) flexible to support a growing range of multimedia data types, traffic flows, and end-to-end QoS (quality of service) requirements; (3) portable to reduce the effort required to support applications on heterogeneous operating-system platforms and compilers; (4) reliable to ensure that applications are robust and tolerant to faults; (5) scalable to enable applications to handle larger numbers of clients simultaneously; and (6) affordable to ensure that the total ownership costs of software acquisition and evolution are not prohibitively high.
Life Beyond Distributed Transactions
An apostate's opinion
META II: Digital Vellum in the Digital Scriptorium
Revisiting Schorre's 1962 compiler-compiler
Major-league SEMAT: Why Should an Executive Care?
Becoming better, faster, cheaper, and happier
Making Money Using Math
Modern applications are increasingly using probabilistic machine-learned models.
Making SIP Make Cents
P2P payments using SIP could enable new classes of micropayment applications and business models.
Making Sense of Revision-control Systems
Whether distributed or centralized, all revision-control systems come with complicated sets of tradeoffs. How do you find the best match between tool and team?
Making a Case for Efficient Supercomputing
A supercomputer evokes images of "big iron" and speed; it is the Formula 1 racecar of computing. As we venture forth into the new millennium, however, I argue that efficiency, reliability, and availability will become the dominant issues by the end of this decade, not only for supercomputing, but also for computing in general.
Making the Mobile Web Faster
Mobile performance issues? Fix the back end, not just the client.
Making the Web Faster with HTTP 2.0
HTTP continues to evolve
Managing Collaboration
Jeff Johnstone of TechExcel explains why there is a need for a new approach to application lifecycle management that better reflects the business requirements and challenges facing development teams.
Managing Contention for Shared Resources on Multicore Processors
Contention for caches, memory controllers, and interconnects can be alleviated by contention-aware scheduling algorithms.
Managing Semi-Structured Data
I vividly remember during my first college class my fascination with the relational database—an information oasis that guaranteed a constant flow of correct, complete, and consistent information at our disposal. In that class I learned how to build a schema for my information, and I learned that to obtain an accurate schema there must be a priori knowledge of the structure and properties of the information to be modeled.
Managing Technical Debt
Shortcuts that save money and time today can cost you down the road.
Massively Multiplayer Middleware
Wish is a multiplayer, online, fantasy role-playing game being developed by Mutable Realms. It differs from similar online games in that it allows tens of thousands of players to participate in a single game world (instead of the few hundred players supported by other games). Allowing such a large number of players requires distributing the processing load over a number of machines and raises the problem of choosing an appropriate distribution technology.
Maximizing Power Efficiency with Asymmetric Multicore Systems
Asymmetric multicore systems promise to use a lot less energy than conventional symmetric processors. How can we develop software that makes the most out of this potential?
Meet the Virts
When you dig into the details of supposedly overnight success stories, you frequently discover that they've actually been years in the making. Virtualization has been around for more than 30 years since the days when some of you were feeding stacks of punch cards into very physical machines yet in 2007 it tipped. VMware was the IPO sensation of the year; in November 2007 no fewer than four major operating system vendors (Microsoft, Oracle, Red Hat, and Sun) announced significant new virtualization capabilities; and among fashionable technologists it seems virtual has become the new black.
Metamorphosis: the Coming Transformation of Translational Systems Biology
In the future computers will mine patient data to deliver faster, cheaper healthcare, but how will we design them to give informative causal explanations? Ideas from philosophy, model checking, and statistical testing can pave the way for the needed translational systems biology.
Metaphors We Compute By
Code is a story that explains how to solve a particular problem.
Mind Your State for Your State of Mind
The interactions between storage and applications can be complex and subtle.
Mobile Application Development: Web vs. Native
Web apps are cheaper to develop and deploy than native apps, but can they match the native user experience?
Mobile Media:
Making It a Reality
Many future mobile applications are predicated on the existence of rich, interactive media services. The promise and challenge of such services is to provide applications under the most hostile conditions - and at low cost to a user community that has high expectations. Context-aware services require information about who, where, when, and what a user is doing and must be delivered in a timely manner with minimum latency. This article reveals some of the current state-of-the-art "magic" and the research challenges.
Model-based Testing: Where Does It Stand?
MBT has positive effects on efficiency and effectiveness, even if it only partially fulfills high expectations.
Modeling People and Places with Internet Photo Collections
Understanding the world from the sea of online photos
Modern Performance Monitoring
Today's diverse and decentralized computer world demands new thinking about performance monitoring and analysis.
Modern System Power Management
The Advanced Configuration and Power Interface (ACPI) is the most widely used power and configuration interface for laptops, desktops, and server systems. It is also very complex, and its current specification weighs in at more than 500 pages. Needless to say, operating systems that choose to support ACPI require significant additional software support, up to and including fundamental OS architecture changes. The effort that ACPI's definition and implementation has entailed is worth the trouble because of how much flexibility it gives to the OS (and ultimately the user) to control power management policy and implementation.
MongoDB's JavaScript Fuzzer
The fuzzer is for those edge cases that your testing didn't catch.
Monitoring and Control of Large Systems with MonALISA
MonALISA developers describe how it works, the key design principles behind it, and the biggest technical challenges in building it.
Monitoring in a DevOps World
Perfect should never be the enemy of better.
Monitoring, at Your Service
Automated monitoring can increase the reliability and scalability of today's online software services.
Multipath TCP
Decoupled from IP, TCP is at last able to support multihomed hosts.
Multitier Programming in Hop
A first step toward programming 21st-century applications
NUMA (Non-Uniform Memory Access): An Overview
NUMA becomes more common because memory controllers get close to execution units on microprocessors.
National Internet Defense - Small States on the Skirmish Line
Attacks in Estonia and Georgia highlight key vulnerabilities in national Internet infrastructure.
Natural Language Translation at the Intersection of AI and HCI
Old questions being answered with both AI and HCI
Network Applications Are Interactive
The network era requires new models, with interactions instead of algorithms.
Network Forensics
The dictionary defines forensics as "the use of science and technology to investigate and establish facts in criminal or civil courts of law." I am more interested, however, in the usage common in the computer world: using evidence remaining after an attack on a computer to determine how the attack was carried out and what the attacker did.
Network Front-end Processors, Yet Again
The history of NFE processors sheds light on the tradeoffs involved in designing network stack software.
Network Virtualization:
Breaking the Performance Barrier
The recent resurgence in popularity of virtualization has led to its use in a growing number of contexts, many of which require high-performance networking. Consider server consolidation, for example. The efficiency of network virtualization directly impacts the number of network servers that can effectively be consolidated onto a single physical machine. Unfortunately, modern network virtualization techniques incur significant overhead, which limits the achievable network performance. We need new network virtualization techniques to realize the full benefits of virtualization in network-intensive domains.
Nine IM Accounts and Counting
The key word with instant messaging today is interoperability. Various standards are in contention.
No Source Code? No Problem!
What if you have to port a program, but all you have is a binary?
Non-volatile Storage
Implications of the Datacenter's Shifting Center
Nonblocking Algorithms and Scalable Multicore Programming
Exploring some alternatives to lock-based synchronization
Not Your Father's PBX?
Perhaps no piece of office equipment is more taken for granted than the common business telephone. The technology behind this basic communication device, however, is in the midst of a major transformation. Businesses are now converging their voice and data networks in order to simplify their network operations and take advantage of the new functional benefits and capabilities that a converged network delivers from greater productivity and cost savings to enhanced mobility.
OCaml for the Masses
Why the next language you learn should be functional
ORM in Dynamic Languages
O/R mapping frameworks for dynamic languages such as Groovy provide a different flavor of ORM that can greatly simplify application code.
Ode to a Sailor
sailor, fleeting mood image of you; all sailor in bear grace, rough hands and poetic dream;
Of Processors and Processing
Digital signal processing is a stealth technology. It is the core enabling technology in everything from your cellphone to the Mars Rover. It goes much further than just enabling a one-time breakthrough product. It provides ever-increasing capability; compare the performance gains made by dial-up modems with the recent performance gains of DSL and cable modems. Remarkably, digital signal processing has become ubiquitous with little fanfare, and most of its users are not even aware of what it is.
On Mapping Alogrithms to DSP Architectures
Our complex world is characterized by representation, transmission, and storage of information - and information is mostly processed in digital form. With the advent of DSPs (digital signal processors), engineers are able to implement complex algorithms with relative ease. Today we find DSPs all around us - in cars, digital cameras, MP3 and DVD players, modems, and so forth. Their widespread use and deployment in complex systems has triggered a revolution in DSP architectures, which in turn has enabled engineers to implement algorithms of ever-increasing complexity.
On Plug-ins and Extensible Architectures
Extensible application architectures such as Eclipse offer many advantages, but one must be careful to avoid "plug-in hell."
One Step Ahead
Security vulnerabilities abound, but a few simple steps can minimize your risk.
Online Algorithms in High-frequency Trading
The challenges faced by competing HFT algorithms
Oops! Coping with Human Error in IT Systems
Errors Happen. How to Deal.
Open Source to the Core
The open source development model is not exactly new. Individual engineers have been using open source as a collaborative development methodology for decades. Now that it has come to the attention of upper and middle management, however, it's finally being openly acknowledged as a commercial engineering force-multiplier and important option for avoiding significant software development costs.
Open Spectrum:
A Path to Ubiquitous Connectivity
Path to Ubiquitous Connectivity
Open vs. Closed:
Which Source is More Secure?
Which source is more secure?
OpenFlow: A Radical New Idea in Networking
An open standard that enables software-defined networking
Orchestrating an Automated Test Lab
Networking and the Internet are encouraging increasing levels of interaction and collaboration between people and their software. Whether users are playing games or composing legal documents, their applications need to manage the complex interleaving of actions from multiple machines over potentially unreliable connections. As an example, Silicon Chalk is a distributed application designed to enhance the in-class experience of instructors and students. Its distributed nature requires that we test with multiple machines. Manual testing is too tedious, expensive, and inconsistent to be effective. While automating our testing, however, we have found it very labor intensive to maintain a set of scripts describing each machine's portion of a given test.
Order from Chaos
There is probably little argument that the past decade has brought the “big bang” in the amount of online information available for processing by humans and machines. Two of the trends that it spurred (among many others) are: first, there has been a move to more flexible and fluid (semi-structured) models than the traditional centralized relational databases that stored most of the electronic data before; second, today there is simply too much information available to be processed by humans, and we really need help from machines.
Other People's Data
Companies have access to more types of external data than ever before. How can they integrate it most effectively?
Outsourcing: Devising a Game Plan
Your CIO just summoned you to duty by handing off the decision-making power about whether to outsource next years big development project to rewrite the internal billing system. That's quite a daunting task! How can you possibly begin to decide if outsourcing is the right option for your company? There are a few strategies that you can follow to help you avoid the pitfalls of outsourcing and make informed decisions. Outsourcing is not exclusively a technical issue, but it is a decision that architects or development managers are often best qualified to make because they are in the best position to know what technologies make sense to keep in-house.
Parallel Processing with Promises
A simple method of writing a collaborative system
Parallel Programming with Transactional Memory
While sometimes even writing regular, single-threaded programs can be quite challenging, trying to split a program into multiple pieces that can be executed in parallel adds a whole dimension of additional problems. Drawing upon the transaction concept familiar to most programmers, transactional memory was designed to solve some of these problems and make parallel programming easier. Ulrich Drepper from Red Hat shows us how it's done.
Passing a Language through the Eye of a Needle
How the embeddability of Lua impacted its design
Passively Measuring TCP Round-trip Times
A close look at RTT measurements with TCP
Patching the Enterprise
Organizations of all sizes are spending considerable efforts on getting patch management right - their businesses depend on it.
People and Process
Minimizing the pain of business process change
People in Our Software
People are not well represented in today's software. With the exception of IM (instant messaging) clients, today's applications offer few clues that people are actually living beings. Static strings depict things associated with people like e-mail addresses, phone numbers, and home-page URLs. Applications also tend to show the same information about a person, no matter who is viewing it.
Perfect Storm:
The Insider, Naivety, and Hostility
Every year corporations and government installations spend millions of dollars fortifying their network infrastructures. Firewalls, intrusion detection systems, and antivirus products stand guard at network boundaries, and individuals monitor countless logs and sensors for even the subtlest hints of network penetration. Vendors and IT managers have focused on keeping the wily hacker outside the network perimeter, but very few technological measures exist to guard against insiders - those entities that operate inside the fortified network boundary. The 2002 CSI/FBI survey estimates that 70 percent of successful attacks come from the inside. Several other estimates place those numbers even higher.
Performance Anti-Patterns
Want your apps to run faster? Heres what not to do.
Pervasive, Dynamic Authentication of Physical Items
The use of silicon PUF circuits
Phishing Forbidden
Phishing is a significant risk facing Internet users today.1,2 Through e-mails or instant messages, users are led to counterfeit Web sites designed to trick them into divulging usernames, passwords, account numbers, and personal information. It is up to the user to ensure the authenticity of the Web site.
Playing for Keeps
Inflection points come at you without warning and quickly recede out of reach. We may be nearing one now. If so, we are now about to play for keeps, and “we” doesn’t mean just us security geeks. If anything, it’s because we security geeks have not worked the necessary miracles already that an inflection point seems to be approaching at high velocity.
Postmortem Debugging in Dynamic Environments
Modern dynamic languages lack tools for understanding software failures.
Power-Efficient Software
Power-manageable hardware can help save energy, but what can software developers do to address the problem?
Powering Down
Power management - from laptops to rooms full of servers - is a topic of interest to everyone. In the beginning there was the desktop computer. It ran at a fixed speed and consumed less power than the monitor it was plugged into. Where computers were portable, their sheer size and weight meant that you were more likely to be limited by physical strength than battery life. It was not a great time for power management.
Principles of Robust Timing over the Internet
The key to synchronizing clocks over networks is taming delay variability.
Privacy, Anonymity, and Big Data in the Social Sciences
Quality social science research and the privacy of human subjects requires trust.
Probing Biomolecular Machines with Graphics Processors
The evolution of GPU processors and programming tools is making advanced simulation and analysis techniques accessible to a growing community of biomedical scientists.
Productivity in Parallel Programming: A Decade of Progress
Looking at the design and benefits of X10
Programmers Are People, too
I would like to start out this article with an odd, yet surprisingly uncontroversial assertion, which is this: programmers are human. I wish to use this as a premise to explore how to improve the programmer’s lot. So, please, no matter your opinion on the subject, grant me this assumption for the sake of argument.
Programming Without a Net
Embedded systems programming presents special challenges to engineers unfamiliar with that environment.
Provenance in Sensor Data Management
A cohesive, independent solution for bringing provenance to scientific research
Proving the Correctness of Nonblocking Data Structures
So you've decided to use a nonblocking data structure, and now you need to be certain of its correctness. How can this be achieved? When a multithreaded program is too slow because of a frequently acquired mutex, the programmer's typical reaction is to question whether this mutual exclusion is indeed required. This doubt becomes even more pronounced if the mutex protects accesses to only a single variable performed using a single instruction at every site. Removing synchronization improves performance, but can it be done without impairing program correctness?
Purpose-Built Languages
While often breaking the rules of traditional language design, the growing ecosystem of purpose-built "little" languages is an essential part of systems development.
Putting It All Together
Component integration is one of the tough challenges in embedded system design. Designers search for conservative design styles and reliable techniques for interfacing and verification.
Quality Assurance:
Much More than Testing
Quality assurance isn't just testing, or analysis, or wishful thinking. Although it can be boring, difficult, and tedious, QA is nonetheless essential.
Rate-limiting State
The edge of the Internet is an unruly place
Reading, Writing, and Code
Forty years ago, when computer programming was an individual experience, the need for easily readable code wasn't on any priority list. Today, however, programming usually is a team-based activity, and writing code that others can easily decipher has become a necessity. Creating and developing readable code is not as easy as it sounds.
Real-World Concurrency
In this look at how concurrency affects practitioners in the real world, Cantrill and Bonwick argue that much of the anxiety over concurrency is unwarranted.
Realtime Computer Vision with OpenCV
Mobile computer-vision technology will soon become as ubiquitous as touch interfaces.
Realtime GPU Audio
Finite difference-based sound synthesis using graphics processors
Realtime Garbage Collection
It's now possible to develop realtime systems using Java.
Reconfigurable Future
The Ability to Produce Cheaper, More Compact Chips is a Double-edged Sword.
Reliable Cron across the Planet
...or How I stopped worrying and learned to love time
Rethinking Passwords
Our authentication system is lacking. Is improvement possible?
Returning Control to the Programmer:
SIMD Intrinsics for Virtual Machines
Exposing SIMD units within interpreted languages could simplify programs and unleash floods of untapped processor power.
Revisiting Network I/O APIs: The netmap Framework
It is possible to achieve huge performance improvements in the way packet processing is done on modern operating systems.
Rules for Mobile Performance Optimization
An overview of techniques to speed page loading
SAGE: Whitebox Fuzzing for Security Testing
SAGE has had a remarkable impact at Microsoft.
SIP:
Basics and Beyond
More than just a simple telephony application protocol, SIP is a framework for developing communications systems.
Scalability Techniques for Practical Synchronization Primitives
Designing locking primitives with performance in mind
Scalable Parallel Programming with CUDA
The advent of multicore CPUs and manycore GPUs means that mainstream processor chips are now parallel systems. Furthermore, their parallelism continues to scale with Moore's law. The challenge is to develop mainstream application software that transparently scales its parallelism to leverage the increasing number of processor cores, much as 3D graphics applications transparently scale their parallelism to manycore GPUs with widely varying numbers of cores.
Scalable SQL
How do large-scale sites and applications remain SQL-based?
Scaling Existing Lock-based Applications with Lock Elision
Lock elision enables existing lock-based programs to achieve the performance benefits of nonblocking synchronization and fine-grain locking with minor software engineering effort.
Scaling Synchronization in Multicore Programs
Advanced synchronization methods can boost the performance of multicore software.
Scaling in Games & Virtual Worlds
I used to be a systems programmer, working on infrastructure used by banks, telecom companies, and other engineers. I worked on operating systems. I worked on distributed middleware. I worked on programming languages. I wrote tools. I did all of the things that hard-core systems programmers do.
Schema.org: Evolution of Structured Data on the Web
Big data makes common schemas even more necessary.
Search Considered Integral
A combination of tagging, categorization, and navigation can help end-users leverage the power of enterprise search.
Searching vs. Finding
Finding information and organizing it so that it can be found are two key aspects of any company's knowledge management strategy. Nearly everyone is familiar with the experience of searching with a Web search engine and using a search interface to search a particular Web site once you get there. (You may have even noticed that the latter often doesn't work as well as the former.) After you have a list of hits, you typically spend a significant amount of time following links, waiting for pages to download, reading through a page to see if it has what you want, deciding that it doesn't, backing up to try another link, deciding to try another way to phrase your request, et cetera.
Securing Elasticity in the Cloud
Elastic computing has great potential, but many security challenges remain.
Securing the Network Time Protocol
Crackers discover how to use NTP as a weapon for abuse.
Securing the Tangled Web
Preventing script injection vulnerabilities through software design
Security - Problem Solved?
There are plenty of security problems that have solutions. Yet, our security problems don’t seem to be going away. What’s wrong here? Are consumers being offered snake oil and rejecting it? Are they not adopting solutions they should be adopting? Or, is there something else at work, entirely? We’ll look at a few places where the world could easily be a better place, but isn’t, and build some insight as to why.
Security Collapse in the HTTPS Market
Assessing legal and technical solutions to secure HTTPS
Security in the Browser
Web browsers leave users vulnerable to an ever-growing number of attacks. Can we make them secure while preserving their usability?
Security is Harder than You Think
Many developers see buffer overflows as the biggest security threat to software and believe that there is a simple two-step process to secure software: switch from C or C++ to Java, then start using SSL (Secure Sockets Layer) to protect data communications. It turns out that this naïve tactic isn't sufficient. In this article, we explore why software security is harder than people expect, focusing on the example of SSL.
Security: The Root of the Problem
Security bug? My programming language made me do it! It doesn't seem that a day goes by without someone announcing a critical flaw in some crucial piece of software or other. Is software that bad? Are programmers so inept? What the heck is going on, and why is the problem getting worse instead of better? One distressing aspect of software security is that we fundamentally don't seem to "get it."
Self-Healing Networks
Wireless networks that fix their own broken communication links may speed up their widespread acceptance.
Self-Healing in Modern Operating Systems
A few early steps show there's a long (and bumpy) road ahead.
Sender-side Buffers and the Case for Multimedia Adaptation
A proposal to improve the performance and availability of streaming video and other time-sensitive media
Sensible Authentication
The problem with securing assets and their functionality is that, by definition, you don't want to protect them from everybody. It makes no sense to protect assets from their owners, or from other authorized individuals (including the trusted personnel who maintain the security system). In effect, then, all security systems need to allow people in, even as they keep people out. Designing a security system that accurately identifies, authenticates, and authorizes trusted individuals is highly complex and filled with nuance, but critical to security.
Sentient Data Access via a Diverse Society of Devices
Today's ubiquitous computing environment cannot benefit from the traditional understanding of a hierarchical file system.
Should You Upload or Ship Big Data to the Cloud?
The accepted wisdom does not always hold true.
Sifting Through the Software Sandbox:
SCM Meets QA
Thanks to modern SCM (software configuration management) systems, when developers work on a codeline they leave behind a trail of clues that can reveal what parts of the code have been modified, when, how, and by whom. From the perspective of QA (quality assurance) and test engineers, is this all just "data," or is there useful information that can improve the test coverage and overall quality of a product?
Simplicity Betrayed
Emulating a video system shows how even a simple interface can be more complex—and capable—than it appears.
Simulators:
Virtual Machines of the Past (and Future)
Simulators are a form of "virtual machine" intended to address a simple problem: the absence of real hardware. Simulators for past systems address the loss of real hardware and preserve the usability of software after real hardware has vanished. Simulators for future systems address the variability of future hardware designs and facilitate the development of software before real hardware exists.
Sink or Swim:
Know When It's Time to Bail
A diagnostic to help you measure organizational dysfunction and take action
SoC: Software, Hardware, Nightmare, Bliss
System-on-a-chip design offers great promise by shrinking an entire computer to a single chip. But with the promise come challenges that need to be overcome before SoC reaches its full potential.
Social Bookmarking in the Enterprise
Can your organization benefit from social bookmarking tools?
Social Perception
Modeling human interaction for the next generation of communication services
Software Development with Code Maps
Could those ubiquitous hand-drawn code diagrams become a thing of the past?
Software Needs Seatbelts and Airbags
Finding and fixing bugs in deployed software is difficult and time-consuming. Here are some alternatives.
Software Transactional Memory: Why Is It Only a Research Toy?
The promise of STM may likely be undermined by its overheads and workload applicabilities.
Software and the Concurrency Revolution
Leveraging the full power of multicore processors demands new tools and new thinking from the software industry. Concurrency has long been touted as the "next big thing" and "the way of the future," but for the past 30 years, mainstream software development has been able to ignore it. Our parallel future has finally arrived: new machines will be parallel machines, and this will require major changes in the way we develop software. The introductory article in this issue ("The Future of Microprocessors" by Kunle Olukotun and Lance Hammond) describes the hardware imperatives behind this shift in computer architecture from uniprocessors to multicore processors, also known as CMPs (chip multiprocessors).
Spam, Spam, Spam, Spam, Spam, the FTC, and Spam
A forum sponsored by the FTC highlights just how bad spam is, and how it's only going to get worse without some intervention.
Spicing Up Dart with Side Effects
A set of extensions to the Dart programming language, designed to support asynchrony and generator functions
Splinternet Behind the Great Firewall of China
Once China opened its door to the world, it could not close it again.
Standardizing Storage Clusters
Data-intensive applications such as data mining, movie animation, oil and gas exploration, and weather modeling generate and process huge amounts of data. File-data access throughput is critical for good performance. To scale well, these HPC (high-performance computing) applications distribute their computation among numerous client machines. HPC clusters can range from hundreds to thousands of clients with aggregate I/O demands ranging into the tens of gigabytes per second.
Statistics for Engineers
Applying statistical techniques to operations data
Storage Systems:
Not Just a Bunch of Disks Anymore
The sheer size and scope of data available today puts tremendous pressure on storage systems to perform in ways never imagined.
Storage Virtualization Gets Smart
Over the past 20 years we have seen the transformation of storage from a dumb resource with fixed reliability, performance, and capacity to a much smarter resource that can actually play a role in how data is managed. In spite of the increasing capabilities of storage systems, however, traditional storage management models have made it hard to leverage these data management capabilities effectively. The net result has been overprovisioning and underutilization. In short, although the promise was that smart shared storage would simplify data management, the reality has been different.
Stream Processors: Progammability and Efficiency
Many signal processing applications require both efficiency and programmability. Baseband signal processing in 3G cellular base stations, for example, requires hundreds of GOPS (giga, or billions, of operations per second) with a power budget of a few watts, an efficiency of about 100 GOPS/W (GOPS per watt), or 10 pJ/op (picoJoules per operation). At the same time programmability is needed to follow evolving standards, to support multiple air interfaces, and to dynamically provision processing resources over different air interfaces. Digital television, surveillance video processing, automated optical inspection, and mobile cameras, camcorders, and 3G cellular handsets have similar needs.
Streams and Standards:
Delivering Mobile Video
Don’t believe me? Follow along… Mobile phones are everywhere. Everybody has one. Think about the last time you were on an airplane and the flight was delayed on the ground. Immediately after the dreaded announcement, you heard everyone reach for their phones and start dialing.
Structured Deferral: Synchronization via Procrastination
We simply do not have a synchronization mechanism that can enforce mutual exclusion.
Successful Strategies for IPv6 Rollouts. Really.
Knowing where to begin is half the battle.
System Administration Soft Skills
How can system administrators reduce stress and conflict in the workplace?
TCP Offload to the Rescue
In recent years, TCP/IP offload engines, known as TOEs, have attracted a good deal of industry attention and a sizable share of venture capital dollars. A TOE is a specialized network device that implements a significant portion of the TCP/IP protocol in hardware, thereby offloading TCP/IP processing from software running on a general-purpose CPU. This article examines the reasons behind the interest in TOEs and looks at challenges involved in their implementation and deployment.
Tackling Architectural Complexity with Modeling
Component models can help diagnose architectural problems in both new and existing systems.
Testable System Administration
Models of indeterminism are changing IT management.
Testing a Distributed System
Testing a distributed system can be trying even under the best of circumstances.
The (not so) Hidden Computer
The growing complexity of purpose-built systems is making it difficult to conceal the computers within.
The API Performance Contract
How can the expected interactions between caller and implementation be guaranteed?
The Age of Corporate Open Source Enlightenment
Like it or not, zealots and heretics are finding common ground in the open source holy war.
The Answer is 42 of Course
Why is security so hard? As a security consultant, I’m glad that people feel that way, because that perception pays my mortgage. But is it really so difficult to build systems that are impenetrable to the bad guys?
The Antifragile Organization
Embracing Failure to Improve Resilience and Maximize Availability
The Balancing Act of Choosing Nonblocking Features
Design requirements of nonblocking systems
The Big Bang Theory of IDEs
Remember the halcyon days when development required only a text editor, a compiler, and some sort of debugger (in cases where the odd printf() or two alone didn't serve)? During the early days of computing, these were independent tools used iteratively in development's golden circle. Somewhere along the way we realized that a closer integration of these tools could expedite the development process. Thus was born the integrated development environment (IDE), a framework and user environment for software development that's actually a toolkit of instruments essential to software creation. At first, IDEs simply connected the big three (editor, compiler, and debugger), but nowadays most go well beyond those minimum requirements.
The Calculus of Service Availability
You're only as available as the sum of your dependencies.
The Case Against Data Lock-in
Want to keep your users? Just make it easy for them to leave.
The Challenge of Cross-language Interoperability
Interfacing between languages is increasingly important.
The Cost of Virtualization
Virtualization can be implemented in many different ways. It can be done with and without hardware support. The virtualized operating system can be expected to be changed in preparation for virtualization, or it can be expected to work unchanged. Regardless, software developers must strive to meet the three goals of virtualization spelled out by Gerald Popek and Robert Goldberg: fidelity, performance, and safety.
The Curse of the Excluded Middle
"Mostly functional" programming does not work.
The Debugging Mindset
Understanding the psychology of learning strategies leads to effective problem-solving skills.
The Deliberate Revolution
Transforming Integration With XML Web Services
The Emergence of iSCSI
Modern SCSI, as defined by the SCSI-3 Architecture Model, or SAM, really considers the cable and physical interconnections to storage as only one level in a larger hierarchy.
The Essence of Software Engineering: The SEMAT Kernel
A thinking framework in the form of an actionable kernel
The Evolution of Security
Security people are never in charge unless an acute embarrassment has occurred. Otherwise, their advice is tempered by “economic reality,” which is to say that security is a means, not an end. This is as it should be. Since means are about tradeoffs, security is about trade-offs, but you knew all that.
The Evolution of Web Development for Mobile Devices
Building Web sites that perform well on mobile devices remains a challenge.
The Family Dynamics of 802.11
The 802.11 family of standards is helping to move wireless LANs into promising new territory.
The Five-Minute Rule 20 Years Later:
and How Flash Memory Changes the Rules
The old rule continues to evolve, while flash memory adds two new rules.
The Flame Graph
This visualization of software execution is a new necessity for performance profiling and debugging.
The Future of Human-Computer Interaction
Personal computing launched with the IBM PC. But popular computing launched with the modern WIMP (windows, icons, mouse, pointer) interface, which made computers usable by ordinary people.
The Future of Microprocessors
The performance of microprocessors that power modern computers has continued to increase exponentially over the years for two main reasons. First, the transistors that are the heart of the circuits in all processors and memory chips have simply become faster over time on a course described by Moore’s law,1 and this directly affects the performance of processors built with those transistors. Moreover, actual processor performance has increased faster than Moore’s law would predict,2 because processor designers have been able to harness the increasing numbers of transistors available on modern chips to extract more parallelism from software.
The Future of WLAN
Overcoming the Top Ten Challenges in wireless networking--will it allow wide-area mesh networks to become ubiquitous?
The Heart of Eclipse
A look inside and extensible plug-in architecture ECLIPSE is both an open, extensible development environment for building software and an open, extensible application framework upon which software can be built. Considered the most popular Java IDE, it provides a common UI model for working with tools and promotes rapid development of modular features based on a plug-in component model. The Eclipse Foundation designed the platform to run natively on multiple operating systems, including Macintosh, Windows, and Linux, providing robust integration with each and providing rich clients that support the GUI interactions everyone is familiar with: drag and drop, cut and paste (clipboard), navigation, and customization.
The Hidden Dividends of Microservices
Microservices aren't for every company, and the journey isn't easy.
The Hitchhiker's Guide to Biomorphic Software
The natural world may be the inspiration we need for solving our computer problems. While it is certainly true that "the map is not the territory," most visitors to a foreign country do prefer to take with them at least a guidebook to help locate themselves as they begin their explorations. That is the intent of this article. Although there will not be enough time to visit all the major tourist sites, with a little effort and using the information in the article as signposts, the intrepid explorer can easily find numerous other, interesting paths to explore.
The IDAR Graph
An improvement over UML
The Ideal HPC Programming Language
Maybe it's Fortran. Or maybe it just doesn't matter.
The Inevitability of Reconfigurable Systems
The introduction of the microprocessor in 1971 marked the beginning of a 30-year stall in design methods for electronic systems. The industry is coming out of the stall by shifting from programmed to reconfigurable systems. In programmed systems, a linear sequence of configuration bits, organized into blocks called instructions, configures fixed hardware to mimic custom hardware. In reconfigurable systems, the physical connections among logic elements change with time to mimic custom hardware. The transition to reconfigurable systems will be wrenching, but this is inevitable as the design emphasis shifts from cost performance to cost performance per watt. Here's the story.
The Invisible Assistant
One lab's experiment with ubiquitous computing
The Long Road to 64 Bits
"Double, double, toil and trouble"... Shakespeare's words (Macbeth, Act 4, Scene 1) often cover circumstances beyond his wildest dreams. Toil and trouble accompany major computing transitions, even when people plan ahead. To calibrate "tomorrow's legacy today," we should study "tomorrow's legacy yesterday." Much of tomorrow's software will still be driven by decades-old decisions. Past decisions have unanticipated side effects that last decades and can be difficult to undo.
The Magic of RFID
Many modern technologies give the impression they work by magic, particularly when they operate automatically and their mechanisms are invisible. A technology called RFID (radio frequency identification), which is relatively new to the mass market, has exactly this characteristic and for many people seems a lot like magic. RFID is an electronic tagging technology that allows an object, place, or person to be automatically identified at a distance without a direct line-of-sight, using an electromagnetic challenge/response exchange.
The Mythos of Model Interpretability
In machine learning, the concept of interpretability is both important and slippery.
The NSA and Snowden: Securing the All-Seeing Eye
How good security at the NSA could have stopped him
The Network is Reliable
An informal survey of real-world communications failures
The Network's New Role
Application-oriented networks can help bridge the gap between enterprises.
The Obama Campaign:
A Programmer's Perspective
The Obama campaign has been praised for its innovative use of technology. What was the key to its success?
The Pain of Implementing LINQ Providers
It's no easy task for NoSQL
The Pathologies of Big Data
Scale up your datasets enough and all your apps will come undone. What are the typical problems and where do the bottlenecks generally surface?
The Price of Performance
In the late 1990s, our research group at DEC was one of a growing number of teams advocating the CMP (chip multiprocessor) as an alternative to highly complex single-threaded CPUs. We were designing the Piranha system,1 which was a radical point in the CMP design space in that we used very simple cores (similar to the early RISC designs of the late ’80s) to provide a higher level of thread-level parallelism. Our main goal was to achieve the best commercial workload performance for a given silicon budget.
The Reincarnation of Virtual Machines
The term "virtual machine" initially described a 1960s operating system concept: a software abstraction with the looks of a computer system's hardware (real machine). Forty years later, the term encompasses a large range of abstractions - for example, Java virtual machines that don't match an existing real machine. Despite the variations, in all definitions the virtual machine is a target for a programmer or compilation system. In other words, software is written to run on the virtual machine.
The Responsive Enterprise: Embracing the Hacker Way
Soon every company will be a software company.
The Rise and Fall of CORBA
Depending on exactly when one starts counting, CORBA is about 10-15 years old. During its lifetime, CORBA has moved from being a bleeding-edge technology for early adopters, to being a popular middleware, to being a niche technology that exists in relative obscurity. It is instructive to examine why CORBA—despite once being heralded as the “next-generation technology for e-commerce”—suffered this fate. CORBA’s history is one that the computing industry has seen many times, and it seems likely that current middleware efforts, specifically Web services, will reenact a similar history.
The Road to SDN
An intellectual history of programmable networks
The Robustness Principle Reconsidered
Seeking a middle ground
The Scalability Problem
Back in the mid-1990s, I worked for a company that developed multimedia kiosk demos. Our biggest client was Intel, and we often created demos that appeared in new PCs on the end-caps of major computer retailers such as CompUSA. At that time, performance was in demand for all application classes from business to consumer. We created demos that showed, for example, how much faster a spreadsheet would recalculate (you had to do that manually back then) on a new processor as compared with the previous year's processor. The differences were immediately noticeable to even a casual observer - and it mattered.
The Science of Managing Data Science
Lessons learned managing a data science research team
The Seven Deadly Sins of Linux Security
Avoid these common security risks like the devil.
The Software Inferno
Dante's tale, as experienced by a software architect
The Story of the Teapot in DHTML
It's easy to do amazing things, such as rendering the classic teapot in HTML and CSS.
The Sun Never Sits on Distributed Development
People around the world can work around the clock on a distributed project, but the real challenge lies in taming the social dynamics.
The Verification of a Distributed System
A practitioner's guide to increasing confidence in system correctness
The Virtualization Reality
A number of important challenges are associated with the deployment and configuration of contemporary computing infrastructure. Given the variety of operating systems and their many versions—including the often-specific configurations required to accommodate the wide range of popular applications—it has become quite a conundrum to establish and manage such systems.
The Web Won't Be Safe or Secure until We Break It
Unless you've taken very particular precautions, assume every Web site you visit knows exactly who you are.
The World According to LINQ
Big data is about more than size, and LINQ is more than up to the task.
The Yin and Yang of Software Development
The C/C++ Solution Manager at Parasoft explains how infrastructure elements allow development teams to increase productivity without restricting creativity.
There is No Now
Problems with simultaneity in distributed systems
There's Just No Getting around It: You're Building a Distributed System
Building a distributed system requires a methodical approach to requirements.
There's No Such Thing as a Free (Software) Lunch
"The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software to make sure the software is free for all its users." So begins the GNU General Public License, or GPL, which has become the most widely used of open source software licenses. Freedom is the watchword; it's no coincidence that the organization that wrote the GPL is called the Free Software Foundation and that open source developers everywhere proclaim, "Information wants to be free."
There's No Such Thing as a General-purpose Processor
And the belief in such a device is harmful
Thinking Clearly about Performance
Improving the performance of complex software is difficult, but understanding some fundamental principles can make it easier.
Thinking Methodically about Performance
The USE method addresses shortcomings in other commonly used methodologies.
Thou Shalt Not Depend on Me
A look at JavaScript libraries in the wild
Thread Scheduling in FreeBSD 5.2
A busy system makes thousands of scheduling decisions per second, so the speed with which scheduling decisions are made is critical to the performance of the system as a whole. This article - excerpted from the forthcoming book, "The Design and Implementation of the FreeBSD Operating System" - uses the example of the open source FreeBSD system to help us understand thread scheduling. The original FreeBSD scheduler was designed in the 1980s for large uniprocessor systems. Although it continues to work well in that environment today, the new ULE scheduler was designed specifically to optimize multiprocessor and multithread environments. This article first studies the original FreeBSD scheduler, then describes the new ULE scheduler.
Threads without the Pain
Multithreaded programming need not be so angst-ridden.
TiVo-lution
The challenges of delivering a reliable, easy-to-use DVR service to the masses
Time is an Illusion.
Lunchtime doubly so. - Ford Prefect to Arthur Dent in "The Hitchhiker's Guide to the Galaxy", by Douglas Adams
Time, but Faster
A computing adventure about time through the looking glass
Titus: Introducing Containers to the Netflix Cloud
Approaching container adoption in an already cloud-native infrastructure
Too Big NOT to Fail
Embrace failure so it doesn't embrace you.
Too Darned Big to Test
The increasing size and complexity of software, coupled with concurrency and distributed systems, has made apparent the ineffectiveness of using only handcrafted tests. The misuse of code coverage and avoidance of random testing has exacerbated the problem. We must start again, beginning with good design (including dependency analysis), good static checking (including model property checking), and good unit testing (including good input selection). Code coverage can help select and prioritize tests to make you more efficient, as can the all-pairs technique for controlling the number of configurations.
Too Much Information
Two applications reveal the key challenges in making context-aware computing a reality. As mobile computing devices and a variety of sensors become ubiquitous, new resources for applications and services - often collectively referred to under the rubric of context-aware computing - are becoming available to designers and developers. In this article, we consider the potential benefits and issues that arise from leveraging context awareness in new communication services that include the convergence of VoIP (voice over IP) and traditional information technology.
Toward Energy-Efficient Computing
What will it take to make server-side computing more energy efficient?
Toward Higher Precision
An introduction to PTP and its significance to NTP practitioners
Toward Software-defined SLAs
Enterprise computing in the public cloud
Toward a Commodity Enterprise Middleware
Can AMQP enable a new era in messaging middleware? AMQP (Advanced Message Queuing Protocol) was born out of my own experience and frustrations in developing front- and back-office processing systems at investment banks. It seemed to me that we were living in integration Groundhog Day - the same problems of connecting systems together would crop up with depressing regularity. Each time the same discussions about which products to use would happen, and each time the architecture of some system would be curtailed to allow for the fact that the chosen middleware was reassuringly expensive.
Trials and Tribulations of Debugging Concurrency
We now sit firmly in the 21st century where the grand challenge to the modern-day programmer is neither memory leaks nor type issues (both of those problems are now effectively solved), but rather issues of concurrency. How does one write increasingly complex programs where concurrency is a first-class concern. Or even more treacherous, how does one debug such a beast? These questions bring fear into the hearts of even the best programmers.
Triple-Parity RAID and Beyond
As hard-drive capacities continue to outpace their throughput, the time has come for a new level of RAID.
UML Fever:
Diagnosis and Recovery
Acknowledgment is only the first step toward recovery from this potentially devastating affliction. The Institute of Infectious Diseases has recently published research confirming that the many and varied strains of UML Fever continue to spread worldwide, indiscriminately infecting software analysts, engineers, and managers alike. One of the fevers most serious side effects has been observed to be a significant increase in both the cost and duration of developing software products. This increase is largely attributable to a decrease in productivity resulting from fever-stricken individuals investing time and effort in activities that are of little or no value to producing deliverable products.
Under New Management
Autonomic computing is revolutionizing the way we manage complex systems.
Undergraduate Software Engineering: Addressing the Needs of Professional Software Development
Addressing the Needs of Professional Software Development
Understanding DRM
The explosive growth of the Internet and digital media has created both tremendous opportunities and new threats for content creators. Advances in digital technology offer new ways of marketing, disseminating, interacting with, and monetizing creative works, giving rise to expanding markets that did not exist just a few years ago. At the same time, however, the technologies have created major challenges for copyright holders seeking to control the distribution of their works and protect against piracy.
Understanding Software Patching
Developing and deploying patches is an increasingly important part of the software development process.
Unified Communications with SIP
SIP can provide realtime communications as a network service.
Unifying Biological Image Formats with HDF5
The biosciences need an image format capable of high performance and long-term maintenance. Is HDF5 the answer?
Unikernels: Rise of the Virtual Library Operating System
What if all the software layers in a virtual appliance were compiled within the same safe, high-level language framework?
Uninitialized Reads
Understanding the proposed revisions to the C language
Unlocking Concurrency
Multicore architectures are an inflection point in mainstream software development because they force developers to write parallel programs. In a previous article in Queue, Herb Sutter and James Larus pointed out, “The concurrency revolution is primarily a software revolution.
Untangling Enterprise Java
Separation of concerns is one of the oldest concepts in computer science. The term was coined by Dijkstra in 1974.1 It is important because it simplifies software, making it easier to develop and maintain. Separation of concerns is commonly achieved by decomposing an application into components. There are, however, crosscutting concerns, which span (or cut across) multiple components. These kinds of concerns cannot be handled by traditional forms of modularization and can make the application more complex and difficult to maintain.
Uprooting Software Defects at the Source
Source code analysis is an emerging technology in the software industry that allows critical source code defects to be detected before a program runs.
Usablity Testing for the Web
Today’s Internet user has more choices than ever before, with many competing sites offering similar services. This proliferation of options provides ample opportunity for users to explore different sites and find out which one best suits their needs for any particular service. Users are further served by the latest generation of Web technologies and services, commonly dubbed Web 2.0, which enables a better, more personalized user experience and encourages user-generated content.
Use-Case 2.0
The Hub of Software Development
Using Free and Open Source Tools to Manage Software Quality
An agile process implementation
Verification of Safety-critical Software
Avionics software safety certification is achieved through objective-based standards.
Virtualization: Blessing or Curse?
Managing virtualization at a large scale is fraught with hidden challenges.
Visualizing System Latency
Heat maps are a unique and powerful way to visualize latency data. Explaining the results, however, is an ongoing challenge.
VoIP Security: Not an Afterthought
Voice over IP (VoIP) promises to up-end a century-old model of voice telephony by breaking the traditional monolithic service model of the public switched telephone network (PSTN) and changing the point of control and provision from the central office switch to the end user's device.
VoIP: What is it Good for?
VoIP (voice over IP) technology is a rapidly expanding field. More and more VoIP components are being developed, while existing VoIP technology is being deployed at a rapid and still increasing pace. This growth is fueled by two goals: decreasing costs and increasing revenues.
Voyage in the Agile Memeplex
Agile processes are not a technology, not a science, not a product. They constitute a space somewhat hard to define. Agile methods, or more precisely 'agile software development methods or processes', are a family of approaches and practices for developing software systems. Any attempt to define them runs into egos and marketing posturing.
Weapons of Mass Assignment
A Ruby on Rails app highlights some serious, yet easily avoided, security vulnerabilities.
Weathering the Unexpected
Failures happen, and resilience drills help organizations prepare for them.
Web Services and IT Management
Web services aren't just for application integration anymore.
Web Services: Promises and Compromises
Much of web services' initial promise will be realized via integration within the enterprise.
What DNS Is Not
DNS is many things to many people - perhaps too many things to too many people.
Whither Sockets?
High bandwidth, low latency, and multihoming challenge the sockets API.
Who Must You Trust?
You must have some trust if you want to get anything done.
Why Cloud Computing Will Never Be Free
The competition among cloud providers may drive prices downward, but at what cost?
Why Is It Taking So Long to Secure Internet Routing?
Routing security incidents can still slip past deployed security defenses.
Why LINQ Matters:
Cloud Composability Guaranteed
The benefits of composability are becoming clear in software engineering.
Why Logical Clocks are Easy
Sometimes all you need is the right language.
Why Writing Your Own Search Engine Is Hard
There must be 4,000 programmers typing away in their basements trying to build the next "world's most scalable" search engine. It has been done only a few times. It has never been done by a big group; always one to four people did the core work, and the big team came on to build the elaborations and the production infrastructure. Why is it so hard? We are going to delve a bit into the various issues to consider when writing a search engine. This article is aimed at those individuals or small groups that are considering this endeavor for their Web site or intranet.
Why Your Data Won't Mix
When independent parties develop database schemas for the same domain, they will almost always be quite different from each other. These differences are referred to as semantic heterogeneity, which also appears in the presence of multiple XML documents, Web services, and ontologies—or more broadly, whenever there is more than one way to structure a body of data. The presence of semi-structured data exacerbates semantic heterogeneity, because semi-structured schemas are much more flexible to start with. For multiple data systems to cooperate with each other, they must understand each other’s schemas.
Workload Frequency Scaling Law - Derivation and Verification
Workload scalability has a cascade relation via the scale factor.
XML
XML, as defined by the World Wide Web Consortium in 1998, is a method of marking up a document or character stream to identify structural or other units within the data. XML makes several contributions to solving the problem of semi-structured data, the term database theorists use to denote data that exhibits any of the following characteristics:
XML Fever
Don't let delusions about XML develop into a virulent strain of XML fever.
You Don't Know Jack About Software Maintenance
Long considered an afterthought, software maintenance is easiest and most effective when built into a system from the ground up.
You Don't Know Jack About VoIP
Telecommunications worldwide has experienced a significant revolution over recent years. The long-held promise of network convergence is occurring at an increasing pace. This convergence of data, voice, and video using IP-based networks is delivering advanced services at lower cost across the spectrum, including residential users, business customers of varying sizes, and service providers.
You Don't Know Jack about Disks
Whatever happened to cylinders and tracks?
You Don't Know Jack about Network Performance
Why does an application that works just fine over a LAN come to a grinding halt across the wide-area network? You may have experienced this firsthand when trying to open a document from a remote file share or remotely logging in over a VPN to an application running in headquarters. Why is it that an application that works fine in your office can become virtually useless over the WAN? If you think it's simply because there's not enough bandwidth in the WAN, then you don't know jack about network performance.
You Don't Know Jack about Shared Variables or Memory Models
Data races are evil.
Your Mouse is a Database
Web and mobile applications are increasingly composed of asynchronous and realtime streaming services and push notifications.