ACM Queue - Distributed Development

Online Event Processing

Martin Kleppmann, Alastair R. Beresford, Boerge Svingen — Sun, 24 Mar 2019 16:42:54 GMT

Support for distributed transactions across heterogeneous storage technologies is either nonexistent or suffers from poor operational and performance characteristics. In contrast, OLEP is increasingly used to provide good performance and strong consistency guarantees in such settings. In data systems it is very common for logs to be used as internal implementation details. The OLEP approach is different: it uses event logs, rather than transactions, as the primary application programming model for data management. Traditional databases are still used, but their writes come from a log rather than directly from the application. The use of OLEP is not simply pragmatism on the part of developers, but rather it offers a number of advantages. Consequently, OLEP is expected to be increasingly used to provide strong consistency in large-scale systems that use heterogeneous storage technologies.

Titus: Introducing Containers to the Netflix Cloud

Andrew Leung, Andrew Spyker, Tim Bozarth — Tue, 07 Nov 2017 13:57:15 GMT

We believe our approach has enabled Netflix to quickly adopt and benefit from containers. Though the details may be Netflix-specific, the approach of providing low-friction container adoption by integrating with existing infrastructure and working with the right early adopters can be a successful strategy for any organization looking to adopt containers.

Functional at Scale

Marius Eriksen — Tue, 20 Sep 2016 14:30:47 GMT

Modern server software is demanding to develop and operate: it must be available at all times and in all locations; it must reply within milliseconds to user requests; it must respond quickly to capacity demands; it must process a lot of data and even more traffic; it must adapt quickly to changing product needs; and in many cases it must accommodate a large engineering organization, its many engineers the proverbial cooks in a big, messy kitchen.

The Verification of a Distributed System

Caitie McCaffrey — Mon, 01 Feb 2016 21:33:33 GMT

Leslie Lamport, known for his seminal work in distributed systems, famously said, "A distributed system is one in which the failure of a computer you didn't even know existed can render your own computer unusable." Given this bleak outlook and the large set of possible failures, how do you even begin to verify and validate that the distributed systems you build are doing the right thing?

Testing a Distributed System

Philip Maddox — Wed, 01 Jul 2015 18:21:35 GMT

Distributed systems can be especially difficult to program, for a variety of reasons. They can be difficult to design, difficult to manage, and, above all, difficult to test. Testing a normal system can be trying even under the best of circumstances, and no matter how diligent the tester is, bugs can still get through. Now take all of the standard issues and multiply them by multiple processes written in multiple languages running on multiple boxes that could potentially all be on different operating systems, and there is potential for a real disaster.

Corba: Gone but (Hopefully) Not Forgotten

Terry Coatta — Mon, 14 Jul 2008 15:03:31 GMT

Corba: Gone But (Hopefully) Not Forgotten

There is no magic and the lessons of the past apply just as well today.

Terry Coatta

Back in the June 2006 issue of Queue, Michi Henning wrote a very good condensed history of CORBA and discussed how some of its technical limitations contributed to its downfall. While those limitations certainly aided CORBA's demise, there is a very widespread notion that the ultimate cause was the ascendance of Web Services, a notion that is compounded with the further belief that Web Services' dominance of the distributed computing landscape is indicative of its technical superiority to the systems that preceded it, such as CORBA and DCOM.

Having worked in distributed systems for a number of years—indeed far enough back in time that building a distributed system meant actually packing bits into UDP packets with lovingly hand-crafted C code—I think this assumption is unwarranted. Worse, it indicates a failure to appreciate the aspects of these previous systems that were well-engineered. This is a symptom of a "silver bullet" mentality that sees Web Services as a radical departure from the past that will finally remove the complexity from designing and building distributed systems.

A Passage to India

Mark Kobayashi-Hillary — Wed, 16 Feb 2005 10:36:56 GMT

A passage to India

Pitfalls that the outsourcing vendor forgot to mention

MARK KOBAYASHI-HILLARY, OFFSHORE ADVISORY SERVICES

Most American IT employees take a dim view of offshore outsourcing. It’s considered unpatriotic and it drains valuable intellectual capital and jobs from the United States to destinations such as India or China. Online discussion forums on sites such as isyourjobgoingoffshore.com are headlined with titles such as “How will you cope?” and “Is your career in danger?” A cover story in BusinessWeek magazine a couple of years ago summed up the angst most people suffer when faced with offshoring: “Is your job next?”

This article, however, is not designed to make the economic case for or against offshore outsourcing. Economists across the world are debating this issue and producing evidence on both sides of the macro-economic fence. What cannot be disputed is that the American IT industry is changing. According to the U.S. Bureau of Labor Statistics, in 1983, computer programmers accounted for 62 percent of all IT jobs in the U.S. with the remainder being computer engineers, analysts, and scientists. By 2002 this ratio had almost reversed, with programmers accounting for just 26 percent of IT jobs.

Outsourcing: Devising a Game Plan

Adam Kolawa — Mon, 06 Dec 2004 10:49:54 GMT

Outsourcing: Devising a Game Plan

What types of projects make good candidates for outsourcing?

Adam Kolawa, Parasoft

Your CIO just summoned you to duty by handing off the decision-making power about whether to outsource next year’s big development project to rewrite the internal billing system. That’s quite a daunting task! How can you possibly begin to decide if outsourcing is the right option for your company?

There are a few strategies that you can follow to help you avoid the pitfalls of outsourcing and make informed decisions. Outsourcing is not exclusively a technical issue, but it is a decision that architects or development managers are often best qualified to make because they are in the best position to know what technologies make sense to keep in-house. Deciding what should and should not be outsourced is key to a successful game plan.

Sink or Swim: Know When It's Time to Bail

Gordon Bell — Thu, 29 Jan 2004 10:33:28 GMT

There are endless survival challenges for newly created businesses. The degree to which a business successfully meets these challenges depends largely on the nature of the organization and the culture that evolves within it. That's to say that while market size, technical quality, and product design are obviously crucial factors, company failures are typically rooted in some form of organizational dysfunction. To help investors recognize signs of trouble before catastrophe strikes, I started working more than a decade ago on the Bell-Mason Diagnostic, a quantitative evaluation method that includes a set of rules for examining a company and comparing it with an "ideal" organization.

Culture Surprises in Remote Software Development Teams

Judith S. Olson, Gary M. Olson — Thu, 29 Jan 2004 10:29:21 GMT

Technology has made it possible for organizations to construct teams of people who are not in the same location, adopting what one company calls "virtual collocation." Worldwide groups of software developers, financial analysts, automobile designers, consultants, pricing analysts, and researchers are examples of teams that work together from disparate locations, using a variety of collaboration technologies that allow communication across space and time.