Development

Vol. 7 No. 11 – December 2009

Development

Articles

Data in Flight

How streaming SQL technology can help solve the Web 2.0 data crunch.

Data in Flight

How streaming SQL technology can help solve the Web 2.0 data crunch.

Julian Hyde, SQLstream

Web applications produce data at colossal rates, and those rates compound every year as the Web becomes more central to our lives. Other data sources such as environmental monitoring and location-based services are a rapidly expanding part of our day-to-day experience. Even as throughput is increasing, users and business owners expect to see their data with ever-decreasing latency. Advances in computer hardware (cheaper memory, cheaper disks, and more processing cores) are helping somewhat, but not enough to keep pace with the twin demands of rising throughput and decreasing latency.

The technologies for powering Web applications need to be fairly straightforward for two reasons: first, because it must be possible to evolve a Web application rapidly and then to deploy it at scale with a minimum of hassle; second, because the people writing Web applications are generalists and are not prepared to learn the kind of complex, hard-to-tune technologies used by systems programmers.

by Julian Hyde

Curmudgeon

Some Rules and Restrictions May Apply

An inquiry into contracts and the Next Big Thing

Some Rules and Restrictions May Apply

An inquiry into contracts and the Next Big Thing

Stan Kelly-Bootle, Author

In many of our interactions with the outside world (solipsists can stop reading now, if indeed they ever started) we enter into contracts with diverse entities, some up front, some lurking below the surface. The commonly construed contractual theme is a mutual agreement where each party accepts certain costs and responsibilities, and in return can rely on certain benefits and rewards. Some sort of symmetry is implied, in that the rational parties favored by the economists (when they come out of their recessionary hiding holes) will weigh the positive and negative impacts of the agreement before "signing." I will put aside the notions of Hobbes, Rousseau, Rawls, and that crowd that in merely being born—a mixed blessing to which nobody freely consents—we enter into some form of social contract with the combined prevailing laws of nature, nurture, and parliamentary whoredom.

We do our best, of course, to discern, codify, and obey the often-conflicting rules set by Life's Grand Game. I refer you to the vast literature on ethics, altruism, selfish genes, crime and punishment, and the dangers of universal gravity, hot pokers, and programming in Basic. But where is the free choice, predictability, or mutuality? Einstein was doubly mistaken. Nature does play dice, and, be warned, the dice are randomly loaded! Einstein was right about space-time: geometry is cruelly bent. Don't be fooled by Playfair's axiom.1 I support the Irish Olympic luge captain who demanded a level playing field.

by Stan Kelly-Bootle

Standards Advice

Easing the pain of implementing standards

Standards Advice

Easing the pain of implementing standards

I would like to dedicate this column to my first editor, Mrs. B. Neville-Neil, who passed away after a sudden illness on December 9, 2009. She was 65 years old.

My mother took language, both written and spoken, very seriously. The last thing I wanted to hear upon showing her an essay I was writing for school was, "Bring me the red pen." In those days I did not have a computer; all my assignments were written longhand or on a typewriter, so the red pen meant a total rewrite. She was a tough editor, but it was impossible to question the quality of her work or the passion that she brought to the writing process. All of the things Strunk and White have taught others throughout the years my mother taught me, on her own, with the benefit of only a high school education and a voracious appetite for reading.

by George Neville-Neil

Articles

Triple-Parity RAID and Beyond

As hard-drive capacities continue to outpace their throughput, the time has come for a new level of RAID.

Triple-Parity RAID and Beyond

Adam Leventhal, Sun Microsystems

As hard-drive capacities continue to outpace their throughput, the time has come for a new level of RAID.

How much longer will current RAID techniques persevere? The RAID levels were codified in the late 1980s; double-parity RAID, known as RAID-6, is the current standard for high-availability, space-efficient storage. The incredible growth of hard-drive capacities, however, could impose serious limitations on the reliability even of RAID-6 systems. Recent trends in hard drives show that triple-parity RAID must soon become pervasive. In 2005, Scientific American reported on Kryder's law,11 which predicts that hard-drive density will double annually. While the rate of doubling has not quite maintained that pace, it has been close.

Problematically for RAID, hard-disk throughput has failed to match that exponential rate of growth. Today repairing a high-density disk drive in a RAID group can easily take more than four hours, and the problem is getting significantly more pronounced as hard-drive capacities continue to outpace their throughput. As the time required for rebuilding a disk increases, so does the likelihood of data loss. The ability of hard-drive vendors to maintain reliability while pushing to higher capacities has already been called into question in this magazine.5 Perhaps even more ominously, in a few years, reconstruction will take so long as to effectively strip away a level of redundancy. What follows is an examination of RAID, the rate of capacity growth in the hard-drive industry, and the need for triple-parity RAID as a response to diminishing reliability.

by Adam Leventhal