January/February 2018 issue of acmqueue

The January/February issue of acmqueue is out now

File Systems and Storage

  Download PDF version of this article PDF

A Pioneer’s Flash of Insight

Jim Gray’s vision of flash-based storage anchors this issue’s theme.


In the May/June issue of Queue, Eric Allman wrote a tribute to Jim Gray, mentioning that Queue would be running some of Jim’s best works in the months to come. I’m embarrassed to confess that when this idea was first discussed, I assumed these papers would consist largely of Jim’s seminal work on databases—showing only that I (unlike everyone else on the Queue editorial board) never knew Jim. In an attempt to learn more about both his work and Jim himself, I attended the tribute held for him at UC Berkeley in May.

I came away impressed not only that Jim was a big, adventurous thinker, but also that he was one who remained moored to reality. This delicate balance pervades his life’s work: The IBM System R folks talked about how in a year he not only wrote nine papers, but also cut 10,000 lines of code. Database researchers from the University of Wisconsin described how Jim, who had pioneered so much in transaction processing, also pioneered the database benchmarks, developing the precursor for what would become the Transaction Processing Council in a classic Datamation paper. His Tandem colleagues talked about how he reached beyond research and development to engage with the field and with customers to figure out (and publish!) why systems actually failed—which was quite a reach for a company that famously dubbed every product “NonStop”! And Jim’s Microsoft coworkers talked about his driving vision to put on the Web a large database that people would actually use, leading him to develop TerraServer, and his ability to inspire and assist subsequent projects such as the Sloan Digital Sky Survey and Microsoft Research’s WorldWide Telescope—systems that solved hard, abstract problems and delivered appreciable concrete results.

Given that Jim insisted on designing and implementing actual systems in the present while also focusing on big, future-looking ideas, it is no surprise that he and Queue had a natural affinity for one another. While listening to the presentations at Jim’s tribute, I began asking myself a question that I imagine was also occurring to others: if Jim were here and if I could have but one conversation with him, what would it be about? For me, the answer was clear: I would want to talk with him about the coming revolution in flash-based storage, the focus of this month’s issue of Queue. This has been an exciting issue to put together, because as Adam Leventhal’s article, “Flash Storage Today,” discusses, the economics of flash have promoted it from sideshow to main event: flash is growing from mere curiosity to a new tier in the storage hierarchy—perhaps the first such tier since the introduction of the IBM RAMAC in 1956!

This development is exactly the kind of change that Jim clearly savored—so much so that in listening to his tribute, I began to wonder if he had thought of or published anything on flash-based storage prior to his disappearance. With the rapidly changing economics of flash, Jim would have had to be particularly insightful (at the time of his disappearance, flash was more than twice as expensive as it is now), but it seemed within the realm of possibility. His colleagues at Microsoft didn’t know of anything specific he had written on the topic, but directed me to his Web site for his list of papers. Thinking the published work was likely too old and that I had thus probably hit a dead end, I nonetheless went to the site—and what I saw there almost took my breath away: the second-most-recent link was “A Radical View of Flash Disks,” with a link to both a document and a talk. Thanks to the support of Jim’s family, we have been able to publish that paper, “Flash Disk Opportunity for Server Applications,” written with his colleague Bob Fitzgerald, in this issue. It is raw, and the numbers are now out of date—but Jim clearly sees the future in front of him, and it is a singular pleasure for the reader to be hoisted onto his shoulders.

Finding Jim’s work inspired us to add two other papers to this issue. First, since the time that Jim worked on this problem, one important impediment has been removed: the device-level issues that he describes—issues that have plagued consumer-grade flash-based SSDs (solid-state drives)—have been largely dealt with in the new class of enterprise-grade flash-based SSDs. It is important for practitioners to understand these problems and their solutions; STEC’s Pat Wilkison and Mark Moshayedi explain the innards of these important new devices in their article in this issue.

Second, in looking for other work that Jim might have done on flash, we ran across the work of Goetz Graefe, a Hewlett-Packard Fellow who had revisited Jim’s “five-minute rule” (the Jim Gray classic that we pointed to online in the May/June issue), wondering if flash had changed the equation. Goetz, too, saw Jim in this work (he dedicated this work to Jim), and it seems especially fitting that Goetz’s update of Jim’s rule now lives alongside Jim’s radical view of a flash-based future.

Enjoy this collection of articles—and the coming revolution in the storage hierarchy that they describe—and take a moment to mourn the loss of a great computer scientist who would have (once again) been in the thick of it all!

BRYAN CANTRILL is a Distinguished Engineer at Sun Microsystems, where he has spent more than a decade working on system software, from the guts of the kernel to client code on the browser and much in between. Along with colleagues Mike Shapiro and Adam Leventhal, Cantrill designed and implemented DTrace, a facility for dynamic instrumentation of production systems that won the Wall Street Journal’s top Technology Innovation Award in 2006. In 2005, Cantrill was named by MIT’s Technology Review as one of the top 35 technologists under the age of 35, and by InfoWorld as one of its Innovators of the Year. He received an Sc.B magna cum laude with honors in computer science from Brown University.


Originally published in Queue vol. 6, no. 4
see this item in the ACM Digital Library



Mihir Nanavati, Malte Schwarzkopf, Jake Wires, Andrew Warfield - Non-volatile Storage
Implications of the Datacenter's Shifting Center

Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Ramnatthan Alagappan, Samer Al-Kiswany, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau - Crash Consistency
Rethinking the Fundamental Abstractions of the File System

Adam H. Leventhal - A File System All Its Own
Flash memory has come a long way. Now it's time for software to catch up.

Michael Cornwell - Anatomy of a Solid-state Drive
While the ubiquitous SSD shares many features with the hard-disk drive, under the surface they are completely different.


(newest first)

Leave this field empty

Post a Comment:

© 2018 ACM, Inc. All Rights Reserved.