Weather station: experiments
So now I have a little program that can talk to the weather station. What other secrets does it hide? Steve Woodford sent me one program, but it seems that this machine has about 64 kB of memory accessible via the USB bus, and the manual states that it can store 4080 readings. At 16 bytes per reading, that suggests that it's all except for the first 256 bytes. The previous incarnation of the program shows that it updates a specific memory location at frequent intervals, and that there's a field stating how old the entry is. Every 30 minutes it moves on to a new slot, so the archive entries show 48 records per day and should last for 85 days.
Friday Squid Blogging: New Squid Discovered
An expedition to study seamounts in the Indian Ocean has discovered some new species, including some squid....
Radio Berkman interview
Here's an interview I did with Radio Berkman -- the podcast of the Berkman Center at Harvard. Radio Berkman 137: Cory Doctorow ¿ In Defense of © MP3 Link
Android Splintering?
There's this flood of new Android phones hitting the market; we're starting to see some new form factors and hardware setups. I'm hearing concerns here and there about the market “splintering”, making the platform less attractive to developers. Only I don't buy it. I'm pretty sure anybody who's been to the mat with the Android APIs shares my unconcern. First of all, a high proportion of most apps is just lists of things to read and poke at; another high proportion of Android apps are decorated Google maps and camera views. I bet most of those will Just Work on pretty well any device out there.
Interview with Me
Yet another interview with me. This one is audio, and was conducted in Rotterdam in October....
Dear ISPs
Dear ISPs, By far the largest set of support requests that we end up seeing for Thunderbird have to do with being unable to receive or send mail. By far the largest single cause of these failures is some unilateral change by the ISP which cause previously working configurations to stop working. In other [...]
Why SHMConfig is off by default
Bastien mentioned the Chromium OS xorg.conf file, which includes an irritating wart - namely, Option "SHMConfig" "on". This tells the Synaptics touchpad driver to export its configuration data to a shared memory region which is accessible to any user on the system. The reason for this is that in the past, there was no good way for configuration information to be passed to input drivers through the X server at runtime. This got fixed with the advent of X input properties, and synaptics can now be configured sensibly over the X protocol.But why was it off by default? Because, as I said, the configuration data is exported to a shared memory region which is accessible to any user on the system.
ACM Symposium on Cloud Computing
I'm on the program committee for the ACM Symposium on Cloud Computing. The conference will be held June 10th and 11th 2010 in Indianapolis Indiana. SOCC brings together database and operating systems researchers and practitioners interested in cloud computing. It is jointly sponsored by the ACM Special Interest Group on Management of Data (SIGMOD) and the ACM Special Interest Group on Operating Systems (SIGOPS). The conference will be held in conjunction with ACM SIGMOD in 2010 and with SOSP in 2011 continuing to alternate between SIGMOD and SOSP in subsequent years. Joe Hellerstein is the SOCC General Chair and Surajit Chaudhuri and Mendel Rosenblum are Program Chairs.
My wife kicks ass
Congrats to my wife Alice, on winning four British Interactive Media Awards! What a clever spouse -- I'm a lucky, lucky man..
Denial-of-Service Attack Against CALEA
Interesting: The researchers say they've found a vulnerability in U.S. law enforcement wiretaps, if only theoretical, that would allow a surveillance target to thwart the authorities by launching what amounts to a denial-of-service (DoS) attack against the connection between the phone company switches and law enforcement. [...] The University of Pennsylvania researchers found the flaw after examining the telecommunication industry...
Jive turkey...
Interview with Straylight Magazines
Here's an interview I conducted with Straylight Magazine, ranging over many subjects -- Makers, the craft of writing, and the business of publishing. MP3 Link
Weather station: success
Continued with my weather station software today, gradually cleaning up the strangenesses. It now reads the station every 30 seconds and prints the results: Page sv IH IT OH OT hPa abs rel Wind Gust Dir Rain 1490 11 45 22.6 24 30.2 93.6 93.6 983.4 5.6 13.3 NW 0.0 1490 11 45 22.6 24 30.2 93.6 93.6 983.4 5.6 13.3 NW 0.0 The pressures are completely wrong, of course, but the device is defective. My new one arrived today (and I'm back on the net), and I was able to confirm with it that the pressures are read correctly, modulo the fact that the device wants you to set the relative pressure manually.
Sigh.
If only eeepc-laptop sent standard keycodes, or something.Oh, wait.Writing a Linux distribution is hard. There's a huge range of interconnected dependencies. It takes a long time to learn how everything fits together, and fixing things properly rather than adding device-specific hacks often requires rewriting a lot of code. I'm sure Google will figure it out in time[1], and I'm also sure that the majority of their work is going into their UI rather than the underlying infrastructure.
A Taxonomy of Social Networking Data
At the Internet Governance Forum in Sharm El Sheikh this week, there was a conversation on social networking data. Someone made the point that there are several different types of data, and it would be useful to separate them. This is my taxonomy of social networking data. Service data. Service data is the data you need to give to a...
Stabbing People with Stuff You Can Get Through Airport Security
"Use of a pig model to demonstrate vulnerability of major neck vessels to inflicted trauma from common household items," from the American Journal of Forensic Medical Pathology. Abstract. Commonly available items including a ball point pen, a plastic knife, a broken wine bottle, and a broken wine glass were used to inflict stab and incised wounds to the necks of...
MAKERS tour ends this weekend at Philcon near Philadelphia
This weekend, I'll be wrapping up my US/Canada tour for Makers, my new novel, with a weekend at Philcon, near Philadelphia. I'll be signing books, doing a reading, giving a speech, and appearing on several panels. Hope to see you there! Important note: I had previously announced a couple of readings tomorrow at the Philadelphia Free [...]
Debugging despite black box libraries
Back to looking at my weather station code today. I was hanging in a call to usb_interrupt_read, and the obvious assumption was that I had not copied the sample code correctly. But how do I compare that? Discovered that, with a couple of tricks, I could use gdb on wviewd: I just needed to stop it becoming a daemon, which you do with a specific parameter to the undocumented function radProcessInit—why use the standard tools when you can write your own? --- daemon.c 2009-11-18 12:32:01.000000000 +1100 +++ daemon.c~ 2009-09-07 10:00:54.000000000 +1000 @@ -852,7 +852,7 @@ if (radProcessInit (PROC_NAME_DAEMON, wviewdWork.fifoFile, PROC_NUM_TIMERS_DAEMON, - FALSE, // TRUE for daemon + TRUE, // TRUE for daemon ...
Concur.next – Parallel I/O
Conclusion first: It turns out that Clojure's concurrency primitives allow you, with a very moderate amount of uncomplicated code, to take advantage of parallel hardware and outperform really fast software when it doesn't take such advantage. I recently wrote about Clojure's concurrency primitives and how you'd use them for a simple logfile-processing problem. The problem divides naturally in two: read the file in parallel, and process the lines in parallel. That last piece discussed the line processing, this one the file reading. [This is part of the Concur.next series.] Does It Work? This code actually runs now; the first-cut unoptimized version presented here computes Wide-Finder-style stats on a Sun T2000 “Niagara” faster than a simple single-threaded Perl script while keeping, on average, over ten of its 32 logical CPUs busy; the Niagara's real maximum throughput is achieved when it reports 16 or so as being busy, so this isn't ...
How Smart are Islamic Terrorists?
Organizational Learning and Islamic Militancy (May 2009) was written by Michael Kenney for the U.S. Department of Justice. It's long: 146 pages. From the executive summary: Organizational Learning and Islamic Militancy contains significant findings for counter-terrorism research and policy. Unlike existing studies, this report suggests that the relevant distinction in knowledge learned by terrorists is not between tacit and explicit...
Interview with The Link (Concordia)
Ginger Coons did a great interview with me last week for Concordia's paper The Link. Good meaty policy questions ahoy! Enhanced Driver's Licenses are being adopted in order to comply with newly-created American regulations on what constitutes an acceptable document for crossing the border. Doctorow did not view this as a sensible excuse. “If all the other [...]
Antifeatures
In preparation for LCA, I'm going to be giving my new Antifeatures talk a few times to smaller local audiences. The first is going to be today in Boston (apologies for the late notice!) at Northeastern University at 11:45 and it's being hosted by the ACM chapter there. The second one will be at my alma mater Hampshire College in Amherst this Friday. A draft flier (ignore the unpluralized "antifeatue") is below.
Quantum Ghost Imaging
This is cool: Ghost imaging is a technique that allows a high-resolution camera to produce an image of an object that the camera itself cannot see. It uses two sensors: one that looks at a light source and another that looks at the object. These sensors point in different directions. For example, the camera can face the sun and the...
Meet 13-year-old Cory Doctorow, at-risk D&D player
Here's a mid-1980s CBC News scare-story about Dungeons and Dragons driving kids to suicide featuring (at 2:49 onwards) me and my classmates (the video is dated 1985, but I'm pretty sure this couldn't have been later than my graduation from Junior High in 1984). Ignoring the crazy-ass fearmongering, it's incredibly nostalgic to see all those [...]
USB library debugging: the pain
Continued with my work on the USB interface of the Fine Offset WH-1081 weather station, and got the code completed relatively quickly. I'm becoming more and more dubious about some of the calls, and there are weirdnesses like having the data stored in two different places, but at least I now understand what's going on. So: ran it. Nothing happened, not even a SIGSEGV. Put the thing in gdb—that's easy, in contrast to wviewd—and discovered it was hanging in the very first read. It seems that to read from the device, you first need to send a control message. The reference code I have is full of manifest literals which make it completely unclear what's really going on: // Construct a 'read page' message msg[0] = 0xa1; msg[1] = (addr >> 8) & 0xffu; msg[2] = (addr >> 0) & 0xffu; msg[3] ...
National Post on Makers
Nice piece in today's National Post about Makers and my approach to publishing: Presently, Doctorow is in the midst of a short North American book tour promoting Makers, which, like all his work, is free to download from his website, craphound.com,under a Creative Commons licence, which allows readers to share and remix the work as long [...]
Secret Knock Lock
Door lock that opens if you tap a particular rhythm....
MAKERS signing 7PM tonight in NYC at Columbus Circle Borders
Hey, New Yorkers! I'm reading from and signing my new novel Makers tonight at the Borders in Columbus Circle at 59th Street, starting at 7PM. Hope to catch you there! Philadelphians, you're next -- Philadelphia Free Library on Friday, then Philcon (in Cherry Hill, NJ) over the weekend. US/Canada Tour
DRM-free, free-as-in-beer Dutch Little Brother ebook
Uitgeverij De Vliegende Hollander is the Dutch publisher for Little Brother, and they've really put a big push behind it. Unfortunately, they're also locked into distributing their catalog as DRM-crippled ebooks through an online retailer that is the only major ebook vendor in the Netherlands. But they're good folks at my publisher, and they're not fond [...]
A Useful Side-Effect of Misplaced Fear
A study in the British Journal of Criminology makes the point that drink-spiking date-raping is basically an urban legend: Abstract. There is a stark contrast between heightened perceptions of risk associated with drug-facilitated sexual assault (DFSA) and a lack of evidence that this is a widespread threat. Through surveys and interviews with university students in the United Kingdom and United...
Talking National Broadband Strategy
Tim O'Reilly on the future web wars
I’ve tended to limit my link referrals to my Twitter feed over the last year, but I wanted to advertise Tim O’Reilly’s latest post on this channel as well (it also feels great to have more than 100 characters to express myself!). Tim explains well what the new battlegrounds for the future of the web [...]
Monday, 16 November 2009
More playing around with the weather station program today, and in the end decided to go with the current libusb implementation, with guidance from the code in wview, after discovering that it's still supported in the more recent versions of libusb, and that there's documentation of a kind: Name usb_open -- Opens a USB device Description usb_dev_handle *usb_open (struct *usb_device dev); usb_open is to be used to open up a device for use. usb_open must be called before attempting to perform any operations to the device. Returns a handle used in future communication with the device.
AMCTV interview about MAKERS and Disney World
I recently conducted an interview with AMCTV's Sci-Fi Scanner about my new novel MAKERS, in which we got into some nice, juicy detail about what makes Disney Parks so fascinating for science fiction treatment. Q: So how did the concept evolve into creating a hacker Disney World in a Wal-Mart? Did it come from your other [...]
Moules Frites @ Devoxx
I'm back in lovely Antwerp for Devoxx. The purgatory we're in over the situation with Oracle has it's pluses and minuses. On the plus side, I don't have to do a keynote.... Steve Harris from Oracle get's that job. I will be doing a talk, but I'll be concentrating on the store we're in the process of launching. The hard part is that the only questions that anyone will be asking are the ones that neither Steve nor I can answer: until the acquisition clears the EC competition commission and closes, we're required to be mostly silent about the future. We're pretty much limited to the official statements.
Martian Chronicles: new story podcast
I've just started podcasting a new story: MARTIAN CHRONICLES is a story I'm working on for Jonathan Strahan's forthcoming LIFE ON MARS young adult anthology. It's a story about the colonization of Mars by free-market absolutists and the video-games they play. They say you can't smell anything through a launch-hood, but I still smelled the pove [...]
Martian Chronicles, Part 01
Here's part one of the podcast of my story in progress, MARTIAN CHRONICLES, being written for Jonathan Strahan's YA Mars book, LIFE ON MARS. MP3 Link
MAKERS US launch tonight, Harvard Bookstore, 7PM
Hey, Bostonites! I'll see you tonight at the Harvard Bookstore (1256 Mass Ave) at 7PM for the US launch of my new novel, Makers! (New Yorkers, and Philadelphians -- see you later this week!) US/Canada tour
Little Brother interview
Intervju med Cory Doctorow from Veslebror Serdeg on Vimeo. Here's an interview I gave to my Norwegian publisher about the writing of Little Brother.
Spider Robinson reads HUMAN READABLE pt 2
Spider Robinson's posted part two of his reading of my story Human Readable on his latest podcast. I'm so happy with how this turned out! MP3 Link Podcast feed
Public Reactions to Terrorist Threats
Interesting research: For the last five years we have researched the connection between times of terrorist threats and public opinion. In a series of tightly designed experiments, we expose subsets of research participants to a news story not unlike the type that aired last week. We argue that attitudes, evaluations, and behaviors change in at least three politically-relevant ways when...
Sunday, 15 November 2009
What does car mean (in a computational sense, of course)? It's one of the oldest LISP constructs, of course, but I've been continually irritated by people who state that it's an acronym for “Contents of the Address Register” (and that cdr stands for “Contents of the decrement register”). All this goes back to the IBM 704, which really did have something called a decrement register (it was a reverse index register: instead of adding the contents to the base address, the computer subtracted them). But the 704 didn't have an address register. So what do they stand for? Went looking for some documentation on the 704 instruction set, which is difficult to find.
Bruce Schneier Action Figure
A month ago, ThatsMyFace.com approached me about making a Bruce Schneier action figure. It's $100. I'd like to be able to say something like "half the proceeds are going to EPIC and EFF," but they're not. That's the price for custom orders. I don't even get a royalty. The company is working on lowering the price, and they've said that...
Saturday, 14 November 2009
Finally swamp finished its upgrade—and it turned out I had installed 9.0-CURRENT! I need to start all over again, preferably with a new disk. The libusb man page for 9.0-CURRENT appears to be the same as on the old 8.0-CURRENT, with these silly libusb20_tr_close and friends. Got confirmation from somebody running 8.0-BETA that the man page there reflects (at least superficially) the libusb 1.0 API. Still, it seems to be too much work to me. < !-- topic p not selected Spent a particularly long time with my house photos again today.
CL IX: Closing Up
For those of us who live North of the 49th parallel, an essential feature of Cottage Life is the sad process of “Closing Up” acknowledging that you're just not going any more till Spring. We did, but remain ambiguous. This time of year in the Pacific Northwest, when your cottage is perched on the north slope of a minor mountain, it gets wet and stays wet and you just have to deal with that. On the other hand, that means that most of the landscape you're looking at us facing south and picks up any fleeting glimpses of the Sun in the most fetching way.
Randy Shroup & John Ousterhout at HPTS 2009
HPTS has always been one of my favorite workshops over the years. Margo Seltzer was the program chair this year and she and the program committee brought together one of the best programs ever. Earlier I posted my notes from Andy Bectolsheim's session Andy Bechtolsheim at HPTS 2009 and his slides Technologies for Data Intensive Computing. Two other sessions were particularly interesting and worth summarizing here. The first is a great talk on high-scale services lessons learned from Randy Shroup and a talk by John Ousterhout on RAMCloud a research project to completely eliminate the storage hierarchy and store everything in DRAM.
Randy Shoup & John Ousterhout at HPTS 2009
HPTS has always been one of my favorite workshops over the years. Margo Seltzer was the program chair this year and she and the program committee brought together one of the best programs ever. Earlier I posted my notes from Andy Bectolsheim's session Andy Bechtolsheim at HPTS 2009 and his slides Technologies for Data Intensive Computing. Two other sessions were particularly interesting and worth summarizing here. The first is a great talk on high-scale services lessons learned from Randy Shoup and a talk by John Ousterhout on RAMCloud a research project to completely eliminate the storage hierarchy and store everything in DRAM.
Reading from last night
Here's the actual reading from last night's MAKERS launch: MP3 Link
Maker Culture video from last night's launch
Here's me chatting with the MakerCulture Project, a student group who came out to my launch last night. Smart folks! They also got pix.
Makers reading
Here's the actual reading from last night's MAKERS launch: MP3 Link
Audio and photos from MAKERS launch in Toronto
Andrew Schwab recorded last night's MAKERS launch in Toronto, a stellar event that was standing-room only, featuring free Ubuntu disks (thanks, John!), presentation of the Sunburst Award, a reading, and Q&A. Photos Makers Launch audio MP3 link
Friday, 13 November 2009
More investigation of the weather station issues this morning. Fired up kimchi and started wview, and—it worked! That proved to be without the archive data, so tried reinstating that, and, not surprisingly, it failed again. Clear case of corrupt old data. Moved it out again and restarted and—it still failed! More playing around. Ran ktrace to see what data was coming across. In the middle of that, saw the strangest thing: === root@kimchi (/dev/pts/0) /var/tmp 21 -> ls -l ktrace.out -rw-r--r-- 1 root wheel 111578 Nov 13 08:43 ktrace.out === root@kimchi (/dev/pts/0) /var/tmp 22 -> ls -l ktrace.out Shared object "libutil.so.7" not found That didn't go away.
Itch scratched - right between the rabbit's ears.
Blowfish in Fiction
The algorithm is mentioned in Von Neumann's War, by John Ringo and Travis Taylor. P. 495: The guy was using a fairly simple buffer overflow attack but with a very nice little fillip of an encryption packet designed to overcome Blowfish. The point seemed to be to create a zero day exploit, which he didn't have a chance of managing....
Video Interview with Me
Here's an interview with me, conducted at the Information Security Decisions conference in Chicago in October....
TVO Agenda on Makers
Here's last night's TVO Agenda interview on Makers -- Steve Paikin and I had a fantastic talk.
Links for Friday, November 13, 2009
Genetic Science Learning Center: Cell Size and Scale - “The smallest objects that the unaided human eye can see are about 0.1 mm long. That means that under the right conditions, you might be able to see an ameoba proteus, a human egg, and a paramecium without using magnification. A magnifying glass can help you [...]
Legacy PC design misery
I've spent chunks of the last couple of days fighting a problem that's existed for about 25 years. The 8086 was a 16-bit processor with a 20-bit address space, limiting the maximum physical address that could be accessed to 1MB. However, quirks of the segmented memory system meant that addresses greater than 1MB could be constructed - these would wrap around to the bottom of memory. Because loading the segment registers was a time consuming operation, some programmers used this behaviour as a performance optimisation.The 80286 introduced 24 bit address space. Unfortunately, this meant that the addresses that previously wrapped to the bottom of memory now pointed at real addresses - not ideal if you were expecting the old behaviour.
Beyond Security Theater
[I was asked to write this essay for the New Internationalist (n. 427, November 2009, pp. 10¿13). It's nothing I haven't said before, but I'm pleased with how this essay came together.] Terrorism is rare, far rarer than many people think. It's rare because very few people want to commit acts of terrorism, and executing a terrorist plot is much...
Video from Quantum 2 Cosmos talks
A couple weeks back I travelled to Waterloo, ON, to appear at the Perimeter Institute's Quantum 2 Cosmos science and technology festival. I gave a talk on copyright and did a panel on robotics and AI, both of which were lots of fun. The TVOntario people just sent me the video links for both, with [...]
Vote for me!
Arch rival to Computing, Computer Weekly, has shortlisted this blog for the IT consultant and analyst blog of the year award. Perhaps it's just a vote of sympathy, getting a blog from a rival magazine on their shortlist, or perhaps they will engage in some Simon Cowell-like skulduggery to ensure I can't win. But either way, I'm shortlisted for the blogger award so I hope that you ¿ dear reader ¿ can help me win! I need your vote.I would note that my editor, Bryan Glick, finishes work at Computing at the end of this week and is moving to be the boss at Computer Weekly so I would dearly like to see myself voted to win an award from his new magazine while I am writing for his former title!You can vote for me now by clicking here.
Thursday, 12 November 2009
The morning after a power failure is always a problem while I pick up the pieces, but this time things looked pretty good. I had powered dereel on (need to do it manually) after the power came back at 2:16, and Yvonne confirmed that she could work normally. brewer and kimchi were up as well, and all seemed OK. But then brewer went away, and when I went out to look it the display was blank (yes, the new one does have a display), and pressing the reset button just gave me repeated beeps. Brought it inside and remounted the memory, after which it came back normally.
FBI/CIA/NSA Information Sharing Before 9/11
It's conventional wisdom that the legal "wall" between intelligence and law enforcement was one of the reasons we failed to prevent 9/11. The 9/11 Comission evaluated that claim, and published a classified report in 2004. The report was released, with a few redactions, over the summer: "Legal Barriers to Information Sharing: The Erection of a Wall Between Intelligence and Law...
Links for Thursday, November 12, 2009
Wall Street Journal: Tinkering Makes Comeback Amid Crisis - “The American tradition of tinkering — the spark for inventions from the telephone to the Apple computer — is making a comeback, boosted by renewed interest in hands-on work amid the economic crisis and falling prices of high-tech tools and materials.“ Seeed Studio: Electronic Bricks? - “By [...]
Security in a Reputation Economy
In the past, our relationship with our computers was technical. We cared what CPU they had and what software they ran. We understood our networks and how they worked. We were experts, or we depended on someone else for expertise. And security was part of that expertise. This is changing. We access our email via the web, from any computer...
Makers Canadian launch in Toronto tonight!
Tonight, I'm launching my latest novel Makers in Canada, at the excellent Toronto sf reference library, the Merril Collection, at 239 College St. (3rd floor), east of Spadina. The event starts at 7PM, and I'll be doing a reading, taking questions, and signing books. Books are being sold by Bakka Phoenix, and if you can't make [...]
Trip Report: October 2008 ISO C++ Standards Meeting
The ISO C++ committee met in Santa Cruz, CA, USA on October 19-24. You can find the minutes here, which include the votes at the whole-group sessions but not the details of the breakout technical sessions where we spend most of the week. The good news is that there's little new technical news. We did a [...]
Technologies for Data Intensive Computing
In an earlier post Andy Bechtolsheim at HPTS 2009 I put my notes up on Andy Bechtolsheim's excellent talk at HPTS 2009. His slides from that talk are now available: Technologies for Data Intensive Computing. Strongly recommended. James Hamilton e: jrh@mvdirona.com w: http://www.mvdirona.com b: http://blog.mvdirona.com / http://perspectives.mvdirona.com
Wed, 11 Nov 2009 23:59:59 UTC
bilbo went down again last night, and again I got a stack backtrace: Message from syslogd@bilbo at Wed Nov 11 03:47:30 2009 ... [61911.308408] Oops: 0002 [#1] SMP [61911.308437] last sysfs file: /sys/devices/pci0000:00/0000:00:1c.5/0000:02:00.0/irq [61911.313002] Stack: [61911.313002] ffff880053137d58 ffffffff802d15b4 0000000000000246 ffff88005318ca48 [61911.313002] 0000000000000000 0000000000000080 0000000000000080 ffffffff802d184b [61911.313002] Call Trace: [61911.313002] [<ffffffff802d13ba>] ? clear_inode+0x6c/0xc3 [61911.313002] [<ffffffff802d15b4>] ? dispose_list+0x58/0xfd [61911.313002] [<ffffffff802d184b>] ? shrink_icache_memory+0x1f2/0x228 [61911.313002] [<ffffffff8029cdda>] ? shrink_slab+0xe0/0x153 [61911.313002] [<ffffffff8029d648>] ? kswapd+0x4a4/0x633 [61911.313002] [<ffffffff8029af7e>] ? isolate_pages_global+0x0/0x22c [61911.313002] [<ffffffff80254742>] ? autoremove_wake_function+0x0/0x2e [61911.313002] [<ffffffff802348e4>] ? __wake_up_common+0x44/0x73 [61911.313002] [<ffffffff8029d1a4>] ? kswapd+0x0/0x633 [61911.313002] [<ffffffff8029d1a4>] ? kswapd+0x0/0x633 [61911.313002] [<ffffffff8025437a>] ? kthread+0x54/0x80 [61911.313002] [<ffffffff80210aca>] ?
Wednesday, 11 November 2009
Concur.next – References
These, “refs” for short, are one of the three tools offered by Clojure to make concurrency practical and manageable. Herewith a walk-through of code that uses them to accomplish a simple task in a highly concurrent fashion. [This is part of the Concur.next series.] The Problem It's one that's been much-discussed in this space: reading all the lines in a large file and computing some simple statistics. It was the discovery that this sort of script ran faster on my Mac laptop than a pricey industrial-grade Sun server that originally motivated the Wide Finder work. What I would really like to come out of all this work would be a set of tools, accessible to ordinary programmers, that help them accomplish everyday tasks with code that makes good use of many of the compute engines on a modern multi-core processor.
Effective Concurrency: Prefer structured lifetimes ¿ local, nested, bounded, deterministic.
This month's Effective Concurrency column, Prefer structured lifetimes ¿ local, nested, bounded, deterministic, is now live on DDJ's website. From the article: Where possible, prefer structured lifetimes: ones that are local, nested, bounded, and deterministic. This is true no matter what kind of lifetime we're considering, including object lifetimes, thread or task lifetimes, [...]
Hacking the Brazil Power Grid
We've seen lots of rumors about attacks against the power grid, both in the U.S. and elsewhere, of people hacking the power grid. Seems like the source of these rumors has been Brazil: Several prominent intelligence sources confirmed that there were a series of cyber attacks in Brazil: one north of Rio de Janeiro in January 2005 that affected three...
Thieves Prefer Stealing Black Luggage
It's obvious why if you think about it: Thieves prefer to steal black luggage because so much of it looks alike. If the thief is caught red-handed by the bag's owner, he only has to say sorry, it looks just like mine. And he's out of there. Scott free. Read the news story that prompted this blog post. I had...
Tuesday, 10 November 2009
Accidentally tried to access one of my pages from the external web site today (http://www.lemis.com/ instead of http://wwww.lemis.com/), and got no response. Suspecting satellite problems again, took a look and discovered that www.lemis.com (really bilbo.ozlabs.org, a Linux machine) was really down, and that nobody knew what had happened. While looking for a spare xterm, discovered one that had been connected to bilbo, conveniently with a lot of syslog messages showing the cause: Message from syslogd@bilbo at Tue Nov 10 04:28:07 2009 ... [918204.839237] Oops: 0002 [#1] SMP [918204.841194] [<ffffffff8029d1a4>] ? kswapd+0x0/0x633 [918204.841194] [<ffffffff8029d1a4>] ? kswapd+0x0/0x633 [918204.841194] [<ffffffff802d13ba>] ?
Protecting OSs from RootKits
Interesting research: "Countering Kernel Rootkits with Lightweight Hook Protection," by Zhi Wang, Xuxian Jiang, Weidong Cui, and and Peng Ning. Abstract: Kernel rootkits have posed serious security threats due to their stealthy manner. To hide their presence and activities, many rootkits hijack control flows by modifying control data or hooks in the kernel space. A critical step towards eliminating rootkits...
The ACPI Embedded Controller
Of course, the event model I described before is far too simple to be worthy of a place in the ACPI spec. At the most basic level, there's more possible events than there are GPEs to attach them to, so there's a need for some further complexity. This manifests itself in the form of the ACPI embedded controller (EC).The EC is typically a small microprocessor sitting on your motherboard, often implemented in the same hardware as the keyboard controller. It shares a lot in common with the keyboard controller - on PCs it'll usually appear in system io space, with one register for writing a command or reading a status, and a second register for passing data back and forth[1].
Hypothetical peek into Rupert Murdoch's mind
I turned my Boing Boing post about Murdoch's mad pronouncements on the Internet into a column for the Guardian, called "For whom the net tolls." What, exactly, is Rupert Murdoch thinking? First, he announces that all of Newscorp's websites will erect paywalls like the one employed by the Wall Street Journal (however, Rupert managed to get [...]
Is Antivirus Dead?
Security is never black and white. If someone asks, "for best security, should I do A or B?" the answer almost invariably is both. But security is always a trade-off. Often it's impossible to do both A and B -- there's no time to do both, it's too expensive to do both, or whatever -- and you have to choose....
Spider Robinson reads my story “Human Readable”
As part of the audiobook for my forthcoming experimental short story collection With a Little Help I asked Spider Robinson to read my story "Human Readable" aloud. He did a smashing job, and now he's serializing the story on his podcast feed. MP3 link Podcast feed
Epoch, Part 08
Here's the eighth and final installment of a story-in-progress, Epoch, commissioned by Mark Shuttleworth for my forthcoming short story collection WITH A LITTLE HELP. MP3 Link
ACPI general purpose events
ACPI is a confusing place. It's often thought of as a suspend/resume thing, though if you're unlucky you've learned that it's also involved in boot-time configuration because it's screwed up your interrupts again. But ACPI's also heavily involved in the runtime management of the system, and it's necessary for there to be a mechanism for the hardware to alert the OS of events. ACPI handles this case by providing a set of general purpose events (GPEs). The implementation of these is fairly straightforward - an ACPI table points at a defined system resource (typically an area of system io space, though in principle it could be something like mmio instead), and when the hardware fires an ACPI interrupt the kernel looks at this region to see which GPEs are flagged.
82 Billion Objects in Amazon S3
At the end of Q3 2009 we counted over 82 billion objects in Amazon S3. Congrats to the team for providing such a rock solid service! When looking at the graph keep in mind that the first 4 markers are a year apart, but the last one only 6 months.
Looking to the past
It's an oft-voiced suggestion that rather than looking at the bad things that happen in our communities, we should focus on the good things. There's a number of highly successful geek women already ¿ should we not be concentrating on encouraging more of them, rather than scaring people away with tales of thoughtlessness, discrimination and outright abuse?Let's draw an analogy. One day, a $20 charge appears on your credit card. You didn't make it. You report it to your credit card company, who assure you that they take fraud seriously and then do nothing. A few days later, another $20 charge. Your credit card company tells you that such events are rare, unrepresentative of the general credit card experience and continue to do nothing.
Autumnal Blues
For the last couple of weeks, I've been really sick; possibly H1N1 (chills, headaches, body pain) but possibly just a really bad cold. Also Lauren's mother died and she had to go away to help so I was in weakened-single-Dad mode. I have explored some new dimensions in fatigue. I might have got better faster if I'd been able to go to bed instead of beating my fledgling Clojure skillz to a bloody pulp against 45,275,432,643 bytes in 218,201,129 lines of test data. I have thoroughly investigated some things that turn out not to work at all but been pleasantly surprised by some others that shouldn't but do.
John Mueller on Zazi
I have refrained from commenting on the case against Najibullah Zazi, simply because it's so often the case that the details reported in the press have very little do with reality. My suspicion was, that as in in so many other cases, he was an idiot who couldn't do any real harm and was turned into a bogeyman for political...
Links for Monday, November 9, 2009
Pixels and Policy: How Tough Economic Times are Encouraging Virtual Workplaces - “There are obvious cost benefits to working in the virtual world that stand out at a time of global recession and rolling bankruptcy. Travel costs are reduced to zero as avatars from around the world come together in one central space.“ Second Life Blog: [...]
Laissez-Faire Access Control
Recently I wrote about the difficulty of making role-based access control work, and how reasearch at Dartmouth showed that it was better to let people take the access control they need to do their jobs, and audit the results. This interesting paper, "Laissez-Faire File Sharing," tries to formalize the sort of access control. Abstract: When organizations deploy file systems with...
When not to outsource
There is an interesting case study for Blank Label shirts in the latest edition of Forbes magazine. As a new venture designed from scratch, Blank Label outsourced both manufacturing of the shirts and the design of its web site. After months of troubles, now only the manufacturing is outsourced. The owner of the company found it too difficult to specify exactly what he wanted on the site and so design decisions were left in the hands of the development team, who invariably made poor choices. It's a nice example of what can be easily specified and outsourced and what's more difficult to pin down.
Sunday, 8 November 2009
cvr2 has died again. Rebooted and it came up from the second disk, which made it look very much like there was something wrong with the first one. Removed the second disk (on second try; the first time I removed the wrong disk), and it came up fine, thank God—I was half expecting a repeat of the XFS offset superblock bug. This time was probably due to overheating. But why do all these Linux distributions have splash screens that hide the boot messages? The only time you ever look at a booting machine is if you want to know what's going on. <
Toronto Star on the Makers launch
Today's Toronto Star has a good piece about my upcoming book-launch for Makers in Toronto on Nov 12, and on the politics that informs my work. Cory Doctorow will have mixed feelings when the news reaches him that employees of the Toronto Public Library system will not be on strike this week. The U.K.-based author and blogger [...]
Mr. Postman
The mailbox in my building is broken. Nobody can remember it being any other way. The lock is busted so anyone in the building can get access to every apartment's individual boxes in the same way that the mailman does. It's not a huge problem since there are only four apartments in the building and the box is behind a locked door to the street. I saw the mailman come one day to deliver mail. He used a key to unlock a box on the outside of the building from which he retrieved a key to first unlock the outside door and then another to "unlock" the mailbox.
Conversation with Butler Lampson at SOSP 2009
Just about exactly one year ago, I posted a summary and the slides from an excellent Butler Lampson talk: The Uses of Computers: What's Past is Merely Prologue. Its time for another installment. Butler was at SOSP 2009 a few weeks back and Marvin Theimer caught up with him for a wide ranging discussion on distributed systems. With Butler's permission, what follows are Marvin's notes from the discussion. Contrast cars with airplanes: when the fancy electronic systems fail you (most-of-the-time) can pull a car over to the side of the road and safely get out whereas an airplane will crash-and-burn.
Interview with Off the Hook/WBAI about Makers
Here's a radio interview I did recently with Off the Hook, the 2600 show, on New York's WBAI, talking about MAKERS, maker politics, and the state of the world. MP3 Link
My Locus column on sex in YA fiction
My latest Locus column, "Teen Sex," explains why I think young adult literature should have sex -- and other "mature" topics -- in it. There's really only one question: "Why have your characters done something that is likely to upset their parents, and why don't you punish them for doing this?" Now, the answer. First, because teenagers have [...]
Friday, 6 November 2009
The Doghouse: ADE 651
A divining rod to find explosives in Iraq: ATSC's promotional material claims that its device can find guns, ammunition, drugs, truffles, human bodies and even contraband ivory at distances up to a kilometer, underground, through walls, underwater or even from airplanes three miles high. The device works on “electrostatic magnetic ion attraction,” ATSC says. To detect materials, the operator puts...
Mossad Hacked Syrian Official's Computer
It was unattended in a hotel room at the time: Israel's Mossad espionage agency used Trojan Horse programs to gather intelligence about a nuclear facility in Syria the Israel Defense Forces destroyed in 2007, the German magazine Der Spiegel reported Monday. According to the magazine, Mossad agents in London planted the malware on the computer of a Syrian official who...
Upgrade your career
Do you like your job? Do you enjoy the people you work with? Would you want to have lunch with them? Every day? Alex Papadimoulis thinks that FogTyler Griffin Hicks-Wright Creek's free lunches are “cultish,” but everyone at Fog Creek loves them. Maybe it's the mandatory brain implant we install in each new worker, but I like to think that we just enjoy eating together because we genuinely like each other and like spending time together. If you can't imagine eating lunch every day with your coworkers, I hate to break it to you: you might not like them. Is it OK to spend most of your waking hours with people you don't like?
The Problems with Unscientific Security
From the Open Access Journal of Forensic Psychology, by a whole llist of authors: "A Call for Evidence-Based Security Tools": Abstract: Since the 2001 attacks on the twin towers, policies on security have changed drastically, bringing about an increased need for tools that allow for the detection of deception. Many of the solutions offered today, however, lack scientific underpinning. We...
Other Concurrency Sessions at PDC09
I mentioned yesterday that I'll be involved in two sessions at PDC09, including a parallel patterns tutorial. I know many of you are interested in concurrency in general and on Microsoft platforms in particular, so I thought I'd share this more complete list of concurrency-related sessions at PDC, put together by my colleague Stephen Toub. Overview: The [...]
Fear and Overreaction
It's hard work being prey. Watch the birds at a feeder. They're constantly on alert, and will fly away from food -- from easy nutrition -- at the slightest movement or sound. Given that I've never, ever seen a bird plucked from a feeder by a predator, it seems like a whole lot of wasted effort against not very big...
Radical Presentism in Tin House
Tin House, a literary magazine, asked me to introduce the current science fiction issue with an overview of the field. I wrote them an essay called "Radical Presentism," about the way that science fiction reflects the present more than the future. Mary Shelley wasn't worried about reanimated corpses stalking Europe, but by casting a technological innovation [...]
PDC'09: Tutorial & Panel
For those of you coming to PDC'09 in Los Angeles a couple of weeks from now, I'll be there for a few hours on Monday and Wednesday participating in two events: Patterns of Parallel Programming: A Tutorial on Fundamental Patterns and Practices for Parallelism. The full-day tutorial is full of useful information. I'll be giving the [...]
Does Slow Growth Equal Slow Death?
My new Inc. column is up. “For a guy who wrote a book on how to hire great programmers, it's mortifying how incompetent I've been at enlarging the sales team, which, right now, consists of one terrific account executive and a dog. (I'm just kidding. There's no dog.)” Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.
Java Store ß: payment and a new client
Put an accountant, a lawyer, an MBA and a software engineer together into a room... Sounds like the lead-in to a bad joke, but it's the exercise that the Java Store team has been living through for the past several months. At the PayPal conference today Eric Klein did an announcement and demo of the next phase in the Java Store's development. We've been working with PayPal on this for some time, using their new PayPal X platform. It always amazes me how complex it is to deal with all the details of global finance. And even so, the store today only handles US issues.
Tuesday, 3 November 2009
While checking links, discovered a new firefox page, advertising the new release 3.5. I hope it renders things better than 3.0.4 that I'm using at the moment. Here's firefox rendering its own web pages: It's quite possible that this is a firefox configuration problem, but it's certainly not an obvious one. Wine problems: shm related? Discussing my problems with Ashampoo photo optimizer and wine.
Upcoming Gig – Scottish Ruby
Next March, I'll visit Edinburgh (for the first time!) to keynote the Scottish Ruby Conference, along with the excellent Jim Weirich. Interestingly, this is the former Scotland On Rails; they've decided to broaden their spectrum.
Clojure N00b Tips
Clojure is the new hotness among people who think the JVM is an interesting platform for post-Java languages, and for people who think there's still life in that ol' Lisp beast, and for people who worry about concurrency and state in the context of the multicore future. Over the last few days I've been severely bipolar about Clojure, swinging from “way cool!” to “am I really that stupid?” Herewith some getting-started tips for newbies like me. This is almost certainly not interesting to anyone except those who are already interested in Clojure, and to one other group: those who might want to package up a programming language in such a way that learning it will be straightforward.
Zero-Tolerance Policies
Recent stories have documented the ridiculous effects of zero-tolerance weapons policies in a Delaware school district: a first-grader expelled for taking a camping utensil to school, a 13-year-old expelled after another student dropped a pocketknife in his lap, and a seventh-grader expelled for cutting paper with a utility knife for a class project. Where's the common sense? the editorials cry....
Meta-Microblogging
So I don't tweet because I'm not ready to hand my data and autonomy over to Twitter. Luckily -- or unluckily perhaps -- that hasn't kept me off the microblogging wagon. I "dent" semi-regularly over at freedom-friendly identi.ca. I've found that microblogging is a great public outlet where one can talk about all those otherwise little meaningless things that we all do in our daily lives. High on my list of meaningless little actions, however, is microblogging itself! But can you microblog about your microblogging -- i.e. can you "metamicroblog" (or "metadent", or "metatweet")?
One Size Does Not Fit All
Last week AWS announced the Amazon Relational Database Service (Amazon RDS) and I blogged that it was big step forward for the cloud storage world: Amazon RDS, More Memory, and Lower Prices. This really is an important step forward in that a huge percentage of commercial applications are written to depend upon Relational Databases. But, I was a bit surprised to get a couple of notes asking about the status of Simple DB and whether the new service was a replacement. These questions were perhaps best characterized by the forum thread The End is Nigh for SimpleDB.
Detecting Terrorists by Smelling Fear
Really: The technology relies on recognising a pheromone - or scent signal - produced in sweat when a person is scared. Researchers hope the 'fear detector' will make it possible to identify individuals at check points who are up to no good. Terrorists with murder in mind, drug smugglers, or criminals on the run are likely to be very fearful...
Monday, 2 November 2009
I've noted in the past severe performance issues running the Ashampoo photo optimizer, a Microsoft-based program, with wine. They're clearly related to the X interface: it's the X server that uses up all the time, sometimes as much as several seconds just to select a box on the window. The problem just “went away” fairly soon after I installed it—otherwise I probably would have given up—but it has returned a few times recently. Today it took over a minute of X server CPU time for the initial screen to complete building, and the remainder was just as slow. Just for the fun of it, I shot down another resource hog, firefox, and the problem went away.
Epoch, Part 7
Here's the seventh installment of a story-in-progress, Epoch, commissioned by Mark Shuttleworth for my forthcoming short story collection WITH A LITTLE HELP. MP3 Link
The FBI and Wiretaps
To aid their Wall Street investigations, the FBI used DCSNet, their massive surveillance system. Prosecutors are using the FBI's massive surveillance system, DCSNet, which stands for Digital Collection System Network. According to Wired magazine, this system connects FBI wiretapping rooms to switches controlled by traditional land-line operators, internet-telephony providers and cellular companies. It can be used to instantly wiretap almost...
ZFS Deduplication
You knew this day was coming: ZFS now has built-in deduplication. If you already know what dedup is and why you want it, you can skip the next couple of sections. For everyone else, let's start with a little background. What is it? Deduplication is the process of eliminating duplicate copies of data. Dedup is generally either file-level, block-level, or byte-level. Chunks of data -- files, blocks, or byte ranges -- are checksummed using some hash function that uniquely identifies data with very high probability. When using a secure hash like SHA256, the probability of a hash collision is about 2^-256 = 10^-77 or, in more familiar notation, 0.00000000000000000000000000000000000000000000000000000000000000000000000000001.
Speaking this week in Cambridge, Sheffield
I'm giving two talks in the UK this week -- the first in Cambridge, as part of the Arcadia Seminar, held at Robinson College; the second is at Sheffield, as part of the DocFest premiere of RIP: A Remix Manifesto, a documentary on copyfighting and art that features some interviews with me. Hope to see [...]
Caching Bites Me Again
I'm on a roll. Two weird caching problems in the space of a week. This one started off looking like a scary security bug. During testing, our QA department observed the following sequence of events: Log into the system as user U1. View page P1 Log out Log into the system as user U2. View page that re-directs to P1. At this point, they were indeed on page P1, but it was showing the data from user U1. My first suspicion was that somehow browser cookies were not being cleared and we somehow managing to pick up the sessions cookies from the initial login.
Figuring out what your company is all about
What is your company about? Recently I got inspired by Kathy Sierra, whose blog Creating Passionate Users and Head First series of books revolutionized developer education. She kept saying the same thing again and again: help your users be awesome. Kathy taught me that if you can't explain your mission in the form, “We help $TYPE_OF_PERSON be awesome at $THING,” you are not going to have passionate users. What's your tagline? Can you fit it into that template? It took us nine years, but we finally worked out what Fog Creek Software is all about, which I'll tell you in a moment, but first, some backstory.
Adam Bosworth on standards
Adam Bosworth: “All successful standards are as simple as possible, not as hard as possible.” Required reading. Need to hire a really great programmer? Want a job that doesn't drive you crazy? Visit the Joel on Software Job Board: Great software jobs, great people.
