Blog Archive: March 2016
Serverlessness
Today Microsoft announced Azure Functions, joining Google Cloud Functions and (from 2014) AWS Lambda. This is fun stuff, and might be a big deal. In the beginning was the server. The last one I ever bought was from Dell; I can remember hauling it out of the cardboard and sliding it into the rack. I pinched a finger badly enough that there was blood on the floor. Then I had to argue with the sysadmin over Debian vs Red Hat. So, now that we have cloud biggies like the three companies named in the first paragraph [disclosure, I work for AWS] nobody has to shed blood deploying servers.
"How SysAdmins Devalue Themselves" now in CACM
CACM reprinted my article in the April edition. They don't usually publish April Fools articles, but I'll consider this the appropriate place for this article. If you subscribe to CACM, you can read the article online, PDF, Ebook. You can also read it in the original publication, ACM Queue for free.
Archer starts tonight! (and other important TV news)
The new season of Archer starts tonight!! (Mar 31, 2016) at 10pm! The Powerpuff Girls 2016 Reboot starts on April 4th. Set your DVRs now! The Detour starts on April 11. I have high hopes for this show. It is created by Samantha Bee and Jason Jones. Speaking of Samantha Bee, her new weekly news program Full Frontal with Samantha Bee is my new favorite show. I think it shows that she should have replaced Jon Stewart. Silicon Valley's season 3 premieres April 24. If you work at a startup, or just wish you did, you can't miss this show. ProTip: If you DVR can only set up new recordings 14 days in advance, set a reminder in your April 10 todo list.
ISIS Encryption Opsec
Tidbits from the New York Times: The final phase of Mr. Hame's training took place at an Internet cafe in Raqqa, where an Islamic State computer specialist handed him a USB key. It contained CCleaner, a program used to erase a user's online history on a given computer, as well as TrueCrypt, an encryption program that was widely available at...
And now for something completely different
Enough Vigra frustration for one day. How about Microsoft Windows 10 to round it off? I had installed it on a VM somewhere, and contrary to my expectations it still wanted me to activate it. How do you do that on a VM? Finally found an old, retired activation key and typed it in: Can't they do anything right? ACM only downloads articles once.
More ports pain
A while back Rainer Hurling sent me mail with some issues relating to the Vigra port. I sent him some patches, but didn't hear back from him, so time to try building it again on FreeBSD -CURRENT. It was like pulling teeth. I did this on my VM current.lemis.com, which I have deliberately kept as bare-bones as possible. That meant building all the dependencies first. But many of them wouldn't build because they were missing dependencies! cmake failed for reasons I forget. When I tried to install the package, that failed too: # pkg install cmake ...
More ports pain
A while back Rainer Hurling sent me mail with some issues relating to the Vigra port. I sent him some patches, but didn't hear back from him, so time to try building it again on FreeBSD -CURRENT. It was like pulling teeth. I did this on my VM current.lemis.com, which I have deliberately kept as bare-bones as possible. That meant building all the dependencies first. But many of them wouldn't build because they were missing dependencies! cmake failed for reasons I forget. When I tried to install the package, that failed too: # pkg install cmake ...
Lawful Hacking and Continuing Vulnerabilities
The FBI's legal battle with Apple is over, but the way it ended may not be good news for anyone. Federal agents had been seeking to compel Apple to break the security of an iPhone 5c that had been used by one of the San Bernardino, Calif., terrorists. Apple had been fighting a court order to cooperate with the FBI,...
David Patterson Retires After 40 Years
David Patterson has had a phenomenal impact on computer architecture and computer science over the last 40 years. Hes perhaps most notable for the industry impact of the projects hes led over these years. I first got to know his work back when the Berkeley Reduced Instruction Set Computer project started publishing. The RISC project...
Mass Surveillance Silences Minority Opinions
Research paper: Elizabeth Stoycheff, "Under Surveillance: Examining Facebook's Spiral of Silence Effects in the Wake of NSA Internet Monitoring": Abstract: Since Edward Snowden exposed the National Security Agency's use of controversial online surveillance programs in 2013, there has been widespread speculation about the potentially deleterious effects of online government monitoring. This study explores how perceptions and justification of surveillance practices...
Epidemic of Vacation Shaming Spreads Across America
This HuffPo article is worth reading: According to an Ernst & Young internal study, for "each additional 10 hours of vacation employees took, their year-end performance ratings improved 8 percent, and frequent vacationers also were significantly less likely to leave the firm". By the way... using your vacation time one day here and a long weekend there is not a "vacation". Generally I find that you're brain doesn't relax until day 3 of a vacation, especially if there is travel involved. If you don't take a long break you never reach that point. If you take a lot of long weekends you end up spending that time doing laundry and you rob yourself of the opportunity to actually relax.
Watch us live today! LISA Conversations: Caskey Dickson on "Why Your Manager LOVES Technical Debt and What to Do About It"
Today (Tuesday, March 29, 2016) we'll be recording episode #8 of LISA Conversations. Join the Google Hangout and submit questions live via this link. Our guest will be Caskey Dickson. We'll be discussing his talk Why Your Manager LOVES Technical Debt and What to Do About It from LISA '15. The video we'll be discussing: Why Your Manager LOVES Technical Debt and What to Do About It Caskey Dickson Recorded at LISA '15 Video and Slides Watch us record the episode live! Tuesday, March 29, 2016 at 3:30-4:30 p.m. Pacific Time (convert) LISA Conversations Episode #8 Co-hosts: Lee Damon and Thomas Limoncelli Guest: Caskey Dickson Join us.
Mobile Aerials
A little while ago I tweeted One thing phone-cams arent much good for is shooting out airplane windows. Since then, Ive noticed my Nexus 5X looking at me in a hurt sort of way. It seems I was wrong. These are both taken somewhere over Australia.
A 1976 Congressional Report on Surveillance
Here's a 1,300-page Congressional report on "surveillance technology" from 1976....
Power on the Internet
Interesting paper: Yochai Benkler, "Degrees of Freedom, Dimensions of Power," Daedelus, winter 2016: Abstract: The original Internet design combined technical, organizational, and cultural characteristics that decentralized power along diverse dimensions. Decentralized institutional, technical, and market power maximized freedom to operate and innovate at the expense of control. Market developments have introduced new points of control. Mobile and cloud computing, the...
Lenses and Cameras in 2016
Im on the way back from a couple of weeks in Australia, and of course Pictures Were Taken. I brought almost all my photo-gear but used it very unevenly; concluded that I have too many lenses, and was left wondering whether you really even need a camera any more. Herewith notes illustrated with Pacific-ocean (mostly) pictures. Numbers Dont know how many I took, but among the photos I kept, 14 were with the Nexus 5X, 119 with the Fujifilm X-T1. Lens stats for the latter: Fuji 35mm 58 Fuji 55-200mm 26 Pentax 50mm 23 Fuji 18-55 4 Pentax 100mm 4 Fuji 10-24mm 4 Wide angle lenses?
Subversion subverted?
More investigation of my build failures today. Simply recreating all files didn't help. How about a complete new working copy? Tried that and was amazed: there were dozens of differences. Why? svn info showed nothing of interest. In particular, the repository root and UUID were the same. And the contents? 60 MB of differences! === grog@eureka (/dev/pts/19) /src/FreeBSD/svn 79 -> diff -wur head-2 head 2>&1 > svn-diffs === grog@eureka (/dev/pts/19) /src/FreeBSD/svn 80 -> l svn-diffs -rw-r--r-- 1 grog lemis 60,039,508 26 Mar 10:42 svn-diffs Strangely some of them seem to be positively ancient. Here head is the new checkout, while head-2 is the old one: --- head-2/sys/boot/sparc64/boot1/_start.s 2016-03-26 08:42:17.953474000 +1100 +++ head/sys/boot/sparc64/boot1/_start.s 2016-03-26 09:42:42.604479000 +1100 @@ -1,4 +1,4 @@ -/* $FreeBSD: head/sys/boot/sparc64/boot1/_start.s 125717 2004-02-11 21:17:04Z ru $ */ +/* $FreeBSD: head/sys/boot/sparc64/boot1/_start.s 93311 2002-03-28 02:41:52Z obrien $ */ ...
Upgrading test boxes
Now that I have resolved my difficulties with VirtualBox, the next step was to upgrade current to the present day. Not an issue: I may not have been keeping a FreeBSD CURRENT system up to date, but I do have a cron job which keeps an up-to-date copy of the source repository. So all I needed was: # cd /usr/src # make buildworld But it failed, repeatedly: /eureka/home/src/FreeBSD/svn/head/lib/clang/libllvmmipsdesc/../../../contrib/llvm/../../lib/clang/include/llvm/IR/Intrinsics.gen:2:10: fatal error: 'Intrinsics.inc.h' file not found Problems with the repository or problems with the snapshot?
Memphis Airport Inadvertently Gets Security Right
A local newspaper recently tested airport security at Memphis Airport: Our crew sat for 30 minutes in the passenger drop-off area Tuesday without a word from anyone, and that raised a number of eyebrows. Certainly raised mine. Here's my question: why is that a bad thing? If you're worried about a car bomb, why do you think length of time...
Reminder: Do your homework for next week's LISA Conversations: Caskey Dickson on "Why Your Manager LOVES Technical Debt and What to Do About It"
This weekend is a good time to watch the video we'll be discussing on the next episode of LISA conversations: Caskey Dickson's talk from LISA '15 titled Why Your Manager LOVES Technical Debt and What to Do About It. Homework: Watch his talk ahead of time. Why Your Manager LOVES Technical Debt and What to Do About It Recorded at LISA '15 Video and Slides Then you'll be prepared when we record the episode on Tuesday, March 29, 2016 at 3:30-4:30 p.m. Pacific Time (convert). Register (optional) and watch via this link. Watching live makes it possible to participate in the Q&A.
Interesting Lottery Terminal Hack
It was a manipulation of the terminals. The 5 Card Cash game was suspended in November after Connecticut Lottery and state Department of Consumer Protection officials noticed there were more winning tickets than the game's parameters should have allowed. The game remains suspended. An investigation determined that some lottery retailers were manipulating lottery machines to print more instant winner tickets...
VirtualBox again
Why am I going to all this trouble to set up physical test boxes? Because I've had difficulties with VirtualBox, mainly. But that should be soluble, and so I had another attempt today. This diary helps, and I noted that it was less than 2 months since my last attempt. But I didn't read it as carefully as I should have: Off to find out from Google, but I couldn't access the Internet: the link wasn't working. Nothing obvious: the interface looked right, routing looked right, but there was no traffic.
Making dental appointments
After my dental appointment, I needed make an appointment for a checkup in about 6 months' time. No worries, they'll send me an SMS. Sorry, don't do SMS. OK, they'll send dead tree mail. And Email? Already dead? ACM only downloads articles once. It's possible that this article has changed since being downloaded, but the only way you can find out is by looking at the original article.
FBI vs. Apple: Who Is Helping the FBI?
On Monday, the FBI asked the court for a two-week delay in a scheduled hearing on the San Bernardino iPhone case, because some "third party" approached it with a way into the phone. It wanted time to test this access method. Who approached the FBI? We have no idea. I have avoided speculation because the story makes no sense. Why...
Cryptography Is Harder Than It Looks
Writing a magazine column is always an exercise in time travel. I'm writing these words in early December. You're reading them in February. This means anything that's news as I write this will be old hat in two months, and anything that's news to you hasn't happened yet as I'm writing. This past November, a group of researchers found some...
More ports pain
I have now committed the latest version of Hugin to the FreeBSD ports tree. And again I've run into problems: new dependencies that I should have known, and for some reason it doesn't build on FreeBSD 9.3: cd /wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/tools && /usr/local/bin/g++48 -DGLEW_STATIC -DHUGIN_HSI -I/wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src -I/wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/hugin_base-I/wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/celeste -I/usr/local/include -I/usr/local/include/OpenEXR -I/wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/foreign -I/usr/local/include/python2.7 -O2 -pipe -Wl,-rpath=/usr/local/lib/gcc48 -fno-strict-aliasing-Wl,-rpath=/usr/local/lib/gcc48 --std=c++11 -O2 -pipe -Wl,-rpath=/usr/local/lib/gcc48 -fno-strict-aliasing -Wl,-rpath=/usr/local/lib/gcc48 -fopenmp -o CMakeFiles/pto_var.dir/ParseExp.cpp.o -c /wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/tools/ParseExp.cpp/wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/tools/ParseExp.cpp: In function 'bool Parser::ConvertToRPN(const string&, const ConstantMap&, std::queue<Parser::RPNTokens::TokenBase*>&)': /wrkdirs/usr/ports/graphics/hugin-2016/work/hugin-2016.0.0/src/tools/ParseExp.cpp:371:23: error: 'stod' is not a member of 'std' val = std::stod(expression.substr(pos), &index); How I hate C++!
Power interruptions: enough is enough
Round 14:50 this afternoon the office UPS started beeping. Out into the garage, and sure enough (thanks, Jim Lannen) the RCD had tripped, the sixth time this year. Turned it on and all was well. Well, almost. My ATA showed an error indication. No worries, that's why I have a spare. Connected it up. It worked, but I discovered that I hadn't configured it as well as I had thought. So: ATA or power supply? Connected the old ATA to the new power supply, and sure enough, it worked. So the power supply died round the time of the power issue.
FBI's Cyber Most Wanted List
The FBI just added two members of the Syrian Electronic Army to its cyber most-wanted list. I had no idea that the FBI had a cyber most-wanted list....
1981 US Document on Encryption Policy
This was newly released under FOIA at my request: Victor C. Williams, Jr., Donn B. Parker, and Charles C. Wood, "Impacts of Federal Policy Options for Nonmilitary Cryptography," NTIA-CR-81-10, National Telecommunications and Information Administration, US. Department of Commerce, June 1981. It argues that cryptography is an important enabling technology. At this point, it's only of historical value....
30% off OReillys Open Source Convention in Austin, May 16-19
O’Reilly’s venerable, essential OSCON is in Austin, Texas this year, meaning that you’ll get to combine brain-thumpingly good talks and workshops of free/open source tools and techniques with some of the world’s best BBQ, millions of bats, my favorite toy store anywhere, and one of the best indie bookstores you could hope to visit. I’m... more
Wasting one million dollars
"Done" means "launched". It isn't "done" until it is launched. It annoys me to hear people say a project is "done... now I just have to launch it". It isn't done if it isn't in production. There are a few reasons for this: people think that launch is "the last 5 percent of a project" but often 80 percent of your time will be consumed by this last 5 percent. Also, you aren't "done" until other people are benefitting from your work (in business speak... " it is delivering value"). Written code has no business value. Launched code does. You can rig this in your favor.
Observations on the Surveillance that Resulted in the Capture of Salah Abdeslam
Interesting analysis from The Grugq: Bottom Line Up Front Intelligence agencies must cooperate more rapidly and proactively to counter ISIS' rapid and haphazard operational tempo. Clandestine operatives must rely on support networks that include overt members of the public. These networks are easily mapped out based on metadata available to nation state level security forces. Fugitives should learn to cook...
Shopping in the Internet era
Two years ago I wrote an essay on The Internet in 2034. Surprise, surprise, 10% of the time has gone by already. There I wrote: Most purchases will occur on-line, and the few remaining shops will mainly exist to order and supply goods available on the Internet. The exceptions will be fresh goods such as food and some services where a view of the items in advance is desired, such as some clothes. This will also have a profound effect on the economy: many companies, notably shops and mall owners, will go bankrupt.
iMessage Encryption Flaw Found and Fixed
Matthew Green and team found and reported a significant iMessage encryption flaw last year. Green suspected there might be a flaw in iMessage last year after he read an Apple security guide describing the encryption process and it struck him as weak. He said he alerted the firm's engineers to his concern. When a few months passed and the flaw...
Brennan Center Report on NSA Overseas Spying and Executive Order 12333
The Brennan Center has released a report on EO 12333, the executive order that regulates the NSA's overseas surveillance. Much of what the NSA does here is secret and, even though the EO is designed for foreign surveillance, Americans are regularly swept up in the NSA's collection operations: Despite a series of significant disclosures, the scope of these operations, as...
Ports install hell
So tiwi's MySQL installation is broken, possibly beyond repair. Since I don't need any of the data in the database, it seemed easier to remove and reinstall it. Tried that today: === root@tiwi (/dev/pts/6) /var/db 21 -> pkg install mysql57-server The following 3 package(s) will be affected (of 0 checked): New packages to be INSTALLED: mysql57-server: 5.7.10_3 mysql57-client: 5.7.10_4 mysql56-client: 5.6.27 Huh? Those numbers are a representation of the version number (5.6 or 5.7). Why should the server version 5.7 install two different clients, one for a different release?
tiwi installation, continued
Yesterday's attempt to install the tuner in tiwi ended with database issues. Before attending to them, considered the cheap and easy approach described on the webcamd page: # # The following command will display webcam contents from /dev/video0 # by default. # pwcview OK, tried that: === grog@tiwi (/dev/pts/2) ~ 1 -> pwcview Failed to access webcam: No such file or directory *********************************************************** Make sure you have connected your webcam to the root hub or to a USB 1.1 hub, also check your dmesg for any errors.
Old file systems
Warren Toomey sent out a message to the Unix heritage society today asking for examples of file systems in the late 1960s. I knew one: the Master File Directory of the Univac 1108. The last time I used it was some time in September 1974, and I don't recall very much about it. But the Programmers Reference Manual is available online, and it contains a description of the MFD, starting on page 333 of the scanunfortunately from such a completely different perspective that it's hard to compare. But Warren's question was whether it was customary to keep the file names separate from other file metadata (inodes in Unix parlance), and it seems that that wasn't the case.
Power failure recovery
Recovering from power failures is getting easier now that I have so much practice. When I came in this morning, eureka was mainly up and running. But I couldn't access my local web cache. Once again it turned out that this was a DHCP issue: after a power outage, dhclient overwrites /etc/resolv.conf with data from the remote DHCP server, not what I want, and in this case it resulted in the external view of my DNS, not the same as the local view served by the local name server. To fix it I needed to check out my local version again and restart squid.
Friday Squid Blogging: Braised Squid With Harissa and Olives
Recommended recipe. As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered....
I'm a Wall Street Journal Acrostic Answer
A quote from Data and Goliath is the answer to a Wall Street Journal acrostic. It's not the same as being a New York Times crossword puzzle answer, but it's close....
Parrot Story
Were visiting friends in Australia and I watched a pair of parrots interact; photographed them, but didnt understand. Oh, and a koala. This was actually at a Koala Reserve on Phillip Island, not terribly far from Melbourne; my 9-year-old insisted. Koala-watching has a Wheres-Waldo flavor, since they are usually asleep, wedged into a tree-fork near where they last stopped eating. So I wandering around trying to spot the stars of the show, and discovered these charming birds. There was interaction going on; one parrot was occasionally snuggling and nuzzling the other totally cute. I looked away and suddenly, with a discordant bird-noise outbreak, one of them was flying away, while the one left behind had expanded his-or-her crest.
Companies Handing Source Code Over to Governments
ZDNet has an article on US government pressure on software companies to hand over copies of their source code. There's no details because no one is talking on the record, but I also believe that this is happening. When asked, a spokesperson for the Justice Dept. acknowledged that the department has demanded source code and private encryption keys before. These...
Maintain Separate GitHub accounts
Someone recently commented that with Github it is "a pain if you want to have a work and personal identity." It is? I've had separate work and personal Github accounts for years. I thought everyone knew this trick. When I clone a URL like [email protected]:TomOnTime/tomutils.git I simply change github to either github-home or github-work. Then I have my ~/.ssh/config file set with those two names configured to use different keys: # TomOnTime / [email protected] Host home-github.com HostName github.com User git IdentityFile ~/.ssh/id_rsa-githubpersonal PreferredAuthentications publickey PasswordAuthentication no IdentitiesOnly yes # tlimoncelli / [email protected] Host work-github.com HostName github.com User git IdentityFile ~/.ssh/id_rsa-githubwork PreferredAuthentications publickey PasswordAuthentication no IdentitiesOnly yes I also have things set up so that if I leave the name alone, my work-owned machines default to the work key, and my personal machines default to my personal key.
Verifying the Substitution Cipher Folklore
A substitution cipher has each letter substituted with another. Cryptography folklore has it that simple substitution ciphers are trivial to break by looking at the letter frequencies of the encrypted text. I tested the folklore and the results were not quite what I was expecting.
Finally! The tuner!
I've been waiting for a USB tuner for tiwi since the beginning of the year. The first one I ordered never materialized, and I ordered a second one on 12 February. It has taken until now for it to arrive! Off to Napoleons to pick it up. Surprise, surprise! Wrong connector! When I ordered the original tuner, I went to some lengths to ensure it had a 75© antenna connector. But when I ordered the replacement, I forgot, and it has an MCX connector. That's not the end of the world, but it means Yet Another Delay while I get the adapter.
Meaningless Anti-Virus Software Features Are Profitable
Tavis Ormandy, Google security expert, is getting press for criticizing Meaningless Antivirus Excellence Awards. This is a good opportunity to mention some thoughts I've had about anti-malware software. I believe that enterprise security defense software (anti-virus, anti-malware, host-based firewall, etc.) should have these qualities: Silent Updating: The software should update silently. It does not need to pop up a window to ask if the new antivirus blacklist should be downloaded and installed. That decision is made by system administrators centrally, not by the user. Hidden from view: The user should be able to determine that the software is activated, but it doesn't need an animated spinning status ball, nor popup windows to announce that updates were done.
New NIST Encryption Guidelines
NIST has published a draft of their new standard for encryption use: "NIST Special Publication 800-175B, Guideline for Using Cryptographic Standards in the Federal Government: Cryptographic Mechanisms." In it, the Escrowed Encryption Standard from the 1990s, FIPS-185, is no longer certified. And Skipjack, NSA's symmetric algorithm from the same period, will no longer be certified. I see nothing sinister about...
Screw optimism, we need hope instead
I wrote an essay called “Fuck Optimism” for a print project from F-Secure, about how we’ll make the Internet a 21st century electronic nervous system that serves humanity and stop it from being a tool to oppress, surveil and displace humans. In honor of Digital Freedom Month, F-Secure and Little Atoms have republished it online.... more
Another FBI Filing on the San Bernardino iPhone Case
The FBI's reply to Apple is more of a character assassination attempt than a legal argument. It's as if it only cares about public opinion at this point. Although notice the threat in footnote 9 on page 22: For the reasons discussed above, the FBI cannot itself modify the software on Farook's iPhone without access to the source code and...
Financial Cryptography 2016
Ross Anderson liveblogged this year's Financial Cryptography conference....
Possible Government Demand for WhatsApp Backdoor
The New York Times is reporting that WhatsApp, and its parent company Facebook, may be headed to court over encrypted chat data that the FBI can't decrypt. This case is fundamentally different from the Apple iPhone case. In that case, the FBI is demanding that Apple create a hacking tool to exploit an already existing vulnerability in the iPhone 5c,...
Towards understanding Olympus Maker Notes
Somehow software has become so complicated nowadays that you can't just hack away and incrementally create something: first you need a library a stack of twisty little libraries, all different, and you have to learn how to use them. My experience with libexif a couple of days ago didn't exactly fill me with confidence. How does it relate to yesterday's view of the structures? What I really want is something that doesn't try to understand and interpret too much of the data, just present it in vaguely dissected form. Enter my up-and-coming program exifdump, not exactly the only program of that name.
Tracing the exiftime bug
What caused the error messages from my exiftime script that I saw a couple of days ago? Today Yvonne borrowed my camera (after over 5 weeks her own camera still isn't back from repairs), and I put a couple of debug lines in the script. But it ran without problems. Later I took my own photos and the problem occurred again. Why? Does exiftime hate me? Ran wh, another little script that extends which to show all possible executables, and found: === grog@eureka (/dev/pts/27) ~/Photos/20160314 1078 -> wh exiftime 263011 -r-xr-xr-x 1 root wheel 47824 6 Mar 10:33 /usr/local/bin/exiftime 1910037 -r-xr-xr-x 1 grog lemis 448 7 Jul 2012 /Photos/Tools/exiftime Clearly the second one is my script.
Punishment and Trust
Interesting research: "Third-party punishment as a costly signal of trustworthiness, by Jillian J. Jordan, Moshe Hoffman, Paul Bloom,and David G. Rand, Nature: Abstract: Third-party punishment (TPP), in which unaffected observers punish selfishness, promotes cooperation by deterring defection. But why should individuals choose to bear the costs of punishing? We present a game theoretic model of TPP as a costly signal...
EXIF data: finally an overview
Further searching for the format of EXIF data has finally paid off, probably because it wasn't what I was looking for. I was looking for a description of TIFF format and came up with this page, which contains this illustration: And that's exactly the kind of diagram I'm looking for. Further investigation shows that there's also a similar page for JPEG, and that mentioned APP0 but not APP1. OK, we're making progress, so went looking for jpeg app1 file format and came up with this page, which looks like it will finally give the overview I've been looking for.
Analysis of Yemeni Cell Phone Metadata
This research shows the power of cell phone metadata. From an article by the author: Yemen has experienced an array of violent incidents and political turmoil in recent years, ranging from al Qaeda militant attacks to drone strikes, Arab Spring protests, and now Saudi Arabian air strikes. Call patterns can capture political or violent activities as they unravel in real...
EXIF: where's the Big Picture?
Spent a lot more time looking at processing EXIF data. There's lots of documentation of the individual components, though not all details (for example, what are the other components of the entries I looked at yesterday?) . But given a file, how do I dissect the structure? Wikipedia tells me that JPEG images store the data in APP1, segment marker 0xffe1, and TIFF images store it in a sub-IFD with the TIFF Private Tag 0x8825.
Strange exifcopy bug
As if I didn't have real problems understanding EXIF, today I got these messages when copying photos from the camera: mcopy -pm C:/dcim/100OLYMP/P3126248.ORF /photowork/P3126248.ORF exiftime: doesn't appear to be a JPEG file; searching for start of image exiftime: skipped spurious bytes in JPEG exiftime: start of image not found The first line is the command to copy the file from the camera, and exiftime is a script that sets the timestamps for the files once they have been read in.
Analysing Olympus MakerNotes
Spent a little time analysing the Olympus MakerNotes for one of my Hibiscus photos. exiftool renders the parts of interest like this: 0x0300 Zoom Step Count : 0 0x0301 Focus Step Count : 9481 0x0303 Focus Step Infinity : 1348 0x0304 Focus Step Near : 18597 0x0305 Focus Distance : 0.285 m 0x0308 AF Point : Left (or n/a) And that's all in the range 0x300 to 0x3ff.
Trip report: Winter ISO C++ standards meeting
On March 5, the ISO C++ committee completed its winter meeting in Jacksonville, FL, USA. We had record-tying attendance, with over 110 experts officially representing eight national bodies. As usual, we met for six days Monday through Saturday, and around the clock from 8:30am till 10pm most days, after which many people still went back […]
Trip report: Winter ISO C++ standards meeting
On March 5, the ISO C++ committee completed its winter meeting in Jacksonville, FL, USA. We had record-tying attendance, with over 110 experts officially representing eight national bodies. As usual, we met for six days Monday through Saturday, and around the clock from 8:30am till 10pm most days, after which many people still went back […]
Friday Squid Blogging: Squid Scientists on Tumblr
Really great Tumblr feed. As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered....
Next on LISA Conversations: Caskey Dickson on "Why Your Manager LOVES Technical Debt and What to Do About It"
Our next guest will be Caskey Dickson. We'll be discussing his talk from LISA '15 titled Why Your Manager LOVES Technical Debt and What to Do About It. Watch live! We'll be recording the episode on Tuesday, March 29, 2016 at 3:30-4:30 p.m. Pacific Time. Particpate in the live Q&A by submitting your questions during the broadcast. Pre-registration is recommended. Register and/or watch via this link. Homework: Watch his talk ahead of time: Why Your Manager LOVES Technical Debt and What to Do About It Recorded at LISA '15 Video and Slides Watch live! LISA Conversations Episode #8 Co-hosts: Lee Damon and Thomas Limoncelli Guest: Caskey Dickson Will be recorded: Tuesday, March 29, 2016 at 3:30-4:30 p.m.
10 Lessons from 10 Years of Amazon Web Services
The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Looking back over the past 10 years, there are hundreds of lessons that weve learned about building and operating services that need to be secure, reliable, scalable, with predictable performance at the lowest possible cost. Given that AWS is a pioneer in building and operating these services world-wide, these lessons have been of crucial importance to our business. As weve said many times before, There is no compression algorithm for experience. With over a million active customers per month, who in turn may serve hundreds of millions of their own customers, there is no lack of opportunities to gain more experience and perhaps no better environment for continuous improvement in the way we serve our customers.
10 Lessons from 10 Years of Amazon Web Services
The epoch of AWS is the launch of Amazon S3 on March 14, 2006, now almost 10 years ago. Looking back over the past 10 years, there are hundreds of lessons that we?ve learned about building and operating services that need to be secure, reliable, scalable, with predictable performance at the lowest possible cost.
Leaked ISIS Documents
Looks like tens of thousands of ISIS documents have been leaked. Where did they come from? We don't know: Documents listing the names of Islamic State fighters have been touted around the Middle East for months, dangled in front of media outlets for large sums of money. [...] Ramsay said he met the source of the documents in Turkey, an...
More EXIF investigations
Before I can do much more with my investigation of Olympus focus stacking, I need to establish if I can get more accurate focus information from the EXIF Maker Notes. First step was to look through the many alternatives to exiftool to see if they do any better. Tried exif, exiv2 and exiftags, all of which produced very little output and nothing at all about focus distance. My guess is that they don't analyse the Maker Notes. exifprobe does show some of the Maker Notes, but there's no mention of focus distance there either. OK, how difficult can it be to go through the EXIF data manually?
Espionage Tactics Against Tibetans
A Citizen Lab research study of Chinese attack and espionage tactics against Tibetan networks and users. This report describes the latest iteration in a long-running espionage campaign against the Tibetan community. We detail how the attackers continuously adapt their campaigns to their targets, shifting tactics from document-based malware to conventional phishing that draws on "inside" knowledge of community activities. This...
LISA Conversations Episode 7: Kris Buytaert on "DevOps: The past and future are here. It's just not evenly distributed (yet)"
Episode 7 of LISA Conversations is Kris Buytaert, who presented DevOps: The past and future are here. It's just not evenly distributed (yet) at LISA '11. Watch the Episode here: Usenix LISA Episode #7 with Kris Buytaert Co-hosts: Lee Damon and Thomas Limoncelli Guest: Kris Buytaert Recorded Tuesday, February 23, 2016 In this episode we discuss his talk: DevOps: The past and future are here. It's just not evenly distributed (yet) Recorded at LISA '11 Talk Description Video and Slides YouTube You won't want to miss this!
Hidden Credit Card Skimmers
New credit card skimmers are hidden inside the card readers, making them impossible to spot. EDITED TO ADD (3/11): Brian Krebs on this from over a year ago....
A Decade of Innovation
March 14, 2006 was the beginning of a new era in computing. That was the day that Amazon Web Services released the Simple Storage Service (S3). Technically, Simple Queuing Services was released earlier but it was the release of S3 that really lit the fire under cloud computing. I remember that day well. At the...
Still more focus stacking
Still more playing around with focus stacking today. Peter Jeremy asked why I took the photos from such a distance; that's clearly because they're big flowers, the lens has a ridiculously long focal length, and I want to get the whole flowers in the image. But clearly I can take photos of parts of the flower too. To do that, I needed flash, and fortunately the camera is prepared to wait for the flash to recharge (with the rather strange sequence 1, 2 or 4 seconds and more, but not the 3 I really need): Once again there seems to be little difference between these two images, though one was taken with a step size of 3, the other with a step ...
Plagiarism in Crossword Puzzles
Yet another fraud discovered through data analysis. EDITED TO ADD (3/11): More....
How to prepare for April Fools Day
Step 1. Buy this for your boss or coworkers. Step 2. Prepare for hijinks. #yourwelcome #dropthemic
Tom Reacts to Teens React to Windows 95
We're in the process of updating The Practice of System and Network Administration (read the drafts here) and I discovered an old section that was written with the assumption that DHCP was newish and readers would need encouragement to use it. Of course, I ripped it out and replaced it with something more modern. However, I couldn't help but include an explanation for new sysadmins what life was like before DHCP (see The Importance of DHCP). Which leads me to this video of teens reacting to Windows 95 (and associated article). The best quote is, "How do you get Internet without WiFi?"
Hacking Ukraine's Power Grid
This is an excellent article on the December hack of Ukraine's power grid....
Focus stacking, continued
Yesterday's experiments with focus stacking left a number of questions open. The first one was the influence of focus step setting. But when I turned on the camera and fought my way down the menus, it showed a step setting of 3, not 1. After a bit of messing around (and taking unintended single shots), I discovered that the menu system really requires me to press the OK button after making a setting. Otherwise the displayed value just goes away again. And that meant that my comparisons yesterday were in fact taken with the same focus step setting. So I took two sets of the same image, one at f/4 and the other at f/8, both with step size 10: Yes, ...
Everspan Optical Cold Stroage
Optical Archive Inc. was a startup founded by Frank Frankovsky and Gio Cogliatore. I first met Frank many years ago when he was Director of the Dell Data Center Solutions team. DCS was part of the massive Dell Computer company but they still ran like a startup. And, with Jimmy Pike leading many of their...
Desired program #16 not found in PAT: solved
Yesterday I discovered duplicate channel number entries in my mythconverg.channel table, and suspected that, along with a bug in MythTV, might explain my inability to record certain programmes. Today took another look. Sure enough, there were many: mysql> SELECT chanid, channum, sourceid, name, finetune, recpriority, visible, useonairguide, mplexid, serviceid FROM channel where mplexid = 1 ORDER BY channum; +--------+---------+----------+--------------------+----------+---------+---------------+---------+-----------+ | chanid | channum | sourceid | name | finetune | visible | useonairguide | mplexid | serviceid | +--------+---------+----------+--------------------+----------+---------+---------------+---------+-----------+ | 2008 | 8 | 2 | WIN Western Vic | 1 | 0 | 1 | 1 | 1 | | ...
Urbanity
Cities are our rule now, anything else the exception. Im biking most workdays, on concrete over the ocean into the stone heart of a small big city, getting ten dozen channels of nonstop urban input and every day I wonder where were all going. The future is distributed unevenly and cities concentrate the unevenness. The balconies and windows are full of stories. That buildings in the West End, a super-dense part of a pretty-dense city. As you can see, its not all pretty; the upscale condo developers, fueled (the story goes) by overseas capital, circle these towers like vultures, looking for a chance to evict retirees and immigrants and singles to build anew.
Chasing down TV reception issues
So what's wrong with cvr2 that it can't receive GEM any more? It can't be the tuner or the frequency, because there are five programmes that I am interested in (visible = 1) on this specific frequency (mplexid, i.e. multiplexer ID): mysql> SELECT chanid, channum, callsign, serviceid FROM channel WHERE mplexid = 1 AND visible = 1 ORDER BY channum; +--------+---------+--------------------+-----------+ | chanid | channum | callsign | serviceid | +--------+---------+--------------------+-----------+ | 2203 | 80 | WIN Western Vic HD | 1938 | | 2081 | 81 | WIN Western Vic | 1924 | | 2204 | 82 | GEM Western Vic | 1926 | | 2083 | 83 ...
More MythTV pain
There has been another reshuffle of the terrestial TV channels, with the result that a number of programmes didn't record. Time to run another mythtv-setup. But this time it didn't work: there were no error messages, but the window didn't display. More X breakage? No --listen-tcp, maybe? No, it's not that simple, but the only way I could get it to display was to connect cvr2 (the MythTV machine) to the TV and display it on its own X server, made more complicated by things like GNOME. And when I did, I couldn't navigate the screens properly. Finally I got my rescan done, but the old channel numbers were still there, and the only way I could tell them apart was because mythtv-setup is too polite to set the finetune attribute, so the new ones all had it set to NULL.
More Microsoft strangenesses
When firing up dischord to check the iview issues, it was, as so often, flat out doing something with the disk. Looking at the Task Manager showed some surprises: No, the representation 1,075... is no surprise. That's modern. But what's this MATLAB stuff? I don't use it, though it seems I have something of that nature on the machine. But it appears to only be an installer. So why is this thing messing around there? And what's ZPSAutoUpdate.exe? How I wish I understood this mess.
Using iview
Discussion on IRC yesterday: it seems there's a new series of The Doctor Blake Mysteries on ABC TV. It's set in Ballarat, and 3 years ago I tried to watch it, hampered at the time by abysmal TV reception. So I tried it with iview, ABC's web service. At the time I was refused because iview knows that I'm not in Australia. Yesterday I tried again: Now isn't that helpful? I sent a comment to ABC, and was surprised to get a response.
Whuffie would be a terrible currency
My latest Locus column, Wealth Inequality Is Even Worse in Reputation Economies, explains the ways in which “reputation” makes a poor form of currency — in a nutshell, reputation doesn’t fulfill most of the roles we expect from currency (store of value, unit of exchange, unit of account), and it is literally a popularity contest... more
Apple vs FBI: The privacy disaster is inevitable, but we can prevent the catastrophe
My new Guardian column, Forget Apple’s fight with the FBI our privacy catastrophe has only just begun, explains how surveillance advocates have changed their arguments: 20 years ago, they argued that the lack of commercial success for privacy tools showed that the public didn’t mind surveillance; today, they dismiss Apple’s use of cryptographic tools... more
New port: no issues?
Finally got round to committing a new FreeBSD port, graphics/hugin-2016, with the latest release candidate for Hugin. Went without a hitch. I was just saying (on IRC) I wonder what will go wrong now when I got a mail message, sent fully 2 minutes after the commit message: Looks like it was not repocopied, may I ask why? Well, the answer is that I didn't know it should have been. There's nothing about it in the Porter's Handbook. No, it seems, I should have been looking in the Committer's Guide, a document that I didn't even know existed, and which isn't referenced in the Porter's Handbook.
Scroogled: the day Google went evil
My short story Scroogled has been reprinted on Lithub, as part of the promotion for Watchlist: 32 Stories by Persons of Interest, a forthcoming anthology about surveillance with stories by Etgar Keret, T.C. Boyle, Robert Coover, Aimee Bender, Jim Shepard, Alissa Nutting, Charles Yu and others. Scroogled is the story of “the day Google became... more
How does Google manage Oncall?
"Login", the Usenix Newsletter, has an excellent article about how Google manages oncall. Authors Andrea Spadaccini and Kavita Guliani did an excellent job of providing an overview of how Google seeks to balance oncall time with non-oncall time so that engineers have time for actual engineering. While most of the article deals with how to prevent operations people from getting overloaded, they also raise the issue that operations underload is dangerous too. SREs get out of practice if they don't get paged enough. They describe games and simulations that SRE teams do to stay in practice. The article is available for free to Usenix members and newsletter subscribers, or for a nominal charge to everyone else.