Blog Archive: October 2016
How to Give Away an Android
You take your nice modern Android, you factory-reset it, and you give (or sell) it to a relative (or friend, or stranger). (In my case, give and son .) But when they turn it on they see a screen labeled Verify your account with text reading This device was reset. To continue, sign in with a Google Account that was previously synced on this device. Heres one way to fix the problem. You, the original answer, grab the phone. Sign in with the primary account you used. Say No to all the sync options you get offered (to save time). Say No when asked if you want fingerprint protection, nodding at the text that warns you that if you bypass it, your device will be usable by a thief after reset.
After Mac?
So, there are new MacBooks and many people are unhappy. Tl;dr: Apple thinks thin-and-light is more important than well-equipped-and-powerful. I griped on Twitter and got a storm of responses, mostly on the subject of other ways I might be able to get what I want from a computer. Sidebar 1 First off: Apple may well be right. Over the past couple of decades, the vast majority of their product launches have hit the sweet spot, turning out to be what people needed even if thats what they didnt think they wanted. Sidebar 2 At work, I use a Mac and it suits me just fine.
Sold My Google Shares
I want the author piece linked in the sidebar to be a full disclosure of the financial interests of the person whos writing what youre reading. I think online writing without such a disclosure is troubling. Anyhow, my financial disclosures no longer include Google, because I sold all my shares last week. Not an insider Because this ongoing fragment touches on money, I should be clear that I have no material non-public information that might give me insight into the future of the Google share price. (In fact, Im not sure I ever did, even when I was working there.)
Pre-Order Kindle edition of 3rd edition of TPOSANA
The Kindle edition of the 3rd Edition of "Volume 1: The Practice of System and Network Administration" can now be pre-ordered. Previously you could only pre-order the paper and PDF versions. The new edition will be available on November 4th, 2016. It includes 22 new chapters, 6 updated chapters, and thousands of updates all over the book. The new edition is nearly twice as long as the previous editions. We worked hard to make this the best edition yet. Here's where you can pre-order the book: Paper: Pre-order from Amazon or InformIT. Kindle: Pre-order from Amazon. PDF (no DRM): Pre-order from InformIT.
Searching random data is not an O(N) problem.
The September/October issue of ACM Queue Magazine has a column I wrote about how to search random data. In theory the best you can do is a linear search. I came up with 10 ways to do better. http://queue.acm.org/detail.cfm?id=2984631 Computer scientists should be upset that I write these things.
Usenix LISA '16 Early Bird Registration Deadline is November 10!
The "Early Bird Registration" deadline is November 10, 2016. After that, the price goes up. LISA is Dec 4-9 and you can register here. I'll be teaching 2 tutorials and Christine Hogan and I will be co-presenting some talks. We also are planning a book signing during the conference. We'll announce the time/location when it gets closer to the event. Presetations: Talk: "Stealing the Best Ideas from DevOps: A Guide for Sysadmins without Developers" (Tom+Christine) Tutorial: "How to Not Get Paged: Managing On-Call to Reduce Outages" (Tom) Tutorial: "Personal Time Management: The Basics for Sysadmins That Are Overloaded" (Tom) SESA'16 (2016 USENIX Summit for Educators in System Administration) is colocated with LISA.
Netdata on a Raspberry Pi
A couple of days ago I had the privilege to see a demo of netdata by Costa Tsaousis , the person behind this project. The project offers comprehensive real time monitoring of a Linux computer with low overhead in a single easily-installed and self-contained system. I thought this was too good to be true, but the number of users and installations hinted that this could well be the case. I therefore decided to install the system on a Raspberry Pi I'm configuring to replace an ancient 20 year old IBM PS/2 server .
Pixel Notes
I pre-ordered the basic Pixel (5", 32G, Silver) because the 5X was getting on my nerves (more below); here are early-days notes. Tl;dr: Ugly, solid, fast, cool camera. No new build So, heres the secret insider story of where little mobile builds come from. Word goes around that the software has to ship by Date X to meet the hardware launch date. Usually its not ready, so what happens is they ship an interim build and then, some weeks later when the phones hit the shelves and people are starting to turn them on, when people turn them on they get prompted to download the real build.
Welcoming Adrian Cockcroft to the AWS Team.
I am excited that Adrian Cockcroft will be joining AWS as VP of Cloud Architecture. Adrian has played a crucial role in developing the cloud ecosystem as Cloud Architect at Netflix and later as a Technology Fellow at Battery Ventures. Prior to this, he held positions as Distinguished Engineer at eBay and Sun Microsystems. One theme that has been consistent throughout his career is that Adrian has a gift for seeing the bigger engineering picture. At Netflix, Adrian played a key role in the company's much-discussed migration to a "cloud native" architecture, and the open sourcing of the widely used (and award-winning) NetflixOSS platform.
Welcoming Adrian Cockcroft to the AWS Team.
I am excited that Adrian Cockcroft will be joining AWS as VP of Cloud Architecture. Adrian has played a crucial role in developing the cloud ecosystem as Cloud Architect at Netflix and later as a Technology Fellow at Battery Ventures. Prior to this, he held positions as Distinguished Engineer at eBay and Sun Microsystems.
Message Processing Styles
Recently Im thinking about how we process messages in networked software. Consider this Java snippet, for example. < !-- HTML generated using hilite.me -->boolean isPassNode(final JsonNode node) { if (node.isObject()) { final JsonNode child = node.get(Constants.TYPE_FIELD); if (child != null) { if (child.isTextual()) { return Constants.PASS_TYPE.equals(child.asText()); } } } return false; } Which asks: Is this a JSON object with a top-level Type field whose value is the string Pass ? Is this a sane thing to want to do? And if so, whats a good way to do it?
Watch my Punk Rock DevOps talk live on Friday at 2:30pm PT
My talk "DevOps Where You Wouldn't Have Expected" will be live-streamed from PuppetConf on Friday, Oct 21 at 2:30pm PT / 5:30 ET. The talk happens to summarize the major points of Chapter 1-4 of the new edition of The Practice of System and Network Administration, which is due out on November 4, 2016. You need to pre-register, which takes time. Please preregister early. A full description of the talk is at http://sched.co/6fk7 To register and watch the live-steam visit https://puppet.com/puppetconf/livestream/signup P.S. The unofficial title of the talk is "Punk Rock DevOps".
Interview with IEEE-USA Insight Podcast
I was interviewed for the IEEE-USA Insight Podcast last summer in New Orleans, during their Future Leaders Summit, where I was privileged to give the keynote (MP3)
Expanding the AWS Cloud: Introducing the AWS US East (Ohio) Region
Today I am very happy to announce the opening of the new US East (Ohio) Region. The Ohio Region is the fifth AWS region in the US. It brings the worldwide total of AWS Availability Zones (AZs) to 38, and the number of regions globally to 14. The pace of expansion at AWS is accelerating, and Ohio is our third region launch this year. In the remainder of 2016 and inq 2017, we will launch another four AWS regions in Canada, China, the United Kingdom, and France, adding another nine AZs to our global infrastructure footprint. We strive to place customer feedback first in our considerations for where to open new regions.
Expanding the AWS Cloud: Introducing the AWS US East (Ohio) Region
Today I am very happy to announce the opening of the new US East (Ohio) Region. The Ohio Region is the fifth AWS region in the US. It brings the worldwide total of AWS Availability Zones (AZs) to 38, and the number of regions globally to 14. The pace of expansion at AWS is accelerating, and Ohio is our third region launch this year.
Debugging chromium
I'm not the only person to have problems with chromium (correct name, but the executable is called chrome) version 52. Last week I entered a bug report and had it almost immediately closed as a duplicate of this report. And that one has been open since 22 August, without any action. I don't know how to interpret this: you'd think that something as important as a web browser would get better attention. On IRC today, Callum Gibson told me that he had no problems with exactly the same release. On his suggestion, I created a new user, logged on as he, and started chrome.
File, folder or directory?
Mail from Bartosz Fabianowski today, wondering why I, of all people, should refer to folders in an article yesterday. But that was in connection with email, and there the term makes sense. And its meaning is distinct from either file or directory: MUAs typically store messages either in files (Mbox) or directories (Maildir), though I suppose many products in the Microsoft space now store them in databases. In each case, though, the usage folder applies. Yet another reason not to apply it to directories. ACM only downloads articles once.
A clever head
Mail from the Frankfurter Allgemeine Zeitung today: Wir danken Ihnen für den großen Zuspruch. Leider sind unsere IT Server zusammen gebrochen. Unsere IT Experten haben das Problem gelöst- Sie können das Abo jetzt wieder bestellen. For once, Google Translate does a reasonable job of translation of the original intent: We thank you for your great support. Unfortunately, our IT servers have collapsed. Our IT experts have solved the problem - you can now order the subscription again.
Private email server: good or bad?
One recurring theme in the current United States of America election campaign is Hillary Clinton's use of a private email server. Three months ago I commented on the matter. But I still don't understand: I also run a private mail server (in fact, at least 3 of them). But in my language a mail server is a Mail Transfer Agent, something like a post office: mail is only stored there until it can be delivered. And indeed, Wikipedia agrees with me: A mail server is a computer that serves as an electronic post office for email.
Talking about Allan Sherman on the Comedy on Vinyl podcast
Jason Klamm stopped my office to interview me for his Comedy on Vinyl podcast, where I talked about the first comedy album I ever loved: Allan Sherman’s My Son, the Nut. I inherited my mom’s copy of the album when I was six years old, and listened to it over and over until I discovered... more
Debugging a File Synchronization Problem
In Effective Debugging I write that if a web search doesn't return you any useful results, then maybe you're barking at the wrong tree. Here's an example.
Accelerating Data: Faster and More Scalable ElastiCache for Redis
Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis, a fully managed in-memory data store that operates at sub-millisecond latency. Since then weve introduced Amazon Kinesis for real-time streaming data, AWS Lambda for serverless processing, Apache Spark analytics on EMR, and Amazon QuickSight for high performance Business Intelligence. While caching continues to be a dominant use of ElastiCache for Redis, we see customers increasingly use it as an in-memory NoSQL database.
Pharmaceutical web sites
What concentration is the Bromhexine hydrochloride in the tablets I bought? Off to the Boehringer Ingelheim web site to take a look. It has a feature I haven't seen before: the country localization page shows pretty (and useless) emblems with a stylized map and a that goes as far as the Czech Republic: Then you have to click and wait several seconds for the next page. If you live in remote places like the United Kingdom or the United States of America, you have to click four times, taking about half a minute in all.
Accelerating Data: Faster and More Scalable ElastiCache for Redis
Fast Data is an emerging industry term for information that is arriving at high volume and incredible rates, faster than traditional databases can manage. Three years ago, as part of our AWS Fast Data journey we introduced Amazon ElastiCache for Redis, a fully managed in-memory data store that operates at sub-millisecond latency.
Fall Dark
Most years I hate this season; less light every day, and with every gust a whirl of summer leaves torn from winter branches. Maybe I dislike the resonance with my lifes own greybeard season. Maybe its the trio of huge Pacific storms were dealing with. Lets be honest: Mostly, its shitty US politics. Some of the colors are beautiful though. These days, it needs an effort of will to look away from Americas slow titanic electoral trainwreck. We should make that effort. All it takes is going for a walk and leaving your mobile in your pocket. At least for a few minutes.
Suspicious rcfile
I automatically file a copy of all incoming mail in ~/Mail/backup, and quite frequently refer to the folder after deleting the primary copy of a message. So it was today. But the last saved message was yesterday at 14:20, 20 hours before. What went wrong? Looking in ~/Mail/procmailerr, I saw lots of: procmail: Suspicious rcfile "/home/grog/.procmailrc" procmail: Couldn't read "/home/grog/.procmailrc" What does that mean? Took a look at the file, and it looked perfectly normal. No reason to not be able to read it. Neither it nor the procmail executable had changed in months.
Still more MythTV agony
What happened to this mornings Deutsche Welle news? For some reason, it didn't get recorded. Usually that's an indication that something much more important has come up, like a football game in Uraguay. But no, today it was broadcast, it just didn't get recorded. And when I tried to schedule another recording, the system just hung. OK, rebootafter all, Ubuntu emulates Microsoftand then the recordings were already scheduled. But what went wrong? That's a problem I've never had before. But that reminded me that I was still planning to retry a Mythbuntu installation on /dev/sda1. The intention was to set the IP address and system name as it was supposed to remain.
Apply for a Shuttleworth Fellowship!
https://vimeo.com/54762523 I’m the “Honourary Steward” for this year’s Shuttleworth Fellowship, this being a valuable and prestigious prize given to people who are undertaking to make the world a better, more open place (“social innovators who are helping to change the world for the better and could benefit from a social investment model with a difference”).... more
Next day of MythTV pain
My current status is that MythTV is running on a machine called greg-GA-MA785GT-UD3H, and I can record up to four programmes at a time. Done? Hardly: Clearly the system name (and DHCP served IP address) is Just Plain Stupid. It should be called ceeveear.lemis.com, which has a public IP address. To use NFS, my personal user ID should be the same as the IDs on other systems in the network.
Wrong About Cameras
Todays Wrongness Exhibit is iPhone 7 vs Leica M9-P: A Side-by-Side Photo Comparison by Michael Zhang, which demonstrates any fool can plainly see, look at the photos that an iPhone 7 takes pictures just as well as a $9K Leica setup. The wrongness here is extreme and, I think, instructive. Lets start with the picture in Zhangs piece, of a Japanese shrine in the rain. Lets see; it is medium-distance, even-depth-of-field, well-lit, and low-dynamic-range. Which is to say, about as easy to get right as a photograph can be. The kind of scenario where you dont need a Leica; in fact, maybe you dont even need a recent iPhone.
DevOps Handbook hitting stores Oct 6!
One of the most anticipated DevOps books in years is about to start shipping! DevOps Handbook: How to Create World-Class Agility, Reliability, & Security in Technology Organizations by Gene Kim, Jez Humble, Patrick Debois, and John Willis is the practical guide to doing all the wonderful things that The Phoenix Project talks about. I've received an early copy of the book and it is excellent. It is very down-to-earth, practical, advice. I'll write more next week when I've had time to read the entire thing. You can pre-order it directly from IT Revolution or via Amazon. Check it out!
MythTV: First success
So yesterday's issue with MythTV was a MySQL configuration issue? Dug through the bug report, which offered at least two alternative solutions, and chose the correct one: --- /etc/mysql/conf.d/mythtv.cnf~ 2016-10-03 16:56:56.682980378 +1100 +++ /etc/mysql/conf.d/mythtv.cnf 2016-10-04 13:49:21.424379549 +1100 @@ -1,3 +1,4 @@ [mysqld] #bind-address=:: max_connections=100 +sql_mode=NO_ENGINE_SUBSTITUTION After that, oh wonder, MythWeb worked! Done! After only 9 days! Well, not quite. I still need: Finish configuring the system, including system name, IP address, file systems and my user ID.
Lost: Emacs bindings
Decades ago, my first experience with firefox was anything but positive. Since then I've learnt to live with it, but it seems to continually try to annoy me. On the very first occasion, I discovered that it had replaced the standard X key bindings with, as I said at the time, something Microsoft-like . At the time I found a way round the problem, but after today's upgrade it stopped working. What's wrong this time? Sometimes I despair. Twenty years of web browsers, twelve years of multimedia software, and they continually conspire to make life hard. But this time I found a firefox addon to do the job for me, firemacs.
MythTV pain, the fourth
I've been trying for a week to install a new version of MythTV, so far without success: On 26 September I installed LinHES. It proved difficult to find a version of shepherd to match, the machine seemed to be continually accessing the disk, and I had difficulties with MythWeb that seemed to depend on how I accessed it.
Package upgrade: the pain
So I am finally forced to upgrade firefox on eureka. I've been reluctant; the package system is much better now, but there seems to be a maze of twisty little interdependencies, all different. Every time I have tried to upgrade firefox, it has tried to remove Emacs. Not an option. So how about rebuilding from source? Started thatnot for the first timeand not for the first time and into endless pain: all the packages that needed removal before I could build the new version. The worst seems to be rust, which bills itself as an open-source systems programming language that runs blazingly fast... .
External editor for firefox
A week or so ago the It's All Text addon to firefox stopped working. Why? It seems that the developer uploaded a corrupt version to the web site. Why should that stop the locally installed version working? Because. My best guess is that automatic updates are so clever that they first remove the old version, then check if the new one works.. I waited a while, sent a message to the developer, which remains unanswered, and went out looking for an alternative. What I want is a way to get a sane editor (Emacs) to edit text fields in firefox. Surprisingly, there are very few.
Identifying motherboards
Yesterday I commented on the strange system name that Ubuntu thought out for my new ceeveear system, apparently the model number of the motherboard. Peter Jeremy also came out with an explanation of how to find the number: it seems that the BIOS provides it. With FreeBSD I can get it with: === grog@eureka (/dev/pts/8) ~ 44 -> kenv | grep smbios.planar smbios.planar.maker="ASRock" smbios.planar.product="Z87 Pro4" smbios.planar.serial="E80-37011700126" smbios.planar.version=" " That was on eureka, and it looks like some kind of sysctl, but it isn't.
Early Experiments in Visualizing Pattern-Development for Viking-age Blades
Ubuntu system names
I had puzzled over the strange default system name that Ubuntu gave to my computer: greg-GA-MA785GT-UD. Peter Jeremy came out with an explanation: it appears to include the model number of the motherboard! Apart from the question about how it found the name, what earthly use is that? It would make (marginally) more sense to describe the appearance of the box. I never cease to be amazed. ACM only downloads articles once.
Next day of MythTV pain
On rather half-heartedly with my Ubuntu installation today. Over the course of time I discovered that the base Ubuntu installation is missing a lot more than just MythTV. Here's a first cut at what needs to be done to get the thing working: apt install emacs nfs-server rwho rwhod openssh-server mailutils mutt On the positive side, Emacs on Mythbuntu used the clipboard for the top of the kill ring, which greatly irritated me.