Blog Archive: November 2016
Modern file transfer
Yvonne back from shopping having scraped somebody's car in the car park. It seems that her car sustained most of the damage, conveniently limited to a single panel: The owner of the other car showed up and showed her how to take video with her phone, for which I admire her (the owner of the other car). We deliberately don't have smart phones because of the interface, and we've never used the photographic capabilities of our phones.
Sprinkler controller arrives
The Ethernet-connected relay board that I ordered a couple of weeks ago has now arrived. No power supplythat was almost to be expected from the descriptionbut absolutely no instructions!. That must be a new low. Hopefully I'll be able to make sense of the markings on the circuit board. Found a suitable power supply (I hope): the board has a marking 12 VDC , but no information about the current required. The description above says 7-24V DC , again with no current spec. My power supply delivers 1 A, which will hopefully be sufficient. I should find time tomorrow to look at it.
Bringing the Magic of Amazon AI and Alexa to Apps on AWS.
From the early days of Amazon, Machine learning (ML) has played a critical role in the value we bring to our customers. Around 20 years ago, we used machine learning in our recommendation engine to generate personalized recommendations for our customers. Today, there are thousands of machine learning scientists and developers applying machine learning in various places, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection. There are many more application areas where we use ML extensively: search, autonomous drones, robotics in fulfillment centers, text processing and speech recognition (such as in Alexa) etc. Among machine learning algorithms, a class of algorithms called deep learning has come to represent those algorithms that can absorb huge volumes of data and learn elegant and useful patterns within that data: faces inside photos, the meaning of a text, or the intent of a spoken word.After over 20 ...
Bringing the Magic of Amazon AI and Alexa to Apps on AWS.
From the early days of Amazon, Machine learning (ML) has played a critical role in the value we bring to our customers. Around 20 years ago, we used machine learning in our recommendation engine to generate personalized recommendations for our customers. Today, there are thousands of machine learning scientists and developers applying machine learning in various places, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection.
Your IP address is compromised!
Call from James Watson of Telstra today to tell me that my computer was in danger. I had a bit of time, so I strung him along. First, of all, he wanted to know whether I was running Windows , Mac OS or Linux, to which I honestly answered No , and volunteered that I was running FreeBSD. What? I repeated a couple of times, and he decided to call his team leader, who introduced himself as Stephen. He told me the same spiel, but didn't ask about what operating system. Instead he asked me to open a browser and go to Google to find support.me.
Copying files with Microsoft
A while back I recorded a film from SBS TV for Chris Bahlo. She gave Yvonne a USB stick to put it on, but unfortunately it was too small. So I tried it again today with a spare 8 GB SD card. I didn't expect what I saw: I've seen that beforeI think with Chris' USB stickbut at the time I attributed it to lack of space. This time I had lots of space. It took a while to discover that the real issue is that the file was larger than 4 GB, the maximum that FAT-32 file systems can handle.
Microsoft programs take over
A while back I bought some hooks for the kitchen. They came with the usual instructions in 3 point font. Reading them, I found an apparently unrelated reference to a Microsoft program: What's that? It proved to be a naked URL: http://www.command.com/ is the base URL of their web site, though of course they have to redirect you immediately. ACM only downloads articles once.
GIMP 2.9
So finally I have a machine to run GIMP 2.9. How do I get it? There are instructions here, and they work. GIMP has developed a singularly ugly new appearance, but it seems to work, and unlike far too many programs nowadays, it doesn't have any issues running over the net (apart from a slowness which I expect wouldn't otherwise be there). Finally I can start to recover my photos. ACM only downloads articles once.
Preparing for GIMP 2.9
Where do I go to get the latest version of GIMP? The right thing to do would be to port it, but I couldn't be bothered. Instead I went looking for precompiled versions for Linux. No, don't have them, but try your distro. OK, I have a Linux box running, with the stupid and immutable system name greg-GA-MA785GT-UD3H, but that's (barely) running MythTV, and I don't want to break it. But I had a virtual machine called eucla, which had Ubuntu 14.04. It seems I never completely installed it: the Ethernet card was set up to talk to the wrong interface on the host machine (and thus take me off the net).
Making friends with Android, again
I had to to to the doctor today, a typical time for using my Android tablet (in the waiting room). Spent some time uploading documentation, in the process discovering that AirDroid has completely changed its behaviour, and it uploaded the files (apparently to /sdcard0), where I couldn't find them any more. Somehow Android has completed the destruction of file system hierarchies started by Microsoft and Apple. OK, I still need something to display them with, and that's clearly Acrobat Reader. The good news is that it instantly found the documents, though it's not clear whether this was luck or design. Acroread has changed its interface too, and it took me something like 10 minutes to discover how to get it to display a page at a time.
Photo restoration continued
Yesterday's search for photo restoration software was held up by my registration for a second Facebook user. The email finally arrived this morning at 8:00, after 17 hours! What held it up? Manual checks for validity? NSA vetting? Who knows? So off I went to the Softonic web site, where I ran into many false leads trying to download other software that I didn't want. Sorry, people, you've just lost my trust. Who knows how good the software is? I no longer want to find out. Another lead was Fotor, which I downloaded, but at first sight it seemed to do nothing useful.
Advice to Someone Just Entering Our Industry
This morning I’m thinking through what I’m going to say at the AWS re:Invent Conference next week and I suppose I’m in a distractable mood. When an email came in asking “what advice would you give someone entering the information technology industry in 2017?” I gave it some thought. This was my short take: Play...
Photo restoration software
I'm working on some photos that I took on 5 August 1969. They need a lot of work to improve them. DxO Optics Pro helps up to a point, but somehow nothing seems to help recovering gradations, particularly in green. Then I have COLOR projects 4 , which, like all the PROJECTS software I bought, seems to offer nothing worthwhile. I tried that a couple of months ago and failed. It's really special effects software, not something that I can use for fixing colour casts. In general, the money I spent on PROJECTS software has proved to be a complete waste. So I went looking for photo recovery software .
$90 Android tablet
I've been doing a lot of macro photography lately, and one of the issues is the viewfinder. Clearly the one on the camera is seldom in a place where I can access it easily, and the LCD screen is not really ideal for careful focusing. If my Android tablet hadn't died, I could try Olympus' appalling OI.Share. But as it happens, ALDI has a special at the moment, a 10" Android tablet for only $90, so while in town, I picked one up. What can you expect for that price? Not even a charger, just a USB cable to connect to (and charge from) a computer.
Car Wars: a dystopian science fiction story about the nightmare of self-driving cars
Melbourne’s Deakin University commissioned me to write a science fiction story about the design and regulation of self-driving cars, inspired by my essay about the misapplication of the “Trolley Problem” to autonomous vehicles. The story, Car Wars, takes the form of a series of vignettes that illustrate the problem with designing cars to control their... more
NBN: not communicating
Yesterday I pondered over the meaning of the status lights on the NTD, but today things were different again: only amber. So it looks as if the signal strength is deteriorating. Posts on Facebook show that I'm not the only one to notice that. At the moment, that's not an issue, but how is this going to continue? Amber, amber/red, red, nothing? Is NBN monitoring the situation? You'd expect them to, and maybe they're in the process of replacing a defective transmitter module: there are three antennae on each tower, and if something relating to one of them has failed, then they can fall back to another until they replace it.
MXNet - Deep Learning Framework of Choice at AWS
Machine learning is playing an increasingly important role in many areas of our businesses and our lives and is being employed in a range of computing tasks where programming explicit algorithms is infeasible. At Amazon, machine learning has been key to many of our business processes, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection. And there are many more application areas where we use machine learning extensively: search, autonomous drones, robotics in fulfillment centers, text and speech recognitions, etc. Among machine learning algorithms, a class of algorithms called deep learning hascome to represent those algorithms that can absorb huge volumes of data and learn elegant and useful patterns within that data: faces inside photos, the meaning of a text, or the intent of a spoken word.
TPOSANA on Amazon Lightning Deals today only!
Upgrade to the new edition! Amazon is having a time-limited sale today that includes the new edition of The Practice of System and Network Administration. This is a good way to get the new edition at a deep discount: Follow this link. It should be somewhere on this page. The deal is only available today (Tuesday, 22-Nov) from 12:05 PST to 6:05 PST.
MXNet - Deep Learning Framework of Choice at AWS
Machine learning is playing an increasingly important role in many areas of our businesses and our lives and is being employed in a range of computing tasks where programming explicit algorithms is infeasible. At Amazon, machine learning has been key to many of our business processes, from recommendations to fraud detection, from inventory levels to book classification to abusive review detection.
Understanding the NTD display
My NTD is still showing alternate amber and green signal status: What does that mean? It took a lot of differential reading the documentation and the markings on the NTD to come to a potentially meaningful conclusion: the signal strength LEDs do show progressive signal strength. One LED means low signal strength. Two LEDs mean medium (acceptable) signal. Three LEDs mean good signal. But in addition, the one LED is red, two LEDs are amber, and three LEDs are green.
Firefox bloat
Yvonne complained that her system was slow, particularly firefox. That's nothing unusual, but I wasn't prepared for what top(1) showed me: last pid: 77440; load averages: 0.51, 0.63, 0.79 up 20+21:20:14 17:17:53 72 processes: 1 running, 71 sleeping CPU: 3.7% user, 0.0% nice, 0.6% system, 0.0% interrupt, 95.7% idle Mem: 523M Active, 3358M Inact, 1129M Wired, 469M Cache, 806M Buf, 2444M Free Swap: 20G Total, 91M Used, 20G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 60584 yvonne 54 20 0 26491M 1533M select 0 226:21 36.38% firefox 1275 yvonne 1 20 0 957M 74324K select 1 107:32 0.00% Xorg 26 GB memory size!
Another Google Maps failure
Used Google Maps to show me the distance from here to Rokewood Junction. But it doens't know where it is either: Rokewood Junction is the road junction at the extreme south-west of that map, as the name of the road leading there indicates. How did Google Maps place it in the middle of nowhere? ACM only downloads articles once.
NTD status docco: found
More searching the web for documentation for the NTD today, and finally I was successful. Not on the National Broadband Network web site, of course, but on boombroadband.com.au, apparently an RSP. The document clearly comes from nbn", but I can't find it on their site. Here's the description of the signal strength LEDs: In reality, they look like this: Based on the markings, any sane person would assume that the LEDs light from left to right with increasing signal strength.
Nonstandard nbn installation
A number of people in Dereel have problems connecting to the National Broadband Network because of the lie of the land, trees, etc., including at least one property in Stones Road. While looking for status LED information, found this page by an antenna company with the descriptive name Telco . But what they offer looks interesting, and next time somebody has a problem, I'll point them to it. ACM only downloads articles once.
Off the net
Lorraine doesn't have an Internet connection, so Yvonne wanted to show her some of the caricatures of Trump that she had found. But then she came to me and said we're off the net . How would she know? It could be a firefox hang or anything in the local network. But she was right. We went off the net at 11:25, and didn't come back until 13:46, only to go off again for another 6 minutes at 13:53. When it did come back, the status indicators on the NTD were not normal: at irregular intervals the left two status LEDs flashed amber (should be green; the status LED is a primitive signal strength indicator).
Achromat Photowalk
On a recent cold and damp (but not actually rainy) evening I was invited on a Fujifilm-sponsored photowalk on Vancouvers Granville Island. The day before, Id received the fruits of a whimsical Kickstarter splashout from months and months back: a Daguerrotype Achromat 2.9/64 Art Lens from Lomography. It was great fun among the sparkles in the dark. The lens looks like this: It has a Pentax K-mount, fastened here to my Fuji XT-1 using a Fotodiox adapter. Look at the fancy writing on the narrow part of the lens, then look a little further away from the camera body, above the writing, and you see a black slot in the top of the lens.
The State of Software Engineering Research
At the 2016 Foundations of Software Engineering conference a panel discussed the state of software engineering research. The panelists were asked to reflect on three questions.
Counting in the time of the web
I have a number of saved searches on eBay for which I get graphic, incomplete, badly rendered and often misspelt emails every day. The number of entries displayed is limited to 12 and the text is broken almost beyond recognition. They made some changes recently. The good news is that they have discovered that the singular of matches is match , and not matche (presumably because they know that the singular of lenses is lense ). But that's about the only improvement. The limit of 12 entries is still there, and the rendering, at least on firefox under FreeBSD, is completely broken: But the one I don't understand here is that there are 5 matches, and it only shows two of them.
Fujifilm Photowalk
On Sunday, I bought new shoes. On Monday, I got my Kickstarter-backed long-delayed Daguerrotype Achromat 2.9/64 Art Lens. Thus, on Tuesday I was well-prepared for a Fujifilm-sponsored photowalk on Vancouvers Granville Island. Thats the Granville Street bridge,with the downtown condo wall behind it and a supermoon above. Sometime in the 3½ years Ive been in Fuji-land I got on their mailing list, which is OK because its low-volume and, well, got me this invite. The ultimate hosts were Fujifilm Canada (whose President and SVP were there), but also present and helping were The Fuji Guys and Beau Photo. I wish I had the cycles to do more photowalks.
Im helping launch Echoes of Sherlock Homes at LAs Chevalier Books tomorrow night
In 2014, lawyer and eminent Sherlockian Les Klinger comprehensively won the legal battle to establish that Sherlock Holmes is in the public domain and available for anyone to use, abuse, alter, celebrate or mock; now with a new anthology of completely unauthorized Sherlock tales, Echoes of Sherlock Holmes, Klinger and co-editor Laurie R. King have... more
Spice up your Analytics: Amazon QuickSight Now Generally Available in N. Virginia, Oregon, and Ireland.
Previously, I wrote about Amazon QuickSight, a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. QuickSight is a very fast, cloud-powered, business intelligence service for the 1/10th the cost of old-guard BI solutions. Today, I am very happy to announce that QuickSight is now generally available in the N. Virginia, Oregon, and Ireland regions. When we announced QuickSight last year, we set out to help all customersregardless of their technical skillsmake sense out of their ever-growing data. As I mentioned, we live in a world where massive volumes of data are being generated, every day, from connected devices, websites, mobile apps, and customer applications running on top of AWS infrastructure.
Trip report: Fall ISO C++ standards meeting (Issaquah)
[ETA: Mentioning specific TSes expected to be merged soon post-C++17.] On Saturday, the ISO C++ committee completed its fall meeting in Issaquah, WA, USA, hosted by Microsoft and the Standard C++ Foundation. We had over 110 people at the meeting, representing 10 national bodies. We also had more than usual local visitors ? note that […]
Trip report: Fall ISO C++ standards meeting (Issaquah)
[ETA: Mentioning specific TSes expected to be merged soon post-C++17.] On Saturday, the ISO C++ committee completed its fall meeting in Issaquah, WA, USA, hosted by Microsoft and the Standard C++ Foundation. We had over 110 people at the meeting, representing 10 national bodies. We also had more than usual local visitors ? note that … Continue reading Trip report: Fall ISO C++ standards meeting (Issaquah) →
Spice up your Analytics: Amazon QuickSight Now Generally Available in N. Virginia, Oregon, and Ireland.
Previously, I wrote about Amazon QuickSight, a new service targeted at business users that aims to simplify the process of deriving insights from a wide variety of data sources quickly, easily, and at a low cost. QuickSight is a very fast, cloud-powered, business intelligence service for the 1/10th the cost of old-guard BI solutions.
Investigating medical records
Now I have my medical records, it's time to scan them in. There were over 60 pages of them! First the notes made by the doctors, conveniently in reverse chronological order, then the pathology results in chronological order. Neither set had page breaks, making it very difficult to understand. Somehow I begin to understand how a new doctor can come to incorrect conclusions after reading them, especially as some of the information about me is just plain incorrect (no allergies, never smoked). How can we do it better? A few months ago I made a table of various blood test results.
Stealing the Best Ideas from DevOps
Christine Hogan and I will be co-presenting a talk at Usenix LISA '16 entitled "Stealing the Best Ideas from DevOps: A Guide for Sysadmins without Developers". Full details are on the LISA website. The talk will cover a lot of the devops-y material from our newest book, the 3rd edition of TPOSANA. We'll be doing a book-signing shortly after the talk. In addition, I'll also be teaching two half-day tutorials: "Personal Time Management: The Basics for Sysadmins That Are Overloaded" and "How to Not Get Paged: Managing On-Call to Reduce Outages". Links to all of this here.
Meet the Teams Competing for the Alexa Prize
On September 29, 2016, Amazon announced the Alexa Prize, a $2.5 million university competition to advance conversational AI through voice. We received applications from leading universities across 22 countries. Each application was carefully reviewed by senior Amazon personnel against a rigorous set of criteria covering scientific contribution, technical merit, novelty, and ability to execute. Teams of scientists, engineers, user experience designers, and product managers read, evaluated, discussed, argued, and finally selected the twelve teams who would be invited to participate in the competition. Today, were excited to announce the 12 teams selected to compete with an Amazon sponsorship. In alphabetical order, they are: Carnegie-Mellon University: CMU Magnus Carnegie-Mellon University: TBD Czech Technical University, Prague: eClub Prague Heriot-Watt University, UK: WattSocialBot Princeton University: Princeton Alexa Rensselaer Polytechnic Institute: BAKAbot University of California, Berkeley: Machine Learning @ Berkeley University of California, Santa Cruz: SlugBots University of Edinburgh, UK: Edina University of ...
Meet the Teams Competing for the Alexa Prize
On September 29, 2016, Amazon announced the Alexa Prize, a $2.5 million university competition to advance conversational AI through voice. We received applications from leading universities across 22 countries. Each application was carefully reviewed by senior Amazon personnel against a rigorous set of criteria covering scientific contribution, technical merit, novelty, and ability to execute.
More Hugin debugging
I consider myself relatively proficient at debugging code I have never seen before. But somehow Hugin has me beat. The problem, as I have identified it so far, is that the configuration variable tempDir (which, as the name suggests, is the name of a directory for storing temporary files) somehow ends up as the path for searching for executables. As I said, shades of firefox. So an obvious approach to searching for it would be to find where the configuration file gets read in. The file is called ~/.hugin, so I can brute force search the source tree for that. And it's not mentioned!
Hugin bug: identified
More examination of the Hugin bug today. It's quite simple, in fact: I have seen it before, but on that occasion I ran into the combined problem of C++ and poor problem reporting. This time I was able to establish that the real problem is that Hugin uses the temporary file path as the executable pathshades of firefox: 39331 hugin CALL execve(0x81b60d380,0x81b605fd0,0x81b007000) 39331 hugin NAMI "/home/var/tmp/icpfind" 39331 hugin RET execve -1 errno 2 No such file or directory But where does it set it?
Pixel Abuse
What happened was, I went down to the dock in the dark, and took pictures that no phone-cam should ever be asked to take. You might be amused. I was with my ten-year-old daughter; The sky was black but the moon was full, its shadows knife-edged, first moonshadows shed ever seen. Its light, edging round the slope of our island, looked cool, only I didnt have a real camera, just my Google Pixel. Hmm, I sense a challenge. Heres the unimproved all-the-defaults version, which has a bit of primitive poetry perhaps. So for the first time I got serious with the manual controls on the Lightroom camera app; cranked the ISO up and the shutter speed down, held the phone against one of the dock pilings, and growled at the girl to stop bouncing.
A bug in Hugin!
A problem report in the mail today: Hugin crashes. For once I had time to look at it, and in the course of the day made quite good progress. It seems that Hugin's way of saying can't find control point detector is: /usr/local/include/wx-3.0/wx/strvararg.h(456): assert "(argtype & (wxFormatStringSpecifier<T>::value)) == argtype" failed in wxArgNormalizer(): format specifier doesn't match argument type It's high time that Hugin saves its log files. By the end of the day it was looking like a configuration problem, and (fortunately) not a FreeBSD port issue.
Software-Defined Cameras
When I pull out my Pixel for a picture, I have to pick which app to tap: Googles Android camera (which oddly seems to have no link) and Adobe Lightrooms. The choice isnt hard, but is interesting. Let me lead with some actual pictures. In each case, the G-cam is first, the Lr-cam second. Howe Sound at dusk. Cabin overlooking Howe Sound, at dusk. The differences are subtle, and probably only matter if youre going to be using a really big screen or piece of paper for delivery. If youre the kind of person who cares, click-to-enlarge and, before I talk about the differences, see if you have opinions.
New sprinkler controller
My current sprinkler controller is suboptimal. It is difficult to use, even more difficult to reprogram, and it seems to have a problemafter only 1½ yearsthat interrupts the flow at random. Twelve years ago in Wantadilla I had a relay board connected to an old computer, and a program and cron job to go with it. Why can't I do that again? For one thing, I don't want a computer running all the time just for that. But what about an Ethernet-connected relay board? Do they exist? A quick search on eBay shows: Yes. Accordinng to the description, 8 Channel Relay Network IP Relay Web Relay Dual Control Ethernet RJ45 interface Ethernet ...
Big-Show Tech
Im all stressed out getting ready for re:Invent, Nov 28th through Dec 2nd this year, in Vegas. Im attending, and may even be speaking if certain pieces fall into place. Ive seen this movie before. At Sun, it was Java One. At Google, I/O. Every big tech company has one. They define the rhythm of the year, and Im wondering if theyre a good idea. Big! Most readers here have probably been to one or more, but for those who havent, heres how it goes. First of all, big is relative. Google and Apple favor San Franciscos Moscone West, a nice welcoming light-filled venue which maxes out at 5,500 heads.
No news today
Recording the Deutsche Welle news is always touch and go: just about any excuse is good enough for the Special Broadcasting Service to forget its charter and broadcast football games from obscure places at the other end of the world. And today it didn't record either. What came instead? Nothing. It was in the programme listings, at any rate. So what happened? Was the football game cancelled, and they updated the EPG too late for greg-GA-MA785GT-UD3H (the immutable name of what should be called ceeveear) to notice? There was no mention in the log files that it had even tried to record the programme.
Old Datamation humour
Decades ago I read a surprising amount of humorous material in Datamation, almost certainly the April 1975 edition. I made (pre-Xerox) photocopies of some of them, but I have since lost them, and all I have is this page, part of the content of which Josh Paetzel recently quoted (and attributed to me) on Facebook: K is the a kludge that you say You require to avoid some delay But that interim fix Forms a habit that sticks And you'll find that it's in there to stay.
Do you teach System Administration?
You do? Then you should attend SESA '16 in Boston, Dec 6, 2016. For more info go to https://www.usenix.org/conference/sesa16 (I'm particularly looking forward to Nicole's opening keynote.)
Sole and Despotic Dominion: how a 20th century copyright law is abolishing property for humans (but not corporations)
In the 18th century, William Blackstone wrote the seminal “Commentaries on the Laws of England,” which contained one of the foundational definitions of property: “that sole and despotic dominion which one man claims and exercises over the external things of the world, in total exclusion of the right of any other individual in the universe.”... more