Another Sale!

The last one was "Best Sellers." Now the video is also "Best Reviewed." Nice!

InformIT.com - Best of 2015 Sale

Sale on My Git Under the Hood LiveLessons!

Well, I haven't blogged much lately, have I? :-)

Heather Fox, from Pearson, tells me that my Git Under the Hood, LiveLessons, is a best-seller, and ison sale this weekend.

InformIT Labor Day Sale - Save up to 70%

Seeing My DNA

There's a photomicrograph of my chromosomes on my living-room wall. I made it myself in a Human Cytogenetics course. When folks ask, I say it's a self-portrait.

I've been thinking about subscribing to 23andme, which analyzes your DNA, for several years, but the price was always too high. I knew it would drop over time.

Two things kicked me over the edge. One was posts by folks like Razib Khan about recent moves by the FDA to shut down or hyperregulate 23andme. The second was a trip to the Boulder Android Meetup, where I met G. Hussain Chinoy.

Chuck Coughlin and I went over to recruit an Android developer or two for a project we're working on, and Hussain sent me email saying he was interested. He wasn't a great fit for the particular slots we were trying to fill -- it happens -- but I told him I was very interested to see that he was a 23andme customer.

A few days later, Hussain dropped me a quick note to say that 23andme was having a "free day." Normally, the base price had been $199, but for that day, it was free.

That did it.

I'm now a happy customer.

I should say more about 23andme, but this post's long enough for now. Go look for yourself.

My company's still hiring, by the way -- contract and full-time positions.

Audiobooks, Sort Of

I began looking for mp3 audiobooks to put in my Amazon Cloud Storage, so I could play them on my Android tablet, and ended up somewhere I hadn't guessed.

I've read that folks use their phones as MP3 players. Mine has its battery run down so fast when I use it as a phone that I can't imagine doing that. Perhaps I'm missing something.

But my NookieTab is a toy I'm exploring the boundaries of, so I bought some music, put it up in Amazon's free, Cloud Storage service, and tried playing it with their Cloud Player. Works fine.

Great.

How about an audiobook? Amazon doesn't sell many audiobooks in its MP3 store, because of DRM, but I saw Indian Tales, by Kipling and bought it. Kipling spins great yarns -- very listen-able.

Then I tried it. The book is machine-read. The synthesized voice has a British accent, which is a nice touch.

I went back and saw a book on Italian. This time, I sampled it before buying. Yup! It's lessons in synthesized English on how to speak Italian. Hearing is believing.

My Nookie Tablet

The Web is filled with instructions on how to root and re-flash the Nook Color.

It has much richer hardware than, say, a Kindle, so although it's sold as a locked-down, e-book reader, a chunk of its market is folks turning it into something else.

Think pogoplug. Think LinkSys WRT54G. Think DEC PDP-7.

BestBuy had stacks of Kindles, but the shelf under the Nook was cleaned out. The salesman had to go to the stockroom to find me a box.

It comes as a locked-down reader, but the base OS is Android so you have two choices: root or re-flash. They'll both void your warranty, so it's a matter of personal taste.

If you re-flash it, there's plenty to choose from. Google will find you instructions for putting on anything from vanilla Froyo to Ubuntu. Someone probably has a port of Windows CE. To each, his own.

I opted for my usual route: I asked the nearest beautiful woman, "Could you help me turn this into an Android Tablet?"

She said, "Sure. It'll take about five minutes, unless you want to back it up first."

Five minutes was not hyperbole.

She had a micro-SD chip with the necessary files on her -- the OS, the Google Android apps, and an over-clocker. We sliced off the Nook's shrink wrap, took it out, popped open the SD port, slipped in her card, brought it up, reformatted the disk and loaded on the new software, and re-booted.

Never even un-boxed the instructions.

"Don't we need to charge it first?" I asked.

"I'm sure it comes with plenty of power," she said.

We looked after it was back up. The battery was 97% full.

It found someone's near-by, open WAP easily, so I configured it to use my Google account, checked my email, and downloaded a couple of apps from the Android marketplace.

I'm running Cyanogenmod's Gingerbread port (Android 2.3.4, with a 2.6.29 kernel).

The display is beautiful, but before I could watch a free movie, from Amazon Instant Video, I had to download Flash Player from the Android Marketplace. Their service does whine that I'm using an unsupported device, and the video's a little jerky, but I don't know whether that's the tablet or the format and bandwidth.

The rest is fiddling. For example, making the terminal usable meant using the menu key (it's all soft keys) to make the fonts bigger. Pairing with a blue-tooth headset required figuring out how to turn the blue-tooth headset on.

The earphone jack works. There's no built-in camera or microphone, and I don't think the hardware's there to turn it into a standard cellphone, but it seems like I should be able to make VOIP calls with the blue-tooth headset. I've poked at that one a little, but not succeeded. Yet.

I'd wiped out the Barnes & Noble book reader, so no Nook e-books for me; instead, Amazon's free, Kindle-for-Android app lets me download and read the books I buy for my Kindle.

I suppose this is a great time to plug the Kindle. Want a tablet computer? Lots of choices, including the Color Nook. Want an e-book reader? Buy a Kindle.

Sturdy, simple, beautiful, easy-to-read, dirt cheap. Connects free everywhere. Electronic paper that's as foolproof and power-thrifty as an Etch-a-Sketch. Unbeatable selection of books.

It is the C-47 of e-book readers. Jeremy Hinegardner tells me his grandpa's on his second one.

Toys

Jealous of Steve Ballmer's buying Skype, I drove to Best Buy, Thursday afternoon, and bought a Color Nook.

The salesman offered me a protection package for $99. I said, "I'm going to void the warranty on this so fast it'll make your head spin."

The board has roughly the same chip set as the phone we're building at work. As a Nook, it was a locked-down, single-purpose reader. So far as I know, B&N won't even let you get to some of its hardware, like the bluetooth. Reflashed, I can now even load on whatever we write for our phone and play with it.

He said, "Next few hours?"

I said, "I'd guess the warranty on this will be void fifteen minutes after I leave the store."

I wasn't wrong. I now own a $250 Android tablet.

Before I went to bed, I slammed on Amazon's Kindle for Android app. Heh.

I'm waiting to see what Skype looks like after Ballmer re-flashes it.

RMBB!

RMBB!

'bout time. See you there.

Free AWS Book Chapters

In my email this morning, from Amazon:

*Free AWS Book Chapters*
Download a free PDF copy of the first three chapters of AWS Evangelist Jeff Barr's book, Host Your Website in the Cloud at http://www.sitepoint.com/books/cloud1/samplechapters.php. Jeff's book covers principal AWS services including EC2, S3, SimpleDB, CloudWatch, and RDS. The book makes use of the AWS SDK for PHP and includes plenty of working examples. Find the book on Amazon.com: http://www.amazon.com/Host-Your-Web-Site-Cloud/dp/0980576830

If you went to BLUG's AWS-Installfest-in-the-Cloud event, this ties right in. Good book, too.

Sort of Balkan Dancers

Another forehead-smacking episode of "Why didn't I think of that?"

HT: Chuck Hipschman

23-and-me one-day sale

23-and-me gives you information about your own DNA. I just became a customer. They're running a sale.

Normally, it's $199, plus a $9 a month subscription to updates for a year (after which you can continue to subscribe or cancel).

Today only: you still have to subscribe, but the $199 is waived, so it's about $100 instead of about $300. I'll find out whether the subscriptions are worth it over the next year.

Note that Federal bureaucrats want to shut them down because they think your DNA is something that only an M.D. should explain to you. Even without that, though, I'd have still bought it.

Handling Failure

Here are two coding styles for handling failure. I read a lot of code that uses the first. I always use the second.

(1)

do-something;

if (success) {

do-this;

do-that;

...

[many lines of stuff]

do-some-other-thing;

}

else

{

print-error-message-and-exit;

}

(2)

do-something;

if (failure) {

print-error-message-and-exit;

}

do-this;

do-that;

...

[many lines of stuff]

do-some-other-thing;

I could go on at length to try to justify my strong preference, but I won't. In the end, it's just taste.

No, I take it back, I will rationalize a little. Here are some reasons I prefer #2.

• It's shorter overall.
• The short alternative comes first, so you don't overlook it in the noise.
• There's one fewer nesting level to puzzle through. This also occasionally permits longer lines without word-wrap.
• It feels more like exception handling, and saying what I mean: A problem? Let's bail.

In Perl, the solution is even more stereotyped and easy-to-read:

do-something OR die "it didn't work";

...

In bash, I use a shell function, die(), which I wrote to let me code in the same style:

do-something || die "It didn't work"

...

Process Whack-a-Mole: An Alternative Approach

Gosh. I haven't blogged in, like, forever. Inspired, as is so often the case, by Hal Pomeranz and friends, here's something.

Over at his Command-Line Kung Fu blog, Hal has posted a solution to the problem of writing a process-whack-a-mole command, that watches for any new process and kills it.

(Hal's self-imposed constraint is always that his solutions really be command lines -- they can't be full-blown scripts -- and that they not use Perl, or other languages that are powerful enough that their use would feel like cheating.)

Nothing wrong with his approach at all, but here's another. (I mailed it to Hal, who said, "Post it somewhere, Dude.")

$ ps -e o pid,ppid,cmd | grep -v $$ > baseline; while : ; do

> sleep 5

> ps -e o pid,ppid,cmd | grep -v $$ > now

> join -v 2 baseline now

done

This is, at least, a single command line. :-)

It shows off a couple of things:

1. join -v
2. the output of ps is pre-sorted by process-number

The didactic downside? It doesn't show off arrays or "[[" . (I spent a minute in philosophical reverie, contemplating the elevated virtue of using files to store data for shell scripts, but then realized this is engineering and snapped out of it.)

Oh, and I completely love Hal's "| grep -v $$", which I never would have thought of.

On reflection, the only thing the join would normally see is the ps itself, which is gone by the time of the join, so for the real deal, this can be pared down a titch.

# ps -e o pid,cmd > baseline; while : ; do

> sleep 5

> ps -e o pid,cmd > now

> kill -9 $(join -v2 -o2.1 baseline now)

done 2>/dev/null &

Just kill everything new and ignore the error message from not being able to kill the ps:

"Kill them all and let The Kernel sort them out."

Just in case

I just learned something new about shell syntax -- specifically, about the case statement.

The shell's big enough that I expect I'll be able to keep learning things about it for a long time, even though I've been writing shell scripts for ... lessee ... about 30 years.

Here's today's:

I'm used to writing this:

case $key in

whatever) do-something ;;

*) some-default-behavior ;;

esac

I just read a Fedora system script that looks like this, instead:

case $key in

(whatever) do-something ;;

(*) some-default-behavior ;;

esac

The leading paren is optional, but legal. And not just for bash, but for any POSIX shell. Amazing. Also, the last item doesn't have to have the semis, so it could even be this:

case $key in

(whatever) do-something ;;

(*) some-default-behavior

esac

Back Off Man, I'm a Scientist

"I have no faith in anything short of actual measurement & the Rule of Three"

-- Charles Darwin

In an interminable thread about recycling computer parts, on the Boulder Linux Users' Group mailing list, Davide Del Vento injects a completely unsubstantiated and largely dubious claim:

The point of the discussion is what we agree is ok to "kill" or "destroy", and what is not. Is it ok to kill a cow to feed my kids? Most people would say "yes". Is it ok to kill Jeffrey Haemer to feed my kids? Most people would say "no".

Doing science means collecting data. The polls in the left sidebar test his claim.

We shall see, eh?

NCAR Software Manufacturing Presentation, the Movie

Courtesy of Davide Del Vento, here's the video in H.264.

NCAR Manufacturing Presentation

What I'll Say

How I'll Say It

Having The Exact Code You Need

Building What You Have

Testing What You've Built

Looking Closely at What You Tested

Sharing It After You've Looked At It Long Enough

"What Was He Talking About?"

"Who Was That Masked Man?"

Performance Tuning Shell Scripts? Why, yes.

Can I write a command-line that lists open TCP ports as quickly as nmap? No way. But can I make one that's fast? Yes, indeedy.

The trick is to do it all in a single shell command.

I always read Hal Pomeranz's weekly, Command Line Kung Fu blog. Inevitably, either I learn something because he tells me, or I learn something because it gets me thinking about how I might do what he's done differently. (That's not an exclusive-or.)

This week, Hal writes a command-line that looks for open TCP ports.

Here's his command. (For a dramatic reading, see his post.)

for ((i=1; i<65535; i++)) ( echo > /dev/tcp/localhost/$i ) 2>/dev/null && echo $i; done

It isn't fast, and he ends with, "But really, if speed were a factor you'd be using nmap instead."

But can I get it to run faster? At least a little? Why, yes I can.

Replacing this:

for (( i=1; i<65535; i++ ))

by this:

for i in {1..65535}

and this:

echo > /dev/tcp/localhost/$i

by this:

> /dev/tcp/local/host/$i (or even < /dev/tcp/localhost/$i )

make minor improvements.

But a bigger win comes from getting rid of subshells.

The parens around the echo create a subshell, which requires a fork() and an exec(), each time through the loop.

By getting rid of those, and discarding error messages at the end of the loop, all the work takes place right in the parent shell.

How much does that improve things? A lot. Here are the numbers.

$ time nmap -p1-65535 --open localhost

real 0m1.366s

user 0m0.280s

sys 0m0.850s

$ time for (( i=1; i<65535; i++ )) ( echo > /dev/tcp/localhost/$i ) 2>/dev/null && echo $i; done

real 1m55.727s

user 0m12.640s

sys 1m28.200s

$ time for i in {1..65536} ; do >/dev/tcp/localhost/$i && echo $i; done 2>/dev/null

real 0m6.203s

user 0m3.290s

sys 0m2.780s

Tom Christiansen claims he can usually write Perl scripts that run within a factor of 'e' (2.718281828...) of the equivalent C program. Here, I'm only doing half that well, but that's not bad.

Even in the shell, sometimes a little tweak makes a big difference.

I'd offer extra points to the reader who knows an attribution for the quote "Make it work, then make it fast," but that would require readers.

It was, however, Frank Zappa who said, "Speed will turn you into your parents."

There's a Lesson Here, but I Can't Remember What

Perhaps you are graced with a mind like a steel trap. I have always had a mind like a steel colander.

I frequently read stuff I wrote, and think, "That's clever. Too bad I can't remember ever knowing that, much less writing about it."

Just now, I was reading an RHCE-prep guide that was explaining pr. I thought, "pr? Geez. There's some ancient history. Next they'll be explaining FORTRAN line-printer-carriage-control codes."

This starts me reminiscing.

"There was an old, Software Tools filter, in RATFOR, that would interpret those codes, called asa (a reference to the American Standards Association, a progenitor of ANSI's). At some point, it was ported to C/Unix. I should see if it's on my Ubuntu desktop."

(As an aside, and before I forget to say it, Kernighan and Plauger's Software Tools is the best book ever written about software engineering.)

It's not there. I think, "Well, okay, I'll install it.

I try apt-cache search and don't find it. Rats.

I google for an Ubuntu version. Nothing. A Linux version? Nothing. Humph.

Well, surely it was in UNIX Version 7. I remember some work Tom Christiansen put in, collecting Perl implementations of old, V7 commands. Maybe he found an implementation of asa(1) that I can just port.

Except I can't remember what he called his collection. I go back to googling, this time for Tom's collection. After a bunch of failed tries, I finally get a hit. You guessed it: a column by Jeff Copeland and, um, me -- Software Ptools -- which I have no recollection of ever having written. How embarrassing.

I should have given up right there, while I was behind, but Noooo .... ("What would you pay? But wait! There's less!")

Had we provided the name of Tom's project? Sho 'nuff: "Perl Power Tools." Maddeningly, the link in our column has gone dead. The universe hates me and there's no beer in the fridge.

I scroll down, hoping for another link. Ooh! Look! There's code! We implement a V7 command, right there in the column, to contribute to PPT ourselves.

We implement asa(1)

Oh, ow.

(I've now learned that the entire Perl Power Tools project has been moved to the CPAN by Casey West.)

Estimating: The Envelope, Please.

How much does it cost Amazon to ship me a Kindle book? About a nickel.

How much did it cost us to get letters saying we're going to get census forms? About $50 million.

How do I get these? Back-of-the-envelope calculations.

Back-of-the-envelope calculations are the quick calculations we do, from simple assumptions, to give us a sense of rough sizes. They may not let us tell whether the answer is 5 or 9, but they can let us see the answer isn't 5 billion -- a 5 followed by 9 zeros.

My sister, Jo, the Tattooed Lady, wondered out loud, this week, "... just how many millions of dollars it cost The US Commerce Dept (read 'us, the taxpayers'), to send everybody in the US a letter this week that says that they will be sending us a census report to fill out. 'Ooooooo. Look out!!!! Here it comes!!!' "

Let's do a back-of-the-envelope calculation. (No pun intended.) It's not hard.

How much does it cost to send a letter? A first-class stamp costs $0.44. The USPS loses money, which is why they want to cut back to 5-day-a-week delivery. So the real cost of processing and delivering a letter is something like $0.50. Could it be $0.30? Or $0.72? Maybe. But it's less than five bucks and more than a farthing.

What's the cost of producing each letter -- printing, stuffing, and so on? At Kinko's, they'd charge you somewhere between a nickel and a dime. Ditto for the public library. Real money, but we're still talking a total cost of around half a buck per letter.

They sent one to each household, and America has over 100 million of those.

We paid at least fifty million bucks for those letters. $50,000,000 . As Jo says, "Here it comes."

But what did we pay to draft the letter, translate it into a bunch of other languages, and get all that approved and processed through our Federal bureaucracy? Probably not even an extra fifty million.

Here's a second example: What sort of profit is Amazon making on Kindle books? I wondered this last year when I bought my Kindle.

Let's see .... Once they've payed the publisher for the book, they probably get a machine-readable version for next-to-nothing -- maybe free. Converting to the Kindle data format is probably done by a piece of software that they wrote once, and amortize across all their books, which means that probably doesn't contribute much either. Amazon's big cost is probably delivery -- what they pay Sprint to get it to us.

So how much is that? Hand me that envelope.

They'll sell me a subscription to a blog for about $2/month. The content is free if I have a browser, and I can't imagine they're trying to make a lot of money from these, either. The $2 is probably Amazon's delivery cost.

The kind of person who reads a blog on his Kindle is a junkie who, I'll guess, might read it three times a day. That's 30*3 = 90, or about a hundred deliveries a month: two cents a day. Books are bigger, but they come over so fast that I bet connection-set-up and -tear-down costs dominate the price.

Amazon's sells new releases at $9.99. This calculation says almost all of that is profit. Their delivery cost, I guessed, was under a nickel a copy.

How close did I come? In a January press release, Amazon revealed it was "less than six cents."

When the government can send us useless letters by Kindle (or email), they'll cost us far less.

"But what could the government do with its vast inventory of surplus envelopes?" the politicians will ask.

Two suggestions come to mind.

Collatz Conjecture

I like this shell script, by Kyle Anderson.

I found out about it because Paul Hummer's created a Northern Colorado Linux Blog aggregator.

Thanks, Paul! And Kyle.