NCAR Manufacturing Presentation

What I'll Say

How I'll Say It

Having The Exact Code You Need

Building What You Have

Testing What You've Built

Looking Closely at What You Tested

Sharing It After You've Looked At It Long Enough

"What Was He Talking About?"

"Who Was That Masked Man?"

Performance Tuning Shell Scripts? Why, yes.

Can I write a command-line that lists open TCP ports as quickly as nmap? No way. But can I make one that's fast? Yes, indeedy.

The trick is to do it all in a single shell command.

I always read Hal Pomeranz's weekly, Command Line Kung Fu blog. Inevitably, either I learn something because he tells me, or I learn something because it gets me thinking about how I might do what he's done differently. (That's not an exclusive-or.)

This week, Hal writes a command-line that looks for open TCP ports.

Here's his command. (For a dramatic reading, see his post.)

for ((i=1; i<65535; i++)) ( echo > /dev/tcp/localhost/$i ) 2>/dev/null && echo $i; done

It isn't fast, and he ends with, "But really, if speed were a factor you'd be using nmap instead."

But can I get it to run faster? At least a little? Why, yes I can.

Replacing this:

for (( i=1; i<65535; i++ ))

by this:

for i in {1..65535}

and this:

echo > /dev/tcp/localhost/$i

by this:

> /dev/tcp/local/host/$i (or even < /dev/tcp/localhost/$i )

make minor improvements.

But a bigger win comes from getting rid of subshells.

The parens around the echo create a subshell, which requires a fork() and an exec(), each time through the loop.

By getting rid of those, and discarding error messages at the end of the loop, all the work takes place right in the parent shell.

How much does that improve things? A lot. Here are the numbers.

$ time nmap -p1-65535 --open localhost

real 0m1.366s

user 0m0.280s

sys 0m0.850s

$ time for (( i=1; i<65535; i++ )) ( echo > /dev/tcp/localhost/$i ) 2>/dev/null && echo $i; done

real 1m55.727s

user 0m12.640s

sys 1m28.200s

$ time for i in {1..65536} ; do >/dev/tcp/localhost/$i && echo $i; done 2>/dev/null

real 0m6.203s

user 0m3.290s

sys 0m2.780s

Tom Christiansen claims he can usually write Perl scripts that run within a factor of 'e' (2.718281828...) of the equivalent C program. Here, I'm only doing half that well, but that's not bad.

Even in the shell, sometimes a little tweak makes a big difference.

I'd offer extra points to the reader who knows an attribution for the quote "Make it work, then make it fast," but that would require readers.

It was, however, Frank Zappa who said, "Speed will turn you into your parents."