Daily Show fodder

This came from a heretical Lutheran friend of mine:

Vatican airline takes to the skies

I love the Ryanair quote:

“Ryanair already performs miracles that even the Pope’s boss can’t
rival, by delivering pilgrims to Santiago de Compostela for the
heavenly price of 10 euros,” Ryanair said in a statement.

So many questions:

Will the Swiss Guard perform TSA or Air Marshal duties? Will they get
to carry their halberds?

Is there any point in the in-flight safety briefing, since anything
that happens would be God’s will? Seems like they could get by with a
priest going down the aisle swinging a censer, if they disabled the
smoke detectors.

Will they take any special precautions with unaccompanied minors?

Here’s hoping the Daily Show doesn’t let this one pass.

Bencode, Bdecode in shell script

I have been playing around with OpenWRT recently, to see what IT tools you could deploy to the bush using cheap, low power computers as a base.

One of the really interesting things about writing systems to live in the embedded Linux world is that you want to try to do as much as possible with the stuff that’s there out of the box. This is because smarter people than me have stuffed the tiny 8 meg flash images with useful tools, leaving no room for my bloated Perl-ware. So the question becomes, how can I live with what they gave me and still get what I want done.

I wanted to do some secure file exchange stuff, and have it work over multiple protocols. So I was investigating OpenSSL. It turns out you can do everything you need to run a custom PKI with the commandline OpenSSL program and shell scripts. You need a way to marshal the various pieces together (the public key, the session key, the data itself, a hash, etc) and wrap them up, then unwrap them at the other end. The professionals use complicated systems based on BER and so on. Then they need to link against huge binaries to make it all work right. But I only had busybox available to me. Could I still make something useful?

I started off thinking about tar, which can put a bundle of data together. But I really like the tight encoding in BitTorrent’s Bencode format, and sending around tarfiles seemed kind of hokey (but don’t laugh, that’s all .deb files are, and RPMs are just CPIO files). So I thought it would be nice to have Bencode/Bdecode available for me to use. But the only standalone tools I could find used interpreted languages, and adding Python to my little Linksys box was exactly what I was trying to avoid.

I ended up, basically as a kind of sick joke, writing my own Bencode/Bdecode in shell script. Don’t laugh… people were doing this in the 80’s with shar files, which had the added benefit that they could uncompress themselves! Let’s see a Bencoded structure do THAT!

I succeeded, at least for what I needed. My Bencode/Bdecode pair are below. Note that these guys only implement non-recursive dictionaries, which was all I needed for my application. Adding full support would be easy, though deciding on a filesystem representation of an ordered list is a bit of a problem.

Bencode in shell script

#!/bin/sh

# usage: bencode [dir]
# encodes all the files in [dir] in Bencode format.
# See http://en.wikipedia.org/wiki/Bencode for the defn.
# Note this tool does not implement recursive structures,
# only dictionaries.

# for the inevitable port because some echos are less compatible
e="echo -n"

if [ "$1" = "" ]; then
echo "usage: bencode [dir]"
exit 1
fi

dir=$1
if [ ! -d "$dir" ]; then
echo "$dir is not a directory."
exit 1
fi

keys=`echo $dir/* | sort`

$e d
for f in $keys
do
if [ ! -f $f ]; then
echo "$f is not a file."
exit 1
fi

key=`basename $f`
klen=`echo $key | wc -c`
klen=`expr $klen - 1`

vlen=`ls -l $f | awk '{print $5}'`

$e "$klen:$key"
$e "$vlen:"
cat $f

done
$e e

Bdecode in shell script

#!/bin/sh

# usage: bdecode [dir]

# decodes stdin into [dir]
# Stdin should be a a Bencoded dictionary of strings (but not general
# Bencoded forms, just a dict of strings)

# dd wrapper
substr () {
if [ "$2" != "" ]; then
count="count=$2"
else
count=
fi
dd bs=1 skip=$1 $count 2>/dev/null
}

# put tmp2 on top of tmp
m () {
mv $tmp2 $tmp
}

dir=$1
if [ "$dir" = "" ]; then
echo "usage: bdecode [dir]"
exit 1
fi

mkdir -p $dir

tmp=/tmp/bd1.$$
tmp2=/tmp/bd2.$$
cat > $tmp

d=`substr 0 1 < $tmp`
if [ "$d" != "d" ]; then
echo "Not a dictionary."
exit 1
fi
substr 1 < $tmp > $tmp2
m

while [ -f $tmp ]
do
n=`awk -F: '{print $1; exit}' < $tmp`

if [ "$n" != "e" ]; then
nlen=`echo $n | wc -c`
substr $nlen < $tmp > $tmp2
m
key=`substr 0 $n < $tmp`
substr $n < $tmp > $tmp2
m

n=`awk -F: '{print $1; exit}' < $tmp`
nlen=`echo $n | wc -c`
substr $nlen < $tmp > $tmp2
m

substr 0 $n < $tmp > $dir/$key
substr $n < $tmp > $tmp2
m
else
rm $tmp
fi
done

Note: these routines won’t win any speed records, but they do work reliably when all you have is BusyBox’s ash, dd, and awk.

A Look at Field Life for Tool Makers

I have recently been talking with some folks about IT tools for humanitarian aid workers. I gave a real-world reality check to them, not to discourage them, but to make sure they knew their audience as well as possible. After all, what good is a tool that looks good in demos, but fails with the users? A good step towards avoiding abject failure is to sit with your users and watch them work. That’s not so easy with humanitarian field workers. A tolerable substitute is to collect testimonies from them with the same kind of data, and try to design a system that has a chance of working in the context.

One of the guys I was writing to asked me to post my experiences publicly. Ok! No problem! And hopefully I can find a few more people to post their own, and we’ll link them together so if a tool-maker can find one, she can find all of them!

My experience was in Liberia, as the log for an MSF-operated basic health care project in a stable post-conflict context. Our security rules limited our movements to either a handful of pre-approved places inside the city, or pre-planned movements. The rest of the time we were in our compound (our curfew for all people and vehicles was sunset). We had twice-weekly car trips to the capital city, which was 6 hours by car away.

My day was not the crazy 20 hour day you live on a vaccination project. Instead, my day started at 7am with a normal breakfast. From then on, I chased technical details (security, food inventory,
vehicle maintenance, construction management) and human resources
problems until 5pm, with a solid hour and a half of quiet time around
lunch. I worked 10-15 hours a week after hours on accounting work. I chose to work this way because of the workload, and because these tasks requiring concentration are best accomplished in the peace and quiet of the evening. Also counting thousands of US dollars with extra people around the office is discouraged by the security rules (and common sense).

Once a week or so we would have a movie night where the guy from Action Contre le Faim next door came over and shared their 80 gigs of DIVX movies with us. This was the maximum extent of inter-organization data sharing! I slept 8 hours a night, except 2 nights a week on average when I was awakened for an hour or so for some problem (generator, transport problem, freight arrival). I spent 4 hours a week helping MSF nurses with their data entry into (brain damaged) Excel worksheets. Having a better health metrics system, and trained users, would have reduced the stress of monthly reports on everyone.

I had a GSM phone on the global network (in/out to USA and Europe as
easy as: 00 country-code local-number). I had reliable e-mail
connectivity via satellite with a bit rate of around 100 kbit/sec. I
could not do HTTP nor HTTPS. I submitted my reports and did “offsite
backups” by burning a CD and sending it on a car to Monrovia. We used Windows 2000 on Thinkpad laptops. We used Thunderbird for e-mail, Microsoft Office tools, VLC for playing movies. We had a projector, which we used for showing training films and presentations, and for movie night.

I think I would have had some time and attention available (perhaps 3 hours a week) for IT-based mapping tools and some data entry, especially when it could generate instant gratification in our local area of operations. We did not have the resources to debug stuff, download patches, or write programs. I could not use Google Maps, let alone Google Earth. It would have been easy for me to receive GeoRSS-formatted e-mail attachments and see them rendered, but only if I had a local map drawing system.

If other field workers would like to give a short description of their work environment to set the stage for tool makers, I would be happy to host your comments here, or to link to them if you put them on your own website.