After last year’s success with Hack My Derby, a hacked up derby had that we created for Derby Con 5, I decided to create a sequel for Derby Con 6!  Much of the build is similar, so I’ll cover the differences.  If you want a more complete write-up on this project from start to finish, you might want to check out last years write-up linked above for some background. I’ll give you a summary though. 

In 2015 I worked together with a co-worker to build a CTF inside of a derby, so that other DerbyCon participants could connect to it, via a WiFi access point that was hosted in the Derby.  With that access, you could then attack several containers that run on the derby. 

This year I expanded that by adding in Cell connectivity using Google’s Project Fi.  Over the Fi connection, I used Amazon’s EC2 to host an OpenVPN server, which the Derby would connect to.  Then the EC2 instance forwards ports over the tunnel back to the derby.  So the url “web.hackmyderby.com” would resolve to the EC2 istance, and then the EC2 instance would forward it over to the container on the derby. 

Build

I was mostly on my own this year. I did get some guidance from Rich, who helped me last year, on some of the electrical level stuff that I don’t have as much experience with. For instance, he helped me work out the internal battery. I also had some network help from Friz, an old friend and network pro, on some of the routing from ec2 to the derby and back.

Last year’s derby was great, but it had a few problems.  First, it was heavy and a tad uncomfortable to wear all weekend.  This was mostly because of the battery. We wanted a full day of runtime from the battery so we went with this huge 20Ah battery.  The second problem was range. The little wireless adapter worked great but had limited range. 

The solution to the battery problem came in the form of a dual battery setup.  I bought a little 250mAh battery and battery controller from adafruit.  That got mounted inside the derby and powered the pi directly. Then I mounted an external 4Ah battery outside if the derby. So my derby essentially had an internal UPS.  The internal battery holds the derby while I exchange external batteries.  The external battery runs about 5 hours.

For the range problem… We got creative.  At the beginning of the project this year, my CIO asked if i needed any help with the derby. I know he’s busy so it was unrealistic to expect him to devote a ton of time to the project. He is the sort of guy that questions what is possible though, and he’s got a networking background.  So I asked him if he had any ideas on how to solve the range problem. I was thinking of building cheap wifi repeaters and planting them around the con, that seemed problematic though.  Between the two of us we cooked up an idea.  We run a system somewhere stable. Then connect the derby over a cellular adapter to the internet.  Then vpn the derby into the gateway system.  Then connect the world though the vpn tunnel to the derby.  We bought a 3g adapter compatible with T-Mobile then i got a project Fi sim to get it online.  The whole thing sounds complicated, but in the end, its really quite elegant.  Making sure the cell connection stayed active was the tricky part.

Cell Connection

This seems so simple in todays connected age, but when you’re dealing with linux with no gui, and a cell provider with less support, it’ gets tricky.  I wanted to connect the derby to the internet via cellular.  I also didn’t want to add some line to an existing cell plan that was going to lock it into some sort of contract.  So I decided to give Project Fi a go.  See if I could find an adapter compatible with Fi, and then get a Fi SIM, and get it online from the command line.  I bought a generic Huawei E353 adapter.  Which is only 3g, but it was relatively cheap.  The one I ended up with is the “BAM 3G”. Fi uses both TMobile, and Sprint, and I’m not really sure if this adapter is supporting Sprint as well, but it does support TMobile, so it worked well enough for my purposes.

Like many 3G adapters, the E353 has two modes.  On Windows or Mac, it first presents itself as a mass storage device, and lets you install software to make it work.  Then it can switch modes to a 3G network adapter.  If I plugged it into my Gnome3 Fedora 24 laptop, it just switched right into 3G mode, and fedora was able to recognise and use it.  On Raspbian, with no gui, things werent so simple.  I had to use usbmodeswitch to flip it from mass storage mode, to network adapter mode.  This is supposed to be automatic once you get usbmodeswitch configured.  I didn’t have much luck.  I ended up getting usbmodeswitch installed, and then I had to import a bunch of definitions for various hardware.  Once I had them, I could manually switch modes from mass storage to 3G.  The problem is, whenever the USB dongle lost power (like, when I unplugged it, or powered down the Pi) it defaulted back to mass storage mode.  So I had to write a script which checked for the existence of the device, checked for its current mode, and switched it if it was in mass storage mode. 

With usbmodeswitch installed, I put the following content in /etc/usbmodeswitch.d/12d1:14fe

# Huawei E352 (T-Mobile NL), E173s (Variant)
TargetVendor=0x12d1
TargetProductList="14fe"
HuaweiNewMode=1

The way I understand it, this *should* have made my cell modem dongle switch to 3d mode automatically when I plugged it in.  It didn’t, I worked around the issue by creating a dialler script, wihch would check lsusb for the existence of the dongle, and determine which mode it was in, then switch it, if needed.

#!/bin/bash

STORMODE="12d1:14fe"
GMODE="12d1:1506"
GCONNECTCMD="sudo wvdial 3gconnect"
LOGFILE="/derby/HackMyDerby/scripts/cell-log"

#Make sure the dongle is in 3g mode
DONGLERET=`lsusb | grep $STORMODE`
DONGLERET=$?
if [ "$DONGLERET" -eq "0" ] ;
  then
    #echo "switching mode, return was $DONGLERET from lsusb | grep $STORMODE"
    DONGCHN=`lsusb | grep 12d1:14fe | awk '{print $4}' | sed s/00//g | sed s/://g`
    USBMODESWITCHCMD="sudo usb_modeswitch -I -W -D -s 20 -u -1 -b 1 -g $DONGCHN -v 12d1 -p 14fe -c /etc/usb_modeswitch.d/12d1:14fe >/dev/null"
    $USBMODESWITCHCMD
    echo "`date` - Switching USB Modes" >> $LOGFILE
    sleep 5
fi

The script would also then check to see if the Pi could already reach to the internet.  If it couldn’t it would connect the 3G adapter.  This was done using WVdial, and a dialler script.  Which was very reminicent of the old dial-up days.  WVDial would initiate the connection, using the script, then call pppd to actually start up the TCP/IP session.  Once I had all of this worked out, it worked beautifully.  I put my script in a cron job that ran every few minutes, and if it was already connected, it simply exited.

INETRET=`nc -w 5 -z 8.8.8.8 53`

INETRET=$?
if [ "$INETRET" -ne '0' ] ;
  then
    # we're not connected...
    # Here we should try open wifi networks, if they fail, 3g
    # check if 3g connection is active (but maybe dead?) by checking for pppd in ps
    echo "`date` - Still cannot see tha netz, trying to kill wvdial" >> $LOGFILE
    sudo killall -9 wvdial
    sleep 2
    sudo killall -9 pppd
    sleep 5
    echo "`date` - Attempting $GCONNECTCMD" >> $LOGFILE
    screen -d -m $GCONNECTCMD
    sleep 10
    sudo ip route add 52.43.253.73 dev ppp0
    sudo ip route add 10.177.0.0/24 dev ppp0
    sudo ip route add 8.8.8.8 dev ppp0
    sudo ip route add 8.8.4.4 dev ppp0
    exit 2
  else
    #we're golden
    echo "`date` - I can see the netz, we're good" >> $LOGFILE
    exit 0
fi

I’m sure there are better ways of handling all of this, but this worked for me on short notice.

VPN

I couldn’t count on Cell providers to allow connections on ports 22, 80, 443, and 23.  So I needed some way to get my mobile derby accessible to the internet.  I ended up configuring an OpenVPN server on Amazon’s  EC2.  Then I configured OpenVPN client on the derby.  Whenever the derby could get to the internet, it would automatically establish its VPN session.  once the vpn session was up, I could access the derby from the EC2 instance.

With that connectivity established, I setup some iptables NAT rules which port forwarded the desired ports over the VPN back to the derby.  This worked out really well, once I got it working.  A huge thanks to @xenophage for some routing assitance.  I was outright stuck, he stopped in one night and we spend a few hours hacking away, and in the end, he found a simple config change which got things working. 

Containers

I went with Docker containers again.  It worked out well last year.  Still no SELINUX support on the raspberry pi though.  I’m just waiting for that to bite me one of these days. 😉  I did a web container, an ssh container, and a CirclMUD (by popular demand) container.  I never checked in to see if anyone bothered with the mud.  The web and ssh containers were both part of the CTF though, and they were hit many times. 

Twitter Integration

Last year I ran out of time, but I wanted to add in some twitter integration, to make the whole thing a little more interactive.  With the derby being internet connected this year, it was easy.  I setup a python script on the derby to tweet whenever someone hit the “display” flag, allowing them to write to the display.  I also made a flag where you got half the code from the web page on the derby, and then other half by tweeting at Dave Kennedy letting him know how much you love him. 😉  That was a bot running on the EC2 instance.  I had it, why not use it?

Conclusion

Well, I have to say, this year was pretty awesome. Not only did I get to improve on the derby, and add in some of the features we missed out on last year, but I got to meet a ton of people because of the derby.  Oh, did I mention?  The derby took first place in the Home-Built derby Hack Your Derby competition. 😉 

That’s integroll in the scoreboard picture. He made a freaking plasma speaker in a derby, it was quite the feat of engineering.  You mgiht also notice Dustin at the top of the Built On Site category?  You were able to get a free plastic derby from Deviant when you registered, and you were expected to build a derby on site, with only what you had on hand.  So…  Dustin is a co-worker that came along with Xenophage.  They work together, and he decided it’d be a good experience to get him to derbycon.  He’s a cool guy, and wanted to build a derby.  I happened to have a freaking crate full of spare parts, including… A pi zero, and an extra OLED screen. 

In under a day, we hacked together a wifi pinapple derby, which would collect the manufacturers of all of the surrounding wifi devices, and output the top 5 on the screen, and… It won first place in the on-site category. 

Derby master, you can call me derby master. 😉