sslhin my HTTPS port (bypassing several firewall limitations in various places)
|Update, 2018: I built my own.|
A few years later, I "graduated" to building my own ARM server from scratch.
When you finish reading this post, I recommend you go read that one as well.
About a year ago, a no longer functional NAS device made by Western Digital came into my hands - a MyBook World Edition. This is a rather old, low-powered, ARM-based file server... and even though it is considered obsolete by today's standards, I knew the moment I got it that I would very much enjoy hacking it to do my bidding :‑)
I am a programmer by vocation, but I don't really believe in the separation of programming and administration - in my humble opinion, there are a lot of advantages in practising both.
First, I had to break the thing down, so I could build it up to what I wanted. Following in the tradition of many engineers before me, I slaved over it for half an hour and succeeded in disassembling it - only to realize afterwards that other kind souls had documented the process. Oh well - figuring things out on your own is the best part about all this :‑)
I proceeded to remove the two drives in the machine, and after attaching them
to my external USB/SATA enclosure, run a quick SMART check on them:
-a /dev/sdX showed that both drives had bad sectors (Reallocated_Event_Count,
Current_Pending_Sector were both non-zero - that's not the kind of drive you
can trust)... I therefore cannibalized an old 160GB external USB disk, and
attached it to the empty enclosure.
Since this is an embedded platform, with no VGA or serial plug on it, I needed to find a way to monitor its boot process. After a bit of Googling, it turned out that the board in fact had soldering pads for traditional RS-232 serial interfaces. The "specs" provided by the almighty Internet were simple:
...and since we are way past the age of motherboards with RS-232 interfaces, I ordered a TTL/RS-232 cable from an online electronics store (translation: a cable that I can attach to the board on one end, and plug the other end into a normal PC USB port - where the serial interface will be accessible by any serial port program).
Two days later, the package arrived - so I soldered the 3 pins (2,3 and 4) and hooked them up to the USB/serial cable.
On my main machine (an Atom 330 based ArchLinux), I attached the USB cable,
fired up a serial interface program (
minicom -D /dev/ttyUSB0 -b 115200),
and then powered up the little board...
I was expecting this to show some kind of BIOS, but maybe this tiny board had no such luxury... maybe it read everything it needed from the attached drives?
I mounted the old drives up to my main PC's USB/SATA enclosure, and sure
enough, I saw clear signs of a Linux-based machine: the software RAID driver in
my ArchLinux detected raid devices (
cat /proc/mdstat showed information about
the RAID structure). Apparently, MyBook had the two drives in a RAID formation
- which I proceeded to successfully mount. There were 4 partitions in each of
the two drives, clearly in a RAID1 mirror formation - with the 4th and final
partition being the one with the "File Server" storage area.
I proceeded to copy the first three partitions (including the partition table)
to my 160GB drive (via
dd). I then used
fdisk to fix the size of the 4th
partition to be the remaining space of my drive.
Attaching the 160GB drive and booting again, minicom logged tremendous improvement:
Welcome to minicom 2.6.2 OPTIONS: I18n Compiled on Mar 5 2013, 16:40:55. Port /dev/ttyUSB0, 12:22:35 Press CTRL-A Z for help on special keys �NASOx_0800 Mon Aug 5 21:45:27 EEST 2013 U-Boot 1.1.2 (Jan 21 2008 - 08:50:09) U-Boot code: 48D00000 -> 48D17648 BSS: -> 48D1B2B8 RAM Configuration: Bank #0: 48000000 32 MB In: serial Out: serial Err: serial Initialising disks No FIS received from device 1 Detecting SATA busses: Bus 0: Found first device OK Device 0: Model: TOSHIBA MK1652GSX Firm: LV010J Ser#: 29GGF8WNS Type: Hard Disk Supports 48-bit addressing Capacity: 152627.8 MB = 149.0 GB (312581808 x 512) Device 1: not available IDE read: device 0 block # 63, count 1 ... 1 blocks read: OK
...and proceeded to boot a normal Linux kernel.
Having access to the Western Digital provided Linux of the board was a very good start.
I then succeeded in using the GPL sources from Western Digital to
built my own kernel and busybox-based mini-distro.
I wanted more than just a toy, though ;
and since I use Debian at work, I followed Mario's consolidated version of instructions
('Quick install steps'), and in 15min, installed the
debootstrap-ed main parts
of Debian on my 160GB drive.
When possible, I always prefer doing things from the console - not only does it improve my knowledge of the OS I work with, it also allows me to do things over serial lines or SSH connections. In the case of this little ARM box, I applied the same knowledge that I am using for normal machines: editing /etc/network/interfaces, /etc/resolv.conf, etc.
Sadly, this is a skill that is fading away, with people becoming increasingly dependent on "wizards"... and facing the inevitable side-effect: clicking on buttons while being unaware of what is going on behind the scenes, ends up with systems that keep running slower and slower, and can only eventually be "rescued" with... clean reinstalls. In contrast, under UNIXes (Linux, the BSDs, etc) your applications' settings are simple files and folders under /etc and $HOME... Personally, I have always kept them under revision control (e.g my main dot files and vim configuration), happily migrating them over the last 15 years across the many machines I've worked with. Backing them up and restoring them, e.g. when I need to work on a brand-new machine at work, is simply a matter of checking out files and folders from my repositories...
And that alone, from what I can see in my dealings with some of my colleagues,
is something they would kill for. And if they ever realized what
does, and has been doing for decades...
But I digress - people reading this blog probably already know all this.
I quickly setup the network interface, and got up to a working...
# apt-get update && apt-get upgrade
At that point, I knew the hard part was over - I now had a Debian/ARM distribution, which I could configure to do whatever I wanted.
# cat /proc/cpuinfo Processor : ARM926EJ-Sid(wb) rev 5 (v5l) BogoMIPS : 99.73 Features : swp half thumb fastmult edsp java CPU implementer : 0x41 CPU architecture: 5TEJ CPU variant : 0x0 CPU part : 0x926 CPU revision : 5 Cache type : write-back Cache clean : cp15 c7 ops Cache lockdown : format C Cache format : Harvard I size : 32768 I assoc : 4 I line length : 32 I sets : 256 D size : 32768 D assoc : 4 D line length : 32 D sets : 256 Hardware : Oxsemi NAS Revision : 0000 Serial : 0000000000000000
...the machine definitely doesn't win any contest: it's very low powered, but that can also be seen as an advantage: nothing wrong with an always on "server" that consumes 0.7 watt when idling! With the 160GB drive added in the mix, I get up to 3-3.5W... not bad at all.
I am behind a DSL line, so my IP address is constantly changing. I needed to be able to get to my machine no matter where I am, so I opened up a (free) account on DynDNS, and set it up.
Since the board only has 32MB RAM, I chose against using the DynDNS Perl script, and instead used a native C client, inadyn:
# apt-get install gcc ... # wget https://.../inadyn.v1.96.2.zip # unzip inadyn.v1.96.2.zip # cd inadyn # make ... # cp bin/linux/inadyn /usr/local/bin
I then configured it to run at startup, via cron:
# cat /var/spool/cron/crontabs/root ... @reboot /usr/local/bin/inadyn
...and set up my DynDNS credentials:
# cat /etc/inadyn.conf --username USER --password PASSWORD --alias UBER-SECRET.dyndns.org --update_period_sec 300 --background
That's it - once every 5 minutes (5 x 60 = 300), the tiny server communicates it's current IP address to DynDNS:
# host UBER-SECRET.dyndns.org UBER-SECRET.dyndns.org has address AA.BB.CC.DD
Update, a year later: DynDNS removed its free plan, so I registered
to two more free DNS providers, EntryDNS and
DuckDNS. They both work via 'magic' web requests,
so I simply set up two cron jobs (running every 5 min) that invoke
passing in the string that identifies my machine.
So, now that the ARM box had an Internet 'name', it was time to setup... "stuff" :‑)
First, I installed a web server - and was now able to export everything that I wanted, to friends and family. As long as they had a browser, they could connect to my little ARM.
Initially, I went with NGINX - but since the board only has 32MB of RAM, I switched to Lighttpd, which is designed to have a smaller memory footprint.
Not that the tiny ARM server and my 700 Kbit upstream DSL speed would ever survive a Slashdoting, of course :‑) Still, there are other uses: I rsync-ed the photo folder of my (jailbroken, company-provided) iPhone, and published it in a password-protected folder...
# rsync -av mobile@iphone:/private/var/mobile/Media/DCIM/100APPLE/ \ /var/www/site/Media/
My pictures therefore became accessible from anywhere in the world, just by browsing to my mini-server (and using the folder password). Nifty!
I added this rsync to a cronjob, so my iPhone's photos are rsync-ed automatically every night, while I sleep and the iPhone is charging.
Privacy issues not withstanding, it's nice to be able to have a mail presence
that doesn't depend on others.
apt-get install exim4, and my friends can now
e-mail me at ttsiod@UBER-SECRET.dyndns.org. I read the mail over SSH, via
'mutt' - which runs fine even in this tiny little CPU.
In a world populated by self-respecting human beings, that would be the end of it ; unfortunately, even though GMail accounts accepted mail sent from me with no problems, others (e.g. Yahoo) considered me a spammer, since I was sending mail from the dark pits of hell (i.e. an IP belonging to a DSL line). How can they be sure that I am not a zombie Windows machine, infected with malware and serving The Spam Lords?
Neither SPF nor DomainKeys convinced them - so I switched my outgoing direction
smarthost - and therefore route outgoing mail via my ISP.
There are places where hitting my SSH server (e.g. to use it as a SOCKS proxy
ssh -D ...) is impossible, because there are firewalls in place that only
allow HTTP traffic.
Initially, I tried exposing the server over HTTPS's port (443), but that was
not enough. I ended up using sslh,
which cleverly sits between a port and a number of daemons. In my case, since
it speaks enough of the SSH and HTTPS protocols, it can determine when an
incoming connection is hunting for SSH responses, and when for HTTPS responses
- and tunnel the request to the proper local daemons (
firewall therefore sees me as a legitimate HTTPS site (which I am, thanks to
Lighttpd) and lets
ssh -p 443 ... pass.
I can download anything I want with my little server.
rtorrent is a text-based
torrent client that works fine, but since I am not exactly a mainstream guy,
the things that actually interest me are usually found elsewhere.
Mostly, I prefer gathering up interesting URLs and setting up
scripts that use
youtube-dl to download them inside
sessions. This way, when I get back from work, I mount the tiny server's
download folder from my jailbroken Android tablet (over Samba - i.e. with
mount -t cifs ...), and enjoy watching without network 'hiccups'.
In case you're wondering, I am currently watching Drew Neil's amazing Vimcasts
series, and egghead.io's videos on AngularJS.
Scrapping is relatively easy - I scrap enough HTML to find the video URLs,
and then feed them to a
wget with a rate limit - so that I don't overload
the kind people that share these treasures.
The Debian rootfs from Mario was made back in 2007, in the days of Lenny.
I wanted to move past that, to today's Debian Wheezy. Starting with a chroot
test in a
debootstrap-ed Wheezy, I realized this would not be easy:
# # From inside the little ARM machine running Debian Lenny $ sudo debootstrap --arch armel wheezy /wheezy \ http://ftp.whateverCountry.debian.org/debian $ sudo -i # mount -t proc none /wheezy/proc # mount -t sysfs none /wheezy/sys # mount -t devpts none /wheezy/dev/pts # chroot /wheezy Fatal: Kernel too old
The fatal message comes from the GLIBC of wheezy, informing that GLIBC was not compiled to work with this old kernel (2.6.17). So, can we recompile a newer kernel for this SoC?
No - not easily. Even though WD provides the sources of the custom 2.6.17
kernel it used, it never updated them to later versions, and they were never
merged with the mainline kernel. The
oxnas SoC is doomed to die a peaceful
But not before its time :‑)
Because even though I can't spend the time to port all the device drivers in the custom WD 2.6.17 to newer kernels, I can try something else: I can compile the Wheezy's GLIBC to support older kernels - including this one!
I asked for help on StackOverflow and UnixStackexchange with this, but eventually I figured it out on my own and
posted my own answer. I basically used
crosstool-ng to cross-compile GLIBC
on my Core i5 and generated the ARM-ie version of GLIBC that could run on my
After all that, I can probably proclaim my WD MyBook World Edition to be the only one on the planet running Debian Wheezy. Maybe it should now be called WD MyBook Galaxy Edition :‑)
If anyone else is interested, I can share a tarball of my updated libc files.
If you think about it, the end result is rather amazing - and done completely over ARM processors, not Intel ones:
rsyncmy phone's photos daily to the little server, and therefore my photos are always available over any browser. Update: I also ended up using lazygal to create a nice arrangement of my photos in galleries with previews and navigation controls.
sshto the tiny server and use Wake-On-LAN on my main desktop, to easily access any of the work I've done in my coding life. Or my music collection :‑)
youtube-dl) from within
screen-ed sessions, and share them over Samba to my house-bound devices (e.g. watch movies from my jailbroken Android tablet)
sslh, I can bypass several firewall limitations that exist in various places, and access my tiny server.
crosstool-ng, I was able to cross-compile GLIBC, and bring the machine up-to-date with today's Debian packages. It is a server, after all - it needs to get security updates (
In plain words, UNIX power - in the tiny server, in my phone, in my tablet. All 3 of them, running on ARM processors. To be honest, I didn't expect that ; 15 years ago I was sure that Intel and Microsoft had cornered the galaxy... but somehow, Linux managed to change all that.
I still have to jailbreak them and/or hack them to do my bidding, of course - the world is still not perfect.
But that's why this is fun :‑)
Updated: Sun Oct 22 14:41:45 2023