Gophers and Crabs

Since I have gradually drifted away from coding regularly over the past few months (something I hope to fix soon), I have been trying to find something to keep my wits sharp. And, preferably, as low level as possible.

This last bit may well be my trying to compensate for the mega-complex stuff I am into during working hours, given that dipping my toes into tiny, manageable problems that I can tackle in one sitting has always proven to be very satisfactory.

And I quite enjoy doing low-level programming, preferably in tiny machines and as close to the metal as possible.

The only significant issue around that these days is in what language I should be doing it in.

I don’t really feel like doing Rust (I might be a little optimistic here, but I never had any trouble with C), and I’ve always been a sucker for dead simple cross-compiling, so I keep circling around Go.

People who find Go “weird” are often unaware of the connection to Plan9, Alef and Limbo (the core language of Inferno, which shares similar quirks).

But even if you’re aware of the heritage and keep the stated goals of Go in mind, there are a few things I actively dislike and find to be perennial downers:

• Error handling forces people to add lots of noise and often unreviewed boilerplate code, all of which feels like a rather pointless chore.
• Dependency management is downright weird if you want to use it on internal projects. I really don’t care if it’s go get, vendoring or modules, I just wish packages weren’t namespaced to GitHub and that this was done right from the start¹.
• Data serialization. Marshaling data via structs feels incredibly kludgy and is painfully ugly to write.

I have a few more complaints, but those are the ones that put me off the most.

Still, I have written a few interesting things in Go. In particular, I had a Python daemon which did the following:

• Every half an hour or so, fetch metadata from a bunch of URLs to compile a list of files to collect from various places.
• Reliably authenticate and download each (sizeable) file as soon as possible, but throttle the transfer to avoid saturating DSL connections (whose uplink is often a much smaller fraction of downlink speeds).
• Drop each download into the closest-matching folder (measured by Levenshtein distance).
• (Optionally) send out a progress notification.

The challenge I had was that the hardware it was supposed to run in was severely constrained (I had much less than 128MB of RAM to work with), and it was an ancient ARMv5 device, so CPU usage was also a concern (the script would take up well over 50% CPU in occasion).

So I decided to port it to Go, which took me a weekend (and change, since I eventually wrote a new configuration file parser for it as well). The code was already largely functional in nature, but I changed it to be an almost pure CSP implementation (spawning a goroutine for each URL or file at various steps in the process, and piping the results onwards to a single channel at the end).

And guess what? CPU usage went down to 5% (effectively zero), and RAM footprint was negligible as well. Better still, I built the 9MB binary on my Mac² and deployed it via ssh in around 10 seconds, so the turnaround time was really quick.

This ran for at least a thousand days without a hitch, until the whole system was overhauled and we had to change how things were moved around, but it just worked for the entirety of the time.

Good thing I didn’t need to update it, though, since ARMv5 support has since been dropped–although the last build I did was generated from a container image I can still run if needed be.

And, in general, I’ve found Go to be a great replacement for many of my Python projects.

The thing is that when you’re always pressed for time, you start to value enjoyment as well as performance and features, and I just don’t enjoy writing Go (which is one of the reasons why piku, for instance, is written in Python).

Serendipitously, as I was writing this, Dave Cheney wrote about the Zen of Go, and borrowing heavily from the Zen of Python in the process. That caused a bit of cognitive dissonance in my mind, for as I read it I found myself generally agreeing with the issues, but not the way Go tackles them.

Circling back to the initial section, it’s too early to say that I’m going to have a serious go at Rust (pun intended) since I don’t think I could enjoy it either.

After all, whether or not I can put up with another programming language that looks like line noise is actually a better question–Clojure and LISPs have spoilt me in many regards (readability, sane concurrency and functional programming being the top 3 things I love about them).

But I find myself wanting something that will be really efficient on ARM hardware, so it might well happen. Given time, of course.