And Then There Were Five

After spending 3D printing a rack enclosure, this weekend I finally put together an upgraded cluster.

Like before, I set up Disco, Spark and Jupyter on it, as well as doing some pretty significant upgrades to my monitoring tools and the cluster dashboard itself:

(Everything’s available here under a MIT license, as usual.)

So far I’m pretty happy with the overall performance, but very annoyed at Raspbian, since the default install (which I didn’t use on my previous cluster) uses a swapfile instead of a swap partition. That means nodes stall every time I try to handle large chunks of data, which kind of negates the point of building a cluster in the first place.

(The previous Model B cluster was very slow, but at least it didn’t flake out on me.)

I’m also getting weird “illegal instruction” errors when building some packages I need, which confirm my that Raspbian on ARMv7 is still half-baked at best.

I suspect I’ll be re-installing the lot fairly soon, but at least Jupyter makes it trivial to migrate my notebooks…

This page is referenced in: