Self-hosting Mastodon is all the rage, but having to deal with a full-blown installation of Ruby (which is always a pain to install properly, even if you use
rbenv), plus the abomination that is Sidekiq and the overall Rube Goldberg-esque architectural approach that is almost mandatory to deal with the complexities of ActivityPub is just something I don’t want to maintain. Ever. Even inside Docker.
Which is why I have been developing my own ActivityPub server using Sanic and a very lightweight
asyncio-based approach at handling all the transactional aspects of ActivityPub atop SQLite. And let me tell you, I honestly wish the protocol was less about doing what boils down to P2P webhooks with PEM signatures embedded in requests.
But Takahē is now aiming to support client apps as of version
0.6, is built on Django (which I have always loved as a framework), and it saves me from the trouble of building everything from scratch, so… I had to try it out.
Besides running as a
WSGI app, Takahē uses an async stator to handle all the background tasks (which is also exactly the pattern I aim for and designed Piku to support), so I just had to see how easy it was to get it running under Piku on very low-end hardware.
I have a 4GB Raspberry Pi 4s set up as an SSD-backed Proxomox server, hosting several different
arm64 LXC containers I use for developing stuff. I love it because I can use LXC CPU allocations to throttle things and make sure they run fast enough on very low-end hardware, plus I can just snapshot, mess up and restore entire environments.
So I set up an Ubuntu 22.04 container with 1GB of RAM and access to 2 CPU cores, capped to 50% overall usage–which is roughly the performance of a Raspberry Pi 2 give or take, albeit with a fully 64-bit CPU.
I deployed Piku, set up a CloudFlare tunnel, and then went to town.
Zero Code Changes Required
- Clone the repository.
- Create a
productionremote pointing to Piku.
- Edit the supplied
- Do a
git push production main.
It was that simple.
Here’s the configuration I used, annotated. First the
# Yes, I went and got it to use SQLite, and it nearly worked 100%
# This is what I eventually migrated to (more below)
# I actually love Django debugging, and with it on I can see the inner workings
# You know who uses this password, don't you?
# No, it's not the one I'm actually using.
# Anyway, this next one breaks a little on Piku, so I need to revise parsing for this case.
# I also added a Redis server later, without any relevant impact in RAM usage
# This is all Piku config from here on down
# I need IPv6 off for sanity inside Proxmox
# This ensures nginx only accepts requests from CloudFlare, plus a few extra tweaks
# These are caching settings for my dev branch of Piku
# This has nginx cache these prefixes
# This maps static user media directly to an nginx route
# You want to set these, trust me. I should make them defaults in Piku.
# This tells uWSGI to shut down idle HTTP workers
# Saves RAM, but startup from idle is a bit more expensive CPU-wise
# We need to run at least 2 uWSGI workers for Takahe
# Each worker will have this many threads
# (even though I'm only giving this 2 cores)
# to match the original gunicorn config.
…and only very minor changes to the
worker: python manage.py runstator
release: python manage.py collectstatic --noinput; python manage.py migrate
In essence, I removed
gunicorn (which I could use anyway) to let
uWSGI handle HTTP requests and scale down to zero (saving RAM). And yes, Piku also supports
release activities, thanks to Chris McCormick.
And that was it. Zero code changes. None. Nada. And I can use exactly the same setup on any VPS on the planet, thanks to Piku.
After a little faffing about with the media storage settings (which I got wrong the first time around, since Takahē also uses
/static for its own assets), I had a fully working ActivityPub instance, and, well… John Mastodon just happened to sign up:
The upshot of this was that the stator
worker was very sad and bombed out when trying to handle hashtags–but all critical stuff worked, so there might well be a workaroud.
But I took some time after breakfast to migrate the database, and since my Django skills are rusty, here are my notes:
# Open a shell to Piku
ssh -t [email protected] run takahe bash
sudo apt install postgresql
python manage.py dumpdata > /tmp/dump.json
sudo su - postgres
-- Set up the database
create user piku;
create database takahe;
alter role piku with password '<mysecret>';
grant all privileges on database takahe to piku;
alter database takahe owner to piku;
# Reset all the migrations, just in case
find . -path “*/migrations/*.py” -not -name “__init__.py” -delete
find . -path “*/migrations/*.pyc” -delete
# Reapply them
python manage.py makemigrations
python manage.py migrate
# Wipe all default entities
python manage.py shell
from django.contrib.contenttypes.models import ContentType
# Load everything back
python manage.py loaddata /tmp/dump.json
Overall, I’m quite impressed with the whole thing. Even with such measly resources and Linux’s tendency to take up RAM with buffers, Takahē under Piku is taking up around 100MB per active worker (2 web handlers, plus the stator worker), plus less than 50MB for PostgreSQL and
So I’m seeing less than 512MB of RAM in actual use, and a steady <10% CPU load inside the container as the stator keeps picking up inbound updates, handling them (including any outbound requests) and doing all the messy housekeeping associated with ActivityPub:
But here’s the kicker: Since this is being capped inside LXC, that is actually around 5% overall CPU load on the hardware–which should translate to something like 2% of CPU usage on any kind of “real” hardware.
With only one active user for now (but following a few accounts already), this is very, very promising.
I have no real plans to leave
mastodon.social for my own domain, but using Takahē to host a small group of people (or a company) with nothing more than a tiny VPS seems entirely feasible, and is certainly in my future.
Right now, I’m going to try to contribute by testing various iOS clients (I will be using the Takahē public test instance as well) and do some minor tweaks to my install, namely:
- Setting up
nginxcaching. Cloudflare is already caching one third of the data, but I want to bulk up this setup so that I can eventually move it to Azure, and I’ve been meaning to add that to Piku anyway.
- Fine-tuning the stator to see how it scales up or down (I might want to try to scale it down further).
gunicornto see if it makes any difference in overall RAM and CPU.
- Seeing if I can get it to work on Azure Functions (that is sure to be fun, although the current SDK failed to install on my M1 and I haven’t tried since).
- Look at how media assets are handled and see if I can add a patch to support Azure Storage via my own
- Deploy on my k3s cluster, to get a feel for how much it would cost to run on spot instances.
There goes my holiday break, I guess…
Update: A Few Days Later
I’ve since sorted out
nginx caching in Piku (and will soon be merging it to
main), which makes things significantly snappier. I’ve also filed #287 to improve caching via Cloudflare and #288 to have
nginx immediately cache assets (which works for me, at least).
Before that, I had some fun tuning stator pauses and filed #232, which resulted in a tweak that lowered idle CPU consumption to a pretty amazing 3% in my test instance.
With the caching tweaks,
gunicorn doesn’t have any real advantage against
uWSGI workers, although I suspect that may be different in higher-load instances.
I’ve also tossed the source tree into an Azure Function and got it to “work”, but not fully. Right now I’m not sure that is worth pursuing given I still need an external database, but I’m really curious to try again in a few months’ time.