Getting Takahē to run on Piku

Last night after work I decided to see how easy it would be to run a Takahē instance under Piku, my tiny -oriented PaaS.

Self-hosting is all the rage, but having to deal with a full-blown installation of (which is always a pain to install properly, even if you use rbenv), plus the abomination that is Sidekiq and the overall Rube Goldberg-esque architectural approach that is almost mandatory to deal with the complexities of is just something I don’t want to maintain. Ever. Even inside Docker.

Which is why I have been developing my own server using Sanic and a very lightweight asyncio-based approach at handling all the transactional aspects of atop . And let me tell you, I honestly wish the protocol was less about doing what boils down to P2P webhooks with PEM signatures embedded in requests.

Enter Takahē

Blue birds are cool. Well, at least flightless ones.

But Takahē is now aiming to support client apps as of version 0.6, is built on Django (which I have always loved as a framework), and it saves me from the trouble of building everything from scratch, so… I had to try it out.

More to the point, Django is exactly what Piku was originally designed to run.

Besides running as a WSGI app, Takahē uses an async stator to handle all the background tasks (which is also exactly the pattern I aim for and designed Piku to support), so I just had to see how easy it was to get it running under Piku on very low-end hardware.

The Hardware

I have a 4GB 4s set up as an SSD-backed Proxomox server, hosting several different arm64 LXC containers I use for developing stuff. I love it because I can use LXC CPU allocations to throttle things and make sure they run fast enough on very low-end hardware, plus I can just snapshot, mess up and restore entire environments.

So I set up an Ubuntu 22.04 container with 1GB of RAM and access to 2 CPU cores, capped to 50% overall usage–which is roughly the performance of a 2 give or take, albeit with a fully 64-bit CPU.

I deployed Piku, set up a CloudFlare tunnel, and then went to town.

Zero Code Changes Required

In short, what I needed to get Takahē up and running under Piku was to:

  1. Clone the repository.
  2. Create a production remote pointing to Piku.
  3. Edit the supplied ENV and Procfile.
  4. Do a git push production main.

It was that simple.

Here’s the configuration I used, annotated. First the ENV file:

# Yes, I went and got it to use SQLite, and it nearly worked 100%
# This is what I eventually migrated to (more below)
# TAKAHE_DATABASE_SERVER=postgres://piku:<password>@localhost/takahe
# I actually love Django debugging, and with it on I can see the inner workings
# You know who uses this password, don't you? 
# No, it's not the one I'm actually using.
# Anyway, this next one breaks a little on Piku, so I need to revise parsing for this case.
# I also added a Redis server later, without any relevant impact in RAM usage

# This is all Piku config from here on down
# I need IPv6 off for sanity inside Proxmox
# This ensures nginx only accepts requests from CloudFlare, plus a few extra tweaks

# These are caching settings for my dev branch of Piku
# This has nginx cache these prefixes

# This maps static user media directly to an nginx route
# You want to set these, trust me. I should make them defaults in Piku.
# This tells uWSGI to shut down idle HTTP workers
# Saves RAM, but startup from idle is a bit more expensive CPU-wise
# We need to run at least 2 uWSGI workers for Takahe
# Each worker will have this many threads 
# (even though I'm only giving this 2 cores)
# to match the original gunicorn config.

…and only very minor changes to the Procfile:

wsgi: takahe.wsgi:application
worker: python runstator
release: python collectstatic --noinput; python migrate

In essence, I removed gunicorn (which I could use anyway) to let uWSGI handle HTTP requests and scale down to zero (saving RAM). And yes, Piku also supports release activities, thanks to Chris McCormick.

And that was it. Zero code changes. None. Nada. And I can use exactly the same setup on any VPS on the planet, thanks to Piku.

After a little faffing about with the media storage settings (which I got wrong the first time around, since Takahē also uses /static for its own assets), I had a fully working instance, and, well… John Mastodon just happened to sign up:

Every ActivityPub server ought to make this their demo account.

Teething Issues

Takahē nearly works with , but sadly it relies on JSON_CONTAINS, which is an unsupported feature in (but one which excels at).

The upshot of this was that the stator worker was very sad and bombed out when trying to handle hashtags–but all critical stuff worked, so there might well be a workaroud.

But I took some time after breakfast to migrate the database, and since my Django skills are rusty, here are my notes:

# Open a shell to Piku
ssh -t [email protected] run takahe bash 
sudo apt install postgresql
python dumpdata > /tmp/dump.json
sudo su - postgres
-- Set up the database
create user piku;
create database takahe;
alter role piku with password '<mysecret>';
grant all privileges on database takahe to piku;
alter database takahe owner to piku;
# Reset all the migrations, just in case
find . -path “*/migrations/*.py” -not -name “” -delete
find . -path “*/migrations/*.pyc” -delete
# Reapply them
python makemigrations
python migrate
# Wipe all default entities
python shell
from django.contrib.contenttypes.models import ContentType
# Load everything back
python loaddata /tmp/dump.json


Overall, I’m quite impressed with the whole thing. Even with such measly resources and Linux’s tendency to take up RAM with buffers, Takahē under Piku is taking up around 100MB per active worker (2 web handlers, plus the stator worker), plus less than 50MB for and nginx together.

So I’m seeing less than 512MB of RAM in actual use, and a steady <10% CPU load inside the container as the stator keeps picking up inbound updates, handling them (including any outbound requests) and doing all the messy housekeeping associated with :

These are the stats a few hours later in the day, after publishing this post.

But here’s the kicker: Since this is being capped inside LXC, that is actually around 5% overall CPU load on the hardware–which should translate to something like 2% of CPU usage on any kind of “real” hardware.

With only one active user for now (but following a few accounts already), this is very, very promising.

I have no real plans to leave for my own domain, but using Takahē to host a small group of people (or a company) with nothing more than a tiny VPS seems entirely feasible, and is certainly in my future.

Next Steps

Right now, I’m going to try to contribute by testing various iOS clients (I will be using the Takahē public test instance as well) and do some minor tweaks to my install, namely:

  • Setting up nginx caching. Cloudflare is already caching one third of the data, but I want to bulk up this setup so that I can eventually move it to Azure, and I’ve been meaning to add that to Piku anyway.
  • Fine-tuning the stator to see how it scales up or down (I might want to try to scale it down further).
  • Trying gunicorn to see if it makes any difference in overall RAM and CPU.
  • Seeing if I can get it to work on Azure Functions (that is sure to be fun, although the current SDK failed to install on my M1 and I haven’t tried since).
  • Look at how media assets are handled and see if I can add a patch to support Azure Storage via my own aioazstorage library.
  • Deploy on my k3s cluster, to get a feel for how much it would cost to run on spot instances.

There goes my holiday break, I guess…

Update: A Few Days Later

I’ve since sorted out nginx caching in Piku (and will soon be merging it to main), which makes things significantly snappier. I’ve also filed #287 to improve caching via Cloudflare and #288 to have nginx immediately cache assets (which works for me, at least).

Before that, I had some fun tuning stator pauses and filed #232, which resulted in a tweak that lowered idle CPU consumption to a pretty amazing 3% in my test instance.

With the caching tweaks, gunicorn doesn’t have any real advantage against uWSGI workers, although I suspect that may be different in higher-load instances.

I’ve also tossed the source tree into an Azure Function and got it to “work”, but not fully. Right now I’m not sure that is worth pursuing given I still need an external database, but I’m really curious to try again in a few months’ time.

This page is referenced in: