Inconsequentials

The past few weeks have gone by in a blur of insomnia, stress and a bunch of random entropy that has triggered my penchant for radical simplification, so I turned off nearly all notifications on work apps, set some recurring reminders to go offline earlier in the day, and decided to spend some time tying up loose ends (as well as building Docker containers to run R, but that’s another story).

In summary, I’m exhausted, and the amount of typos in the first version of this post is ample evidence of that. So I’m cutting back and doing satisfying little hacks to relax.

Even though this site is still managed via Dropbox (instant publishing and trivial updates are addictive), I’m planning to change that someday.

The trigger, ironically, was their move to restrict filesystem support on Linux. There’s an ingenious fix that injects a different filesystem check into the client, but since their CLI daemon is a nightmare to keep running (it insists on taking up 100% CPU without apparent reason now and then) and I’ve had an Office 365 family plan since well before joining Microsoft, the extra expense of paying for Dropbox and not being able to use it properly on two of the machines where it actually made a difference (my VM and my Linux laptop) doesn’t really pay off for me.

So I’m preparing to cancel it next Summer, and trying out abraunegg/onedrive, which has the great side benefit of being easy to run inside Docker and get running on ARM–although, to be fair, it being written in D, of all things, is a major turn-off.

Someone really ought to rewrite the entire thing in Go because it’s single-threaded and slow, but it seems promising for my use cases, and this post was actually drafted on my Linux laptop while it synced my Arduino development tree across.

Another thing that has been bugging me is the amount of CPU omsagent uses in my Azure VMs, so I decided to deal with that too by using raw cgroups to limit its CPU usage to roughly 10%.

Limiting CPU usage for a single user is relatively trivial (but hard to piece together for Ubuntu 18.04), so after installing cgroups-bin I created a few files that create a limitcpu group that limits usage to around 10% of CPU shares, and that apply that policy to anything running under the omsagent user:

$ cat /etc/cgconfig.conf
group limitcpu {
  cpu {
    cpu.shares = 100;
  }
}

$ cat /etc/cgrules.conf
omsagent cpu limitcpu/
*:dropbox cpu limitcpu/

$ cat /etc/rc.local
#!/bin/bash
echo 1 > /proc/sys/vm/oom_dump_tasks
cgconfigparser -l /etc/cgconfig.conf
cgrulesengd
exit 0

You can check if it’s working by poking around inside /sys/fs/cgroup and checking what PIDs are included inside the limitcpu group. Limiting RAM may require a bit more tweaking, but I did’t see a need for it just yet.

This was a bit of a surprise because I had no idea rc.local was still actually usable under systemd (it’s run at startup if it exists and marked as executable), but there you go. There are still bits of the Azure agents running as root, but at least now I don’t have the bloated Ruby components taking up resources.

A few months down the road, I realized that the Ruby agent was burning up to 72% of a single core by running the OMS change tracking script (which is extremely I/O intensive, and also vastly inneficient to boot), so I changed /etc/cgconfig.conf and the like to read:

$ cat /etc/cgconfig.conf
group halfcpu {
  cpu {
    cpu.shares = 512;
    cpu.cfs_period_us = 1000000;
    cpu.cfs_quota_us = 500000;
  }
  cpuacct {
    cpuacct.usage="0";
  }
}
group halfram {
  memory {
     memory.limit_in_bytes="512M";
     #memory.memsw.limit_in_bytes="1024M"; # not available in my kernel defaults and I don't want to set swapaccount=1
  }
}
group limitcpu {
  cpu {
    cpu.shares = 100;
    cpu.cfs_period_us = 1000000;
    cpu.cfs_quota_us = 100000;
  }
  cpuacct {
    cpuacct.usage="0";
  }
}
group limitio {
  blkio {
    blkio.throttle.read_bps_device="8:0 1048576";
    blkio.throttle.write_bps_device="8:0 1048576";
  }
}


$ cat /etc/systemd/system/cgrulesgend.service
[Unit]
Description=cgroup rules generator
After=network.target cgconfigparser.service uwsgi.service

[Service]
User=root
Group=root
Type=forking
EnvironmentFile=-/etc/cgred.conf
ExecStart=/usr/sbin/cgrulesengd
Restart=on-failure

[Install]
WantedBy=multi-user.target


$ cat /etc/systemd/system/cgconfigparser.service
[Unit]
Description=cgroup config parser
After=network.target

[Service]
User=root
Group=root
ExecStart=/usr/sbin/cgconfigparser -l /etc/cgconfig.conf
Type=oneshot

[Install]
WantedBy=multi-user.target

$ cat /etc/cgrules.conf 
omsagent cpu,cpuacct limitcpu/
% blkio limitio/
% memory halfram/
nxautomation cpu limitcpu/
% blkio limitio/
% memory halfram/
piku cpu,cpuacct halfcpu/
% memory halfram/
*:/opt/microsoft/omsagent/ruby/bin/ruby cpu,cpuacct limitcpu/
*:/opt/microsoft/omsagent/ruby/bin/ruby blkio limitio/
piku:/home/piku/.dropbox-dist/dropbox-lnx.x86_64-72.3.127/dropbox cpu,cpuacct limitcpu/
% blkio limitio/

blkio needs device numbers to do its thing (which you get from cat /proc/partitions), so this limits access to /dev/sda to 1MBps, and I added Dropbox to it as well–mind you, Dropbox’s autoupdater will routinely change the version number and binary paths, so if you do this, keep that in mind.

I’m still having some trouble with this, though, largely because both the omsagent and nxautomation users spawn multiple processes and I’ve already caught a few with the “wrong” blkio cgroup. And systemd appears to override some of the cgroup allocations…

So I went with a humungous hack of restarting cgrulesgend inside the root crontab. Ah well.

I’ve got a weirdly-shaped flat with reinforced concrete walls, which besides being a bane for wireless coverage is also a problem for package deliveries, since it’s hard to hear the doorbell when you’re vacuuming or cleaning around the house. Over the past few weeks I came to appreciate that first hand as I worked from home for a while, and decided to do something about it.

My doorbell is a classic Friedland Type 4, which means it’s about the most analog thing in the house except for a couple of clocks, and getting a modern Honeywell doorbell “converter” kit is unaccountably hard (it’s been in and out of stock).

Plus I have a feeling a single extra doorbell isn’t likely to be enough, and it’s more fun to try to “fix” things yourself.

I started by looking at building a powered ESP8266 sensor that could work off the 8-12VAC it’s powered with and was putting together a rectifier bridge when it hit me that the doorbell has a coil and that I could do something that wasn’t going to have to reboot at every bell push.

At first I thought I didn’t have any magnetic reed switches handy, but then the second round of that particular epiphany hit me–I grabbed one of my Xiaomi Aqara contact sensors, removed it from its casing, and taped it directly to the coil.

Sure enough, I now had a Zigbee-enabled doorbell. Which, by the magic of MQTT, I soon tied to HomeKit, after I fixed the irritating quirk of it sending out multiple messages a second.

Why? Because doorbells run on AC, and thus the reed switch actually vibrates every time the doorbell rings¹. So I had to ignore new events after the first within a given time window (easily done with a trigger node in Node-RED acting as a software debouncer), and then it was just a matter of turning on notifications on the Home app (which are instant on my Mac).

That having been dealt with, I started working on the hardware to cater to resident humans–a set of small, noisy devices to scatter about the house. Since I have a bunch of ESP-01S devices² left over from my initial shenanigans with Azure IoT and a handful of passive buzzers, those were the obvious choice, although it doesn’t really feel like a finished solution and I’d have preferred something able to carry a tune:

The buzzer is driven by GPIO2 going low, and the 100Ω resistor limits the current so as to not fry the ESP-01S. I should have made it something like 200Ω to keep the sink current under 12mA³, but the buzzer isn’t visibly rated and my multimeter is MIA, so I chanced it and it has been working fine.

The ESP-01S is going to be mounted on a DIP connector on the final board in order to make it easily removable and re-flashable, and that will probably help keep the layout compact by moving the buzzer fully under it.

I initially meant to write everything from scratch, but as I started putting together the Wi-Fi Manager library, an MQTT client and all the rest, I decided I wasn’t going to spend an afternoon in C++ land and took another look at Sonoff-Tasmota, which is what my smart sockets use anyway, and which has support for extra sensors and GPIO control over MQTT.

As it turns out, I was able to flash the ESP-01S with it, set it up as a Generic device and have it control the buzzer as an “inverted” relay, which was just great and saved me a lot of time, because:

• It has the same MQTT topic structure as my smart sockets
• I had to do pretty much zero extra work to get it working under Node-RED

Or, that is, it would have saved me a lot of time if both the Arduino libraries for the ESP8266 weren’t riddled with Wi-Fi bugs, which meant I spent around six hours flashing and re-flashing a couple of ESP-01S and a WeMos D1 with various firmware variants built atop three versions of the ESP8266/Arduino core–2.3.0, 2.4.2, and 2.5.0b3, of which only the latter actually worked for me (and isn’t easy to find, since it’s not directly linked from the project page).

I had previously reflected upon this firmware being a work in progress, and in retrospect I’m very happy I didn’t try to upgrade my smart sockets in the meantime. But I might end up doing just that, since it’s perfectly possible to place one of these passive buzzers inside a Sonoff S20 socket and have a dual-purpose smart socket and doorbell extender…

I might even go all the way and try out the baked-in MP3 player.

But first, I want to make sure we don’t miss any deliveries.