A few months after writing up the Cudy AX3000 units and moving the house over to OpenWRT, I ended up revisiting the one bit I had deliberately waved away as “good enough”: roaming.

A real house, with a mix of phones, tablets, laptops and a few stubborn IoT things that insist on staying in 2016, has… issues. But they’re not always obvious, and given we’d both upgraded the 5GHz band and changed the locations of the access points, it took a while to figure out where the new rough spots were.

If you’re just tuning in, I have a hard split between a legacy 2.4GHz network and the modern 5GHz one. I already had client-managed roaming and basic handoff guidance, but now I added usteer, 802.11k neighbour reports (because hostapd was not cooperating), and things are now pretty much perfect.

The long version is below, with anonymised data and enough detail for future me to remember why I did this.

The obvious advice for roaming is “use one SSID everywhere”, and that is often correct if you’re running Wi-Fi in an office, a public venue, or generally somewhere where you don’t have (or care about) legacy devices. It is also not what I did, because the 2.4GHz side needs to remain friendly to older and slightly terrible IoT devices, which means WPA2 compatibility and a conservative setup.

The 5GHz side is where the more modern clients live, and despite losing 5GHz access for a couple of things, I was happy to move it to WPA3. So this is what things look like from a high level:

• 2.4GHz: legacy-compatible WPA2-ish network for IoT and old clients.
• 5GHz: modern client network with WPA3/SAE
• 2.5GbE backhaul across four “dumb” APs
• Zero cloud management or vendor-specific software. Nada. Zilch. Non-negotiable.

However, I got a few complaints that when moving about the house, iPhones, iPads and MacBooks would not switch to another AP. Since our flat is wrapped around a couple of elevator shafts and there are a few spots (like the kitchen) where tiling, pipes and tiny RF nuisances like fridges were prevalent, that sort of tended to happen a lot–and Apple devices are notorious for being opinionated about that base station they want to stick to.

The baseline seemed fine. All four APs had 802.11r/k/v-related options enabled. Fast Transition was also demonstrably happening–the AP logs had auth_alg=ft entries that showed fast transition was happening, I had installed wpad-mbedtls for “mesh” support, but roaming clearly needed to be improved.

And my setup meant it had to be improved within each band/SSID, not across bands. Cross-band roaming is the client’s job, and many clients are not especially good at it.

But two things stood out:

• There was no steering daemon installed. Clients were making all roaming decisions on their own, which usually means they hang on to a far-away AP until their signal is frankly embarrassing.
• rrm_nr_list was empty on every radio. In other words, even though 802.11k was enabled, hostapd was not exposing neighbour reports to clients, so… no real way to steer anything.

So I installed usteer and its LuCI companion package on all four APs, enabled it, and left the initial configuration at defaults:

opkg update
opkg install usteer luci-app-usteer
/etc/init.d/usteer enable
/etc/init.d/usteer restart

The default configuration is minimal: LAN gossip, syslog enabled, IPv6 disabled for the daemon (because, for reasons, I don’t trust our current ISP router to do anything reliably except act as an ONT), and a moderate debug level. That was enough for all APs to see one another and exchange client data, which is exactly what I wanted.

However, the 802.11k neighbour list wasn’t being populated. After poking through the OpenWRT forums, I realized the missing piece was static-neighbor-reports, which is one of those tiny OpenWRT packages that does exactly what it says and nothing more.

Each AP can generate its own 802.11k neighbour report element via:

ubus call hostapd.<iface> rrm_nr_get_own

But clients only get useful neighbour lists if each AP is told about the other APs. So I generated per-band lists and installed them per AP:

opkg install static-neighbor-reports
/etc/init.d/static-neighbor-reports enable
/etc/init.d/static-neighbor-reports restart

The important detail is that the reports are band-specific: 2.4GHz radios only advertise 2.4GHz peers, and 5GHz radios only advertise 5GHz peers. No cross-band mixing, because the two networks intentionally have different SSIDs and security settings.

After that, every AP had three neighbours per radio, usteer had AP/client state, and hostapd has explicit 802.11k neighbour data to hand to clients that ask for it.

The first comparison is a little boring, but useful. Here is the 2.4GHz SNR before and after the change (this, like the other charts here, was generated from Graphite data):

There is no miracle here. 2.4GHz remains 2.4GHz–crowded, noisy, full of junk devices and crowded by all my neighbors. Two of the APs improved or stayed roughly level, two got worse in the sampling window, and I have zero expectations about ever clearing this kind of congestion without moving to the countryside.

The 5GHz side is more encouraging, even if you do need to know when we were near which AP at what time when you look at active bitrates:

The interesting part, though, is that at least between two APs, there was a noticeable shift in usage–which seems to reflect where clients should be registered in practice:

But the best sanity check is the sticky-client view, because that is what started this in the first place:

The number of merely weak clients did not disappear–one extra client fell below -75dBm in the later sample–but the very weak clients went away. That is the bit I care about: the previous -90dBm-ish sticky associations were gone in the later check, which seems to indicate clients are not getting hung up on their previous AP and are indeed roaming.

A single sample is not science, and Wi-Fi is a swamp of client decisions, radio noise and domestic entropy. I also saw one new Fast Transition log entry after the rollout:

FT: Missing required pairwise in pull response from a peer AP

That happened once in the latest check. It is not enough to call the setup broken, but it is worth watching–especially because SAE and FT have enough moving parts that I would rather trust logs than assumptions.

I will be keeping an eye on this over the next few weeks… somehow. I got an LLM to do the Graphite queries and chart scripting for me, and ain’t nobody got time to build dashboards only I would look at, but the metrics aren’t going to go away and the stable config lives in my local Gitea instance now, so there’s really no excuse not to do a spot check in a few months.

But I really like my Cudy APs. No cloud controller, no meshing, no mobile app and no secret sauce. Just OpenWRT, collectd/Graphite, and the odd ssh session to check configs.

That is still the main thing I like about this setup: when it gets weird, it gets weird in ways I can inspect.

I know this blog has strayed a fair distance from its Mac-centric origins, but I’ve been keeping a mental list of all the things that are broken, missing or inexplicably neglected in Apple’s software, and it’s gotten long enough that writing it down feels like a public service¹.

This isn’t about Liquid Glass or grand design failures–those are well documented elsewhere. This is about the small stuff. The papercuts that, individually, you learn to live with, and collectively make you wonder whether anyone at Apple actually uses their software.

Despite the somewhat surprising length of this post after stitching together all the notes, I’m actually focusing on the things I hit every week (not trying to put together an exhaustive catalogue), and others will have their own lists–and that’s part of the problem.

Apple Mail is the first app open every day and the one I find hardest to defend, and I’ve been defending it for twenty years (longer if you remember the original NeXT mail client).

The broader story is one of abandonment. Mail.app used to be extensible–there was a plugin API that third parties used to build genuinely useful tools (GPGMail, SpamSieve, Act-On, all manner of filing and productivity helpers), and I used it to, among other things, have HJKL keybindings.

Apple deprecated that API, replaced it with a (much more restrictive) MailKit surface in 2021, and proceeded to lock MailKit down so hard that barely anyone shipped an extension.

And then they quietly stopped mentioning it. The result is that Mail is now less extensible than it was in 2010.

In particular, in this age of desktop AI agents, I come time and again across the fact that AppleScript support in Mail has been left to rot. I wrote about getting raw message source via AppleScript years ago, and even then it was a workaround for missing functionality.

Today the dictionary is unchanged, the bugs are unchanged, and the “Apply Rules” menu option–which used to let you re-run rules on selected messages–no longer works consistently on multiple selections, if it works at all.

And searching for messages is such a mockery of a user experience that I’m not even sure how to describe it–suffice it to say that it never searches solely inside the folder I’m in and that it often fails to find messages that I know are there, even with the most basic criteria.

And then there are the basics that have simply never arrived on iOS:

• There is no way to filter messages on an iPad. Not “limited filtering”–none. You cannot create a rule, you cannot sort by sender, you cannot batch-select by criteria.
• Smart folders don’t exist on any iOS version (no, the stupid Categories thing doesn’t count). They’ve been on the Mac since… 2004?
• And, of course, there is no way to have Mail rules sync from the Mac to iOS. For a company that talks endlessly about ecosystem coherence, this is bizarre.
• Download progress is opaque. When Mail is pulling thousands of messages from an IMAP server, the feedback is either nothing or a tiny spinner.
• Apple Intelligence in Mail amounts to a summary button that occasionally produces useful one-liners.

There’s no smart filing, no suggested rules, no priority inbox–nothing that would actually reduce the cognitive load of managing email. Gmail had most of this a decade ago.

I wrote an entire blogpost about how bad it is these days, and if I had the patience, I could probably write twice as much.

But I’ll just add that the performance is abysmal if you have thousands (or millions) of small files, and that things like asimov (or manually setting the right extended attributes manually for excluding development folders, something I routinely forget to do) shouldn’t exist, because it should work properly in the first place:

• It should have much more transparent progress indications
• It should never fail silently
• It should recover gracefully from failures
• It really should suggest automatic exclusions and have a proper UI that is not “Add this huge top-level folder” for exclusions

Again, this isn’t rocket science. I installed Borg Backup the other day on some of my Linux VMs, and it is so good that it defies explanation how Apple still hasn’t gotten this right.

Craig Hockenberry recently wrote up an experience that captures the problem perfectly: his iPhone’s Spotlight index corrupted, search stopped working across App Library, Mail, Notes, Messages and Settings, and after trying every remedy he could find online–forced restarts, language changes, toggling Siri, developer mode reindexing–the only “fix” was a full device backup and restore.

Which took hours, broke Apple Pay, reset FaceID for two dozen apps, wiped TestFlight builds, and generally made his life miserable for days.

On the Mac, rebuilding the Spotlight index is a one-line terminal command that somehow I keep not memorizing despite needing it once a month. On iOS, that affordance doesn’t exist.

“It just works, my ass” was Craig’s summary, and it’s hard to improve on it.

Search on iOS is slow, inconsistent, and returns incomplete results across every app that relies on it. On macOS it’s marginally better but still loses to most third-party tools, solely because Spotlight completely made a mess of the user experience and Finder, well, can’t even find itself sometimes.

This, again, is something that I come across every single time I need to manage personal time, and that is essential if we want any form of serious AI assistants to work (or integrate with Apple stuff).

But I’ll cut right to the point: the Calendar app has barely changed since iOS 7, and the parts that have changed are worse.

• Event metadata parsing is broken. If someone sends you a calendar invite with a video call link, Calendar will sometimes pick it up, sometimes not, and sometimes create a phantom “location” that’s actually a URL fragment.
• There’s no way to see a compact list of upcoming events without also seeing the full calendar grid.
• Calendar sharing within a family is functional but graceless.
• CalDAV support is just… not there. It sort of works, but ever since Apple decided to move both Calendar and Reminders to CloudKit (or whatever), all you will get (for Reminders, at least) are the leftover entries that they left in the store before the migration.

Oh, and need I mention that Siri is terrible at calendar operations, including the extremely basic “at what time did my wife book dinner”?

I know. Most of the parts about some apps above are also about automation, and I did post about this in my Siri for Families piece, but it deserves a dedicated entry because in this age of Codex and Claude being able to control your desktop, it rankles.

• Shortcuts actions break between OS versions. Not occasionally–routinely.
• AppleScript is unmaintained, and despite what I wrote earlier, is now presumed dead.
• There is no cross-platform automation story whatsoever. No, Shortcuts is not useful there, save for the laudable exception of being able to use my iPhone to automate switching watchfaces (which is something very few people are likely to use).
• Accessibility sort of works, but it is so clunky in practice that some of the workarounds I’ve seen implemented in Claude and Codex border on the hilarious.

The bottom line, for me, is that Siri Shortcuts integration is shallow compared to what Android offers through intents, or what Windows offers through COM automation (or even Win32, which surprisingly still works so well that it took me 15 minutes to do an agent tool).

In keeping with Apple’s inability to make the iPad truly useful, iOS has no hypervisor support today–it was removed in iOS 16.4, and nothing has effectively replaced it since. The result is that you can’t run a Linux VM on an iPad, and you can’t run Docker containers on it either, which means that the entire ecosystem of local LLMs, coding agents, development environments and monitoring tools that I rely on for work and play is completely inaccessible on the iPad.

macOS has had Hypervisor.framework since… Yosemite, and Apple Silicon Macs run Linux VMs beautifully–but on iPad and iPhone, the entire concept doesn’t exist, and we are forced to run half-assed emulators like ios-linuxkit (which I’ve been banging on for a month as a way to prove my point).

This matters to me because a huge amount of the software I use daily–local LLMs, coding agents, development environments, monitoring tools–runs in containers or lightweight VMs. I can do all of this on an EUR 50 ARM board running Linux. I cannot do any of it on an EUR 1,400 iPad Pro with an M4 chip, without jumping through hoops to get AltStore to run on it so that UTM can pretend it has proper virtualization.

I know that Apple doesn’t care about this now that they feel buoyed by the MacBook Neo’s runaway success, but I am actually looking forward to trying out a GoogleBook solely because Google has reasonably decent support for running Linux userlands on ChromeOS and Android, and I want to see how that compares to the iPad’s non-existent support.

I could possibly write a book about this by now, considering that I’ve been at this for many, many years. HomeKit could be so much better, but it is also a part of the Apple experience where the gap between promise and reality is most painful.

Yes, Matter is coming, etc., etc., but a new protocol will never solve any of the shortcomings of the Home app:

• Scene chaining doesn’t exist.
• If-this-then-that logic is barely functional.
• Presence detection is flaky and not granular enough for room-level logic.
• There is no scripting layer. Shortcuts can trigger HomeKit actions, but HomeKit automations can’t call Shortcuts.
• Adaptive lighting is half-baked.
• Multi-home support is a mess.

I’ve papered over most of that with Node-RED and Homebridge, and of course Home Assistant can do all of the above, but, again, my main point is that it shouldn’t need to exist for people who’ve bought into the Apple ecosystem.

At this point, Apple should just buy Homey and can their entire HomeKit stack.

The Watch deserves its own entry because it’s the device where Apple’s failure to prioritise timekeeping is most absurd, and with the rebirth of Pebble, I was reminded of how awesome smartwatch UX can be and how Apple never even got close.

In particular, the “Smart” Stack (the thing you get when you swipe up from the bottom) is never aligned with what I actually want to see, or what is up on my calendar.

The Pebble’s timeline view remains the high-water mark for watch UX–one button tap, chronological day view, no widget carousel.

Apple’s Calendar app on the Watch tries to replicate the iPhone calendar grid on a 45mm screen, which is about as useful as reading a newspaper through a keyhole.

A watch should be the single best device for time-aware context. Instead of building a timeline, Apple built a widget carousel.

I once spent a week building a CalDAV client to talk to iCloud Reminders and Calendar, and the experience was a masterclass in Apple’s backwards-compatibility approach: it works, except when it doesn’t.

• Newer Reminders lists silently migrate to CloudKit and disappear from CalDAV entirely.
• Apple Notes is completely gone from IMAP–all content is now behind CloudKit’s protobuf CRDT format, which Eric Migicovsky recently reverse engineered
• Calendar event recurrence expansion doesn’t work properly through CalDAV.
• App-specific passwords are required if you want to have third party clients sort of work, but limitations are documented nowhere.

The pattern mirrors a lot of my gripes about the original iCloud services: Apple builds new infrastructure, migrates data silently, leaves old APIs running but progressively useless, and provides no supported path for third-party access.

Yes, it got updated recently. No, it is neither good nor fast nor consistent when you use vim daily, and that is why I use Ghostty. Like a lot of other core Mac tools, I have feelings about it, some of which I cannot express politely.

I write Swift because I have to, not because Apple makes it easy.

The language itself has been through enough breaking revisions that code from three years ago often won’t compile without changes. SwiftUI is worse–views that worked on iOS 17 already behaved differently on 18 and now seem broken in 26, and the abstraction leaks the moment you need anything beyond a list and a navigation stack.

The result is a UI framework that feels modern in tutorials and feels like debugging a black box in production. I’ve lost count of how many times I’ve had to drop to UIKit to work around a SwiftUI layout bug that, once I start searching for it, I realize has been reported for years and yet nobody at Apple acknowledges.

And then there’s the $99/year developer fee, which Apple charges you for the privilege of running your own code on your own hardware. Not to publish on the App Store–just to run an app on a device you already paid for. The certificate expires annually, and if you don’t renew, your sideloaded apps stop launching. In 2026, on hardware I own, I need a subscription to run my own software.

The App Store itself is a whole separate set of papercuts–review delays, opaque rejections, the 30% cut, the inability to distribute updates outside the store–but those are well-documented grievances.

The one that gets me is simpler: the entire developer toolchain assumes you are building a product for sale, not a tool for yourself. Xcode doesn’t have a “just let me run this on my phone” mode that doesn’t involve provisioning profiles, entitlements, and a certificate chain.

Until I started using Codex, every personal project started with ten minutes of ceremony. Now I never even open Xcode.

I still have an iPhone 6 in a drawer, and every time I pick it up I’m reminded of what a phone that fits in your hand actually feels like. It’s delightful to hold–thin, light, one-handable without gymnastics, and the screen is perfectly usable for everything I actually do on a phone.

Every iPhone since has been bigger, heavier, and harder to use one-handed, and the Max/Plus variants are actively hostile to anyone with normal-sized hands or normal-sized pockets. Apple keeps making the screens taller and the bezels thinner, but the fundamental ergonomic regression–that phones stopped being things you hold comfortably and became things you grip–has never been acknowledged, let alone reversed.

The iPhone SE was the last concession to people who wanted a small phone, and Apple killed it. The Mini lasted two generations before being quietly shelved. The message is clear: you will hold the slab and you will like it.

Every one of these is fixable. Most have been fixable for years. The pattern isn’t technical inability–it’s neglect.

Apple has the engineers, the money, and the platform control. They’ve chosen not to, repeatedly, and I suspect writing about it won’t make any difference, but as someone who has been using Macs since the System 6 days and writing about OSX here since the very beginning, I like to keep a scorecard.

And right now, it’s neither looking good nor reassuringly future-proof, unless, of course, you happen to love Liquid Glass.

I’m done waiting for Apple to fix things. And one of the things I think should exist is a decent way to run Linux binaries on my iPad.

And after almost six months messing about with ARM emulation in various forms, I can finally do something about it.

Put bluntly, the lack of hypervisor support on iOS should be an embarrassment to Apple–an EUR 1400 iPad Pro with an M4 chip can’t run Docker, can’t run a VM, can’t do any of the things I do daily on an EUR 50 ARM board. Apple has the hardware support, the kernel entitlements, and has chosen to keep it locked away.

ios-linuxkit is my answer to that, or at least as much of an answer as you can get without Apple’s cooperation. It’s a Linux runtime for iOS that provides a working AArch64 userland on iPhone and iPad–shells, compilers, package managers, language runtimes, the lot–without JIT, without RWX memory, without MAP_JIT, without any of the things Apple won’t let you have.

The base is the ish-arm64 branch of iSH, which implements a threaded-code interpreter (they call it “Asbestos”) that translates ARM64 Linux instructions through precompiled gadget dispatch. No runtime code generation means no App Store policy violations, which means it can actually ship. The trade-off is performance–you’re not getting native speed, you’re getting “fast enough for a shell and some compilers.”

It’s fast in human terms, although my use of Go and Bun mask a lot of the underlying limitations.

The timing comes down to converging interests: I have been deep in emulation land since the holidays, and even though the ish-arm64’s “gadget” emulator is quite a different beast from the naïve block-level JITs I’ve been bolting onto BasiliskII and SheepShaver, I have been developing all of them on the Orange Pi 6 Plus board I have been testing for a few months, so they share roughly the same approach:

• Bolt on a VNC server (or an emulated console) so I can connect to it from my iPad
• Build out several test harnesses (build, base smoke tests, tracing harnesses and automated application testing)
• Figure out what to do (this is the hard part, and I’ve learned quite a bit across the various emulators)
• Figure out where it breaks (Go, Java, etc.) and why
• Hand out the drudgery (like test runs and automated fixes) to a piclaw instance in clearly defined piecemeal specs so I get nice reports and debugging output I can review in a clean web UI

I wouldn’t have had the time or energy to do this without Codex, but I certainly wouldn’t have been able to do it without the Orange Pi as a test bed. Having an ARM 12-core SBC with 16GB RAM I could devote to this, despite a tad constraining (I would have preferred 32 so I could run more builds and test matrices concurrently) was a major enabler here.

The fork started as a bring-up exercise, but has turned into something more focused: making the runtime stable and tested enough that you can actually develop on it. The current validation gate has 82 core tests passing on Alpine ARM64, with workload coverage across Go, Rust, Bun, Node, Python, Java, Zig, and a few others.

And since I’ve seen quite a few people trying to run AI coding agents on iOS, there’s a separate set of AI CLI harness tests that installs, runs and does cursory tests on most current agent tools (spoiler: Claude Code was a complete and utter pain to get to run. Everyone else’s mostly “just worked” after a few cycles of JS runtime/kernel call cleanup passes, theirs was just broken).

The harness testing is AI-driven–I pointed piclaw at it with a custom gdb skill and let it grind through failures, fix them, and re-run. The strategy is mine (which syscalls to prioritise, what the “gadget” fixes should look like, where to invest in performance), but the mechanical detection/fix loop is the kind of thing that would have taken months by hand.

Because I think it is a thing that should exist, yes, but also because I want to run things like gi (which is still WIP) on my iPad.

Especially Go binaries, which never ran in the original iSH. And despite my love for remote sessions, I don’t want to run all of it on a server, nor via a UI proxied from somewhere else–I want to do some of it locally, in a terminal, with my workspace on the device.

Bun, V8 and Go work. Alpine’s apk means I can easily get pretty much every single CLI tool I need to work too, without the compromises a-Shell (which I still love, by the way) imposed. And since I have been hacking away at my own flavor of Ghostty in rcarmo/ghostty-web, I was able to swap the dated iSH terminal with something that looks right.

It’s not fast (well, it is, much faster than the original, but not native fast). It’s not a replacement for HyperKit on iOS (if we ever get it back). But it’s mine, I can fix it and make it faster to some degree, and works for me.

And since I have zero intention of bringing it to the App Store myself (or even paying Apple $99 for the privilege of running it on my own hardware without plugging my iPad into my laptop weekly, which is something the EU should really ding on Apple for), I am going to maintain it and add more fixes, keeping it open source so that other people can build better, more polished tools.

You’re welcome.

Update, May 19: I’ve since ported Terax, an AI “terminal” to it and have started poking at the Asbestos emulator to see if I can both speed it up and fix a few more corner cases around Go compilation, which is still very slow.

Last weekend my DS1019+ decided, for some unfathomable reason, to stop working after I took it out of the closet, dusted it and put it back, and I have feelings about it.

In fact, I’ve had them throughout the whole week, because it’s taken forever to get most of my home services up again.

Fortunately, my home automation and a few other things are spread among my Proxmox nodes, but I had a bunch of things running on that NAS, and I wanted to document what happened because someone else might have the same issues I did and end up here.

The machine booted up (power LED initially blinking, solid green status LED, disk activity almost immediately), but would not show up on the network.

Both LAN interfaces would be up, but issued zero packets. No DHCP requests, no link-local addressing, not even replies to arping (and yes, I knew the MAC addresses of the machine, because that’s the kind of thing I keep tabs on). I plugged in my MacBook and my MiniBook X on each interface, rebooted, and saw… nothing.

tcpdump saw nothing at all. I thought it might be some sort of OS glitch (which is why I tried both laptops), but no luck.

So I tried to reset it to factory configuration. You have two reset levels, the first of which only resets your admin password and network settings, the second has you reinstall the OS without losing data.

But nothing worked, and Synology’s tooling just couldn’t find the NAS or connect to it.

The first thing I did was set up Virtual DSM on borg to see if I could, in the direst of emergencies, access our off-site Azure backups. That sort of worked, but the experience was so fiddly that I was reminded of all of HyperBackup’s pitfalls in one fell swoop–most notably that I effectively need a Synology to get at that data, which is not something I want to rely on.

Yes, there is a HyperBackup desktop application. No, it did not work for me–it apparently expects you to download backup files from the cloud to your local machine, and I need to be able to directly restore files from Azure, period.

After filing a ticket with Synology about my unresponsive system, they sent me an AI-generated troubleshooting list, in the middle of which was a step I could not find anywhere in their online documentation: booting the machine without any disks.

That apparently also automatically reset settings (which is, in retrospect, weird, because it feels like something should be stored in the chassis for this kind of emergency), and I was finally able to discover it on the network, reset the admin password, reconfigure the network, etc.

So if you have the same symptoms, this might save your day. And, as it turns out, be the prelude to an entire week of pain, because mine spent the past five days or so grinding through data scrubbing. Because that is a thing it felt like doing, and I’ve been coping with the fallout since then–extremely slow access, very slow response times as I tried to double-check services and settings, etc.

First of all, all my Docker containers were gone. Container Manager, for some reason, does not preserve any settings in this scenario, and if I didn’t have Portainer installed and a copy of (most of) my stacks in Gitea, this would have been enough for me to never again run containers on a Synology.

As it was, I was able to point piclaw to the machine and have it reconstruct all critical services in a few hours (it would have been much faster if it wasn’t doing scrubbing). And, as it turns out, there was also enough residual info in the underlying Docker daemon itself to fill in most of the gaps.

But barring that, there were a bunch of things that made recovery a pretty stressful endeavor:

• The mobile apps (DS Finder and the like) were useless in finding or diagnosing the issue at every step.
• The web site did not list disk removal as a troubleshooting step (at least not that I could see, since it went straight into the dual-step reset procedure).
• The timing documented for holding the reset button for reset 1 (4 seconds) was not accurate. It was more like 20, and I feared for a moment I might end up triggering reset 2, which would require reinstalling the OS.
• Synology’s desktop tools are, to be brief, very poorly maintained and look like something out of the 90s, even down to the Windows look on macOS.

So even for an “appliance” NAS, the experience could be much better.

Resetting the configuration had zero impact on my data–at least so far as I can tell. Shares, users, all the regular stuff was preserved, and after a few glitches with cloud backups (because disk scrubbing made them fail overnight twice), everything seems in order.

But since the machine spent so long simultaneously scrubbing and swapping as I tried to restore services, it’s clear that I cannot rely on it for interactive use anymore.

Synology doesn’t really let me upgrade RAM on the thing (you sort of can, but it’s already capped at the maximum RAM the J4125 can officially support), so I’ve started removing stuff from it–most of the Docker services I’ve been running there for years are now moving into Proxmox microVMs or LXCs running elsewhere, and are either going to use the Synology as a “dumb” NAS and mount storage directly, or be backed up to it using Borg Backup Server (which is going to be the only new Docker container running on it).

I’ve already moved Portainer and Gitea off it, and having them run (even with very constrained resources) on separate microVMs in an N150 makes a world of difference–so much so that I have to wonder why I put up with the J4125’s slowness for years.

I set Proxmox to snapshot both VMs daily (and added a temporary direct-to-cloud restic backup), and am now slowly moving the rest. Or, rather piclaw is doing that. I had it draft a plan to group containers and create target VMs/LXCs, and the agent is now merrily rsyncing data and container configs out of the Synology.

After the dust settles, I am going to move all of my backups out of the Synology ecosystem–I currently rely on HyperBackup to back up my data to Azure, but the recovery attempt was so off-putting that I am going to look into using restic directly to Azure.

Backrest looks like a nice way to do that, with the added benefit that restic backups (which I have already been using for years) seem to work better with Azure storage tiering (and thus might even be cheaper in the long run).

The Ternus announcement got me thinking about the one thing I keep wishing Apple would build and almost certainly never will: a family-scoped AI assistant that actually works across all our devices.

Read More...

I’ve been mulling the UX arc I’ve been going through over the past couple of years, and I think it was mostly the same for everybody:

Read More...

Regular readers will know that I’ve spent most of the past two years shoehorning LLMs into single-board computers, partly as a learning exercise and partly because there are lots of local/”edge” applications where semantic reasoning (no matter how limited) and “interpretation” of sensor data are actually useful.

Read More...

I’ve been building MCP servers for a while now–I wrote about the general approach last year, started out by creating umcp, and I’ve recently opened up an Office server that’s been battered by enough models against enough real documents that the patterns have settled.

Read More...

I have been having feelings about Apple lately. This blog may have drifted a fair way from its original focus on macOS, but I am still, first and foremost, an Apple user – just not an exclusively Apple user, and perhaps not even a particularly obedient one anymore, since I use both Windows and Linux every day and have grown used to judging platforms by what they let me get done rather than by whatever story they are trying to tell about themselves.

Read More...