When OpenCode decides to use a Chinese proxy

So here’s my cautionary tale for 2026: I’ve been testing toadbox, my very simple, quite basic coding agent sandbox, with various .

I’m running the agent containers inside , and I decided to tweak the VLAN configuration I was using for the VM running the containers, with the result that it temporarily lost DNS access (so, effectively, “internet access”, except for connections that were already established).

When I connected back to one of the containers, I noticed that OpenCode (which I’m running inside toad, since I very much prefer its text UI) had decided to route the package installations through a Chinese proxy server:

Now that was a surprise...
Now that was a surprise...

I terminated the container immediately, and checked the workspace files (nothing untoward there that I could see, except for a couple more test files in the target project), but it was a stark reminder that LLMs are not to be trusted blindly.

Just for clarification, I was using OpenCode’s out-of-the-box default settings, with no custom configuration. toadbox just installs toad, which then installs OpenCode off its startup menu, and I did no setup whatsoever. So I was using one of their default free models.

Digging around in a fresh container the default model I’m getting is big-pickle, and I have to assume that this is the one that decided to use that proxy–and going forward, now that I know where OpenCode is saving its log files, I’ll be adding a docker volume to toadbox to capture those logs outside the container for future reference.

Like I posted on Hacker News, this is interesting for several reasons–the first is that even with tool whitelisting this kind of thing might happen, and the second is the why. I happen to know about how popular Go is in China and that the proxy approach is… well… kinda official, but it’s also a great demonstration of how leverage “knowledge” in a completely non-linear way.

Considering that one of Simon Willison’s predictions for 2026 was that we are finally going to solve sandboxing for LLMs this year, I guess this is a timely reminder that we still have a long way to go.

But his other prediction about there being a “Challenger disaster” for coding agent security feels a lot closer to reality now…

Update: I did some spelunking since the container was actually still around (I just killed it, didn’t remove it) and confirmed that this was indeed big-pickle. Here’s the relevant excerpt from the logs showing the agent seting the Go proxy (“fortunately” it goes through google.cn, but… you get the idea):

INFO  2026-01-12T17:27:08 +0ms service=acp-agent event={"part":{"id":"prt_bb33f4a91001cjhLK5GlGb6JtS","sessionID":"ses_452fc8c2affeug9efc7DE65VRd","messageID":"msg_bb33f37d3001j20t9IKK6MN569","type":"tool","callID":"call_1ed63d94eb3140709de6c67c","tool":"bash","state":{"status":"running","input":{"command":"cd /home/me/Sync/Development/Experimental/gotel && go env -w GOPROXY=https://goproxy.cn,direct","description":"Try Chinese Go proxy"},"time":{"start":1768238828180}}}} message part updated
INFO  2026-01-12T17:27:08 +1ms service=server method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +0ms service=server status=started method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +0ms service=server status=completed duration=0 method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +0ms service=acp-agent event={"part":{"id":"prt_bb33f4a91001cjhLK5GlGb6JtS","sessionID":"ses_452fc8c2affeug9efc7DE65VRd","messageID":"msg_bb33f37d3001j20t9IKK6MN569","type":"tool","callID":"call_1ed63d94eb3140709de6c67c","tool":"bash","state":{"status":"running","input":{"command":"cd /home/me/Sync/Development/Experimental/gotel && go env -w GOPROXY=https://goproxy.cn,direct","description":"Try Chinese Go proxy"},"metadata":{"output":"","description":"Try Chinese Go proxy"},"time":{"start":1768238828182}}}} message part updated
INFO  2026-01-12T17:27:08 +0ms service=server method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +0ms service=server status=started method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +1ms service=bus type=message.part.updated publishing
INFO  2026-01-12T17:27:08 +1ms service=server status=completed duration=2 method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +0ms service=acp-agent event={"part":{"id":"prt_bb33f4a91001cjhLK5GlGb6JtS","sessionID":"ses_452fc8c2affeug9efc7DE65VRd","messageID":"msg_bb33f37d3001j20t9IKK6MN569","type":"tool","callID":"call_1ed63d94eb3140709de6c67c","tool":"bash","state":{"status":"completed","input":{"command":"cd /home/me/Sync/Development/Experimental/gotel && go env -w GOPROXY=https://goproxy.cn,direct","description":"Try Chinese Go proxy"},"output":"","title":"Try Chinese Go proxy","metadata":{"output":"","exit":0,"description":"Try Chinese Go proxy"},"time":{"start":1768238828180,"end":1768238828185}}}} message part updated
INFO  2026-01-12T17:27:08 +0ms service=server method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +0ms service=server status=started method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request
INFO  2026-01-12T17:27:08 +1ms service=server status=completed duration=1 method=GET path=/session/ses_452fc8c2affeug9efc7DE65VRd/message/msg_bb33f37d3001j20t9IKK6MN569 request

Oh, and I was just sent this RCE report for OpenCode too, so… yeah. Be careful out there.

Lisbon Film Orchestra

Great start to the show
A little while ago, in a concert hall not that far away…

How I Manage My Personal Infrastructure in 2026

As regular readers would know, I’ve been on the homelab bandwagon for a while now. The motivation for that was manifold, starting with the pandemic and a need to have a bit more stuff literally under my thumb.

But I also have a few services running in the cloud (in more than one cloud, actually), and I’ve seldom written about that or the overlaps between that and my homelab.

For cloud provisioning , but that is not what this post is really about–it’s more about how I design what gets deployed.

Zero Exposed Endpoints

One of my key tenets is zero exposed endpoints. That means no web servers, no , no weird port knocking strategies to get to a machine, nothing.

If it’s not exposed to the Internet, then it’s not something I ever need to worry about. But of course there’s a flip side to that: how do I get to my stuff when I need to?

This won’t be popular in many circles, but everything I have exposed to the Internet is behind Cloudflare in one form or another:

  • This site is fronted by Cloudflare (even though it is static HTML in an Azure storage account)
  • I use Cloudflare Tunnels to have a couple of services (web and ) accessible from outside the house (including some whimsical things like a way to share the screen from my when doing whiteboarding), and they are shut down automatically overnight.

To get at anything else, I use (extensively, from anywhere to anywhere). The VMs or VPSes I use across providers are only accessible via (and, of course, whatever native console the provider exposes), and typically have no exposed ports.

Static Is Faster, Lighter and Simpler

I’ve come to the conclusion over the years that there is no real reason for me to run a public web server for 90% of what I do, so things like this site, my RSS feed summarizer, and anything else that needs to publish Web content are designed from scratch to generate static content and push it out to blob storage.

That way, serving HTTP, managing certificates, and handling traffic spikes are literally someone else’s problem, and I never have to worry about it again.

Anything fancy or interactive I typically deploy on my homelab or inside Tailscale.

But, interestingly enough, I also:

  • don’t need to worry about response times
  • need a lot less CPU and memory altogether to get something done
  • can pack a lot more services into cheaper, often single-core VMs

Squeezed down, Concentrated Compute

Over the years I dabbled with many forms of service deployments, and after a long time building enterprise stuff and then refactoring those as microservices (typically using message queues, since HTTP makes you fall into all sorts of synchronous traps), I gradually came to a point where I started questioning how cost-effective some of those approaches were.

So today I don’t use any form of serverless compute. I have often been tempted by Cloudflare Workers, but since I don’t need that kind of extremely distributed availability and I can pack 12 small services inside a dual-core ARM VPS for a negligible fixed amount of money, I don’t have to worry about spikes.

If something somehow becomes too popular, the VM acts as a containment boundary for both performance and cost, which is much better than an unbound serverless bill.

I have been sticking to that approach for years now, since it’s cheap, predictable, and extremely easy to maintain or back up. And since I deploy most of my services as plain docker compose, I can set CPU and RAM limits if needed.

Keep It Dead Simple

Although it’s inescapable in many modern production environments, I don’t use for my own stuff, and the key reason for it is simplicity.

I don’t want to have to worry about managing a cluster, handling volume claims, or deal with the additional CPU and memory overhead that comes with it.

So I have been deploying most of my stuff using , docker compose and kata, my minimalist, ultra-pared-down deployment helper, which has an order of magnitude less complexity.

If I need redundancy or scale-out, it’s much simpler to deploy docker swarm, mount the local provider’s external storage of choice on all the nodes (if needed) and have an ultra-low overhead, redundant deployment. In fact, I have been for years now.

And, again, it’s easy to set up VM backups with point-in-time restore in the vast majority of providers. If there’s one boring technology that everyone got right, it’s definitely VM backups.

SQLite is awesome

I work a lot with huge data warehouses, data lakes, and various flavors of , plus all the madness around data medallions, ETL, data marts, and the various flavors of semantic indexing the agentic revolution needs but nobody really mentions.

But for my own stuff, I always go back to because it is both much simpler and surprisingly flexible:

  • I can store timeseries data in it
  • it is stupidly fast (there’s a 10GB SQLite file with all my home automation telemetry for a year, and it’s surprisingly zippy for the hardware it’s running on)
  • I can enrich it with indexable JSON without breaking the whole schema
  • it has baked-in full-text indexing
  • I can use it as a vector store with a couple of extensions
  • I can back it up trivially

So I hardly ever deploy any kind of database server–but when I do, it’s always .

And yet, I only have two instances of it running these days.

Secrets Management

Even with zero exposed endpoints, secrets management is still a thing I need to worry about. To reduce the number of moving parts, I’ve been using docker swarm secrets for most of my apps (or just the provider’s secrets management: Azure Key Vault, AWS Secrets Manager, etc.).

On my homelab I’ve been using Hashicorp Vault, but it is far too complex for most of my needs and I’ve been dabbling with a replacement.

Bringing It All Home

My homelab approach is pretty much the same: everything is behind , (almost) nothing is directly exposed to the Internet, and I use docker compose for most of my application deployments, except that the hypervisor is and I use containers extensively instead of full VMs.

There is a lot of FUD out there around running docker inside , but backing up an entire and its multiple docker compose applications as a single unit is incredibly convenient and much more efficient than a VM.

The only real issue I’ve had (a couple of times) is that a misbehaving container can bog down the entire host if resource limits are not set properly or if it abuses I/O (which is particularly easy to do in a NAS with HDDs), so those services live inside regular VMs.

Incidentally, podman has a bunch of issues running inside containers, largely related to cgroup and UID management.

As to service definitions themselves, docker compose and the like, everything is backed, of course, and I use to manage all of it, together with and a few custom actions.

Observability

This is the bit that I have been sorting out, and I’m converging towards a combination of for metrics and a custom OpenTelemetry collector I’m working on called Gotel to gather traces and logs from my applications.

I use the cloud provider’s managed backends for those (Azure Application Insights, AWS CloudWatch, etc.), but I want something simpler and more portable for my own stuff–and I’m building it now.

Notes for December 25-31

OK, this was an intense few days, for sure. I ended up going down around a dozen different rabbit holes and staying up until 3AM doing all sorts of debatably fun things, but here’s the most notable successes and failures.

Read More...

TIL: Restarting systemd services on sustained CPU abuse

I kept finding avahi-daemon pegging the CPU in some of my LXC containers, and I wanted a service policy that behaves like a human would: limit it to 10%, restart immediately if pegged, and restart if it won’t calm down above 5%.

Well, turns out systemd already gives us 90% of this, but the documentation for that is squirrely, and after poking around a bit I found that the remaining 10% is just a tiny watchdog script and a timer.

Setup

First, contain the daemon with CPUQuota:

sudo systemctl edit avahi-daemon
[Service]
CPUAccounting=yes
CPUQuota=10%
Restart=on-failure
RestartSec=10s
KillSignal=SIGTERM
TimeoutStopSec=30s

Then create a generic watchdog script at /usr/local/sbin/cpu-watch.sh:

#!/bin/bash
set -euo pipefail

UNIT="$1"
INTERVAL=30

# Policy thresholds
PEGGED_NS=$((INTERVAL * 1000000000 * 9 / 10))   # ~90% of quota window
SUSTAINED_NS=$((INTERVAL * 1000000000 * 5 / 100)) # 5% CPU

STATE="/run/cpu-watch-${UNIT}.state"

current=$(systemctl show "$UNIT" -p CPUUsageNSec --value)
previous=0
[[ -f "$STATE" ]] && previous=$(cat "$STATE")
echo "$current" > "$STATE"

delta=$((current - previous))

# Restart if pegged (hitting CPUQuota)
if (( delta >= PEGGED_NS )); then
  logger -t cpu-watch "CPU pegged for $UNIT (${delta}ns), restarting"
  systemctl restart "$UNIT"
  exit 0
fi

# Restart if consistently above 5%
if (( delta >= SUSTAINED_NS )); then
  logger -t cpu-watch "Sustained CPU abuse for $UNIT (${delta}ns), restarting"
  systemctl restart "$UNIT"
fi

…and mark it executable: sudo chmod +x /usr/local/sbin/cpu-watch.sh

It’s not ideal to have hard-coded thresholds or to hit storage frequently, but in most modern systems /run is a tmpfs or similar, so for a simple watchdog this is acceptable.

The next step is to make it executable and figure out how to use it via systemd templates:

sudo chmod +x /usr/local/sbin/cpu-watch.sh
# cat /etc/systemd/system/[email protected]
[Unit]
Description=CPU watchdog for %i
After=%i.service

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/cpu-watch.sh %i.service
# cat /etc/systemd/system/[email protected]
[Unit]
Description=Periodic CPU watchdog for %i

[Timer]
OnBootSec=2min
OnUnitActiveSec=30s
AccuracySec=5s

[Install]
WantedBy=timers.target

The trick I learned today was how to enable it with the target service name:

sudo systemctl daemon-reload
sudo systemctl enable --now [email protected]

You can check it’s working with:

sudo systemctl list-timers | grep cpu-watch
# this should show the script restart messages, if any:
sudo journalctl -t cpu-watch -f

Why This Works

The magic, according to Internet lore and a bit of LLM spelunking, is in using CPUUsageNSec deltas over a timer interval, which has a few nice properties:

  • Short CPU spikes are ignored, since the timer provides natural hysteresis
  • Sustained abuse (>5%) triggers restart
  • Pegged at quota (90% of 10%) triggers immediate restart
  • Runaway loops are contained by CPUQuota
  • Everything is systemd-native and auditable via journalctl

It’s not perfect, but at least I got a reusable pattern/template out of this experiment, and I can adapt this to other services as needed.

Ovo

Yeah, I don’t know what the grasshoppers want with the egg either
Another great evening spent in the company of Cirque du Soleil

Predictions for 2026

I had a go at doing predictions for 2025. This year I’m going to take another crack at it—but a bit earlier, to get the holiday break started and move on to actually relaxing and building fun stuff.

Read More...

Notes for December 9-24

Work slowed down enough that I was able to unwind a bit more and approach the holiday season with some anticipation–which, for me, invariably means queueing up personal projects. So most of what happened in my free time over the past couple of weeks was coding-related.

Read More...

The Big Blue Room

A lovely mirror
This part of town never disappoints, even in winter.

2025 In Review

Like , this is my somewhat rushed recollection of the year that was, and as usual it’s a mix of personal and professional reflections, with some thoughts on technology and trends thrown in for good measure.

Read More...

Archives3D Site Map