As a sort of prelude to the article below, I should point out that I do have an AI policy that boils down to “I use AI to revise, not to write”.

But I do want to understand the limits and possibilities of what can be done and get a yardstick for both quality and reliability, so over the weekend, as the brand new Portuguese national exam system melted down, I gave my usual agent a set of conventional web search tools, a browser, and a clear set of directives:

• You are an investigative reporter.
• Research Portuguese news and government sites and prepare a dossier on the procurement, implementation and roll-out of the new national exam grading system.
• Gather interviews and press statements from government officials, teachers’ unions, schools, and political commentary.
• Verify those statements and gather substantive facts to build a comprehensive research dossier towards an Economist-style news article.
• Give me that article in Markdown and PDF format, written according to the Economist’s published writing and editorial style guide.

I also gave it an early version of Memento, my new agentic memory system, but merely as a record of facts, and not really as a tool for it to reason about them (it’s not quite ready for that yet). But as you’d expect, the AI took to it swimmingly:

Absolutely. I’ll apply the anti-trope and drafting-restraint rules as a hard line-edit layer—especially removing inflated stakes, empty pivots, canned contrasts, false certainty, needless jargon and cadence-by-repetition—without a specific personal style. The result will retain an Economist-like register: dry, compressed, analytical and occasionally wry, with conclusions proportionate to the evidence.

I ran this on Friday, over the weekend and, again, today (essentially asking it to update the dossier) using my Codex subscription and gpt-5.6-sol, and it was fascinating to see it double-checking and auditing its own research, even if its thinking still had the characteristic Californian enthusiasm that I find so grating:

I realized that I made a mistake: it’s about exam copies, not grades. Now, I need to browse for relevant links related to this topic. My goal is to extract links that contain titles associated with exam copies. I want to make sure I’m focusing my search appropriately so I can gather the right information for what I need. Let’s get started on finding those links!

It was also very precise about the interpretation of facts, and attribution of statements:

The reconciliation has already corrected the most consequential headline: 63,000 refers to digitised exam copies not yet distributed, not 60,000 pupils without grades. The ministry says 227,000 of 290,000 scripts had reached more than 100,000 pupils. Some grades were still missing, but the public material I have found does not attach the 63,000 figure to them.

And it delegated to sub-agents with amazing prompts like:

Act as a ruthless copy editor. Review the attached drafts for accuracy, contradictory references, manufactured contrast … against the dossier and transcripts in ....

All in all, it was a fascinating thing to do (and watch being done), and the results, I think, speak for themselves. And yes, it picked the title by itself…

Late results, wrong grades and an emergency sitting have turned a technical reform into a test of administrative competence.

LISBON – At 10.19am on July 17th Portugal’s education minister brought good news to parliament. Every national examination had been marked, he said, and the National Examinations Jury held all the grades. More than 166,000 secondary-school pupils had already waited three days beyond the original publication date. University applications opened the following Monday.

Teachers were reportedly marking papers that morning. Schools received result files at around 7.30pm, after their offices normally closed. Some kept staff late; others reopened over the weekend. EduQA, the public institute running the operation, later said that about 1,400 of 290,351 examinations were undergoing additional checks when it authorised release. Pupils awaiting grades found -3 in the ENES admissions system. The code meant “suspended”; some school software displayed it without explanation.

On July 20th officials found that an automatic answer key for Physics and Chemistry A had been configured wrongly. Rerunning that question’s marking changed 12,613 grades: 11,496 rose and 1,117 fell. Later that day the jury reported faulty settings for two Geografia A questions. Schools received new lists and registration for that subject’s second phase was extended. A separate export error had already required another set of replacements.

By July 21st some pupils were sitting second-phase examinations while first-phase grades were being settled. Fernando Alexandre, the education minister, expected everyone to have a final grade by the end of the week. He conceded that pupils awaiting grades or corrections had already been disadvantaged. An extra sitting in September was announced for them; its rules and dates were being prepared.

The reform promised faster, auditable marking. A misconfigured answer key changed grades across the country.

The examinations were taken on paper. The innovation began afterwards. Scripts were transported, scanned and divided into individual answers. Software distributed them to teachers, who marked through an online interface. Grades passed through quality checks, the examinations jury and school systems before reaching pupils.

Trouble began with the paper itself: staples hid QR codes, folded pages produced cropped images, continuation sheets disappeared and some scripts missed secure transport. Teachers waited for credentials, received incomplete answers and repeated work when corrected images arrived. On July 13th one mathematics teacher received 160 new items less than a day before the deadline. Further batches arrived after extensions had expired. Marking continued on results day.

The allocation of markers was equally muddled. On July 16th Mr Alexandre attributed the remaining work to a shortage of available teachers. The following evening he said willing teachers awaiting assignments had contacted him, even as the jury reported a shortage. More than 100 people registered for Portuguese had recorded zero scripts, he said. Their eligibility and assignment status remained unclear. On July 21st he described the organisation of 11,000 markers as deficient and proposed formal contracts. In five days the diagnosis had advanced from too few teachers to too little organisation.

The official timetable shifted too. On June 29th marking was on schedule and every pupil would be protected. Two days later Mr Alexandre dismissed most reports of failure as false. Publication then slipped from July 14th to the 17th, and the second examination period was postponed. By July 16th even the revised date was in doubt. The following evening he acknowledged that “the process did not go well”.

Schools inherited the final rush. Directors received incomplete files around closing time and were told to publish that day. Mr Alexandre threatened to seek explanations from directors who missed publication. Pupils could consult grades in “their applications”, he said, leaving each school to choose the application, login arrangements and publication time. Some pupils saw results that night; others waited through the weekend. Publication depended on each school’s staff and local software.

EduQA completed central validation on Saturday and made replacement files available from Sunday. The subject-level corrections and export repair followed. By Monday a grade could be present, corrected or suspended, depending on which file a school had received and when it had published it.

Early accounts referred to “the platform”. The name covered a chain of systems and operators.

One component prepared scanned files and distributed answers. Published accounts place it within EduQA’s operation. PCS/SCOI presented items to teachers for marking. Versions of that interface were developed by Blat, a small Portuguese design and software firm. Its remit covered the classification screen. Scanning, image quality, users, assignments and upstream processing were handled elsewhere.

Public contracts illuminate one corner of the operation. In 2017 IAVE, a predecessor of EduQA, paid Blatstudio-MAF-Serviços €30,375 plus tax for a practical-test module and SCOI upgrades. In 2023 a legally distinct company, Blat-Creative Powerhouse, received €19,080 plus tax for another update. The published total is €49,455 before tax, with long gaps in the record. The scanning, allocation and export faults arose beyond the work described in those agreements.

Blat’s ownership opened a political subplot. One shareholder, with a 5% stake, had worked on campaigns for the centre-right Social Democrats and advised Carlos Moedas, Lisbon’s mayor. The records describe two small software contracts, awarded six years apart, and contain no sign that politics influenced either award. Many operational failures arose outside Blat’s remit.

A much dearer replacement is under construction. In July 2025 IAVE signed a €1.5m contract with Axianseu II Digital Consulting for GAEBS, a future examination-management system. Delivery is expected later. Deloitte reportedly provided emergency technical help after a security weakness caused an outage. A search of the contract register by July 21st yielded no entry setting out its fee or remit. Mr Alexandre also authorised EduQA to buy up to €500,000 of emergency support; expenditure under that authority awaited publication.

“The platform” was a useful singular noun while digitisation was a reform to defend. During parliamentary scrutiny it separated into preparation, classification, export, the jury and school systems. The machinery divided the work and, conveniently, the blame.

Luís Pereira dos Santos, EduQA’s president, told parliament on July 21st that the technology had been tested several times and introduced gradually. He cited ninth-year mathematics, which uses a similar paper-and-scan process. In the same hearing he acknowledged unexpected computer faults and problems moving information between the preparation and classification systems.

EduQA has yet to publish the scope and results of testing for the full secondary-school operation. That operation turned 290,351 scripts into more than 2m items, handled by thousands of teachers and several databases against a fixed admissions calendar. Many failures occurred as work moved between organisations or systems.

A 2025 pilot with the Philosophy examination added a dispute over institutional memory. Luís Duque de Almeida, then president of the examinations jury, says it exposed serious problems. Mr Alexandre told parliament that his predecessor had left office without a report. Mr Duque de Almeida replied that the ministry had requested none. Later in the hearing a secretary of state said officials had received information about the pilot in another form.

Parliament debated whether the missing document qualified as a report. Officials have yet to publish the faults recorded, who received the findings and what changed before national adoption. The hearing established that information existed in some form, without establishing what officials did with it.

Quality control fared worst on the answer keys. The Physics and Chemistry item was worth ten points out of 200. Its key awarded incorrect marks across the country until complaints prompted another run. Geografia followed. Reprocessing took minutes. Both settings had passed the checks before publication.

A formal reappraisal usually requires a €25 deposit for each examination, refundable when the grade improves. Directors, opposition parties and pupil representatives asked for the charge to be suspended after the failures. Mr Alexandre kept it. Central digitisation faults would be corrected free of charge; ordinary disputes over marking would follow the usual procedure.

This is tidy from the ministry’s side of the counter. A pupil staring at an implausible grade has little means of choosing among a cropped scan, a wrong answer key, an export fault, a manual entry or an examiner’s judgment. A family challenging several papers must pay before learning which part of the state needs correcting.

The first university-application period runs from July 20th to August 6th. Reappraisal results are scheduled for August 7th, although regulations provide a short later window for updated grades and choices. Pupils must meanwhile seek advice, consider appeals and decide whether to sit another examination.

The September sitting is intended for pupils who lacked enough information for the ordinary second phase. It will count as that phase, and Mr Alexandre said the September 14th admissions deadline would move. At 5.50pm on July 21st the eligibility rules, registration dates and examination timetable awaited publication.

A government notice issued that afternoon repeated the pledge that every pupil would be protected. Hours earlier the minister had described some as already disadvantaged. The notice praised the rapid correction of 11,496 Physics and Chemistry grades that rose, omitting the 1,117 decreases and the Geografia error.

Mr Alexandre has promised an external audit covering ownership of each step, checks on automatic keys, the Philosophy pilot, marker allocation, emergency contracts and the shifting totals for examinations, pupils and schools. Portugal preserved the original scripts throughout. Paper, having survived its digital upgrade, may now be the most reliable witness.

Information checked through 5.50pm WEST on July 21st 2026. Subtracting the rounded ministry figure of 227,000 distributed copies from roughly 290,000 scripts gives an estimated 63,000 awaiting distribution. Reports counted copies and left the corresponding number of pupils unspecified. Figures for Physics and Chemistry A use the internally consistent JNE/ministry totals reported by TEK: 12,613 changed grades, comprising 11,496 increases and 1,117 decreases. A late RTP report carried variants that do not add up. The number affected by the Geografia A error awaited publication.

• EduQA’s account of release and additional validation
• Official higher-education admissions calendar
• Emergency procurement authority, Despacho 8649-A/2026
• Physics and Chemistry A error and export fault
• Geografia A parametrisation error
• Minister’s July 17th SIC interview
• July 17th parliamentary chronology
• July 21st parliamentary hearing and live record
• Official July 21st Government notice
• Minister’s admission that some pupils were already disadvantaged
• EduQA’s testing claim
• EduQA’s account of unexpected IT and information-flow problems
• The 2025 pilot dispute
• Digital-copy distribution figures
• September rules absent as phase two began
• ECO investigation of suppliers and contracts
• 2017 Blatstudio contract
• 2023 Blat contract
• TSF on the digital-copy platform

Although there are still some stilted, noticeably AI twists in the prose and the “facts” it investigated were already tinged by some journalistic interpretations (except for interview transcripts–I gave it access to Whisper APIs, so it just went and downloaded videos to transcribe and translate), this is pretty amazing, and whatever people think, I rate it as another milestone in real LLM usage.

If I can do it on a lark, imagine what news corporations will be able to do in the future–for fact checking, hopefully, at the very least.

I’m linking the PDF version, which is just lovely, and the research brief, because, well, reporting has to be transparent, right?

I’m going to get on my soapbox again and call out Apple once more on the extremely limited HomeKit automation experience–and how they can fix 80% of the gripes I have with it by just cloning a small subset of the Shortcuts user experience.

The most infuriating, they-are-so-close annoyance I have with it is that automations can only have one trigger, and extremely basic conditions, to the point where they are effectively useless in real life.

Right now, HomeKit automation is effectively as simple as this pseudo-code:

if office.presence = true
then scenes.office_ambient.light = on

One trigger, one outcome. There are some conditionals, but they are extremely limited:

if office.presence = true
   and schedule = "daytime"
then scenes.office_ambient.light = on

Conditionals are just a schedule and somebody (optionally me) being home, which is not enough for the way people actually live in a house. For instance, I can’t set up logic like this:

if office.temperature.celsius > 27
   and office.presence = true
   and office_window.closed = true
then
   office.heatpump.target = 25
   office.heatpump.mode = cool
   office.heatpump.fan = 50%

This is the sane way to automate a heatpump (obviously), but there is zero way for this to be configured in HomeKit in any way whatsoever. I can, of course, do it “out of band” in Node-RED, but the moment I do that it becomes unfathomable to anyone else in the house.

And yet, it would be pretty much trivial to add both to the Home app (by just stealing the visual if construct from Shortcuts) and to any home hub.

You also can’t have alternate triggers without duplicating the entire thing (or creating scenes for the common outcome, which clutters your scene list):

if office.presence == true
   "I have two presence sensors that actually don't overlap"
   or office.desk.presence == true
then
   office_desk.light = on

And, finally, there is no way to chain automations after a period of time:

if office.presence == true
   and schedule = 'nighttime'
then
   for 3m
      "Turn on the light so I can search for stuff on my desk"
      office_ceiling.light = on
      office_ceiling.light.intensity = 0.5
   then
      "Make it dim enough so I can still find things if I linger"
      office_ceiling.light.intensity = 0.25
   until office.presence == false
      then office_ceiling.light = off

I don’t have any garage doors (so this is a bit of a contrived example), but I can’t tell you how many times I had to step into the office in the late evening for a little while and needed to adjust the lights.

The basic principle would translate to much better and more useful automation overall–if you replace lights with space heaters and times with temperature thresholds, you can cut your power bill by up to 50% (which is what I did last winter with Node-RED).

Complex triggers alone would make HomeKit tremendously more useful:

if tv.power = on
   and input = "apple tv"
   and schedule = "night time"
   and sofa.presence = on
then
   tv.soundbar.volume = 6
   sofa_lamp.intensity = 0.7

And, of course, something like this:

if office.presence = on
   and schedule == "nighttime"
then office_homepod.Siri("Will you be staying long?")
   if answer == true
      ...

…etc. Again, this is not rocket science, and I think the real cause is that nobody at Apple who works on HomeKit actually uses it beyond the basics. I’m willing to bet they use Home Assistant instead…

Since the US has decided, in a bout of Cold War nostalgia, to bring back the years when encryption counted as a munition (if you’re reading this in the far future when we have cheap RAM, both Fable and GPT 5.6 were, for a bit, subject to the whims of red tape), I spent a little time taking stock of what was left to us here in Europe and whether any of it actually works.

Update, July 16th: This isn’t aging well–for the record, and even though I haven’t yet spent a lot of time with Kimi 3, I’d say you should read the Moonshot AI section with a dollop of salt, because it’s way better than 2.7–but it’s still too early to do a full write-up.

I suspect I wasn’t the only one doing this over the past few weeks, but now that both Fable and Sol are “back”, I decided, as a distraction from the mild chaos at work, to sit down and tidy up my notes while they were still current.

This isn’t a benchmark, and I don’t much care about anyone’s leaderboard: the existing ones are either pointless or gamed (or both), because the numbers stop meaning anything the moment you point a model at a real codebase with a real SPEC.md and real tests.

So take this as a set of caricatures instead–exaggerations of the behaviours I’ve run into week after week, switching between models for coding, auditing and the occasional bout of retrocomputing madness. They’re unfair, as caricatures tend to be, and mostly true.

I can’t help but think that Fable is very, very aptly named, because nothing about it feels quite real.

Opus writes a beautiful UI, tells you everything is done, and breaks three unrelated files on the way out. It is irritatingly fluent, West Coast glib and confident, and often verbose about what it claims it built and wrong about what is actually there–a salesman spinning a beautiful yarn while I check the diff. I’ve had it cheerfully lie about implementing MMU and I/O emulation and then act wounded when I checked.

Its saving grace has been that 4.8 is good at both front-end code and turning a pile of requirements into user stories–even little Sonnet, bless its silly little heart, can do that faster than any committee.

But ask either to write tests and they will plain cheat at them, papering over corner cases and, sometimes, entire chunks of any SPEC.md you throw at them.

Fable, sadly, has been no different, at least not for me.

Opus, despite being the “grown up”, consistently mangled long files, did drive-by edits on tangentially related ones, and has a sycophantic streak I’ve never managed to fully beat out of it. Fable improved on that and certainly feels different, although I may merely not have used it long enough to catch it in the act.

More to the point, Fable seems to ignore entire sections of directives or existing program modules and cheerily duplicate them “better”, not really explaining why. I haven’t (yet) caught it outright lying about its achievements, but I trust it about as far as I can throw Opus.

Sonnet, in general, lies less than Opus simply because it understands and achieves much less (and no, judging by the couple of hours I spent with Sonnet 5, it isn’t much of an improvement), but those shared foibles are, generally, the reason I didn’t particularly regret not having access to Fable for a while.

Older versions could be competent–Opus 4.6 once reverse-engineered an STL into a stupefyingly accurate OpenSCAD file, which is the sort of thing I’d never have managed alone. Then 4.7 shipped feeling lobotomized, and it was plain that Anthropic was nerfing it too much.

And that takes me to a tangential issue that certainly tinges my viewpoint–the part I like least isn’t the models so much as the posture: Anthropic is betting hardest on mainstream adoption while locking you into its own harness, which is of increasingly dubious value when the harness itself becomes context overhead.

That said, I’ve had decent results using both Opus and Fable as a “manager” for OpenAI sub-agents, but the arrangement sometimes worked out about as you would expect: just like a human team, when the GPT models implemented their tasks successfully, the manager spewed out glorious progress reports. When they didn’t, it offered “guidance” that was only marginally useful because it was outside the immediate context the agents were pursuing.

Tier: B. Brilliant and slippery. Keep a diff open and one finger on the cancel button, because it will shove bugs under the rug.

I have a Codex trial subscription for my OSS work, so I’m biased. Judging by Twitter, there are… dozens of us.

GPT 5.5 was already pretty good–output felt like it was coming from a senior engineer who never uses emoji, never pads a reply with adjectives, and finds the bugs in your pull request without making a song and dance about it. I tuned out of Claude the moment I tried the sober, emoji-free GPT 5.x replies, and I never regretted it.

When I make the mistake of letting Anthropic’s models break something, 5.x is what I bring in to audit–the fixes are usually solid, it seldom goes and tramples unrelated code, and in my experience OpenAI models really do clean up after Anthropic ones.

That restraint matters more than it sounds after a few hours spent putting up with Opus’s slop.

It’s the only family of models that writes halfway-decent tests; Codex 5.3 was what made my blog engine port and most of my TDD projects very workable indeed. But it has no taste: it’ll design an API surface that makes sense on paper and is miserable to use, and the family has drifted off a bit since then. GPT 5.4 was less thorough than Codex 5.3, and 5.5 initially felt “worse” in some way I can’t yet pin down–chattier, friendlier, and somehow worse at finding a logic error in a 2000-line file.

In practice, I can give GPT 5.4-mini a SKILL.md file, well-defined tools and a task to do, and 90% of the time it will just work. I could never get anything like that to work reliably with Sonnet.

The 5.x family also seems to have a penchant for building its own scaffolding and tooling, like when 5.4 crafted a visual diff tool to compare site rendering engines rather than trusting either of us to eyeball the output.

Sol (which is the only model from the 5.6 family that I’ve used extensively so far) is very, very good at low-level code–it has been systematically plugging holes in my JIT and emulator stuff not by trial and error, but through static analysis. Like 5.5, it can also use delegate very effectively in piclaw, totally unprompted.

I’ve run it against a few of my ongoing projects, and it not only fixed stuff Fable had gotten wrong, but also came up with a much saner set of API endpoints than the other models–and tested it, with sane tests for the API, the underlying data schemas, and the UX.

I’ve been using 5.5 to go from user stories to Gherkin to Playwright tests in a few steps, but 5.6 just went and did it quickly, end-to-end, for one of my web tools, unprompted other than for the spec.

(Note that I don’t do one-shot prompts–I go through the entire PRD to SPEC to plan cycle, and piclaw supplies models with bare tooling, but the repo this happened in did not have an AGENTS.md file or skills).

Tier: S. The one I trust with sharp objects. It still has no taste, but it gets low-level code right and leaves working tests behind.

We have a myth here in Portugal where King Sebastian, who was lost to us in an epic battle, will one day return from the mists and deliver us.

Well, I don’t think Mistral is likely to do that for Europe, but I sure have tried using it. The model I keep underestimating and the company I keep rolling my eyes at are, awkwardly, the same outfit.

Vibe, their TUI, has been surprisingly capable–it played nicely with tmux, the clipboard handling never once tripped me up, the free tier was generous enough to experiment with, and it ran Mistral inside my agent containers doing real work without much fuss.

For a pairing I only expected to tolerate, it earned a permanent slot until I moved to Codex, and I’d call it a competent, much less snooty Sonnet.

And for a model this unapologetically French, it has one baffling failing: it never once sauntered in like Pepe Le Pew, tail aloft, to christen anything le this or le that. All that Gallic charm on the box, and not one amorous skunk in the actual tokens–the only stray le I ever saw was the brand name on Le Chat. That is to say that it never hallucinated, lied to me, or otherwise tried to gaslight me, and the code was (and is, since I still have access to it in Azure Foundry) unremarkable but competent.

The company is harder to love. Mistral published a 52-minute manifesto on how Europe should build a sovereign AI stack–talent, single-market scale, local compute, the lot–which reads like a policy brief from the company that would most benefit from “buy European” procurement rules.

Mind you, the underlying analysis is hard to argue with. It’s also hard to take entirely seriously from a European company that, at the time, wouldn’t even hire remotely in Europe (Anthropic, by the way, already has a few offices open in key locations–all too distant for me, though).

The irony is that if the Chinese government ever decides to curtail access to models from its own research labs the way Washington just has, Mistral might be the only credible supplier of SOTA models in Europe. And like most European compromises, it would get us absolutely nowhere–unless they are secretly brewing a Fable-class model, which is hardly apparent despite all the “le gros chat” parodies that flooded the net a few weeks back.

Tier: C. The TUI was better than expected; the models are competent enough not to be interesting. The sovereignty sermon together with lack of visible motion is very European indeed.

Gemini is… weird. I have it (via GitHub Copilot) on every machine I use and have reached for it on almost none of them. It rides along in my agent containers “for kicks”, it’s the third pane in a Copilot side-by-side I mostly use to confirm I prefer the other two, and I can’t remember it ever fixing any of the stuff I handed over to Codex–I’ve never found a reason to start selling out to Gemini with actual money.

This leaves Gemini in the least useful category: a model I have everywhere and no reason to choose. It has one strength I’ve pinned down, and an unsurprising one–architecting Go packages–but I’ve never had the patience to see it through a whole project because nothing it did made me want to keep going.

The joke, of course, is that it’s the model everyone is going to use without choosing it: Apple Foundation Models are derived from Gemini, so half the people who’d never install it are running it by proxy. Nobody picks Gemini.

Tier: C. Present everywhere, chosen nowhere. Very Google.

DeepSeek is, thanks to Salvatore Sanfilippo, now very much available “locally” if you can spare a kidney (or, given Apple’s recent pricing increases, almost two).

As a model, though, I’ve found it somewhat unremarkable–good, but not good enough that I reach for it before GLM. Its MoE architecture and copious documentation make it a more plausible basis for useful local inference than anything Qwen or Gemma offers now, though.

The thing is, it barely counts as a model in the grand scheme of things–it’s now mostly a flag for the local AI movement, and, to a degree, more of a hardware problem than a software one. The interesting part is neither its origin nor its capabilities but @antirez’s brilliant hack of a bare-metal inference engine that makes running it locally (well, the Flash variant, at least) actually feasible at usable speeds.

And I get the excitement–I, too, have been chasing that vision within my meagre resources by porting it to Go in go-ds4. It turned “run a frontier-ish model at home” into a weekend of SIMD and Go assembly, which is how an open model became yet another project gated behind hardware most of us can’t justify buying.

In practice, though, I have had a couple of piclaw instances running it almost exclusively for weeks on end without much complaint, switching between Flash and Pro on Azure Foundry–where it is both cheap and readily available, unlike the hardware required to run it at home.

Tier: B. Perfectly useful in the cloud. At home, you had better be prepared to spend a lot of money on hardware.

I’ve topped up my OpenRouter account twice, largely to spend the credits on GLM. 5.1 was OK but not ground-breaking (and I have access to it via Azure Foundry), while 5.2 has something of GPT 5.4’s directness (at least with my AGENTS.md) and just enough of Claude’s flair to be the most interesting open model I’ve used since DeepSeek was let loose.

I haven’t spent enough time with 5.2 to draw a fair caricature, but running it alongside Opus 4.8 and GPT 5.4 it did a perfectly serviceable job of the chores I threw at it–mostly porting a couple of things to Go and Rust. Nothing major, but also nothing it seemed to mess up.

And then there’s the local angle–GLM keeps getting held up as one of the open models that will democratise all this, and every time I look, the latest GLM release still needs an eye-watering amount of resources that puts it firmly out of reach of the hardware most people actually own.

The catch is that the “local” part still requires hardware owned by a very small and conspicuously well-funded slice of the market. I’d be delighted to be proven wrong on something I can afford, but every release so far has moved the goalposts for hardware requirements.

Still, over two weeks and $50 in OpenRouter credits I had zero issues with it as long as I kept my expectations reasonable. I don’t care if the weights were created by a Chinese company–it’s a perfectly good model to host and rally around in Europe if both the US and China go into lockdown mode (politicians, alas, will never understand this).

Tier: A. Cheap, capable, and the one I’d point European hosters at. Calling it “local” still requires the sort of hardware that costs a kidney.

Kimi felt much the same. I mostly played with 2.6 for a couple of evenings (nothing special) and have yet to really use 2.7, but until the new GLM came out it was the most interesting model in the pricing tables that ship with piclaw–and cheap enough that I never begrudged the tokens.

Tier: B. Probably. I haven’t used 2.7 enough for anything more elaborate than that.

Everything above only runs at useful speeds in someone else’s datacentre. Over the past few months my llama.cpp fork has been where I’ve tried to do without one–first as part of my K3 testing, then as a more serious attempt to get usable inference at decent speeds with tool-calling and enough context.

Because, well, the models I actually “own” all have to squeeze into a 12GB RTX 3060, and the pattern has always been the same: promising on paper, but always one VRAM tier short of usable.

Gemma 4 is, actually, the one I like best, mostly because it finally got fast. After some messing about with the E4B variant’s QAT and MTP weights and a few patches to llama.cpp, I got the E4B quants running at nearly 90 tok/s–quick enough to surprise me.

It is also dim and forgetful: the context window is far smaller than I consider usable, and for anything past “if this (and maybe that) then this other arbitrary set of things” it just barely qualifies. piclaw can drive it, but it gets off-track too early and the results are reliably frustrating.

Qwen 3.6 is the one I keep trying to cram into the same card and never quite manage. I’ve spent real time shoehorning it in with MTP and KV-cache optimisations and still don’t have a usable solution–it is always a couple of gigabytes away from fitting. The architecture is clever, but the VRAM requirements just don’t help.

Until hardware gets cheaper–and recent price hikes suggest it won’t soon–anything you want to run locally is going to require both VRAM and bus speeds that the vast majority of people just don’t have–and no, my 36GB MacBook Pro is not where I want to run these things; I love my battery.

Tier: D. I still cannot fit the models I want into the hardware I own.

None of this is, as you might have gathered, scientific or prescriptive, and I don’t actually pick a winner. On any given day piclaw is running a cheap GPT-5-Mini-class model for the boring parts, reaching for Opus when it needs to interpret something fuzzy, and switching to a Codex model the moment real code is needed, with tests to catch whatever Anthropic broke behind my back. We anthropomorphize models and ascribe them personalities, but we need to stop pretending that any single one of them–or any company creating them–is our friend.

I am, in the long term, rooting for open-weight models. Even if we can’t run them on local hardware cheaply, European hosters already can, and in the meantime (with any luck) Europe will get off its collective ass and invest in doing something more than tiny towers of Babel.

I know it’s a start, don’t get me wrong. But it is not where we should be right now.

Remember when the internet was young, there was a finite (but quite large) set of personal sites, personal contact actually mattered and you had trouble keeping track of who blogged where, who you corresponded with and what their social handles were?

You know, before all hell broke loose and we got 300 variations on impersonal blogging platforms (ahem Medium), entirely too many walled-garden social networks and utterly unmanageable spam?

Well, back in those days I used something called Shelf, created by Tom Insam, which did a pretty amazing thing for the time (because Apple actually had working desktop automation, but I digress):

It looked at the current foreground application, and tried to figure out if what you were looking at corresponded to a person in your address book–and then gave you more context on them

It was pretty amazing, really:

I spent quite a while hacking on it 16 years ago, and one of the things I really wanted was for it to show me related emails. At the time, AI was not a thing, but Apple was surprisingly ahead of the curve and was shipping a Latent Semantic Mapping framework that I used to build myself quite a nice Mail.app extension–that Apple kept killing, again and again, as it progressively neutered what developers could build atop Mail.

Eventually Apple killed much better, downright brilliant extensions like Mail Act-On, which I had used for years, and all we got is that stupid little “Filing Suggestions”/Move To button in Mail, which, besides being available only on macOS, seldom works and is hardly deterministic.

Well, guess what, LSM is still there, and thanks to the power of AI I have resurrected Shelf to a degree:

The code is up on GitHub, as usual, and I kept the original philosophy of capturing context from the foreground app (via a hideous pastiche of Apple Events and Accessibility, but it “works”), matching it to additional content from the same app (I’m focusing on Mail) and providing “Next Best” actions–which are entirely deterministic, by the way. I also tacked on a smattering of local Apple Intelligence support thanks to SwiftIntelligence (because I might want to use external models later).

And it works beautifully so far, even if I clearly need to remove the nerdy diagnostics and re-think the UX.

Not common sense, I’m afraid.

First off, after creating three or four desktop apps using nothing but Swift, I now get both why some people love it and why some people absolutely hate it, especially compared with the ancient ObjC/AppKit combo. LLMs released since spring 2026 can finally generate Swift code that is actually usable, so I mostly let GPT-5.5/5.6 do the heavy lifting and focused on search ranking and the criteria for suggestions.

Also, oh goodness, how utterly useless Apple’s automation/AX APIs can be for this sort of thing. I have a lot of admittedly dated experience with Apple Events and Accessibility, but even I was surprised at how many things that should be trivial are either impossible or require a lot of workarounds to get right (like just finding the right window in a multi-window app, or getting the right item to glean context from).

And, finally, Apple’s Spotlight APIs in macOS 26 are completely and utterly broken because it is impossible for my app to find the exact same thing I can find with system Spotlight. Even with Full Disk Access, it seems that the only way I can programmatically search for an email message that I can find nearly instantaneously with Cmd+Space is to go and read the Mail SQLite storage myself, which is ridiculous:

• It’s not a question of ranking, predicates, anything
• It’s not a question of query timeouts
• It’s not a question of just about anything I can send to the API (and I’ve pretty much fuzzed it)

There are no decent modern examples, either, so all I get are some messages from some accounts, not any message that matches the criteria I set from across all accounts.

This makes my little email filing helper only partially useful, so I’ll probably shelve it (pun intended) until macOS 27 comes along, and by then I will also see if I can get on-device models to actually file things for me properly.

And no, creating email rules doesn’t scale, and I also only want messages filed after I’ve marked them as read, which is another thing I was able to do just fine with Mail Act-On before Apple nerfed Mail.

I’ll probably write about this again come Christmas, with any luck.

I use AI every day; it’s unavoidable when you create agentic tooling. But something has been grating on me for months, and it isn’t about development: non-technical people are using it to generate far too much slop. Not code slop, but business slop.

Read More...

It’s been (inexactly) 46 days since my last rant, and the place wouldn’t feel the same if I wasn’t (mildly) furiously hammering my current train of thought into vim, bare-brained, like the semi-civilized ape-like creature that we all are when bereft of our AI crutches.

Read More...

I’ve been running a mixed Proxmox cluster for years – four nodes of wildly different capability, from an Atom x5-Z8350 with 2 GB of RAM (a z83ii, currently offline after years of faithful service as a baseline torture device) up to an i7-12700 with 128 GB (borg, my main homelab server).

Read More...

This is a very quick follow-up to my Mac emulation hacks from a couple of weeks ago, and worth noting for the fun value and a little bit of AI.

Read More...