Nov 27^th 2022 · 5 min read

A Diffuse Return

Progress over the past two weeks has been slow, but I’ve progressively been feeling well enough to get back to exercising and taking a stab at my hobbies again (work is another matter–I’ve been ramping up, but Thanksgiving holiday has slowed down everything).

And one of the things that has been on my mind is the recent release of Stable Diffusion 2.0, partly because it intersects with ML (which I wish I was doing more of, not necessarily of that kind), hardware (I keep wishing I had a beefy desktop machine with a hardware GPU, although my MacBook Pro and my iPad Pro can both run Stable Diffusion models just fine), and fun (because I had so much fun with the early version).

So I decided to do the kind of thing I usually do–I added another angle to this, which was automated deployment, and built an Azure template to deploy Static Diffusion on my own GPU instance.

an enginer at a desktop computer — An accurate rendition of how my head felt while I was doing this, courtesy of sinus troubles.

Alternatives

Like everyone else, I’ve been using a mix of free Google Colab workbooks (which have an annoying set up time, even though most of them will cache data on Google Drive to speed up repeated invocations), my MacBook Pro (where I’ve been using either imaginAIry for Stable Diffusion 2.0 or DiffusionBee for older versions), and even my iPad Pro with Draw Things–which works surprisingly well, although it does burn down the battery.

imaginAIry, in particular, appeals to me because it makes no attempt to hide the internals and exposes a nice Python API, so it’s my favorite for reproducible results.

But I also wanted to get a feel for how people were building services around this and needed an “easy” thing to get back up to speed, so I decided to host my own¹.

Picking a VM

The thing is, GPU-enabled instances don’t come cheap. I already went down this track back in 2018 when I fiddled with NVIDIA Geforce NOW, and the only thing I didn’t do at the time was play around with spot pricing.

I have been using spot instances for my Kubernetes clusters, so I started by checking out spot pricing for a couple of interesting SKUs:

The Azure Graph Explorer, being queried for spot pricing — Looking for the cheapest price on the GPU instances I was interested in.

US $0.114/hour is really not bad considering that the retail price for a Standard_NV6 (which is exactly the same instance type I used in 2018) is, as of this writing, exactly ten times that amount, so that seemed OK. The additional surcharges for a standard SSD volume (a tiny one) and networking are residual, so I can certainly fit this into my personal budget for only a few hours of fooling around each month.

And getting the machine preempted would not be a big deal (you can also look up eviction rates, and they’re currently at 0-5%).

NVIDIA Driver Setup

Next, I went and hacked down my k3s template to deploy a single machine and started to figure out how to set up NVIDIA drivers in Linux. Given the hassle involved, I decided to go with Ubuntu 20.04 (which is still LTS, and has a couple more years of testing).

However, first I had to make sure I was picking the right image. And in case you’ve been living under a rock or are still doing Stone Age style manual provisioning in the Azure portal, I’m one of those people who only do automated deployments, so when I rewrote the Azure template I had to account for the fact that the Standard_NV6 uses an older generation of OS images (for a different hypervisor baseline):

    "imageReference": {
        "publisher": "Canonical",
        "offer": "0001-com-ubuntu-server-focal",
        "sku": "[if(contains(parameters('instanceSize'),'Standard_NV6'),'20_04-lts','20_04-lts-gen2')]",
        "version": "latest"
    },

Note that the if conditional above may seem lazy because I’m using contains, but that’s because I have also been playing around with the Standard_NV6_Promo SKU.

The next step was getting the NVIDIA drivers to actually install. And that turned out (after a couple hours of investigation and a couple of tests) to be as easy as making sure my cloud-init script did two things:

Installed ubuntu-drivers-common and nvidia-cuda-toolkit
Ran ubuntu-drivers autoinstall as part of my “preflight” script to set up the environment.

This last command does all the magic to get the kernel set up as well, although there is a caveat–you’ll need a reboot to actually be able to install some PyTorch dependencies (especially those that are recompiled upon install and need nvidia-smi and the like to have valid output, which only happens if the drivers are loaded).

But after a reboot, it works just fine:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla M60           Off  | 00000001:00:00.0 Off |                  Off |
| N/A   47C    P0    38W / 150W |   2505MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      3766      C   python3                          2502MiB |
+-----------------------------------------------------------------------------+

I’ve yet to test this on Ubuntu 22.04, but I suspect it will work just the same (I’m quite curious as to how docker-nvidia will work, although I didn’t need it for this).

And, of course, in the real world I would likely automate a pre-baked OS image so I could have fully reproducible deployments.

But one thing I made sure to set up already (besides adding Tailscale for easier remote access) was configuring the VM for automated shutdown at dinner time. You can do that via the portal in a few clicks, and I will be adding it to my template later.

After all, you absolutely must set guardrails on how you use any kind of cloud resource… Especially if you’re experimenting–that’s usually when things get out of control.

Picking a Version

I started out using imaginAIry and JupyterLab for Stable Diffusion 2.0, but I quickly realized that the new training set used for 2.0 is vastly different and omits most of the nice art styles that I’m rather fond of.

So I parked that for a while and decided to revert to 1.5 and the AUTOMATIC1111 web UI (which also supports 2.0 somewhat).

And I’m not at all sad about missing out on 2.0 for now, especially considering I can still get this kind of output out of 1.5:

An engineer messing with shiny crystals — Ooooh, shiny!

There was also a serendipitous coincidence in that I was asked to help in a couple of things, one of which also included setting up a GPU instance, so this was a good way to keep the momentum going once I clocked off work. ↩︎

Nov 25^th 2022 · 1 min read

TIL - Minimal Go WebDAV server

I had an interesting little conundrum the other day, which was that I needed to help someone access a remote Linux filesystem via Windows Explorer (long story…) but had no direct connectivity, no way to use network filesystems, no way to use VS Code remote editing (I was going through a bastion) nor an SSH client that would do sshfs.

I could, however, set up a TCP tunnel, so my initial reaction was to go and check if there were any single-file WebDAV servers in Python, but then I realized that Go has a perfectly serviceable WebDAV module and hacked this together:

package main

import (
    "flag"
    "golang.org/x/net/webdav"
    "net/http"
)

func main() {
    var address string
    flag.StringVar(&address, "a", "localhost:8080", "Address to listen to.")
    flag.Parse()

    handler := &webdav.Handler{
        FileSystem: webdav.Dir("."),
        LockSystem: webdav.NewMemLS(),
    }

    http.ListenAndServe(address, handler)
}

With go 1.13 (which is what I had at hand), you need to do:

go get golang.org/x/net/webdav
GOOS="linux" GOARCH="amd64" go build -o webdav main.go

So I copied the binary across, typed http://hostname:port into Windows Explorer, and everything worked: uploading large files, creating directories, editing configurations, etc.

Of course I had to tidy up permissions here and there, but overall, it “just worked”.

Nov 20^th 2022 · 2 min read

A Week Later

It’s been over a week now and we’re all still testing positive for COVID, plus a few random things have happened.

Nov 13^th 2022 · 3 min read

COVID Update

So it finally happened. Nearly three years in, and (coincidentally or not) just after my first real trip in years, I got COVID, and so did all of my family, which I guess qualifies this as the latest post in that particular series.

Nov 6^th 2022 · 6 min read

The Elon Gambit

Well, that was a “fun” week. I was originally planning to clean up another of my long-standing drafts and post about one of my projects, but in between a small trip and the chaos at Twitter, I thought I’d hop on the bandwagon and write about that as well.

Nov 3^rd 2022 · 1 min read

WeMeet Europe 2022

mid-morning break — I went out to Porto to speak about Microsoft’s strategy for the telco market, and it was nice (but weird) to stand on a stage again after three years.

Nov 1^st 2022 · 1 min read

The Ventura Problem ※

This is the kind of “but it worked perfectly before” reason why I don’t usually upgrade to new macOS versions before there’s at least one point release.

In short, macOS Ventura is the only operating system where you can’t program a Raspberry Pi Pico (and, potentially, any other board that supports MicroPython or CircuitPython, not just RP2040 ones) by just dragging files onto its virtual USB drive.

Like pretty much every single filesystem-related weirdness we’ve been subject to throughout the history of macOS, it may look like an edge case to the Finder team at Apple, but it is a breaking issue for anyone using their otherwise perfectly good Mac for a perfectly normal thing that just works on every other operating system.

I am mostly at peace with the fact that we will never be rid of .DS_Store in USB pen drives because Apple can’t find the willpower to be good removable media citizens, but like all the extended attribute stupidity we’ve seen over the past twenty years, preventing the Finder from writing them by default to non-macOS volumes (or even thinking about that in the first place) would definitely have paid off in this instance…

I’m tracking the GitHub issue to see how long it takes to get this fixed, but I’m not holding my breath here.

Oct 29^th 2022 · 6 min read

Homelab Update

Every now and then I spend an hour or so in the evenings poking at my homelab, which kind of adds up into actual productivity over a couple of months.

Oct 28^th 2022 · 1 min read

Elon Musk completes Twitter purchase ※

I don’t have any plans to leave Twitter (and I can be found on Mastodon if you look hard enough), but to be honest I haven’t been deriving any real enjoyment from it for a few years, so I hope this will at least make it a bit more interesting.

However, it needs saying that “general purpose” social networks have, by and large, become less and less appealing. Messaging has spun off into its own thing, Instagram and TikTok are largely view-only or have become a Darwinian meme zoo, and smaller, more focused communities are a lot more engaging (at least for me).

The one thing Twitter has accomplished over the past couple of years is demonstrating how we are all, everywhere, still in thrall to the US tech industry (well, discounting the WeChat universe, of course).

And that, in turn, reminds me that the EU is still trying to sponsor an European search engine, which feels almost steampunkish in this day and age.

Oct 28^th 2022 · 1 min read

TIL - VIM spelling and plugin interaction

After months of writing, I realized that I was using vim spell checking all wrong, which led to quite a few typos making their way online.

The reason for that was that I had added a few plugins (like vimwiki) that overrode formatting and highlighting settings, so a lot of my typos were actually there and flagged by the spell checker, just not visibly highlighted.

So I went and moved my Markdown and spell checking settings after all the other plugins and color settings were loaded, and they now look like this:

"--- Writing ---
Plug 'tpope/vim-markdown'
" enable fenced code highlighting
let g:markdown_fenced_languages = ['html', 'python', 'bash=sh', 'javascript']
set spell spelllang=en_us
hi SpellBad term=standout cterm=standout,underline gui=underline

The last line is the critical thing, but I would rather manage settings as a block for each use case, and I’m somewhat annoyed that this has been going on for so long.

Oct 24^th 2022 · 1 min read

Stage Manager in iPadOS 16 ※

Federico makes a number of good points about Stage Manager, and although I have exactly zero interest in using it on a Mac, I do share his view that it is just nice to have it on the iPad¹.

I have been trying it out for a couple of hours and although it is clearly somewhere between half baked and awkward (and yes, it’s already blown up on me, from random keyboard failures to full-on Springoard reloads), Stage Manager is already pretty useful to people like me, especially with both a-Shell and the new Blink update (provided you do the sane thing and use tmux in case iOS wipes your sessions).

The workspace management is definitely an acquired taste–I simply hid the workspace strip, and keep wishing the whole thing worked like regular macOS desktops. Some system keybindings for arranging windows would be great, but I can’t wait to have this working in an external display.

I also have a nagging, sinking feeling that proper external display support would be the kind of feature the modern Apple would want to defer for the M2 iPads–that way they could nickel and dime us for the privilege of finally using an iPad with a monitor in a sensible way.

But I hope that won’t be true, and am still somewhat amazed that we’ve come a long, long way. Twelve years later, and I can finally use an iPad without it feeling like a bigger iPod Touch.

And I find it simply hilarious that Instagram now works properly under Stage Manager… ↩︎

Oct 23^rd 2022 · 9 min read

Fast Remote Desktop on a Raspberry Pi, with Glamor and an iGPU

If anyone were to tell me that I would be hacking X11 server modules in 2022, I’d say they were nuts. And yet, here we are.

Oct 18^th 2022 · 1 min read

30 Years of Vodafone Portugal

António Carrapatoso, Mário Vaz and António Coimbra — The three (former and current) CEOs, all of whom I’m very proud to have worked with.

Oct 16^th 2022 · 6 min read

Seven Years At Microsoft

I’ts now been seven years since I joined, and a year since I moved from what is now Industry Solutions to Azure for Operators.

Oct 12^th 2022 · 1 min read

NVIDIA RTX 4090 review ※

This thing is a beast both price-wise and size-wise.

Although the morality of jamming a three-slot GPU that takes 450W of power into a desktop machine is dubious in a time of energy constraints, the sheer amount of raw performance it delivers is… sobering.

And then there’s the insane price (and requirements that may require entirely new cases and PSUs). I’ve been keeping tabs on the GPU market during the crypto madness and we’re long overdue more affordable GPUs, but this isn’t it.

Alas, I’m not sure Intel or AMD are (or will be) in a position to drive costs down - and their support for ML is comparatively lacking…

Oct 9^th 2022 · 1 min read

Wiki Link Visualization ※

I was a bit annoyed at how hard it was to visualize the sheer amount of pages this site contains, so I replaced the draft 2D visualization I had on my 20 year anniversary post with a pretty nice WebGL one, which is going to live here for the foreseeable future.

The library I found to replace my hand-rolled D3.js code also supports VR, so if that’s your thing, use this link instead.

Warning: The VR version will not work directly an Oculus Quest 2 for now – it is simply too detailed, I will be paring it down a bit ASAP.

Also, don’t expect this to work on mobile anything. It will render on desktops with relatively modern GPUs and it is quite speedy on my M1 Pro, but realistically it is hardly optimized.

The graph is updated automatically, and I will be playing with it (as time permits) to see if I can improve navigation and interactivity (right now it’s just plain visualization).

Oct 8^th 2022 · 8 min read

On Mind Mapping

I don’t think of myself as particularly organized, but I do like to prioritize and reason things out both visually and conceptually with at least a modicum of order, and over the years the only approach (other than notetaking) that has consistently stuck with me is mind mapping.

Oct 5^th 2022 · 2 min read

TIL - Octoprint for Android Works On Doorstops

I was combing through a set of To Dos when I came across a note to try Octoprint for Android, which is an Android application that wraps a Linux userland where it installs OctoPrint, allowing it to run on pretty much any old Android phone you might have lying around.

And since Raspberry Pis are still impossible to get hold of (even though I still have spares), I thought it worthwhile to investigate whether it would run on an old Android set-top-box I have been meaning to re-purpose, the Minix Neo X5 mini:

it looked awesome in 2013 — Those two USB-A ports are much less hassle than an OTG adapter.

The thing runs Android Kitkat (4.4), which means that by today’s standards it’s barely more useful than a doorstop.

I kept a couple around because it has both Ethernet and optical audio out, which I considered using for room speakers a long while ago, but also for nostalgia.

These were the boxes we used for the Codebits 2014 digital signage system, and at the time they had very decent WebKit graphics acceleration and were able to render all sorts of things on top of live video, although its paltry dual core CPU can barely handle doing anything of substance:

An early performance stress test. You can see the final results on Flickr.

But what about Octoprint for Android, then?

Well, in the limited testing I could do (my Prusa now runs Klipper, and although Octo4a seems to support it, I only found out about that after unplugging it), it works pretty great - once it gets started, mind you.

You can set it to run automatically when your device powers on, and you get the full Octoprint experience over the browser: remote printing, G-code viewing, printer monitoring, and (if you’re using a phone) access to the camera to view your printer remotely.

There are two caveats worth keeping in mind, though:

Although it downloads and installs the latest Octoprint, the sandbox poses some challenges, so depending on your device the USB serial tricks it does might not work well, and you might need to hunt around for it on your network since it lacks the ability to do Bonjour announcements.
On low-end hardware like the Minix it can take the better part of three minutes to be available over the web.

But if you’re in a pinch and really want to get your 3D printer online on your LAN (some people don’t, although I fail to see the fascination of wrestling with USB pens or memory cards when you can just click on a button), consider giving it a shot.

Oct 1^st 2022 · 15 min read

20 Years of Tao of Mac

By any sort of measure It has now been twenty years since this site has been up, and a lot has changed. When I “launched” this site (a pompous way to say I cobbled it together and got it online), I was in telecoms and trying to escape the world of corporate IT by using the Mac as a UNIX workstation at home¹.

Sep 29^th 2022 · 1 min read

Google is shutting down Stadia ※

…surprising no-one.

Sep 29^th 2022 · 1 min read

Testing an Apple Watch Ultra in the Scottish Highlands ※

Awesome landscape shots, and a pretty good review of the Ultra in real life conditions, but I cannot help but remember Garmin’s quip about measuring battery life in months, not hours.

I actually expected the Ultra to last three days without a charge, because that is what I would want if I went on a weekend trip. But no, in low power mode, after nearly 12 hours of trekking, it was at 47%, and only 10% above the Series 7 (although it would still arguably be doing more, at least mapping-wise).

It can probably last an entire weekend in “regular” use, but as someone who charges their current Apple watch during breakfast and otherwise wears it 24/7, what I want is a watch that lasts me at least an entire week in regular use, and 2-3 days in “outdoor” mode.

Not that I’ve been truly outdoors in the past few years, sadly (only a few times to the countryside, and even then to places where I would have power), but the expectation is still there, and seemingly remains unfulfilled.

Sep 28^th 2022 · 1 min read

Amazon’s Kindle Scribe ※

Not really sure about this one.

I love my Kindle, but I don’t need it to be larger, and even though I do use Scribble and occasionally annotate stuff on my iPad, those feel like marginally useful things at best. More to the point, I haven’t taken significant amounts of handwritten notes in years (and even then I used pocket notebooks).

(Pen input, or at least graphical notetaking, does make sense for languages with complex scripts, but I am considering my own use cases here.)

Also, paying an extra $30 for the “premium” of having an “eraser” on the pen seems… exactly the kind of penny-pinching Amazon would do to differentiate SKUs.

Time will tell if it is really a Remarkable competitor, although I should say that it inspires me with a lot more confidence than the vast majority of e-ink tablets – because like it or not (and even being wary of its software failings), the Kindle hardware and its store are known quantities that have been around for 14 years.

Sep 24^th 2022 · 1 min read

Sunset

Sep 21^st 2022 · 1 min read

NVIDIA GeForce RTX 40 Series GPUs ※

Well, at nearly €2.000 a pop and almost needing a pocket nuclear reactor for power, I would say NVIDIA does not see gaming PCs as a sizable target market anymore.

With the Ethereum merge finally done and GPU mining drying up, I’d say they are just maintaining the “top dog” status quo on the PC market and aiming for specialist applications now, while keeping their 30xx range still available.

(I also expect some datacenter SKUs to come out of this, since, to be honest, that is where the real money is for mass GPU sales now, but that’s kind of besides the point here.)

As to my personal plans of building a workstation PC with a GPU for ML stuff, some gamedev and occasional AAA gaming, well… I guess I will keep playing on the Xbox and use my M1 Pro to run PyTorch.

After all, I can now get a compact, Ryzen 9 5900HX system for around €900 if I really need a powerful multicore PC, and that comes with good enough integrated graphics to play some games without turning into a space heater.

Update: And, as it turns out, there is also a new RTX 6000 series workstation card with 48GB that seems two to three times as expensive, so they are definitely going for the high end…

Sep 19^th 2022 · 1 min read

The iPhone 14 Is Much Easier To Repair ※

This is very, very good. In short, the internals were completely redesigned to make it easy to repair from both the front and the back, which puts an end to almost a decade of nearly inscrutable hardware assemblies.

I love it. It completely flips the usual narrative of right to repair on its head, and puts the ball squarely on its competitors’ camp.

And am going to love it even more next year when it comes with an USB-C port.

Archives • 3D Site Map