A Diffuse Return

Progress over has been slow, but I’ve progressively been feeling well enough to get back to exercising and taking a stab at my hobbies again (work is another matter–I’ve been ramping up, but Thanksgiving holiday has slowed down everything).

And one of the things that has been on my mind is the recent release of Stable Diffusion 2.0, partly because it intersects with ML (which I wish I was doing more of, not necessarily of that kind), hardware (I keep wishing I had a beefy desktop machine with a hardware GPU, although and my can both run Stable Diffusion models just fine), and fun (because I had ).

So I decided to do the kind of thing I usually do–I added another angle to this, which was automated deployment, and built an Azure template to deploy Static Diffusion on my own GPU instance.

An accurate rendition of how my head felt while I was doing this, courtesy of sinus troubles.

Alternatives

Like everyone else, I’ve been using a mix of free Google Colab workbooks (which have an annoying set up time, even though most of them will cache data on Google Drive to speed up repeated invocations), my (where I’ve been using either imaginAIry for Stable Diffusion 2.0 or DiffusionBee for older versions), and even my with Draw Things–which works surprisingly well, although it does burn down the battery.

imaginAIry, in particular, appeals to me because it makes no attempt to hide the internals and exposes a nice API, so it’s my favorite for reproducible results.

But I also wanted to get a feel for how people were building services around this and needed an “easy” thing to get back up to speed, so I decided to host my own1.

Picking a VM

The thing is, GPU-enabled instances don’t come cheap. I already went down this track back in 2018 when I fiddled with , and the only thing I didn’t do at the time was play around with spot pricing.

I have been using spot instances for my Kubernetes clusters, so I started by checking out spot pricing for a couple of interesting SKUs:

Looking for the cheapest price on the GPU instances I was interested in.

US $0.114/hour is really not bad considering that the retail price for a Standard_NV6 (which is exactly the same instance type I used in 2018) is, as of this writing, exactly ten times that amount, so that seemed OK. The additional surcharges for a standard SSD volume (a tiny one) and networking are residual, so I can certainly fit this into my personal budget for only a few hours of fooling around each month.

And getting the machine preempted would not be a big deal (you can also look up eviction rates, and they’re currently at 0-5%).

NVIDIA Driver Setup

Next, I went and hacked down my k3s template to deploy a single machine and started to figure out how to set up NVIDIA drivers in Linux. Given the hassle involved, I decided to go with 20.04 (which is still LTS, and has a couple more years of testing).

However, first I had to make sure I was picking the right image. And in case you’ve been living under a rock or are still doing Stone Age style manual provisioning in the Azure portal, I’m one of those people who only do automated deployments, so when I rewrote the Azure template I had to account for the fact that the Standard_NV6 uses an older generation of OS images (for a different hypervisor baseline):

    "imageReference": {
        "publisher": "Canonical",
        "offer": "0001-com-ubuntu-server-focal",
        "sku": "[if(contains(parameters('instanceSize'),'Standard_NV6'),'20_04-lts','20_04-lts-gen2')]",
        "version": "latest"
    },

Note that the if conditional above may seem lazy because I’m using contains, but that’s because I have also been playing around with the Standard_NV6_Promo SKU.

The next step was getting the NVIDIA drivers to actually install. And that turned out (after a couple hours of investigation and a couple of tests) to be as easy as making sure my cloud-init script did two things:

  • Installed ubuntu-drivers-common and nvidia-cuda-toolkit
  • Ran ubuntu-drivers autoinstall as part of my “preflight” script to set up the environment.

This last command does all the magic to get the kernel set up as well, although there is a caveat–you’ll need a reboot to actually be able to install some PyTorch dependencies (especially those that are recompiled upon install and need nvidia-smi and the like to have valid output, which only happens if the drivers are loaded).

But after a reboot, it works just fine:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla M60           Off  | 00000001:00:00.0 Off |                  Off |
| N/A   47C    P0    38W / 150W |   2505MiB /  8192MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      3766      C   python3                          2502MiB |
+-----------------------------------------------------------------------------+

I’ve yet to test this on Ubuntu 22.04, but I suspect it will work just the same (I’m quite curious as to how docker-nvidia will work, although I didn’t need it for this).

And, of course, in the real world I would likely automate a pre-baked OS image so I could have fully reproducible deployments.

But one thing I made sure to set up already (besides adding Tailscale for easier remote access) was configuring the VM for automated shutdown at dinner time. You can do that via the portal in a few clicks, and I will be adding it to my template later.

After all, you absolutely must set guardrails on how you use any kind of cloud resource… Especially if you’re experimenting–that’s usually when things get out of control.

Picking a Version

I started out using imaginAIry and JupyterLab for Stable Diffusion 2.0, but I quickly realized that the new training set used for 2.0 is vastly different and omits most of the .

So I parked that for a while and decided to revert to 1.5 and the AUTOMATIC1111 web UI (which also supports 2.0 somewhat).

And I’m not at all sad about missing out on 2.0 for now, especially considering I can still get this kind of output out of 1.5:

Ooooh, shiny!

  1. There was also a serendipitous coincidence in that I was , one of which also included setting up a GPU instance, so this was a good way to keep the momentum going once I clocked off work. ↩︎

TIL - Minimal Go WebDAV server

I had an interesting little conundrum the other day, which was that I needed to help someone access a remote Linux filesystem via Windows Explorer (long story…) but had no direct connectivity, no way to use network filesystems, no way to use VS Code remote editing (I was going through a bastion) nor an SSH client that would do sshfs.

I could, however, set up a TCP tunnel, so my initial reaction was to go and check if there were any single-file servers in , but then I realized that has a perfectly serviceable module and hacked this together:

package main

import (
    "flag"
    "golang.org/x/net/webdav"
    "net/http"
)

func main() {
    var address string
    flag.StringVar(&address, "a", "localhost:8080", "Address to listen to.")
    flag.Parse()

    handler := &webdav.Handler{
        FileSystem: webdav.Dir("."),
        LockSystem: webdav.NewMemLS(),
    }

    http.ListenAndServe(address, handler)
}

With 1.13 (which is what I had at hand), you need to do:

go get golang.org/x/net/webdav
GOOS="linux" GOARCH="amd64" go build -o webdav main.go

So I copied the binary across, typed http://hostname:port into Windows Explorer, and everything worked: uploading large files, creating directories, editing configurations, etc.

Of course I had to tidy up permissions here and there, but overall, it “just worked”.

A Week Later

It’s been over and we’re all still testing positive for COVID, plus a few random things have happened.

Read More...

COVID Update

So it finally happened. Nearly three years in, and (coincidentally or not) just after my , I got COVID, and so did all of my family, which I guess qualifies this as the latest post in .

Read More...

The Elon Gambit

Well, that was . I was originally planning to clean up another of my long-standing drafts and post about one of my projects, but in between and the chaos at , I thought I’d hop on the bandwagon and write about that as well.

Read More...

WeMeet Europe 2022

I went out to Porto to speak about Microsoft’s strategy for the telco market, and it was nice (but weird) to stand on a stage again after three years.

Homelab Update

Every now and then I spend an hour or so in the evenings poking at my , which kind of adds up into actual productivity over a couple of months.

Read More...

TIL - VIM spelling and plugin interaction

After months of writing, I realized that I was using spell checking all wrong, which led to quite a few typos making their way online.

The reason for that was that I had added a few plugins (like vimwiki) that overrode formatting and highlighting settings, so a lot of my typos were actually there and flagged by the spell checker, just not visibly highlighted.

So I went and moved my Markdown and spell checking settings after all the other plugins and color settings were loaded, and they now look like this:

"--- Writing ---
Plug 'tpope/vim-markdown'
" enable fenced code highlighting
let g:markdown_fenced_languages = ['html', 'python', 'bash=sh', 'javascript']
set spell spelllang=en_us
hi SpellBad term=standout cterm=standout,underline gui=underline

The last line is the critical thing, but I would rather manage settings as a block for each use case, and I’m somewhat annoyed that this has been going on for so long.

Fast Remote Desktop on a Raspberry Pi, with Glamor and an iGPU

If anyone were to tell me that I would be hacking X11 server modules in 2022, I’d say they were nuts. And yet, here we are.

Read More...

30 Years of Vodafone Portugal

The three (former and current) CEOs, all of whom I’m very proud to have worked with.

Seven Years At Microsoft

I’ts now been since I joined, and since I moved from what is now Industry Solutions to Azure for Operators.

Read More...

On Mind Mapping

I don’t think of myself as particularly organized, but I do like to prioritize and reason things out both visually and conceptually with at least a modicum of order, and over the years the only approach (other than notetaking) that has consistently stuck with me is mind mapping.

Read More...

TIL - Octoprint for Android Works On Doorstops

I was combing through a set of To Dos when I came across a note to try Octoprint for Android, which is an Android application that wraps a Linux userland where it installs OctoPrint, allowing it to run on pretty much any old Android phone you might have lying around.

And since are still impossible to get hold of (even though I still have spares), I thought it worthwhile to investigate whether it would run on an old Android set-top-box I have been meaning to re-purpose, the Minix Neo X5 mini:

Those two USB-A ports are much less hassle than an OTG adapter.

The thing runs Android Kitkat (4.4), which means that by today’s standards it’s barely more useful than a doorstop.

I kept a couple around because it has both Ethernet and optical audio out, which I considered using for room speakers a long while ago, but also for nostalgia.

These were the boxes we used for the Codebits 2014 digital signage system, and at the time they had very decent WebKit graphics acceleration and were able to render all sorts of things on top of live video, although its paltry dual core CPU can barely handle doing anything of substance:

An early performance stress test. You can see the final results on Flickr.

But what about Octoprint for Android, then?

Well, in the limited testing I could do (my now runs Klipper, and although Octo4a seems to support it, I only found out about that after unplugging it), it works pretty great - once it gets started, mind you.

You can set it to run automatically when your device powers on, and you get the full Octoprint experience over the browser: remote printing, G-code viewing, printer monitoring, and (if you’re using a phone) access to the camera to view your printer remotely.

There are two caveats worth keeping in mind, though:

  • Although it downloads and installs the latest Octoprint, the sandbox poses some challenges, so depending on your device the USB serial tricks it does might not work well, and you might need to hunt around for it on your network since it lacks the ability to do Bonjour announcements.
  • On low-end hardware like the Minix it can take the better part of three minutes to be available over the web.

But if you’re in a pinch and really want to get your 3D printer online on your LAN (some people don’t, although I fail to see the fascination of wrestling with USB pens or memory cards when you can just click on a button), consider giving it a shot.

20 Years of Tao of Mac

By It has now been twenty years since this site has been up, and a lot has changed. When I “launched” this site (a pompous way to say I cobbled it together and got it online), I was in telecoms and trying to escape the world of corporate IT by using the as a UNIX workstation at home1.

Read More...

Sunset

I used to arrive at this very place about this time of day, some 20-odd years ago.
Archives3D Site Map