VPN
service that provides seamless connectivity across your devices no matter where they reside, with a few interesting features:
I have been using it for a couple of years as a way to remotely access my home machines (and other stuff) when traveling, and have been very happy with it.
Category | Date | Link | Notes |
---|---|---|---|
Server | 2024 | headscale | a self-hosted Tailscale control server |
UI | trayscale | a system tray icon for Linux desktops |
|
tailscale-systray | another system tray icon for Linux desktops |
I have a few devices running it, including a pair of NEXX WT3020s that are still, for my money, the nicest low-power travel routers ever made, being able to run off a tiny power pack.
192.168.1.2
is a common default IP address for the initial setup client device, and the CIDR
range for that is overused (I typically change it to another subnet range).I am so, but so sorry to see this happen. I paid my way through college doing design, but decided I would never again use Adobe software after Creative Cloud became the mess it is today.
Even though I don’t do graphical design as part of my profession (well, not explicitly at least), I willingly paid for Affinity to sponsor them as a viable alternative to Adobe.
And it sort of worked (until it backfired today): The Affinity suite is (for the moment) good quality native Mac software that does not rely on cloud features nor has a subscription model.
As much as their FAQ claims that will not change, I think we’ve all seen this before–in short, I don’t trust Canva one whit and fully expect to revisit this post in a year when Serif/Affinity breaks one of those three tenets above and forces me to move away from their software.
The gist of the matter is that more stuff has happened, I am not at all comfortable with it, and for a number of reasons, I’ve had to put all my AMD
and ARM
-related AI stuff on hold for at least two more weeks–perhaps even for an extended period of time.
That also meant pulling on hold a RK3588
board that I was getting some great results out of (it’s the Banana Pi M7).
And with Easter coming, I’m likely just going to give up and try to catch up on my reading (or doing something more prosaic like modeling some 3D printed MIDI
adapters).
But since I can’t sit around doing nothing, I’ve pivoted to ESP32
and electronics. One of the things I’m trying to figure out is why my ESP32
cameras, which are based on a fork of this code, can actually stream to HomeKit when a freshly built image (using PlatformIO) just crash.
I can extract the old firmware, copy it across and get a working camera:
# Copy original image
esptool.py -b 115200 --port /dev/ttyUSB0 read_flash 0x00000 0x400000 flash_4M.bin
# Flash on new camera
esptool.py -b 115200 --port /dev/ttyUSB0 write_flash --flash_freq 80m 0x000000 flash_4M.bin
# Reset Homekit settings
esptool.py -b 115200 --port /dev/ttyUSB0 erase_region 0x3a0000 4096
…but somehow new builds are unstable. I am poring over my archives and most online forks to try to divine what I did back then, but this just goes to show that if you want stable firmware for anything, you really need to vendor all of your dependencies (which I didn’t at the time).
I’m really close to getting Ghidra out to just dive into the firmware and see what’s going on, but I need better hardware.
Because, you see, to add to the frustration, I have spent the past couple of days using an Intel MacBook Pro with a Touch Bar. I honestly had forgotten how bad this thing was. Although it is still quite serviceable and can run most of what I need, this is the kind of experience that proves that (short-term) retro computing is not a nostalgia trip.
Since I’ve been testing a Keychron K7 Max with my iPad (more on that later), the MacBook Pro keyboard’s stiffness and the continuous frustration at failing to hit the Esc
key in any meaningful way have been driving me insane.
Also, I am again acutely aware of battery life. How blissfully ignorant of past foibles we can be until we are thrown back into it.
As such, I’ve decided to put up a list of resources that might be useful to those who are considering alternatives or need to migrate away from Redis while retaining compatibility with the protocol:
Category | Date | Link | Notes |
---|---|---|---|
Alternatives | 2024 | garnet | a remote cache-store from Microsoft Research that offers strong performance (throughput and latency), scalability, storage, recovery, cluster sharding, key migration, and replication features. |
Examples | miniredis | my minimal Redis server implementation in Python |
|
Forks | redict | A more opinionated fork |
|
placeholderkv | What seems to be the main fork of the original project |
Instabuy. I’ve been a long-term PICO-8 fan, and this, even in such an early stage, is simply sublime.
I can’t wait to see what people come up with. I’m a bit worried that one of my kids joked that he wanted to do a PowerPoint clone, though.
The only flaw I can see is that it’s not available for ARM
Linux yet (so I can’t run it on a Pi or any of my development boards), but I’m sure that will be fixed soon enough.
I’ve read through a bunch of articles on this and a fair chunk of the filing, and it seems like the DOJ doesn’t really have that much of a case here. Their accusations are so broad and have so many holes (well, most of the technical ones, and except some of the paywalls) that it’s almost as if they wanted to make the European Commission look good by comparison.
Or maybe, just maybe, someone had to justify their job. Stranger things have happened.
Archive link, HN Discussion, John Gruber.
I watched this the other day and it was simply sublime. My own attempt at doing something similar (but far less realistic) has been languishing for the better part of two years, and it may take another two until I even print the first parts…
5700U
-based laptop for almost three years now and it’s been a great experience, but I wanted to explore the desktop side of things, especially since my meanderings into edge computing have shown that ARM
-based servers are at an interesting inflection point for running small AI models, and I just had no real data on AMD
‘s capabilities in that space.
Ryzen-based mini-PCs have been around for a while, but the previous generation of mobile processors (like the 4700U
and 5700U
) never quite managed to get traction in that space. Now, with Intel leaving the NUC
segment to its licensees and a resurgence of interest in small, powerful desktops with good integrated GPU
s (thanks in large part to the Steam Deck and its custom APU
), there’s a lot of possibilities to explore.
In short, I was looking for something that had a better iGPU
and could handle virtualization workloads without excessive power consumption, since my experiences with borg
and the U59 has shown me that Intel CPUs have spread out into extremes where they either consume too much power for the performance they deliver or are just built too cheaply.
So when I stumbled upon the AceMagic AM18 and saw that it was based on the 7840HS
–a 4nm process, 8C/16T processor that can go up to 5.1GHz–I decided to give it a try.
Disclaimer: AceMagic sent me a review unit (for which I thank them), and this article follows my Review Policy. Also, as usual, I purposely avoid doing any detailed price comparisons.
However, with ollama
starting to support AMD graphics this week and my having some trouble in (and lack of time for) dealing with ROCm
and ZLUDA
, I decided to split this review in two:
ROCm
) will circle back to the AI and homelab side of thingsSo let’s get started.
The AceMagic AM18 is, as you would expect, nice and compact, following the usual NUC
-like flattened cube form factor with ports on the front and back. It’s a bit larger than a NUC
and has a large cooling fan on top, but it’s small enough to fit under a monitor or under a TV (which is actually where I kept it the week it arrived).
The model I received had the following specs:
DDR5
RAM
(in a nice dual-channel configuration using Crucial CT16G56C46S5.M8D1
SODIMM
s, rated for 5600MT/s, and, interestingly enough, mentioned by Crucial as having on-die ECC
support)NVME
SSD (in a PCIe 4.0 slot) that identifies itself as a RS512GSSD710
from TenaFe and (unusually for mini-pcs) has a built-in temperature sensor.USB-A
3.0 ports (two in the front, two in the back. Confusingly, the back ones are black and the front ones are blue)USB-C
in the front, HDMI
and DisplayPort
in the back–I would have preferred two HDMI
ports, but they are all rated for 8K@60Hz, and use the amusingly named “Pink Sardine” AMD
chipset.)RTL8852BE
Wi-Fi 6/Bluetooth 5.2, which sits underneath the SSD
.USB-C
100W PSU
, rated at 20V/5A.The front also has a TTRS
audio jack and a power button. There are no Kensington lock slots, which might deter business users, but the machine is small enough to be easily hidden away.
I also got a VESA
mount, display cables, etc., but, interestingly, no SATA
cables or brackets–the reason for that only became clear when I opened up the machine:
The machine is relatively easy to open for cleaning and expansion–the top four decorative screws provide easy access to the fan for cleaning (but not removal), whereas (sadly) you need to remove the rubber feet to get at the bottom screws that allow you to pop the bottom cover.
Once you remove the bottom, it’s easy to see why there is no SATA
cable supplied–the single NVME
slot is all that you get for storage expansion, and even though the plastic is moulded to hold a 2.5” drive, the bottom fan is mounted in that slot.
It’s not necessary to remove the internal bracket to access the Wi-Fi card, but I did it anyway, and it revealed the CMOS
battery, a UART
header tucked near the USB
ports (which seems really interesting to play with later) and a (pinless) debug header, plus a red CMOS
clearing jumper and assorted wiring.
Before you ask (since I have purposefully delayed mentioning it), the RGB
lighting can be controlled only via the front button, and can fortunately be turned off entirely.
I did find the white setting to quite useful when trying to work around the machine under my TV, and I suspect it will come in even handier when I move it to my server closet.
BIOS
SettingsThe machine arrived with a basic Windows 10 Home install that has been customized to (at least) include some chipset drivers and Google Chrome:
I am not a fan of anything being added to the Windows images except drivers (and AceMagic has recently had to change their base images), but I understand that there are markets where this is a requirement or preference. I did not find anything wrong with the install, but since I need at least Windows Pro for working and my plan was to use it as a Linux machine, I wiped the install and set up Bazzite on the internal SSD
, and then later replaced the internal SSD
with a 1TB one to run Proxmox (which I will write about in a follow-up post).
I did, however, take a look at the BIOS
settings, and found them to be quite comprehensive:
Things I liked to see, and intend to use in a server configuration:
PXE
bootIOMMU
and PCIe
featuresUnlike my 5700U
machine, I did’t see any obvious way to pre-allocate RAM to the iGPU
, but I suspect that’s just because it is a different chipset.
If you’re curious, I have an annex with the output from sensors
and lshw
that I am using as a basis for tweaking the Proxmox setup I’ll be posting about in the second part of this review.
Since the machine arrived on a day when I had not a whit of clear desk space in my office on account of my testing another 3D printer it went directly to my living room, where I plugged it into a power monitoring socket next to my TV.
Linux setup was as easy as installing the bazzite-gnome-deck
spin of Bazzite, rebooting, and then setting the resolution to 1080p–and after going through the initial install wizard and configuring Steam, it felt pretty much like any game console would.
A pleasant surprise was that the M780
iGPU
detected my TV without any issues–it is a relatively old LG
that goes up to 4K 120Hz with HDR
support, and I was able to set it at anywhere between 1080p/60Hz to 4K/120Hz without any issues.
I eventually trimmed it down to 1080p/120Hz to make sure I could get a decent frame rate–the AMD
Radeon M780
iGPU
can handle the full resolution for regular desktop work, but there was no point in harming the gaming experience1.
I then paired an Xbox controller via Bluetooth, and started downloading my Steam library.
That helped me get an early start on testing power consumption and Wi-Fi speeds–impressively, it was able to download my Steam library at a peak 480Mbps, which is only slightly less what my streaming server gets on a wired Gigabit connection–and I don’t have Wi-Fi 6 in my house yet:
The power consumption was also quite good, and I was able to measure it at various loads:
Operating System | Power (W) | Notes |
---|---|---|
Bazzite (Fedora 39) | 60.3 | Gaming (Steam, average over the games listed) |
50.9 | Steam downloads over Wi-Fi |
|
45.6 | Gaming (Emulation) |
|
21.7 | Booting |
|
11.2 | Idle, TV/display on |
|
0.1 | Suspend |
This will, of course, be expanded upon in the server part of this review, but I was quite pleased to see that the machine was able to idle at around 12W and consume less than 100W under heavy load.
As to noise, and being extraordinarily fan-averse to the point of preferring streaming over local gaming, I was quite pleased to find that the fan was generally barely audible even when gaming, and I could only notice it for a few seconds at a time when under heavy load.
But I did place the machine squarely under the center speaker of my audio setup, so it was a bit of a moot point–I suspect it would be more noticeable (and annoying) if I had it right beside me–but I will eventually do more detailed measurements when I get around to posting the server part of this review.
The good thing is that tweaking the fan curves in the BIOS
is a feasible option on this machine–plus the sheer size of the top fan (in mini-PC terms, it’s pretty massive) means that it is relatively quiet and completely non-whiny even when running at full speed.
Throughout that first week I installed a few games and emulators to test over the evenings, and was able to play a few hours of Halo Infinite, Borderlands 3, Hades and Stray without any issues.
I also synced my RetroDECK setup and experimented with a few emulators, and everything worked fine other than dual control presses in the 8bitdo Ultimate Controller I tried to use for a particular game–but that’s just the combination of a couple of rough edges in Bazzite and RetroDECK.
As you’d expect, everything flagged as Steam compatible “just worked”. I only had trouble with Horizon: Zero Dawn, which is known to be fickle and does work on my streaming setup, but I suspect it’s just a matter of getting a few more Proton/AMD
updates down the line:
I also emulated that which cannot be named anymore on the very evening it was rendered extinct, and it was… glorious, both in 1080p and in 4K (although I think the M780
is smack in the middle between smooth 1080p/30fps and 4K/30fps). Even without any immediate way to tweak the frame rate, I suspect 1080p/60fps is well within reach.
I will be doing more detailed benchmarking in the follow-up post, but I did want to run a throughput test on the shipping PCIe
4.0 SSD
for later comparison:
# fio --filename=/<path>/file --size=5GB --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth=64 --runtime=120 --numjobs=4 --time_based --group_reporting --name=random-read-write --eta-newline=1
random-read-write: (groupid=0, jobs=4): err= 0: pid=7251: Sun Mar 17 18:20:12 2024
read: IOPS=115k, BW=7218MiB/s (7568MB/s)(846GiB/120001msec)
slat (usec): min=4, max=2551, avg= 9.39, stdev= 2.54
clat (usec): min=10, max=4139, avg=1091.34, stdev=72.41
lat (usec): min=17, max=4155, avg=1100.73, stdev=72.50
clat percentiles (usec):
| 1.00th=[ 938], 5.00th=[ 979], 10.00th=[ 1004], 20.00th=[ 1037],
| 30.00th=[ 1057], 40.00th=[ 1074], 50.00th=[ 1090], 60.00th=[ 1106],
| 70.00th=[ 1123], 80.00th=[ 1156], 90.00th=[ 1188], 95.00th=[ 1205],
| 99.00th=[ 1270], 99.50th=[ 1287], 99.90th=[ 1418], 99.95th=[ 1532],
| 99.99th=[ 2057]
bw ( MiB/s): min= 7004, max= 7437, per=100.00%, avg=7221.87, stdev=17.80, samples=956
iops : min=112078, max=119006, avg=115549.98, stdev=284.76, samples=956
write: IOPS=115k, BW=7215MiB/s (7565MB/s)(845GiB/120001msec); 0 zone resets
slat (usec): min=6, max=2872, avg=23.47, stdev= 6.27
clat (nsec): min=1272, max=4136.1k, avg=1092150.09, stdev=72495.42
lat (usec): min=9, max=4156, avg=1115.62, stdev=72.91
clat percentiles (usec):
| 1.00th=[ 938], 5.00th=[ 979], 10.00th=[ 1004], 20.00th=[ 1037],
| 30.00th=[ 1057], 40.00th=[ 1074], 50.00th=[ 1090], 60.00th=[ 1106],
| 70.00th=[ 1123], 80.00th=[ 1156], 90.00th=[ 1188], 95.00th=[ 1205],
| 99.00th=[ 1270], 99.50th=[ 1287], 99.90th=[ 1418], 99.95th=[ 1532],
| 99.99th=[ 2057]
bw ( MiB/s): min= 7014, max= 7312, per=100.00%, avg=7218.74, stdev= 9.64, samples=956
iops : min=112230, max=117002, avg=115499.83, stdev=154.17, samples=956
lat (usec) : 2=0.01%, 20=0.01%, 50=0.01%, 100=0.01%, 250=0.01%
lat (usec) : 500=0.01%, 750=0.01%, 1000=8.92%
lat (msec) : 2=91.07%, 4=0.01%, 10=0.01%
cpu : usr=9.18%, sys=90.07%, ctx=11180, majf=0, minf=54
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued rwts: total=13858307,13852536,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: bw=7218MiB/s (7568MB/s), 7218MiB/s-7218MiB/s (7568MB/s-7568MB/s), io=846GiB (908GB), run=120001-120001msec
WRITE: bw=7215MiB/s (7565MB/s), 7215MiB/s-7215MiB/s (7565MB/s-7565MB/s), io=845GiB (908GB), run=120001-120001msec
That’s a whopping 115k IOPS
, which is very impressive for a vendor-supplied SSD
and should be more than enough for most workloads. It might even be slower than the one I got for testing Proxmox… but we’ll see.
From a gaming perspective (and from someone who just can’t retire from gaming), the AceMagic AM18 is very solid, and as a first hands-on experience with a 7840HS
-based machine, it’s quite impressive in terms of performance and power consumption.
This becomes ever more apparent if I compare it with my NVIDIA RTX 3060-based streaming setup, which is nominally more powerful and acting as a streaming server but, in the end, provides a similar gaming experience on my TV at the expense of a lot more power (and heat).
And yes, the AceMagic AM18 can be a Steam/Moonlight server as well–I tried it out of curiosity and it worked fine on lighter games, but the reason I wanted to test it was really to understand the limits of the AMD
APU
architecture, and I’m trying really hard not to go down any more rabbit holes right now.
But if you came here looking for a small, powerful, and quiet desktop that can, perhaps, replace your conventional console, I’d have to say maybe. And that’s simply because comparing the AceMagic AM18 directly with a console is a bit unfair in either direction.
If you’re still looking for a bias on my part instead of a fully formulated opinion, then I’d say that spec-wise it is a good match for the current generation of consoles if you don’t care about AAA 4K gaming or console exclusives2.
And I’d leave it at that until Proton gets more optimizations for this new crop of Ryzen APU
s.
But at this power envelope, I’m really impressed. It’s not Apple Silicon, but definitely the next best thing in terms of average performance per watt.
The only teething issue I had was apparent lack of HDMI-CEC
support–I do have an HDMI
switch in between that also has other (working) devices plugged into it, and I still had to fiddle with buttons to switch inputs. I suspect that’s a combination of the M780
iGPU
and Bazzite, and there were no obvious BIOS
settings to tweak. ↩︎
And if I think about it, the idea of replacing a console with a Steam-based home theatre PC is something I might do myself some day. ↩︎
Again, like last year, weekly notes were a way to keep my mind off my work situation, keep track of progress on projects, and to generally refer back to later when I was looking for a specific thing (like some of the scripts I hacked together, which are easier to find in context).
And since I really don’t like using most of the current set of note taking applications full of clutter (ahem Obsidian) and vastly prefer having them public, they are very, very easy to maintain.
But the optimistic mood they helped sustain was broken this week by yet another corporate re-org, and even having fully recovered from spending most of last year in a state of shock at the layoffs–or perhaps especially because of that, I decided to accelerate things this time around and do something else for a change. Literally.
I’ve been slowly gravitating towards smaller and smaller hardware over time, and although my homelab is still in an expansion phase, ARM
nodes already outnumber Intel nodes on my Proxmox cluster.
Low power, high performance hardware with a very tight power envelope is something that really appeals to me, and even the code I’ve written (or tweaked) has been going lower and lower down the stack.
Add to that well over a decade of doing analytics and “traditional” AI, plus the LLM
hype fest, and everything’s pretty much converging towards my spending more and more “free” time (as well as my independently advisory hours, which are another thing I don’t write about) around those topics.
It’s very early days, but yes, edge LLM
s (and, let’s be honest, more useful models) are feasible now within very specific parameters, and that’s something I’d like to pursue–likely with a few detours into other hardware-centric projects, since I have literally dozens of small MCU
boards to play with.
Besides, amid all the insanity of web technology, it just feels nice to write C++
again, even if I’m slower at it.
Regular readers have noticed that I’ve been doing more and more product reviews. That’s always been something I enjoyed doing because a) I greatly enjoy writing and b) the object or software under review provides me with a focal point around which I can wrap not just my (sometimes extreme) curiosity and my (definitely not neurotypical) attention to detail.
There will be a few more reviews coming (there’s a software/music one that I have as yet failed to complete due to lack of proper inspiration, for instance), and derivative pieces as I dig further into things.
I have no intent to let those take over the site, or even turn a profit (that’s the gist of the policy), but they did take over my notes in the sense that they provided a bit of extra escapism from work and afforded me the ability to accelerate a couple of my personal projects–and that’s the kind of balance between research, tinkering and writing that I think I need to get over the upcoming months.
This page is a reference table for resources I’m keeping tabs on.
That grand, spacious (and long ignored) branch of computing that, these days, deals not just with thinking machines (so-called “hard” AI), but mostly with enhancing the usefulness of machines in general by exploring mechanisms to express and manipulate harvested knowledge.
Or just pretty pictures and weird chatbots.
Field | Category | Date | Link | Notes |
---|---|---|---|---|
General | Examples | 2024-03 | flash-attention-minimal | a minimal implementation of Flash Attention |
Frameworks | 2023-12 | MLX | An array framework for Apple silicon |
|
mlx-examples | MLX examples |
|||
2023-05 | marvin | A generic wrapper for various AI APIs |
||
Jupyter | 2023-04 | jupyter-ai | an official Jupyter plugin that can handle multiple AI back-ends (although it seems less flexible than the others right now) |
|
Libraries | 2023-06 | unstructured | a library for handling and segmenting unstructured data of various kinds, from text to common file formats |
|
2023-03 | ml-ane-transformers | Apple’s transformers library, optimized for the Neural Engine |
||
2009-06 | Alchemy | A toolkit providing a series of algorithms for statistical relational learning and probabilistic logic inference, based on Markov logic representation |
||
Techniques | 2022-12 | awesome-chatgpt-prompts | might be a short-lived resource, but an interesting one |
|
Tools | 2024-01 | pico-tflmicro | a port of TensorFlow Lite Micro to the Raspberry Pi Pico |
|
2023-05 | explainerdashboard | a web app that explains the workings of a (scikit-learn compatible) machine learning model |
||
2023-04 | basaran | An Open-Source alternative to the OpenAI text completion API, with a compatible streaming API for privately hosted models. |
||
2023-03 | NVIDIA Triton Inference Server | A high-performance inference server |
||
2022-12 | Project Bumblebee | Pre-trained and transformer neural models in Elixir. |
||
Generative Audio | models | 2023-04 | bark | a text-prompted genereative audio model |
Large Language Models | Applications | 2023-07 | Chie | A cross-platform dekstop application with chat history and extension support |
Copilots | 2023-06 | Obsidian Copilot | an interesting take on how to use semantic search and OpenSearch’s BM25 implementation |
|
Demos | 2024-01 | WhisperFusion | an ensemble setup with WhisperSpeech, WhisperLive and Phi |
|
Frameworks | 2023-11 | Tanuki | yet another LLM framework using decorators for data validation |
|
2023-07 | litellm | a simple, lightweight LLM wrapper |
||
AutoChain | Yet another alternative to langchain |
|||
griptape | a |
|||
llama_index | a data framework for LLM applications |
|||
txtai | has spinoffs for chat, workflows for medical/scientific papers, semantic search for developers and semantic search for headlines and story text |
|||
llmflows | Yet another alternative to langchain, but with an interesting approach at defining workflows |
|||
2023-05 | langchain | a composable approach for building LLM applications |
||
guidance | Control modern language models more effectively and efficiently than traditional prompting or chaining. |
|||
Front-Ends | 2024-01 | jan | an open-source ChatGPT alternative that runs 100% offline (uses nitro) |
|
2023-12 | SecureAI-Tools | a self-hosted local inference front-end for chatting with document collections |
||
gpt4all | another self-hosted local inference front-end |
|||
Jupyter | 2023-04 | LLMBook | A VS Code notebook interface for LLMs |
|
jupytee | a Jupyter plugin that can handle code generation and image generation, but not switching models (GPT-4) |
|||
genai | a Jupyter plugin that can handle code generation and fixes based on tracebacks |
|||
ipython-gpt | a Jupyter plugin that can handle multiple models |
|||
Libraries | 2024-02 | DataDreamer | library for prompting, synthetic data generation, and training workflows |
|
2023-10 | MemGPT | a memory management/summarization technique for unbounded context |
||
2023-09 | instructor | a clever library that simplifies invoking OpenAI function calls |
||
2023-06 | simpleaichat | A simple wrapper for the ChatGPT AI |
||
2023-05 | guardrails | a package for validating and correcting the outputs of large language models |
||
Models | 2024-01 | TinyLlama | pretraining of a 1.1B Llama model on 3 trillion tokens. |
|
2023-12 | ml-ferret | a multi-modal model from Apple |
||
2023-04 | turbopilot | a GitHub CoPilot replacement that can run locally (CPU only) |
||
Reference | 2023-12 | Native JSON Output from GPT-4 | tips on how to use OpenAI JSON and function calling |
|
GPT Prompt Archive | A set of sample base prompts for various LLMs |
|||
2023-03 | Using LLaMA with M1 Mac | Manual instructions for Apple Silicon |
||
Resources | 2023-12 | Prompt Engineering Guide | a set of lecture notes and detailed examples of prompting techniques |
|
promptbase | Another set of prompting techniques and detailed examples |
|||
2023-04 | awesome-decentralized-llm | a collection of LLM resources that operate independently |
||
Samples | 2024-01 | SimpleTinyLlama | a simple PyTorch-based implementation |
|
devlooper | a program synthesis agent that autonomously fixes its output by running tests |
|||
2023-12 | gpt-researcher | an agent that does online research on any given topic |
||
LibreChat | A self-hosted ChatGPT alternative |
|||
sharepoint-indexing-azure-cognitive-search | provides an example of how to use Graph navigation and Cognitive Search indexing |
|||
gpt4all | open-source LLM chatbots |
|||
GPT in 60 Lines of NumPy | a tutorial on how to build a GPT model from scratch |
|||
Bash One-Liners for LLMs | a collection of one-liners for various LLMs |
|||
2023-11 | David Attenborough narrates your life | A pretty hilarious image-to-description example |
||
Demystifying Advanced RAG Pipelines | An LLM-powered advanced RAG pipeline built from scratch |
|||
Wanderlust OpenAI example using Solara | A simple interactive web shell with some nice features |
|||
Tools | 2024-03 | gpt-pilot | a prototype development tool that leverages GPT |
|
R2R | a framework for or rapid development and deployment of production-ready RAG systems with SQLite support |
|||
2024-02 | geppetto | a bot for integrating ChatGPT and DALL-E into Slack |
||
GPTFast | a set of acceleration techniques |
|||
notesollama | a plugin for Apple Notes that uses the Accessibility APIs |
|||
llm-ls | a local language server that leverages LLMs |
|||
llm-vscode | a VSCode extension that uses llm-ls |
|||
crewAI | a framework for orchestrating autonomous AI agents |
|||
llmware | a framework for developing LLM-based applications including Retrieval Augmented Generation |
|||
hqq | an implementation of Half-Quadratic Quantization (HQQ) |
|||
reor | a note taking tool that performs RAG using a local LLM |
|||
NeuralFlow | a Python script for plotting the intermediate layer outputs of Mistral 7B |
|||
2024-01 | ollama-bot | a rudimentary IRC bot that communicates with a local instance of ollama |
||
koboldcpp | nn easy-to-use AI text-generation software for GGML and GGUF models based on llama.cpp |
|||
gguf-tools | a set of tools for manipulating GGUF format files |
|||
nitro | a self-hosted inference engine for edge computing with an OpenAI API |
|||
emacs-copilot | an Emacs extension for using a local LLM |
|||
nlm-ingestor | a set of parsers for common file formats |
|||
lorax | a framework that allows users to serve thousands of fine-tuned models on a single GPU |
|||
privy | An open-source alternative to GitHub copilot that runs locally. |
|||
2023-12 | microagents | an interesting experiment on self-editing agents |
||
BricksLLM | an OpenAI gateway in Go to create API keys with rate limits, cost limits and TTLs |
|||
macOSpilot-ai-assistant | An Electron app for macOS |
|||
TinyChatEngine | A local (edge) inference engine in C++ without any dependencies |
|||
LocalAI | A local, drop-in replacement for the OpenAI API |
|||
Serve | A containerized solution for using local LLMs via web chat |
|||
wyGPT | another C++ local inference tool |
|||
2023-10 | localpilot | a MITM proxy that lets you use the GitHub Copilot extension with other LLMs |
||
2023-09 | embedchain | another framework to create bots from existing datasets |
||
llm_agents | a simplified agent framework (doesn’t use OpenAI functions) |
|||
2023-08 | pykoi | a unified interface for data and feedback collection, including model comparisons |
||
PromptTools | self-hostable toools for evaluating LLMs, vector databases, and prompts |
|||
2023-07 | a1gpt | A C++ implementation of a GPT-2 inference engine |
||
khoj | an intriguing personal assistant based on local data |
|||
promptfoo | A tool for testing and evaluating LLM prompt quality. |
|||
2023-06 | SuperAGI | another AutoGPT-like harness for building GPT agents |
||
2023-05 | ChainForge | a visual programming environment for benchmarking prompts across multiple LLMs |
||
langflow | a node-based GUI for quick iteration of langchain flows |
|||
2023-04 | Auto-GPT | an attempt to provide ChatGPT with a degree of autonomy |
||
2023-03 | dalai | An automated installer for LLaMA |
||
llama.cpp | A C++ port of Facebook’s LLaMA model. Still requires roughly 240GB of (unoptimized) weights, but can run on a 64GB Mac. |
|||
minillm | A GPU-focused Python wrapper for LLaMa |
|||
simple-llama-finetuner | A way to do LoRA adaptation of LLaMa |
|||
chatbot-ui | a more or less sensibly designed self-hosted ChatGPT UI |
|||
content-chatbot | A way to quickly create custom embeddings off a web site |
|||
chatblade | a CLI wrapper for ChatGPT |
|||
GPTQ-for-LLaMa | a way to quantize the LLaMA weights to 4-bit precision |
|||
llama-rs | A Rust port of llama.cpp |
|||
alpaca-lora | Another way to do LoRA adaptation of LLaMa |
|||
Vector Databases | 2023-08 | vectordb | A simple vector database that can run in-process |
|
marqo | A vector database that performs vector generation internally |
|||
2023-07 | chroma | an embedding database |
||
USearch | A Single-File Vector Search Engine |
|||
Workflows | danswer | a pretty complete GPT/search integration solution with GitHub, Slack and Confluence/JIRA connectors |
||
Multi-modal Models | Samples | 2024-02 | ml-mgie | instruction-based image self-editing |
NeRFs | Tools | 2022-12 | nerfstudio | A tool for manipulating Neural Radiance Fields (NeRF) and rendering the scenes out as video |
Speech Recognition | Models | 2024-01 | WhisperLive | a real-time text-to-speech system based on Whisper |
2023-11 | distil-whisper | a distilled version of whisper that is 6 times faster |
||
2022-12 | whisper | a general purpose speech recognition model |
||
2022-02 | whisper.cpp | a C++ implementation of whisper that can run in consumer hardware |
||
Tools | 2023-12 | insanely-fast-whisper | An opinionated CLI for audio transcription |
|
Speech Synthesis | Models | 2024-01 | Real-Time-Voice-Cloning | a PyTorch implementation of a voice cloning model |
WhisperSpeech | a text-to-speech system built by inverting Whisper |
|||
2023-12 | StyleTTS2 | A text to speech model that supports style diffusion |
||
Stable Diffusion | Apps | 2023-03 | swift-coreml-diffusers | Hugging face’s own app, using Swift and CoreML for Apple Silicon |
2022-11 | Draw Things | Pre-packaged app for iOS, downloads and allows re-use of .ckpt files. |
||
DiffusionBee | Pre-packaged app for macOS (M1 and Intel) |
|||
CGI | 2023-03 | Blender-ControlNet | A Blender plugin to generate ControlNet inputs for posing figures |
|
2022-12 | dream-textures | A Blender plugin for texturing models based on a text description. |
||
Implementations | 2023-10 | OnnxStream | Stable Diffusion XL 1.0 Base on a Raspberry Pi Zero 2 (or in 298MB of RAM) |
|
Libraries | 2024-01 | sd4j | a Java library for Stable Diffusion that uses ONNX |
|
Models | 2023-03 | Upscale Model Database | Too wide a choice, perhaps |
|
2022-12 | Fast Stable Diffusion | Another tactic to accelerate inference |
||
CoreML Stable Diffusion | Apple’s optimizations for CoreML |
|||
Reference | 2024-01 | comflowy | a set of reference workflows and documentation for ComfyUI |
|
Tools | 2024-03 | comflowyspace | a ComfyUI desktop wrapper |
|
2023-10 | ComfyUI-AnimateDiff-Evolved | An AnimateDiff integration for ComfyUI |
||
ComfyUI-Manager | A component manager for ComfyUI |
|||
2023-08 | stable-diffusion.cpp | stable diffusion inference on the CPU, in pure C++ |
||
Opendream | A layer-oriented, non-destructive editor |
|||
2023-03 | ComfyUI | pretty impressive node-based UI |
||
InvokeAI | A polished UI |
|||
2022-11 | imaginAIry | Works well on Apple Silicon, pure CLI interface to all SD models. Does not reuse .ckpt files, however, so requires separate disk cache. |
||
2022-08 | Stable Diffusion WebUI | Nearly always the best, bleeding edge WebUI for SD |
Woke up in a country where the right-wing party increased their representation four-fold, wherein by “right-wing” I actually mean “those who do not know history are doomed to repeat it” or far, far worse. I have full faith in incompetence to assert itself over time, but it’s going to be a rough ride.
And yes, there’s a metaphor in the way Internet Explorer crashes the whole thing.
Pretty impactful corporate reorg dropped–at about the same time as my building water main ruptured.
As Douglas Adams famously quipped, I definitely can’t get the hang of Tuesdays.
Before that happened:
snapdrop
fork to GitHub. Needs a few more tweaks and excision of odd bits of code, but has held up to regular use over the past few weeks.SKADIS
pegboard at lunchtime, which of course meant 3D printing some hooks and adapters.Nothing much happened after that that I can mention. I was too busy dealing with the fallout from the water main and the reorg, and that went on for a while.
It leverages the DDC
protocl and is written in Swift.
Not sure if genius, madness, or both, but generating near real time video of performance stats and viewing it in picture in picture mode is not exactly the first approach that comes to mind for designing an operating system monitor…
This page is a stub.
RAM
LCD
and potato webcamSSD
(I upgraded to 128GB)USB-A
portsSD
card slot (which does not work in all OSes)HDMI
outFinally, it is powered by a small barrel jack (plug is 3mm wide, 7.5mm deep, ~1mm bore). That takes 19V/3.42A, ~65W, so can be powered by a $2 USB-PD
adapter in theory, although it will have to be an external one.
The case is plastic, and yet the thing seems pretty much indestructible. The battery still claims to last 6 hours on occasion (more like three, really, but it’s still useful), and I tend to use it as a travel laptop when on vacation.
Once unlocked, you can use SeaBIOS
to install a normal Linux system. Mine started out with Lubuntu 14.04 and is (in 2024) running Fedora 39.
Edit /etc/default/grub
and add tpm_tis.force=1
to the default boot options to work around a delay in booting.
This script was required to compile a set of suitable kernel modules to support the trackpad. It took a fair amount of time to come up with sensible sensitivity settings that allowed for two-finger scrolling the “right” way.
Those were added as an LXDE startup script, since it is easier to tweak than X11 configs:
#!/bin/sh
echo "Areas"
synclient AreaRightEdge=850 AreaLeftEdge=50
echo "Pressure"
synclient FingerLow=10 FingerHigh=16
echo "Buttons"
synclient TapButton1=1 TapButton2=3 TapButton3=2
echo "Scroll"
synclient VertScrollDelta=-19 HorizScrollDelta=-19 HorizTwoFingerScroll=1
Bound these using obkey to Super+<key>
:
pactl set-sink-mute 0 toggle pactl set-sink-volume 0 -- +5% pactl set-sink-volume 0 -- -5% xbacklight -inc 5% xbacklight -dec 5%
…but using xbindkeys
seems to be a better option. Here’s a configuration file for it, taken from here.
As always under Linux that is a work in progress, which is a shame considering that Chrome OS works flawlessly.
The following steps gave me workable suspend/resume in Lubuntu at the expense of Bluetooth (they are not necessary for Fedora anymore, but I do get the occasional slowdown on resume, so I’m keeping the notes around)
/etc/pm/sleep.d
and set it executable with chmod +x
/etc/rc.local
GRUB_CMDLINE_LINUX_DEFAULT
in /etc/default/grub
to contain these options
sudo update-grub
and rebootAll that remains then is to set the LXDE
power manager to suspend on lid close, etc.
To get secure swap to work properly, you need to patch the ecryptfs-setup-swap
script to include an offset=8
option in /etc/crypttab
. This appears to be an Ubuntu bug,which, unaccountably, hasn’t been caught during testing or fixed yet.
alias pbcopy="xclip -selection c" alias pbpaste="xclip -o"
Woke up at 4AM, which kind of set the mood for the day.
Had to make a trip to the office again, which is becoming far more usual this year than I expected, so my free time was shot.
PCB
s and where I could fasten both an endoscope camera and a large (20cm across) Fresnel lens I have someplace. And yes, I should get a soldering microscope, but all the nice HDMI
ones are outrageously expensive.PETG
holders to fasten them to the SK1.I have only vague recollections of the day, which was so productive work-wise that it risked spilling over and nuking my quality time.
iGPU
side. I suspect that 30fps with some aliasing tweaks are entirely possible, but am going to move on to other workloads ASAP because I need to get back to my AI stuff.Woke up at 4AM again with a splitting headache (either travel doesn’t agree with me or late night meetings, and I suspect both).
XFCE
remote desktop to start a Basilisk build in the wee hours of the morning. I have plans for something portable, but am not sure it will get done this month. I would love to join the Global AppleTalk Network people are setting up, because it is so wonderfully nostalgic…Miserably rainy day. Woke up exhausted.
btop
, which despite looking either like a “l33t” tool or like pants on a standard terminal is a great all-in-one solution for monitoring CPU, GPU, I/O, disk usage and network traffic and saves me the trouble of constantly installing iotop
and htop
.from os import environ, listdir, makedirs
from os.path import join, exists, getmtime, isdir
from shutil import rmtree
from json import loads
from asyncio import run, create_task, sleep, create_subprocess_shell
from asyncio.subprocess import PIPE, STDOUT
from aiohttp import ClientSession
from logging import basicConfig, INFO, DEBUG, WARNING, getLogger
basicConfig(level=INFO, format='%(asctime)s %(levelname)s %(funcName)s:%(lineno)s %(message)s')
log = getLogger(__name__)
PROJECTS = environ.get('PROJECTS', 'Ryujinx/release-channel-master').split(',')
ARCHIVE_PATH = environ.get('ARCHIVE_PATH', 'releases')
RELEASE_COUNT = int(environ.get('RELEASE_COUNT', 5))
async def fetch(path: str, url: str) -> None:
proc = await create_subprocess_shell(f"wget -P {path} -N {url} -qnv", stdout=PIPE, stderr=PIPE)
log.info(f"fetching {url} to {path}")
stdout, stderr = await proc.communicate()
log.debug(stdout)
log.debug(stderr)
async def main() -> None:
async with ClientSession() as session:
for project in PROJECTS:
log.debug(f'Fetching releases for {project}')
async with session.get(f'https://api.github.com/repos/{project}/releases') as response:
data = await response.json()
filtered_data = [element for element in data if ((element["draft"] != True) and (element["prerelease"] != True))][:RELEASE_COUNT]
for release in filtered_data:
folder = join(ARCHIVE_PATH, project, release['tag_name'])
if not exists(folder):
makedirs(folder)
for asset in release['assets']:
if not exists(join(folder, asset['name'])):
create_task(fetch(folder, asset['browser_download_url']))
await sleep(0.1)
folders = [f for f in listdir(join(ARCHIVE_PATH, project)) if isdir(join(ARCHIVE_PATH, project, f))]
log.info(f"checking for old releases in {project}")
if len(folders) > RELEASE_COUNT:
folders.sort(reverse=True)
folders = folders[RELEASE_COUNT:]
for folder in folders:
log.warning(f"removing {project} release {folder}")
rmtree(join(ARCHIVE_PATH, project, folder))
if __name__ == '__main__':
run(main())
Decided this was going to be a procrastination-friendly weekend, so after reading the news I went full on random geekery mode:
moonlight-embedded
on a Raspberry Pi Zero 2W and actually played stuff on it:This may turn out to be addictive, but not yet. I don’t really fancy accumulating pretty dust collectors and am happy with sticking to functional parts…
Election day, which I had completely forgotten about. I mean, it’s not as if Portugal has been under particularly effective government for a few years now.
IKEA
for a bit of strategic storage implement acquisition (I always knew there would be a pegboard in my life sometime).VINDSTYRKA
air quality sensor (that uses Sensirion hardware and Thread) to try out, because my homegrown solution is tying up a Zero W I have other uses for and the poor Enviro+ board design generates inaccurate temperature readings since the cursor is too close to the Pi CPU. I do lose the ability to do my own metrics, but the VINDSTYRKA
“just worked”–I paired it with my zigbee2mqtt
network by tapping the pair button four times (took me a while to figure that out, as it isn’t documented in the booklet), and it immediately popped up in HomeKit and I can get centralized metrics from it, so… Net win, I guess. And yes, I’ll be checking the actual metrics in detail as soon as I have some time (I’ve got a vague “revise home IoT telemetry” entry in my TODOs since last year).ollama
on my test machines and resumed fiddling with LLM
s–this time I did some RAG
over my notes, which might not have been the best of ideas:Decided to call it a day and do non-computer stuff afterwards.
Update: I went to check on my desktop Mac (a Mac Mini M2 Pro) and it had spontaneously rebooted barely 15 minutes before I got to it. I started digging into the logs (
log show --last 45m
is your friend) and saw nothing obvious except this:
... 2024-03-10 18:43:55.246304+0000 0x1743 Default 0x0 0 0 kernel: (AppleThunderboltNHI) AppleThunderboltGenericHAL::lateSleep - complete - took 0 milliseconds 2024-03-10 18:43:55.246312+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformSleepAction -> AppleThunderboltHALType5 2024-03-10 18:43:55.246326+0000 0x1743 Default 0x0 0 0 kernel: (AppleThunderboltNHI) AppleThunderboltGenericHAL::lateSleep - complete - took 0 milliseconds 2024-03-10 18:43:55.246332+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformSleepAction -> AppleThunderboltHALType5 2024-03-10 18:43:55.246346+0000 0x1743 Default 0x0 0 0 kernel: (AppleThunderboltNHI) AppleThunderboltGenericHAL::lateSleep - complete - took 0 milliseconds 2024-03-10 18:43:55.246355+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformSleepAction -> AppleThunderboltHALType5 2024-03-10 18:43:55.246369+0000 0x1743 Default 0x0 0 0 kernel: (AppleThunderboltNHI) AppleThunderboltGenericHAL::lateSleep - complete - took 0 milliseconds 2024-03-10 18:43:55.246378+0000 0x1743 Default 0x0 0 0 kernel: PMRD: trace point 0x18 2024-03-10 18:43:55.246641+0000 0x1743 Default 0x0 0 0 kernel: PE_cpu_power_disable>turning off power to cluster 1 2024-03-10 18:43:55.246810+0000 0x1743 Default 0x0 0 0 kernel: PE_cpu_power_disable>turning off power to cluster 2 # Last line before crash 2024-03-10 18:43:55.246836+0000 0x1743 Default 0x0 0 0 kernel: PMRD: trace point 0x19 2024-03-10 18:52:57.788824+0000 0x1743 Default 0x0 0 0 kernel: PMRD: trace point 0x23 # ^^^ First line after crash #...lots of IOPlatformQuiesceAction -> AppleT8101GPIOIC and similar stuff elided 2024-03-10 18:52:57.788995+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 1 2024-03-10 18:52:57.789010+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 2 2024-03-10 18:52:57.789017+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 3 2024-03-10 18:52:57.789026+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 4 2024-03-10 18:52:57.789039+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 5 2024-03-10 18:52:57.789041+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592453]: arm_cpu_init(): cpu 1 online 2024-03-10 18:52:57.789054+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 6 2024-03-10 18:52:57.789055+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592462]: arm_cpu_init(): cpu 2 online 2024-03-10 18:52:57.789057+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592458]: cpu_start() cpu: 5 2024-03-10 18:52:57.789068+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 7 2024-03-10 18:52:57.789080+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 8 2024-03-10 18:52:57.789086+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592498]: arm_cpu_init(): cpu 4 online 2024-03-10 18:52:57.789098+0000 0x1743 Default 0x0 0 0 kernel: cpu_start() cpu: 9 2024-03-10 18:52:57.789100+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592508]: arm_cpu_init(): cpu 5 online 2024-03-10 18:52:57.789111+0000 0x1743 Default 0x0 0 0 kernel: PMRD: trace point 0x22 2024-03-10 18:52:57.789113+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592522]: arm_cpu_init(): cpu 6 online 2024-03-10 18:52:57.789121+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformWakeAction -> AppleThunderboltHALType5 2024-03-10 18:52:57.789134+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformWakeAction -> AppleThunderboltHALType5 2024-03-10 18:52:57.789145+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformWakeAction -> AppleThunderboltHALType5 2024-03-10 18:52:57.789150+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592555]: arm_cpu_init(): cpu 8 online 2024-03-10 18:52:57.789152+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592564]: arm_cpu_init(): cpu 9 online 2024-03-10 18:52:57.789165+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformWakeAction -> AppleThunderboltHALType5 2024-03-10 18:52:57.789167+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592564]: arm_cpu_init(): cpu 9 online 2024-03-10 18:52:57.789169+0000 0x1743 Default 0x0 0 0 kernel: [ 7257.592564]: IOPlatformWakeAction -> AppleThunderboltHALType5 2024-03-10 18:52:57.789181+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformWakeAction -> AppleMCA2Switch 2024-03-10 18:52:57.789373+0000 0x1743 Default 0x0 0 0 kernel: IOPlatformWakeAction -> IODTNVRAMPlatformNotifier ...
…so I went and filed Feedback FB13682232
, in hope that Apple can make some sense of it. Smells like a kernel bug to me.
Update 2: I managed to get the panic log by logging in as an admin user, and the first lines are:
panic(cpu 0 caller 0xfffffe002fc97ce8): Wake transition timed out after 35 seconds while calling power state change callbacks. Suspected bundle: com.apple.driver.AppleThunderboltNHI. Thread 0x1bff1f.
Debugger message: panic
And guess what, this particular kind of issue with com.apple.driver.AppleThunderboltNHID
has been reported since the M1 days.
I’m going to update Feedback FB13682232
, but I’m not holding my breath given how long this issue has been around.
RK3588
, but on the AllWinner H618
SoC, which is a lower end quad-core ARM Cortex-A53 running at 1.5GHz:
Disclaimer: Orange Pi sent me a review unit (for which I thank them), and this article follows my Review Policy.
The Zero 3 is a bit of an odd duck, but with a few interesting features:
LPDDR4
RAM
USB-C
for powerI quite like to see Ethernet ports in smaller SBC
s, since it makes them more useful as thin clients, and the generous RAM
allocation is also a plus for a board this size. The rest of the specs are a mix of the usual and the unusual:
H618
SoC
with 4xCortex-A53 cores running at 1.5GHz and a Mali G31 MP2 GPU
.20U5622
Wi-Fi/Bluetooth module that supports 802.11ac at 5GHz and Bluetooth 5.0 (a pigtail antenna is supplied)HDMI
port (which is a bit of an annoyance, but I have a few cables for that)SD
card slot (which is all you get for storage, since there is no EMMC
on this board)GPIO
header (which is also a bit of an oddity, but is documented on their wiki)USB-A
ports (it has one on the main board)Given the mixing and matching that is going on in the BOM
, I suspect this board (or its ancestors) was originally designed for use in a set-top box and re-purposed for the maker market by replacing an EMMC
with a card slot, but that’s just a hunch.
As usual, getting stable, maintainable software was a bit of a challenge. While fiddling with the board I realized that it booted into a mini-userland saved to SPI
flash when no card was present (which has interesting implications for field service and is something I’d like to see more often), but booting a full OS has to be done from an SD
card.
As to Linux distributions, it took me a while to actually get around to testing it, for two reasons:
On the Android front, things are just… strange. To flash the official image to an SD
card, you need to use an extraordinarily specific Windows tool (you read that right, it does something to the partitioning scheme that apparently can’t be done with your usual dd
or Mac/Linux imaging tools), and there is no real way to customize or build your own image–so after a few weeks of poking around I gave up on that–part out of principle (it’s an unmaintainable solution) and part out of frustration.
Then I went and spent a fair chunk of time searching for a way to get a Linux image I could easily maintain or upgrade, and things were better, but still far from perfect.
There are unofficial Armbian images available, but even though the wiki had quite a bit of documentation, it took me a while to realize I had to use specific system images for the 4GB RAM
version of the board.
And, you guessed it, there were no prebuilt ones. This is apparently due to the need of having specific DTB
files for uboot
depending on board version, and instructions for patching the images were a bit hard to find (they are in Chinese, which I personally don’t really mind, but by that time I was already out of time).
So, in short, the software situation is a bit… complicated. And for folk like me, who are used to embedded systems and can boot their own kernels, Orange Pi does have build instructions for building a Linux 6.1 kernel on GitHub.
But since at the time I was already trying to backport an RK3588
kernel to Armbian, I just didn’t have the time to deal with another SoC
and its quirks.
I eventually flashed the official Debian desktop image onto a 32GB SD
card, and even though the system booted to a US English locale, I still had to do a few changes:
huaweicom
repository from apt
sources, because, well… I’d rather update from other places, or use geo-distributed mirrors.xfce-terminal
, because (as usual in Chinese distributions) the default terminal is picked due to its ability to display Hanzi.firefox-esr
as a browser since it was lighter than Chromium.avahi-daemon
and nss-mdns
to be able to ssh
into it using hostname.local
and discover my other machines on the network.I was also pleasantly surprised by orangepi-config
, which is pretty feature-packed:
Running Debian, I measured the bare board at the USB-C
port, and saw it taking in a maximum of 2.2W while booting without anything connected, and idling at 0.8W in the same situation.
Plugging in a keyboard and mouse bumped these by almost 1W, and adding Ethernet another 0.3W.
With a monitor, keyboard and mouse plugged in, it was taking in a little above 2W idle, which is in line with expectations for a board with this kind of feature set.
With my original idea of flashing Android on it and test Moonlight alongside the Logitech G CLoud out the window, I plugged the Zero 3 into an LCD
panel I have on my electronics bench and used it to look up documentation and well as a thin client to connect to my desktops.
And it worked–pretty well, really. With 4GB of RAM
and a quad-core CPU
, it was snappy enough to open PDF
s and browse the web (as long as you don’t expect to watch a lot of video–anything over 720p was a bit sluggish):
I used it without the expansion board, since the keyboard I was using has a built-in USB
hub (and didn’t need audio), and it was almost invisible thanks to its small size.
So as a thin client, yes, it does work perfectly–I had zero issues with performance over VNC
or RDP
on the 1080p display I was using. And I did eventually try out Moonlight as well, although clearly the Linux image I was using was not optimized for making the best out of the Mali G31 GPU
:
The Orange Pi Zero 3 is a strange SBC
. Like many, it is let down by its software support, but the only real flaw I can point to hardware-wise (within comparable boards) is the lack of an EMMC
.
I would have loved to have a good, easy way to run Android on it (where the GPU
would shine), but as a Linux board I see it as being more useful for thin clients and running Klipper/Octoprint alongside a 3D printer than, say, digital signage (since the video decoding is lacking), but it’s a very capable one.
It is a bit of a shame that it doesn’t ship with EMMC
storage. But overall it is a capable board that can be used for a variety of purposes.
My plans for it are to (eventually) build it into a monitor (the panel on my electronics desk is just bare, literally hanging by the driver board, and I have been slowly designing a suitable enclosure). That way I can use it as a thin client, and, eventually, plug in my oscilloscope via USB.
But given its small size (it’s almost exactly the size of the now useless LCD on my KP3S Pro), I’ve also considered using it to run Klipper (although with 4GB of RAM
it could probably handle two or three printers), and I suspect a lot of people will find the lower-specced boards pretty handy for that.
Another use (given its Ethernet port and low consumption) is as a home router “sidekick”–you can confidently power it off a USB
port on your router and use it to run Pi-hole
for ad blocking or OpenWRT
for a personal VPN–which would be a nice and compact way to add those features to a home network.
It is a CoreXY machine with a 256×256×256mm print volume, direct drive extrusion and shipping with Klipper firmware.
Category | Date | Link | Notes |
---|---|---|---|
Models | 2024 | Two Trees SK1 Base Plate STL | A base plate STL to include in SK1 print profiles |
Mods | Bowden tube guides | A set of guides for placing the Bowden tube differently |
|
Fan Shroud with LED mountings | Allows you to mount Voron-style LEDs near the nozzle and makes it easier to remove for maintenance |
||
120mm fan adapter for the mainboard | A mount for a 120mm fan to cool the mainboard |
||
Aux fan holder | A mount for an auxiliary 12032 part fan |
||
Anti-vibration feet | These take third-party rubber feet from the Bambu P1P |
||
Webcam mount | A mount for the Creality K1 Webcam |
Reading through this makes me realize there has been no real progress on app distribution or improvements in Apple’s App Store ecosystem regarding openness over the part decade, and that besides these changes flying in the face of EU requirements, real, live, actual sideloading is still a pipe dream–there is absolutely no way it’s become feasible for the individual user, and new app stores and browser engines are still hampered by various restrictions.
(As an aside, I am particularly amused at the availability of game streaming apps, and would like to take this opportunity to thank Apple for forcing me to buy an NVIDIA Shield to use them for the past four years, because it has turned out to be a much better piece of hardware than the Apple TV, which they continue to blunder and neglect as a product–but I digress.)
Most of the “additions” like payment features and NFC access are things that should have been there from the start, and it is clear to anyone with a modicum of technical knowledge (let alone folk like me who worked literally decades in the mobile industry) that Apple has done the absolute minimum to “comply” to a degree that should be transparent to even the dimmest bureaucrats in Brussels.
Your move, EU. And try to make sure fining Apple for hindering music streaming wasn’t a one-off heavily driven by industry lobbyists, OK? Because multi-year investigations just don’t cut it anymore.