I have now officially reached the point where I have more KVM devices than I actually need (at least for now), and I think this one will be the last for a while–so this is my attempt to finish the series on a high note.

The JetKVM has been popping up a bit all over the place in the homelab scene, and I was very curious to try it given its unique Apple Watch-like look and form factor.

Disclaimer: JetKVM sent me a free sample unit (for which I thank them), and this article follows my review policy.

The device is very compact, and its all-metal construction feels very sturdy, an impression that is compounded by its small size and considerable weight:

As you can see above, all the connectors are on the back of the device:

• An RJ-45 Ethernet port (sadly, this doesn’t support Wi-Fi)
• An RJ-11 port extension port (for ATX power control, serial console, etc.)
• Mini-HDMI for video in
• USB-C for power and controlling your PC.

One thing I noticed almost immediately is that the spacing between the mini-HDMI port and the Ethernet jack is too tight. This means that if, like me, you have RJ-45 plugs with a reinforced plastic guard; they might actually not fit–I had to fish out another patch cable with a slim RJ-45 plug:

Despite the space constraints at the back of the device, I actually wish the JetKVM had a full HDMI port, as it would simplify things a bit even if we do get a nice mini-HDMI cable in the box.

I suppose it’s marginally better than micro HDMI, but you still need a specific kind of cable, which might be annoying to find if you need to replace it with something of a specific length.

But you do get three cables in the box:

• A mini-HDMI to HDMI cable
• A USB A-C cable for power
• A nice 7-8cm USB-C Y splitter (which lets you use separate power and data connections in case your PC does not provide standby USB power)

The build quality is excellent, and of course the front display steals all the attention. But, somewhat to my dismay, there is actually very little you can do with it–you can switch to a second screen listing the firmware version, and that’s about it:

Also, in my experience, the display sometimes turns off after a while–which means you have to touch it to read the IP address if you forget.

However, the remote experience is much more sophisticated. Compared to most KVMs I’ve been using, the JetKVM’s WebUI borders on the luxurious, with a very nicely designed web interface featuring multiple panes:

In terms of making it easy to discover and access all of the features, the UI is excellent–and I’ve used it as my preferred device for testing new mini-PCs and single-board computers for a couple of months now.

Although I am very happy that I need zero proprietary software to access the JetKVM and am quite impressed at the feature set (including, for instance, switching between several EDIDs to account for monitor recognition vagaries) I do think that a “simple” mode that used less screen real estate would be useful–perhaps collapsing things into a simple unobtrusive bar like in full screen mode.

I would also love to have it announce itself over Rendezvous (my PiKVMs do so as pikvmX.local, which means I never have to remember their IP addresses). Apparently that was a feature in the past, but it was removed for some reason.

Like the NanoKVM I looked at a few weeks back, the JetKVM’s software is also open-source on GitHub–but unlike it, the JetKVM hasn’t had a lot of controversy around it, and the cloud services behind it are actually open source and can be self-hosted.

That is perhaps a step too far for me for a single device, but there is an open issue mentioning a Tailscale plugin, so support might not be too far off–and of course I can still access it remotely via Tailscale by routing though my home exit node.

Regarding JetKVM’s cloud service, I ended up not trying it because their only OpenID Connect method supported is Google, and I’m actively trying to remove that kind of dependency from my life.

Feature-wise, I only have two software complaints: The first is that there isn’t a screenshot feature (neither in the 0.1.18 version it shipped with nor in the latest 0.2.3, which would be very useful for quick sharing and documentation (and no, taking a screenshot of the browser or using extensions doesn’t replace having this as a built-in feature).

So I just hacked my own and created this JavaScript bookmarklet, which works great:

javascript:"use strict";!function(window,document){var canvas=document.createElement("canvas"),video=document.querySelector("video"),ctx=canvas.getContext("2d");canvas.width=parseInt(video.offsetWidth);canvas.height=parseInt(video.offsetHeight);ctx.drawImage(video,0,0,canvas.width,canvas.height);var currentDateTime=new Date().toISOString().replace('T',' at ').replaceAll(':','.').substring(0,22);var a=document.createElement("a");a.download=`Screenshot ${canvas.width}x${canvas.height} ${currentDateTime}.png`;a.href=canvas.toDataURL("image/png");document.body.appendChild(a).click();a.remove()}(window,document)

The second is a bit stranger, and I suspect it is a video encoding setting–I have been unable to get the video to show truly white images.

The video stream and screenshots I get are all off-white, which is hardly relevant when using this to fix something but very noticeable when I need to take screenshots of remote machines with the-actually I swapped back and forth with the NanoKVM, and the image quality was quite different:

To me the JetKVM’s “High” video quality is arguably worse than the NanoKVM’s default but feels slightly snappier, which is why I think this is an encoding compromise that I hope can be fixed in a later firmware version.

I’ve been using the JetKVM on a daily basis to set up development boards and a couple of mini-PCs, and it’s been rock solid other than occasional (likely browser related) input glitches and the slight video encoding issue I point out above.

Software-wise, I’m just a bit hung up on the display.

Although vastly more readable than other devices I’ve looked at lately, it’s nevertheless stupendously luxurious for what it can actually do, and It’s a bit strange to have a touchscreen device that I can interact with so little.

Either way, I think all the above can be fixed in firmware upgrades–except for the lack of Wi-Fi, which is a particular preference of mine for temporary setups.

I know I’m late to this party, but I’ve been reading John Gruber’s piece “Something Is Rotten in the State of Cupertino”, and finally found time to share my thoughts.

And, to be fair, I at first I let it linger in my RSS reader due to its size, but during a quiet evening yesterday I finally got a few paragraphs down and I thought “OK, John’s really upset about this”.

But I get why. When you see claims like personalized Siri, meant to pull context from your emails, messages, and even your screen, only to have them delayed repeatedly, you can’t help but ask: isn’t this reminiscent of the old “so why the fuck doesn’t it do that?” moment from the Jobs era?

John cuts through the hype with a dry reminder that good marketing must be backed by working technology, and since I too appreciate a firm demo over slick concept videos and glossy commercials I have to say it seems Apple’s credibility is starting to show cracks. Well, new cracks, at least.

While it is true that Tim Cook has transformed Apple into a profitable, well-oiled machine. However, promising breakthrough AI features without providing demos or hands-on trials risks undermining the trust Apple has built-—except perhaps among long-time Apple developers who’ve been burned by similar things in the past.

So there are three points I would like to make here:

• Siri has been woefully mismanaged for many years.
• Apple Intelligence doesn’t just lack focus, it’s unattainable as is.
• Even if Apple could sort out the two points above, they currently lack enough of an automation foundation to make either meaningful.

So, here is what I found on the web about Siri improvements in recent years (and if you don’t get that reference, you must have been using an Android device for the past decade):

Nothing. Nada. Zilch.

Apple’s recent announcements about “more personalized Siri” and Apple Intelligence felt like the same old song and dance from Cupertino—big promises with little substance.

Oh, there are a couple of pieces about corporate reorgs (including this one from January, which seems to fall mostly into the “rearranging deck chairs” category), but essentially nothing got upgraded or improved since 2021 except a shift to on-device speech processing.

Everything else is still largely the same, and anyone who’s asked Siri to do the simplest of things and hasn’t (yet) given up will readily attest to that.

This level of staleness in a voice assistant isn’t unique (we did have a pandemic, and Alexa was also pretty much neglected until Panos Panay moved to Amazon), but given that Apple was already so comically far behind, it’s more likely that the engineering effort required to do anything useful fell somewhere between the cracks of their shift to Apple Silicon and creating bundles to increase services revenue.

A part of me also wants to rant on about the general decay of macOS and iOS in anything other than cosmetics, but we can skip that for the moment.

Let’s look at its current state.

As an EU citizen, I have had to finagle my way into actually being able to test a few of Apple Intelligence’s “features”, and the text-oriented ones are only marginally better than my homebrew shortcuts because they’re baked deeper into the system and not restricted to the (woefully unmaintained) Services menu.

But, other than notification summaries, which I’ve kept on solely for wry amusement, I’ve only used them to reply to an actual e-mail once–and that was pretty much by mistake.

I used Image Playground less than a dozen times (mostly to see if I could break the content filtering, honestly), and there are so many free image generation tools out there (and they’re moving so quickly into video) that I feel like I’m wasting perfectly good pixels in pointing out its irrelevance.

The main issue I have with Apple Intelligence is that all the features that have been surfaced so far (except perhaps Writing Tools, which can actually be a powerful accessibility enabler) are utterly pointless, and even if Private Cloud Compute seems like a major technical achievement to the uninitiated, there are loads of confidential computing solutions out there right now.

But here’s another thing: any confidential AI computing solution, regardless of how it manages context and compute, requires a working AI model–preferably one smarter than the one powering Writing Tools. And Apple has, so far, shown zero public interest in creating the kind of frontier models that people like me deal with on a daily basis.

I initially thought that their punting on some replies and deferring to ChatGPT was a way to defer accountability (although they had no compunction in having “older” Siri defer to Wolfram Alpha, something that seems to have stopped), but now I think that Apple is simply incapable (or, most likely, unwilling) to invest in training their own large-scale models and is sticking to gemma and phi grade on-device SLMs for the time being.

And that might be a bit of a problem, because (taking gemma3 as this week’s example) those models are not really that useful. I have been trying to coax some of them to tackle limited contexts like home automation for a while, and the results are not encouraging.

Prior to Tim Cook, I would hope for something magic being done someplace in a vault under Apple Park. Now, it’s either happening Severance-style or not at all, because I honestly don’t think Apple is that good at software anymore.

Just go into Shortcuts and check what actions you can automate in any of Apple’s core apps (at least the ones that seem relevant to me in the context of a “Smarter Siri”):

• Two actions for mail, none of which tackles searching for an e-mail or getting its contents
• Two for Messages, none of which are contextual
• Seven for Contacts, none of which are contextual either
• Eleven for Calendars (surprisingly)
• Twenty(ish) for Notes (even more surprising - these people are my heroes)
• Eleven for Photos, none of which deal with search or context
• Nine for Reminders
• Three for Home (which is kind of a major thing in “personal contexts”)

There is pretty much zero “public” API surface to do anything remotely like what Apple Intelligence requires to fulfill its promises where it regards context awareness or application actions.

But here’s the thing: even if you consider that Apple would primarily be using some sort of private automation hooks, the foundations for those have long rusted away (or never been implemented in iOS).

I would like to believe that gap, and the apparent extinction of the notion of proper app platform features in macOS and iOS (as well as an overemphasis on sandboxing all the things) is very probably why Apple is blending AI into the apps themselves in a rather uneven way and (along with the blind alley of LLMs) another major stumbling block into building “proper” Apple Intelligence (even if it doesn’t excuse Siri).

As someone who relied a lot on e-mail plugins until Apple effectively killed them, still tries to use JXA and Shortcuts to automate their workflows and bore witness to AppleScript’s progressive demise over the years, I’d say Apple not just painted themselves into a corner here, but they did that to themselves due to their general neglect of macOS and iOS foundational technology.

And, of course, third party developers aren’t happy with this at all–not just because Siri would abstract (and hide away) their apps to fulfill user intents, but also because there is no real return from building app intents atop such a shaky foundation.

In short, Apple is screwed, and it’s kind of ironic that they got bit by their own self-inflicted AI hype without going all in on AI themselves.

This is going to be a slightly different review, since it is not about a finished product–I’ve been sent an early sample of ArmSom/BananaPi’s upcoming AI module 7 (which will be up on CrowdSupply soon), and thought I’d share my early impressions of it.

Disclaimer: ArmSom provided me with an AIM7 and an AIM-IO carrier board free of charge (for which I thank them), and besides this piece following my review policy, I should reinforce that this is about a crowdfunding project, so there’s no guarantee that the final product will be exactly the same as the one I received.

The AIM7 itself is a straightforward module patterned after the NVIDIA Jetson, with the same DIMM-like connector and onboard RAM and eMMC storage:

In case you’re not familiar with the chipset, these are the baseline specs:

• 4xARM Cortex-A76 + 4xCortex-A55 CPU cores
• Mali-G610 MP4 GPU
• 3-core NPU capable of 6 TOPS
• up to 32GB of LPDDR4 RAM
• up to 128GB of eMMC storage

I got an 8GB RAM/32GB EMMC board, which, to be honest, is very tight both RAM and storage-wise–it is certainly too little RAM for most of the AI workloads I tried, and fairly limiting for development considering that there is no other way to get high-speed storage hooked up.

But as we’ll see later, it might be perfectly suitable for industrial applications, not GenAI flights of fancy.

The module did not come with a thermal solution, so I resorted to my usual set of copper shims to keep it (quietly) cool. In that respect (and power consumption) it was not substantially different from any of the many RK3588 devices I’ve tested, ranging from 4 to 11W (typical use vs full CPU load).

The carrier board is where all the I/O action is–besides being powered via a 12V barrel jack or USB-C (I chose the former this time around), it has 4 USB-A ports, HDMI, DisplayPort, a single Gigabit Ethernet port, and the usual set of hardware ports–40-pin GPIO header, MIPI-CSI/DSI, a microSD card reader, and an M.2 E-key slot for Wi-Fi:

Besides the single carrier board, the BananaPi wiki page hints at a 4-module AI edge gateway, so there’s clearly potential for building ARM clusters here.

Putting the AIM7 side-by-side with a Jetson Nano, the similarities are obvious:

ArmSom’s Crowdsupply page has a quick comparison with the Jetson Nano with some benchmarking information, so I won’t spend a lot of time on that here other than say that the modules are actually meant to be pin-compatible.

That said, I couldn’t run a set of comparative benchmarks for a couple of reasons: The 5V supply I got for the Jetson Nano died sometime late last year, so I can’t power it right now–but even if I tried exchanging carrier boards, the Nano I have was running a pretty old version of Ubuntu and CUDA, so it would actually be hard to get a meaningful comparison.

In truth, there’s just too little modern code that will actually run on the Nano unless you craft your own.

So, does the AIM7 replace the NVIDIA Jetson? Well, considering software alone the short answer is “no if you really need to use CUDA”, but I address that point below.

In short, it offers an interesting alternative if you care to venture into the waters of Rockchip’s various NPU libraries–which have actually improved quite a bit over the past few months.

The board shipped with Debian Bullseye (11) and kernel 5.10.160, which feels a little dated but is par for the course in the Rockchip universe.

Since there is no Armbian support for this board yet, I just reset the locale settings to en_US.UTF-8 (it came with the usual Chinese locale) and made do with it:

uname -a
Linux armsom-aim7 5.10.160 #98 SMP Thu Jan 2 15:14:22 CST 2025 aarch64 GNU/Linux

locale
sudo vi /etc/locale.gen
sudo locale-gen
sudo vi /etc/default/locale

And yes, of course I ran DeepSeek on it as soon as I got it. It’s a great party trick, but it doesn’t really do much other than show off the NPU’s capabilities.

What I’ve found much more interesting is running a few other models directly on the NPU, like marco-o1 and gemma-2:

However, any of them was a pretty tight fit–the 8GB of RAM was the main constraint, and I couldn’t get gemma-3:4b to run at all–although I was able to run gemma-3:1b under ollama (because I couldn’t convert it–more on that below).

So, in short, if LLMs are what you intend to play with, go for a bigger SKU with more eMMC storage and RAM–the 8GB/32GB version is just too tight for anything other than a few small models, and even then only one at a time.

One of my perennial gripes with the Rockchip toolchains is that a lot of the rkllm code has been patchy and hard to use. However, a few brave souls have forged on, and these days rkllm-1.1.4 is actually pretty usable.

And there are a few other tools available for the RK3588’s NPU that might make things easier for people looking to experiment with LLMs, including two I’m using in the screenshot above:

• rkllama gives you an ollama-like experience (even down to being able to download pre-converted models from the cloud, in this case directly from Huggingface)
• rknputop is the closest thing to an nvtop-like view of NPU usage (albeit still a little buggy, and including CPU, RAM and temperature)

There are also (finally) documented and reproducible ways to convert models to .rkllm format, and (even better) a surprising amount of pre-converted models on Huggingface iike these.

Converting models has actually become somewhat trivial, provided you use relatively old model families–for instance, I could not convert phi4 or the latest gemma3 to .rkllm format because:

• I had a bunch of issues with the phi4 tokenizer, so even though I could convert phi3 the same conversion code wouldn’t work for phi4.
• gemma3, which just came out as I am finishing this post, is simply not supported by rkllm since the model actually declares itself as a different “family”.

Now, rkllm won’t be updated for a while yet, but hacking at the model tree with torch and transformers isn’t something I have the time or patience to do these days, so I wasn’t able to devise a suitable workaround–yet.

But the general approach is pretty simple:

from rkllm.api import RKLLM
from os.path import basename
import torch

# tried to use these to convert the tokenizer
from transformers import AutoTokenizer, AutoModel, PreTrainedTokenizerFast
#...

modelpath = './Phi-3-mini'
llm = RKLLM()

# options ['cpu', 'cuda']
ret = llm.load_huggingface(model=modelpath, model_lora = None, device='cuda')
# ret = llm.load_gguf(model = modelpath)
if ret != 0:
    print('Load model failed!')
    exit(ret)

# break out the parameters for quicker tweaking
ret = llm.build(do_quantization=True, 
                optimization_level=1,
                quantized_dtype="W8A8",
                quantized_algorithm="normal",
                target_platform="RK3588", 
                num_npu_core=3, 
                extra_qparams=None, 
                dataset=None)
if ret != 0:
    print('Build model failed!')
    exit(ret)

# Export rkllm model
ret = llm.export_rkllm(f"./{basename(modelpath)}_{quantized_dtype}_{target_platform}.rkllm")
if ret != 0:
    print('Export model failed!')
    exit(ret)

There are a few other options you can tweak, but with my 3060 converting a couple of small models took mere minutes.

However, I don’t think the AIM7 is going to be used for running LLMs in real life.

For most people rknn-toolkit2 is likely to be of more interest, since it allows you to use whisper and various versions of YOLO (that you can get from the model zoo ) for speech and image recognition.

I know for a fact that the Jetson Nano was extensively used in image processing and defect detection in manufacturing lines, and although I haven’t had the time to actually test YOLO on this board yet, I know that the AIM7 is a suitable replacement because that’s what many other RK3588 boards are already being used for.

Which is why having an RK3588 board that is a direct replacement for a Jetson (which, by the way, tends to be picky about power and much hotter and noisier) makes a lot of sense.

I am not sure about video frame rates since that will always be heavily model-dependent, but earlier RK3588 boards with the original rknn toolkit were able to go well over 25fps, and I suspect that will at least double with some careful optimization.

But, most importantly, it isn’t hard to find example code that uses rknn for object detection and there is now even official support from tools like Frigate, so it’s certainly no longer an unexplored path.

I like the AIM7–I can’t really compare it directly to the other RK3588 SBCs I reviewed in terms of features (other than the chipset and base I/O) because it is very specifically designed to replace or compete with the Jetson, but, ironically, out of all of them it is the best suited for industrial applications–where you don’t really need NVMe storage, and the even 32GB will be plenty to have multiple copies of vision models like YOLO.

I just wish it had come with more RAM as far as I’m concerned…

Reviewing things is an interesting experience–no matter what, I learn something from dealing with the various hardware and software bits I get, and every now and then you come across something you really like despite some challenges along the way.

Read More...

Some six months ago, in the middle of a somewhat troubled summer, I decided to stock up on IP KVMs and ordered a pair of Sipeed NanoKVMs–a full kit and a barebones one. The price was simply irresistible, and I wanted something that could eventually replace my homegrown PiKVMs.

Read More...

In retrospect, I find it vaguely amusing that “doomscrolling” only really took off during the pandemic, because, well, for me it started a little earlier.

Read More...

I’ve been tinkering with deepseek-r1 for a couple of weeks now, but with its newfound popularity and cries of impending doom for developers, I thought I’d do a little bit more systematic testing with a trivial use case and post the results.

Read More...