More Raspberry Pi 2 Benchmarking

So what do you do on a Saturday morning? Me, I benchmark ARM development boards using Clojure.

Following up on Wednesday’s quick test, I decided to even out the scores a bit and include the original Raspberry Pi Model B.

To begin with, I overclocked the Pi 2 to 1GHz, connected everything via Ethernet and re-ran the benchmark for 15m to minimize any effects from other processes.

All machines were running the exact same code like so (again, I’m using JDK 1.8):

<```bash> $ java -cp standalone.jar clojure.main -m ink-clj.main > /dev/null

What that does is serve a plain HTML template that does a little markup processing. You can see what that looks like here, where I keep a running instance (obviously, the benchmark doesn’t load the CSS – that’s just a template of mine for new projects).

I used wrk again, and this is what the results looked like:

# Pi 2

$ wrk -c 100 -d 15m --latency -t 50 http://quad-damage.lan:8080
Running 15m test @ http://quad-damage.lan:8080
  50 threads and 100 connections
load: 1.84  cmd: wrk 1215 waiting 13.44u 61.92s
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   105.17ms    9.27ms 321.76ms   86.72%
    Req/Sec    18.60      1.28    24.00     59.90%
  Latency Distribution
     50%  103.64ms
     75%  108.50ms
     90%  114.13ms
     99%  128.28ms
  857155 requests in 15.00m, 3.38GB read
Requests/sec:    952.31
Transfer/sec:      3.84MB

This time around (via Ethernet but with 1GHz overclocking), the test took the Pi’s four cores up to 80% sustained CPU load.

ODROID-U2

The ODROID, on the other hand, went a little higher than last time (at around 55-60%), with these results:

# ODROID-U2

$ wrk -c 100 -d 15m --latency -t 50 http://odroid-server.lan:8080
Running 15m test @ http://odroid-server.lan:8080
  50 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    71.08ms   11.23ms 229.91ms   73.13%
    Req/Sec    28.12      3.66    38.00     67.24%
  Latency Distribution
     50%   66.97ms
     75%   77.10ms
     90%   88.04ms
     99%   94.99ms
  1284929 requests in 15.00m, 5.06GB read
Requests/sec:   1427.64
Transfer/sec:      5.76MB

For good measure, I flashed a brand new Class 10 SD Card with the exact same Raspbian image as the Pi 2 and overclocked an original Model B to 1GHz as well (as well as setting its GPU split to 16MB).

When I ran the benchmark against it, the thing died. It threw up a kernel panic, corrupted the SD card and that was that.

So I re-flashed and tried overclocking it to 900MHz instead, yielding the following results:

# Model B

$ wrk -c 100 -d 15m --latency -t 50 http://model-b.lan:8080
Running 15m test @ http://model-b.lan:8080
  50 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.06s   195.15ms   3.53s    95.26%
    Req/Sec     1.47      0.53     2.00     50.26%
  Latency Distribution
     50%    1.03s 
     75%    1.06s 
     90%    1.08s 
     99%    2.13s 
  86382 requests in 15.01m, 348.55MB read
  Socket errors: connect 0, read 0, write 0, timeout 70
Requests/sec:     95.90
Transfer/sec:    396.23KB

Whereas the Pi 2 put up a decent fight, the Model B can hardly keep up.

Here are all the results in a nice little table:

The good news here is that the Pi 2 is almost ten times faster than the original board, with the ODROID-U2 being one and a half times faster still (which is no big surprise, given that it’s clocked at 1.7GHz).

I have to wonder about the Pi 2’s CPU load (I have a nagging feeling at least part of it is likely to be due to it still having Ethernet hang off the USB bus), but I don’t have any meaningful computing benchmarks handy, and the differences between a (still not ARMv7 optimized) Raspbian and Ubuntu mean that we’ll only be able to get proper comparisons when things progress a bit further on the software front¹.

In the meantime, I’ve also taken the time to set up NetBeans on the Pi 2, and found it almost as speedy as on the U2 – perfectly usable for small Java projects, including debugging and profiling.

I haven’t tried any normal desktop applications (other than Chromium), but I’d say the Pi 2 is finally worth considering as an full-fledged educational computer (i.e., beyond the introductory toy stage).

All things considered (and I don’t go in much for popularity), it’s now the best bang for the buck out there, and judging from my brief experiments with arcade emulation (I’m currently having another go at my Christmas experiment), there’s going to be a lot of (literally) entertaining things to do with the Pi 2.

Now all I need is Ubuntu for it. Why? Because the packages on it are way better than Raspbian’s, and they’ve been doing ARMv7 for a good while.

Here’s an example of what we’re missing out on:

Pro Tip: Install a Decent VNC Server

For some reason (I suppose licensing or other Debianisms), Raspbian ships with tigthvncserver, which is markedly inferior to the “real” VNC server. Even on the Pi 2, the thing has a lot of lag, and I found it extremely annoying to use when compared to the smooth experience I’ve had on my ODROID for a couple of years.

So I decided to rebuild the vnc4server package from Ubuntu (instructions are on my Raspberry Pi page), and mostly as I expected the result was a staggering improvement in terms of responsiveness.

I am also able to do something that never seemed to work properly in tightvncserver: switch the desktop size on the fly.

To do so, just start vncserver with a set of geometry settings, like so:

vncserver -depth 16 -geometry 1024x768 -geometry 1280x800 -geometry 1366x768 -geometry 1600x1200 -geometry 1280x720 -geometry 1920x1080 -geometry 2048x1536

…and you can then use xrandr -r <number> to switch between them on the fly, which significantly improves things when you access the same machine from multiple devices.

And yes, I had tried using the exact same settings with tightvncserver, from bit depth to encoding. I don’t think the marked increase in performance came from it being a new, optimized binary (although that may have contributed to it), but rather because it’s a better package for the task at hand.

Either way, both are more than enough justification for me to keep looking for a “regular” Ubuntu build for this board – I’ve half a mind to try building a network install rootfs for it, but I need to get up to speed on other stuff…

As a final note, I still haven’t figured out why Chromium is much faster than the built in browser (either on the console or remotely), but I’ve since ruled out IPv6 or DNS shenanigans. Ping me on Twitter if you’ve found a fix.