Test Performance and Functionality on Raspberry Pi OS 64-bit

[geerlingguy]

Test with the new 64-bit Pi OS ARM image.

[geerlingguy]

Upstream issue filed: geerlingguy/ansible-role-docker_arm#10

[geerlingguy]

Testing again with ansible-playbook -i inventory -c local main.yml -e "docker-version=5:19.03.9~3-0~debian-buster"

That didn't work, because it's docker_version. Oops! Anyways, better to override in a config.yml anyways...

[geerlingguy]

I'd also like to run an informal https://www.phoronix-test-suite.com/?k=downloads to see if there's a notable difference between the armv7l and arm64 version.

Related: https://medium.com/@matteocroce/why-you-should-run-a-64-bit-os-on-your-raspberry-pi4-bd5290d48947

[geerlingguy]

One interesting difference:

$ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
HYPRIOT_DEVICE="Raspberry Pi"
HYPRIOT_IMAGE_VERSION="v1.12.0"

# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

[geerlingguy]

The mysql arm image I'm using from Hypriot hasn't been updated in two years, and isn't compatible with arm64... so maybe consider one of the following for mysql_docker_image:

# If this works, it could be used for both AMD64 and ARM64.
mysql_docker_image: mysql/mysql-server:8.0

# This would be more compatible with MySQL 5.6/5.7...
mysql_docker_image: arm64v8/mariadb:10.4

Note that making the change as a default would break any existing users if they are not on Raspberry Pi OS 64-bit and/or have an older Pi that can't run it.

[geerlingguy]

Phoronix:

sudo apt-get install -y php7.3-cli php7.3-xml
wget https://phoronix-test-suite.com/releases/phoronix-test-suite-9.6.1.tar.gz
tar -xvf phoronix-test-suite-9.6.1.tar.gz
cd phoronix-test-suite
./phoronix-test-suite system-info # make sure it's working, agree to terms
./phoronix-test-suite list-recommended-tests # see what tests are recommended

# Run a few benchmarks (run as root to avoid prompts; don't do this on an instance you care about, it'll install a lot of stuff!)
export FORCE_TIMES_TO_RUN=3
./phoronix-test-suite benchmark pts/john-the-ripper
./phoronix-test-suite benchmark pts/compress-7zip
./phoronix-test-suite benchmark pts/iperf # requires separate server: `iperf -s`
./phoronix-test-suite benchmark pts/ramspeed

[geerlingguy]

Pi 4, 64-bit

(Note: Long-term test runs with a bare Pi (no fan, no cooling heatsink) resulted in the Pi CPU overheating (vcgencmd measure_temp) and throttling (vcgencmd get_throttled) quite a bit. I had to put the Pi in my modified Pi Case with Fan and it was much more consistent.

john-the-ripper

Test: Blowfish
Average: 1223 Real C/S
Deviation: 3.50%
Samples: 15

Test: MD5
Average: 32790 Real C/S
Deviation: 0.16%
Samples: 5

iperf

Note: Done manually iperf -s on Mac, iperf -c [ip] on Pi, phoronix was erroring out.

917 Mbits/sec

(Gave up after a few hours; Phoronix suites were having some issues (lots of results had high variance or were erroring out), so I'm going to use other benchmarking for simplicity's sake.)

[geerlingguy]

Pi 4 64-bit

All benchmarks run three times, averaged result in table.

Benchmark	Result
iperf	917 Mbits/sec
stress-ng	1552 bogo ops/s

# stress-ng --cpu 4 --cpu-method fft --cpu-ops 50000 --metrics-brief
stress-ng: info:  [13789] defaulting to a 86400 second (1 day, 0.00 secs) run per stressor
stress-ng: info:  [13789] dispatching hogs: 4 cpu
stress-ng: info:  [13789] cache allocate: using defaults, can't determine cache details from sysfs
stress-ng: info:  [13789] successful run completed in 32.50s
stress-ng: info:  [13789] stressor       bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [13789]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [13789] cpu               50000     32.44    129.16      0.00      1541.21       387.12

Pi 4 32-bit

Benchmark	Result
iperf	937 Mbits/sec
stress-ng	1377 bogo ops/s

# stress-ng --cpu 4 --cpu-method fft --cpu-ops 50000 --metrics-brief
stress-ng: info:  [980] defaulting to a 86400 second (1 day, 0.00 secs) run per stressor
stress-ng: info:  [980] dispatching hogs: 4 cpu
stress-ng: info:  [980] cache allocate: using defaults, can't determine cache details from sysfs
stress-ng: info:  [980] successful run completed in 36.53s
stress-ng: info:  [980] stressor       bogo ops real time  usr time  sys time   bogo ops/s   bogo ops/s
stress-ng: info:  [980]                           (secs)    (secs)    (secs)   (real time) (usr+sys time)
stress-ng: info:  [980] cpu               50000     36.34    145.26      0.00      1376.07       344.21

[geerlingguy]

For comparison on my Mac with it's i9, stress-ng (slightly different test, processor, OS, etc.) gives 209,486 (compared to ~50,000 for a float on the Pi). So a tad bit faster :P

[geerlingguy]

Testing with https://gist.github.com/geerlingguy/570e13f4f81a40a5395688667b1f79af now.

Just noting—when I run the above script I currently get a prompt Would you like to save these test results (Y/n): n... need to either add an expect/echo, or figure out some other way to tell phoronix to stop asking so many darn questions!