How to run netdev selftests CI style

netdev selftest runner

This page describes how we run selftests in netdev CI. It should be helpful for reproducing CI failures, but it's not the one and only way the tests can be run!

Netdev CI: https://netdev.bots.linux.dev/status.html

"Flakes" view: https://netdev.bots.linux.dev/flakes.html

vng

We run tests in virtme-ng: https://github.com/arighi/virtme-ng virtme-ng builds the kernel and runs the tests.

Runner types

There are 3 runner types which can be distinguished by looking at the "Remote". Runners with "metal" in the name of remote run on a bare metal machine but still in vng, meaning the VMs which execute the tests have KVM virtualization. This is the most usual way to run tests.

Runners without "metal" in the name ("ksft") run in a VM, so vng has to deal with nested virtualization and no KVM support.

Then there are runners with a "-dbg" suffix, those have extra debug Kconfig options enabled.

How to build

Kernels are built with just the relevant options enabled, for instance for net selftests:

vng --build  --config tools/testing/selftests/net/config

and for forwarding selftests we'd use the forwarding config:

vng --build  --config tools/testing/selftests/net/forwarding/config

The "-dbg" runners get extra debug options:

vng --build  \
        --config tools/testing/selftests/net/forwarding/config \
        --config kernel/configs/debug.config \
        --config kernel/configs/x86_debug.config

Tests are built separately, e.g. for forwarding:

make -C tools/testing/selftests/ TARGETS=net/forwarding

How to run

Executor runs tests one by one:

vng -v --run . --user root --cpus 4 -- \
        make -C tools/testing/selftests TARGETS=net TEST_PROGS=pmtu.sh TEST_GEN_PROGS="" run_tests

Tips

• To simulate the "ksft" (i.e. non-"metal") runners add --disable-kvm to vng options.
• Skip the -v option if you don't want to see kernel logs.
• If QEMU fails to start try using --disable-microvm.
• some tests try to write into the PWD and virtme-ng gives by default RO access to the host filesystem, to override that add --rwdir tools/testing/selftests/net/