test-backend-ops: add support for specifying output format

[yeahdongcn]

Make sure to read the contributing guidelines before submitting a PR

Testing Done

root@xiaodongye-s80:/ws# ./build/bin/test-backend-ops --help
Usage: ./build/bin/test-backend-ops [mode] [-o <op>] [-b <backend>] [-p <params regex>] [--output <console|sql>]
    valid modes:
      - test (default, compare with CPU backend for correctness)
      - grad (compare gradients from backpropagation with method of finite differences)
      - perf (performance evaluation)
    op names for -o are as given by ggml_op_desc() (e.g. ADD, MUL_MAT, etc)
    --output specifies output format (default: console)

root@xiaodongye-s80:/ws# ./build/bin/test-backend-ops perf -b CPU -o ADD
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 MUSA devices:
  Device 0: MTT S80, compute capability 2.1, VMM: yes
Testing 2 devices

Backend 1/2: MUSA0
  Skipping
Backend 2/2: CPU
  Device description: 12th Gen Intel(R) Core(TM) i5-12400
  Device memory: 31859 MB (31859 MB free)

  ADD(type=f32,ne=[4096,1,1,1],nr=[1,1,1,1]):   ADD(type=f32,ne=[4096,1,1,1],nr=[1,1,1,1]):                1572480 runs -     0.64 us/run -       48 kB/run -   71.97 GB/s
  ADD(type=f32,ne=[4096,1,1,1],nr=[1,512,1,1]):   ADD(type=f32,ne=[4096,1,1,1],nr=[1,512,1,1]):                13338 runs -    76.12 us/run -    24576 kB/run -  308.22 GB/s
  Backend CPU: OK

2/2 backends passed
OK

root@xiaodongye-s80:/ws# ./build/bin/test-backend-ops perf -b CPU -o ADD --output sql
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 MUSA devices:
  Device 0: MTT S80, compute capability 2.1, VMM: yes
CREATE TABLE IF NOT EXISTS test_results (
  test_time TEXT,
  backend_name TEXT,
  op_name TEXT,
  op_params TEXT,
  test_mode TEXT,
  supported INTEGER,
  passed INTEGER,
  error_message TEXT,
  time_us REAL,
  flops_per_sec REAL,
  bandwidth_gb_s REAL,
  memory_kb INTEGER,
  n_runs INTEGER
);

INSERT INTO test_results (test_time, backend_name, op_name, op_params, test_mode, supported, passed, error_message, time_us, flops_per_sec, bandwidth_gb_s, memory_kb, n_runs) VALUES ('2025-06-25T03:35:20Z', 'CPU', 'ADD', 'type=f32,ne=[4096,1,1,1],nr=[1,1,1,1]', 'perf', '1', '1', '', '0.666059', '0.000000', '68.732809', '48', '1506960');
INSERT INTO test_results (test_time, backend_name, op_name, op_params, test_mode, supported, passed, error_message, time_us, flops_per_sec, bandwidth_gb_s, memory_kb, n_runs) VALUES ('2025-06-25T03:35:21Z', 'CPU', 'ADD', 'type=f32,ne=[4096,1,1,1],nr=[1,512,1,1]', 'perf', '1', '1', '', '74.107237', '0.000000', '316.573511', '24576', '13680');

root@xiaodongye-s80:/ws# ./build/bin/test-backend-ops perf -b CPU -o ADD --output sql | sqlite3 add.sqlite
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 MUSA devices:
  Device 0: MTT S80, compute capability 2.1, VMM: yes

Edit (added build_commit and build_number for future use):

[Copilot]

Pull Request Overview

This PR adds support for specifying the output format (console or SQL) for the test-backend-ops tool and unifies logging via a new printer interface.

• Introduces a printer interface with concrete implementations for console and SQL output.
• Updates test evaluation functions and the CLI to use the printer for logging.
• Adds new command-line option "--output" along with helper functions to parse the output format.

[JohannesGaessler]

From my end I think these changes to test-backend-ops would be fine but keep in mind that it's an important piece of the project with many stakeholders.

[JohannesGaessler]

TODO

[yeahdongcn]

This was added to record the timestamp (the same as llama-bench), but I haven’t decided how to use it.

[yeahdongcn]

From my end I think these changes to test-backend-ops would be fine but keep in mind that it's an important piece of the project with many stakeholders.

Thanks for pointing that out! I’ll add the recent contributors as reviewers.