Skip to content

Commit ec34e01

Browse files
authored
Handle errors thrown from benchcomp column exprs (rust-lang#3145)
Prior to this commit, errors thrown when evaluating the text of a benchcomp extra column would crash benchcomp. This could happen, for example, if a column tries to compare an old variant with a new one, but no data for the old variant exists, as seen in this run: https://github.com/model-checking/kani/actions/runs/8700040930/job/23859607740 Forcing the user to do error handling in the column text would make the text even more unwieldy than it already is, so this commit makes the column text evaluate to **<ERROR>** if an exception is raised during evaluation. By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.
1 parent 7b08d7f commit ec34e01

File tree

1 file changed

+14
-1
lines changed

1 file changed

+14
-1
lines changed

tools/benchcomp/benchcomp/visualizers/__init__.py

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -358,7 +358,20 @@ def _add_extra_columns(self, metrics):
358358
for bench, variants in benches.items():
359359
tmp_variants = dict(variants)
360360
for column in columns:
361-
variants[column["column_name"]] = column["text"](tmp_variants)
361+
if "column_name" not in column:
362+
logging.error(
363+
"A column specification for metric %s did not "
364+
"contain a column_name field. Each column should "
365+
"have a column name and column text", metric)
366+
sys.exit(1)
367+
try:
368+
variants[column["column_name"]] = column["text"](tmp_variants)
369+
except BaseException:
370+
# This may be reached when evaluating the column text
371+
# throws an exception. The column text is written in a
372+
# YAML file and is typically a simple lambda so can't
373+
# contain sophisticated error handling.
374+
variants[column["column_name"]] = "**ERROR**"
362375

363376

364377
@staticmethod

0 commit comments

Comments
 (0)