Skip to content

Make it possible to customise the assertions inserted for coverage instrumentation #5274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 19, 2020

Conversation

thomasspriggs
Copy link
Contributor

This PR makes it possible to customise the assertions inserted for coverage instrumentation. We have a requirement in a downstream repository to be able to consider coverage goals only to be met when additional criteria are satisfied. The additional configure-ability this PR adds to CBMC allows us to achieve this.

  • Each commit message has a non-empty body, explaining why the change was made.
  • Methods or procedures I have added are documented, following the guidelines provided in CODING_STANDARD.md.
  • The feature or user visible behaviour I have added or modified has been documented in the User Guide in doc/cprover-manual/ No user visible changes.
  • Regression or unit tests are included, or existing tests cover the modified code (in this case I have detailed which ones those are in the commit message).
  • My commit message includes data points confirming performance improvements (if claimed). Non claimed.
  • My PR is restricted to a single feature or bugfix.
  • White-space or formatting changes outside the feature-related changed lines are in commits of their own.

@@ -36,18 +36,30 @@ class cover_instrumenter_baset
{
}

/// The type of function used to make goto_program assertions.
using assertion_factoryt =
std::function<decltype(goto_programt::make_assertion)>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⛏️ I'd prefer an explicit type here

Copy link
Contributor

@chrisr-diffblue chrisr-diffblue Mar 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, there's no documentation here for what type signature the factory must satisfy. So if I wanted to add a new assertion factory, I'd have to reverse engineer the signature of the current one. Not a massive problem, but we should make the type explicit and document it. And as this now becomes part of the API, its an additional reason the contract needs documenting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Will do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -36,18 +36,30 @@ class cover_instrumenter_baset
{
}

/// The type of function used to make goto_program assertions.
using assertion_factoryt =
std::function<decltype(goto_programt::make_assertion)>;
Copy link
Contributor

@chrisr-diffblue chrisr-diffblue Mar 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, there's no documentation here for what type signature the factory must satisfy. So if I wanted to add a new assertion factory, I'd have to reverse engineer the signature of the current one. Not a massive problem, but we should make the type explicit and document it. And as this now becomes part of the API, its an additional reason the contract needs documenting.

That this change compiles shows that this member function is not
specialised in any derived class. This means that we can make further
changes to its signature and implementation without considering derived
classes.
Copy link
Contributor

@chrisr-diffblue chrisr-diffblue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks for the updates - and especially for the extra docs beyond what I asked.

/// The type of function used to make goto_program assertions.
using assertion_factoryt = std::function<
goto_programt::instructiont(const exprt &, const source_locationt &)>;
static_assert(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice :-)

This allows for additional logic to be added to the existing coverage
instrumenters. For example instead of generating an assertion of the
form `assert(guard)` we want to be able to generate an assertion of the
form `assert(!in_region_of_interest || guard)`. These customised forms
of instrumentation will still add assertions in the same locations and
be based on top of the original expressions.
@thomasspriggs thomasspriggs force-pushed the tas/flag_goals branch 2 times, most recently from 223329a to 443e085 Compare March 19, 2020 09:19
@codecov-io
Copy link

Codecov Report

Merging #5274 into develop will decrease coverage by <.01%.
The diff coverage is 93.75%.

Impacted file tree graph

@@            Coverage Diff             @@
##           develop   #5274      +/-   ##
==========================================
- Coverage    67.51%   67.5%   -0.01%     
==========================================
  Files         1170    1170              
  Lines        96289   96287       -2     
==========================================
- Hits         65005   65003       -2     
  Misses       31284   31284
Flag Coverage Δ
#cproversmt2 42.4% <25%> (ø) ⬆️
#regression 64.01% <87.5%> (-0.01%) ⬇️
#unit 31.74% <31.25%> (-0.01%) ⬇️
Impacted Files Coverage Δ
src/goto-instrument/cover.h 100% <ø> (ø) ⬆️
src/goto-instrument/cover_instrument_mcdc.cpp 93.86% <100%> (ø) ⬆️
src/goto-instrument/cover_instrument_decision.cpp 100% <100%> (ø) ⬆️
src/goto-instrument/cover_instrument_other.cpp 95.65% <100%> (-0.19%) ⬇️
src/goto-instrument/cover_instrument.h 92.1% <100%> (-0.21%) ⬇️
src/goto-instrument/cover_instrument_branch.cpp 100% <100%> (ø) ⬆️
src/goto-instrument/cover_instrument_location.cpp 100% <100%> (ø) ⬆️
src/goto-instrument/cover_instrument_condition.cpp 100% <100%> (ø) ⬆️
src/goto-instrument/cover.cpp 84.28% <66.66%> (+0.11%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1002ac5...443e085. Read the comment docs.

@thomasspriggs thomasspriggs merged commit 094fc97 into diffblue:develop Mar 19, 2020
@thomasspriggs thomasspriggs deleted the tas/flag_goals branch March 19, 2020 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants