Use mock sample in tests

[williambdean]

Saving about 50s in the test suite by mocking pm.sample

---

📚 Documentation preview 📚: https://causalpy--489.org.readthedocs.build/en/489/

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.59%. Comparing base (632c457) to head (c3b6423).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #489   +/-   ##
=======================================
  Coverage   94.59%   94.59%           
=======================================
  Files          28       28           
  Lines        2053     2053           
=======================================
  Hits         1942     1942           
  Misses        111      111           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[williambdean]

66s -> 16s test suite runs. nice

[williambdean]

Doing for doctest would be greater reduction of time

[williambdean]

6m31s down to 1m58s

[drbenvincent]

This is pretty cool - thanks for the contribution.

On my local machine...

• make test runs in 24 seconds (down from 50 seconds on main) ✅
• make doctest speed is unchanged❓

I guess I just have some questions to make sure I understand this

• mock_sample_for_doctest kind of looks transparent in that we're overriding pm.sample with pymc.testing.mock_sample. Though that's not working or doing much?
• mock_pymc_sample is provided as an input to tests (like test_did for example) but it's never used anywhere in those test functions. So this is a bit confusing. Does this make the use of sample_kwargs irrelevant now?

[williambdean]

Noticing that too. Weird thing is pytest --doctest-modules seems to work for me but pytest --doctest-modules causalpy/ does not

[williambdean]

mock_pymc_sample is a fixture that doesn't have to be used directly in the test function since it replaces pm.sample but then puts back original at the end of the test. Just like many of those tests are not using pm.sample directly (i.e. fit method), the fixture doesn't have to be used directly.

[williambdean]

So the pm.sample is being mocked in the CI/CD tests. We can see with the drop in the run times. Need to figure out why calling pytest --doctest-modules causalpy/ doesn't work when pytest --doctest-modules does. Is this a blocker for merging this PR?

EDIT: Figured out with 990104d. The conftest file was not being picked up based on the test structure