Skip to content

Flaky test TestSimulator::test_one_gaussian #5467

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 15, 2022

Conversation

LukeLB
Copy link
Contributor

@LukeLB LukeLB commented Feb 13, 2022

Flaky test was being caused by sampling error thus threshold has been changed from 0.1 to 0.15 to reduce the probability of failure. Fixes #5409.

Depending on what your PR does, here are a few things you might want to address in the description:

@LukeLB LukeLB changed the title Flaky test TestSimulator::test_one_gaussian #5409 Flaky test TestSimulator::test_one_gaussian Feb 13, 2022
@codecov
Copy link

codecov bot commented Feb 13, 2022

Codecov Report

Merging #5467 (d367385) into main (d018cba) will increase coverage by 0.02%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #5467      +/-   ##
==========================================
+ Coverage   81.39%   81.41%   +0.02%     
==========================================
  Files          82       82              
  Lines       14218    14218              
==========================================
+ Hits        11573    11576       +3     
+ Misses       2645     2642       -3     
Impacted Files Coverage Δ
pymc/parallel_sampling.py 87.70% <0.00%> (+0.99%) ⬆️

@ricardoV94
Copy link
Member

Should this assert below also be adjusted or is the distribution more narrow there?

https://github.com/pymc-devs/pymc/blob/a04e49007e0c87f8f70542efdb02cd0bdf5db12f/pymc/tests/test_smc.py#L321

@LukeLB
Copy link
Contributor Author

LukeLB commented Feb 14, 2022

Should this assert below also be adjusted or is the distribution more narrow there?

https://github.com/pymc-devs/pymc/blob/a04e49007e0c87f8f70542efdb02cd0bdf5db12f/pymc/tests/test_smc.py#L321

Just ran this and from 5000 samples 0% pass the 0.1 threshold so I think we're safe here. Graph below.
image

Copy link
Member

@ricardoV94 ricardoV94 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for this work!

@ricardoV94 ricardoV94 merged commit 4210431 into pymc-devs:main Feb 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky test TestSimulator::test_one_gaussian
2 participants