-
-
Notifications
You must be signed in to change notification settings - Fork 20
Pass atol
parameter to FITSDiff
#33
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't grok the reference_dir
setup. Is there no way to generate the truth in memory at run time? Seems weird to have to copy the same file over for a new test.
Not sure if this answers your question, but since these tests are testing the machinery that compares generated data to a reference file, I think there has to be a reference file to compare to, even if only trivial data is being generated in this case. And since the reference file name is autogenerated from the test name, I think a new test needs a new copy of the file. I suppose the reference files could be programmatically generated in a temp dir as the tests run, but I'm not sure how to easily pass that temp dir to the I could easily believe I'm missing a better way to do this (or misunderstanding the tests), in this weird context where the testing framework is being used to test the testing framework itself. |
You are probably doing it right. I am not familiar with this package. 🤷 |
I had to re-enable CI in the settings. It was disabled due to inactivity. So I have to close/re-open to kick it off. Please do not be alarmed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks! Could you add an entry in the CHANGES.rst under 0.6?
Done! |
Thanks, all! |
I noticed that the
atol
parameter doesn't get passed toastropy.io.fits.diff.FITSDiff
when the reference file is FITS, and so settingatol
has no effect. (e.g.@pytest.mark.array_compare(file_format='fits', atol=1)
still used anatol
of 0.) This PR adds a test foratol
, ensuresatol
is passed through, and mentions theatol
parameter in the readme alongsidertol
(which was already mentioned).As I was adding the test, I noticed that the existing test for the
rtol
parameter was buggy, in that the generated and reference files both contained only values of 1.6, sortol
could be zero or any other value and the test would still pass. (Maybe the reference files were regenerated at some point, forgetting that in this particular case, the reference file must be different from the generated file to test the tolerance values.) I've modified that test by scaling up the generated values so that the test fails ifrtol
isn't set to a large value, to ensure that the value ofrtol
is being used.