Skip to content

taw read from file depends on X/Y instead of constant #321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
May 2, 2023

Conversation

remicousin
Copy link
Contributor

I ran into a couple of issues in order to this and what I propose here is probably the start of the discussion rather than the final solution. I am also going to add some comments to point out some specific problems that led me to do some changed.

rr_mrg's and taw's X/Y are the same by design but they respectively are double and single precision. That caused the tiling to create tiles of different sizes (by 1 point), resulting in water balance algo to fail (see more details below -- but water balance algo expects coordinates of same name to be identical througout its inputs).

converting rr_mrg's X/Y to single fixed the tiling problem, but water balance was still failing. Turns out that one of the two sets of X/Ys had many values with tailing ...00001. Yet, there are a series of broadcast used in water balance algo to set the stage for the computation, and so as a result, say latitudes y and y+0.000001 were both broadcasted, leading to wrong objects, of differing sizes not allowing the water balance algo to go through.

Inspired by this conversation, I resorted to override-align taw against rr_mrg. This made the initial precision conversion useless and so I commented out to keep it just for the story-telling here for now.

@remicousin remicousin requested a review from aaron-kaplan April 19, 2023 20:22
@remicousin remicousin self-assigned this Apr 19, 2023
Copy link
Collaborator

@aaron-kaplan aaron-kaplan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine. I don't have a precise explanation for the errors you described, but they both have to do with concurrent access to a global resource by multiple concurrent threads or processes. It might be between python threads (though I don't think so because maproom_monit.py sets threaded=False) or between dask workers. In any case, eliminating the global variable taw is the right solution.

@remicousin remicousin requested a review from aaron-kaplan May 1, 2023 13:48
@remicousin remicousin merged commit d862db5 into master May 2, 2023
@remicousin remicousin deleted the remic/tawfile branch May 2, 2023 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants