Skip to content

dtype control for corrcoef #17395

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
orenbenkiki opened this issue Sep 29, 2020 · 2 comments · Fixed by #17456
Closed

dtype control for corrcoef #17395

orenbenkiki opened this issue Sep 29, 2020 · 2 comments · Fixed by #17456

Comments

@orenbenkiki
Copy link

Feature

I am computing a cross-correlation matrix of 150K * 150K matrix of float32.

Numpy insists on making the results dtype be float64. This causes the amount of memory to be "too much".

I would debate that default, but changing it would probably be too disruptive (not backward compatible).

However, how about a dtype parameter for corrcoef?

In general I would have liked to see a dtype parameter "everywhere" to override the float64 default...
@lschwetlick lschwetlick mentioned this issue Oct 5, 2020
Merged
@lschwetlick
Copy link
Contributor

lschwetlick commented Oct 5, 2020 •
edited
Loading

Looks like corrcoef relies on cov in which deliberately casts to float64.

I made an experimental PR (#17456) for this, and it seems to work without much of a hitch.

[This is my first PR to numpy so I'm happy to get any feedback you might have!]

@orenbenkiki
Copy link
Author

Wow, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

ENH: add dtype option to numpy.lib.function_base.cov and corrcoef lschwetlick/numpy
2 participants
@orenbenkiki @lschwetlick