-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Add numpy SPECTRL2 implementation #1062
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I think this is ready for review whenever folks get a chance. For the sake of ease of testing I've aligned all the inconsistencies (see table in original comment) with I created a new Thanks! |
I'm in favor the new |
Nice work. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, but I didn't compare the implementation to the reference.
pvlib/spectrum.py
Outdated
|
||
Parameters | ||
---------- | ||
surface_tilt : float or numpy array |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with numpy array but worth considering numpy.ndarray for precision.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you need to say something about the dimensions of the array arguments?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll volunteer to check this against the reference.
Is there a reason to exclude pandas.Series from the input types?
pvlib/spectrum.py
Outdated
|
||
Parameters | ||
---------- | ||
surface_tilt : float or numpy array |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you need to say something about the dimensions of the array arguments?
pvlib/spectrum.py
Outdated
# Is = Ir + Ia + Ig # Eq 3-1 | ||
Is = (Ir + Ia + Ig) * Cs # Eq 3-1 | ||
|
||
# calculate spectral irradiance on a tilted surface, Eq 3-18 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could transposition be a separate function? Or could the use be expected to do it using the existing transposition functions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could be a separate function, but I'm inclined to implement the complete model.
If users want to do their own transposition, is there any issue with them passing surface_tilt=0
and transposing afterwards (other than the inefficiency of doing an unnecessary transposition in this function)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a reason to exclude pandas.Series from the input types?
Probably, but I'll have to remind myself what it is. I'd guess to make broadcasting easier. Whatever the reason, re-wrapping in a Series before returning would probably not be an issue.
pvlib/spectrum.py
Outdated
# Is = Ir + Ia + Ig # Eq 3-1 | ||
Is = (Ir + Ia + Ig) * Cs # Eq 3-1 | ||
|
||
# calculate spectral irradiance on a tilted surface, Eq 3-18 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It could be a separate function, but I'm inclined to implement the complete model.
If users want to do their own transposition, is there any issue with them passing surface_tilt=0
and transposing afterwards (other than the inefficiency of doing an unnecessary transposition in this function)?
I'm thinking of users who would bring e.g. |
Inputs now allowed to be To-do items:
|
This module is already quite big isn't it? What about creating |
Regarding the conversion from Series to arrays, I find that numpy usually plays pretty well with Series, so I would be interested to learn where it causes problems for you. Maybe you don't really need to convert them all? |
pvlib/spectrum.py
Outdated
several ways from the original report [1]_. The report itself also has | ||
a few differences between the in-text equations and the code appendix. | ||
The list of known differences is shown below. Note that this | ||
implementation follows ``spectrl2_2.c``. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you happen to compare with the Excel version from NREL as well?
If there are two distinct variations, we may give the user the choice as we have done for other models.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't compare with the Excel version in any meaningful way. I agree it would be nice to be able to switch between the two (if there is indeed a difference), but I hesitate to add additional complexity to this PR. Future work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe someone at NREL would know if the C code or the Excel version is in some sense authoritative. I agree with deferring any kind of switch.
I support this, even though a pvlib SMARTS won't appear any time soon. |
Thanks @adriesse for the numpy lesson :) The numpy vs pandas issue is that I want to cleanly produce the outer product of two vectors where one axis indexes time and the other indexes wavelength, for example: N = 2
airmass = pd.Series(range(N), index=pd.date_range('2019-01-01', freq='h', periods=N))
vapor_coeff = np.array([1, 2, 3])[:, np.newaxis]
airmass * vapor_coeff # fails, but airmass.values * vapor_coeff works and returns an array with shape (3, 2) Is there a nice way to get this behavior when one is a Series? |
I guess I haven't run into this exact situation often. When one argument is a Series, the A clean way to get the desired behavior would be to use I won't hold anything up though. I just provide my thoughts on what is overall a very good contribution to pvlib! |
('water_vapor_absorption', 'float64'), | ||
('ozone_absorption', 'float64'), | ||
('mixed_absorption', 'float64'), | ||
])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There seems to be little benefit to putting the constants into a structured array. How about 5 constant column vectors like this?
WAVELENGTH = np.transpose(np.array([[
300.0, 305..0 etc.
]]))
A side benefit is that one can recognize the use of the constants in the code more easily because they are CAPS.
Of course if you do end up using np.outer
the constants can remain row vectors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work @kanderso-nrel easily traceable to the reference and well-organized.
pvlib/spectrum/spectrl2.py
Outdated
wavelength = _SPECTRL2_COEFFS['wavelength'][:, np.newaxis] | ||
spectrum_et = _SPECTRL2_COEFFS['spectral_irradiance_et'][:, np.newaxis] | ||
|
||
optical_thickness = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use pvlib.atmosphere.angstrom_aod_at_lambda
, see
pvlib-python/pvlib/atmosphere.py
Line 620 in 6e936f5
def angstrom_aod_at_lambda(aod0, lambda0, alpha=1.14, lambda1=700.0): |
pvlib/spectrum/spectrl2.py
Outdated
ALG = np.log(1 - aerosol_asymmetry_factor) # Eq 3-14 | ||
BFS = ALG * (0.0783 + ALG * (-0.3824 - ALG * 0.5874)) # Eq 3-13 | ||
AFS = ALG * (1.459 + ALG * (0.1595 + ALG * 0.4129)) # Eq 3-12 | ||
cosZ = cosd(apparent_zenith) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
duplicate line
Co-authored-by: Cliff Hansen <[email protected]>
FYI I updated the comparison notebook using the current HEAD of this branch. No significant changes from the original plots at the top of this thread. https://gist.github.com/kanderso-nrel/0dc7e7ebff7b5a9c13b52f8b48eb71ee |
What's a good and convenient source for precipitable_water, ozone, aerosol_turbidity_500nm data? Preferably multiple values per day, USA and/or worldwide. |
ECMWF, try get_ecmwf_macc |
I will! There's a lot of stuff in iotools. Where have I been? |
Are we ready to merge this? |
I never responded to @adriesse's last suggestion -- I had the vague notion that someone might appreciate the constants already being in tabular form so that they can easily export it for another purpose. Other than that, you're right that columns only get used individually, so not much need for a structured array. For this and using |
I liked the structured array when I first read the code, but I also see @adriesse's point. I don't think there's a wrong choice here so I'm going to go ahead and merge. Thanks all! |
Closes #xxxxdocs/sphinx/source/api.rst
for API changes.docs/sphinx/source/whatsnew
for all changes. Includes link to the GitHub Issue with:issue:`num`
or this Pull Request with:pull:`num`
. Includes contributor name and/or GitHub username (link with:ghuser:`user`
).Additional to-do:
Calculating the spectral distribution for a single timestamp takes under a millisecond on my laptop. I've been comparing against the solar_utils wrapper around the NREL C implementation (thanks @mikofski for making it so easy to use!) and have discovered several discrepancies between the equations in the report, the report code appendix, and the C implementation. It doesn't help that the C implementation uses its own (non-SPA) solar position calculator. By matching the python code to the C implementation for each of the differences, the difference in output spectra is small enough for me to start blaming the accumulated error from the C implementation's single-precision floats:
Discrepancies: