Skip to content

tmy with coerce year fails if February contains leap year, eg Greensboro-NC #865

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
mikofski opened this issue Jan 24, 2020 · 0 comments · Fixed by #866
Closed

tmy with coerce year fails if February contains leap year, eg Greensboro-NC #865

mikofski opened this issue Jan 24, 2020 · 0 comments · Fixed by #866
Assignees
Labels
Milestone

Comments

@mikofski
Copy link
Member

mikofski commented Jan 24, 2020

Describe the bug
Try to read Greensboro TMY3 with coerce_year=1990

raises the follow traceback:

In [1]: import pvlib

In [2]: greensboro = pvlib.iotools.read_tmy3('723170TYA.CSV', coerce_year=1990)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in converter(*date_cols)
   3280                 result = tools.to_datetime(
-> 3281                     date_parser(*date_cols), errors="ignore", cache=cache_dates
   3282                 )

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pvlib\iotools\tmy.py in <lambda>(*x)
    194         parse_dates={'datetime': ['Date (MM/DD/YYYY)', 'Time (HH:MM)']},
--> 195         date_parser=lambda *x: _parsedate(*x, year=coerce_year),
    196         index_col='datetime')

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pvlib\iotools\tmy.py in _parsedate(ymd, hour, year)
    215     # and dateutil's inability to handle that.
--> 216     offset_hour = int(hour[:2]) - 1
    217     offset_datetime = '{} {}:00'.format(ymd, offset_hour)

TypeError: only size-1 arrays can be converted to Python scalars

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in converter(*date_cols)
   3291                             parser=date_parser,
-> 3292                             dayfirst=dayfirst,
   3293                         ),

pandas/_libs/tslibs/parsing.pyx in pandas._libs.tslibs.parsing.try_parse_dates()

pandas/_libs/tslibs/parsing.pyx in pandas._libs.tslibs.parsing.try_parse_dates()

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pvlib\iotools\tmy.py in <lambda>(*x)
    194         parse_dates={'datetime': ['Date (MM/DD/YYYY)', 'Time (HH:MM)']},
--> 195         date_parser=lambda *x: _parsedate(*x, year=coerce_year),
    196         index_col='datetime')

TypeError: _parsedate() missing 1 required positional argument: 'hour'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-2-6078abbb94dd> in <module>
----> 1 greensboro = pvlib.iotools.read_tmy3('723170TYA.CSV', coerce_year=1990)

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pvlib\iotools\tmy.py in read_tmy3(filename, coerce_year, recolumn)
    194         parse_dates={'datetime': ['Date (MM/DD/YYYY)', 'Time (HH:MM)']},
    195         date_parser=lambda *x: _parsedate(*x, year=coerce_year),
--> 196         index_col='datetime')
    197
    198     if recolumn:

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)
    683         )
    684
--> 685         return _read(filepath_or_buffer, kwds)
    686
    687     parser_f.__name__ = name

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
    461
    462     try:
--> 463         data = parser.read(nrows)
    464     finally:
    465         parser.close()

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in read(self, nrows)
   1152     def read(self, nrows=None):
   1153         nrows = _validate_integer("nrows", nrows)
-> 1154         ret = self._engine.read(nrows)
   1155
   1156         # May alter columns / col_dict

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in read(self, nrows)
   2132             data = {k: v for k, (i, v) in zip(names, data)}
   2133
-> 2134             names, data = self._do_date_conversions(names, data)
   2135             index, names = self._make_index(data, alldata, names)
   2136

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in _do_date_conversions(self, names, data)
   1883                 self.index_names,
   1884                 names,
-> 1885                 keep_date_col=self.keep_date_col,
   1886             )
   1887

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in _process_date_conversion(data_dict, converter, parse_spec, index_col, index_names, columns, keep_date_col)
   3355
   3356             _, col, old_names = _try_convert_dates(
-> 3357                 converter, colspec, data_dict, orig_names
   3358             )
   3359

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in _try_convert_dates(parser, colspec, data_dict, columns)
   3388     to_parse = [data_dict[c] for c in colnames if c in data_dict]
   3389
-> 3390     new_col = parser(*to_parse)
   3391     return new_name, new_col, colnames
   3392

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\parsers.py in converter(*date_cols)
   3295                     )
   3296                 except Exception:
-> 3297                     return generic_parser(date_parser, *date_cols)
   3298
   3299     return converter

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pandas\io\date_converters.py in generic_parser(parse_func, *cols)
     36     for i in range(N):
     37         args = [c[i] for c in cols]
---> 38         results[i] = parse_func(*args)
     39
     40     return results

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pvlib\iotools\tmy.py in <lambda>(*x)
    193         csvdata, header=0,
    194         parse_dates={'datetime': ['Date (MM/DD/YYYY)', 'Time (HH:MM)']},
--> 195         date_parser=lambda *x: _parsedate(*x, year=coerce_year),
    196         index_col='datetime')
    197

~\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\pvlib\iotools\tmy.py in _parsedate(ymd, hour, year)
    219     true_date = offset_date + dateutil.relativedelta.relativedelta(hours=1)
    220     if year is not None:
--> 221         true_date = true_date.replace(year=year)
    222     return true_date
    223

ValueError: day is out of range for month

To Reproduce
Steps to reproduce the behavior:

  1. download tmy3 for greensboro
  2. start python import pvlib
  3. execute pvlib.iotools.read_tmy3('723170TYA.CSV', coerce_year=1990)
  4. See error

Also, seems to works fine if coerce_year=None, but actually raises a different silent issue:
image
Note the February 29th at midnite in between Feb. 28th and March 1st

Versions:

  • pvlib.__version__: 0.7.1
  • pandas.__version__: 0.25.3
  • python: 3.7.6
  • numpy: 1.18.1

Additional context
Look at cells A1416 around end of February:

2/28/1996 22:00
2/28/1996 23:00
2/28/1996 24:00:00
3/1/1990 1:00
3/1/1990 2:00
3/1/1990 3:00

PS you can paste from Excel into GitHub and it GH automatically formats the data as MD table, nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants