Replaced PANDAS_DATETIMEUNIT with NPY_DATETIMEUNIT

[WillAyd]

progress towards #21852

• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

Working on my C-fu as well. There may be a better way to do this (ex: even adding NPY_DATETIMEUNIT to Cython/includes) but figured I'd offer this up for review

@jbrockmendel

[WillAyd]

Not sure if there is a better way to do this declaration. Right now I copy/pasted in the three files where it is need but is there a shared place to put that for shared access across the modules?

[jbrockmendel]

Since it is in np_datetime.pxd, it can be cimported from there.

[jbrockmendel]

Since this is done in np_datetime.pxd, I'd just cimport from there rather than copying this.

[jbrockmendel]

Aside from that note on repeated imports, this looks like a step in the right direction. The other super-easy change-over is pandas_datetimestruct.

[jbrockmendel]

This file isn't linted, but if it were, I think it would want two lines before the cdef class

[codecov]

Codecov Report

Merging #21863 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #21863   +/-   ##
=======================================
  Coverage   91.92%   91.92%           
=======================================
  Files         160      160           
  Lines       49906    49906           
=======================================
  Hits        45875    45875           
  Misses       4031     4031

Flag	Coverage Δ
#multiple	90.3% <ø> (ø)	⬆️
#single	42.11% <ø> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5d0daa0...8aa70bb. Read the comment docs.

[jbrockmendel]

I still can’t find a way to access functions in numpy’s _datetime.h. Not even sure what is possible in C-land. @chris-b1 are you the right person to ask about this?

[WillAyd]

@jbrockmendel not sure what you are looking for out of that but it doesn't appear that header is included in numpy's core/include directory along with the rest of the headers. You might have to mess around with setup.py if you are trying to include header files outside of that location:

pandas/setup.py

Line 128 in 5d0daa0

numpy_incl = pkg_resources.resource_filename('numpy', 'core/include')

Though that most likely violates a lot of best practices

[WillAyd]

Though looking at it I don't believe src/multiarray/_datetime.h is included as part of your numpy installation in site-packages so may not be possible with how the code is currently structured (could be wrong)

[jbrockmendel]

not sure what you are looking for out of that but it doesn't appear that header is included in numpy's core/include directory along with the rest of the headers

The hope was that it is somehow passed through via an #include in one of the files that is part of core/include. After all, numpy itself somehow manages to call these functions

[chris-b1]

Yeah, even if it were possible I don't think we should be depending on private API of numpy - presumably why some bits (e.g. datetime string parsing) were vendored in the first place.

[jreback]

cc @charris

do you recall if these existed in numpy < 1.6 ?

[WillAyd]

AFAICT goes at least back to 1.4

[jbrockmendel]

AFAICT the code was vendored b/c at the time pandas supported older numpy versions that did not yet have datetime64. Also because pandas handles timezones differently in np_datetime_strings. It'd still be nice to not re-implement the large portion that is identical.

It would also be super-convenient to be able to use the numpy versions in src/ujson/python/obj2JSON.c, as that would let us de-couple the remaing src/datetime code.

[jreback]

thanks @WillAyd and @jbrockmendel for the comments. Agree that the vendored code, just wanted the rationale documented here :>