BUG: TimeSeries compat from < 0.13

[jreback]

fixes a pickle compat issue for < 0.13 when presented with a TimeSeries (was an oversight on the initial PR).

additional tests as well

[jreback]

@jtratner @cpcloud

can you guys have a look at this under py3. There are some 'errors' when I try to read a py2.7 in the compatibility mode (e.g. when I have to 'fake' a ndarray sub-class to make it work). Works fine for py2 reading py2 and py3 reading py3. But get a very strange error when reading py2 with py3.

It just prints the output for now.

Try this (you can also drop into the debugger at the print statement as well)

[goat-jreback-~/pandas] nosetests-3.2 build/lib.linux-x86_64-3.2/pandas/io/tests/test_pickle.py -s

(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x456b8c0>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.10.1/AMD64_windows_2.7.3.pickle : initializing object must be a string
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x456b908>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.10.1/x86_64_linux_2.7.3.pickle : initializing object must be a string
./mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.11.0/test.pickle : 'numpy.int64' object has no attribute 'items'
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x457e3b0>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.11.0/x86_64_linux_2.7.3.pickle : initializing object must be a string
.(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x45c4ef0>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.12.0/0.12.0_i686_linux_2.7.3.pickle : initializing object must be a string
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x45c4f80>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.12.0/0.12.0_x86_64_linux_2.7.3.pickle : initializing object must be a string
(<class 'TypeError'>, TypeError('initializing object must be a string',), <traceback object at 0x45c4b48>)
<built-in function scalar> (dtype('float64'), '\x00\x00\x00\x00\x00\x00ø\x7f')
/mnt/home/jreback/pandas/build/lib.linux-x86_64-3.2/pandas/io/tests/data/legacy_pickle/0.12.0/x86_64_linux_2.7.3.pickle : initializing object must be a string
..
----------------------------------------------------------------------
Ran 4 tests in 0.125s

[jtratner]

I'll definitely take a look...

[jtratner]

@jreback the test cases are setup now such that they never fail :P (you need to add a raise after the print exception part or something...

[jreback]

that was on purpose, so u could look

any idea about the unpickle error?

[jtratner]

ah okay - no I have no idea, I was trying to grep/grin around in the Python source to find what raises the error and I couldn't find anything with 'initializing object'. Any ideas on where it's raised?

[jreback]

yep that's the problem
it's unpicking a np.dtype('float'64) object
but for some reason it has a weird string encoding - cant see any reason for that

[jtratner]

maybe you're overriding some part that sets an encoding internally in the pickle module? Also, have you tried unpickling but removing pickle_compat? (do you still see the 'initialize string' error)?

[jreback]

I first try regular unpickle (this is the try_read bizness in io/pickle.py, then fallback to the compat decoding, then try encoding in the same way.

This 'fails' on the final try, e.g. using pickle_compat with the encoding. AFAIK I am doing it correctly. for example. in 3.3 it correctly unpickles a 3.3 from 0.11.

Its just the py2 with pickle_compat that is failing with something I am not even touching.....

[jtratner]

Whee... Did 3.3 unpickle a Python 2 Series pickle before you did the Series
refactor?

[jreback]

Found it.

(Pdb) p args[1]
'\x00\x00\x00\x00\x00\x00ø\x7f'
(Pdb) p func(args[0],args[1].encode(self.encoding))
nan

I think this might be a numpy/python bug. args[0] here is a np.dtype('float64') object and the value (args[1]) is a string. But the dtype is expecting bytes. So I just catch the error and reencode the strings and it works.

really odd, like the bytes were magically converted to a string w/o properly decoding. (and the result is a nan!)