python3 test failures

[juliantaylor]

on ubuntu 13.04 several tests fail with 3.3 while they suceed with python2
13.04 has python 3.3 and numpy 1.7
current pandas head 61766ec

======================================================================
FAIL: test_quoting (pandas.io.tests.test_parsers.TestParsers)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jtaylor/tmp/pandas-0.9.1/build/lib.linux-x86_64-3.3/pandas/io/tests/test_parsers.py", line 528, in test_quoting
    sep='\t')
AssertionError: Exception not raised by read_table

======================================================================
FAIL: test_cant_compare_tz_naive_w_aware (pandas.tseries.tests.test_timeseries.TestTimestamp)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jtaylor/tmp/pandas-0.9.1/build/lib.linux-x86_64-3.3/pandas/tseries/tests/test_timeseries.py", line 2349, in test_cant_compare_tz_naive_w_aware
    self.assertRaises(Exception, a.__eq__, b.to_pydatetime())
AssertionError: Exception not raised by __eq__

----------------------------------------------------------------------

======================================================================
FAIL: test_more_flexible_frame_multi_function (__main__.TestGroupBy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jtaylor/tmp/pandas-0.9.1/build/lib.linux-x86_64-3.3/pandas/tests/test_groupby.py", line 1909, in test_more_flexible_frame_multi_function
    assert_frame_equal(result, expected)
  File "/home/jtaylor/tmp/pandas-0.9.1/build/lib.linux-x86_64-3.3/pandas/util/testing.py", line 167, in assert_frame_equal
    assert(left.columns.equals(right.columns))
AssertionError

----------------------------------------------------------------------

======================================================================
ERROR: test_yahoo (pandas.io.tests.test_yahoo.TestYahoo)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/tests/test_yahoo.py", line 25, in test_yahoo
    pd.DataReader("F", 'yahoo', start, end)['Close'][-1],
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/data.py", line 58, in DataReader
    retry_count=retry_count, pause=pause)
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/data.py", line 149, in get_data_yahoo
    parse_dates=True)[::-1]
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/parsers.py", line 364, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/parsers.py", line 195, in _read
    return parser.read()
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/parsers.py", line 592, in read
    ret = self._engine.read(nrows)
  File "/tmp/pandas/build/lib.linux-x86_64-3.3/pandas/io/parsers.py", line 844, in read
    data = self._reader.read(nrows)
  File "parser.pyx", line 597, in pandas._parser.TextReader.read (pandas/src/parser.c:5342)
  File "parser.pyx", line 619, in pandas._parser.TextReader._read_low_memory (pandas/src/parser.c:5562)
  File "parser.pyx", line 668, in pandas._parser.TextReader._read_rows (pandas/src/parser.c:6143)
  File "parser.pyx", line 655, in pandas._parser.TextReader._tokenize_rows (pandas/src/parser.c:6027)
  File "parser.pyx", line 1385, in pandas._parser.raise_parser_error (pandas/src/parser.c:14807)
pandas._parser.CParserError: Error tokenizing data. C error: Expected 7 fields in line 106, saw 3

test_more_flexible_frame_multi_function (__main__.TestGroupBy) ... > /home/jtaylor/tmp/pandas-0.9.1/build/lib.linux-x86_64-3.3/pandas/util/testing.py(167)assert_frame_equal()
-> assert(left.columns.equals(right.columns))
(Pdb) p left.columns

[(D, mean), (D, std), (C, mean), (C, std)]
(Pdb) p right.columns
MultiIndex
[(C, mean), (C, std), (D, mean), (D, std)]

this one is random probably related to hash randomization

the yahoo one might be due to missing internet connection?

[juliantaylor]

another one with matplotlib installed

======================================================================
ERROR: test_bootstrap_plot (pandas.tests.test_graphics.TestSeriesPlots)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pandas/tests/test_graphics.py", line 158, in test_bootstrap_plot
    _check_plot_works(bootstrap_plot, self.ts, size=10)
  File "/usr/lib/python3/dist-packages/pandas/tests/test_graphics.py", line 625, in _check_plot_works
    ret = f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/pandas/tools/plotting.py", line 376, in bootstrap_plot
    samplings = [random.sample(data, size) for _ in range(samples)]
  File "/usr/lib/python3/dist-packages/pandas/tools/plotting.py", line 376, in <listcomp>
    samplings = [random.sample(data, size) for _ in range(samples)]
  File "/usr/lib/python3.3/random.py", line 298, in sample
    raise TypeError("Population must be a sequence or set.  For dicts, use list(d).")
TypeError: Population must be a sequence or set.  For dicts, use list(d).

seems like random.sample does not take np arrays, its no abc.Sequence (which might be a bug in numpy?)

I can reproduce Intermittent failures on py3.3 with numpy 1.7.0rc1.dev-3a52aa0' (git master).
I cannot reproduce failures on py3.2 with numpy 1.7.
I cannot reproduce failures on py2.7 with numpy 1.7.

test_dict sometimes also fails.

[juliantaylor]

the random failures can only be seen with python 3.3 and python 2.7.3, hash randomization does not exist earlier
with python2.7.3 it might have to explicitly enable it

verified. can reproduce on 2.7.3 with hash randomization enabled.
here's the announcement.
The change mitigates an algorithmic complexity attack.

[juliantaylor]

actually no, the python "fix" is completely ineffective (see issue14621)
nevertheless relying on ordering of dicts was never supported an should thus be fixed

afaict, the seed is only randomized between sessions, traversal order within a given process is
determinstic.
Of course relying on order should be fixed, but I'm not sure why the problem is surfacing,
I don't see any state being saved across runs.

dear lord, the order of keys is this form of construction is not well-determined:

OrderedDict(foo=0,bar=1)

[wesm]

Python 3.3 with NumPy git master passing cleanly for me now. Could you give it a shot with your environment and let me know so I can close this issue?

[juliantaylor]

I still get this failure (f4b5cbb):

======================================================================
FAIL: test_quoting (pandas.io.tests.test_parsers.TestPythonParser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/pandas/build/lib.linux-x86_64-3.3/pandas/io/tests/test_parsers.py", line 462, in test_quoting
    sep='\t')
AssertionError: Exception not raised by read_table

----------------------------------------------------------------------