Parametrized type annotation raises error

[adamklein]

In cloudpickle 1.2.1

In [6]: import typing                                   
                                                                                            
In [7]: def myfun(x: typing.Union[int, list]):          
   ...:     return x           
   ...:                                                                                                                       
                                                   
In [8]: import cloudpickle                                   
                           
In [9]: cloudpickle.dumps(myfun) 

results in

PicklingError: Can't pickle typing.Union[int, list]: it's not the same object as typing.Union

whereas in 1.1.1

In [1]: import typing

In [2]: def myfun(x: typing.Union[int, list]):
   ...:     return x
   ...: 

In [3]: import cloudpickle

In [4]: cloudpickle.dumps(myfun)
Out[4]: b'\x80\x04\x95[\x01\x00\x00\x00\x00\x00\x00\x8c\x17cloudpickle.cloudpickle\x94\x8c\x0e_fill_function\x94\x93\x94(h\x00\x8c\x0f_make_skel_func\x94\x93\x94h\x00\x8c\r_builtin_type\x94\x93\x94\x8c\x08CodeType\x94\x85\x94R\x94(K\x01K\x00K\x01K\x01KCC\x04|\x00S\x00\x94N\x85\x94)\x8c\x01x\x94\x85\x94\x8c\x1e<ipython-input-2-c47f6a8f7a19>\x94\x8c\x05myfun\x94K\x01C\x02\x00\x01\x94))t\x94R\x94J\xff\xff\xff\xff}\x94(\x8c\x0b__package__\x94N\x8c\x08__name__\x94\x8c\x08__main__\x94u\x87\x94R\x94}\x94(\x8c\x07globals\x94}\x94\x8c\x08defaults\x94N\x8c\x04dict\x94}\x94\x8c\x0eclosure_values\x94N\x8c\x06module\x94h\x16\x8c\x04name\x94h\x0f\x8c\x03doc\x94N\x8c\x08qualname\x94h\x0f\x8c\nkwdefaults\x94NutR.'

[cicdw]

I'm guessing you are running a Python version of 3.5 or 3.6. I ran into this problem a while back and made this PR to aggressively remove type annotations from earlier versions of Python altogether.

It looks like this newer PR updates those changes to include annotations for 3.5-3.6, and was included in the 1.2.0 release of cloudpickle.

The ultimate problem is that certain types are simply not pickleable in 3.5 / 3.6, leading to inconsistent serialization behavior depending on the type annotations someone uses.

I'm not a core cloudpickle dev or anything, and don't necessarily have a strong opinion here, but thought I'd provide some context.

[pierreglaser]

@cicdw thanks for this clarification, I was not aware of #197 . It seems like #276 was a mistake -- I'm going to revert it, and make it clear that type hints are not supported for Python3.5-3.6.

[ogrisel]

#299 was merged and released in cloudpickle 1.2.2.