Pickling Annotations can fail

[cicdw]

Now that cloudpickle is including annotations, there are certain typing classes that fail to pickle in versions of python < 3.7 that are now raising errors when included as annotations:

## python 3.6.6

import typing
import cloudpickle

tt = typing.Union[str, typing.List[str]]
cloudpickle.dumps(tt, protocol=cloudpickle.DEFAULT_PROTOCOL) # always fails
# _pickle.PicklingError: Can't pickle typing.Union[str, typing.List[str]]: it's not the same object as typing.Union

def f(x: tt) -> tt:
    return x

## passes in cloudpickle==0.5.3 because type annotations were ignored
## but fails in 0.5.4+
cloudpickle.dumps(f, protocol=cloudpickle.DEFAULT_PROTOCOL)

All of this works fine in python 3.7

Related: python/typing#511

[jamesstidard]

I'm getting some strange behaviour related to this. Sorry to open the typing can of worms.

I'm using Python 3.7 and cloudpickle==0.5.6 but have been using cloudpickle via distributed==1.23.2. Which I reported over at: dask/distributed#2291

The above code sample works for me. However, the following does not:

from typing import Optional
from distributed import Client

def f2() -> Optional[int]:
    return 1

def f1() -> int:
    return 1

def main():
    client = Client()
    print(client.submit(f1).result())  # works. prints 1
    print(client.submit(f2).result())  # fail. raises
    client.close()

if __name__ == '__main__':
    main()

Appologies that this includes a dask dependancy. I tried to repoduce with the above code snippet but I don't get an exception:

from typing import Optional
import cloudpickle

cloudpickle.dumps(Optional[int], protocol=cloudpickle.DEFAULT_PROTOCOL)

I'm a little bemused to where the problem is coming from now with those results.

[ogrisel]

Can you please open a new issue?

[jamesstidard]

Sure thing, thanks. I wasn't sure the etiquette (like if this issue should be reopened or not.

New issue is here: #209