cloudpickle breaks dill deserialization across servers. #82
Description
Following up on this issue on Stackoverflow:
In a nutshell, with Python 3.5:
Server A imports cloudpickle this causes types.ClassType to become defined.
>>> import types
>>> dir(types)
['BuiltinFunctionType',
'BuiltinMethodType',
'ClassType',
'CodeType',
...
]
Server B does not import cloudpickle, so types.ClassType is left undefined.
>>> import types
>>> dir(types)
['BuiltinFunctionType',
'BuiltinMethodType',
'CodeType',
...
]
Objects which are serialized in server A also seem to serialize a reference to ClassType. Then, when they are deserialized on server B, we encounter the following error:
Traceback (most recent call last):
File "/home/streamsadmin/git/streamsx.topology/test/python/topology/deleteme2.py", line 40, in <module>
a = dill.loads(base64.b64decode(a.encode()))
File "/home/streamsadmin/anaconda3/lib/python3.5/site-packages/dill/dill.py", line 277, in loads
return load(file)
File "/home/streamsadmin/anaconda3/lib/python3.5/site-packages/dill/dill.py", line 266, in load
obj = pik.load()
File "/home/streamsadmin/anaconda3/lib/python3.5/site-packages/dill/dill.py", line 524, in _load_type
return _reverse_typemap[name]
KeyError: 'ClassType'
I've found a workaround, which you can see on Stackoverflow.
Here's my question: types.ClassType was removed in 3.5, yet cloudpickle re-adds it. Is this strictly necessary? It seems to be having side effects.