Remove version caps on package dependencies.

[davidt0x]

We are doing this, there are a lot of reasons that this is not a good idea for packages that are meant to be libraries like MDF. Here is a very long :) discussion about it by one of my fellow RSEs here at Princeton. Here is a the TLDR;. This is already causing issues between PsyNeuLink (where we also version cap) and MDF. I tried to install a fresh install of psyneulink the other day on a clean environment and the pip solver got stuck backtracking. Additionally, we have PyTorch pinned to 1.8 now and @mraunak last merged in PR has tests that fail on PyTorch 1.9 on my end. These are not being caught in CI. This is especially true in the PyTorch case because we are using several internal APIs that are subject to change. Anyway, I vote to remove the version caps and to deal with breaking changes and deprecations from libraries as they come. We can always provide an optional locked set of dependencies for a reproduceable environment down the line.

[ALopez3531]

Hello, my name is Alonso Lopez and I am an Outreachy applicant. I am interested in taking on this task.

[pgleeson]

Please see #225 (reply in thread) for discussions around the context for this. It will basically involve forking the project and changing the version constraints in https://github.com/ModECI/MDF/blob/development/setup.cfg and making sure the GitHub actions are still passing. Make sure to do this from the development branch.

[ALopez3531]

Hello,

I have been working on changing the constraints and was hoping I could receive some advice for the issue I have come across. I changed the constraints for the pytorch dependency within setup.cfg and ran the tests, which the new program did not pass. I then modified the arguments in the _model_to_graph() function contained within the importer.py file, and the program passed two tests. However now I received the following error:

self = <onnxruntime.capi.onnxruntime_inference_collection.InferenceSession object at 0x7f47082f6670>, providers = [], provider_options = []
def _create_inference_session(self, providers, provider_options):
available_providers = C.get_available_providers()
# validate providers and provider_options before other initialization
providers, provider_options = check_and_normalize_provider_args(providers,
provider_options,
available_providers)
# Tensorrt can fall back to CUDA. All others fall back to CPU.
if 'TensorrtExecutionProvider' in available_providers:
self._fallback_providers = ['CUDAExecutionProvider', 'CPUExecutionProvider']
else:
self._fallback_providers = ['CPUExecutionProvider']
session_options = self._sess_options if self._sess_options else C.get_default_session_options()
if self._model_path:
sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
else:

      sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)

E onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : This is an invalid model. Type Error: Type 'tensor(int64)' of input parameter (input_sequence) of operator (ConcatFromSequence) in node (Co_ConcatFromSequence) is invalid.
../../../.local/lib/python3.8/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:309: InvalidGraph

[pgleeson]

@ALopez3531 Are you making these changes in a fork of the repository? The simplest way to share your changes and for us to see the precise errors (for multiple python versions) is to commit your changes to a fork, and enable tests to be run at the Actions tab.

[ALopez3531]

@pgleeson Alright my changes should be available now.

[pgleeson]

Thanks @ALopez3531, but you'll have to be a bit more specific about what you've done, why you think it should work and which tests are failing.

[ALopez3531]

Hello @pgleeson, I made some comments under the edits that I made, I was hoping you could check them out.

[pgleeson]

Thanks again @ALopez3531. I'm afraid it wasn't possible to look at your edits in detail before the Outreachy application deadline.

Can you point here to the specific edits/comments you made (simplest is to mention @pgleeson in the comment)?

[parikshit14]

@davidt0x i cloned your branch remove_version_caps and setup a new local environment using

pip install -e .[dev] --no-cache-dir

now when I run the pytest command on the test folder, it gives one error(giving same error on main branch as well). So what needs to be updated more here?

[davidt0x]

Hey @parikshit14,

What is the error you are getting? These are the ones I get on my end, the same as in CI logs

This looks like an issue with sympy trying to evaluate the strings we use to encode onnx functions. I am not sure why this started happening. Perhaps I need to pin the sympy version further back (I left that pin in and removed all others) or maybe it is related to an issue with the new pytorch. I didn't have more time to dig into this.

[parikshit14]

I tried running in the normal env as well (the one with version caps intact) and in that too getting the same error

[davidt0x]

Hey @parikshit14, which directory are you running pytest from?

[davidt0x]

Ok, you are probably running the pytest command pytest. This is different in one subtle way from running python -m pytest, it doesn't prepend the current working directory to sys.path. See here here. For now, I would use python -m pytest if running pytest from command line. This is probably what PyCharm's pytest runner does as well so I never caught this issue. That being said, I am not sure I like this practice in these tests of importing from examples. We already have a test that runs all examples. @mraunak, I believe you wrote these tests. I am trying to understand them. Are these examples automatically generated pytorch code from MDF? If so, where is the code that generates them?