Dependencies: add compatibility with newer libraries, list some missing ones

[mojca]

Description:

• I would like to request adding support for a newer version of jsonschema and urllib3.

• In addition to that urllib3 doesn't seem to be listed as a dependency, but it gets fetched when running the tests (I would like to avoid fetching any libraries whatsoever by providing them as a dependency of the package that I'm trying to create from aws-sam-translator), and yaml is also not listed as a "testing dependency".

• The issue below might need to be reported as a separate issue. I'm trying to package this in a package manager (just because it's listed as a dependency of another package that we want) and something seems wrong with the way the files are searched when running the tests (before actually installing the package):

--->  Building py37-aws-sam-translator
Executing:  cd "/path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0" && /opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 setup.py --no-user-cfg build 

...

--->  Testing py37-aws-sam-translator
Executing:  cd "/path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0" && /opt/local/Library/Frameworks/Python.framework/Versions/3.7/bin/python3.7 setup.py --no-user-cfg test 
running test
Searching for jsonschema~=2.6
Reading https://pypi.org/simple/jsonschema/
Downloading https://files.pythonhosted.org/packages/77/de/47e35a97b2b05c2fadbec67d44cfcdcd09b8086951b331d82de90d2912da/jsonschema-2.6.0-py2.py3-none-any.whl#sha256=000e68abd33c972a5248544925a0cae7d1125f9bf6c58280d37546b946769a08
Best match: jsonschema 2.6.0
Processing jsonschema-2.6.0-py2.py3-none-any.whl
Installing jsonschema-2.6.0-py2.py3-none-any.whl to /path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/.eggs
writing requirements to /path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/.eggs/jsonschema-2.6.0-py3.7.egg/EGG-INFO/requires.txt

Installed /path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/.eggs/jsonschema-2.6.0-py3.7.egg
Searching for urllib3<1.25,>=1.20
Reading https://pypi.org/simple/urllib3/
Downloading https://files.pythonhosted.org/packages/01/11/525b02e4acc0c747de8b6ccdab376331597c569c42ea66ab0a1dbd36eca2/urllib3-1.24.3-py2.py3-none-any.whl#sha256=a637e5fae88995b256e3409dc4d52c2e2e0ba32c42a6365fee8bbd2238de3cfb
Best match: urllib3 1.24.3
Processing urllib3-1.24.3-py2.py3-none-any.whl
Installing urllib3-1.24.3-py2.py3-none-any.whl to /path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/.eggs
writing requirements to /path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/.eggs/urllib3-1.24.3-py3.7.egg/EGG-INFO/requires.txt

Installed /path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/.eggs/urllib3-1.24.3-py3.7.egg
running egg_info
writing aws_sam_translator.egg-info/PKG-INFO
writing dependency_links to aws_sam_translator.egg-info/dependency_links.txt
writing requirements to aws_sam_translator.egg-info/requires.txt
writing top-level names to aws_sam_translator.egg-info/top_level.txt
reading manifest file 'aws_sam_translator.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'tests'
writing manifest file 'aws_sam_translator.egg-info/SOURCES.txt'
running build_ext
helpers (unittest.loader._FailedTest) ... ERROR
yaml_helper (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: helpers (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: helpers
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/samtranslator/public/helpers.py", line 3, in <module>
    from samtranslator.translator import add_default_parameter_values
ImportError: cannot import name 'add_default_parameter_values' from 'samtranslator.translator' (/path/to/py37-aws-sam-translator/work/aws-sam-translator-1.11.0/samtranslator/translator/__init__.py)

The yaml error goes away once I add it to a testing dependency. The error seems to be a consequence of maybe the files not yet residing at their final destination (the same procedure works for a couple of hundred other packages).

[yan12125]

it gets fetched when running the tests

For urllib3, I believe it's due to out-dated botocore. MacPorts ships botocore 1.12.116 and urllib3 1.25.x is declared as supported since botocore 1.12.154 [1].

For jsonschema, 3.x is supported in the develop branch [2].

yaml is also not listed as a "testing dependency"

It's here: https://github.com/awslabs/serverless-application-model/blob/master/requirements/dev.txt

[1] boto/botocore@aae1cf8
[2] #927

[jfuss]

Our main requirements file can be found here: https://github.com/aws/serverless-application-model/blob/master/requirements/base.txt and we support 1.5 of boto3 or greater and 3.2 or greater for jsonschema. urllib3 is not used in this repo: https://github.com/aws/serverless-application-model/search?q=urllib3

Given how old this issue is, I assume some of the newer practices are might have solved this. If this is not the case, please create a new issue.

Closing