Tests fail with cfn-lint 0.22.0

[yan12125]

Description:

I ran the test suite. Some tests fail with cfn-lint 0.22.0. I remember all tests were fine when I use cfn-lint 0.21.6 + the patch aws-cloudformation/cfn-lint@0ff8769 for jsonschema 3.x compatibility.

Steps to reproduce the issue:

1. Clone this repo
2. Install needed Python packages
3. Run pytest tests

Observed result:

On branch master, 4 tests fail:

TestTranslatorEndToEnd.test_transform_success_054_api_with_auth_all_minimum
TestTranslatorEndToEnd.test_transform_success_057_api_with_auth_no_default
TestTranslatorEndToEnd.test_transform_success_255_global_handle_path_level_parameter
TestTranslatorEndToEnd.test_transform_success_261_globals_for_api

Output for a sample run can be found here: https://tests.reproducible-builds.org/archlinux/community/python-aws-sam-translator/build1.log

On branch develop, 7 tests fail:

TestTranslatorEndToEnd.test_transform_success_054_api_with_auth_all_minimum
TestTranslatorEndToEnd.test_transform_success_057_api_with_auth_no_default
TestTranslatorEndToEnd.test_transform_success_267_global_handle_path_level_parameter
TestTranslatorEndToEnd.test_transform_success_273_globals_for_api
TestTranslatorEndToEnd.test_transform_success_openapi3_18_api_with_auth_all_minimum_openapi
TestTranslatorEndToEnd.test_transform_success_openapi3_21_api_with_swagger_and_openapi_with_auth
TestTranslatorEndToEnd.test_transform_success_openapi3_24_api_with_openapi_definition_body_no_flag

Expected result:

All tests pass.

[beck3905]

I've run into this issue to. It is causing my PR to fail when it previously passed.

I did some digging and found that in the latest version of cfn-lint (v0.22.0) AWS::Cognito::UserPool was added as a resource type to the ap-southeast-1 region, but does not exist in any other region's resource type library. However, it appears the properties of AWS::Cognito::UserPool were not added and thus validation of that resource type fails. Those tests contain that type of resource in their templates.

I've submitted an issue with cfn-lint: aws-cloudformation/cfn-lint#1002

[yan12125]

Now there is aws-cloudformation/cfn-lint#1003, which fixes the test suite of aws-sam-translator for me.

[yan12125]

Thanks @sambattalio and @kddejong - the issue is now fixed 🎉