Sharing a single API dns name across varying base path names that represent API extensions

[bmckinle]

Describe your idea/feature/enhancement

We need a way to extend an API across different repositories by creating a new base path name for an existing, shared API DNS name. For example:

api.mycompany.com/service1 created in template1.yaml using a single API Gateway and Serverless Function resources.
api.mycompany.com/service2 created in template2.yaml using the template1.yaml API Gateway and different Serverless Function resources.

Proposal

Our understanding is that is a limitation of SAM but using a combination of CloudFormation and SAM can achieve automation that would achieve a shared api domain goal that supports extensions via basename changes only. This should be supportable via SAM itself.

Things to consider:

Sharing API elements across SAM templates. We are primarily looking ways to simplify maintaining a growing API over time without resorting to creating a unique domain name for each API extension that emerges from collaborating teams over time. Ostensibly,
service1.mycompany.com
service2.mycompany.com
...
is not as valuable as
api.mycompany.com/service1
api.mycompany.com/service2
...

Additional Details

AWS Support suggested we create this issue as the limitation of sharing a single API domain across SAM templates is a known limitation.

[hawflau]

@bmckinle Thanks for your proposal. Yeah, it's a limitation that SAM doesn't support sharing API resource across multiple stacks seamlessly.

We've been thinking a lot about how we can solve that. We posted a workaround in a previous thread. I was using a root stack and nested stack to illustrate in that example but I guess it is solving the problem as yours. Can you take a look and give us feedback? This will help us formulate a better solution.

[bmckinle]

Referring to the workaround, while solution 1 heads in the right direction, it suggests we need to create a single API (openapi/swagger). Yet, we need multiple APIs managed in decoupled (multi-team) manner. Solution 2 removes openapi/swagger altogether, which is something we want to avoid entirely. We want multiple openapi.yaml and template.yaml module/api sets that links to a common dns_domain_template.yaml. Here are two examples of this type of architecture, with the first from AWS blogs and the second a third party vendor:

1. AWS Microservices Blog
   Notice the following multi-api-gateways within one domain is supported for a single or central account:
   my-custom-domain.com/path1
   my-custom-domain.com/path2
   Can this be achieved in cloudformation alone? How? Can this be supported in multiple accounts, such as dev, qa, and production?

2. Serverless
   This is supported out of the box. Why can't SAM emulate what serverless is doing? What blocks SAM from achieving the same?

In summary, we need a way to use SAM to deploy multiple API Gateways that share a domain name with each gateway using different base paths.

[troycampano]

@bmckinle I've been able to achieve this URL format using multiple SAM projects:

api.mycompany.com/service1
api.mycompany.com/service2

I used AWS::ApiGateway::BasePathMapping to associate various paths to my custom domain used in API Gateway. It looks like this:

  Microservice02Mapping:
    Type: 'AWS::ApiGateway::BasePathMapping'
    Properties:
      BasePath: 'app02'
      DomainName: api.hostname.com
      RestApiId: !Ref ApiGatewayApi
      Stage: Prod

I have mappings like this in each of my SAM projects. In the API Gateway console, under "Custom domain names", the "API mappings" tab looks like this:

I posted more details here: #2703 (comment)

[ricott]

@troycampano We use this approach extensively and it works ok. We use it for the HTTP API Gateway. It took a session with an AWS serverless specialist SA to come up with the solution a year ago when we adopted the pattern.

  ApiGatewayMapping:
    Type: AWS::ApiGatewayV2::ApiMapping
    DependsOn: Gateway
    Properties:
      ApiId: !Ref Gateway
      ApiMappingKey: !If [IsFeatureDeployment, !Sub '${AWS::StackName}/users', 'users']
      DomainName: !FindInMap [DeploymentEnvironment, !Ref Environment, DomainName]
      Stage: !Ref Gateway.Stage

You get a lot of API Gateways but at least it solves the very basic problem of being forced to build a single monolithic API stack. With this approach, you can build and deploy multiple stacks independently and still use the same custom domain name, e.g. api.mycompany.com.

There is a problem though with paths. Imagine you have a users endpoint as per the example above. Gateway path mapping points to /users. Since users is now part of base path there is a problem with the path property of the individual serverless functions. If you want to expose a get with /users/{user_id} and a get with /users where you for instance want to allow a lookup via email - then this is not possible using this approach.

This works

      Events:
        GetUser:
          Type: HttpApi
          Properties:
            ApiId: !Ref Gateway
            Method: get
            Path: /{user_id}
            Auth:
              Authorizer: OAuth2Authorizer
              AuthorizationScopes:
                - "read:users"

This works but looks awful since the path has to be /users/?email=abc@xyz.se with the ending /.

        GetUsers:
          Type: HttpApi
          Properties:
            ApiId: !Ref Gateway
            Method: get
            Path: /
            Auth:
              Authorizer: OAuth2Authorizer
              AuthorizationScopes:
                - "read:users"

This means you have to add some artificial API mapping path like this, which we currently use
ApiMappingKey: !If [IsFeatureDeployment, !Sub '${AWS::StackName}/u/v1', 'u/v1']

Then for the serverless endpoints, you can use
Path: /users/{user_id} and /users

Which results in these endpoints

get api.mycompany.com/u/v1/users/{user_id}
get api.mycompany.com/u/v1/users?email=

If we could import and share an API Gateway instance instead and only attach the serverless functions we wouldn't have this problem. My understanding is that the serverless framework supports this but I haven't gotten around to trying it yet.

[bmckinle]

We've switch to AWS CDK over the last year, and using a BasePathMapping object and existing apigateway.from... existing domain, are able to add a new base path to an existing domain quite easily. Here's a snippet of how easy it is:

       #
        # Create api gateway for proxy integration with lambda
        #
        existing_domain_name = apigateway.DomainName.from_domain_name_attributes(self,
                                                                                 "Company API Domain",
                                                                                 domain_name=api_domain_name,
                                                                                 domain_name_alias_hosted_zone_id=hosted_zone_id,
                                                                                 domain_name_alias_target=api_gateway_domain_name)

        rest_api = apigateway.RestApi(self, "ipset_refresher-api",
                                      rest_api_name="My REST API Service",
                                      description=f"This service updates ipset.yaml on {target_s3_bucket_name}",
                                      deploy_options=apigateway.StageOptions(stage_name=stage_name))

        apigateway.BasePathMapping(self,
                                   "MyBasePathMapping",
                                   domain_name=existing_domain_name,
                                   rest_api=rest_api,
                                   base_path=base_path
                                   )

The world moves on.

[hoffa]

Potentially relevant: #2703 (comment)