Failed to build request panic

[IndigoCurnick]

Hi there,

Working with the local testing environment for lambdas with cargo lambda watch and cargo lambda invoke <lambda> --data-file test_payload.json panics the main Tokio thread with failed to build request from here. Looking at older issues #368 #423 #445 the answers seem to be revolving around the test jsons are malformatted and need extra fields. However, I specifically want to test the behavior of the lambda when it does receive a bad request. I'd there to be a way to get an error from the conversion to an HTTP request, so I can explicitly handle this and return an error to the user?

[nmoutschen]

Hey @NathanielCurnick! IIRC, the fact that those JSON requests are malformatted just means that they won't correspond to what Lambda actually receives from API Gateway/ALB, thus it's unlikely that an actual user would face that situation unless you're not invoking the functions from APIGW/ALB/Lambda URLs.

[IndigoCurnick]

Thanks for the reply @nmoutschen! Looking at it, I think you're right that some formatting is done by the API Gateway, at least from what I gathered from those old threads. It still irks me slightly that there's a potential crash I can not handle. Maybe an alternative solution is to have the cargo lambda invoke wrap test data in the same way as the API Gateway does?

[calavera]

Can you share the payload that you're trying to use? What do you exactly mean by "bad request", API Gateway guarantees that the JSON has a consistent format. Testing payloads with missing fields that API Gateway guarantees might not be very useful.

[calavera]

Btw, if you're testing HTTP requests, the function URL endpoint might be more useful in your case:

https://github.com/cargo-lambda/cargo-lambda/#watch---function-urls

[calavera]

@NathanielCurnick this is an example of the payload that API Gateway sends to Lambda: https://github.com/LegNeato/aws-lambda-events/blob/master/aws_lambda_events/src/generated/fixtures/example-apigw-request.json

In that example, your app's payload goes into the body field at the bottom.

[IndigoCurnick]

Hi @calavera, thanks for your reply!

I'm using the cargo lambda invoke <lambda> --data-file test_payload.json command to test requests. I understand that when on Lambda itself the API gateway wraps the incoming request from a client into a consistent JSON format like the one in the link you provide, where their request body would go in the body field. However, locally, that wrapping does not happen.

The sample data I am using is a POST request JSON like this
{ method: 'post', url: <URL>, headers: { 'Content-Type': 'application/json' }, data : <SOME-JSON>}

If I make this request to the Lambdas deployed on AWS it is successful, but locally this fails since it is missing the extra wrapping from the API Gateway. I would rather the way I interact with and test both local running Lambda functions and Lambda functions actually on AWS remains consistent. Right now it seems the only option is for me to manually reproduce this request wrapping, which I would rather avoid because I want to test exactly how a client would interact with the Lambda function

[calavera]

However, locally, that wrapping does not happen.

Right, this doesn't happen because cargo lambda invoke is a generic invocation that goes to the lambda function directly, like aws lambda invoke does for the AWS CLI.

A way that we could fix this would be by adding a flag to cargo lambda invoke to wrap the given payload with a specific service's event shape. That would give you the ability to send the data in the shape you're looking for. Some examples of how that can work:

cargo lambda invoke --data-file payload.json --wrap apigw_rest
cargo lambda invoke --data-file payload.json --wrap sns

Contributions to make that happen are very welcome 😉

[IndigoCurnick]

That sounds like a perfect solution. Is there any documentation on how the API Gateway specifically wraps requests? I can make a fork for this soon

[calavera]

Actually, now that I look more closely at you example, I don't think that's going to work because that's not really how ApiGW translates payloads. ApiGW is going to take your http request and translate it into a Lambda event that looks like the one in the example above. It's not a simple wrapper. There are attributes in the requests mapped to specific fields in the payload. Fields like httpMethod, headers, and path, are guaranteed to exist. The data that you send in every request, which goes in the body, is the aleatory data that you want to test. That's a very specific behavior that other services like SNS don't follow. That's why a simple wrapping option won't work.

The three options I see are these:

1. Take the example payloads and modify them to include the data that you want to test, continue using lambda invoke with them.
2. Use the function URLs feature to send HTTP requests and test the responses.
3. Add more flags to cargo lambda, or maybe a whole new command, to test HTTP requests.

I would personally focus on 1, or 2, because they will allow you to test what you need to test without waiting for major development changes.

[IndigoCurnick]

No 2 has proven to be the winner for my needs!
For anyone else who comes across this, you can use if cfg!(debug_assertions) to switch between http://localhost:9000/lambda-url/FUNCTION_NAME and your AWS Lambda URL for tests or production code respectively, for when you need to call other Lambdas.

Thanks for all your help @calavera @nmoutschen !