Skip to content

Support for SQS Lambda source event #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Paviel opened this issue Jul 12, 2018 · 7 comments
Closed

Support for SQS Lambda source event #31

Paviel opened this issue Jul 12, 2018 · 7 comments

Comments

@Paviel
Copy link

Paviel commented Jul 12, 2018

AWS now supports using SQS as a Lambda event source. https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/ Events containing messages created with the extended client are hard to handle. Would be great to have a mechanism of deserializing the message and fetching the document from S3 if the message contains a reference.
Thanks.

choeflake added a commit to choeflake/amazon-sqs-java-extended-client-lib that referenced this issue Aug 16, 2018
@jmcazaux
Copy link

jmcazaux commented Feb 1, 2019

Second that !

@Slava96
Copy link

Slava96 commented Mar 17, 2020

+1

1 similar comment
@Spiderpig86
Copy link

+1

@msailes
Copy link

msailes commented Aug 26, 2020

How would you want this to work? Would you like a AWS provided library which had a special handler which fetched the S3 object behind the scenes and passed you it?

@Spiderpig86
Copy link

For documentation purposes, I believe the ask is to make it easier for anyone currently consuming messages that were published by an SQSExtendedClient library and sent to an SQS queue easier.

The flow looks like this: SQSExtendedClient -> SQS Queue -> Lambda

As of now, I don't believe I have found any particular info regarding this use case. Probably this SO question can shed some light: https://stackoverflow.com/questions/57042235/how-to-use-amazonsqsextendedclient-to-recieve-data-from-an-sqsevent

We cannot directly use the SQSExtendedClient to consume messages sent to Lambda, so it sounds like a hacky solution is needed in the mean time to determine if the message is an S3 pointer or the actual payload.

This is especially painful when the SQSExtendedClient is configured to send regular messages via SQS and oversized messages via S3.

@msailes
Copy link

msailes commented Sep 1, 2020

Lambda Powertools for Java now provides a seamless way to consume large messages in Lambda.

https://github.com/awslabs/aws-lambda-powertools-java

You can find the docs here: https://awslabs.github.io/aws-lambda-powertools-java/utilities/sqs_large_message_handling/

@adam-aws
Copy link
Contributor

adam-aws commented Sep 1, 2020

Thanks a lot @msailes, closing this issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants