Skip to content

s3 example - thumbnail creator (#613) #621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 1, 2023

Conversation

peterborkuti
Copy link
Contributor

Issue #613, if available:

Description of changes:
S3 example: donwload + create thumbnail + upload
It uses a shared client, based on the example #619

By submitting this pull request

  • I confirm that my contribution is made under the terms of the Apache 2.0 license.
  • I confirm that I've made a best effort attempt to update all relevant documentation.

@peterborkuti peterborkuti force-pushed the s3-example-thumbnail branch from b6d0ed6 to 07cf14f Compare March 19, 2023 20:08
#[async_trait]
impl GetFile for S3Client {
async fn get_file(&self, bucket: &str, key: &str) -> Option<Cursor<Vec<u8>>> {
println!("get file bucket {}, key {}", bucket, key);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

np: what do you think about making these tracing::info calls?


#[async_trait]
pub trait GetFile {
async fn get_file(&self, bucket: &str, key: &str) -> Option<Cursor<Vec<u8>>>;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend this trait also return a Result and let the upper layer handle the error as it sees fit

@peterborkuti
Copy link
Contributor Author

Thank you for the thorough review. Give me some days to fix your findings.

@peterborkuti
Copy link
Contributor Author

peterborkuti commented Mar 24, 2023

Dear @greenwoodcm and @timClicks,
I improved the PR based on your remarks.
I also fixed a bug: it stopped after uploading the first thumbnail.

@peterborkuti
Copy link
Contributor Author

I checked the performance, so I uploaded the same image with different names:

for i in seq 1 10; do aws s3 cp image.png s3://bpimage1/image-run-slow2-$i.png; done

It created different events, so every upload initiated one handler run.
The duration was between 40ms and 100ms.

I moved the client instantiation into the handler and checked the same way, the duration was between 122ms and 185ms.
So instantiating client in the main is a must have.

lambda-fast

lambda-slow

@peterborkuti peterborkuti force-pushed the s3-example-thumbnail branch from b1ccc2f to 6984cdd Compare March 25, 2023 07:43
}

fn get_file_props(record: &S3EventRecord) -> (String, String) {
let empty_response = ("".to_string(), "".to_string());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of using the empty string to represent nothing here, i recommend having this function return (Option<String>, Option<String>)

Ok(())
}

fn get_file_props(record: &S3EventRecord) -> (String, String) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like you'd be able to create your iterator using .into_iter() and then pass an owned S3EventRecord into this method. then you don't have to .to_owned() below, which i believe implicitly clones.

}

let bucket_name = record.s3.bucket.name.to_owned().unwrap();
let object_key = record.s3.object.key.to_owned().unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of doing the if checks above then unwrapping here, i recommend a match statement.

match (record.event_name, record.s3.bucket.name) {
    (Some(event_name), Some(bucket_name) => {
        // additional validation 
    }
    _other => (None, None),
}

this can even be used as the return statement, without using the return keyword

let reader = Cursor::new(vec);
let mut thumbnails = create_thumbnails(reader, mime::IMAGE_PNG, [ThumbnailSize::Custom((size, size))]).unwrap();

let thumbnail = thumbnails.pop().unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you might want to propagate errors up through this method so that if the method fails (for whatever reason) it doesn't crash the function

@peterborkuti
Copy link
Contributor Author

Hi @greenwoodcm,
I modified the code based on your second review, thank you.
I think, the code now is more "rusty" than before.
Péter

Copy link
Contributor

@greenwoodcm greenwoodcm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, thanks for the contribution!

@greenwoodcm greenwoodcm merged commit 3888a1b into awslabs:main Apr 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants