[Grpc]Increase MaxMessageLength to int.MaxValue

[F-Schmidt1]

Investigative information

Please provide the following:

• Timestamp: 2020-02-14
• Function App version (1.0 or 2.0): 2.0
• Invocation ID: 482c2b8e-ffbd-48fb-93ee-926bd17f96f8
• Region: West Europe

Repro steps

1. Create a function with a Blobtrigger binding and deploy to Azure (Linux, consumption plan):
2. Upload a file smaller than 120 MB to the Blob Storage Directoy declared in the functions.json
3. Upload a file larger than 120 MB to the Blob Storage Directory declared in the functions.json

Expected behavior

The functions-app should trigger on both files, and be executed twice (with two entries showing up in the monitoring tab)

Actual behavior

The functions-app only triggers on the smaller file, and is only executed once. One successful, and zero non-successful executions show up in the monitoring tab.

In The Application-Insights-Logs the entry

Executing 'Functions.BlobTrigger' (Reason='New blob detected: file.csv', Id=482c2b8e-ffbd-48fb-93ee-926bd17f96f8)

shows up under traces, but not under requests. There are no entries under exceptions.

When running the functions locally in the VS-code debugger, the error

Error Writing message type InvocationRequest to workerId

System.Private.CoreLib: Error sending from server

is thrown.

Related information

Running the equivalent functions-app using c# on windows runs without any issues, so this problem might be specifically related to python and/or linux under consumption plan.

[anthonychu]

There's a 128MB limit on the trigger and binding payload size for gRPC based workers (i.e., non-.NET languages). @anirudhgarg to confirm

The solution to this, currently, is to use an Event Grid trigger to start the function whenever file(s) are added to the container, then use the storage SDK to download the file. This also allows you to stream the file so you don't load it all into memory at once (which can be an issue especially on the consumption plan).

@craigshoemaker Can we add this to the docs? Probably should document that all out of proc languages have a binding payload size of 128MB, and specifically call this out in the blob trigger docs and suggest Event Grid alternative. Thanks!

[F-Schmidt1]

Thank you for your reply.
I just tested the EventGrid trigger, and it seems to be working fine. Is there a reason for why this limit for blobtriggers exists? As far as i know streaming the file to reduce memory workload is also possible with blobtriggers.

[paulehr]

just wanted to chime in that i hit this as well and it didnt matter if i was using powershell or python as a language, I kept hitting the same GRPC resource exhausting when trying to push a ~400MB file.

[pragnagopa]

MaxMessageLength is configured here: https://github.com/Azure/azure-functions-host/blob/dev/src/WebJobs.Script.Grpc/Server/GrpcServer.cs#L19 for gRpc server. This effects all out-of-proc languages.

We should remove the size limitation.

Note: This would require all the language workers being updated to set max value for the channel message size.