Node.js 20.x. Longer cold starts than prior versions (18,16) with 3X top values compared to the average

[adriantanasa]

The recently released nodejs20.x runtime for AWS Lambda is establishing a concerning performance degradation trend for Node runtime and larger variation in the cold start values with maximum values 3X more than previous node version.

Based on running a benchmark over 100 points for a function (1KB code size, 256MB memory, us-east-1) with nodejs20.x runtime the average value is 240ms (compared to 171ms node18.x and 159ms for node16.x).

There is an increased variation in the initDuration for this runtime compared to the previous version, with the maximum values increased by several times. For the same 1KB-sized function, its maximum cold start value is now 793ms compared to 248ms (node18), and the P90 is 584ms compared to 182ms on node18.

This "instability" is confirmed for functions with variable package sizes (1K to 64MB) over 1000 cold start executions.

Gist results for all Nodejs runtimes:

• nodejs20.x
• nodejs18.x
• nodejs16.x

Details of how the benchmark is designed and executed here: Size is (almost) all that matters for optimizing AWS Lambda cold starts

This issue was raised also by @o-alexandrov on the Nodejs20.x release topic:

@jtuliani I was very hyped about Node.js 20 after your comment above on what AWS did to decrease cold starts.
Unfortunately, the cold starts are a lot worse than Node.js 18/16/14 for non-Graviton (non-ARM) and pretty much the same as Node.js 16 for Graviton.
Originally posted by @o-alexandrov in #91 (comment)

Hopefully, this is something that can be tuned in the next while to fulfill the high community expectations from the announcement of cold starts optimization actions such as the removal of extra root CA certs, lazy-loading of non-essential modules, and further tuning of the AWS SDK 3 library.

[electricsheep]

I have experienced the same issue.

Here is a graph of my lambda duration - red line indicates the upgrade from nodejs18.x to nodejs20.x:

Runtime average for v18: 195ms
Runtime average for v20: 310ms

[adriantanasa]

I have run the benchmark recently and it seems the peak Init Duration issues are fixed now. I am curios what caused them though. Also the overall performance is improved almost matching the nodejs16.x runtime, You can see here the latest results for nodejs20.x. Great job, AWS team for addressing this ;)

PS @electricsheep please create another issue for your case. It might be something from the libraries used or other factor. This issue here was only for Init Duration with fairly plain functions with no npm libraries or AWS SDK calls.