-
Notifications
You must be signed in to change notification settings - Fork 707
combination of managed layer AWSSDKPandas-Python312 and AWSLambdaPowertoolsPythonV2 result in ModuleNotFoundError: No module named '_cffi_backend' #2688
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
an extra test I did: the combination of arn:aws:lambda:eu-west-1:336392948345:layer:AWSSDKPandas-Python311:6 and arn:aws:lambda:eu-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV2:11 in Python 3.11 did work |
So here's what is going on here. On the other hand, recent versions of Now on the exception import json
from aws_lambda_powertools.utilities.data_masking.provider.kms.aws_encryption_sdk import (
AWSEncryptionSDKProvider,
)
def lambda_handler(event, context):
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
} which also raises In short, this is a powertools specific issue that they need to solve. I can open an issue with them to inform them. |
Powertools for AWS Lambda on-call checking in - we're investigating. Thank you so much for the preliminary investigation. |
we're making a patch release now to remove |
After testing with version 2.34.2 of powertools using layers:
on Python 3.12, I can confirm that the issue is fixed, thanks @heitorlessa. Closing the ticket |
an extra note for whoever bumps into this -- we will be investigating the operational effort in adding a new layer per Python version in the near future. |
Describe the bug
For my Python 3.12 Lambda function I need to combine Powertools and SDK for Pandas. I would like to use the managed layers for this.
I added layers: arn:aws:lambda:eu-west-1:336392948345:layer:AWSSDKPandas-Python312:4 and arn:aws:lambda:eu-west-1:017000801446:layer:AWSLambdaPowertoolsPythonV2:64
this results in:
Function Logs
rom cryptography.hazmat.primitives.hashes import HashAlgorithm
File "/opt/python/cryptography/hazmat/primitives/hashes.py", line 9, in
from cryptography.hazmat.bindings._rust import openssl as rust_openssl
pyo3_runtime.PanicException: Python API call failed
INIT_REPORT Init Duration: 4103.15 ms Phase: init Status: error Error Type: Runtime.ExitError
ModuleNotFoundError: No module named '_cffi_backend'
thread '' panicked at /github/home/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.20.2/src/err/mod.rs:788:5:
Python API call failed
note: run with
RUST_BACKTRACE=1
environment variable to display a backtraceTraceback (most recent call last):
File "/var/runtime/bootstrap.py", line 63, in
main()
File "/var/runtime/bootstrap.py", line 60, in main
awslambdaricmain.main([os.environ["LAMBDA_TASK_ROOT"], os.environ["_HANDLER"]])
File "/var/lang/lib/python3.12/site-packages/awslambdaric/main.py", line 21, in main
bootstrap.run(app_root, handler, lambda_runtime_api_addr)
File "/var/lang/lib/python3.12/site-packages/awslambdaric/bootstrap.py", line 478, in run
request_handler = _get_handler(handler)
^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.12/site-packages/awslambdaric/bootstrap.py", line 53, in _get_handler
m = importlib.import_module(modname.replace("/", "."))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.12/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1387, in _gcd_import
File "", line 1360, in _find_and_load
File "", line 1331, in _find_and_load_unlocked
File "", line 935, in _load_unlocked
File "", line 994, in exec_module
File "", line 488, in _call_with_frames_removed
File "/var/task/lambda_function.py", line 2, in
import awswrangler as wr
File "/opt/python/awswrangler/init.py", line 10, in
from awswrangler import (
File "/opt/python/awswrangler/mysql.py", line 27, in
pymysql = _utils.import_optional_dependency("pymysql")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python/awswrangler/_utils.py", line 147, in import_optional_dependency
module = importlib.import_module(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.12/importlib/init.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python/pymysql/init.py", line 78, in
from . import connections # noqa: E402
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/python/pymysql/connections.py", line 13, in
from . import _auth
File "/opt/python/pymysql/_auth.py", line 9, in
from cryptography.hazmat.primitives import serialization, hashes
File "/opt/python/cryptography/hazmat/primitives/serialization/init.py", line 7, in
from cryptography.hazmat.primitives._serialization import (
File "/opt/python/cryptography/hazmat/primitives/_serialization.py", line 10, in
from cryptography.hazmat.primitives.hashes import HashAlgorithm
File "/opt/python/cryptography/hazmat/primitives/hashes.py", line 9, in
from cryptography.hazmat.bindings._rust import openssl as rust_openssl
pyo3_runtime.PanicException: Python API call failed
INIT_REPORT Init Duration: 58089.99 ms Phase: invoke Status: error Error Type: Runtime.ExitError
START RequestId: 5d231810-7e77-403c-8727-30dba414f36f Version: $LATEST
RequestId: 5d231810-7e77-403c-8727-30dba414f36f Error: Runtime exited with error: exit status 1
I also tried the same with Python 3.11 and corresponding managed layers, but tis resulted in the same behaviour.
removing one of the layers resulted in behaviour I expected.
How to Reproduce
as code in the lambda function I only added (as test):
The text was updated successfully, but these errors were encountered: