Unable to use build_in_docker feature with Rancher Desktop or Colima

[mdesoeuv]

Description

The build of a Poetry Project in docker with the lambda module fails. It seems related to a permissions problem with the temporary folder created and bound to the container during the build process.

Versions

• Module version [Required]: 7.2.1
• Terraform version: v1.5.2
• Docker: 25.0.3, build 4debf411d1
• MacOS: 13.6.3

Reproduction Code [Required]

Steps to reproduce the behavior:

• An Docker engine alternative to Docker Desktop (Rancher Desktop or Colima)
• terraform init
• terraform apply

Project Structure

.
├── builds
│   └── lambda
├── lambda
│   ├── README.md
│   ├── docker
│   │   └── Dockerfile
│   ├── poetry.lock
│   ├── pyproject.toml
│   ├── src
│   │   └── api
│   │       ├── __init__.py
│   │       └── main.py
│   └── tests
│       └── __init__.py
├── main.tf

Terraform

module "lambda_backend" {
  source = "terraform-aws-modules/lambda/aws"

  function_name              = "api-lambda"
  description                = "FastAPI in AWS Lambda"
  handler                    = "api.main.handler"
  publish                    = true
  create_lambda_function_url = true

  source_path = [
    "${path.module}/lambda/src",
    {
      path           = "${path.module}/lambda/pyproject.toml"
      poetry_install = true
    }
  ]

  artifacts_dir = "${path.root}/builds/lambda/"

  build_in_docker = true
  runtime         = "python3.10"
  docker_image    = "build-python3.10-poetry"
  docker_file     = "${path.module}/lambda/docker/Dockerfile"

  cloudwatch_logs_retention_in_days = 3
}

output "lambda_url" {
  value = module.lambda_backend.lambda_function_url
}

Lambda Source Code

from fastapi import FastAPI
from mangum import Mangum


app = FastAPI()

handler = Mangum(app, lifespan="off")


@app.get("/")
def root():
    return {"Hello World !"}

pyproject.toml

[tool.poetry]
name = "api"
version = "0.1.0"
description = ""
authors = ["mdesoeuvre <mehdi.desoeuvre@gmail.com>"]
readme = "README.md"
packages = [{include = "api", from = "src"}]

[tool.poetry.dependencies]
python = "^3.10"
fastapi = "^0.109.2"
mangum = "^0.17.0"


[tool.poetry.group.dev.dependencies]
uvicorn = {extras = ["standard"], version = "^0.27.1"}

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Dockerfile

FROM public.ecr.aws/sam/build-python3.10:latest

RUN pip install poetry==1.7.1

Expected behavior

The build should succeed as it does with Docker Desktop

Actual behavior

The build fails with the error: Poetry could not find a pyproject.toml file in /var/task or its parents

module.lambda_backend.null_resource.archive[0]: Creating...
module.lambda_backend.null_resource.archive[0]: Provisioning with 'local-exec'...
module.lambda_backend.null_resource.archive[0] (local-exec): Executing: ["python3" ".terraform/modules/lambda_backend/package.py" "build" "--timestamp" "1708445266894506000" "./builds/lambda/49f733814a0efd219e4697d12f75fcafa79f7c1d8ed6cce68f062dc7e8a2ef23.plan.json"]
module.lambda_backend.null_resource.archive[0] (local-exec): zip: creating './builds/lambda/49f733814a0efd219e4697d12f75fcafa79f7c1d8ed6cce68f062dc7e8a2ef23.zip' archive
module.lambda_backend.null_resource.archive[0] (local-exec): zip: adding content of directory: ./lambda/src
module.lambda_backend.null_resource.archive[0] (local-exec): zip: adding: api/
module.lambda_backend.null_resource.archive[0] (local-exec): zip: adding: api/__init__.py
module.lambda_backend.null_resource.archive[0] (local-exec): zip: adding: api/main.py
module.lambda_backend.null_resource.archive[0] (local-exec): > docker images '--format={{.ID}}' build-python3.10-poetry
module.lambda_backend.null_resource.archive[0] (local-exec): Installing python dependencies with poetry & pip: ./lambda/poetry.lock
module.lambda_backend.null_resource.archive[0] (local-exec): > mktemp -d terraform-aws-lambda-XXXXXXXX # /var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5
module.lambda_backend.null_resource.archive[0] (local-exec): Using poetry.lock file: ./lambda/poetry.lock
module.lambda_backend.null_resource.archive[0] (local-exec): > cd /var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5
module.lambda_backend.null_resource.archive[0] (local-exec): > docker run --rm -w /var/task -v /private/var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5:/var/task:z -v /Users/mdesoeuvre/.ssh/known_hosts:/root/.ssh/known_hosts:z --entrypoint '' fdbb6bb2ed34 /bin/sh -c 'poetry config --no-interaction virtualenvs.create true && poetry config --no-interaction virtualenvs.in-project true && poetry export --format requirements.txt --output requirements.txt --with-credentials && python3.10 -m pip install --no-compile --no-deps --prefix= --target=. --requirement=requirements.txt && chown -R 503:20 .'

module.lambda_backend.null_resource.archive[0] (local-exec): Poetry could not find a pyproject.toml file in /var/task or its parents
module.lambda_backend.null_resource.archive[0] (local-exec): zip: Error during zip archive creation
module.lambda_backend.null_resource.archive[0] (local-exec): Traceback (most recent call last):
module.lambda_backend.null_resource.archive[0] (local-exec):   File "/Users/mdesoeuvre/Downloads/FastApi/.terraform/modules/lambda_backend/package.py", line 1625, in build_command
module.lambda_backend.null_resource.archive[0] (local-exec):     bpm.execute(build_plan, zs, query)
module.lambda_backend.null_resource.archive[0] (local-exec):   File "/Users/mdesoeuvre/Downloads/FastApi/.terraform/modules/lambda_backend/package.py", line 900, in execute
module.lambda_backend.null_resource.archive[0] (local-exec):     with install_poetry_dependencies(query, path) as rd:
module.lambda_backend.null_resource.archive[0] (local-exec):   File "/Users/mdesoeuvre/.asdf/installs/python/3.11.2/lib/python3.11/contextlib.py", line 137, in __enter__
module.lambda_backend.null_resource.archive[0] (local-exec):     return next(self.gen)
module.lambda_backend.null_resource.archive[0] (local-exec):            ^^^^^^^^^^^^^^
module.lambda_backend.null_resource.archive[0] (local-exec):   File "/Users/mdesoeuvre/Downloads/FastApi/.terraform/modules/lambda_backend/package.py", line 1229, in install_poetry_dependencies
module.lambda_backend.null_resource.archive[0] (local-exec):     check_call(
module.lambda_backend.null_resource.archive[0] (local-exec):   File "/Users/mdesoeuvre/.asdf/installs/python/3.11.2/lib/python3.11/subprocess.py", line 413, in check_call
module.lambda_backend.null_resource.archive[0] (local-exec):     raise CalledProcessError(retcode, cmd)
module.lambda_backend.null_resource.archive[0] (local-exec): subprocess.CalledProcessError: Command '['docker', 'run', '--rm', '-w', '/var/task', '-v', '/private/var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5:/var/task:z', '-v', '/Users/mdesoeuvre/.ssh/known_hosts:/root/.ssh/known_hosts:z', '--entrypoint', '', 'fdbb6bb2ed34', '/bin/sh', '-c', 'poetry config --no-interaction virtualenvs.create true && poetry config --no-interaction virtualenvs.in-project true && poetry export --format requirements.txt --output requirements.txt --with-credentials && python3.10 -m pip install --no-compile --no-deps --prefix= --target=. --requirement=requirements.txt && chown -R 503:20 .']' returned non-zero exit status 1.

Error: local-exec provisioner error

  with module.lambda_backend.null_resource.archive[0],
  on .terraform/modules/lambda_backend/package.tf line 67, in resource "null_resource" "archive":
  67:   provisioner "local-exec" {

Error running command './builds/lambda/49f733814a0efd219e4697d12f75fcafa79f7c1d8ed6cce68f062dc7e8a2ef23.plan.json': exit status 1. Output: zip: creating
'./builds/lambda/49f733814a0efd219e4697d12f75fcafa79f7c1d8ed6cce68f062dc7e8a2ef23.zip' archive
zip: adding content of directory: ./lambda/src
zip: adding: api/
zip: adding: api/__init__.py
zip: adding: api/main.py
> docker images '--format={{.ID}}' build-python3.10-poetry
Installing python dependencies with poetry & pip: ./lambda/poetry.lock
> mktemp -d terraform-aws-lambda-XXXXXXXX # /var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5
Using poetry.lock file: ./lambda/poetry.lock
> cd /var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5
> docker run --rm -w /var/task -v /private/var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5:/var/task:z -v
/Users/mdesoeuvre/.ssh/known_hosts:/root/.ssh/known_hosts:z --entrypoint '' fdbb6bb2ed34 /bin/sh -c 'poetry config --no-interaction virtualenvs.create true && poetry
config --no-interaction virtualenvs.in-project true && poetry export --format requirements.txt --output requirements.txt --with-credentials && python3.10 -m pip install
--no-compile --no-deps --prefix= --target=. --requirement=requirements.txt && chown -R 503:20 .'

Poetry could not find a pyproject.toml file in /var/task or its parents
zip: Error during zip archive creation
Traceback (most recent call last):
  File "/Users/mdesoeuvre/Downloads/FastApi/.terraform/modules/lambda_backend/package.py", line 1625, in build_command
    bpm.execute(build_plan, zs, query)
  File "/Users/mdesoeuvre/Downloads/FastApi/.terraform/modules/lambda_backend/package.py", line 900, in execute
    with install_poetry_dependencies(query, path) as rd:
  File "/Users/mdesoeuvre/.asdf/installs/python/3.11.2/lib/python3.11/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/Users/mdesoeuvre/Downloads/FastApi/.terraform/modules/lambda_backend/package.py", line 1229, in install_poetry_dependencies
    check_call(
  File "/Users/mdesoeuvre/.asdf/installs/python/3.11.2/lib/python3.11/subprocess.py", line 413, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--rm', '-w', '/var/task', '-v',
'/private/var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-s8qm4qk5:/var/task:z', '-v', '/Users/mdesoeuvre/.ssh/known_hosts:/root/.ssh/known_hosts:z',
'--entrypoint', '', 'fdbb6bb2ed34', '/bin/sh', '-c', 'poetry config --no-interaction virtualenvs.create true && poetry config --no-interaction virtualenvs.in-project
true && poetry export --format requirements.txt --output requirements.txt --with-credentials && python3.10 -m pip install --no-compile --no-deps --prefix= --target=.
--requirement=requirements.txt && chown -R 503:20 .']' returned non-zero exit status 1.

Additional context

• Further investigation seems to point to a permissions problem with the temporary folder created during the build process and bound in the container :

ls command inside the container with an other folder mounted in the container : OK

❯ docker run --rm -w /var/task -v ~/Downloads/FastApi/lambda:/var/task:z -v /Users/mdesoeuvre/.ssh/known_hosts:/root/.ssh/known_hosts:z --entrypoint '' fdbb6bb2ed34 /bin/sh -c 'ls -la /var/task'
total 100
drwxr-xr-x 1  503 games   288 Feb 20 15:40 .
drwxr-xr-x 1 root root   4096 Feb  6 19:58 ..
drwxr-xr-x 1  503 games    96 Feb 20 15:33 docker
-rw-r--r-- 1  503 games 65930 Feb 20 15:27 poetry.lock
-rw-r--r-- 1  503 games   448 Feb 20 15:27 pyproject.toml
-rw-r--r-- 1  503 games     0 Feb 20 15:22 README.md
drwxr-xr-x 1  503 games    96 Feb 20 15:26 src
drwxr-xr-x 1  503 games    96 Feb 20 15:22 tests
-rw-r--r-- 1  503 games    15 Feb 20 15:25 .tool-versions

ls command inside the container with temp folder : fail

❯ mktemp -d "$TMPDIR/terraform-aws-lambda-XXXXXXXX"
/var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T//terraform-aws-lambda-eq7lvce5
❯ cp -r ./* /var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T//terraform-aws-lambda-eq7lvce5
❯ docker run --rm -w /var/task -v /var/folders/tt/zl9rh1j93pz1b_05kfm1p9lr0000gq/T/terraform-aws-lambda-eq7lvce5:/var/task:z -v /Users/mdesoeuvre/.ssh/known_hosts:/root/.ssh/known_hosts:z --entrypoint '' fdbb6bb2ed34 /bin/sh -c 'ls -la /var/task'
total 12
drwxr-xr-x 2 root root 4096 Feb 20 16:11 .
drwxr-xr-x 1 root root 4096 Feb  6 19:58 ..

• This issue is not the same as #524 because the build works fine with Docker Desktop but fails with Rancher Desktop or docker cli

[Bogyie]

same here

[Bogyie]

I solve it! @mdesoeuv

Based on code that module, create temp directory under /var/... and mount it.
So, colima can't mount that directory.

Here is several options to solve this problem.

1. On starting colima, add option --mount
2. Edit colima template, run colima template

# Configure volume mounts for the virtual machine.
# Colima mounts user's home directory by default to provide a familiar
# user experience.
#
# EXAMPLE
# mounts:
#   - location: ~/secrets
#     writable: false
#   - location: ~/projects
#     writable: true
#
# Colima default behaviour: $HOME and /tmp/colima are mounted as writable.
# Default: []
mounts: []

4. Set temp directory path at source block( but poetry is not support now )

source_path = [
    "${path.module}/src",
    {
      path           = "${path.module}/requirements.txt"
      pip_requirements = true
      pip_tmp_dir = "${path.module}/.temp"
    }
  ]

5. Edit module's source code , and use it with local source

[mdesoeuv]

Thank you for responding to this post with your fix, your help is appreciated

[github-actions]

This issue has been automatically marked as stale because it has been open 30 days
with no activity. Remove stale label or comment or this issue will be closed in 10 days

[github-actions]

This issue was automatically closed because of stale in 10 days

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

[antonbabenko]

This issue has been resolved in version 7.18.0 🎉