Buildkit ignores .dockerignore in Git repository

[phoenix-bjoern]

When using docker build with a Git repo as build context and DOCKER_BUILDKIT=1 the .dockerignore contents in the Git repo gets ignored and the files/directories get copied in the Docker image.
When DOCKER_BUILDKIT env is not set, the ignored files/directories are correctly excluded and can't be found in the Docker image.

Steps to reproduce the issue:

1. export DOCKER_BUILDKIT=1
2. docker build https://github.com/PHOENIX-MEDIA/buildkit-test.git
3. docker run -ti --rm <image> sh
4. ls -la /var/www/html/
   The files in https://github.com/PHOENIX-MEDIA/buildkit-test/blob/master/.dockerignore shouldn't be in the html folder.

You can repeat the steps with a clone of the repository on your local machine:

1. git clone https://github.com/PHOENIX-MEDIA/buildkit-test.git .
2. docker build .
3. docker run -ti --rm <image> sh
4. ls -la /var/www/html/
   The folder only contains the copy.me file, which is the expected result according to the .dockerignore contents.

I'm using Docker version 19.03.13, build 4484c46d9d on Centos7.

[thaJeztah]

Thanks for reporting

@tonistiigi @tiborvass PTAL

[tonistiigi]

.dockerignore only applies to transferring local files from client to the daemon. We should make it clear in the docs if it isn't already. When building from remote sources files are not removed from there. This is also true for the legacy builder and has always worked like this. The difference is that due to some unrelated changes old cli doesn't in some cases perform a build from git even if user thinks they do. Instead git clone is run locally by the cli followed by docker build .. This method is slower, duplicates more data, removes the cache references to original sources, and requires specific host dependencies that are not always present.

It is possible we could add a completely separate implementation of dockerignore that removes files on git clone and another one for tarball download but then that would be different from the legacy build(although unlikely to cause practical issues). If your usecase prefers removing files you can always just run git clone && docker build .

[phoenix-bjoern]

@tonistiigi thanks for the insights. The reason why we used the build from remote sources (git) is that it can retrieve only the delta and therefore should be faster than cloning and then copying the whole files (could be several hundreds MBs). We didn't expect there would be any difference in behavior.
So if there is a way to respect declarations in .dockerignore even for remote sources we would definitely prefer that option.

[mauke]

@tonistiigi

.dockerignore only applies to transferring local files from client to the daemon. We should make it clear in the docs if it isn't already.

Oh, the docs are clear, they just say the opposite: https://docs.docker.com/build/building/context/

When you pass a URL pointing to the location of a Git repository as an argument to docker build, the builder uses the repository as the build context.

[...]

When you run a build command, the build client looks for a file named .dockerignore in the root directory of the context. If this file exists, the files and directories that match patterns in the files are removed from the build context before it's sent to the builder.

Note how it explicitly says .dockerignore is looked up and evaluated in the build context. Nothing even hints at .dockerignore files being ignored outside of local directories.

[tonistiigi]

cc @dvdksn