Preserving read-only flags on package data causing access denied when building

[miuliano]

I'm working with a read-only source directory on Windows and running into access denied errors when calling python setup.py bdist_wheel. I noticed that it was only files included in package_data that were causing this, which helped me track it down to:

setuptools/setuptools/command/build_py.py
outf, copied = self.copy_file(srcfile, target)

which is slightly different from its distutils version:

cpython/Lib/distutils/command/build_py.py
self.copy_file(os.path.join(src_dir, filename), target, preserve_mode=False)

So it looks like it comes down to this preserve_mode flag.

I don't know if there is a good reason to preserve things like read-only flags? There's a comment about preserving mode further up in the distutils code, which seems to suggest not preserving is the way to go.

[jorikdima]

Let me vote up. I was going to report about this issue, but @miuliano has already done it.
My version control system keeps all unchecked files in RO mode, so when I build a wheel it copies sources to build folder. But ONLY package data is keeping mode then and build wheel crashes as it can't delete it then.

[acerv]

Hello everyone,

this is actually a major issue in Windows systems, where some versioning tools might stick with read-only files by default when creating their local repo.

And this is the case of Perforce, where all files are read-only by default. I tried to put my own project inside the CI system using Jenkins. After bdist_wheel command, packaging failed because copied files with MANIFEST.in setup had read-only flag and it was not possible to removed them after build folder was created.

My 2 cents.

Andrea

[miuliano]

I finally had some time to come back to this issue and dig a bit deeper. It's definitely messier than it looks, and @jorikdima did a great job breaking it down in the PR. For simplicity's sake, there are really two issues here:

1. setuptools implementation of the build_py command, unlike the distutils version, preserves file modes for package_data when copying to the build folder (but not for the package itself). However, on Windows the read-only attribute is the only supported one as os.chmod doesn't support anything else (see https://docs.python.org/3/library/os.html#os.chmod). Read-only on Windows isn't the same as having the write bits unset on Linux - it also means the file can't be deleted. This prevents deleting the file in the build folder as well as prevents copying an updated version if it has changed in source.

2. The bdist_wheel command finishes by attempting to call shutil.rmtree on the folder containing the intermediate files used to generate the wheel. These files are created via the install (install_lib) command, which copies them from the build folder. This always preserves file modes so that the read-only flag gets propagated over. rmtree comes with the caveat that attempting to delete read-only files on Windows will cause an exception (https://docs.python.org/3/library/shutil.html#rmtree-example), but the wheel package doesn't handle this correctly so it errors out.

As for solutions... My gut feeling is that preserving modes on Windows during the build step is just a gotcha with very few benefits. If setuptools is open to a Windows-specific hack then I'd turn off copy_file's preserve_mode for just that platform. The wheel package also needs better rmtree error handling, so I'll log a bug there.