Include more than one license file or other arbitrary files in metadata

[agronholm]

Originally reported by: pombredanne NA (Bitbucket: pombredanne, GitHub: pombredanne)

---

With #47 it is now possible to get a SINGLE license file included in a built wheel by using setup.cfg

[metadata]
license_file = <some file>

This is however a severe limitation: this works for very simple licenses... but even for common license such as the Apache-2.0, you cannot include both the license file and the NOTICE file.
Beyond this, there could be other metafiles you want to include in a built wheel.
If they are not package-data, you could use data_files, but they will be installed in some generally unpleasant location.

We should generalize the notion of license-file to include more than one and include arbitrary files defined in a setup.cfg as metafiles.

---

• Bitbucket: https://bitbucket.org/pypa/wheel/issue/138

[agronholm]

Original comment by Daniel Holth (Bitbucket: dholth, GitHub: dholth):

---

Appending them into a single file is not an option?

The main concern here is that an arbitrary file in .dist-info should not share the same filename as any of the machine readable metadata. If you decided to call your arbitrary file 'METADATA' then the resulting wheel would either overwrite your file or it would not work.

[agronholm]

Original comment by pombredanne NA (Bitbucket: pombredanne, GitHub: pombredanne):

---

See schematics/schematics#420 (comment) on why appending them in a single file is not an (easy) option: for instance a license and authors file.

[agronholm]

Original comment by pombredanne NA (Bitbucket: pombredanne, GitHub: pombredanne):

---

@dholth you wrote:

The main concern here is that an arbitrary file in .dist-info should not share the same filename as any of the machine readable metadata. If you decided to call your arbitrary file 'METADATA' then the resulting wheel would either overwrite your file or it would not work.

Could the workaround be simply to add these files after all .dist-info files have been generated and raise an exception if you tried to add a file that already exist in there with a clear error message?

[agronholm]

Original comment by Daniel Holth (Bitbucket: dholth, GitHub: dholth):

---

I would like the long-term solution to be for wheel to define additional categories of files in addition to purelib and platlib, making it easy to put files in places like /usr/share/doc/

[agronholm]

Original comment by pombredanne NA (Bitbucket: pombredanne, GitHub: pombredanne):

---

@dholth Can you elaborate a bit by what you mean exactly by categories of files?

[agronholm]

Original comment by pombredanne NA (Bitbucket: pombredanne, GitHub: pombredanne):

---

do you see these as sections in the setup.cfg where you define what?
I guess the difficulty for a generic solution would be to be very explicit and play well with system as well as virtualenv installations. Such that there is no guess work when packaging and no guess work where you will find the the files once installed.
Which is something data_files have been lacking IMHO.
So ideally for each "metafile" you need to state the exact source(s) and where it will go exactly.
And that where should be somewhere in the .dist-info ? but what aboutd your /usr/share/doc case?

[agronholm]

Original comment by Daniel Holth (Bitbucket: dholth, GitHub: dholth):

---

So the model of installation in wheel is that you have categories of files
that are copied onto destination directories:

package-1.0.data/scripts/
package-1.0.data/platlib/

and so on, with either purelib or platlib going in the root of the wheel
archive instead of in the .data/ subdirectory.

If you added more categories like

package-1.0.data/docs/

then that folder in the wheel archive would be copied onto a configurable
target path, like /usr/share/docs/. Of course it would all be relative to
the virtualenv when using a virtualenv.

In review, the archive contains a set of directory trees each containing a
category of files. Each tree is copied onto a target path in the install
environment. This is how wheel works now but it only knows about a few
distutils-defined categories.

(We also have data_files which is copied to different places depending on
the installer, but generally is copied onto / or the root of the
virtualenv. It has been hard to use correctly for a long time and so is not
very useful.)

Questions remain. Which paths? How do you configure them? How does a
package know where its constituent pieces have been installed? How do you
specify which sources become which "category of data_file"? Is this
sufficiently flexible? Do we include a package-specific path like
/usr/share/docs/packagename-1.0/?

Tried to work this out in a PEP last year but it got stalled.

[agronholm]

Original comment by pombredanne NA (Bitbucket: pombredanne, GitHub: pombredanne):

---

Daniel you wrote:

Tried to work this out in a PEP last year but it got stalled.

I am game. Do you have the start of some draft?

[agronholm]

Original comment by Daniel Holth (Bitbucket: dholth, GitHub: dholth):

---

This section but removing any mention of "a new version of wheel" from the pep, rename pep to "fine grained installation of Python packages". https://www.python.org/dev/peps/pep-0491/#install-paths

[agronholm]

Original comment by Daniel Holth (Bitbucket: dholth, GitHub: dholth):

---

@pombredanne please take a look at https://bitbucket.org/dholth/python-install-paths/src/tip/pep-0491.txt

We have a pull request with most of an implementation of the same for wheel, and probably in /dholth/wheel I have an alternative implementation of a very similar proposal.

[agronholm]

The current plan would seem to be to concatenate all the license related files to .dist-info/LICENSE.txt. With upcoming packaging standards a better solution might emerge. Any objections?

[pombredanne]

@agronholm Thank you for keeping this alive! .dist-info/LICENSE.txt is what we have today, but it is limited and might need combining multiple texts in one (such as in the common Apache + NOTICE text) with some loss of precision in some cases... Yet I have no objection to the status quo, even if not perfect.