Tracking issue for tracking licensing metadata with REUSE

[pietroalbini]

This is a tracking issue for MCP 519: track licensing metadata with REUSE

Steps

• Implement scaffolding to track metadata with REUSE
• Add step in CI to verify REUSE can parse all the metadata in the repository.
• Import copyright metadata from the Debian rustc package.
• Enhance the metadata with the existing text in the COPYRIGHT file and issues labeled A-licensing.
• Upgrade to REUSE 4.x and REUSE.toml #127361
• Add REUSE annotations for benchmarks in collector/compile-benchmarks rustc-perf#1939
• Automatically include dependencies licensing in tarballs' .reuse/dep5 in rustbuild.
• Remove LICENSE-MIT and LICENSE-APACHE.
• Auto-generate src/librustdoc/html/static/COPYRIGHT.txt
• Auto-generate COPYRIGHT in source distributions and tarballs
• Replace COPYRIGHT to explain we're using REUSE and instructions on how to lookup actual metadata.

Implementation history

• Initial implementation of REUSE #99415
• Clarify licensing situation of MPSC and SPSC queue #104139
• Add prototype to generate COPYRIGHT from REUSE metadata #104439
• Add more license annotations #104527
• Let reuse look inside git submodules #118445
• Use reuse tool 4.0 #127923
• Use Reuse 4 to manage licenses rustc-perf#1951

[pietroalbini]

There is a problem right now when running REUSE with the --include-submodules flag in our repository (which we'll eventually want to do), as REUSE doesn't handle the formatting of some LLVM comments. Opened fsfe/reuse-tool#560 to fix that issue.

[cuviper]

One caution I want to raise about REUSE is that they currently recommend uncopyrightable files to be listed as CC0, but Fedora Legal is planning to reclassify CC0 as "not allowed" for code, only "allowed-content".

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YUQJHPX6EC2RGXUM7VCKXKWNW6W7DKS6/

[infinity0]

The metadata maintained in the Debian package (in the past few years, mostly by me) is valid only for a particular version of the Debian package, currently 1.60/1.61 from 4-5 months ago. It does not apply to current rust git master. Hopefully whoever owns this issue has the next two steps (Enhance.. & Automatically include..) lined up for soon, otherwise the imported metadata will become more and more wrong as time goes by.

We also specifically exclude a LOT of stuff from the Debian rust package, one of the reasons being convenience, so that we don't have to maintain the licensing metadata for it. These are listed in the imported file under the very large Files-Excluded section near the top, and includes things like mdbook, llvm, cargo, and various binary blobs. In other words, the licensing metadata imported from Debian is incomplete for the purposes of rust itself, assuming this issue is to fully-document the licensing of the rustc source tarball which pulls in stuff external to this main rust.git repo.

[pietroalbini]

@infinity0 thanks for the comment! Indeed I know the metadata imported from Debian is incomplete, and I plan to work in the coming weeks to improve it, after which I plan to ask the foundation for help with a legal review of the metadata. I've just been surprisingly busy in the past couple of months :(

[pietroalbini]

Added a list of REUSE issues I'm tracking in the issue body.

[pietroalbini]

Draft PR with more licensing annotations is open at: #104527

[pietroalbini]

REUSE 4.0.0 is now out, which adds REUSE.toml. That file fixes all of the concerns we had with REUSE, and I thus opened #127361 to migrate to it. rust-lang/rustc-perf#1939 is also something we should do to make our annotations more accurate.