Valid JSON document reported as invalid because exception text cannot be found

[pombredanne]

Using v1.1.0 with openjdk 19 2022-09-20 on Linux x64, the attached document fails to validate even though it contains the proper text for the exception as a LicenseRef

$ openjdk.java -jar tools-java-1.1.0-jar-with-dependencies.jar Verify qt_core_5.4.1.spdx.json.txt json
This SPDX Document is not valid due to:
	Missing required exception text for LicenseRef-dejacode-qt-company-exception-2017-lgpl-2.1 in Qt Core

qt_core_5.4.1.spdx.json.txt

@tdruez ping

[pombredanne]

@goneall Thanks!

[goneall]

@pombredanne In looking at the attached JSON file, it looks like one of the hasExtractedLicensingInfos is being used for the exception.

This is not supported in the spec or in the library.

There was a comment in a LicenseListXML issue suggesting we support this: spdx/license-list-XML#1022 (comment) - however, it has not been adopted.

I recall when we first introduced license exceptions, we decided not to add support for "ExceptionRef-"'s with the logic that the producer of the SPDX document could just create a "LicenseRef-" with the main license + the license exception text. In the case of LicenseRef-dejacode-qt-company-exception-2017-lgpl-2.1 - it would be the LGPL-2.1 text followed by the LicenseRef-dejacode-qt-company-exception-2017-lgpl-2.1 text.

Turns out the error isn't handled very well - I'll create a PR which will fail rather than display an odd verification error. BTW - I attempted to add support for this in the library but ran into a design problem in not being able to distinguish between exceptions and licenses when reading the hasExtractedLicensingInfos. To solve this, we could either have a separate array - something like hasExtractedLicExceptionInfos, or use the ID syntax to distinguish the type (e.g. ExceptionRef- for exceptions). Probably more of a spec issue/discussion than a library discussion.

[pombredanne]

This is not supported in the spec or in the library.

That's rather unfortunate as we are tracking over 100 exceptions in ScanCode and DejaCode that do not exist in the SPDX list. I guess that if we move to namespacing this would become a non-issue?

we decided not to add support for "ExceptionRef-"

Using "ExceptionRef" instead of "LicenseRef" would not have been a good implementation anyway as IMHO exception and licenses should not use different field names as this makes any support for exception more complicated to implemented and harder to understand and read. To me an exception is simply a license with a flag that states that this is an exception.

the producer of the SPDX document could just create a "LicenseRef-" with the main license + the license exception text.

Any LicenseRef should be usable as an exception. Otherwise having exception support is rather moot and promotes everyone using LicenseRef for anything, which in turn makes license expressions mostly useless as the ids (e.g., the words in the "licensing" language) carry no meaning whatsoever and require the text to be rescanned and reinterpreted to make sense of it: this defeats the principle of having license ids in the first place; and this leads to using license refs for everything which is correct but mostly useless for the recipient and is growing commonplace.

Another possibility is to report all exceptions with an AND which defeats the purpose of having a WITH syntax to support exceptions but this is likely the least incorrect yet spec-compliant approach.

[goneall]

@pombredanne

That's rather unfortunate as we are tracking over 100 exceptions in ScanCode and DejaCode that do not exist in the SPDX list. I guess that if we move to namespacing this would become a non-issue?

I believe namespacing would solve it since the same rules for listed licenses should apply to namespaced licenses.

Any LicenseRef should be usable as an exception. Otherwise having exception support is rather moot and promotes everyone using LicenseRef for anything, which in turn makes license expressions mostly useless as the ids (e.g., the words in the "licensing" language) carry no meaning whatsoever and require the text to be rescanned and reinterpreted to make sense of it: this defeats the principle of having license ids in the first place; and this leads to using license refs for everything which is correct but mostly useless for the recipient and is growing commonplace.

Good point. Although, in my use of the libraries it is important to distinguish between text used as exceptions and text used as the primary license language (for validation). This would require some way of distinguishing between the two independent of it's use in the context of an expression.

Another possibility is to report all exceptions with an AND which defeats the purpose of having a WITH syntax to support exceptions but this is likely the least incorrect yet spec-compliant approach.

I agree that this may be the best (or least bad) approach since we don't currently support user defined exceptions. The only thing you loose is a subtle distinction that the exception is intended to only be used along with the license rather than as a standalone license. I can't think of too many cases where this distinction would be important in analysis since both WITH and AND have the semantics that the terms of both apply.

[goneall]

The bad error message is resolved with SPDX Java Library PR #123

Closing this issue. We could open a separate issue against the spec for the WITH operator to support License/Exception ref's.