GPL-2.0 in test tests/php

[OliverFendt]

Hi,
thank you very much for providing this package.
I am currentliy doing a license analysis of markdown and came across the GPL-2.0 license text in tests/php.
How is the relation of the GPL-2.0 to the files in this repo, does it apply to any of the files, if yes it would be very useful that those files contain a reference to the GPL-2.0 otherwise no one can determine to which files it applies and so it will be very hard to comply to the license.

[waylan]

Well, I had no idea that file was there. In fact, the php tests were added before I joined the project, so we are not likely to get a definitive answer. My best guess would be that the license was added with the php test files. As I understand it, back when those tests were added. they were copied from the php markdown project as-is. Presumably, this license is the license which applies to them (that is all files in the php directory and any subdirectories).

Curiously, the current license for PHP Markdown does not match at all (BSD 3 clause), so that makes my answer suspect. I also checked PHP Markdown classic and that license doesn't match either (I downloaded the distribution and checked the included license file). In fact, that license goes back 17 years.

I have no idea what the GPL license applies to.

it would be very useful that those files contain a reference to the GPL-2.0

The problem is that the testing framework doesn't allow for that. We have had a strict policy that we do not edit these files (same for the pl tests). If a specific test fails and we have no intention of making it pass, then we skip the test (as defined in tests/test_legacy.py), we do not edit it.

It is not clear what the right path forward is here.

[facelessuser]

Obviously, GPL 2.0 is not compatible with BSD, but I question whether it matters in this case. And what I mean by this, do we need to keep the license around for using the test cases as spec. Or does following a spec under a license make your code need to follow that license? I suspect not. Google copied the API of Java in Android 🤷🏻.

While I think we definitely should reference where the test cases came from, I question whether the Markdown test cases are something copyrightable as they are just Markdown prose. Do I have mark my documentation GPL 2.0 if I use <HTTP://WWW.SOMEURL.COM> in my docs because it was in a test? I doubt it.

I'm not sure what the best way to represent this is, but I don't think this is something that puts us in danger, but this may be misleading.

[facelessuser]

Obviously, text/prose is copywritable, but we are essentially monkeying the behavior of PHP Markdown , not copying their code.

Maybe for safety, we recopy the tests cases from PHP Markdown and throw out those that we don't want to follow and update the license? I guess you can re-evaluate which test cases have changed since, and write your own that test the same thing...

[facelessuser]

Here's a discussion on the matter: https://opensource.stackexchange.com/questions/7503/implications-of-using-gpl-licenced-code-only-during-testing. Granted, I don't think anyone taking in that thread is a lawyer, and I'm not either. So, 🤷🏻.

[waylan]

Maybe for safety, we recopy the tests cases from PHP Markdown and throw out those that we don't want to follow and update the license? I guess you can re-evaluate which test cases have changed since, and write your own that test the same thing...

That is were I'm going with this. For that matter, maybe we don't even need any of the PHP tests at all. Back when those tests were new, PHP was one of the few leading Markdown implementations. Today, it is one of many. So unless there are some important tests in there that don't exist elsewhere, I'm inclined to just delete the entire directory. And if there are some important tests, then we should replace them with new tests using our new framework and delete the directory.

[OliverFendt]

@waylan and @facelessuser thank you for picking this up.
The most clear solution would be to follow your proposal @waylan to delete the entire directory, if it is not of value anymore.

[waylan]

I should point out that according to test coverage reports, we are relying on the PHP and/or Perl tests for some stuff. With all tests run (1038 tests, 24 skipped) , the coverage report looks like this:

Name                                  Stmts   Miss  Cover   Missing
-------------------------------------------------------------------
markdown/__init__.py                      5      0   100%
markdown/__main__.py                     42      0   100%
markdown/__meta__.py                     14      0   100%
markdown/blockparser.py                  30      0   100%
markdown/blockprocessors.py             310      7    98%   214, 219-220, 229, 281, 629, 635
markdown/core.py                        151      0   100%
markdown/extensions/__init__.py          29      0   100%
markdown/extensions/abbr.py             102      7    93%   64, 177-178, 181-184
markdown/extensions/admonition.py        86      0   100%
markdown/extensions/attr_list.py         97      0   100%
markdown/extensions/codehilite.py       115     18    84%   152-173, 237
markdown/extensions/def_list.py          58      2    97%   105-106
markdown/extensions/extra.py              9      0   100%
markdown/extensions/fenced_code.py       98      1    99%   126
markdown/extensions/footnotes.py        227      6    97%   139-140, 160, 167, 398-399
markdown/extensions/legacy_attrs.py      23      0   100%
markdown/extensions/legacy_em.py         12      0   100%
markdown/extensions/md_in_html.py       207      0   100%
markdown/extensions/meta.py              43      0   100%
markdown/extensions/nl2br.py              8      0   100%
markdown/extensions/sane_lists.py        23      0   100%
markdown/extensions/smarty.py            86      0   100%
markdown/extensions/tables.py           145      0   100%
markdown/extensions/toc.py              256     14    95%   72-78, 84-96
markdown/extensions/wikilinks.py         48      0   100%
markdown/htmlparser.py                  139      0   100%
markdown/inlinepatterns.py              407      2    99%   175, 852
markdown/postprocessors.py               56      2    96%   140, 143
markdown/preprocessors.py                27      0   100%
markdown/serializers.py                  95      0   100%
markdown/treeprocessors.py              227      5    98%   52, 112, 240-242
markdown/util.py                        151      7    95%   108, 128-133, 256-261
-------------------------------------------------------------------
TOTAL                                  3326     71    98%

However, on a local machine which doesn't have all of the necessary dependencies installed and the PHP and Perl tests are automatically skipped (1038 tests, 132 skipped), the coverage report looks like this:

Name                                  Stmts   Miss  Cover   Missing
-------------------------------------------------------------------
markdown\__init__.py                      6      0   100%
markdown\__main__.py                     42      0   100%
markdown\__meta__.py                     14      0   100%
markdown\blockparser.py                  31      0   100%
markdown\blockprocessors.py             304      8    97%   214, 219-220, 229, 247, 281, 629, 635
markdown\core.py                        148      0   100%
markdown\extensions\__init__.py          29      1    97%   112
markdown\extensions\abbr.py             102      7    93%   64, 177-178, 181-184
markdown\extensions\admonition.py        87      0   100%
markdown\extensions\attr_list.py         98      0   100%
markdown\extensions\codehilite.py       115     97    16%   48-52, 116-133, 146-188, 212-247, 260-265, 269-288, 296-335, 339-343
markdown\extensions\def_list.py          57      2    96%   105-106
markdown\extensions\extra.py              9      0   100%
markdown\extensions\fenced_code.py       97     10    90%   90, 115, 120-137, 162, 176
markdown\extensions\footnotes.py        227      7    97%   139-140, 160, 167, 290, 398-399
markdown\extensions\legacy_attrs.py      24      0   100%
markdown\extensions\legacy_em.py         13      0   100%
markdown\extensions\md_in_html.py       208      0   100%
markdown\extensions\meta.py              44      0   100%
markdown\extensions\nl2br.py              9      0   100%
markdown\extensions\sane_lists.py        21      0   100%
markdown\extensions\smarty.py            86      0   100%
markdown\extensions\tables.py           145      0   100%
markdown\extensions\toc.py              256     14    95%   72-78, 84-96
markdown\extensions\wikilinks.py         48      0   100%
markdown\htmlparser.py                  140      0   100%
markdown\inlinepatterns.py              384      2    99%   175, 852
markdown\postprocessors.py               57      2    96%   140, 143
markdown\preprocessors.py                28      0   100%
markdown\serializers.py                  96      0   100%
markdown\treeprocessors.py              228      5    98%   52, 112, 240-242
markdown\util.py                        143      7    95%   106, 128-133, 256-261
-------------------------------------------------------------------
TOTAL                                  3296    162    95%

It is clear the those reports are different. It is possible that those differences are all attributed to the Perl tests and removing the PHP make no difference, but we would need to confirm that. If any of the PHP tests are contributing to coverage, those need to be identified and replicated in the new test framework before we delete them. For that matter, we should also be replicating the relevant Perl tests as well, but that is not necessary to remove the PHP tests.

[facelessuser]

I'll likely start working through this as it is pretty important to get this taken care of before we do another release. My focus will be just on removing PHP tests, with an emphasis to make sure we don't drop in line coverage. I will try to identify unique, significant cases that I think should be replicated, but it's a lot of tests, so I'm not going to worry too much if some get missed.

[facelessuser]

It ended up not being too bad. Many tests were already disabled, and the remaining ones only had a handful of cases not covered elsewhere (at least I think, it's hard to know for sure). Anyway, a PR is up and I believe the coverage percentage is still intact.