Skip to content

Commit 809489a

Browse files
committed
Add a proposed mechanism for tracking known "quality" implementations.
There is obvious slight overlap here with our existing list of implementations on the web page -- note though that we have loose (or no) criteria for listing implementations on the page. Here instead we wish to specifically raise languages for which we know no good implementation to exist, despite the language being popular or widespread, if any. What's here does *not* yet render this data anywhere, nor does it implement the dynamic API calling mentioned in the body (for retrieving statistics from any listed repository). What it does do is identify a simple initial list of languages (from TIOBE and GitHub's own lists), and propose a format for tracking data points about implementations meeting the proposed criteria. The initial goal is to get feedback on this template, and to solicit help in filling it in for these languages. Refs: #418
1 parent 48e6d9c commit 809489a

File tree

1 file changed

+74
-0
lines changed

1 file changed

+74
-0
lines changed
Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Cross-language Implementation Quality
2+
3+
This directory collects metrics to assist in confirming the presence of (at least one) "high quality" implementations across programming languages.
4+
5+
Absence from this directory is not meant to be a slight against anyone or anyone's work certainly.
6+
7+
The goals are to identify gaps in the ecosystem that might need assistance, and ultimately to help the JSON Schema ecosystem and its users in finding solid, compliant implementations.
8+
9+
The intial collection of TODO programming languages which appears below is simply a starting point, based largely on:
10+
11+
* the [GitHub popularity list](https://octoverse.github.com/2022/top-programming-languages)
12+
* the [TIOBE index](https://www.tiobe.com/tiobe-index/)
13+
14+
where it combines languages at the top of the popularity list with those on the rising / fastest growing list.
15+
16+
You can help this exercise!
17+
For each language, we wish to track whether one or more known implementations exist which:
18+
19+
* Fail less than 2% (around 30) tests from the official JSON Schema test suite
20+
(This 2% number was chosen simply because it represents the dividing point between one cluster of implementations supported by Bowtie.
21+
Above this number is a collection of implementations failing larger orders of magnitude of tests).
22+
* Supports one of the most recent 2 releases of JSON Schema
23+
* Has had a release of itself in the last 3 years (which is around the timeline on which JSON Schema itself has seen its last release)
24+
* Has had some human-authored commit in the last year
25+
* Does not have explicit indication, by its maintainer or documentation, that it is not production ready
26+
27+
and then highlight any languages for which this is not the case.
28+
29+
For languages that *do* have one or more such implementations, we then wish to include:
30+
31+
* the name and link to the implementation
32+
* the date of its last commit
33+
* the date of its last release
34+
* the number of open security vulnerabilities it has, if known
35+
* the number of open issues it has on its issue tracker
36+
* the number of open pull requests it has
37+
* whether it implements some or all of the interfaces specified on our Common Interfaces page (https://json-schema.org/implementers/interfaces),
38+
and if so, which
39+
40+
A number of the above data points are likely to change often.
41+
We therefore assume only some of the above should be tracked in this directory, and the remainder will be retrieved programmatically if or when this information is used.
42+
43+
An example entry for this directory is:
44+
45+
```yaml
46+
language: Blub
47+
implementations:
48+
- name: jsonschema-ftw
49+
url: https://example.com/someone/jsonschema-ftw
50+
known_to_have:
51+
many_failing_tests: false
52+
recent_draft_support: true
53+
recent_release: true
54+
recent_commit: true
55+
non_production_status: false
56+
metrics:
57+
# last_commit_date: fetched from GitHub
58+
# last_release_date: fetched from GitHub
59+
# open_security_vulnerabilities: fetched from GitHub
60+
# open_issues: fetched from GitHub
61+
# open_prs: fetched from GitHub
62+
known_interfaces:
63+
instance-validation: https://docs.jsonschema-ftw.com/validation/validate_function
64+
version-detection: https://docs.jsonschema-ftw.com/validation/version_of_schema
65+
```
66+
67+
Note that *not* all of the above must precisely match in order to be considered for inclusion!
68+
Specifically, an implementation with *no* recent commits is not disqualified from being considered "high quality".
69+
One with a high number of failing tests however, is indeed likely not to meet the definition.
70+
The above is simply meant to track these data points without necessarily imposing all of them as gospel -- users of these libraries should make their own judgements there.
71+
72+
Please pick a language you're familiar with and feel free to fill in such an entry!
73+
74+
Feedback on this format itself is also welcome in an issue!

0 commit comments

Comments
 (0)