Switch to APIDevTools/json-schema-ref-parser

[philsturgeon]

Currently we use our home grown json-ref-resolver, but we've switched most of our ecosystem over to APIDevTools/json-schema-ref-parser for it's superior bundling strategy (https://github.com/stoplightio/studio/issues/275) amongst other things.

We've got some outstanding pull requests going on, but so to avoid extra work let's wait for APIDevTools/json-schema-ref-parser#153 to be merged, then switch Spectral over.

[philsturgeon]

Work was started on this a while back but not completed. Feel free to use this as a base: https://github.com/stoplightio/spectral/tree/feat/schema-ref-parser

[lehphyro]

Are you guys planning to finish the switch soon? The linked issues forced us to disable linting in our projects due to random errors produced by failure to resolve references properly.

[philsturgeon]

@lehphyro most of the examples in the linked issue (#870) have workarounds, and some of them even seem to indicate user error, so we've focused on frying bigger fish for now. Is there no workaround for your infinitely circular dependency scenario?

[lehphyro]

@philsturgeon It's not about only circular refs, I've created a repo that you can use to reproduce the issue: https://github.com/lehphyro/spectral-ref-issue

I usually use maven to run node tools, so you can reproduce it by cloning that repo and then running mvn clean install from the root directory, running spectral binary directly yields the same result.

This is the error you should see:

$ spectral lint src/main/resources/openapi.yaml --ruleset openapi-rules.yaml
OpenAPI 3.x detected

/repo/spectral-ref-issue/src/main/resources/examples/response.json
 1:1  error  examples-responses-are-valid  can't resolve reference schemas/statement.yaml from id #

✖ 1 problem (1 error, 0 warnings, 0 infos, 0 hints)

[philsturgeon]

Interesting. There may well be some sort of issue from having a model load up the main openapi file again, which is in turn trying to load up a bunch of refs and then some of them call back to the main file, and on it goes.

I'd switch this:

  documents:
    description: List of documents provided by the bank
    items:
      $ref: '../openapi.yaml#/components/schemas/AccountStatementIdentification'
    type: array

to this:

  documents:
    description: List of documents provided by the bank
    items:
      $ref: schemas/statement-id.yaml
    type: array

See if that sorts you out.

[lehphyro]

I cannot unfortunately, I need to have all $refs point to the components section in order to have https://openapi-generator.tech/ generate code with good class names.

[lehphyro]

Should I assume that this won't be done any time soon? Since I'll have to switch to another linter entirely, it'd be nice to know it before making the effort.

[philsturgeon]

@lehphyro Lots of work is being done on json-schema-ref-parser and several releases have been made since these discussions started, so I'm sure whatever concerns you have with it will be solved soon enough, if not already.

The plan is to wrap up the resolution logic in an adapter pattern as part of v6, and provide an adapter for both resolvers - or allow people to pass in their own entirely - therefore separating the resolution and linting logic entirely. You can keep using the old one until json-schema-ref-parser has solved whatever problems are keeping you off it.

Consolidation of efforts is the goal here, meaning two teams are working on improving one tool instead of us both playing whackamole with various edge cases. Neither are good or bad, they've just both got different problems, but json-schema-ref-parser generally has fewer of them, and is getting a lot of PRs from us to remove the rest. Soon comes the new $ref logic for $id and OAS3.1/JSON Schema 2019-09 logic (APIDevTools/json-schema-ref-parser#145), so... no point us all doing that twice.

[lehphyro]

@philsturgeon thank you for the update, I'll give it a try when a working solution becomes available.