cannot describe a typical XML-based REST API in OpenAPI/Swagger

[abstratt]

I cannot describe a XML request/response that has different elements typed by the same complex type. That should be possible, as it is common practice in XML modeling (typically using XSD schemas).

Simple example: a customer has a billing address (element name: billing-address) and a residential address (residential-address).

Reasons:

• in order to reuse a complex type, in OpenAPI I must create a definition object, and refer to this definition in the $ref field in the body of request/response
• in order to declare the name, namespace, prefix and other XML-specific metadata in OpenAPI, I must declare a XMLobject (via a "xml" field)
• however, as per the spec, whenever you use $ref, other values are ignored, IOW, I cannot define the name/namespace/prefix of the XML element

See swagger-api/swagger-ui#3469 and #556.

[darrelmiller]

I'm not sure this limitation is specific to XML based payloads. Whenever you reference a "reusable" resource you cannot modify or add to that reusable component. That is a limitation of the $ref construct that we borrowed from JSON Schema. The JSON Schema folks are working on alternative mechanisms but currently there is no ability to do this.

You are limited to defining the xml field in the reused definition. Is there a reason this does not work for you?

[abstratt]

@darrelmiller This is the key requirement: "different elements typed by the same complex type" - they would all be deemed to have the same element name. The linked swagger-ui issue includes an example.

[darrelmiller]

@abstratt Ah, I understand. Yes that is a limitation, however I would argue that doing that in XML isn't a particularly wise thing. Elements usually have a different structure. If you just want a descriptive difference between elements then create a name attribute on the element. I realize that is probably not possible to change the XML in your case.

From a longer term perspective, I would like to see the addition of XSD support to OpenAPI rather than trying to fix the Xml annotations.

[abstratt]

Thanks for the idea @darrelmiller, but I hope we agree that is a workaround, and with some serious limitations:

A customer may have two addresses (not three or more), a residential one, and a billing one. The latter is optional.

That is trivial to model with XSD. It should be possible to model with Swagger as well. But it is not, actually not even if you use JSON (because you cannot reuse complex types, but only entire property definitions, including things like descriptions, 'requiredness', etc - please correct me if that statement is incorrect, as I see it as the root cause for the present issue, and a major design issue with OpenAPI/Swagger. TBH, I am still hoping I am just missing something obvious, and someone shows me how to reuse complex types in Swagger).

[darrelmiller]

@abstratt JSON Schema doesn't make a distinction between complex types and schemas. Unfortunately you are trying to do something that JSON Schema wasn't designed to do. Currently OpenAPI is somewhat constrained by the limitations of JSON Schema when it comes to description of payloads. Hopefully that won't always be the case.

[abstratt]

@darrelmiller Thanks. Here's hoping that changes soon. That is a serious drawback in JSON Schema, one which Swagger/OpenAPI should not be inheriting, as it limits the description of JSON-based APIs, and prevents description of many XML-based APIs.

[handrews]

I think that draft-07 (which should be published by November 20th) and draft-08 will help with this sort of thing. And the relevant ideas in draft-07 don't technically require an implementation change.

draft-07 classifies keywords as data model, assertions, or annotations, and is more formal about how annotations are applied to instance data. xml is an annotation keyword: it adds information to the instance that is used by other tools. Note that this is not a change from draft-05, it's just giving names to things that were not called out clearly before.

Based on how we define annotations and the concept of applicability in draft-07, I would expect this to work:

propA:
  allOf:
    - $ref: '#/definitions/MyType2'
    - xml:
        name: prop-a

I assume that it does not in current OpenAPI implementations?

As for draft-08, it will focus on re-usability, including several issues around how things with allOf and/or $ref function. There is a draft-08 milestone if you want to keep an eye on that.

[chippyash]

This may seem off the wall, but JSON schema definition seems to be in the same state that XML was about a millenia ago at version 0.5 or some such. My 2 cents worth is that OpenApi should not tie itself to an emergent standard that could well take many years to become operable. By comparison, XML, XSLT, XSD family of technologies could well have resolved most of the issues being found with OpenAPI 3 et al had they been adopted as the underlaying schema definition. Not suggesting it changes, just saying it's all been done before.

[patricekrakow]

Wow, a good old XML vs JSON discussion. Just wanted to bring back https://blog.jclark.com/2010/11/xml-vs-web_24.html in the discussion. Coming from XML (maybe even from SGML) this post has changed my relationship with JSON almost 9 years ago. BTW, James is back after (also) almost 9 years of Blog silence to talk about Ballerina ;-)
Disclaimer: I still enjoy writing parsers for X12 and EDIFACT ;-)

[darrelmiller]

@chippyash The alternate schema work that is ongoing will ensure that no-one is tied to any particular schema language. Despite it's "draft" status, JSON Schema is far from emergent. I would argue it has more real world traction than XML Schema ever did.
XML Schema has a fundamentally different world view than JSON Schema. The XML Schema "closed world" principle does not fit nearly as well with the loosely coupled web.

I am and always will be a fan of XML and XSLT. Less so of XML Schema, but XML tech just isn't the best tool for the job when describing payloads for HTTP APIs.

[abstratt]

This issue is not really a contest between JSON or XML, but about a relevant gap in OpenAPI spec in which it does not allow a fairly common modeling pattern (multiple child objects of the same base type with different roles/under different names) for XML-based APIs, ironically due to a limitation in JSON Schema.

Hopefully OpenAPI can loosen this restriction soon, as many well designed XML based APIs out there cannot be modeled with OpenAPI, as it stands now.

[handrews]

@chippyash @patricekrakow @abstratt @darrelmiller Please note that we finally, after over a year of work, released the 2019-09 draft of JSON Schema yesterday.

In IETF terms, this consists of several draft-handrews-*-02 documents (there is a giant table of all of the versions and identifiers on json-schema.org now, explaining IETF vs meta-schema and why we're using dates now).

Of particular interest may be the hypothetical code generation example in Appendix E of the Core spec. This is intended to illustrate some techniques related to how JSON Schema 2019-09 could be extended for code generation.

@darrelmiller and the rest of the OpenAPI TSC have been working with @philsturgeon and myself on the JSON Schema side to ensure that JSON Schema is a better platform for the needs of OAS. Between proposals for improving compatibility of the Schema Object in OAS 3.1, and the alternate-schema work that @darrelmiller mentioned, I am optimistic about future functionality and alignment.

[chippyash]

Oops. @patricekrakow I have to grin. I read James' blog about Ballerina too. Kinda how I ended up here. Was very interested in it's OpenApi (although limited) capability, started to play, delved deeper into the standard and started to find some fundamental issues particularly in the nesting and multiple child arenas that @abstratt refers to. I'm going to use OpenApi, but instruct my devs to use copious examples to get around the limitation in the definition itself. Just means we'll have to hand code those bits in our implementations. Anyone remember the processors you could inject in an XSLT definition to get round local difficulties? ;-) Still going to use XML for my DI container definitions though! And @handrews, thanks for all the good work you and your colleagues are doing to even try and establish a standard for API definitions.

[handrews]

@chippyash 🙏

For XML definitely the alternate-schema work will be more helpful for you! While the flexibility we've put into JSON Schema 2019-09 should eventually make it more possible to describe other things, having the ability to use XML Schema for XML payloads is something that I very much support.

[chippyash]

@handrews I wasn't really saying I want to have the ability to specify XML schema, although having to support clients that use XML and JSON (and a few other strange things that I can't remember the names of) would most certainly be useful. It was more about applying the pain that was gone through in XMLs birth to the construction of a JSON schema definition. I've shied away from using JSON as a reliable data transport mechanism simply because I couldn't validate it reliably and have built whole libraries around doing just that. I welcome a standardised mechanism to do that validation whole heartedly. Sure XSDs can be a PITA to design and implement, but at least there is only one way to do it, (barring a few processor 'flavours'!). I guess my observation was kind of asking does the JSON schema group have old buggers with that pain involved in bringing a new standard into the world? I don't know but I guess I kinda hope so.

[philsturgeon]

@chippyash I think it sounds like you have feedback on how JSON Schema works, or used to work, or need support on a certain use case, or something. This doesn't seem like a "XML in OpenAPI" conversation anymore though, so lets close this one down?

If JSON Schema cannot do what you want, and the XML Object is not good enough, then XML Schema in Alternative Schema is going to solve your problems when it gets done. So its a solved problem however you spin it.

[abstratt]

Can we please have a link to the document describing how to model XML APIs in OpenAPI using the a "Alternative Schema" approach (which I confess I know nothing about) which avoids the limitation reported here?

If that is doable with a new approach (and the new approach is already available even if in beta form etc), yup, closing this issue makes sense.

[philsturgeon]

Alternative Schema is a draft feature still being worked on, but it will allow XSD as @chippyash is requesting. People who want alternative Schema can join in various conversations about alternative Schema. #1993

The main “doing this in JSON Schema + xml keyword” issue is solved by OpenAPI raising support for JSON Schema Draft 2019-09 in #1977

So anyone interested in those two topics can track progress there. This issue is now superfluous, confusingly named, and sinking in slightly in accurate person opinions about JSON and JSON Schema in general, which will probably not lead anywhere constructive.

[chippyash]

@philsturgeon i didn't start this thread. Far be it from me to close it down. It does however leave some unanswered questions, however uncomfortable they are. By all means close it down, but how about raising issue tickets to resolve the the open questions (many here in this thread alone) that have been identified. This is OpenAPI. The JSON schema issue aside, OpenAPI is not dealing with some of the real world use cases identified in this thread alone.

[darrelmiller]

The TSC have done work to bring the Alternative Schema proposal to the community as a draft feature. This will allow XML based APIs to be described accurately. The next step in the draft feature process is for people building tools to implement the draft feature and provide feedback on its feasibility. If we get multiple implementations then we will incorporate the feature into the specification.

The OpenAPI TSC has done what it can at this point to support this feature. If you need proper support for XML, then please encourage the people building the tools you use to implement the alternative schema draft feature and provide us with real world implementation experience.

There doesn't need to be any further discussion on this thread.