[UR] Implement extension mechanism #458

veselypeta · 2023-04-24T14:44:27Z

This MR:

Introduces a basic concept of extensions for unified runtime.
The extensions specification is largely based on L0 see here
Introduces urPlatformGetExtensionProperties to query which extensions are supported.

veselypeta · 2023-04-24T14:48:42Z

scripts/core/EXT.rst

+- Extensions must have globally unique extension names reported from ${x}PlatformGetExtensionProperties
+- All extensions must be defined in this specification
+- Extensions must not break backwards compatibility of the core APIs
+- Standard extension versions must be backwards compatible with prior versions


I copied this from L0 extension scheme, but perhaps we could use semantic versioning with extensions with major/minor versions?

scripts/core/EXT.rst

scripts/core/platform.yml

AerialMantis · 2023-05-02T12:57:41Z

scripts/core/EXT.rst

+- Extensions must use globally unique names for macros, enums, structures and functions
+- Extensions must have globally unique extension names reported from ${x}PlatformGetExtensionProperties
+- All extensions must be defined in this specification
+- Extensions must not break backwards compatibility of the core APIs


Does this also include semantic breaks as well as interface breaks in the core APIs, i.e. does this forbid an extension from altering the behavior of an existing API?

I think an extension can alter the behavour of an existing API. For example you might want to chain an additional structure to the input or use extension specific enums, but default usage should remain the same.

Hmm, I'm not sure I'm so confortable with extensions that change semantics because we have no way to opt-out with this architecture. If we were able to enable extensions, like Vulkan, maybe I'd be more conformtable with that.

AerialMantis · 2023-05-02T12:58:59Z

scripts/core/EXT.rst

+- Extensions must have globally unique extension names reported from ${x}PlatformGetExtensionProperties
+- All extensions must be defined in this specification
+- Extensions must not break backwards compatibility of the core APIs
+- Standard extension versions must be backwards compatible with prior versions


Does this mean that experimental extensions are permitted to break backwards compatibility?

Yes I think that makes sense. Experimental extensions should have no ABI or backwards compatibility requirements.

AerialMantis · 2023-05-02T13:57:59Z

scripts/core/EXT.rst

+There are two types of extensions defined by this specification:
+
+1. **Standard** -  extension will be included into the current and all future version of the specification.
+2. **Experimental** - extensions require additional experimentation and development, before becoming a standard extension. 


Will there be implementation or coverage requirements for when an extension should move from experimental to standard?

AerialMantis · 2023-05-02T14:00:49Z

scripts/core/EXT.rst

+* Document the extension in `EXT_<ext-name>.rst` file based on the `EXT_Template.rst`, ensuring it is added to the list of
+  extensions below.
+
+* The extension will only be accepted if all conditions are met and its addition is agreed upon by the Working Group.


Does this refer to an extension becoming a standard extension?

AerialMantis · 2023-05-02T14:07:18Z

scripts/core/EXT.rst

+* The extension will only be accepted if all conditions are met and its addition is agreed upon by the Working Group.
+
+
+List of Standard Extensions


Just a thought, maybe these lists should also track the implementation status of the extensions, so whether they are implemented, and then which version of UR supports which version of the extension, particularly for the experimental extensions which may change, so high-level languages know what they can use when updating the UR interface.

scripts/core/EXT.rst

EwanC · 2023-05-03T10:25:08Z

scripts/core/EXT_Template.rst

+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In this section describe in detail the purpose of the extension, along with its
+valid usage.


Quite like it when extensions have a version history at the bottom, could we add that to the template?

This is useful info, I do wonder if having the ability to view different extension versions side-by-side would also be useful though (other than looking at the git history)?

That would be really good. @kbenzie I've seen versioned docs before on the Zephyr project link - there is a dropdown in the bottom left where you can select the version of the API you want to view. I'm not really sure how this is implemented, but perhaps we can have something like that as well as a changelog (like ewan suggested).

Looking at the Zephyr page I think that's for the entire page rather than just the extension specs, my gut feeling is you'd need to have a separate Sphinx build for each extension to get something exapctly like that.

scripts/core/EXT_Template.rst

veselypeta · 2023-05-04T10:00:52Z

In the WG there was a strong response against implementing extensions in this way - especially standard extensions. Instead we will create a Contribution Guide which will outline how anyone will be able to add "experimental features" to extend the API for their purposes. We have not yet decided on how we should then make an "experimental feature" a core feature, but we cannot make any references to the working group as this would inhibit 3rd party contributions.

veselypeta · 2023-05-18T09:56:28Z

Closing this in favour of Experimental Features

Rather than the user deciding whether to use the emulation mode or backend command-buffer mode, it is less error prone to do it programmatically as a user can't select an unsupported config. This also fixes #90 where currently the `SYCL_EXT_ONEAPI_GRAPH` macro isn't defined when in emulation mode. Done by using a PI device info query in finalization. This also corresponds to a new UR device info query until the extension mechanism is decided oneapi-src/unified-runtime#458 but will be superceeded by whatever extension reporting mechanism is decided on. The only PI backend which reports support for command-buffer implementation is Level Zero, the other backends/adapters report no support. The vendor test macro test-e2e test is re-enabled with this change.

veselypeta marked this pull request as draft April 24, 2023 14:46

veselypeta commented Apr 24, 2023

View reviewed changes

EwanC reviewed Apr 27, 2023

View reviewed changes

scripts/core/EXT.rst Outdated Show resolved Hide resolved

scripts/core/EXT.rst Show resolved Hide resolved

scripts/core/EXT.rst Show resolved Hide resolved

kbenzie reviewed Apr 27, 2023

View reviewed changes

scripts/core/EXT.rst Outdated Show resolved Hide resolved

scripts/core/EXT.rst Outdated Show resolved Hide resolved

kbenzie added the needs-discussion This needs further discussion label Apr 28, 2023

kbenzie reviewed May 1, 2023

View reviewed changes

scripts/core/platform.yml Outdated Show resolved Hide resolved

scripts/core/platform.yml Outdated Show resolved Hide resolved

veselypeta force-pushed the petr/document-extensions branch 2 times, most recently from ef696fc to 5534b9d Compare May 2, 2023 10:25

kbenzie mentioned this pull request May 2, 2023

Unified Runtime Command Buffers #13

Closed

Petr Vesely added 4 commits May 2, 2023 14:32

[UR] Implement extension mechanism

620f680

[UR] Address review comments

5acf665

[UR] Add extension documentation template

5c629a0

[WIP] make char*

ccb757e

veselypeta force-pushed the petr/document-extensions branch from 68764b5 to ccb757e Compare May 2, 2023 13:38

AerialMantis reviewed May 2, 2023

View reviewed changes

EwanC reviewed May 3, 2023

View reviewed changes

scripts/core/EXT.rst Outdated Show resolved Hide resolved

EwanC reviewed May 3, 2023

View reviewed changes

[UR] add opaque data extension

554315b

EwanC mentioned this pull request May 8, 2023

[SYCL][Graph] Query for backend command-buffer support reble/llvm#161

Merged

veselypeta closed this May 18, 2023

veselypeta deleted the petr/document-extensions branch June 15, 2023 08:27

		* The extension will only be accepted if all conditions are met and its addition is agreed upon by the Working Group.


		List of Standard Extensions

[UR] Implement extension mechanism #458

[UR] Implement extension mechanism #458

Uh oh!

Conversation

veselypeta commented Apr 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

veselypeta May 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

veselypeta May 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kbenzie May 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

veselypeta May 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

veselypeta commented May 4, 2023

Uh oh!

veselypeta commented May 18, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

veselypeta commented Apr 24, 2023 •

edited

Loading

veselypeta May 3, 2023 •

edited

Loading

veselypeta May 3, 2023 •

edited

Loading

kbenzie May 3, 2023 •

edited

Loading

veselypeta May 3, 2023 •

edited

Loading