Skip to content

[AutoDiff] Support custom derivatives for @_alwaysEmitIntoClient functions #78908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kovdan01
Copy link
Contributor

@kovdan01 kovdan01 commented Jan 25, 2025

Consider an @_alwaysEmitIntoClient function and a custom derivative defined
for it. Previously, such a combination resulted different errors under different
circumstances.

Sometimes, there were linker errors due to missing derivative function symbol -
these occurred when we tried to find the derivative in a module, while it
should have been emitted into client's code (and it did not happen).

Sometimes, there were SIL verification failures like this:

SIL verification failed: internal/private function cannot be serialized or serializable: !F->isAnySerialized() || embedded

Linkage and serialization options for the derivative were not handled properly,
and, instead of PublicNonABI linkage, we had Private one which is unsupported
for serialization - but we need to serialize @_alwaysEmitIntoClient functions
so the client's code is able to see them.

This patch resolves the issue and adds proper handling of custom derivatives
of @_alwaysEmitIntoClient functions. Note that either both the function and
its custom derivative or none of them should have @_alwaysEmitIntoClient
attribute, mismatch in this attribute is not supported.

The following cases are handled (assume that in each case client's code uses
the derivative).

  1. Both the function and its derivative are defined in a single file in
    one module.

  2. Both the function and its derivative are defined in different files which
    are compiled to a single module.

  3. The function is defined in one module, its derivative is defined in another
    module.

  4. The function and the derivative are defined as members of a protocol
    extension in two separate modules - one for the function and one for the
    derivative. A struct conforming the protocol is defined in the third
    module.

  5. The function and the derivative are defined as members of a struct
    extension in two separate modules - one for the function and one for the
    derivative.

The changes allow to define derivatives for methods of SIMD.

Fixes #54445

@kovdan01
Copy link
Contributor Author

Tagging @asl

@kovdan01 kovdan01 force-pushed the autodiff-alwaysEmitIntoClient branch from 196d2d4 to c75bc66 Compare January 25, 2025 07:53
@kovdan01 kovdan01 force-pushed the autodiff-alwaysEmitIntoClient branch from c75bc66 to bb90693 Compare January 30, 2025 08:44
@kovdan01 kovdan01 marked this pull request as ready for review January 30, 2025 08:45
@kovdan01
Copy link
Contributor Author

Tagging @JaapWijnen @rxwei

@asl
Copy link
Contributor

asl commented Jan 30, 2025

@swift-ci please test

@kovdan01 kovdan01 force-pushed the autodiff-alwaysEmitIntoClient branch from bb90693 to ffa5f81 Compare January 30, 2025 17:43
@asl
Copy link
Contributor

asl commented Jan 30, 2025

@swift-ci please test

@kovdan01 kovdan01 force-pushed the autodiff-alwaysEmitIntoClient branch from ffa5f81 to 6bb9def Compare February 3, 2025 15:37
@asl
Copy link
Contributor

asl commented Feb 3, 2025

@swift-ci please test

@kovdan01 kovdan01 force-pushed the autodiff-alwaysEmitIntoClient branch from 6bb9def to 0b9bf45 Compare February 6, 2025 00:34
@asl
Copy link
Contributor

asl commented Feb 6, 2025

@swift-ci please test

@kovdan01
Copy link
Contributor Author

kovdan01 commented Feb 6, 2025

Would be glad to see feedback from everyone interested

@asl asl added the AutoDiff label Feb 13, 2025
@kovdan01
Copy link
Contributor Author

Would be glad to see feedback from everyone interested

4 similar comments
@kovdan01
Copy link
Contributor Author

Would be glad to see feedback from everyone interested

@kovdan01
Copy link
Contributor Author

Would be glad to see feedback from everyone interested

@kovdan01
Copy link
Contributor Author

kovdan01 commented Mar 6, 2025

Would be glad to see feedback from everyone interested

@kovdan01
Copy link
Contributor Author

Would be glad to see feedback from everyone interested

@asl
Copy link
Contributor

asl commented Mar 13, 2025

@lorentey will you please take a look on stdlib changes?

@asl
Copy link
Contributor

asl commented Apr 7, 2025

@jckarter Can you take a look into sil linker part? Thanks!

@kovdan01
Copy link
Contributor Author

@lorentey Will you please take a look on stdlib changes?
@jckarter Can you take a look into sil linker part?
Thanks!

@asl
Copy link
Contributor

asl commented Apr 22, 2025

@kovdan01 Will you please update the description? To summarize what is done, what is the issue, etc.

@kovdan01
Copy link
Contributor Author

@kovdan01 Will you please update the description? To summarize what is done, what is the issue, etc.

Sure, I've updated the PR description.

@kovdan01
Copy link
Contributor Author

@lorentey Will you please take a look on stdlib changes?
@jckarter Can you take a look into sil linker part?
Thanks!

2 similar comments
@kovdan01
Copy link
Contributor Author

kovdan01 commented May 5, 2025

@lorentey Will you please take a look on stdlib changes?
@jckarter Can you take a look into sil linker part?
Thanks!

@kovdan01
Copy link
Contributor Author

@lorentey Will you please take a look on stdlib changes?
@jckarter Can you take a look into sil linker part?
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[SR-13689] derivative of @_alwaysEmitIntoClient function
2 participants