[Model] GPTBigCodeForEmbedding supporting token span classification #13684
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a new model GPTBigCodeForEmbedding. The task this model supports is classifying spans of tokens, either based on both the first and last token of the span or based only on the last token. It reuses the PoolerOutput so that the model can be called with existing APIs.
The token span classification can also be added to other models, but I have kept the PR contained to only the new
gpt_bigcode_embedding.py
file, and the necessary change toregistry.py