Skip to content

[Feature][Encoder-Decoder]: Add support for cuda graph during decoding in encoder-decoder models #7447

@sroy745

Description

@sroy745

🚀 The feature, motivation and pitch

Currently for encoder-decoder models we don't support cuda graph during the decode phase. This fr tracks adding support for cuda graph during decode phase. Adding this support will help speed up the decode phase.

#7366

cc: @afeldman-nm

Alternatives

No response

Additional context

No response

Metadata

Metadata

Assignees

Labels

feature requestNew feature or requeststaleOver 90 days of inactivity

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions