Skip to content

fused_moe configs for MI325X #300

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Dec 3, 2024
Merged

fused_moe configs for MI325X #300

merged 3 commits into from
Dec 3, 2024

Conversation

JArnoldAMD
Copy link

New fused_moe configs for Mixtral-8x7B and Mixtral-8x22B with TP=1,2,4,8 for both FP8 and FP16 on the recently announced MI325X.

maleksan85 and others added 2 commits November 13, 2024 10:22
@maleksan85
corrected types for strides in triton FA (ROCm#274) (ROCm#276)
efb0432 
Co-authored-by: Aleksandr Malyshev <[email protected]>
(cherry picked from commit 9a46e97)
@JArnoldAMD
fused_moe configs for MI325X
a405057 
New fused_moe configs for Mixtral-8x7B and Mixtral-8x22B with
TP=1,2,4,8 for both FP8 and FP16 on the recently announced MI325X.
shajrawi
shajrawi reviewed Dec 3, 2024
View reviewed changes
Copy link
Collaborator

@shajrawi shajrawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please make the PR vs develop branch? since that is the one we are currently using and main is frozen at the moment?

@JArnoldAMD JArnoldAMD changed the base branch from main to develop December 3, 2024 19:26
@JArnoldAMD
Copy link
Author

Can you please make the PR vs develop branch? since that is the one we are currently using and main is frozen at the moment?

Oops, sorry about that. Switched to the develop branch now.

shajrawi
shajrawi approved these changes Dec 3, 2024
View reviewed changes
Copy link
Collaborator

@shajrawi shajrawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for doing this! Ship it :)

@shajrawi shajrawi merged commit 0cee60d into ROCm:develop Dec 3, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shajrawi shajrawi shajrawi approved these changes

@divakar-amd divakar-amd Awaiting requested review from divakar-amd

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@JArnoldAMD @shajrawi @maleksan85