Skip to content

Commit 294aeec

Browse files
committed
Corrections and clean-up
Back to Q8_0 for attn_k and attn_v if 8 experts or more. for attn_v and attn_k if experts>=4 GQA>=12 brought back to expert>=4 quant level instead of 8 GQA8 brought to GQA7, and GQA7 brought to GQA4.
1 parent e7c5163 commit 294aeec

File tree

1 file changed

+116
-141
lines changed

1 file changed

+116
-141
lines changed

0 commit comments

Comments
 (0)