Skip to content

Fix the max window length fetching in local mode #1016

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Oct 4, 2023
Merged

Fix the max window length fetching in local mode #1016

merged 3 commits into from
Oct 4, 2023

Conversation

jinhongyii
Copy link
Member

Fixes #1013

@jinhongyii
Copy link
Member Author

cc: @tqchen @MasterJH5574

@MasterJH5574
Copy link
Member

Let’s take the minimum value of the one specified in config and the one in the metadata.

@MasterJH5574 MasterJH5574 merged commit 1384f23 into mlc-ai:main Oct 4, 2023
@MasterJH5574 MasterJH5574 mentioned this pull request Oct 4, 2023
Closed
2 tasks
@junrushao
Copy link
Member

This PR breaks the multi-GPU integration which relies on reading num_shards from mlc-chat-config.json

@jinhongyii jinhongyii mentioned this pull request Oct 5, 2023
Merged
junrushao pushed a commit that referenced this pull request Oct 5, 2023
@jinhongyii
[Bugfix] Fix Disco break (#1022)
b6d7958 
Fix disco break introduced by #1016
@CharlieFRuan CharlieFRuan mentioned this pull request Nov 18, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@MasterJH5574 MasterJH5574 MasterJH5574 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Tracking] Fix the context length fetching in local mode
3 participants
@jinhongyii @MasterJH5574 @junrushao