Skip to content

Conversation

ngxson
Copy link
Collaborator

@ngxson ngxson commented Jul 23, 2024

Ref discussion: #8636 (comment)

llama_lora_adapter_clear() can be used when user want to switch the adapter, but don't know which adapters are loaded into llama_context to be removed.

For simple task switching application, user can write their own code based on this function:

void switch_my_adapter(struct llama_context * ctx, struct llama_lora_adapter * adapter) {
  llama_lora_adapter_clear(ctx);
  llama_lora_adapter_set(ctx, adapter, 1.0);
}

@ngxson ngxson requested a review from ggerganov July 23, 2024 17:45
@ngxson ngxson mentioned this pull request Jul 23, 2024
4 tasks
@zhipenghan
Copy link

Thank you for implementing this change. Now, users can manage adaptors outside of the library as if the existing adaptors were invisible.

@ngxson ngxson merged commit b115105 into ggml-org:master Jul 24, 2024
@ngxson
Copy link
Collaborator Author

ngxson commented Jul 24, 2024

Oops sorry I didn't modify the merge commit message to specify module name

arthw pushed a commit to arthw/llama.cpp that referenced this pull request Jul 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants