feat: Add HTTPS serving option #1000
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ashwinb
requested review from
dineshyv,
dltn,
ehhuang,
hardikjshah,
raghotham,
sixianyi0721,
terrytangyuan,
vladimirivic and
yanxi0830
as code owners
facebook-github-bot
added
the
CLA Signed
leseb
reviewed
Feb 7, 2025
Collaborator
leseb
left a comment
leseb
left a comment
There was a problem hiding this comment.
TLS has replaced SSL, so we should s/ssl/tls/. This also seems like a great addition to the CHANGELOG! 😊 Thanks!
raghotham
approved these changes
Feb 7, 2025
Contributor
|
Looks great! |
kaushik-himself
pushed a commit
to fiddlecube/llama-stack
that referenced
this pull request
Feb 10, 2025
# What does this PR do? Enables HTTPS option for Llama Stack. While doing so, introduces a `ServerConfig` sub-structure to house all server related configuration (port, ssl, etc.) Also simplified the `start_container.sh` entrypoint to simply be `python` instead of a complex bash command line. ## Test Plan Conda: Run: ```bash $ llama stack build --template together $ llama stack run --port 8322 # ensure server starts $ llama-stack-client configure --endpoint http://localhost:8322 $ llama-stack-client models list ``` Create a self-signed SSL key / cert pair. Then, using a local checkout of `llama-stack-client-python`, change https://github.com/meta-llama/llama-stack-client-python/blob/main/src/llama_stack_client/_base_client.py#L759 to add `kwargs.setdefault("verify", False)` so SSL verification is disabled. Then: ```bash $ llama stack run --port 8322 --tls-keyfile <KEYFILE> --tls-certfile <CERTFILE> $ llama-stack-client configure --endpoint https://localhost:8322 # notice the `https` $ llama-stack-client models list ``` Also tested with containers (but of course one needs to make sure the cert and key files are appropriately provided to the container.)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Enables HTTPS option for Llama Stack.
While doing so, introduces a
ServerConfigsub-structure to house all server related configuration (port, ssl, etc.)Also simplified the
start_container.shentrypoint to simply bepythoninstead of a complex bash command line.Test Plan
Conda:
Run:
$ llama stack build --template together $ llama stack run --port 8322 # ensure server starts $ llama-stack-client configure --endpoint http://localhost:8322 $ llama-stack-client models listCreate a self-signed SSL key / cert pair. Then, using a local checkout of
llama-stack-client-python, change https://github.com/meta-llama/llama-stack-client-python/blob/main/src/llama_stack_client/_base_client.py#L759 to addkwargs.setdefault("verify", False)so SSL verification is disabled. Then:Also tested with containers (but of course one needs to make sure the cert and key files are appropriately provided to the container.)