feat(UI): Adding Conversation History #3203
Conversation
meta-cla
bot
added
the
CLA Signed
franciscojavierarceo
changed the title
franciscojavierarceo
force-pushed
the
session-manager
branch
from
92f2944 to
df0710a
Compare
| name: string; | ||
| messages: Message[]; | ||
| selectedModel: string; | ||
| selectedVectorDb: string; |
There was a problem hiding this comment.
forthcoming
franciscojavierarceo
force-pushed
the
session-manager
branch
2 times, most recently
from
502615d to
83aa78f
Compare
Signed-off-by: Francisco Javier Arceo <[email protected]>
franciscojavierarceo
force-pushed
the
session-manager
branch
2 times, most recently
from
842f272 to
77ed07f
Compare
franciscojavierarceo
requested review from
ashwinb,
bbrowning,
ehhuang,
hardikjshah,
leseb,
mattf,
raghotham,
reluctantfuturist,
slekkala1,
terrytangyuan and
yanxi0830
as code owners
our agents API offers sessions right? to me this was the reason why we had them. could you explain why that doesn't or won't work? |
|
I was uncertain about the state of the agents API given some discussions during the office hours, but if we intend on maintaining agents API I'll move it there. |
Signed-off-by: Francisco Javier Arceo <[email protected]> chore: Enable keyword search for Milvus inline (llamastack#3073) With milvus-io/milvus-lite#294 - Milvus Lite supports keyword search using BM25. While introducing keyword search we had explicitly disabled it for inline milvus. This PR removes the need for the check, and enables `inline::milvus` for tests. <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> Run llama stack with `inline::milvus` enabled: ``` pytest tests/integration/vector_io/test_openai_vector_stores.py::test_openai_vector_store_search_modes --stack-config=http://localhost:8321 --embedding-model=all-MiniLM-L6-v2 -v ``` ``` INFO 2025-08-07 17:06:20,932 tests.integration.conftest:64 tests: Setting DISABLE_CODE_SANDBOX=1 for macOS =========================================================================================== test session starts ============================================================================================ platform darwin -- Python 3.12.11, pytest-7.4.4, pluggy-1.5.0 -- /Users/vnarsing/miniconda3/envs/stack-client/bin/python cachedir: .pytest_cache metadata: {'Python': '3.12.11', 'Platform': 'macOS-14.7.6-arm64-arm-64bit', 'Packages': {'pytest': '7.4.4', 'pluggy': '1.5.0'}, 'Plugins': {'asyncio': '0.23.8', 'cov': '6.0.0', 'timeout': '2.2.0', 'socket': '0.7.0', 'html': '3.1.1', 'langsmith': '0.3.39', 'anyio': '4.8.0', 'metadata': '3.0.0'}} rootdir: /Users/vnarsing/go/src/github/meta-llama/llama-stack configfile: pyproject.toml plugins: asyncio-0.23.8, cov-6.0.0, timeout-2.2.0, socket-0.7.0, html-3.1.1, langsmith-0.3.39, anyio-4.8.0, metadata-3.0.0 asyncio: mode=Mode.AUTO collected 3 items tests/integration/vector_io/test_openai_vector_stores.py::test_openai_vector_store_search_modes[None-None-all-MiniLM-L6-v2-None-384-vector] PASSED [ 33%] tests/integration/vector_io/test_openai_vector_stores.py::test_openai_vector_store_search_modes[None-None-all-MiniLM-L6-v2-None-384-keyword] PASSED [ 66%] tests/integration/vector_io/test_openai_vector_stores.py::test_openai_vector_store_search_modes[None-None-all-MiniLM-L6-v2-None-384-hybrid] PASSED [100%] ============================================================================================ 3 passed in 4.75s ============================================================================================= ``` Signed-off-by: Varsha Prasad Narsing <[email protected]> Co-authored-by: Francisco Arceo <[email protected]> chore: Fixup main pre commit (llamastack#3204) build: Bump version to 0.2.18 chore: Faster npm pre-commit (llamastack#3206) Adds npm to pre-commit.yml installation and caches ui Removes node installation during pre-commit. <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> <!-- Describe the tests you ran to verify your changes with result summaries. *Provide clear instructions so the plan can be easily re-executed.* --> Signed-off-by: Francisco Javier Arceo <[email protected]> chiecking in for tonight, wip moving to agents api Signed-off-by: Francisco Javier Arceo <[email protected]> remove log Signed-off-by: Francisco Javier Arceo <[email protected]> updated Signed-off-by: Francisco Javier Arceo <[email protected]> fix: disable ui-prettier & ui-eslint (llamastack#3207) chore(pre-commit): add pre-commit hook to enforce llama_stack logger usage (llamastack#3061) This PR adds a step in pre-commit to enforce using `llama_stack` logger. Currently, various parts of the code base uses different loggers. As a custom `llama_stack` logger exist and used in the codebase, it is better to standardize its utilization. Signed-off-by: Mustafa Elbehery <[email protected]> Co-authored-by: Matthew Farrellee <[email protected]> fix: fix ```openai_embeddings``` for asymmetric embedding NIMs (llamastack#3205) NVIDIA asymmetric embedding models (e.g., `nvidia/llama-3.2-nv-embedqa-1b-v2`) require an `input_type` parameter not present in the standard OpenAI embeddings API. This PR adds the `input_type="query"` as default and updates the documentation to suggest using the `embedding` API for passage embeddings. <!-- If resolving an issue, uncomment and update the line below --> Resolves llamastack#2892 ``` pytest -s -v tests/integration/inference/test_openai_embeddings.py --stack-config="inference=nvidia" --embedding-model="nvidia/llama-3.2-nv-embedqa-1b-v2" --env NVIDIA_API_KEY={nvidia_api_key} --env NVIDIA_BASE_URL="https://integrate.api.nvidia.com" ``` cleaning up Signed-off-by: Francisco Javier Arceo <[email protected]> updating session manager to cache messages locally Signed-off-by: Francisco Javier Arceo <[email protected]> fix linter Signed-off-by: Francisco Javier Arceo <[email protected]> more cleanup Signed-off-by: Francisco Javier Arceo <[email protected]>
franciscojavierarceo
force-pushed
the
session-manager
branch
from
44b044c to
6620b62
Compare
| }); | ||
| }; | ||
|
|
||
| const response = await client.agents.turn.create( |
There was a problem hiding this comment.
now using agents api
| } | ||
|
|
||
| console.log("📡 Fetching agent config from API..."); | ||
| const agentDetails = await client.agents.retrieve(agentId); |
There was a problem hiding this comment.
getting agents
| const createDefaultSession = useCallback( | ||
| async (agentId: string) => { | ||
| try { | ||
| const response = await client.agents.session.create(agentId, { |
There was a problem hiding this comment.
creating agent sesion
| ) | ||
| ) { | ||
| try { | ||
| await client.agents.delete(agentId); |
There was a problem hiding this comment.
deletes from DB
| await client.agents.delete(agentId); | ||
|
|
||
| // clear cached data for agent | ||
| SessionUtils.clearAgentCache(agentId); |
There was a problem hiding this comment.
deletes from cache.
| const fetchToolgroups = async () => { | ||
| try { | ||
| console.log("Fetching toolgroups..."); | ||
| const toolgroups = await client.toolgroups.list(); |
There was a problem hiding this comment.
i'm calling toolgroups as I believe this maps to the preferred field in the AgentConfig.
I think we are keeping it for now for exactly this use case. With Responses, you don't need "turns" etc. and each session can be more flexibly arranged but independent sessions still need to be kept somewhere and so it doesn't take care of that from the client's perspective. |
| @@ -0,0 +1,345 @@ | |||
| import React from "react"; | |||
| // ensures this only happens client side | ||
| const safeLocalStorage = { | ||
| getItem: (key: string): string | null => { | ||
| if (typeof window === "undefined") return null; |
There was a problem hiding this comment.
why doesn't this (and other methods) throw when window is undefined?
There was a problem hiding this comment.
Next.js does server side rendering so there's no window at when the page is compiled, the null ensures nothing breaks during page rendering, once the client renders the page the content is then hyrdated.
| return globalThis.crypto.randomUUID(); | ||
| }; | ||
|
|
||
| export function SessionManager({ |
There was a problem hiding this comment.
this is a react component -- the suffix Manager feels wrong to it? it must have some UI state? how will this be used in UI?
There was a problem hiding this comment.
The state is retrieved from the client.agents.list() and cached locally. I had originally called it SessionManager because my first implementation had sessions stored locally and this was the component managing the local storage of those sessions. I still use much of that pattern for faster refreshes. We can definitely rename it.
There was a problem hiding this comment.
i renamed this to conversations. it's not using the API for now but i'm happy to swap it to those APIs in the future and I think that should mostly be straightforward.
|
@franciscojavierarceo see this https://x.com/OpenAIDevs/status/1958660207745409120 -- specifically the Conversations API. { Conversations + Responses } is an alternative to the Agents API and is likely where the ecosystem is likely to shift to. So while we will now likely get rid of Agents API, I think we are still not going to make session state persist on the client. Clients need to be lightweight. |
Signed-off-by: Francisco Javier Arceo <[email protected]>
franciscojavierarceo
changed the title
# What does this PR do? - Introduces the Agent Session creation for the Playground and allows users to set tools - note tools are actually not usable yet and this is marked explicitly - this also caches sessions locally for faster loading on the UI and deletes them appropriately - allows users to easily create new sessions as well - Moved Model Configuration settings and "System Message" / Prompt to the left component - Added new logo and favicon - Added new typing animation when LLM is generating ### Create New Session <img width="1916" height="1393" alt="Screenshot 2025-08-21 at 4 18 08 PM" src="https://github.com/user-attachments/assets/52c70ae3-a33e-4338-8522-8184c692c320" /> ### List of Sessions <img width="1920" height="1391" alt="Screenshot 2025-08-21 at 4 18 56 PM" src="https://github.com/user-attachments/assets/ed78c3c6-08ec-486c-8bad-9b7382c11360" /> <!-- If resolving an issue, uncomment and update the line below --> <!-- Closes #[issue-number] --> ## Test Plan Unit tests added --------- Signed-off-by: Francisco Javier Arceo <[email protected]>
2 participants
What does this PR do?
Create New Session
List of Sessions
Test Plan
Unit tests added