Skip to content

LLM Chat Interface with MCP Enabled Tool Orchestration #1202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 18 commits into from
Oct 11, 2025
Merged

LLM Chat Interface with MCP Enabled Tool Orchestration #1202

merged 18 commits into from
Oct 11, 2025

Conversation

@kevalmahajan
Copy link
Member

@kevalmahajan kevalmahajan commented Oct 10, 2025 •
edited
Loading

✨ Feature / Enhancement PR

🚀 Summary (1-2 sentences)

Closes #1200

This PR introduces a complete LLM Chat interface for virtual MCP servers in Context Forge Gateway with MCP tool orchestration, enabling users to interact with multiple LLM providers (Azure OpenAI, Ollama, OpenAI) through a web-based chat interface.

Key Features

  1. Multi-Provider LLM Support: Connect to Azure OpenAI or Ollama with provider-specific configurations
  2. MCP Tool Integration: Seamless integration with Virtual MCP servers added in the gateway.
  3. Chat Context: Supports context of previous messages
  4. Real-time Streaming: Server-Sent Events (SSE) streaming for responsive token-by-token responses
  5. Tool Execution Visibility: Real-time events for tool invocations (start, end, error)
  6. Session Management: Per-user isolated sessions with JWT authentication
  7. Conversation History: Automatic context management with configurable trimming
  8. Error Resilience: Comprehensive error handling with graceful degradation
  9. ReAct Agent: LangGraph-powered agent for multi-step reasoning and tool orchestration

Pre-requisites to Test:

  1. Install new optionall dependencies required for LLM Chat.
    pip install ".[llmchat]"
  2. Add corresponding env variables for LLM configuration or can skip this step if you want to pass it throught the UI.
# Enable the LLM Chat functionality, false to disable
LLMCHAT_ENABLED=true

## EITHER Azure OpenAI OR Ollama can be configured.

## Azure OpenAI LLM Configurations
AZURE_OPENAI_API_KEY=<api_key>
AZURE_OPENAI_ENDPOINT=https://your-url.openai.azure.com
AZURE_OPENAI_API_VERSION=2024-02-15-preview
AZURE_OPENAI_DEPLOYMENT=gpt4o
AZURE_OPENAI_MODEL=gpt4o

## Ollama LLM Configurations
# OLLAMA_MODEL=qwen3:1.7b
  1. Delete browser cache history to avoid cache conflits in UI.
  2. Run with single worker for now. Multi worker session handling will be supported soon.

Testing:

  1. Once all the prequities as performed, start the UI, navigate to LLM Chat tab.
  2. You will see the list of all the virtual servers configured in your Context Forge Gateway. Select the virtual servers that have tools.
  3. LLM configuration panel will pop up, if you have configure the LLM configurations in .env not need to pass it through UI. Just select the provider that you want to use and click on connect.
  4. Chat window will appear. Test tool calling with input queries as per your configured tools.
Recording.2025-10-10.143050.mp4

🧪 Checks

  • make lint passes
  • make test passes
  • CHANGELOG updated (if user-facing)

@kevalmahajan kevalmahajan marked this pull request as draft October 10, 2025 08:46
@kevalmahajan @crivetimihai
MCP client implemented as new tab LLM Chat
e6c0ca3 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
Improvements
f323582 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
experience changes
d490691 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
Improved MCP client backend
ae0d885 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
Fix scrolling to the latest message
b02b023 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
Fixed jwt token for servers
f2bd64b 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
require auth token for team and private level servers
b7c3c45 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
proper error handling
b495f3c 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
Thinking functionality display
9263681 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
responsive tabs
a2c116e 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
code standard fixes
441d880 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
updated env example
b227175 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
added new optional dependencies
2227b57 
Signed-off-by: Keval Mahajan <[email protected]>
@kevalmahajan @crivetimihai
Fixed web linting
cb2f2de 
Signed-off-by: Keval Mahajan <[email protected]>
@crivetimihai crivetimihai force-pushed the llm_integration_mcp_client branch from a074cde to cb2f2de Compare October 11, 2025 02:08
@crivetimihai
Fix linting issues for PR #1202
8217b13 
- Fix bandit B105 false positive in llmchat_router.py by adding nosec comment for empty string check
- Fix CSS linting issues in admin.css:
  - Rename keyframes to kebab-case (messageSlideIn -> message-slide-in, fadeIn -> fade-in, thinkingStepIn -> thinking-step-in)
  - Replace deprecated word-break: break-word with overflow-wrap: break-word
  - Split single-line declaration blocks into multi-line format
  - Remove duplicate @Keyframes fade-in and .animate-fade-in selectors
  - Merge duplicate .thinking-step selector with animation property

All pytest tests pass (3392 passed, 45 skipped)
All quality checks pass (flake8, bandit, interrogate, pylint, verify)
All web linting passes (stylelint, htmlhint)
@crivetimihai
Fix doctest failures by adding import guards for optional LLM depende…
4090c73 
…ncies

Wrapped langchain imports in try/except block to handle environments where
the optional LLM chat dependencies are not installed. This allows doctest
to run successfully without requiring these dependencies.

- Added _LLMCHAT_AVAILABLE flag to track availability
- Set imports to None with type ignore when unavailable
- Prevents ModuleNotFoundError during doctest runs
@crivetimihai crivetimihai marked this pull request as ready for review October 11, 2025 03:59
@crivetimihai
Linting and test fixes
e15b84e 
Signed-off-by: Mihai Criveti <[email protected]>
@crivetimihai
Web lint
e10345e 
Signed-off-by: Mihai Criveti <[email protected]>
@crivetimihai crivetimihai merged commit 424e563 into main Oct 11, 2025
36 checks passed
@crivetimihai crivetimihai deleted the llm_integration_mcp_client branch October 11, 2025 08:13
This was referenced Oct 11, 2025
Merged
Closed
p4yl04d3r pushed a commit to p4yl04d3r/mcp-context-forge that referenced this pull request Nov 19, 2025
@kevalmahajan @crivetimihai @p4yl04d3r
LLM Chat Interface with MCP Enabled Tool Orchestration (IBM#1202)
c1567c1 
* MCP client implemented as new tab LLM Chat

Signed-off-by: Keval Mahajan <[email protected]>

* Improvements

Signed-off-by: Keval Mahajan <[email protected]>

* experience changes

Signed-off-by: Keval Mahajan <[email protected]>

* Improved MCP client backend

Signed-off-by: Keval Mahajan <[email protected]>

* Fix scrolling to the latest message

Signed-off-by: Keval Mahajan <[email protected]>

* Fixed jwt token for servers

Signed-off-by: Keval Mahajan <[email protected]>

* require auth token for team and private level servers

Signed-off-by: Keval Mahajan <[email protected]>

* proper error handling

Signed-off-by: Keval Mahajan <[email protected]>

* Thinking functionality display

Signed-off-by: Keval Mahajan <[email protected]>

* responsive tabs

Signed-off-by: Keval Mahajan <[email protected]>

* code standard fixes

Signed-off-by: Keval Mahajan <[email protected]>

* updated env example

Signed-off-by: Keval Mahajan <[email protected]>

* added new optional dependencies

Signed-off-by: Keval Mahajan <[email protected]>

* Fixed web linting

Signed-off-by: Keval Mahajan <[email protected]>

* Fix linting issues for PR IBM#1202

- Fix bandit B105 false positive in llmchat_router.py by adding nosec comment for empty string check
- Fix CSS linting issues in admin.css:
  - Rename keyframes to kebab-case (messageSlideIn -> message-slide-in, fadeIn -> fade-in, thinkingStepIn -> thinking-step-in)
  - Replace deprecated word-break: break-word with overflow-wrap: break-word
  - Split single-line declaration blocks into multi-line format
  - Remove duplicate @Keyframes fade-in and .animate-fade-in selectors
  - Merge duplicate .thinking-step selector with animation property

All pytest tests pass (3392 passed, 45 skipped)
All quality checks pass (flake8, bandit, interrogate, pylint, verify)
All web linting passes (stylelint, htmlhint)

* Fix doctest failures by adding import guards for optional LLM dependencies

Wrapped langchain imports in try/except block to handle environments where
the optional LLM chat dependencies are not installed. This allows doctest
to run successfully without requiring these dependencies.

- Added _LLMCHAT_AVAILABLE flag to track availability
- Set imports to None with type ignore when unavailable
- Prevents ModuleNotFoundError during doctest runs

* Linting and test fixes

Signed-off-by: Mihai Criveti <[email protected]>

* Web lint

Signed-off-by: Mihai Criveti <[email protected]>

---------

Signed-off-by: Keval Mahajan <[email protected]>
Signed-off-by: Mihai Criveti <[email protected]>
Co-authored-by: Mihai Criveti <[email protected]>
Signed-off-by: p4yl04d3r <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@crivetimihai crivetimihai Awaiting requested review from crivetimihai crivetimihai is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Feature Request]: In built MCP client - LLM Chat service for virtual servers with agentic capabilities and MCP Enabled Tool Orchestration

3 participants

@kevalmahajan @crivetimihai