Implementation of choosing between OPENAI/Antropic models by setting MODEL_TO_USE

[korjavin]

This pull request introduces enhancements to model selection and logging, along with improvements in API behavior and frontend handling. Key updates include the addition of model aliasing, default model configuration, refined logging for better traceability, and adjustments to frontend model handling logic.

Backend Enhancements

Model Selection and Aliasing:

• Introduced MODEL_NAME_ALIASES to map short model names to their full names and ensure consistency in model resolution. Default model selection now uses config.MODEL_TO_USE if no model is specified in requests (backend/agent/api.py). [1] [2] [3]

Logging Improvements:

• Enhanced logging to provide detailed information about model usage, resolved names, and API calls. Added logs for tool calling strategies and model-specific configurations (backend/agent/api.py, backend/services/llm.py). [1] [2]

Tool Configuration:

• Dynamically configured tool calling strategies (e.g., XML or native tools) based on the model type in run_agent and thread_manager functions (backend/agent/run.py, backend/agentpress/thread_manager.py). [1] [2]

Frontend Enhancements

Model Handling in Chat Input:

• Updated the ChatInput component to handle models more flexibly. The selectedModel state now defaults to null, and model_name is only included in API calls if explicitly selected (frontend/src/components/thread/chat-input.tsx). [1] [2]… parameters

• Adjusted max_tokens logic based on model type, including new conditions for "gpt-4".
• Enhanced MessageTool to include detailed documentation and examples for user interactions.
• Improved ThreadManager to handle new tool choices and reasoning efforts.
• Updated LLM service to incorporate enable_thinking and reasoning_effort parameters.
• Modified chat input component to dynamically manage model selection and thinking options.

[vercel]

@korjavin is attempting to deploy a commit to the projects Team on Vercel.

A member of the Team first needs to authorize it.

[korjavin]

I tested it on MODEL_TO_USE="openai/gpt-4o"
results are worse, but at least it works

and checked backward compatibility with MODEL_TO_USE="anthropic/claude-3-7-sonnet-latest"

[korjavin]

To close this: #102

[markokraemer]

We intentionally had removed the direct use of the 'MODEL_TO_USE' as results are worse with all other models, often mostly unusable. @dat-lequoc is currently architecting it to work with other models as well – but we´ll just recommend 1 or 2

[markokraemer]

The native tool calling often works way more reliable with the other models but we dont have a frontend implementation for it atm & the frontend is very messy, but it can be done

[korjavin]

I understand that quality is different, but does it make any harm to let self-hosted owner to decide if he/she wants to try other model?

I wanted to use chatgpt myself, so I implemented this.