Description

Implement AI capabilities into the 2ly application using a bring-your-own-license (BYOL) approach. This feature enables users to configure their own OpenAI or Anthropic API keys and use AI to intelligently suggest tools for tool sets based on natural language descriptions.

Context

Original request: "I'd like to implement AI into the project, starting with a BYOL in the setting to add API key from OpenAI or Anthropic. Then, the first feature would be in tool set, to have a pop up that allow me to request in natural language the goal of a tool set and AI to suggest the tools required for this tool set. for example i request in NLP 'I want to send email' and AI should suggest for my tool set 'draft email, send email' tools based on the tools available in my Tools section."

Acceptance Criteria

Phase 1: BYOL Settings

• Users can navigate to a new AI settings section in workspace settings
• Users can input and save API keys for both OpenAI and Anthropic providers
• API keys are encrypted before storing in Dgraph (workspace-scoped)
• Users can select which provider to use (OpenAI or Anthropic)
• Users can select specific models from a dropdown (gpt-4, gpt-4-turbo, claude-3-opus, claude-3-sonnet, etc.)
• API keys are validated before saving
• Clear error messages displayed if API key is invalid
• Both providers can be configured simultaneously and users can switch between them

Phase 2: AI Tool Set Builder

• "Build with AI" button/action added to tool sets interface
• Modal/dialog opens for natural language input
• Users can describe their goal in plain language (e.g., "I want to send email")
• System sends tool names and descriptions to configured AI provider
• AI returns specific tool names/IDs that match the user's intent
• Users can review suggested tools before accepting
• Users can modify/deselect suggested tools
• Selected tools are added to the tool set
• If no matching tools found, suggest external MCP servers from registry
• Display error messages if AI service is down or unavailable
• Handle cases where AI returns no suggestions gracefully

Technical Implementation Notes

Files to Create:

Backend:

• packages/backend/src/services/ai-service.ts

  • Abstract AI service interface supporting multiple providers
  • OpenAI client implementation
  • Anthropic client implementation
  • Model selection and validation

• packages/backend/src/services/ai-tool-suggester.ts

  • Prompt engineering for tool suggestions
  • Parse AI responses into tool IDs
  • Fallback to MCP registry search if no matches

• packages/backend/src/middleware/encryption.ts

  • Encrypt API keys before storage
  • Decrypt API keys for use

• packages/backend/src/repositories/workspace-settings-repository.ts

  • CRUD operations for AI settings
  • Fetch workspace AI configuration

Frontend:

• packages/frontend/src/components/settings/ai-settings.tsx

  • Settings panel UI for API keys
  • Provider selection dropdown
  • Model selection dropdown (dynamic based on provider)
  • API key validation feedback
  • Save/update settings

• packages/frontend/src/components/toolsets/ai-tool-suggester-modal.tsx

  • Modal with natural language input textarea
  • Submit button to trigger AI suggestion
  • Display suggested tools with checkboxes
  • Accept/Cancel actions
  • Loading states
  • Error states

• packages/frontend/src/components/toolsets/mcp-server-suggestions.tsx

  • Display external MCP server suggestions
  • Link to install/add MCP servers

• packages/frontend/src/hooks/useAISettings.ts

  • Query current AI settings
  • Mutation to save AI settings
  • Loading and error states

• packages/frontend/src/hooks/useAIToolSuggestions.ts

  • Mutation to request tool suggestions
  • Handle AI response
  • Error handling

• packages/frontend/src/stores/ai-store.ts

  • Client-side AI configuration state
  • Selected provider and model

• packages/frontend/src/graphql/ai/ (new directory)

  • saveAISettings.graphql
  • getAISettings.graphql
  • suggestToolsForGoal.graphql
  • searchMCPServers.graphql

Files to Modify:

• packages/common/schema/apollo.schema.graphql

  • Add AIProviderSettings type
  • Add AIProvider enum (OpenAI, Anthropic)
  • Add AIModel type
  • Add saveAISettings mutation
  • Add getAISettings query
  • Add suggestToolsForGoal query/mutation
  • Add ToolSuggestion type
  • Add MCPServerSuggestion type

• packages/common/schema/dgraph.schema.graphql

  • Add AISettings type with encrypted key fields
  • Link to Workspace

• packages/backend/src/di/container.ts

  • Register AIService
  • Register AIToolSuggester
  • Register encryption middleware

• packages/frontend/src/components/settings/ (existing settings)

  • Add navigation to AI settings tab/section

• packages/frontend/src/components/toolsets/ (existing toolset UI)

  • Add "Build with AI" button
  • Integrate AI suggester modal

Database Schema (Dgraph):

type AISettings {
  id: ID!
  workspace: Workspace!
  openaiApiKey: String @search(by: [exact]) # encrypted
  anthropicApiKey: String @search(by: [exact]) # encrypted
  selectedProvider: String @search(by: [exact]) # "OpenAI" | "Anthropic"
  selectedModel: String
  createdAt: DateTime
  updatedAt: DateTime
}

Approach:

1. AI Service Architecture:

   • Create abstract AIService interface
   • Implement OpenAIService and AnthropicService
   • Use factory pattern to select provider based on settings
   • Send only tool names and descriptions (optimize for tokens)

2. Prompt Engineering:

   const prompt = `You are helping select tools for a tool set. The user wants: "${userGoal}"
   
   Available tools:
   ${tools.map(t => `- ${t.name}: ${t.description}`).join('\n')}
   
   Return a JSON array of tool names that best match this goal.
   Format: ["tool-name-1", "tool-name-2"]
   
   If no tools match, return an empty array.`;

3. Security:

   • Encrypt API keys using AES-256 before storing in Dgraph
   • Store encryption key in environment variable (never in database)
   • Validate API keys on save by making test call to provider
   • Workspace-scoped: each workspace has its own AI settings

4. MCP Server Suggestions:

   • If AI returns empty array, query MCP registry
   • Use semantic search on user's goal to find relevant servers
   • Display as secondary suggestion: "No matching tools found. Consider installing..."

5. Error Handling:

   • API key invalid: Show inline error in settings
   • AI service down: Show error modal with retry option
   • No suggestions: Fall back to MCP server recommendations
   • Rate limiting: Display message and suggest retry time

Considerations:

• Token Optimization: Only send tool names + short descriptions to avoid hitting context limits
• Model Availability: Keep model lists updated as providers release new models
• Cost Awareness: Display estimated token usage/cost to users (optional enhancement)
• Privacy: API keys never leave the backend, never logged
• Caching: Consider caching AI responses for identical queries (optional enhancement)
• Batch Operations: If tool list is large (>100), implement pagination or chunking
• Provider Parity: Ensure both OpenAI and Anthropic provide similar quality results
• Graceful Degradation: Application works normally even if AI features fail

Testing Requirements

• Unit tests for AI service (mock OpenAI/Anthropic responses)
• Unit tests for encryption/decryption middleware
• Unit tests for tool suggestion prompt generation
• Integration tests for AI settings CRUD operations
• Integration tests for complete AI suggestion flow
• Frontend component tests for AI settings panel
• Frontend component tests for AI suggester modal
• E2E test: Configure AI settings and use tool suggester
• Manual testing checklist:
  • Test with valid OpenAI key
  • Test with valid Anthropic key
  • Test with invalid keys
  • Test switching between providers
  • Test various natural language queries
  • Test with no matching tools
  • Test error scenarios (network down, rate limit)

Documentation Updates

• Add AI features section to README
• Document how to configure AI providers
• List supported models for each provider
• Add example use cases for AI tool suggester
• Document encryption approach for API keys
• Add troubleshooting guide for common AI errors
• Update GraphQL schema documentation

Dependencies

New NPM Packages:

• openai - Official OpenAI Node.js SDK
• @anthropic-ai/sdk - Official Anthropic SDK
• crypto (built-in) - For API key encryption

Environment Variables:

• AI_ENCRYPTION_KEY - Secret key for encrypting API keys in database

Related Issues/PRs

None (initial AI feature)