diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..bb706cc4 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,38 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Architecture + +- Frontend: React application with TypeScript, Vite, and Tailwind CSS +- Backend: Python-based LangGraph agent leveraging AWS Bedrock foundation models for iterative research +- Agent Flow: + 1. Uses AWS Bedrock Claude to generate search queries from user input + 2. Performs web research via AWS search capabilities + 3. Uses AWS Bedrock Claude to reflect on results and identify knowledge gaps + 4. Uses AWS Bedrock Claude for iterative query refinement + 5. Uses AWS Bedrock Claude to synthesize answers with citations + +## Development Commands + +Initial Setup: +```bash +make setup # Install all dependencies for frontend and backend +``` + +Development: +```bash +make dev # Run both frontend and backend dev servers +npm run build # Build frontend for production (in frontend/) +npm run lint # Run frontend ESLint +ruff check . # Run backend linter (in backend/) +mypy . # Run backend type checker (in backend/) +``` + +## Environment Setup + +Required environment variables: +- AWS_ACCESS_KEY_ID: AWS access key for Bedrock services +- AWS_SECRET_ACCESS_KEY: AWS secret access key for Bedrock services +- AWS_REGION: AWS region where Bedrock models are deployed (e.g., us-west-2) +- LANGSMITH_API_KEY: LangSmith API key (for production) \ No newline at end of file diff --git a/Makefile b/Makefile index 2e5c9033..204b361b 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,10 @@ -.PHONY: help dev-frontend dev-backend dev +.PHONY: help setup dev-frontend dev-backend dev + +setup: + @echo "Installing frontend dependencies..." + @cd frontend && npm install + @echo "Installing backend dependencies..." + @cd backend && uv sync help: @echo "Available commands:" @@ -12,9 +18,9 @@ dev-frontend: dev-backend: @echo "Starting backend development server..." - @cd backend && langgraph dev + @cd backend && uv run langgraph dev # Run frontend and backend concurrently dev: @echo "Starting both frontend and backend development servers..." - @make dev-frontend & make dev-backend \ No newline at end of file + @make dev-frontend & make dev-backend diff --git a/README.md b/README.md index 275a8d8c..39852530 100644 --- a/README.md +++ b/README.md @@ -1,25 +1,25 @@ -# Gemini Fullstack LangGraph Quickstart +# AWS Bedrock Fullstack LangGraph Quickstart -This project demonstrates a fullstack application using a React frontend and a LangGraph-powered backend agent. The agent is designed to perform comprehensive research on a user's query by dynamically generating search terms, querying the web using Google Search, reflecting on the results to identify knowledge gaps, and iteratively refining its search until it can provide a well-supported answer with citations. This application serves as an example of building research-augmented conversational AI using LangGraph and Google's Gemini models. +This project demonstrates a fullstack application using a React frontend and a LangGraph-powered backend agent. The agent leverages AWS Bedrock foundation models to perform comprehensive research on user queries, dynamically generating search terms, performing web searches, and iteratively refining its research until providing well-supported answers with citations. This application serves as an example of building research-augmented conversational AI using LangGraph and AWS Bedrock. -![Gemini Fullstack LangGraph](./app.png) +![AWS Bedrock Fullstack LangGraph](./app.png) ## Features -- 💬 Fullstack application with a React frontend and LangGraph backend. -- 🧠 Powered by a LangGraph agent for advanced research and conversational AI. -- 🔍 Dynamic search query generation using Google Gemini models. -- 🌐 Integrated web research via Google Search API. -- 🤔 Reflective reasoning to identify knowledge gaps and refine searches. -- 📄 Generates answers with citations from gathered sources. -- 🔄 Hot-reloading for both frontend and backend development during development. +- 💬 Fullstack application with a React frontend and LangGraph backend +- 🧠 Powered by AWS Bedrock foundation models for advanced conversational AI +- 🔍 Dynamic search query generation using AWS Bedrock Claude +- 🌐 Integrated web research capabilities powered by AWS +- 🤔 Reflective reasoning using AWS Bedrock models to identify knowledge gaps +- 📄 Answer synthesis with citations using AWS Bedrock +- 🔄 Hot-reloading for both frontend and backend development ## Project Structure The project is divided into two main directories: -- `frontend/`: Contains the React application built with Vite. -- `backend/`: Contains the LangGraph/FastAPI application, including the research agent logic. +- `frontend/`: Contains the React application built with Vite +- `backend/`: Contains the LangGraph/FastAPI application leveraging AWS Bedrock ## Getting Started: Development and Local Testing @@ -29,25 +29,21 @@ Follow these steps to get the application running locally for development and te - Node.js and npm (or yarn/pnpm) - Python 3.8+ -- **`GEMINI_API_KEY`**: The backend agent requires a Google Gemini API key. - 1. Navigate to the `backend/` directory. - 2. Create a file named `.env` by copying the `backend/.env.example` file. - 3. Open the `.env` file and add your Gemini API key: `GEMINI_API_KEY="YOUR_ACTUAL_API_KEY"` +- uv (https://docs.astral.sh/uv/) +- **AWS Credentials:** + 1. Navigate to the `backend/` directory + 2. Create a file named `.env` by copying the `backend/.env.example` file + 3. Add your AWS credentials: + ``` + AWS_ACCESS_KEY_ID=your_access_key + AWS_SECRET_ACCESS_KEY=your_secret_key + AWS_REGION=your_region + ``` **2. Install Dependencies:** -**Backend:** - -```bash -cd backend -pip install . -``` - -**Frontend:** - ```bash -cd frontend -npm install +make setup ``` **3. Run Development Servers:** @@ -57,21 +53,21 @@ npm install ```bash make dev ``` -This will run the backend and frontend development servers. Open your browser and navigate to the frontend development server URL (e.g., `http://localhost:5173/app`). +This will run the backend and frontend development servers. Open your browser and navigate to the frontend development server URL (e.g., `http://localhost:5173/app`). _Alternatively, you can run the backend and frontend development servers separately. For the backend, open a terminal in the `backend/` directory and run `langgraph dev`. The backend API will be available at `http://127.0.0.1:2024`. It will also open a browser window to the LangGraph UI. For the frontend, open a terminal in the `frontend/` directory and run `npm run dev`. The frontend will be available at `http://localhost:5173`._ ## How the Backend Agent Works (High-Level) -The core of the backend is a LangGraph agent defined in `backend/src/agent/graph.py`. It follows these steps: +The core of the backend is a LangGraph agent defined in `backend/src/agent/graph.py`. It leverages AWS Bedrock foundation models at each step: ![Agent Flow](./agent.png) -1. **Generate Initial Queries:** Based on your input, it generates a set of initial search queries using a Gemini model. -2. **Web Research:** For each query, it uses the Gemini model with the Google Search API to find relevant web pages. -3. **Reflection & Knowledge Gap Analysis:** The agent analyzes the search results to determine if the information is sufficient or if there are knowledge gaps. It uses a Gemini model for this reflection process. -4. **Iterative Refinement:** If gaps are found or the information is insufficient, it generates follow-up queries and repeats the web research and reflection steps (up to a configured maximum number of loops). -5. **Finalize Answer:** Once the research is deemed sufficient, the agent synthesizes the gathered information into a coherent answer, including citations from the web sources, using a Gemini model. +1. **Generate Initial Queries:** Uses AWS Bedrock Claude to analyze the input and generate targeted search queries +2. **Web Research:** Performs web searches using AWS capabilities to find relevant information +3. **Reflection & Knowledge Gap Analysis:** Uses AWS Bedrock Claude to analyze search results and identify knowledge gaps +4. **Iterative Refinement:** Generates follow-up queries using AWS Bedrock Claude and repeats research if needed +5. **Answer Synthesis:** Uses AWS Bedrock Claude to create a comprehensive answer with citations ## Deployment @@ -85,24 +81,24 @@ _Note: If you are not running the docker-compose.yml example or exposing the bac Run the following command from the **project root directory**: ```bash - docker build -t gemini-fullstack-langgraph -f Dockerfile . + docker build -t aws-bedrock-fullstack-langgraph -f Dockerfile . ``` **2. Run the Production Server:** ```bash - GEMINI_API_KEY= LANGSMITH_API_KEY= docker-compose up + AWS_ACCESS_KEY_ID=your_access_key AWS_SECRET_ACCESS_KEY=your_secret_key AWS_REGION=your_region LANGSMITH_API_KEY=your_langsmith_api_key docker-compose up ``` Open your browser and navigate to `http://localhost:8123/app/` to see the application. The API will be available at `http://localhost:8123`. ## Technologies Used -- [React](https://reactjs.org/) (with [Vite](https://vitejs.dev/)) - For the frontend user interface. -- [Tailwind CSS](https://tailwindcss.com/) - For styling. -- [Shadcn UI](https://ui.shadcn.com/) - For components. -- [LangGraph](https://github.com/langchain-ai/langgraph) - For building the backend research agent. -- [Google Gemini](https://ai.google.dev/models/gemini) - LLM for query generation, reflection, and answer synthesis. +- [React](https://reactjs.org/) (with [Vite](https://vitejs.dev/)) - For the frontend user interface +- [Tailwind CSS](https://tailwindcss.com/) - For styling +- [Shadcn UI](https://ui.shadcn.com/) - For components +- [LangGraph](https://github.com/langchain-ai/langgraph) - For building the backend research agent +- [AWS Bedrock](https://aws.amazon.com/bedrock/) - Foundation models for query generation, reflection, and answer synthesis ## License -This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details. \ No newline at end of file +This project is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details. \ No newline at end of file diff --git a/backend/pyproject.toml b/backend/pyproject.toml index 09eb5988..57538fba 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -11,13 +11,14 @@ requires-python = ">=3.11,<4.0" dependencies = [ "langgraph>=0.2.6", "langchain>=0.3.19", - "langchain-google-genai", + "langchain-aws>=0.1.0", + "langchain-community>=0.0.24", "python-dotenv>=1.0.1", "langgraph-sdk>=0.1.57", "langgraph-cli", "langgraph-api", "fastapi", - "google-genai", + "boto3>=1.34.0", ] diff --git a/backend/src/agent/configuration.py b/backend/src/agent/configuration.py index 6256deed..289296dc 100644 --- a/backend/src/agent/configuration.py +++ b/backend/src/agent/configuration.py @@ -9,24 +9,18 @@ class Configuration(BaseModel): """The configuration for the agent.""" query_generator_model: str = Field( - default="gemini-2.0-flash", - metadata={ - "description": "The name of the language model to use for the agent's query generation." - }, + default="anthropic.claude-3-sonnet-20240229-v1:0", + metadata={"description": "AWS Bedrock model for query generation."}, ) reflection_model: str = Field( - default="gemini-2.5-flash-preview-04-17", - metadata={ - "description": "The name of the language model to use for the agent's reflection." - }, + default="us.anthropic.claude-3-5-sonnet-20240620-v1:0", + metadata={"description": "AWS Bedrock model for reflection."}, ) answer_model: str = Field( - default="gemini-2.5-pro-preview-05-06", - metadata={ - "description": "The name of the language model to use for the agent's answer." - }, + default="us.anthropic.claude-3-5-haiku-20241022-v1:0", + metadata={"description": "AWS Bedrock model for answer generation."}, ) number_of_initial_queries: int = Field( diff --git a/backend/src/agent/graph.py b/backend/src/agent/graph.py index dae64b77..661e2f15 100644 --- a/backend/src/agent/graph.py +++ b/backend/src/agent/graph.py @@ -1,4 +1,9 @@ import os +import json +import logging +import time +import random +from typing import List from agent.tools_and_schemas import SearchQueryList, Reflection from dotenv import load_dotenv @@ -7,7 +12,9 @@ from langgraph.graph import StateGraph from langgraph.graph import START, END from langchain_core.runnables import RunnableConfig -from google.genai import Client +from langchain_aws import ChatBedrockConverse +from langchain_community.tools import BraveSearch +import boto3 from agent.state import ( OverallState, @@ -23,29 +30,87 @@ reflection_instructions, answer_instructions, ) -from langchain_google_genai import ChatGoogleGenerativeAI from agent.utils import ( - get_citations, get_research_topic, - insert_citation_markers, - resolve_urls, + extract_text_from_content, ) +# Set up logging +logger = logging.getLogger(__name__) + load_dotenv() -if os.getenv("GEMINI_API_KEY") is None: - raise ValueError("GEMINI_API_KEY is not set") +if not all( + k in os.environ + for k in [ + "AWS_ACCESS_KEY_ID", + "AWS_SECRET_ACCESS_KEY", + "AWS_REGION", + "BRAVE_SEARCH_API_KEY", + ] +): + raise ValueError("Required environment variables are not properly set") + + +def get_bedrock_llm(model_id: str, temperature: float = 0.7) -> ChatBedrockConverse: + """Initialize a Bedrock model with the specified configuration.""" + return ChatBedrockConverse( + model=model_id, + region_name=os.getenv("AWS_REGION"), + temperature=temperature, + max_tokens=2048, + ) -# Used for Google Search API -genai_client = Client(api_key=os.getenv("GEMINI_API_KEY")) + +# Initialize BraveSearch without retry wrapper to avoid interface issues +search_tool = BraveSearch.from_api_key( + api_key=os.getenv("BRAVE_SEARCH_API_KEY"), search_kwargs={"count": 5} +) + + +def safe_search_with_retry(query: str, max_retries: int = 5) -> str: + """Perform search with manual retry logic to avoid tool interface issues.""" + for attempt in range(max_retries): + try: + return search_tool.run(query) + except Exception as e: + if attempt < max_retries - 1: + # Exponential backoff with jitter + delay = (2**attempt) + random.uniform(0, 1) + logger.warning( + f"Search attempt {attempt + 1} failed: {str(e)}. Retrying in {delay:.2f}s..." + ) + time.sleep(delay) + else: + raise e + + +def parse_structured_output(response_text, schema_type: str): + """Safely parse structured output with fallbacks.""" + try: + # First extract text content using our utility function + text_content = extract_text_from_content(response_text) + + # Clean up the response text + if "```json" in text_content: + text_content = text_content.split("```json")[1].split("```")[0].strip() + elif "```" in text_content: + text_content = text_content.split("```")[1].split("```")[0].strip() + + return json.loads(text_content) + except Exception as e: + logger.warning( + f"Failed to parse {schema_type} response: {str(e)} - Response type: {type(response_text)}" + ) + return None # Nodes def generate_query(state: OverallState, config: RunnableConfig) -> QueryGenerationState: - """LangGraph node that generates a search queries based on the User's question. + """LangGraph node that generates search queries based on the User's question. - Uses Gemini 2.0 Flash to create an optimized search query for web research based on - the User's question. + Uses AWS Bedrock Claude to create optimized search queries for web research based on + the User's question with structured output parsing. Args: state: Current graph state containing the User's question @@ -54,31 +119,64 @@ def generate_query(state: OverallState, config: RunnableConfig) -> QueryGenerati Returns: Dictionary with state update, including search_query key containing the generated query """ + logger.info("Starting query generation") + configurable = Configuration.from_runnable_config(config) # check for custom initial search query count if state.get("initial_search_query_count") is None: state["initial_search_query_count"] = configurable.number_of_initial_queries - # init Gemini 2.0 Flash - llm = ChatGoogleGenerativeAI( - model=configurable.query_generator_model, - temperature=1.0, - max_retries=2, - api_key=os.getenv("GEMINI_API_KEY"), - ) - structured_llm = llm.with_structured_output(SearchQueryList) - - # Format the prompt - current_date = get_current_date() - formatted_prompt = query_writer_instructions.format( - current_date=current_date, - research_topic=get_research_topic(state["messages"]), - number_queries=state["initial_search_query_count"], - ) - # Generate the search queries - result = structured_llm.invoke(formatted_prompt) - return {"query_list": result.query} + try: + # Initialize the model + llm = get_bedrock_llm( + model_id=configurable.query_generator_model, + temperature=1.0, + ) + + # Format the prompt + current_date = get_current_date() + research_topic = get_research_topic(state["messages"]) + + formatted_prompt = query_writer_instructions.format( + current_date=current_date, + research_topic=research_topic, + number_queries=state["initial_search_query_count"], + ) + + # Add structured output instructions to the prompt + structured_prompt = f""" +{formatted_prompt} + +Please respond with a JSON object containing: +- "query": array of search query strings +- "rationale": brief explanation of why these queries are relevant + +Example format: +{{"query": ["query1", "query2"], "rationale": "explanation"}} +""" + + logger.info(f"Using model: {configurable.query_generator_model}") + + response = llm.invoke(structured_prompt) + logger.info( + f"Raw response type: {type(response.content)}, content: {str(response.content)[:200]}..." + ) + parsed = parse_structured_output(response.content, "query generation") + + if parsed and "query" in parsed: + logger.info(f"Generated queries: {parsed['query']}") + return {"query_list": parsed["query"]} + else: + # Fallback + logger.warning("Using fallback query generation") + return {"query_list": [research_topic]} + + except Exception as e: + logger.error(f"Error in generate_query: {str(e)}") + # Fallback to a simple query + research_topic = get_research_topic(state["messages"]) + return {"query_list": [research_topic]} def continue_to_web_research(state: QueryGenerationState): @@ -93,9 +191,10 @@ def continue_to_web_research(state: QueryGenerationState): def web_research(state: WebSearchState, config: RunnableConfig) -> OverallState: - """LangGraph node that performs web research using the native Google Search API tool. + """LangGraph node that performs web research using Brave Search. - Executes a web search using the native Google Search API tool in combination with Gemini 2.0 Flash. + Executes a web search using Brave Search API with built-in retry logic + and processes results with Claude 3 Sonnet. Args: state: Current graph state containing the search query and research loop count @@ -104,44 +203,80 @@ def web_research(state: WebSearchState, config: RunnableConfig) -> OverallState: Returns: Dictionary with state update, including sources_gathered, research_loop_count, and web_research_results """ + logger.info(f"Starting web research for query: {state['search_query']}") + # Configure configurable = Configuration.from_runnable_config(config) - formatted_prompt = web_searcher_instructions.format( - current_date=get_current_date(), - research_topic=state["search_query"], - ) - - # Uses the google genai client as the langchain client doesn't return grounding metadata - response = genai_client.models.generate_content( - model=configurable.query_generator_model, - contents=formatted_prompt, - config={ - "tools": [{"google_search": {}}], - "temperature": 0, - }, - ) - # resolve the urls to short urls for saving tokens and time - resolved_urls = resolve_urls( - response.candidates[0].grounding_metadata.grounding_chunks, state["id"] - ) - # Gets the citations and adds them to the generated text - citations = get_citations(response, resolved_urls) - modified_text = insert_citation_markers(response.text, citations) - sources_gathered = [item for citation in citations for item in citation["segments"]] - return { - "sources_gathered": sources_gathered, - "search_query": [state["search_query"]], - "web_research_result": [modified_text], - } + # Perform web search with manual retry logic + try: + search_results = safe_search_with_retry(state["search_query"]) + search_data = json.loads(search_results) + logger.info(f"Found {len(search_data)} search results") + except Exception as e: + logger.error(f"Search failed for query '{state['search_query']}': {str(e)}") + # Return minimal results if search fails completely + return { + "sources_gathered": [], + "search_query": [state["search_query"]], + "web_research_result": [ + f"Search failed for query '{state['search_query']}': {str(e)}" + ], + } + + # Format search results for the model + formatted_results = [] + sources_gathered = [] + + for idx, result in enumerate(search_data): + source = { + "title": result["title"], + "url": result["link"], + "snippet": result["snippet"], + "short_url": f"[{idx + 1}]", + } + sources_gathered.append(source) + formatted_results.append( + f"{source['short_url']}: {source['title']}\n{source['snippet']}\nURL: {source['url']}\n" + ) + + try: + llm = get_bedrock_llm( + model_id=configurable.query_generator_model, + temperature=0.0, + ) + + formatted_prompt = web_searcher_instructions.format( + current_date=get_current_date(), + research_topic=state["search_query"], + search_results="\n\n".join(formatted_results), + ) + + response = llm.invoke(formatted_prompt) + + # Extract text content properly + content = extract_text_from_content(response.content) + + return { + "sources_gathered": sources_gathered, + "search_query": [state["search_query"]], + "web_research_result": [content], + } + + except Exception as e: + logger.error(f"Error processing web research results: {str(e)}") + return { + "sources_gathered": sources_gathered, + "search_query": [state["search_query"]], + "web_research_result": [f"Error processing results: {str(e)}"], + } def reflection(state: OverallState, config: RunnableConfig) -> ReflectionState: """LangGraph node that identifies knowledge gaps and generates potential follow-up queries. Analyzes the current summary to identify areas for further research and generates - potential follow-up queries. Uses structured output to extract - the follow-up query in JSON format. + potential follow-up queries using structured output parsing. Args: state: Current graph state containing the running summary and research topic @@ -150,34 +285,110 @@ def reflection(state: OverallState, config: RunnableConfig) -> ReflectionState: Returns: Dictionary with state update, including search_query key containing the generated follow-up query """ + logger.info("Starting reflection") + configurable = Configuration.from_runnable_config(config) # Increment the research loop count and get the reasoning model state["research_loop_count"] = state.get("research_loop_count", 0) + 1 - reasoning_model = state.get("reasoning_model") or configurable.reasoning_model - - # Format the prompt - current_date = get_current_date() - formatted_prompt = reflection_instructions.format( - current_date=current_date, - research_topic=get_research_topic(state["messages"]), - summaries="\n\n---\n\n".join(state["web_research_result"]), - ) - # init Reasoning Model - llm = ChatGoogleGenerativeAI( - model=reasoning_model, - temperature=1.0, - max_retries=2, - api_key=os.getenv("GEMINI_API_KEY"), - ) - result = llm.with_structured_output(Reflection).invoke(formatted_prompt) - - return { - "is_sufficient": result.is_sufficient, - "knowledge_gap": result.knowledge_gap, - "follow_up_queries": result.follow_up_queries, - "research_loop_count": state["research_loop_count"], - "number_of_ran_queries": len(state["search_query"]), - } + reasoning_model = state.get("reasoning_model") or configurable.reflection_model + + try: + # Initialize the model + llm = get_bedrock_llm( + model_id=reasoning_model, + temperature=1.0, + ) + + # Flatten web_research_result and ensure all items are strings + summaries = [] + for item in state["web_research_result"]: + if isinstance(item, list): + # Recursively flatten nested lists + for subitem in item: + if isinstance(subitem, str): + summaries.append(subitem) + elif isinstance(subitem, dict): + # Skip dictionaries (likely sources_gathered got mixed in) + logger.warning( + f"Skipping dict in web_research_result: {type(subitem)}" + ) + continue + else: + summaries.append(str(subitem)) + elif isinstance(item, str): + summaries.append(item) + elif isinstance(item, dict): + # Skip dictionaries (likely sources_gathered got mixed in) + logger.warning(f"Skipping dict in web_research_result: {type(item)}") + continue + else: + summaries.append(str(item)) + + logger.info(f"Processed {len(summaries)} summaries for reflection") + + # Format the prompt + current_date = get_current_date() + base_prompt = reflection_instructions.format( + current_date=current_date, + research_topic=get_research_topic(state["messages"]), + summaries="\n\n---\n\n".join(summaries), + ) + + # Add structured output instructions + structured_prompt = f""" +{base_prompt} + +Please respond with a JSON object containing: +- "is_sufficient": boolean (true/false) +- "knowledge_gap": string describing missing information +- "follow_up_queries": array of follow-up query strings + +Example format: +{{"is_sufficient": false, "knowledge_gap": "Missing recent data", "follow_up_queries": ["query1", "query2"]}} +""" + + response = llm.invoke(structured_prompt) + logger.info( + f"Raw reflection response type: {type(response.content)}, content: {str(response.content)[:200]}..." + ) + parsed = parse_structured_output(response.content, "reflection") + + if parsed and "is_sufficient" in parsed: + logger.info( + f"Reflection result - sufficient: {parsed['is_sufficient']}, follow-up queries: {parsed.get('follow_up_queries', [])}" + ) + return { + "is_sufficient": parsed["is_sufficient"], + "knowledge_gap": parsed.get("knowledge_gap", ""), + "follow_up_queries": parsed.get("follow_up_queries", []), + "research_loop_count": state["research_loop_count"], + "number_of_ran_queries": len(state["search_query"]), + } + else: + # Fallback + logger.warning("Using fallback reflection") + return { + "is_sufficient": state["research_loop_count"] >= 2, + "knowledge_gap": "Fallback: research completed", + "follow_up_queries": [], + "research_loop_count": state["research_loop_count"], + "number_of_ran_queries": len(state["search_query"]), + } + + except Exception as e: + logger.error(f"Error in reflection: {str(e)}") + # Fallback to simple values + research_topic = get_research_topic(state["messages"]) + return { + "is_sufficient": state["research_loop_count"] + >= 2, # Default to sufficient after 2 loops + "knowledge_gap": "Error in reflection process.", + "follow_up_queries": [research_topic] + if state["research_loop_count"] < 2 + else [], + "research_loop_count": state["research_loop_count"], + "number_of_ran_queries": len(state["search_query"]), + } def evaluate_research( @@ -202,6 +413,11 @@ def evaluate_research( if state.get("max_research_loops") is not None else configurable.max_research_loops ) + + logger.info( + f"Evaluating research: sufficient={state['is_sufficient']}, loop={state['research_loop_count']}, max={max_research_loops}" + ) + if state["is_sufficient"] or state["research_loop_count"] >= max_research_loops: return "finalize_answer" else: @@ -230,39 +446,84 @@ def finalize_answer(state: OverallState, config: RunnableConfig): Returns: Dictionary with state update, including running_summary key containing the formatted final summary with sources """ + logger.info("Finalizing answer") + configurable = Configuration.from_runnable_config(config) - reasoning_model = state.get("reasoning_model") or configurable.reasoning_model + reasoning_model = state.get("reasoning_model") or configurable.answer_model + + # Flatten web_research_result and ensure all items are strings + summaries = [] + for item in state["web_research_result"]: + if isinstance(item, list): + # Recursively flatten nested lists + for subitem in item: + if isinstance(subitem, str): + summaries.append(subitem) + elif isinstance(subitem, dict): + # Skip dictionaries (likely sources_gathered got mixed in) + logger.warning( + f"Skipping dict in web_research_result: {type(subitem)}" + ) + continue + else: + # Extract text content from structured format + extracted = extract_text_from_content(subitem) + summaries.append(extracted) + elif isinstance(item, str): + summaries.append(item) + elif isinstance(item, dict): + # Skip dictionaries (likely sources_gathered got mixed in) + logger.warning(f"Skipping dict in web_research_result: {type(item)}") + continue + else: + # Extract text content from structured format + extracted = extract_text_from_content(item) + summaries.append(extracted) + + logger.info(f"Processed {len(summaries)} summaries for final answer") + logger.info( + f"Sample summary content: {summaries[0][:100] if summaries else 'No summaries'}..." + ) # Format the prompt current_date = get_current_date() formatted_prompt = answer_instructions.format( current_date=current_date, research_topic=get_research_topic(state["messages"]), - summaries="\n---\n\n".join(state["web_research_result"]), + summaries="\n---\n\n".join(summaries), ) - # init Reasoning Model, default to Gemini 2.5 Flash - llm = ChatGoogleGenerativeAI( - model=reasoning_model, - temperature=0, - max_retries=2, - api_key=os.getenv("GEMINI_API_KEY"), - ) - result = llm.invoke(formatted_prompt) - - # Replace the short urls with the original urls and add all used urls to the sources_gathered - unique_sources = [] - for source in state["sources_gathered"]: - if source["short_url"] in result.content: - result.content = result.content.replace( - source["short_url"], source["value"] - ) - unique_sources.append(source) + try: + # init Claude 3 Haiku + llm = get_bedrock_llm( + model_id=reasoning_model, + temperature=0.0, + ) + result = llm.invoke(formatted_prompt) + + # Extract text content properly + content = extract_text_from_content(result.content) + + # Replace the short urls with the original urls and add all used urls to the sources_gathered + unique_sources = [] + for source in state["sources_gathered"]: + if source["short_url"] in content: + content = content.replace(source["short_url"], source["url"]) + unique_sources.append(source) + + logger.info(f"Generated final answer with {len(unique_sources)} sources") + + return { + "messages": [AIMessage(content=content)], + "sources_gathered": unique_sources, + } - return { - "messages": [AIMessage(content=result.content)], - "sources_gathered": unique_sources, - } + except Exception as e: + logger.error(f"Error in finalize_answer: {str(e)}") + return { + "messages": [AIMessage(content=f"Error generating final answer: {str(e)}")], + "sources_gathered": [], + } # Create our Agent Graph @@ -290,4 +551,4 @@ def finalize_answer(state: OverallState, config: RunnableConfig): # Finalize the answer builder.add_edge("finalize_answer", END) -graph = builder.compile(name="pro-search-agent") +graph = builder.compile(name="aws-bedrock-search-agent") diff --git a/backend/src/agent/prompts.py b/backend/src/agent/prompts.py index d8fd3b9a..dabc5243 100644 --- a/backend/src/agent/prompts.py +++ b/backend/src/agent/prompts.py @@ -16,78 +16,55 @@ def get_current_date(): - Don't generate multiple similar queries, 1 is enough. - Query should ensure that the most current information is gathered. The current date is {current_date}. -Format: -- Format your response as a JSON object with ALL three of these exact keys: - - "rationale": Brief explanation of why these queries are relevant - - "query": A list of search queries +Generate search queries for the following research topic and provide a brief rationale for why these queries are relevant. -Example: +Research Topic: {research_topic}""" -Topic: What revenue grew more last year apple stock or the number of people buying an iphone -```json -{{ - "rationale": "To answer this comparative growth question accurately, we need specific data points on Apple's stock performance and iPhone sales metrics. These queries target the precise financial information needed: company revenue trends, product-specific unit sales figures, and stock price movement over the same fiscal period for direct comparison.", - "query": ["Apple total revenue growth fiscal year 2024", "iPhone unit sales growth fiscal year 2024", "Apple stock price growth fiscal year 2024"], -}} -``` -Context: {research_topic}""" +web_searcher_instructions = """Analyze and synthesize information from the provided search results to create a comprehensive summary about "{research_topic}". +Instructions: +- The current date is {current_date}. +- Carefully review each search result and its source. +- Focus on extracting factual, verifiable information from credible sources. +- When citing information, use the provided citation markers in square brackets (e.g., [1], [2], etc.). +- Organize the information logically and create a coherent narrative. +- Only include information found in the search results, don't make up any information. -web_searcher_instructions = """Conduct targeted Google Searches to gather the most recent, credible information on "{research_topic}" and synthesize it into a verifiable text artifact. +Search Results: +{search_results} -Instructions: -- Query should ensure that the most current information is gathered. The current date is {current_date}. -- Conduct multiple, diverse searches to gather comprehensive information. -- Consolidate key findings while meticulously tracking the source(s) for each specific piece of information. -- The output should be a well-written summary or report based on your search findings. -- Only include the information found in the search results, don't make up any information. +Please provide a well-structured summary that incorporates the relevant information with proper citations.""" -Research Topic: -{research_topic} -""" reflection_instructions = """You are an expert research assistant analyzing summaries about "{research_topic}". -Instructions: -- Identify knowledge gaps or areas that need deeper exploration and generate a follow-up query. (1 or multiple). -- If provided summaries are sufficient to answer the user's question, don't generate a follow-up query. -- If there is a knowledge gap, generate a follow-up query that would help expand your understanding. -- Focus on technical details, implementation specifics, or emerging trends that weren't fully covered. - -Requirements: -- Ensure the follow-up query is self-contained and includes necessary context for web search. - -Output Format: -- Format your response as a JSON object with these exact keys: - - "is_sufficient": true or false - - "knowledge_gap": Describe what information is missing or needs clarification - - "follow_up_queries": Write a specific question to address this gap - -Example: -```json -{{ - "is_sufficient": true, // or false - "knowledge_gap": "The summary lacks information about performance metrics and benchmarks", // "" if is_sufficient is true - "follow_up_queries": ["What are typical performance benchmarks and metrics used to evaluate [specific technology]?"] // [] if is_sufficient is true -}} -``` - -Reflect carefully on the Summaries to identify knowledge gaps and produce a follow-up query. Then, produce your output following this JSON format: +Your task is to: +1. Determine if the provided summaries are sufficient to answer the user's question +2. If not, identify specific knowledge gaps or areas that need deeper exploration +3. Generate follow-up queries to address any gaps + +Guidelines: +- If summaries are sufficient, set is_sufficient to true and leave knowledge_gap empty and follow_up_queries as an empty list +- If there are gaps, generate 1-3 specific follow-up queries that would help expand understanding +- Focus on technical details, implementation specifics, or emerging trends that weren't fully covered +- Ensure follow-up queries are self-contained and include necessary context for web search + +Research Topic: {research_topic} +Current Date: {current_date} Summaries: -{summaries} -""" +{summaries}""" + answer_instructions = """Generate a high-quality answer to the user's question based on the provided summaries. Instructions: - The current date is {current_date}. -- You are the final step of a multi-step research process, don't mention that you are the final step. -- You have access to all the information gathered from the previous steps. -- You have access to the user's question. -- Generate a high-quality answer to the user's question based on the provided summaries and the user's question. -- you MUST include all the citations from the summaries in the answer correctly. +- Generate a well-structured answer that directly addresses the user's question. +- Include specific facts and details from your research, always with proper citation markers. +- Be clear and concise while ensuring accuracy and comprehensiveness. +- When citing sources, ensure you maintain the citation markers (e.g., [1], [2], etc.). User Context: - {research_topic} diff --git a/backend/src/agent/utils.py b/backend/src/agent/utils.py index d02c8d91..d99b3dd3 100644 --- a/backend/src/agent/utils.py +++ b/backend/src/agent/utils.py @@ -2,20 +2,64 @@ from langchain_core.messages import AnyMessage, AIMessage, HumanMessage +def extract_text_from_content(content: Any) -> str: + """ + Extract plain text from ChatBedrockConverse response content. + + ChatBedrockConverse returns content in the format: + [{"type": "text", "text": "actual content"}] or similar structures + + Args: + content: The content from a ChatBedrockConverse response + + Returns: + str: The extracted plain text + """ + if isinstance(content, str): + return content + + if isinstance(content, list): + # Handle list of content blocks + text_parts = [] + for item in content: + if isinstance(item, dict) and item.get("type") == "text": + text_parts.append(item.get("text", "")) + elif isinstance(item, str): + text_parts.append(item) + else: + # Fallback to string conversion + text_parts.append(str(item)) + return "".join(text_parts) + + if isinstance(content, dict): + # Handle single content block + if content.get("type") == "text": + return content.get("text", "") + # Try other common keys + for key in ["text", "content", "message"]: + if key in content: + return str(content[key]) + + # Fallback to string conversion + return str(content) + + def get_research_topic(messages: List[AnyMessage]) -> str: """ Get the research topic from the messages. """ # check if request has a history and combine the messages into a single string if len(messages) == 1: - research_topic = messages[-1].content + research_topic = extract_text_from_content(messages[-1].content) else: research_topic = "" for message in messages: if isinstance(message, HumanMessage): - research_topic += f"User: {message.content}\n" + content = extract_text_from_content(message.content) + research_topic += f"User: {content}\n" elif isinstance(message, AIMessage): - research_topic += f"Assistant: {message.content}\n" + content = extract_text_from_content(message.content) + research_topic += f"Assistant: {content}\n" return research_topic diff --git a/backend/test-agent.ipynb b/backend/test-agent.ipynb index d100b7f1..b0349b7c 100644 --- a/backend/test-agent.ipynb +++ b/backend/test-agent.ipynb @@ -8,7 +8,13 @@ "source": [ "from agent import graph\n", "\n", - "state = graph.invoke({\"messages\": [{\"role\": \"user\", \"content\": \"Who won the euro 2024\"}], \"max_research_loops\": 3, \"initial_search_query_count\": 3})" + "state = graph.invoke(\n", + " {\n", + " \"messages\": [{\"role\": \"user\", \"content\": \"Who won the euro 2024\"}],\n", + " \"max_research_loops\": 3,\n", + " \"initial_search_query_count\": 3,\n", + " }\n", + ")" ] }, { @@ -470,7 +476,12 @@ "metadata": {}, "outputs": [], "source": [ - "state = graph.invoke({\"messages\": state[\"messages\"] + [{\"role\": \"user\", \"content\": \"How has the most titles? List the top 5\"}]})" + "state = graph.invoke(\n", + " {\n", + " \"messages\": state[\"messages\"]\n", + " + [{\"role\": \"user\", \"content\": \"How has the most titles? List the top 5\"}]\n", + " }\n", + ")" ] }, { diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 6e68e50b..e375576b 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -4,6 +4,7 @@ import { useState, useEffect, useRef, useCallback } from "react"; import { ProcessedEvent } from "@/components/ActivityTimeline"; import { WelcomeScreen } from "@/components/WelcomeScreen"; import { ChatMessagesView } from "@/components/ChatMessagesView"; +import React from "react"; export default function App() { const [processedEventsTimeline, setProcessedEventsTimeline] = useState< @@ -12,6 +13,7 @@ export default function App() { const [historicalActivities, setHistoricalActivities] = useState< Record >({}); + const [finalAnswerMessages, setFinalAnswerMessages] = useState([]); const scrollAreaRef = useRef(null); const hasFinalizeEventOccurredRef = useRef(false); @@ -55,7 +57,7 @@ export default function App() { data: event.reflection.is_sufficient ? "Search successful, generating final answer." : `Need more information, searching for ${event.reflection.follow_up_queries.join( - ", " + ", ", )}`, }; } else if (event.finalize_answer) { @@ -74,16 +76,53 @@ export default function App() { }, }); + // Filter messages to only show human messages and final AI answers + const displayMessages = React.useMemo(() => { + if (!thread.messages) return []; + + const filtered: Message[] = []; + + // Group messages by conversation pairs (human -> ai responses) + for (let i = 0; i < thread.messages.length; i++) { + const message = thread.messages[i]; + + if (message.type === "human") { + filtered.push(message); + } else if (message.type === "ai") { + // For AI messages, only keep the last one for each conversation + // Find the last AI message index + let lastAiIndex = -1; + for (let j = filtered.length - 1; j >= 0; j--) { + if (filtered[j].type === "ai") { + lastAiIndex = j; + break; + } + } + + if (lastAiIndex >= 0 && !thread.isLoading) { + // Replace with the latest (most complete) AI message + filtered[lastAiIndex] = message; + } else if (lastAiIndex === -1) { + // First AI message in this conversation + filtered.push(message); + } + // If still loading, don't add intermediate AI messages + } + } + + return filtered; + }, [thread.messages, thread.isLoading]); + useEffect(() => { if (scrollAreaRef.current) { const scrollViewport = scrollAreaRef.current.querySelector( - "[data-radix-scroll-area-viewport]" + "[data-radix-scroll-area-viewport]", ); if (scrollViewport) { scrollViewport.scrollTop = scrollViewport.scrollHeight; } } - }, [thread.messages]); + }, [displayMessages]); useEffect(() => { if ( @@ -144,7 +183,7 @@ export default function App() { reasoning_model: model, }); }, - [thread] + [thread], ); const handleCancel = useCallback(() => { @@ -157,10 +196,10 @@ export default function App() {
- {thread.messages.length === 0 ? ( + {displayMessages.length === 0 ? ( ) : (
diff --git a/frontend/src/components/ChatMessagesView.tsx b/frontend/src/components/ChatMessagesView.tsx index 1792e6f7..eaf13fb3 100644 --- a/frontend/src/components/ChatMessagesView.tsx +++ b/frontend/src/components/ChatMessagesView.tsx @@ -74,7 +74,7 @@ const mdComponents = {
@@ -85,7 +85,7 @@ const mdComponents = { @@ -96,7 +96,7 @@ const mdComponents = { 
@@ -117,7 +117,7 @@ const mdComponents = {
     
@@ -150,9 +150,7 @@ const HumanMessageBubble: React.FC = ({
       className={`text-white rounded-3xl break-words min-h-7 bg-neutral-700 max-w-[100%] sm:max-w-[90%] px-4 pt-3 rounded-br-lg`}
     >
       
-        {typeof message.content === "string"
-          ? message.content
-          : JSON.stringify(message.content)}
+        {extractTextFromContent(message.content)}
       
     
   );
@@ -197,20 +195,13 @@ const AiMessageBubble: React.FC = ({
         
       )}
       
-        {typeof message.content === "string"
-          ? message.content
-          : JSON.stringify(message.content)}
+        {extractTextFromContent(message.content)}