vuetify+langchain local RAG

Overview

This system implements a micro-microservice architecture, and uses vuetify+langchain+local large model+local vector database to build a demonstration of the RAG system.

It mainly includes the following functions:

• Use local LLM for language translation
• Use local LLM for professional knowledge questions and answers

From the perspective of system architecture, this system consists of three parts: front-end, API gateway, and back-end service:

Features

The front end is developed using vue3 and vuetify3. The script is mainly written using the Combined API. The main features include:

• User authentication based on OAuth2.0 and JWT standards
• Translation component
• Q&A chat component The front-end code also includes common functions such as local session management, request interception common modules, switch theme, and uses a lot of veutify3 controls.

API Gateway is implemented using FastAPI, and mainly implements the following functions:

• User authentication based on OAuth2.0 and JWT specifications
• Request proxy forwarding

Backend services mainly provide services based on the local LLM, including:

• Use the Ollama platform to deploy and use the Ollama3.1 open source LLM locally
• Use Chroma for local vector data storage and retrieval
• Use Ollama3.1 or nomic-embed-text for vectorization
• Use the local large language model Ollama3.1 to achieve enhanced generation
• Use langchian to integrate the above functions to provide services

File Structure

• image/ The image directory used in the document
• src/ The source code directory
  • backend/ The backend code
    • gateway/ The API gateway
      • common/ The common logic module
      • config/ The configuration file
      • util/ The tool class
      • api gateway.py The gateway startup file
      • start.bat The gateway startup batch file in the windows system, double-click to start the API gateway
    • services/ Application services
      • consulting/ Consulting service
      • legal_consultation/ Legal advice service
      • translation/ Translation service
      • response.py API corresponding entity class
    • start all.bat Windows batch file, run it to start the gateway and all services
  • practice/ Python files for practice, each file implements a function independently
  • webclient/ front-end vuetify code
    • src/ source code directory
    • ...

Installation

Backend

1. It is recommended to use Visual Studio Code as the development tool, and use venv to create a virtual environment in the backend folder. For detailed steps, see: Configure venv in Visual Studio Code

It is recommended to use python 3.12.1 or above

2. Install dependencies The gateway and each service are independent applications. After entering the backend directory, go to the gateway folder and each subfolder of services, and execute the following command to install its runtime environment:

pip install -r requirements.txt

3. Install Ollama and download the large language model to the local computer. Here you need to download llama3.1 and nomic-embed-text
   For detailed steps, please refer to: Use the locally deployed llama3.1 LLM in langchian

4. Start all backend services
   You can use backend/start all.bat to start the gateway and all services with one click. As shown below:
   

At this point, the backend service has been configured.

Front-end

1. It is still recommended to use Visual Studio Code to open the project.
   If you are not familiar with using Visual Studio Code and veutify, it is recommended to take a look at: Use the fastest way to create a vuetify3 project

It is recommended to use V18.20.5(LTS), and it is not recommended to use newer versions.

2. Install node module

npm install

3. Start

pnpm run dev

or

npm run dev

Additional Notes

If you are not familiar with technologies such as langchain and RAG, we recommend that you refer to the following introductory tutorials:
🔗Building a RAG system of langchain+local large language model+local vector database from scratch

If you don't know vuetify, you can take a look at the following introductory tutorial:
🔗Veutify3 Getting Started

---

🪐Good luck🪐