CSE427: Edu Chatbot – FastAPI + Helm + Minikube + Prometheus + Grafana

This project deploys a FastAPI-based chatbot using Haystack, containerized with Docker, and deployed using Helm on Minikube with full observability (Prometheus + Grafana) and autoscaling.

---

Duy Huynh 520644 joshhn Mijung Jung 509822 mijung2024

---

🧰 Prerequisites

Install the following tools:

• Minikube
• kubectl
• Helm
• Docker
• hey (for load testing)

On macOS:

brew install minikube kubectl helm hey

---

🚀 Getting Started

1. Start Minikube

minikube start
minikube addons enable metrics-server

---

2. Build and Load Docker Image

docker build -t edu-chatbot:latest .
minikube image load edu-chatbot:latest

---

3. Deploy the Chatbot via Helm

helm upgrade --install edu-chatbot ./edu-chatbot

Check if the pods and service are running:

kubectl get pods
kubectl get svc

---

4. Get the Service URL

minikube service edu-chatbot-edu-chatbot --url

Example output:

http://127.0.0.1:52876

---

📈 Prometheus + Grafana Setup

1. Add Helm Repos

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

2. Create Monitoring Namespace

kubectl create namespace monitoring

3. Install Prometheus

helm install prometheus prometheus-community/prometheus -n monitoring

4. Create Dashboard ConfigMap

kubectl create configmap chatbot-dashboard \
  --from-file=edu-chatbot/grafana-dashboard.json \
  -n monitoring

5. Install Grafana

helm install grafana grafana/grafana \
  -n monitoring \
  --set adminPassword=admin \
  --set service.type=NodePort \
  --set dashboardsConfigMaps.default=chatbot-dashboard \
  --set datasources."datasources\.yaml".apiVersion=1 \
  --set datasources."datasources\.yaml".datasources[0].name=Prometheus \
  --set datasources."datasources\.yaml".datasources[0].type=prometheus \
  --set datasources."datasources\.yaml".datasources[0].access=proxy \
  --set datasources."datasources\.yaml".datasources[0].url=http://prometheus-server.monitoring.svc.cluster.local \
  --set datasources."datasources\.yaml".datasources[0].isDefault=true

6. Access Grafana

minikube service grafana -n monitoring

Login:

• Username: admin
• Password: admin

---

📊 Test the Application

1. Upload a document

curl -X POST "http://127.0.0.1:52876/upload/" \
  -H "accept: application/json" \
  -H "Content-Type: multipart/form-data" \
  -F "file=@sample_docs/cloud_computing_notes.txt"

2. Ask a question

curl "http://127.0.0.1:52876/query/?question=What+is+FastAPI"

---

🔥 Load Test with hey to Trigger Autoscaling

hey -z 60s -c 50 "http://127.0.0.1:52876/query/?question=What%20is%20SaaS%3F"

• -z 60s: Run for 60 seconds
• -c 50: 50 concurrent users

Watch HPA react

kubectl get hpa -w

You should see pods increase based on CPU load.

---

📬 Clean Up

helm uninstall edu-chatbot
helm uninstall grafana -n monitoring
helm uninstall prometheus -n monitoring
kubectl delete namespace monitoring

---

🧠 Extras

• View Prometheus targets: http://localhost:9090/targets (requires kubectl port-forward)
• Extend HPA to scale by memory or custom metrics via autoscaling/v2