🏦 UOB Credit Card AI Chatbot

(Chatbot Tư vấn Thẻ tín dụng ứng dụng Generative AI & RAG cho Ngân hàng UOB)

📌 Overview

The UOB Credit Card AI Chatbot is a Proof of Concept (PoC) project that leverages Generative AI and Retrieval-Augmented Generation (RAG) to automate credit card consulting services for UOB Bank.

Instead of relying on traditional rule-based scripts, this system is capable of "understanding" user intent, retrieving accurate information from the database, and generating natural, contextual, and highly specialized responses.

Dự án UOB Credit Card AI Chatbot là một Proof of Concept (PoC) ứng dụng Generative AI và Retrieval-Augmented Generation (RAG) để tự động hóa dịch vụ tư vấn thẻ tín dụng cho Ngân hàng UOB.

Thay vì dựa vào kịch bản rule-based truyền thống, hệ thống này có khả năng "hiểu" ý định người dùng, truy xuất thông tin chính xác từ cơ sở dữ liệu, và tạo ra các câu trả lời tự nhiên, đúng ngữ cảnh và mang tính chuyên môn cao.

🚀 Key Features

• Hybrid RAG Pipeline: Combines Vector Search (via Multilingual-E5) and Keyword Search (BM25), followed by Re-ranking using a Cross-Encoder (MS-MARCO) to ensure the highest retrieval accuracy.
• LLM-as-Judge Evaluation: Integrates an automated pipeline to score response quality on a 1-5 scale (supporting both Reference-based and Rubric-based evaluation), eliminating the need for manual review.
• Experience Library (Token Prior): Automatically collects high-quality responses ($\geq$ 4/5) to serve as supplementary context for future inference, simulating the Training-Free GRPO concept.
• Admin Dashboard: Provides a Streamlit interface for real-time monitoring of metrics (Intent Accuracy, LLM Judge Score) and Knowledge Base management.

🛠️ Tech Stack

• Language: Python 3.10
• Backend Framework: FastAPI, Uvicorn
• AI & NLP:
  • LLM: OpenAI API (or compatible Groq/Together)
  • Embeddings & Re-ranking: sentence-transformers, langchain-huggingface
  • Hybrid Search: rank_bm25, Cosine Similarity (scikit-learn, numpy)
• Database: Apache Cassandra (NoSQL)
• Frontend / Dashboard: Streamlit
• DevOps: Docker, Docker Compose

📁 Project Structure

UOB-GenAI-Chatbot/
├── app.py                   # Main FastAPI backend code, RAG workflow & API endpoints
├── dashboard.py             # Admin dashboard built with Streamlit
├── run_golden_test.py       # Automated evaluation script (Golden Test Set)
├── updater.py               # Module for updating/syncing data into Cassandra
├── scraper.py               # Module for scraping source data into Cassandra
├── Dockerfile               # Container build configuration
├── docker-compose.yml       # Multi-container deployment configuration
└── static/                  # Static Chat UI (HTML/CSS/JS)

⚙️ Installation & Setup

System Requirements:

• Docker & Docker Compose installed.
• Python 3.10+ (if running without Docker).

Step 1: Clone the repository

git clone <your-github-repo-url>
cd UOB

Step 2: Configure Environment Variables Create a .env file in the root directory and provide the necessary keys (e.g., OPENAI_API_KEY).

Step 3: Run with Docker

docker-compose up -d --build

The system will run 2 containers:

• cassandra-uob: Database running on port 9042.
• uob-chatbot: Backend API and UI running on port 8080.

Step 4: Access the Application

• Chat UI: Open a browser and navigate to http://localhost:8080
• Admin Dashboard: (Run locally) streamlit run dashboard.py

📊 Evaluation Metrics

Based on the Golden Test Set (20 questions):

• Intent Accuracy: Achieved 95.0%
• LLM-as-Judge ($\geq$ 4/5): Achieved 85.0%
• Fact Hit Rate: 65% exact match with standard facts.

---

Developed during Data Science & AI Internship at POPTECH.