Case study · Startup

GPT-Powered Internal Chatbot with RAG

A retrieval-augmented chatbot for a Malaysian tech startup. Employees query internal PDF documentation through a context-aware, session-persistent chat backed by GPT-4.

  • LangChain
  • ChromaDB
  • GPT-4
  • FastAPI
  • MongoDB
  • OpenAI Embeddings
  • Python 3.10+
tokens per chunk
300
token overlap
100
REST endpoints
5
manual re-indexing
0

The problem

Staff spent significant time explaining what the company does, again and again, to new hires, potential clients and partners. There was no centralised, always-available source of truth for company FAQs and internal knowledge.

The solution

A session-persistent chatbot backed by the company's internal documentation. Employees and clients ask naturally, get answers grounded in the knowledge base, and follow up with full context preserved, removing repetitive explanations and onboarding friction.

Architecture

Press play to follow a request.

request pathindexing / build

Swipe sideways to explore the diagram →

RAG Internal Chatbot architecturePDF documents/data directoryOpenAI embeddings300-token chunks · 100 overlapChromaDBvector databaseGPT-4grounded generationMongoDBchat history · CRUDRetrieverhistory-awareChat sessionscreate · continuedelete · retrieveFastAPIhosted REST APIStaff & clientsask in plain language
OVERVIEW

Documents are embedded once; each prompt pulls the relevant chunks before the model answers.

Interface · REST API

Five endpoints, full session control.

FastAPI · RAGA v1

  • POST/chatCreate a new chat session and get the first response
  • POST/chat/{session_id}Continue an existing conversation
  • GET/session/{session_id}Retrieve the full message history for a session
  • GET/sessionsList all active session IDs and descriptions
  • DELETE/chat/{session_id}Delete a session and its chat history

Capabilities

What it does well.

  • 01

    Persistent session memory

    Every conversation is tied to a UUID session stored in MongoDB, so multi-turn context is preserved across API calls with full history retrieval.

  • 02

    History-aware retrieval

    Follow-up questions are reformulated by the LLM before they hit the vector store, so retrieved chunks are always contextually relevant.

  • 03

    Persistent vector embeddings

    ChromaDB persists the embedding index to disk. The knowledge base survives restarts, with no re-indexing unless documents change.

  • 04

    Full session CRUD API

    REST endpoints for creating sessions, continuing chats, fetching history, listing all sessions and deleting sessions cleanly.

  • 05

    Document-grounded answers

    The system prompt constrains the LLM strictly to indexed document content, eliminating answers from outside the knowledge base.

  • 06

    Zero-friction knowledge updates

    Add PDFs to the /data directory and restart. Chunking (300 tokens, 100 overlap), embedding and indexing run automatically on server startup.

Stack

Built with.

API layer
FastAPI · Python 3.10+ · Uvicorn
RAG framework
LangChain · LangChain-OpenAI · LangChain-Community
Vector database
ChromaDB 0.5 · OpenAI Embeddings
Language model
GPT-4 · OpenAI API · Temperature 0.8
Session storage
MongoDB 7+ · PyMongo · MongoDBChatMessageHistory
Document processing
Unstructured · DirectoryLoader · RecursiveTextSplitter
Next projectProduction Web Development & SEO