Case study · Banking
Semantic API Discovery Platform
A semantic search system that lets developers find API documentation by describing what they need, built for a major Malaysian financial institution.
- FastAPI
- MongoDB
- OpenAI Embeddings
- ChromaDB
- HuggingFace
- GPT-4o
- Redis
- candidates per search
- 25
- kept after reranking
- 5
- response cache
- 10 min
- index sync
- Live
The problem
Developers navigating a large financial institution's API catalogue had to scroll through pages of documentation or rely on exact keyword matches. That created friction, slowed development and left good APIs underused simply because they were hard to discover.
The solution
A search that understands developer intent. Ask “I want to find Authentication APIs” and it returns a ranked recommendation, alternatives, and a plain-English justification of why each result relates to the query, with no exact keywords needed.
Architecture
Press play to follow a request.
Swipe sideways to explore the diagram →
Index once, then answer every query with retrieval, reranking and a grounded recommendation.
Example from the system
Ask in plain English.
Main recommendation: DELETE Notifications API 3
Other options: DELETE Notifications API 51, DELETE Notifications API 79
Justification: API 3 has the simplest, most direct path, so it's easy to integrate and use. APIs 51 and 79 offer similar functionality with slightly different examples; all three are reliable choices.
Response format as shown in the original architecture diagram. Placeholder API names.
Capabilities
What it does well.
- 01
Semantic intent search
Describe what you need in plain English, not exact keywords. The query is vectorised and similarity-searched across the full API catalogue.
- 02
Contextual recommendations
A main recommendation plus alternative options, each with a plain-English justification explaining exactly why it matches the query.
- 03
HuggingFace reranking
Vector search retrieves 25 candidates; a HuggingFace reranking model narrows them to the top 5 before they reach the LLM.
- 04
Live data sync
A MongoDB replica set with change streams keeps ChromaDB in sync as API documentation is updated, with no manual re-indexing.
- 05
Redis response caching
Identical queries are served from cache for 10 minutes, reducing LLM calls and improving latency for repeated searches.
- 06
GPT-4o grounded generation
Answers are grounded strictly in retrieved API data. The LLM formats a tailored answer with ranked suggestions and rationale, not hallucinated content.
Stack
Built with.
- API layer
- FastAPI · Python
- Database
- MongoDB · Replica set · Change streams
- Embedding
- OpenAI text-embedding-3-large
- Vector store
- ChromaDB · similarity_top_k = 25
- Reranking
- HuggingFace reranking model · top_k = 5
- LLM & cache
- GPT-4o · Redis · 10-min TTL