01
List CS courses by semester
Returns Fall 2026 CS courses as a structured list and shows the official sources used for the answer.
AI systems · Backend
BuzzBot
A Georgia Tech assistant that routes exact schedule queries and context-heavy policy questions through separate retrieval paths.
- Role
- Independent project · System architecture, backend, retrieval, and evaluation
- Technologies
- FastAPI · LangGraph · PostgreSQL · pgvector · Hybrid Retrieval
Actual product
See it in action
02
Look up instructors and sections
Groups CS 2200 instructors with their sections and connects the answer to the supporting schedule rows.
03
Verify an academic deadline
Answers a withdrawal-deadline question with the date and time zone, then cites the relevant Academic Calendar text.
- Schedule SQL questions passed
- 150/150
- Course retrieval questions passed
- 120/120
- Policy answers with evidence
- 92%
- Unsupported confident policy answers
- 0%
Hit@1 and Hit@5
01
Context
University information combines data that requires exact rows, such as class times and available seats, with documents that require context and citations, such as policies and calendars. Semantic search makes schedule answers unstable, while SQL alone cannot answer policy questions.
Selecting the right source was the first success criterion. A failed refresh could not damage the currently served data, and retrieval failures needed to remain distinguishable from answer-generation failures.
02
Engineering
FastAPI receives a question and LangGraph selects either the schedule or document path. Schedule questions query normalized public OSCAR data in PostgreSQL without an LLM. Document questions retrieve candidates with pgvector and PostgreSQL FTS, combine them with RRF, and rerank the final evidence.
The generation model receives selected passages and structured citation data. A post-answer check verifies that citations support the response. If evidence is weak, the system performs one bounded retry and then abstains. A failed ingestion never replaces the last known-good dataset.
My contribution
- Designed the architecture that separates schedule SQL from document RAG.
- Implemented ingestion, normalization, hybrid retrieval, citation assembly, and answer validation.
- Measured schedule lookup, document retrieval, answer correctness, citation support, and abstention separately.
Key technical decisions
Query schedules directly with SQL
Course times and seats require exact rows, so public OSCAR data is queried with SQL instead of vector similarity. Document retrieval quality cannot alter schedule correctness.
Allow one retrieval retry
The route, retrieve, and validate sequence is fixed so the model cannot call tools indefinitely. A failed evidence check permits one retry; the system abstains if evidence remains insufficient.
Do not publish partial ingestion results
A refresh replaces current data only after the complete job succeeds. Users continue seeing the last known-good version instead of a partially updated dataset.
03
Validation
Schedule SQL, question routing, and UI rendering passed 150/150, 150/150, and 140/140 cases. Course-information retrieval passed Hit@1 and Hit@5 across 120 questions, while Academic Calendar routing and Hit@5 passed 20/20.
Policy answers measured 71% correctness and 78% citation entailment. Evaluating retrieval and generation separately showed that document retrieval was the first bottleneck to address.
Failures and changes
- Policy Evidence Hit@5 reached 70%, below the 85% target. Oracle retrieval reached 92%, showing that retrieval was a larger bottleneck than answer generation.
- A hierarchical-retrieval approximation reduced Evidence Hit@5 to 62%, so it was not shipped.
Next steps
- Freshness is bounded by the last successful ingestion. Recording collection timestamps and source changes would allow only modified documents to be reindexed.
- Policy retrieval has not reached the 85% Evidence Hit@5 target. Chunking, metadata, and reranking should be compared independently on the fixed evaluation set before expanding the answerable scope.