Scholar_Mate AI
An end-to-end RAG system for document-grounded Q&A over typed, scanned, and handwritten PDF notes — with OCR fallback and prompting strict enough to say "I don't know" instead of hallucinating.
The problem
Students' notes are messy in practice: some pages are typed, some are phone-scanned, some are handwritten. A RAG assistant that only handles clean typed PDFs is useless for real coursework, and an assistant that answers confidently from outside the uploaded material is actively harmful for exam prep. Scholar_Mate AI had to handle every note format a student actually has, and refuse to answer when the material simply isn't there.
Approach
- Upload: the student uploads PDF notes through a vanilla HTML/CSS/JS frontend.
- Extract: a FastAPI backend pulls text page-by-page with pypdf.
- OCR fallback: for scanned or handwritten pages with no embedded text, PyMuPDF renders the page as an image, then PaddleOCR, OpenAI Vision, or an Auto mode (PaddleOCR first, falling back to OpenAI Vision) extracts the text.
- Chunk: LangChain's RecursiveCharacterTextSplitter splits extracted text into overlapping chunks.
- Embed: SentenceTransformers (all-MiniLM-L6-v2) turns each chunk into a vector.
- Store: ChromaDB stores the vectors with metadata — file name, page number, extraction method.
- Retrieve: the user's question is embedded and matched against stored chunks by similarity search.
- Generate: the top chunks plus the question go to Gemini or OpenAI under a strict grounding prompt that requires the answer to come only from the retrieved chunks.
- Reference: the frontend shows page numbers and the source chunk text so answers are verifiable, not just asserted.
Why the core output is deterministic, not LLM-guessed
The grounding prompt is deliberately strict: if the retrieved chunks don't contain the answer, the assistant responds "I could not find this information in the uploaded document" instead of guessing from the model's general knowledge. For exam prep, a wrong-but-confident answer is worse than no answer.
Engineering notes
- Smart background indexing — the first few pages index immediately so a student can start asking questions while the rest of a large PDF indexes in the background.
- OCR text caching, so previously processed pages are never re-OCR'd.
- PaddleOCR runs in a separate worker process to isolate native crashes from the main FastAPI server.
- Configurable OCR quality modes (Fast / Balanced / Accurate) and page-range processing for large PDFs.
- Runs locally via Uvicorn or through Docker Compose, with persistent volumes for uploads, ChromaDB, and OCR caches.
Tech stack
Results
- Handles typed, scanned, and handwritten PDF notes in a single pipeline.
- Grounded, hallucination-resistant answers with page-number citations.
- Auto-generates summaries, MCQs, and viva questions from the same indexed material.
- Deployed on Hugging Face Spaces via Docker.