How we built an AI quiz engine that actually understands Nigerian university exams.
Building AI that works for JAMB, WAEC, and university-specific past questions required us to rethink how we prompt, fine-tune, and validate outputs.
When we started building CourseVault's quiz engine, we made one assumption that turned out to be completely wrong: that a generic AI model could handle Nigerian university exam questions out of the box.
It couldn't. Not even close.
The problem with generic AI for Nigerian exams
Nigerian university exams have quirks. JAMB questions often test specific syllabus points that differ from what a US or UK curriculum would emphasise. WAEC marking schemes have patterns — they reward certain phrasings over others. And many Nigerian university lecturers re-use questions from past papers with slight modifications.
When we tested GPT-4 and Claude on a set of 200 real JAMB past questions, accuracy was around 71%. Good for general knowledge, terrible for a product that students are supposed to trust before an exam.
Our approach: context-heavy prompting + validation layers
The first thing we did was build a curriculum context layer. Instead of sending a raw PDF excerpt to the model, we prepend structured metadata: the subject, course level, Nigerian exam type (if applicable), and specific syllabus objectives for that topic.
This alone pushed accuracy on JAMB-style questions from 71% to 84%.
The second change was building a validation pipeline. Every AI-generated question goes through three checks:
- Factual consistency: Does the correct answer actually appear in the source PDF?
- Distractor quality: Are the wrong answers plausibly wrong, not obviously wrong?
- Question clarity: Is the question unambiguous enough that a student who knows the material will get it right?
What we learned about Nigerian academic PDFs
Most Nigerian university lecture notes are PDFs converted from scanned Word documents from 2009. They're noisy. Equations are images, not text. Tables don't parse. Page numbers are embedded in the middle of sentences.
We built a pre-processing pipeline that handles all of this: OCR for image-heavy PDFs, equation detection, table extraction, and noise removal. It adds 3-4 seconds to upload time but dramatically improves quiz quality.
Where we are now
Our quiz engine now scores 91% on a test set of 500 Nigerian university questions across 12 subjects. We're not perfect — Chemistry and Mathematics are still harder than History and English — but we're good enough that students are trusting us before real exams.
That's the benchmark that matters.
Start studying smarter today.
7-day free trial. No credit card. All features included.