Skip to content
All projects

Case study / 2026

EIGENN

Everything an exam candidate needs in one platform, with grading as the difficult part of it. The pipeline performs the reading; a reviewer signs the result.

Evaluation platformFastAPILangGraphTemporalONNXReact

01

One domain, addressed properly

A candidate preparing for UPSC, a state PSC, CA or CS is not short of tools. They are short of a single environment that understands the examination they are actually sitting. General-purpose products treat an answer sheet as a document; this platform treats it as an answer to a question with a defined marking scheme.

The platform is multi-tenant and two-sided. An institute enrols, brings its own students, and retains its own content, cohorts and branding. The student signs in through their institute and completes the entire cycle in one place: submit the handwritten sheet, have it read, receive an evaluation against the marking scheme — rather than scanning in one tool, querying a model in another, and receiving no mark anyone will stand behind.

  • Institutes administer their own students, content and cohorts; data is separated by tenant.
  • A handwritten answer sheet is scanned and read on entry to the pipeline, so the input is paper rather than typed text.
  • A single step for the student: submit, and the reviewed evaluation returns.

02

A reviewer signs every result

A platform that issues machine-generated marks directly to a student is a liability rather than a product. Institutes are judged on their grading, and one confidently incorrect score costs more trust than a hundred correct ones earn.

The pipeline therefore never issues a final mark. It produces a draft with its reasoning attached, and a reviewer approves, amends or rejects it; only the approved version reaches the student. That makes the model a rapid first reader rather than an authority, which is the role it performs well.

03

A pipeline with declared failure states

The evaluation flow comprises 17 nodes in LangGraph, several of which cycle so the pipeline reviews its own intermediate output before continuing. The cycles matter: a first pass over a handwritten answer is frequently wrong in a way a second pass identifies.

Every node checkpoints, so a failure resumes rather than restarts, and each failure state is typed rather than raised as a generic error. A system that can report only that something went wrong is a system that cannot be operated.

  • Typed failure states, each with a defined owner and recovery path.
  • Per-node checkpoint and resume, so a partial run is not discarded.
  • Long-running work orchestrated on Temporal rather than inside request handlers.
  • Langfuse traces every node, so a poor result can be examined after the fact.

04

Retrieval within a CPU budget

Retrieval runs over more than 120,000 chunks of reference material, embedded with BGE-M3 and reranked by a cross-encoder. Reranking is the step that separates broadly relevant passages from the correct one.

Running that on GPUs would have been the straightforward choice and a recurring cost. Quantising the models to ONNX INT8 brought the entire retrieval stack within approximately 300MB of RAM on ordinary CPU hosts. The accuracy cost was marginal; the infrastructure cost was eliminated.

05

Testing output that is permitted to vary

A model does not return an identical sentence twice, so conventional equality assertions establish nothing. What must remain fixed is the structure: the fields, the types, and the ranges a mark may occupy.

Contract tests verify exactly that, and they are what detect a prompt or model change quietly breaking downstream code — a failure that otherwise reaches a reviewer as a blank screen.

06

Payments require a higher standard

Payments are the one area where a retry must never become a second charge. Writes go to an append-only ledger and every operation is idempotent, so a duplicated webhook or a double submission settles to the same balance.

Most of the 150+ tests in this area are not happy-path tests. They cover the duplicate callback, the timeout mid-write, and the out-of-order delivery — the states that appear in production only if they have been written down first.

Next case study

AlgoPy