Inside bulk grading: how the two-phase pipeline works
When a teacher drops a class set of scans into Zippy and presses grade, two things have to be true at once. The page has to respond right away, and thirty answers have to be marked against a rubric by a language model without running up an unbounded bill or timing out. Those goals pull in opposite directions. Here is how the grading pipeline holds both.
The problem with grading in one pass
The naive version is a single request: receive the upload, OCR every page, send every answer to the model, wait, return the results. It is simple, and it falls over in production. A class of thirty essays is thirty slow model calls behind one HTTP request. The teacher stares at a spinner for a minute or more, the request risks timing out, and if two teachers grade at once you have sixty concurrent model calls with nothing deciding which run first.
So grading is split into two phases with very different shapes.
Phase A: a fast pass that returns in seconds
The first phase is synchronous and does no model grading at all. It OCRs each uploaded scan, extracts one answer per question, and writes the submission and its answers to the database with the feedback column left empty. Then it returns.
Because nothing is graded yet, this is quick, and the front end can render a card for every student immediately, each showing a plain "Grading..." state. The teacher sees the whole class appear at once instead of waiting for the slowest essay. The work of actually marking has been deferred, not skipped.
Phase B: a bounded background fan-out
The second phase is spawned to run in the background after the response has already gone out. It grades each answer, but it does not do so with the throttle wide open. Every grading call takes a permit from a single global pool sized to a small number of concurrent evaluations. That pool is shared across every teacher on the instance, not per request, so ten teachers grading at once cannot stack up into hundreds of simultaneous model calls. The pool is the backpressure valve for the whole system.
As each answer comes back, its row is updated in place. The pipeline is not building one big result object to return at the end; it is filling in the database incrementally, one answer at a time.
Live progress, streamed to the browser
The piece that makes the deferred work feel immediate is streaming. As Phase B finishes each answer it emits an event on the workspace channel, and a terminal event when a whole submission is done. The front end subscribes to that channel and patches the matching card the moment its event arrives. The "Grading..." placeholder becomes a real score and comment without a refresh and without polling.
So the teacher gets the best of both shapes: the page is populated in seconds by Phase A, and it fills in live as Phase B works through the class under a fixed concurrency ceiling.
Why the deterministic answers never reach the model
One more detail keeps the bill down and the quality up. Not every question needs a language model. Multiple choice, multi-select, and fill-in-the-blank questions that carry an answer key are graded by exact comparison and never sent to the model at all. That is both a cost optimisation and a correctness fix: a model asked to grade a checkbox will happily write a paragraph of essay-style critique about it, which is not what a checkbox needs. The objective path is pure, has no network call, and is exhaustively unit tested.
What this buys teachers
None of this is visible in the product, and that is the point. The teacher presses grade, the class appears at once, and the marks fill in while they watch. The engineering underneath is three ideas working together: return fast by deferring the slow work, bound the slow work with a shared permit pool, and stream the results back as they land. It is the difference between a tool that feels like it is thinking and one that feels like it is stuck.
We will go deeper on the evaluation model itself, and on why feedback is written by a separate step from scoring, in a future post.
The classroom view of the same pipeline is in marking a class set, and the rubric it grades against is covered in AI grading is only as good as the rubric behind it.