In a computer-based exam hall, one invigilator cannot watch every student, desk, hand movement, and screen at once. CCTV exists, but reviewing hours of crowded, low-quality footage after an incident is slow and unreliable.

This system explores whether AI can help organise that footage into a small set of reviewable moments; such as an unexplained object near a candidate’s hand or an unusual repeated interaction without claiming that the system can determine cheating.

The idea sounded small until we placed it inside a real exam hall.

We(me and my teammate Mr. Kawaljeet Singh) started with a human problem, not a model problem. Imagine a computer-based test with roughly two hundred candidates, a ceiling camera, a room full of monitors and chairs, and a proctor who cannot continuously watch every hand, desk and side glance. The obvious question was: can a system make that room more reviewable?

The wrong answer is to say “yes, it can detect cheating.” The moment we took the question seriously, that sentence collapsed. A camera does not see intent. It sees compressed pixels, dropped frames, occluded shoulders, screen glow, wrists crossing desk edges and objects that can be twenty or thirty pixels wide. Everything meaningful is inference layered on top of an incomplete view.

So the project became a narrower, harder and more honest question: how can an investigation-support system surface a small number of explainable moments for a trained human to review - without turning ordinary behaviour into an automated allegation?

See it live here!

GitHub

The adversarial questions came before the architecture.

We deliberately tried to break our own premise with ordinary, uncomfortable scenarios.

Two students may lean in and whisper. They may make hand signs. A teacher or invigilator may approach a student to help. Someone may carry wired earphones. A calculator can resemble a phone; a phone can resemble a dark monitor edge. Someone may bring an object for which no detector class exists at all. And an invigilator throwing a chit would be a serious safety incident, but it is not the same task as examination review. Treating all of these as one “suspicious behaviour” problem is exactly how a system becomes unsafe.

The source footage had already set the ceiling for the project.

Before selecting a model, we audited the six available CCTV recordings. They were real computer-based-test footage, not curated object-detection clips. Most were capped at 720p; one was 640×480. Several were variable-frame-rate (VFR), and the bit-rate was roughly 0.08 bits per pixel. The smallest objects that mattered were not “small” in the abstract. A near phone could occupy about 30×20 pixels. Facial detail was even less forgiving: recorded median inter-ocular distances of 9.3, 10.1 and 4.7 pixels in sampled views made gaze, lip movement and subtle facial cues scientifically indefensible.

That changed our language. We stopped saying “detect karke dekh lete hai.” We began recording when a question could not be answered from the source.

This is a critical distinction: an absence of measurement is not an observation of normality.

Pre-processing was not makeup. It was measurement discipline.

The temptation with weak footage is to enhance it until it looks convincing. We tested four branches on identical frames: identity/no preprocessing, CLAHE contrast enhancement, NLMeans denoising and unsharp masking. CLAHE can lift local contrast. NLMeans can reduce noise. Unsharp masking can make borders pop. None of those operations creates information that the camera did not capture; all can alter the very artefacts a detector will read.

The identity branch therefore stayed locked for measurement and model comparison. Enhancement could be used in a reviewer-facing presentation view only when labelled. In one measured run, CLAHE increased the sharpness score by 439.35 but also increased the blocking metric by 0.0457; unsharp masking raised sharpness by 2,011.59 and blocking by 0.0330. Those are not free gains. They are a warning that a “clearer” 30×20 object can be a more persuasive compression artefact.

Why we rejected motion-triggered sampling

Our first instinct was to sample only where motion occurred. It looked efficient. It was also wrong for this problem: the early motion-triggered pass inspected only 20–39% of the available time, and a stationary person with a stationary prohibited object would simply never be observed. We replaced it with uniform sampling : typically 2Hz, with some experiments at 3Hz and treated motion as context and prioritisation, never as the sole admission ticket to the pipeline.

Motion itself needed a baseline. We separated low-area, isolated, non-persistent changes often compression noise - from environmental movement that persisted through at least half a window. We did not delete noisy regions. We recorded and deprioritised them. That “no silent deletion” rule recurs throughout the system.

The architecture backing the entire though process.

This is one thing I can claim - I would bet that technically, in terms of the architecture and thought and efforts that behind it combined with the result we were achivieng across multiple footages, we would have one a blind competition on this system.

1. The pipeline starts with CCTV but never hands a raw camera directly to an allegation model

2. A local GPU agent reads the footage with PyAV and OpenCV, preserves timing and health information, samples it, detects people and candidate objects, estimates body geometry, and builds an evidence packet

3. Cloudflare carries the review surface: the Worker API, D1 for hierarchy and decisions, R2 for media and bundles, and the React console

Detailed - GitHub

Why AlphaPose became the geometry baseline.

We did not choose AlphaPose because it has a memorable name. I chose it partly because I came across it too randomly with too god of a result and I knew for sure no one would implement it - and no one did!

Nah, but on a serious note we chose it after running the same manifest through three pose paths. The task was not face recognition or mind reading; it was geometry: a person box, wrist locations, shoulders and enough stable facial landmarks to compute only coarse head orientation when the source supported it. AlphaPose was used as a top-down model: it starts from a known person crop, then estimates the keypoints inside it. That design is useful in a hall where one missed person crop ruins all later wrist association.

Why D-FINE was just a proposal engine.

For people and object context, we used D-FINE through ONNX Runtime with CUDA. Its stock COCO model knows familiar visual categories such as person, keyboard, mouse, monitor, bottle and cell phone. That is useful context in a computer lab; it is not a policy classifier. The project originally let the COCO “cell phone” label escalate too easily. That path generated 40 flags, 27 of them high priority. It was removed from the escalation path.

For small objects, we also measured SAHI: sliced inference over candidate crops (320×320 slices, 25% overlap, then non-maximum suppression). Slicing improved phone proposals in the experiment, but also created more hard negatives. In the frozen comparison, D-FINE+SAHI produced 159 phone proposals on one condition versus D-FINE native’s 108; the gain cannot be read as better truth because screen edges, monitors and desk details are precisely the false positives that also multiply.

This is one of the less glamorous lessons from the project. A calculator that looks like a phone is not an edge case to solve with a bigger confidence threshold. It is a semantic and policy problem. The visual system can propose “small rectangle near wrist.” It cannot decide whether an otherwise-permitted calculator is a forbidden device without a defined policy, calibrated context and data that actually represents the distinction.

Where SAM 3 entered the picture?

SAM 3 is a segmentation model: given a text prompt, it returns masks for the visual regions it believes match. That makes it useful as a second look at an already-localised question, for example: in this particular crop, can it find “phone” or “paper,” and does it instead find keyboard, mouse, monitor or bottle? It is not a free oracle for every CCTV frame.

We called SAM 3 only after a proposal and geometric gate. Running it across every frame would cost roughly 1.4–3.3 seconds per call and would make the project depend on an online service for a task that originally needed to work locally. A concurrency experiment at 16 was not a breakthrough; it lost 2,937 of 3,439 calls to rate limits. We held the call volume around six concurrent requests and capped an episode at three calls. The bottleneck became an architectural fact, not a benchmark footnote.

The most useful measurement was not a hero result. In a frame where the chit detector assigned 0.74 confidence to a keyboard-like region, SAM 3 returned six keyboards (0.52–0.89) and no prompted target.

That was the exact behaviour we needed from the referee: recognise ordinary workstation equipment as a competing explanation. In a paper study, 392 gated candidates became 133 corroborations across 136 outputs, with one real paper handler represented by a track and a split track; 48 were explicitly suppressed as equipment context. In talking-like episodes, 402 of 1,432 were equipment context, seven were corroborated and 1,052 were unsupported.
“Unsupported” did not mean the person was cleared - it meant the source/model did not establish the proposition

Instead of sending every CCTV frame to SAM 3, we used a staged evidence pipeline to ask it only the questions worth asking. Video is first sampled uniformly, checked for source quality and camera motion, then processed for people, body keypoints, candidate objects and wrist association. Nearby frames are grouped into one episode. Only proposals that are visible enough, linked to a person, persistent or recurring, and not already explained by ordinary equipment reach SAM 3. We cap each episode at three verification calls, so SAM 3 acts as a referee for ambiguous evidence

Uniform sampling replaced motion-only sampling, so stationary objects were not missed.

• PTS-aware decoding kept timing correct for variable-frame-rate CCTV.

• Source-health, motion and alignment checks filtered camera shake and compression noise.

• D-FINE proposed people and object context; AlphaPose supplied wrist/body geometry.

• Geometric gating rejected objects unrelated to the relevant person.

• Native and SAHI object passes were limited to candidate crops, not full-frame exhaustive scans.

• Nearby detections were merged into temporal episodes instead of treated as separate events.

• SAM 3 ran only on gated episodes, with a maximum of three calls per episode.

• Prompts included target objects and ordinary confusers—keyboard, mouse, monitor and bottle—to suppress workstation false positives.

• “Unsupported,” unavailable, or unstable SAM output became needs_better_view, never a silent negative.

Decision Flow in short -

Two models looking at the same pixels are not two independent witnesses.

The easiest fusion rule would have been “if two models agree, flag it.” We rejected that. D-FINE and SAM 3 can agree because they are reading the same crop, the same compression and the same misleading monitor edge. Agreement between them is corroboration within one visual modality, not independent proof.

The evidence record carries the mundane detail that makes this defensible later: detector configuration, frame presentation time, crop and source references, track and wrist relationship, quality information, verifier result, reason-code vocabulary and reviewer revisions. The fusion code rejects machine attempts to write human decision states. This is the opposite of a black box: it is a refusal to let the final state hide where it came from.

Why we rejected using Gemma locally as a VLM ?

(some too technical stuff, i voiced gpt into writing too)

We did not drop Gemma because it is a weak model. On general multimodal benchmarks, it is genuinely capable: Google reports Gemma 4 E4B at 52.6% on MMMU Pro and the 31B model at 76.9%. But those are benchmarks for clear, information-rich images - not variable-frame-rate, low-bitrate ceiling CCTV where a possible phone can occupy 20–30 pixels. In our footage, Gemma could produce a convincing description while describing the wrong candidate; on one contact sheet, it narrated a person at a desk even though the highlighted event belonged to an empty row. When we gave it an annotated crop, it described our orange bounding box back to us as an object. The model was fluent. The evidence was still wrong.

The CUDA route did not solve that fundamental problem. Even a quantised local E4B VLM still required model weights, a vision projector, GPU memory, a KV cache, a local server, health checks and image transfer - on the same RTX 3050 Laptop GPU with only 4GB VRAM that was already running AlphaPose, D-FINE and SAHI. After the pipeline’s safety reserve, several model stages reported only about 2.3GB available. The object passes alone already took 635–2,531 seconds per recording. Adding a VLM did not create more source detail; it created another expensive process competing for the same memory and turning one evidence packet into a slow text-generation job.

The local serving path was also operationally brittle. Gemma vision in the llama.cpp route required both the GGUF model and a separate multimodal projector; its own documentation called the Gemma 3 vision route experimental/demo-oriented. In our run, a 9KB llama-server launcher stub could exist on disk yet fail if started from the wrong directory because its required DLLs were not found. We had already spent five minutes on CV work before discovering that last-stage failure. A health probe fixed the engineering issue, but it did not make the VLM useful enough to justify the cost.

MLX was not the escape hatch either. The rest of Project Classroom was built and measured on NVIDIA CUDA: ONNX Runtime CUDA for D-FINE, PyTorch CUDA for AlphaPose, and one constrained 4GB GPU budget. An Apple-silicon/Metal MLX route would have created a separate runtime, separate memory pool and media-transfer boundary for a model that had already failed the evidence-grounding test. MLX now offers a CUDA backend too, but changing frameworks would not repair the key failure: no runtime optimisation can recover semantic detail that is absent from a 20-pixel CCTV object.

So we kept Gemma out of the final decision path. It could describe an already-selected crop, but it could not create candidates, close evidence, alter rank, identify people, infer intent or write a human decision. SAM 3 remained a bounded referee for object masks; the rest stayed with deterministic geometry, temporal evidence and human review.

• Gemma 4 E4B: 52.6% MMMU Pro; Gemma 4 31B: 76.9% MMMU Pro—strong general vision benchmarks, but not CCTV evidence benchmarks. Google model card

• The official Gemma image-understanding example alone downloads a 10.2GB model file, illustrating why local multimodal deployment is not lightweight on a 4GB GPU. Google vision guide

• The llama.cpp Gemma vision route needs both model weights and an mmproj vision projector, and is described as experimental/demo-oriented. llama.cpp Gemma vision documentation

• MLX is primarily built around Apple-silicon workflows; its modern CUDA backend is Linux/NVIDIA-specific and still would not resolve the evidence-quality issue. MLX installation documentation

The 1,00,000-student question changed the architecture

The ’s real question was not, “Can you add more workers?” It was: before an examination begins, how do you know what kind of load each centre will create; how do you know which centres need more cameras, more storage, more review capacity and more compute; and how do you do that without silently turning a student’s academic or disciplinary history into an AI suspicion score?

My answer was to audit the centre and examination operation first, then build capacity around that audit. A centre with 2,000 registered candidates, 1,750 actual attendees, poor uplink, old 640×480 cameras, repeated power failures and an unclear camera layout has a very different technical profile from a centre with 300 candidates, high-resolution cameras and reliable local storage. Those facts affect how much video exists, how usable it is, how many GPU-hours are required and how many humans must be available to review it. They must not decide that an individual candidate is more likely to cheat.

The audit would therefore build a centre readiness profile, not a “suspicion profile.”

That distinction is non-negotiable. “How many students cheated here previously?” can be useful as an aggregate centre-quality signal, adjusted for centre size and only when incidents were actually confirmed after review. It can justify more reliable cameras, more audit staff or a stricter equipment check at the entrance. It must not cause the next batch of students at that centre to be algorithmically prioritised as likely offenders. The same applies to pass rates: a sudden centre-level anomaly may deserve a human audit of invigilation, network conditions or examination operations; it does not tell us which student to inspect.

What 10 lakh actually means in compute terms

At this scale, the unit is neither “one student” nor “one video.” It is:

centre → examination window → camera → encrypted time shard → evidence episode

Assume, conservatively, one camera covers 30 candidates. One million candidates means about 33,000 camera views. At only 2 sampled frames per second, that is about 66,000 sampled frames every second. Sending all of those frames to a central GPU fleet for person detection, pose estimation, object detection, slicing and VLM verification would be absurdly expensive—and technically irresponsible. It would also create a single point of failure exactly when every centre begins simultaneously.

So the design becomes a multi-layer system:

1. Centre layer — capture and audit

   Each centre has a local gateway connected to its camera network. It records encrypted video locally, validates timestamps, detects camera failure or missing feed, and breaks footage into fixed 5–10 minute shards. It uploads source material when connectivity permits; it does not require every centre to have perfect live bandwidth.

2. Regional ingestion layer — store before processing

   Raw video is stored by centre, exam, camera and time shard. This is where Cloudflare R2 is useful: it is object storage for large recordings and compact evidence artifacts, not a GPU substitute. The system keeps the original source reference so a reviewer can trace every event back to the correct camera and timestamp.

3. Workload-estimation layer — schedule by cost, not by panic

   Every shard receives a workload estimate before it reaches a GPU: duration, resolution, number of visible people, camera quality, expected crop count, selected model version and required SLA. A 640×480 sparse room and a 1080p crowded hall should not be treated as the same job.

   The scheduler uses this estimate to decide which regional GPU pool receives the work, how many jobs that pool can safely run, and whether the examination’s review deadline can still be met. That is actual scaling: modelling the work before allocating compute.

4. GPU layer — distribute video analysis, not accusations

   GPU workers process independent shards in parallel. They do not receive a vague instruction such as “analyse 10,000 students.” They receive a bounded, idempotent job: Centre C, Exam E, Camera 14, 10:00–10:10, pipeline version X. If one worker fails, only that shard is retried; it does not restart a three-hour examination recording or block other centres.

5. Evidence layer — expensive models see only narrowed episodes

   The first layers perform source-health checks, uniform sampling, tracking, pose and object-context work. Only short, geometrically associated and temporally persistent episodes enter the expensive verification path. This is why SAM 3 was bounded behind multiple gates: scale is not achieved by calling a stronger model more times; it is achieved by ensuring the stronger model never sees the 99% of frames that do not justify the cost.

6. Human-review layer — review capacity is planned per centre and exam window

   The final queue is not a queue of “suspicious students.” It is a queue of traceable evidence packets, ordered by evidence completeness, deadline and reviewer capacity. Every packet still retains context_observation, needs_better_view and review_candidate as different outcomes.

Where queues actually fit

The original FIFO claim-token queue was a correct prototype for one GPU. It is not the complete design for a national examination system.

At multi-centre scale, we would use separate queues for separate workload classes:

• Ingestion queue: a centre uploaded a shard; validate manifest, checksum and metadata.

• CV queue: assign that shard to an eligible regional GPU pool.

• Verification queue: only gated evidence episodes enter; strict rate limits protect SAM or a future self-hosted verifier.

• Artifact queue: upload compact crops, JSON evidence and audit logs after compute finishes.

• Human-review queue: route evidence packets to authorised reviewers by centre, examination and deadline.

• Retry/dead-letter queue: isolate corrupt uploads, exhausted retries, unavailable GPUs or unsupported formats for human operations review.

Each message must carry a centre ID, exam ID, camera ID, shard ID, pipeline version and idempotency key. A queue is at-least-once by nature, so every stage has to tolerate a duplicate without generating duplicate evidence or duplicate reviewer work. Failed jobs need leases, expiry, exponential backoff and a dead-letter path—not just “try again forever.”

Cloudflare was chosen for this control plane: Workers for authenticated APIs and routing, R2 for video/evidence objects, Queues for asynchronous events, and D1 for the reviewer-facing hierarchy and audit records. We would not make one D1 table the global brain for ten lakh candidates. The control plane must be partitioned by region, centre and examination window; the GPU fleet must be independently autoscaled; and the analytics/audit store must be separated from the live review path.

The core conclusion is simple: ten lakh students are not solved by “more logical workers.” They are solved by a centre audit, camera and network readiness, local buffering, workload estimation, geographic partitioning, bounded model calls, GPU capacity planning, idempotent recovery and a human review process that never converts centre history or academic history into an automated allegation. Also shoutout to Race Engineering - https://raceengineering.ai/ for their GPU Inference.

I personally built this entire thing in 24 hours with much less help from anyone. On a 45-minute sleep, I pretty much enjoyed the process, but not the outcome. At the end of this, I lost. My team was called up immediately because there was no team before us present in the hall. The flow we had prepared did not get a natural runway. The presentation lost synchronicity. Wi-Fi lag added friction at exactly the wrong moment, and I could not present the system at the level I knew it deserved, at the level I knew I could and somewhere in between 4-6minutes; I gave up too.

That is the honest competition ending. It is also not the ending of the engineering story. If anything, the project makes more sense when we refuse to make the result look cleaner than it was. The architecture itself is built around that refusal: a blurred frame does not become certainty, a high confidence label does not become a finding, an unavailable model does not become a negative, and a rushed presentation does not change what we built (hopefully the learning and a positive outcome in the future stays)

I lost the pitch, more that that - I lost myself there. To next time, and trying again -}