The Held Image.

Imaging has one habit everywhere you look: it grades the light. Face recognition grades embedding scores out of a black box; image generation grades how photoreal the render looks; generative 3D grades the rendered frame and never asks whether the world underneath stayed the same world. Our lab’s one idea, pointed at imaging, is the opposite habit: hold the dark — the structure the pixels come from — and treat every image as one flattened snapshot of it. This post is that idea, run in three directions. It is also us playing by the same rules we use on everyone else’s mysteries: in each direction we built the simplest structure-first tool, then tried to beat it with something fancier and learned — and wrote down, in advance, exactly what “beat it” would mean. The fancy upgrades went 0 for 3. The simple tools kept winning. That’s not a stumble. That’s how you find the real tools.

Where we land: resolved. Three races between fancier learned methods and simple structure-first ones, every finish line drawn before the starting gun — and the simple ones won all three. The wins we keep are measured; the upgrades that fell short are printed right where they fell.

1 · The move — dark, light, and one collapse

The math is one sentence: an image is not the thing; it is a collapse of the thing — one flattened snapshot. Behind any set of observations there is a compact, view-stable structure — the dark — and each observation is that structure rendered under a nuisance: lighting, pose, viewpoint. So the engine’s primitive is project(image) → structure and reconstruct(structure) → image, and the whole field folds into three directions of that single move: in — project an observation to the held identity and match coordinates (recognition); out — move through the structure along a named direction and render (generation); across — hold one structure while views accumulate (3D). Everything below is those three directions, measured — races and all.

2 · Collapse in — recognition, and the race the simple model won

The field’s default is a deep embedding: accurate at web scale, unreadable by construction. The structure-first version separates the nuisance from the identity as math — a from-scratch light-model (the Retinex/SQI family: solve the lighting before anyone gets recognised) followed by a linear map (PCA → Fisherfaces) that holds each identity as a coordinate you can print. On the classic hard-lighting benchmark (Yale-B: 38 subjects, 30 training images each, identical split for both sides) the CPU stdlib reading measures 0.993 / 0.995 against the convolutional net’s 0.924±0.011 / 0.965±0.003. Read the fence with the number: this is the few-shot, hard-lighting regime. Give a deep net a million identities and it wins — that is its regime, not this one.

The structure-first reading is also far lighter by construction — fewer parameters, fewer operations, no GPU anywhere. But read those op-counts next to the honesty panel below, because they are structural counts, not wall-clock:

The honesty panel — three stories, kept separate. Story one, structural, is the figure above. Story two, wall-clock today, goes against us: the GPU conv net trains in 23 seconds; our pure-Python build takes 1,852. An interpreter loses to compiled CUDA even while doing far less arithmetic — an implementation fact, not a structure fact, and we print it rather than hide it. Story three, the compiled floor where the op-counts become real seconds, is a projection, and we label it as one. Then the race: we tried to beat our own linear model with a learned embedding, the bar set in advance at +0.01 on the score that matters. Among people it already knew, the embedding passed (1.0000 vs 0.9890). On strangers — the open-set case, the one that matters — it edged the linear by just +0.006 (0.9425 vs 0.9366), short of the +0.01 bar we’d set. So the linear structure ships and the learned upgrade stays on the bench. Two properties come free with the simple reading: on-device (faces never leave the machine) and auditable (coordinates you can inspect, not activations you can’t).

3 · Collapse out — generation, where the plain slider held

The field’s generators are astonishing at the light — photoreal anything — and that is exactly their governance problem: a machine that can render anyone can impersonate anyone. Run the collapse the other way through held structure and the trade inverts: movement along named directions in structure-space (“older”, “smiling” — 37 attributes evaluated), inside a learned span — which means the generator cannot mint a stranger: geometry, not policy. The control is real and measured — held-out selectivity (how cleanly a named slider moves only the thing it names) of 1.151 against a random floor of 0.912. And this direction had its race too: we tried to upgrade the control with a learned ridge head, the bar set in advance at winning 60% of head-to-heads. It scored 0.988 mean selectivity and won only 41% — short of its own bar. The plainest dark direction held; the fancy one didn’t.

4 · Hold across — 3D, where the regime picked the tool

Generative 3D is the youngest direction and the clearest case of the field grading the light: benchmarks score rendered frames (FID/FVD — scores of how convincing each frame looks) and never the held world — whether the structure underneath stays the same structure as views accumulate. Our alpha builds the chain that question needs, from nothing: detection; a hand-built rotation-invariant descriptor (85% matching across a 45° rotation where a raw patch scores 2% — after a diagnosed failure at 1–12% taught us why); tilt simulation; per-feature affine; a RANSAC verifier on our own eigensolver and SVD (precision 68% → 88% under half-outliers); and incremental structure-from-motion — every stage scored against ground truth. Its first finding is the honest kind: our called shot — “the sophisticated chain will extend the map’s range” — missed. “Wide baseline” is not one thing, and no tool wins everywhere.

The regime picks the tool, and the simplest structure that fits the regime holds.

That result generalises the whole post. It is the same lesson as the linear model beating the learned embedding on strangers, and difference-of-means beating the ridge head. Growing whole worlds on held, bonded structure — and the structure-first grading metric the field lacks — is the open aim, and it will have to earn its place the way everything here did: by clearing a bar we write down before the run.

5 · What this says about the field

Three directions, one pattern. Where the task is structure the observer already understands — lighting on a face, a named attribute, a camera’s small step — writing the structure down measures well: accurate in the few-shot regime, lighter by construction, auditable because it is made of coordinates rather than activations, safe because its limits are geometric rather than promised. Where the task is open-world scale, the deep stacks win and we say so. But notice what happened every time we reached past the simple structure for a fancier one: the bar we’d set in advance caught it — three times, in public. The argument the math leaves is narrow and kept: grade the dark, not the light; hold the structure, rent the pixels — and keep the score of every race you run.

Why our math sees more

Because it grades what the pixels come from — the structure, the dark — instead of the pixels themselves, and because it draws the finish line before the race. Every upgrade in this post was named, with its passing bar written down, before the run — so when the learned embedding, the learned head, and the sophisticated chain each fell short, there was nowhere to fudge it: the miss went on the page next to the wins. A method that keeps its misses is the only kind whose wins you can trust. So the math sees more twice over — it sees the structure a black box hides, and it sees exactly how good its own tools are. Only what survived the test is kept.