There is a moment the previous three essays had been walking toward. I typed a raw JSON-RPC request into a panel of the running app: set color_orange.color to 1 0 0. The orange objects in the viewport turned red, the Save button lit up, and the claim the whole redesign had been making on paper (the GUI is just another agent; both edit one canonical scene document) became something you could watch happen.
Everything before that moment was cage-building, though I did not think of it that way at the time. The plumbing reads like a checklist: a small vocabulary of verbs spoken over JSON-RPC, testable end to end with no LLM anywhere; optimistic concurrency, so a stale agent edit is rejected with a conflict instead of clobbering the human’s work; render-thread and save-path safety, so an agent’s edit can neither tear the scene out from under an in-flight render nor evaporate when the window closes. But the checklist has a shape. Every item makes the environment deterministic, inspectable, and safe, and every item was finished before any model was allowed in, because the plan was always to invite something probabilistic inside. A production surface, in other words, not a chat demo. Then the door opened: a chat loop, multiple model providers, and skills, small teaching documents the model pulls on demand, full of example scenes, every example checked in CI.
Here is the strange part. That cage was the last stretch of the project where the risk lived in the C++. Once a model joined the loop, the dominant risk moved above the renderer: into what the system teaches, what it lets the model do, what it lets the model see, and how I decided whether any of it worked. I went in thinking of the model as a reader of documentation. What I found is a reader unlike any I have written for, one that treats different kinds of text as different classes of interface. A human reads to extract intent; this reader reads the way a machine loads a program. Examples it executes like recipes. Ambient guidance it mostly fails to operationalize. Tool descriptions it treats as part of the action space. The craft is learning which is which, and I learned it in escalating order.
Examples are executable
Every slice of this work went through the adversarial review loop the earlier essays described: fresh reviewer agents, orthogonal lenses, repeat until a round comes back empty. That loop kept an accidental tally of where the serious findings lived. The mechanical layer (syntax tree, derivation, the one edit pathway) produced no further material findings after its first round. The GUI threading layer produced one, a race in the direction nobody designs for first: not “is my dependency dead” but “is my dependency alive and being read by someone else.”
The teaching produced the rest.
The first three arrived together. The skills’ CI contract said every embedded scene snippet must parse and derive cleanly, and every snippet did. Then a reviewer, told to behave like a scene author rather than a code auditor, actually rendered them. Three of seven were broken as teaching examples: two rendered pure black (a glass demo with nothing behind the glass to refract, a directional light that could never connect through a dielectric), and a third presented an invisible glass sphere as one of “four common starters.” The materials skill even retold RISE’s old invisible-glass-sphere war story, then shipped two examples reproducing it by construction; the file whose closing advice was “if everything renders black, check your light direction” was distributing examples that render black for a reason the file never teaches. The CI gate gained its first render-time oracle that day: every snippet must now actually render, with mean luminance inside sane bounds. Coarse, but it ends the era of examples rotting silently, and it encodes the real lesson: deriving cleanly is necessary; rendering visibly is what makes an example an example.
Why do black examples rate as serious defects rather than embarrassing ones? Because this reader follows the recipe. A human skims an example, takes the gist, and course-corrects when the render comes back black; the model executes the example. Among the other teaching findings, one skill contained a rename recipe which, followed verbatim through the real agent surface, wedged the scene in four calls, and saving then reopening amounted to data loss. Another was an “honest limitations” note that over-claimed in the careful direction (“unnamed chunks can never be removed,” false for the one case that mattered) and steered the model into constructing the exact stuck state it warned about. Honesty that overshoots builds the trap it describes.
The doctrine wrote itself: on an agentic surface, prompt text, tool descriptions, and skills are load-bearing code. They get the same review, the same prove-it-fails-first tests, the same drive-it-end-to-end verification as the engine underneath, because a wrong sentence wedges a scene as surely as a wrong pointer.
Then came the harsher half of the lesson. I was reviewing my prose like code because the model read it so literally. Most of my prose, it turned out, the model was not reading at all.
Affordances beat exhortations
My premise for the next feature was simple: a model building a complex scene from a vague brief should be able to ask a clarifying question instead of guessing. Before writing code, we checked the record. On the reactive skill (the user requests something impossible; decline and explain), the stronger frontier models were fluent and the local models never managed it. On the proactive behavior (does a model ever volunteer a clarifying question mid-build), the answer across 132 scene-building runs was zero, including on briefs with genuine ambiguity. The models were mechanically capable of ending a turn with a question; the gap was behavioral, not syntactic. And I knew what a better paragraph would buy, because I had four data points already: four separate guidance interventions in skills and system prompts, measured across the board, zero behavioral effect.
So the question shipped as a tool instead. ask_user presents structured questions with clickable options in the chat panel, and the discipline (ask only when the answer materially changes the scene, at most two or three, never re-ask after being told no user is available) lives in the tool’s own description, on a working theory, still a theory, that text attached to an affordance travels where free-standing text does not.
Grading it meant building something the harness never had: a user. A tool the harness cannot answer is a tool it cannot measure, so the scenario gained a deterministic scripted responder, plus a control fixture that never asks, proving in CI that the ask checkpoint fails exactly when it should. An oracle, not a hope.
The first board came back with nearly every run asking, including a small local model that three previous guidance interventions had failed to move. It was the most flattering result of the whole arc, and it was partly wrong. The scripted responder had shipped with a catch-all default, so any question at all (about mood, about setting, about color) received a pocket-watch-flavored reply, and the board was crediting models with “consuming the answers” for choices the fixture itself had made. The fix removed the default, narrowed the measured claim to the one ambiguity actually under test, and added a checkpoint that inspects what each question was about, since a count only proves the model called the tool. Re-scored under the stricter oracle, the same board held at 12 of 14 runs asking. The finding survived, which is the only reason it is worth quoting, and the rule it left behind is now standing policy: the number that makes you look good is the first one to distrust.
The same shape repeated in the other direction. When one model was burning 143 tool calls on a single scene, my instinct was to soften the skill that tells models to render often. We A/B’d the wording and it did almost nothing; one model shifted slightly, another rendered about 2.6 times per run no matter what the skill said. Then we shipped a batch insert verb, changing no guidance at all, and on the same scene-building scenarios call counts fell 55 to 65 percent; every model on the board adopted it without a word of new instruction. The chattiness was never a personality. It was fifteen chunks and a one-chunk verb.
The honest conclusion is scoped: across this board and these tasks, changing the action space moved behavior far more reliably than changing ambient instructions, in both directions, on and off. I suspect it generalizes, and as a product principle I now act as if it does: when a behavior matters, encode it in the affordance before trying to exhort it into existence.
Widening what it can see
Changing what the model could do had moved it when words could not. The obvious escalation was to change what it could see, and that one started as a single sentence of mine: RISE computes far more than it shows. Alongside the beauty result, the renderer can record primary-hit depth, world-space normals, and diffuse albedo. A human gets one image because human attention is the scarce resource; this reader is not bound to that convention, though every extra channel still spends its input budget.
The theory here is sound but weaker than the sales pitch, and the review loop made us say so. A beauty render is a many-to-one projection; auxiliary channels constrain different parts of the inverse problem, which proves useful information could exist and says nothing about whether this reader can decode it. So the work split into two claims, never conflated: the renderer can expose aligned auxiliary signals cheaply (engineering), and a real model can use them to answer better (empirical, and no amount of information theory gets to declare it shipped). The engineering half is deliberately ordinary: one PNG, a 2×2 atlas of beauty, albedo, normals, and log depth from the same render, pixel-aligned, with structured metadata so the client never reverse-engineers the picture. It fits the image interface every vision model already has.
The empirical half got a small controlled A/B rather than an expensive matrix: one local vision model, twelve authored mirrored cases across three cue families, one arm seeing the beauty image, the other seeing the atlas at a matched input budget. One family produced the result. Asked to judge intrinsic material color under compensating illumination (is the surface dark, or just dimly lit?), the beauty arm scored at chance; the atlas arm answered all twelve material queries correctly, in both call orders. Performance barely changed on the depth-cue cases and declined slightly on the normal-cue ones. We initially reported a p-value and then deleted it, because its independence assumption was false: twelve cases built as six mirrored pairs, each queried three times, are not thirty-six independent samples. The published claim is narrow: this model extracted task-useful information from the atlas on at least one controlled family. Not a general lift, not proof any single panel helped, not a reason to ship six more channels. Weakening the claim did not erase the signal; it made the signal credible.
Notice what the atlas is, though. Not an instruction. Nobody wrote a paragraph urging the model to reason harder about materials. We changed what its eye could reach.
An instrument, not a scoreboard
There is one more reader in this story: me, reading the model. Every result above came out of an evaluation harness that has now survived thirteen epochs of methodology changes, and every epoch was a small confession that the previous one measured slightly the wrong thing. The over-generous scripted user was one. Another looked, from the outside, exactly like a capability wall: one model flatlining on tasks its peers handled, until someone read the wire and found our own provider integration quietly disabling that model’s reasoning. It turns out I am a strange reader too, prone to skimming for the conclusion I want. Measurement infrastructure is load-bearing code, and it gets the same adversarial review, or it lies.
The harness does not make a stochastic collaborator predictable. It makes observations of the collaborator repeatable and inspectable, so conclusions can be challenged against evidence. Deterministic scaffolding is not the opposite of probabilistic systems; it is what makes their behavior measurable.
Four levers
So the strangest reader turns out not to be only a reader. It is a user, and its behavior is shaped jointly by four things you control: what you tell it, what you let it do, what you let it see, and how you decide whether it succeeded. Instructions matter, but affordances dominate; examples shape imitation, tools shape action, observations shape inference, and evaluations determine what you are allowed to believe.
Which is to say that somewhere in this arc my own job description quietly changed. I began as the author of documentation for a new collaborator and ended as the architect of the environment it inhabits, designing its verbs, its senses, and the instruments that tell me the truth about it. You do not write the strangest reader into being a good collaborator. You build the world in which it becomes one. Getting the machine right took three essays; learning that took this one.
The full results table across every frontier model, scores and costs alike, comes in the next essay, once the billing cycle and I have both recovered from the last complete run.
Aravind 