Real time removes the pauses that hide weak architecture
A conventional agent workflow can retrieve evidence, call a model, inspect a tool result, and ask a person to approve the next step. Latency still matters, but the workflow usually has enough time to explain itself. A real-time game, emulator, audio system, XR experience, or interactive visual runtime has a different contract. The next frame, audio quantum, input sample, or machine cycle will arrive whether the agent is ready or not.
That pressure exposes category errors. A renderer that owns the visible picture may begin to look like it owns the world. A predictor that hides network or worker latency may be mistaken for accepted state. A SharedArrayBuffer that makes transport fast may be treated as canonical memory. An AudioWorklet that keeps sound continuous may be allowed to synthesize events the guest machine never produced. A WebGPU path that is dramatically faster may be trusted before it has proved behavioral equivalence.
A real-time system may miss a deadline, lose a device, or degrade presentation. It must not invent a different world.
This is why real-time engineering belongs in the agentic AI story. It forces the system to represent who may observe, predict, execute, deliver, present, degrade, recover, and commit. The model is no longer the only source of uncertainty. Timing itself becomes adversarial.
The first requirement is one authoritative world
In Neon Drift, deterministic world state owns legal movement, bike positions, trails, collision, score, pickups, speed, and round outcome. VANTA and CITADEL may learn patterns, compare paths, coordinate tactics, and decide when confidence is too low. They do not get to move a bike through a wall or reinterpret a collision because a prediction was convenient.
In Arcadia and the emulator suite, the machine core owns registers, memory, device state, input order, guest time, video output identity, and native PCM production. WebGPU, WebGL2, Canvas, AudioWorklet, controller adapters, and presentation queues deliver the machine. They are not alternate emulators with permission to correct the guest.
The design consequence is simple but strict: every fast output is tied to an authoritative epoch. A predicted turn belongs to a particular arena snapshot. A rendered frame belongs to a machine frame and state hash. A block of PCM belongs to a sample sequence. A shared snapshot belongs to a completed generation. When the source epoch changes, the candidate either revalidates, recomputes, or loses influence.
Prediction can protect experience only while it remains visibly provisional
Prediction is useful. It can reduce apparent input delay, choose a tactical route, prebuild a shader pipeline, prefetch a semantic explanation, or interpolate motion between accepted snapshots. The failure begins when a predicted value is allowed to cross into accepted state without the same authority checks as any other proposal.
A robust prediction contract therefore records the source epoch, the features the predictor observed, the candidate path, confidence or ambiguity, expiry, and the exact state transition it is not permitted to perform. If a newer snapshot invalidates the assumptions, the prediction is discarded. Confidence does not extend its lifetime.
This is especially important for learned game intelligence. A short-term model may identify repeated turns or cadence. A stronger contextual model may include corridor width, nearby walls, approach angle, open territory, boosts, or objective state. Those gains are valuable only when the model remains bounded by legal actions and current geometry. Better intelligence should make VANTA more formidable—not clairvoyant, illegal, or opaque.
Workers and shared transport separate ownership from delivery
Moving work off the main thread is not merely a performance optimization. It can establish a cleaner authority boundary. A Worker can own exact machine execution or deterministic world updates while the main thread owns presentation and interaction. MessageChannel can carry typed directives and acknowledgements. SharedArrayBuffer can carry bounded snapshots or ring-buffer data with generation counters and explicit consistency rules.
The dangerous shortcut is to treat shared memory as a free-for-all. A torn snapshot, stale generation, or unbounded queue can create a state that no owner ever accepted. The safe design uses one writer for authoritative structures, versioned slots, atomic generation markers, bounded capacity, and clear behavior when the reader cannot verify a complete update. The reader drops or retries; it does not fill the gap with authority.
Neon Drift illustrates another useful split: shared memory is appropriate for world snapshots and telemetry, while directives, dialogue, and consequential coordination remain typed messages. This keeps the low-latency data path from becoming an ambient command channel.
Audio continuity is a persistent stream with a strict truth boundary
Audio reveals discontinuity immediately. A game can tolerate one repeated visual frame more gracefully than a burst of static, clipping, or a dropped musical phrase. That is why the emulator and Arcadia work increasingly uses persistent AudioWorklet streams and ring buffers rather than repeatedly creating short-lived audio nodes.
Continuity does not authorize replacement sound. The guest machine or arcade hardware profile owns the samples and event sequence. The delivery system may buffer, resample under a declared policy, apply bounded remastering, avoid clipping, and recover from an underrun. It may not invent a weapon, engine, voice, or musical event that did not occur.
The right tests are title-specific and long-running: static, clipping, discontinuity, latency, death and restart, title switching, device suspension, and extended play. A ten-second clean sample is not enough to prove a persistent stream.
WebGPU should be first—and still earn authority
WebGPU is the first/default graphics and presentation path in the OverLift emulator work because it provides the right modern substrate for GPU-resident execution, compute, and presentation. WebGL2 and Canvas remain automatic fail-safe fallbacks after a real initialization, compilation, device-loss, or presentation failure. They are not the default merely because showing any picture is easier than fixing the primary path.
That policy has two parts. The product preference is WebGPU first. The authority rule is that WebGPU output must still be tied to exact state and qualified against a reference. A faster shader, renderer, texture path, or compute kernel can be admitted only after it preserves the behavior that matters. When mismatch appears, the lane is quarantined, the fallback preserves service, and the evidence records why.
Backpressure should shed presentation detail before it sheds truth
Every real-time system eventually produces more work than one lane can consume. Telemetry, semantic explanation, dialogue, frames, audio, controller events, and diagnostics compete for budgets. Unbounded queues convert temporary pressure into permanent latency and memory growth.
The response must be typed. Low-value telemetry can coalesce. Decorative animation can reduce. A frame can repeat. An explanation can update less often. But accepted actions, safety stops, approvals, input sequence, receipts, and rollback markers cannot be silently dropped. Backpressure policy is therefore an authority policy, not just a queue setting.
The portfolio proves the same rule in different machines and worlds
| System | Candidate intelligence or fast path | Authority that remains exact |
|---|---|---|
| Neon Drift / CITADEL | Opponent modeling, path comparison, coordination, dialogue, prediction | Legal movement, geometry, collision, score, input, and round state |
| Arcadia | GPU presentation, remaster profiles, cabinet UX, title switching | Arcade CPU/device execution, native sound events, frame and PCM identity |
| Emulator Suite | WebGPU/WebGL2 rendering, AudioWorklet delivery, compatibility profiles | Guest machine state, timing, input sequence, device behavior, replay |
| C64 XR | Spatial presentation, controllers, asset delivery, diagnostics | C17/Wasm machine, drive/media state, storage, audio, checkpoint identity |
Neon Drift Arcadia C64 XR Emulator Suite
Real-time evaluation measures tails, transitions, and recovery
Averages hide the failures users feel. Real-time qualification needs p50, p95, and p99 latency; underrun and overflow counts; dropped or repeated frames; queue depth; device-loss recovery; input-to-visible response; memory and thermal behavior; state-hash parity; audio and video continuity; and deterministic replay. It also needs title- or scenario-level campaigns rather than one synthetic benchmark.
For agent intelligence, evaluation includes legal-action rate, self-trap rate, stale-plan rejection, abstention accuracy, counterplay variety, and performance against a fixed legacy baseline. For a GPU path, it includes visual and state equivalence plus failure recovery. For audio, it includes long-play continuity and transition behavior. The system must measure the path and consequence, not only whether a demo eventually displayed something.
Recovery is the final proof of authority
A system that cannot explain how it recovers probably does not know what owns state. Exact recovery starts from a verified checkpoint, admitted input ledger, versioned runtime, source identities, and deterministic replay. Speculative presentation is discarded. Failed fast paths remain quarantined until they requalify.
This is the larger agentic lesson. A capable model, clever predictor, fast GPU, or smooth interface can make a system feel intelligent. Authority is proven when the system can lose those conveniences, preserve the ground, reconstruct the accepted world, and show the receipt.