Benchmarks

Two things decide whether video-as-code is worth it: how fast a render is, and how often an agent gets the scene right without a human. Both are measured here on one machine, each with a date and a script you can run. The scripts are open; if you maintain another engine and the numbers are stale or unfair, send a pull request.

Render speed

The same scene written in each tool and in oxframe, rendered at 1920 × 1080. Wall time of the render command, median of three runs after a warm-up, 2026-10-22, on Intel(R) Core(TM) Ultra 7 255H, 16 threads, Linux. oxframe renders on the GPU by default; --cpu forces the software renderer so the number does not depend on the graphics card.

Each row is a different scene, so compare a tool with oxframe across a row, not one row with another. fframes is not timed here yet — it needs codec development packages the benchmark machine lacks — and Motion Canvas has no headless render to time. A batch of 3 personalised cards, 2.42 s each, renders in 14.65 s on the CPU, one after another.

Script: bench/vs/measure.sh. Each tool's version is in the table; the full command per tool is in vs-measurements.json.

Agent success

The harder question for an AI agent is not speed but whether it gets the scene right. The reference set is 47 prompts — algorithms, charts, code walkthroughs, shaders, slides, voice and vertical formats — each given to Claude Code in a fresh oxframe init project. A prompt passes when the scene builds, oxframe check reports no error, and the scene does what the prompt asked.

oxframe 0.20.5, 2026-10-22. The one miss is the useful part: on a code-explanation prompt the agent wrote plain text instead of the Code node, which would have let it highlight the expression a piece at a time — a gap in the skill to close next. (Two 9:16 shorts and one video prompt first failed on bugs in the scorer itself — a vertical pixel size it did not read as 9:16, and an asset a fresh project lacks — both since fixed in the eval.)

Script: evals/run.py in oxframe-cli, with the prompts beside it. It runs any agent (--agent 'cmd:…'), so the same set can time another tool once its skill exists — that cross-tool table is the open invitation above.