How we measure generation quality and speed
This is the method behind any funscript benchmark we publish: how a generated script is scored against a reference, where the reference scripts come from, and the one speed figure we have. There is no accuracy table on this page yet. The first batch scored against hand-made scripts is in, and a second, held-out batch comes before any accuracy figure is published.
Checked against the project's measurement tools and records on 21 September 2026.
How a script is scored
A funscript is a list of positions over time. Scoring starts from its turns: every point where the script stops moving one way and starts moving the other. The generated script's turns are compared with the reference script's turns over the same stretch of video.
- Matching
- A generated turn counts as a match when it lands within ±150 ms of a reference turn. Anything further away is a miss on one side and an extra turn on the other.
- F1
- One number combining how many of the reference turns were found and how many of the generated turns were real. 1.0 means every turn matched and none was invented; missing turns and extra turns both pull it down.
- Timing error
- For the matched turns only, how far apart the two turns are in milliseconds, reported as the median and the 90th percentile. No mean or 95th percentile is computed. Unmatched turns are left out of this figure, so a timing error always has to be read next to the F1 it came with: a low timing error on a low F1 describes only the turns that were found.
- Stroke size ratio
- The median length of the generated script's strokes divided by the median length of the reference's, with no matching. 1.0 means strokes of the right size on average; below 1.0 means strokes that are too small.
- Turn rate and idle motion
- Against hand-made scripts we also count direction changes per second, and how much the script keeps moving through stretches the scripter marked as having no action.
Repeat runs with the same settings on one film varied by about ±0.005 F1, so we treat any difference smaller than 0.01 F1 between two versions or settings as noise.
Where the reference scripts come from
A score is only as good as the script it is compared with, and for most of the project's history that script was not independent.
Until 19 September 2026: Live Track references
Every F1, timing and stroke-size figure recorded before 19 September 2026 compared the engine with Live Track references. Those were the engine's own output from 22 August 2026, steered by a person dragging a tracking box, not scripts made by hand. A new version can disagree with them because it is worse, or because it is better than the engine that made them. So these comparisons can stop a bad change, but they cannot confirm a gain, and any figure from them can only be described as agreement with our references, never as accuracy against hand-made scripts. The "27 ms" timing figure that has appeared on this site came from this kind of comparison.
Since 19 September 2026: blind hand-scripted clips
The truth now comes from short clips scripted by hand under a blind protocol:
- Clips 12 seconds long are cut from the test films at windows picked by a fixed random seed. Each gets a random name and has no sound, title or timecode.
- The key saying which film each clip came from is kept apart and not opened while scripting, and the scripting order mixes the films.
- The scripter makes a funscript for each clip by hand, following written rules that were frozen before the first clip was scripted.
- Only the middle 8 seconds of each clip is scored; the 2 seconds at each end are not.
- The engine's script for the same stretch is scored against the hand-made one with the measures above.
As a check on the truth itself, the scripter's stroke rate matched the video's own dominant motion frequency to within about 0.1 Hz on 7 of the 9 clips checked.
The first batch, 28 clips from five films scored on 19 September 2026, has two limits we state plainly. It was scripted by one person, the developer. And the clips come from the same five films the default settings were tuned on, so nothing in it is held out. It is an exploration batch: good for finding weaknesses, not for a headline number.
The one speed measurement
We have one current speed figure, from one PC. Read it with every condition attached:
| Condition | Value |
|---|---|
| Hardware | NVIDIA RTX 5080 graphics card, Intel i7-13700K processor |
| Accuracy tier | Balanced, with fresh-install defaults (YOLOv8s pose model, MiDaS small depth model, every 3rd frame analysed) |
| Build | Version 2.121.2 defaults, run from source, not the installed app. 2.121.2 is built but not released; customers run 2.121.1 |
| Films | 5 films, 32–39 minutes long, 29–60 fps, 1080p and 2160p |
| Cost per analysed frame | 26.3–36.7 ms |
| Time per film | 8.2–28.5 minutes |
| When | Around 15 September 2026 |
On that PC, every one of the five films took less time to analyse than to play. Because Balanced analyses every 3rd frame, a 60 fps film has twice as many frames to analyse per minute as a 30 fps one and costs roughly twice as much. Time on the clock also depends on what else the PC is doing: the same films took far longer when runs overlapped with other work.
Only the ranges above survive in the project's records; the per-film breakdown of this run was not kept, so we do not pair a time with a particular film.
This is not a promise for your PC. It says nothing about other graphics cards, about running on the processor alone, or about the High and Fast tiers.
What is not measured yet
- Other graphics cards. Only the RTX 5080 has been timed. There is no figure for mid-range cards.
- Processor only. The app runs without an NVIDIA graphics card, more slowly, but that has not been timed.
- The High and Fast tiers at today's settings. No current timing exists for either.
- The installed app. Every measurement so far was run from source, not from the installer build.
- Timing error against hand-made scripts. The hand-scripted batch has F1, turn rate, stroke size and idle motion, but no millisecond timing error yet.
- Device latency and drift in real sessions. Nothing records how closely a device actually follows the video: no measured round-trip latency, drift or start offset for the Autoblow AI Ultra, and none for Intiface devices or DeoVR. The drift thresholds described in how the app corrects Autoblow drift are what the app aims for, not measured results. Only the Autoblow AI Ultra has been tested on real hardware; The Handy, Kiiroo, Lovense and OSR2/SR6 have been tested against simulated devices only.
- A held-out accuracy batch. No batch of clips from films the settings were never tuned on has been scored yet.
- Motion-type classification in the default app. Its accuracy on the features the running app uses has never been measured.
When results will be published
The first hand-scripted batch is scored. It showed us where the engine is weak: it makes fewer turns than the scripter on fast scenes, its strokes run small on the fastest ones, and it keeps moving through stretches with no action. We are not quoting its F1 or stroke-size numbers as an accuracy figure, because the clips came from the films the defaults were tuned on and there was one scripter.
Before any accuracy figure goes on this site, a held-out batch will be cut from films the settings were not tuned on and scored the same way. When it is, this page will show the numbers with the method above, the number of clips and films, and who scripted them. Any speed figure added later will carry the same conditions as the one above: graphics card, tier, frame rate, resolution and build. We have not set a date.
Questions
What happened to the "27 ms median timing error"?
It was measured against the Live Track references, which were engine output, and it counted only turns matched within ±150 ms. At the current Balanced defaults the same measurement gives 21–35 ms per film, but it is agreement with our own references, not accuracy against a hand-made script. No timing error against hand-made scripts has been computed yet.
How fast will it be on my PC?
We cannot say yet. The only measurement is from an RTX 5080, where 32–39 minute films took 8.2–28.5 minutes at Balanced. We have no figure for any other graphics card or for the processor alone. The free trial lets you time it on your own hardware.
Why not publish the hand-scripted results now?
Because the batch is too narrow to stand as an accuracy figure: one scripter, and clips from the same films the defaults were tuned on. A held-out batch fixes the second problem; it comes first.
Does the Accuracy setting change quality?
It changes how many frames are analysed: High every 2nd frame, Balanced every 3rd, Fast every 4th. On one film in early September 2026, with the engine and High settings of that time and scored against the Live Track references, Fast reached about half the F1 of High (0.316 against 0.605). Current-settings comparisons between the tiers against hand-made scripts have not been made.
Read next
- How the engine worksPose, optical flow and depth, and what each contributes to a script.
- Drift correctionHow playback on an Autoblow AI Ultra is kept in step with the video, and what is and is not measured about it.
- Report StudioLook at your own generation's report in the browser.
- Local processingThe analysis being timed here runs on your PC.
Measure it on your own videos
The trial is the full app for one day, so you can time a generation on your own PC and judge the scripts yourself.