Pose estimation in AutoScript Sync
When AutoScript Sync makes a funscript, pose estimation is done by a YOLOv8-pose model running on your PC. It finds each person in a sampled frame and places 17 keypoints on them. A tracker follows those people from frame to frame and turns their movement into candidate position signals. Pose is optional: it is one candidate among several, and without a pose model the engine runs on optical flow alone.
Checked against the app's code on 21 September 2026.
Which model each Accuracy tier uses
There are four YOLOv8-pose sizes. The app downloads the small one automatically; the others come from the in-app Download best chooser, which recommends the large model on a GPU and the small one on a CPU.
| Model | Size | Picked for, on a CUDA GPU |
|---|---|---|
yolov8n-pose (nano) | ~13 MB | Fast |
yolov8s-pose (small) | ~45 MB | Balanced; the default download |
yolov8m-pose (medium) | ~99 MB | Second choice for Balanced and High when their first choice is not installed |
yolov8l-pose (large) | ~168 MB | High |
This mapping applies with Auto-pick model tier from Accuracy switched on, which is the default. The app can only pick a model that is already installed, so on a default install every tier runs the small model until you download others. On a CPU the caps are deliberate: Fast uses nano only, and Balanced and High use the small model at most. You can also point the app at a model file yourself.
What the model sees: 17 keypoints per person
Pose runs on every sampled frame by default, on the same 320 px wide analysis image the rest of the engine uses. That frame is letterboxed to the model's input size: an ONNX model uses the size it was exported at (640 px for the standard exports), and a .pt model run through the bundled Ultralytics package works at 320 px.
Each person detected with more than 0.35 confidence is kept, with the 17 standard COCO keypoints: head points, shoulders, elbows, wrists, hips, knees and ankles. How often pose runs is a stride in config.json (pose_every_n), set to every sampled frame out of the box.
How the person tracker keeps identities
Detections are only useful if the same person is followed through a scene. AutoScript Sync uses its own tracker for this, not an off-the-shelf one. It matches people between frames by the distance between their box centres and by box size, taking the nearest match first.
From the tracked people it builds several candidate signals: the vertical movement of the hips, and the relative movement of pairs of body regions (hips to hips, head to hips, hands to hips), each divided by body size. Each candidate is scored on three things:
- how strongly it repeats: the share of its power in its strongest peak between 0.2 and 4 Hz;
- coverage: how much of the scene it was actually measured for, with anything under 40% scored zero;
- amplitude: how much it moves.
The best-scoring candidate wins separately for each scene chapter, so a change of position between scenes can change which body-region signal pose contributes.
What pose adds to the script
Pose does not write the script on its own. The final position trace is built from optical flow held to an anchor, and pose is one of the anchor candidates alongside the centre of the moving region and depth. It can become the single elected anchor for a video only when it covers at least 25% of the video and its quality-weighted score is more than 0.6 times the region-centre score. When it is not elected it still feeds the default per-sample blend of all readable channels, and that blend is used only if it scores at least as well as the elected channel. The election scores for each candidate are written into the script's metadata, so you can see what won.
Pose also helps label motion type. Keypoint geometry and face position are used as evidence for each 2-second window's label, and they can override or adjust the classifier's call.
How much pose improves the stroke trace has not been shown. On one test film, switching pose on or off moved stroke agreement (F1) with our reference script only between 0.607 and 0.610, below the 0.01 difference the developer treats as meaningful (run-to-run noise was about ±0.005). That reference was engine output guided by a hand-drawn tracking box, not a hand-made script, and no comparison of the light and heavy pose models on stroke accuracy has been recorded.
Running on the GPU, and the CPU fallback
Pose models run locally as ONNX through ONNX Runtime, or as .pt files through the bundled Ultralytics and PyTorch stack. Acceleration is NVIDIA CUDA only; there is no AMD or Intel acceleration for the models, and TensorRT is not used.
ONNX Runtime can quietly load on the CPU when CUDA is not usable, so the app checks which provider a session actually got rather than assuming. If a CUDA GPU was requested but cannot be used, pose falls back to the CPU and the app reports the provider in use. The Check GPU button runs this check on demand; the ms/frame it shows is the pose model's own self-test, not the speed of a whole analysis.
By default the app also turns off cuDNN autotuning so GPU inference gives the same numbers from run to run, at some cost in speed. That does not guarantee an identical script on regeneration, because the app learns between runs.
Limits
- Hidden bodies. A keypoint the model cannot see is not measured. A candidate signal measured for under 40% of a scene scores zero, and pose cannot become the elected anchor below 25% coverage of the video, so in heavily hidden scenes the script leans on optical flow and the region centre instead.
- Crowded frames. The tracker matches people by box position and size only. It has no appearance model, so when people overlap or cross, identities can be swapped between frames.
- Small figures. Pose runs on the 320 px analysis frame, so a person who fills a small part of a wide shot gives the model few pixels to work with.
- Speed. Pose has been the largest share of analysis time in the developer's measurements, which is why the tiers pick smaller models for Fast and cap the model size on a CPU.
Questions
Do I need a pose model to make a script?
No. Without one the engine runs on optical flow alone and the Inference status line says so. The small pose model is downloaded automatically unless you turn automatic downloads off.
Should I download the large pose model?
On an NVIDIA GPU, Download best recommends it and High will use it. We have not measured whether it makes better scripts than the small model, so it is a choice, not a fix.
Does pose run on an AMD or Intel graphics card?
Not accelerated. AI models use NVIDIA CUDA only; on other hardware pose runs on the CPU, which works but is much slower.
Does the pose model send anything online?
No. It runs on your PC, and the app's analysis modules contain no network code. Only the model download itself needs the internet.
Read next
- How AI funscript generation worksThe whole pipeline, and where pose sits in it.
- Optical flow motion trackingThe signal pose is weighed against.
- Depth estimation with MiDaSThe other optional model, and the axis it recovers.
- Running AutoScript Sync on WindowsGPU, CPU, what the installer brings and where models live.
Try it on your own videos
The trial lasts one day and runs on your own PC; your videos are analysed locally and never uploaded.