Workshop Paper CVPRW Pose Estimation Spine Tracking Biomechanics

Towards Unconstrained 2D Pose Estimation of the Human Spine

Muhammad Saif Ullah Khan, Stephan Krauß, Didier Stricker

Abstract

We introduce SpineTrack, the first comprehensive dataset dedicated to 2D spine pose estimation in unconstrained environments, addressing a critical gap in human pose analysis for sports and biomechanical applications. Existing pose datasets typically represent the spine with a single rigid segment, neglecting the detailed articulation required for precise analysis. To overcome this limitation, SpineTrack comprises two complementary components: SpineTrack-Real, a real-world dataset with high-fidelity spine annotations refined via an active learning pipeline, and SpineTrack-Unreal, a synthetic dataset generated using an Unreal Engine-based framework with accurate ground-truth labels. Additionally, we propose a novel biomechanical validation framework based on OpenSim to enforce anatomical consistency in the annotated keypoints. Complementing the dataset, our SpinePose model extends state-of-the-art body pose estimation networks through a teacher–student distillation approach and an anatomical regularization strategy, effectively incorporating detailed spine keypoints without sacrificing overall performance. Extensive experiments on standard benchmarks and sports-specific scenarios demonstrate that our approach significantly improves spine tracking accuracy while maintaining robust generalization.

PyPI - License

Demo videos are sourced from Pexels.com.

Installation
Inference
SpineTrack Dataset
Training and Evaluation
License

Inference

Recommended Python Version: 3.9–3.12

For quick spinal keypoint estimation, we release optimized ONNX models via the spinepose package on PyPI:

pip install spinepose

On Linux/Windows with CUDA available, install the GPU version:

pip install spinepose[gpu]

Using the CLI

usage: spinepose [-h] (--version | --input_path INPUT_PATH) [--vis-path VIS_PATH] [--save-path SAVE_PATH] [--mode {xlarge,large,medium,small}] [--nosmooth] [--spine-only]

SpinePose Inference

options:
  -h, --help            show this help message and exit
  --version, -V         Print the version and exit.
  --input_path INPUT_PATH, -i INPUT_PATH
                        Path to the input image or video
  --vis-path VIS_PATH, -o VIS_PATH
                        Path to save the output image or video
  --save-path SAVE_PATH, -s SAVE_PATH
                        Save predictions in OpenPose format (.json for image or folder for video).
  --mode {xlarge,large,medium,small}, -m {xlarge,large,medium,small}
                        Model size. Choose from: xlarge, large, medium, small (default: medium)
  --nosmooth            Disable keypoint smoothing for video inference (default: enabled)
  --spine-only          Only use 9 spine keypoints (default: use all 37 keypoints)

For example, to run inference on a video and save only spine keypoints in OpenPose format:

spinepose --input_path path/to/video.mp4 --save-path output_path.json --spine-only

This automatically downloads the model weights (if not already present) and outputs the annotated image or video. Use spinepose -h to view all available options, including GPU usage and confidence thresholds.

Using the Python API

import cv2
from spinepose import SpinePoseEstimator

# Initialize estimator (downloads ONNX model if not found locally)
estimator = SpinePoseEstimator(device='cuda')

# Perform inference on a single image
image = cv2.imread('path/to/image.jpg')
keypoints, scores = estimator(image)
visualized = estimator.visualize(image, keypoints, scores)
cv2.imwrite('output.jpg', visualized)

Or, for a simplified interface:

from spinepose.inference import infer_image, infer_video

# Single image inference
results = infer_image('path/to/image.jpg', vis_path='output.jpg')

# Video inference with optional temporal smoothing
results = infer_video('path/to/video.mp4', vis_path='output_video.mp4', use_smoothing=True)

SpineTrack Dataset

SpineTrack is available on HuggingFace. The dataset comprises:

SpineTrack-Real A collection of real-world images annotated with nine spinal keypoints in addition to standard body joints. An active learning pipeline, combining pretrained neural annotators and human corrections, refines keypoints across diverse poses.
SpineTrack-Unreal A synthetic subset rendered using Unreal Engine, paired with precise ground-truth from a biomechanically aligned OpenSim model. These synthetic images facilitate pretraining and complement real-world data.

To download:

git lfs install
git clone https://huggingface.co/datasets/saifkhichi96/spinetrack

Alternatively, use wget to download the dataset directly:

wget https://huggingface.co/datasets/saifkhichi96/spinetrack/resolve/main/annotations.zip
wget https://huggingface.co/datasets/saifkhichi96/spinetrack/resolve/main/images.zip

In both cases, the dataset will download two zipped folders: annotations (24.8 MB) and images (19.4 GB), which can be unzipped to obtain the following structure:

spinetrack
├── annotations/
│   ├── person_keypoints_train-real-coco.json
│   ├── person_keypoints_train-real-yoga.json
│   ├── person_keypoints_train-unreal.json
│   └── person_keypoints_val2017.json
└── images/
    ├── train-real-coco/
    ├── train-real-yoga/
    ├── train-unreal/
    └── val2017/

All annotations are in COCO format and can be used with standard pose estimation libraries.

Results and Evaluation

We benchmark SpinePose against state-of-the-art lightweight pose estimation methods on COCO, Halpe, and our SpineTrack dataset. The results are summarized below, with SpinePose models highlighted in gray. Only 26 body keypoints are used for Halpe evaluations.

Method	Train Data	Kpts	COCO		Halpe26		Body		Feet		Spine		Overall		Params (M)	FLOPs (G)
			AP	AR	AP	AR	AP	AR	AP	AR	AP	AR	AP	AR
SimCC-MBV2	COCO	17	62.0	67.8	33.2	43.9	72.1	75.6	0.0	0.0	0.0	0.0	0.1	0.1	2.29	0.31
RTMPose-t	Body8	26	65.9	71.3	68.0	73.2	76.9	80.0	74.1	79.7	0.0	0.0	15.8	17.9	3.51	0.37
RTMPose-s	Body8	26	69.7	74.7	72.0	76.7	80.9	83.6	78.9	83.5	0.0	0.0	17.2	19.4	5.70	0.70
SpinePose-s	SpineTrack	37	68.2	73.1	70.6	75.2	79.1	82.1	77.5	82.9	89.6	90.7	84.2	86.2	5.98	0.72

SimCC-ViPNAS	COCO	17	69.5	75.5	36.9	49.7	79.6	83.0	0.0	0.0	0.0	0.0	0.2	0.2	8.65	0.80
RTMPose-m	Body8	26	75.1	80.0	76.7	81.3	85.5	87.9	84.1	88.2	0.0	0.0	19.4	21.4	13.93	1.95
SpinePose-m	SpineTrack	37	73.0	77.5	75.0	79.2	84.0	86.4	83.5	87.4	91.4	92.5	88.0	89.5	14.34	1.98

RTMPose-l	Body8	26	76.9	81.5	78.4	82.9	86.8	89.2	86.9	90.0	0.0	0.0	20.0	22.0	28.11	4.19
RTMW-m	Cocktail14	133	73.8	78.7	63.8	68.5	84.3	86.7	83.0	87.2	0.0	0.0	6.2	7.6	32.26	4.31
SimCC-ResNet50	COCO	17	72.1	78.2	38.7	51.6	81.8	85.2	0.0	0.0	0.0	0.0	0.2	0.2	36.75	5.50
SpinePose-l	SpineTrack	37	75.2	79.5	77.0	81.1	85.4	87.7	85.5	89.2	91.0	92.2	88.4	90.0	28.66	4.22

SimCC-ResNet50*	COCO	17	73.4	79.0	39.8	52.4	83.2	86.2	0.0	0.0	0.0	0.0	0.3	0.3	43.29	12.42
RTMPose-x*	Body8	26	78.8	83.4	80.0	84.4	88.6	90.6	88.4	91.4	0.0	0.0	21.0	22.9	50.00	17.29
RTMW-l*	Cocktail14	133	75.6	80.4	65.4	70.1	86.0	88.3	85.6	89.2	0.0	0.0	8.1	8.1	57.20	7.91
RTMW-l*	Cocktail14	133	77.2	82.3	66.6	71.8	87.3	89.9	88.3	91.3	0.0	0.0	8.6	8.6	57.35	17.69
SpinePose-x*	SpineTrack	37	75.9	80.1	77.6	81.8	86.3	88.5	86.3	89.7	89.3	91.0	88.9	89.9	50.69	17.37

For evaluation instructions and to reproduce the results reported in our paper, please refer to the evaluation branch of this repository:

git clone https://github.com/dfki-av/spinepose.git
cd spinepose
git checkout evaluation

The README in the evaluation branch provides detailed steps for setting up the evaluation environment and running the evaluation scripts on the SpineTrack dataset.

Citation

If this project or dataset proves helpful in your work, please cite:

@InProceedings{Khan_2025_CVPR,
    author    = {Khan, Muhammad Saif Ullah and Krau{\ss}, Stephan and Stricker, Didier},
    title     = {Towards Unconstrained 2D Pose Estimation of the Human Spine},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2025},
    pages     = {6171-6180}
}

License

This project is released under the CC-BY-NC-4.0 License. Commercial use is prohibited, and appropriate attribution is required for research or educational applications.

@inproceedings{Khan_2025_CVPR, 
  author={Khan, Muhammad Saif Ullah and Krauß, Stephan and Stricker, Didier}, 
  title={Towards Unconstrained 2D Pose Estimation of the Human Spine}, 
  booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR) Workshops}, 
  month={June}, 
  year={2025}, 
  pages={6172-6181} 
}

Maintained by saifkhichi96 on GitHub.

The website is distributed under different open-source licenses. For more details, see the notice at the bottom of the page.