Spine pose estimation, spine motion analysis, and AI for spine biomechanics across radiographs, CT, MRI, structured light, RGB, and RGB-D.
Work on spine vision is scattered across medical imaging, posture screening, surgical navigation, and markerless motion analysis. I made this page to put those threads next to each other: what people measure, what datasets exist, and which questions are still hard to test.
- clinical imaging and deformity analysis
- surface posture systems and structured light
- markerless motion, benchmarking, and inference
from clinical imaging to markerless sensing
benchmarks, systems, and datasets compared below
showing how large the literature already is
reported by the scoping review of deep learning for spine care
Why this page exists
Public discussion of spine pose estimation, spine motion analysis, and spine biomechanics AI is still scattered: radiology papers focus on anatomy and pathology, commercial posture systems focus on repeated screening, and motion-focused computer-vision work rarely connects back to vertebral biomechanics. This hub compresses that landscape into one map.
- The field is strongest on static anatomical understanding in CT and MRI, where shared segmentation benchmarks already exist.
- Radiograph and biplanar systems are clinically important for deformity and alignment, but open ML workflows are less standardized.
- Surface topography and structured light enable repeated, radiation-free follow-up, yet public benchmarking remains sparse.
- Markerless RGB and RGB-D systems are only beginning to expose spine-aware motion, vertebral kinematics, and deployable inference.
Modalities and systems
These cards separate sensing modalities by what they measure well, how mature their evaluation is, and where public reproducibility still falls short.
Standing radiographs and biplanar X-ray
High value for deformity, alignment, and weight-bearing assessment.
Weight-bearing radiographs remain central to scoliosis, sagittal balance, and deformity analysis. Computer-vision work here focuses on landmarks, Cobb angle estimation, and 3D reconstruction from limited views.
- single-view projection ambiguity
- benchmarking is narrower than clinical usage
- commercial 3D workflows are more mature than open ML tooling
CT anatomy and vertebra segmentation
The best-covered open setting for vertebra labeling and segmentation.
CT provides rich anatomy, robust vertebral boundaries, and multi-center datasets that support automated labeling, segmentation, and morphometric analysis. It is the most benchmark-ready spine-imaging lane today.
- structure is captured better than motion
- benchmarks emphasize anatomy, not downstream biomechanics
- cross-scanner robustness is improving but still matters
MRI and soft-tissue segmentation
Strong for discs, vertebrae, canals, and degeneration-related structures.
MRI supports lumbar structure delineation without ionizing radiation and is central for degenerative disease, spinal canal analysis, and soft-tissue context. Open benchmarking exists, but is still less mature than CT.
- protocol heterogeneity across scanners and institutions
- challenge infrastructure exists, but fewer dominant public baselines
- biomechanics interpretation is mostly indirect
Surface topography and structured light
Radiation-free follow-up and posture analysis are strong practical advantages.
Rasterstereography and structured-light systems measure back-surface geometry to infer posture and scoliosis-related asymmetries. They are attractive for repeated monitoring, but public ML datasets and common benchmarks are rare.
- surface geometry is only an indirect proxy for vertebral anatomy
- public datasets are limited
- commercial devices dominate over open research infrastructure
RGB and multiview markerless motion
This is where unconstrained posture and motion finally meet biomechanics-aware supervision.
Conventional pose datasets barely represent the spine beyond shoulders and pelvis. Newer work starts to recover spine-centric 2D landmarks, multiview 3D positions, and vertebral rotations from natural full-body motion.
- detailed ground truth is scarce
- annotation policies differ sharply between datasets
- benchmarking is only now becoming reusable
RGB-D and intraoperative sensing
Dense 3D sensing matters when anatomy must be registered during a procedure.
RGB-D and point-cloud methods are emerging for exposed-spine capture, navigation, and registration. The research emphasis is less on general posture and more on anatomically faithful geometry under procedure-time constraints.
- datasets are smaller and more specialized
- few field-standard benchmarks exist yet
- generalization outside operating-room conditions is unresolved
What spine vision is actually trying to predict
The field is broader than segmentation. These task families clarify why some modalities look mature and others still look structurally thin.
Localization and labeling
Identify vertebral levels, centroids, landmarks, discs, or anatomical key points before downstream measurements.
Segmentation and structure extraction
Recover vertebrae, discs, canals, or back-surface geometry at dense spatial resolution.
Alignment and deformity quantification
Estimate Cobb angle, sagittal balance, curvature, and postural asymmetry for screening or planning.
3D reconstruction under load
Infer patient-specific 3D spine structure under standing, weight-bearing conditions.
Dynamic motion and biomechanics
Move beyond static posture into vertebral motion, rotational kinematics, and anatomically plausible movement.
Intervention support and registration
Use imaging or RGB-D geometry to support navigation, registration, or procedure-time guidance.
Benchmarks, systems, and representative releases
This matrix is intentionally cross-modal. It mixes public benchmarks, commercial clinical systems, and open research releases so visitors can compare what is measurable, what is reproducible, and what is still missing.
| Resource | Modality | Primary focus | Supervision | Evaluation | Access and tooling | Biomechanics value |
|---|---|---|---|---|---|---|
|
AASCE / SpineWeb
radiographs
landmarks
Cobb angle
|
Single-view radiographs | Scoliosis landmark detection and curvature estimation | Annotated vertebral landmarks and alignment targets on AP X-rays | Challenge-style ranking centered on landmark and curvature accuracy |
Public challenge resources via SpineWeb and Grand Challenge
Benchmark-oriented ecosystem, but no dominant reusable inference stack
|
Useful for coronal alignment and deformity; not dynamic |
|
EOS / sterEOS
biplanar
3D reconstruction
weight-bearing
|
Biplanar low-dose X-ray | Weight-bearing 2D/3D reconstruction and alignment analysis | Expert-guided reconstruction from paired standing radiographs | Clinical workflow rather than open leaderboard |
Commercial platform
No public inference stack
|
Very high for structural alignment under load |
|
VerSe
CT
segmentation
labeling
|
CT | Vertebra localization, segmentation, and identification | Volumetric vertebra masks and labels across heterogeneous CT scans | Standardized Grand Challenge tasks with shared metrics and reports |
Public benchmark
Open challenge reports and baseline implementations are available
|
Indirect; excellent for anatomy, weak for motion |
|
SPIDER
MRI
segmentation
lumbar
|
Lumbar MRI | Lumbar spine segmentation and clinically relevant structure delineation | Pixelwise annotations for lumbar structures on MRI | Continuous challenge setup with hidden test evaluation |
Public challenge infrastructure
Shared baselines and evaluation framework
|
Indirect; more anatomy and pathology than motion |
|
DIERS formetric 4D
surface topography
screening
structured light
|
Structured light / rasterstereography | Back-surface posture analysis and radiation-free monitoring | Surface-shape capture and anatomical proxies rather than direct vertebral labels | Clinical repeatability and correlation studies |
Commercial system
Public code and shared ML benchmarks are sparse
|
Moderate; useful for posture and follow-up, indirect for vertebral anatomy |
|
SpineDepth
RGB-D
surgery
registration
|
RGB-D and point clouds | Dense 3D reconstruction of exposed spine anatomy for surgical registration | Paired RGB-D depth completion and anatomy-aware geometry targets | Research-task metrics for depth and geometry quality |
Research release
Useful reference point, but not yet a field-standard benchmark stack
|
High anatomical fidelity in procedure-specific settings |
|
SpinePose
RGB
2D pose
inference
|
Monocular RGB | 2D spine-centric landmark estimation in natural images | Manual 2D spine landmarks with biomechanics-aware evaluation | Public paper benchmarks on spine-centric 2D tasks |
Paper page, dataset, and inference entry point
Inference library available
|
Moderate; stronger than standard pose skeletons, still 2D |
|
SIMSPINE
RGB
3D motion
benchmark
|
Multiview RGB with simulation-derived supervision | 3D spinal keypoints, rotational kinematics, and benchmark baselines | Simulated vertebral markers, multi-view detections, and inverse-kinematics labels | Reference baselines across 2D, multiview 3D, monocular 3D, and biomechanics-aware checks |
Paper page plus dataset on Hugging Face
Benchmark assets released; inference support is the next tooling step
|
High; anatomy-aware motion is the core point |
Where my own research sits
My work currently lives in the smallest but least-served part of this map: spine-aware understanding from external sensing, especially RGB and multiview motion, with a stronger emphasis on vertebral structure and biomechanics than conventional pose estimation.
Put differently: most of the open field is still strongest on anatomy under clinical imaging, while my current line is trying to make spine-aware structure and motion usable from external sensing.
SpinePose
Moves the field from generic human joints toward spine-centric 2D landmarks in natural RGB images.
SIMSPINE
Extends the problem into multiview 3D supervision, vertebral kinematics, and biomechanics-aware benchmarking.
SpinePose Inference Library
Turns released models into a usable CLI and Python API, so the work is not locked inside papers.
PoseAdapt
Targets continual adaptation and efficient optimization once pose models leave one fixed benchmark and meet changing domains.
Where the field still looks structurally weak
These are the failure modes that show up repeatedly across modalities, even when individual methods look strong in isolation.
Static anatomy is easier to benchmark than motion
CT and MRI already support shared segmentation challenges, but vertebral motion, rotations, and full-body spine biomechanics still lack common public evaluation.
Clinical utility and open reproducibility diverge
Commercial systems like EOS and DIERS are important in practice, yet their maturity does not translate into open datasets, code, or challenge infrastructure.
Biomechanics validation is still exceptional
Many methods optimize localization or segmentation metrics without checking whether the predicted posture or motion remains anatomically plausible.
Cross-modal fusion is underdeveloped
The field still treats radiographs, MRI, surface scans, and RGB motion as separate silos instead of complementary evidence streams.
Deployment quality is lagging behind publication quality
Even when methods are strong on paper, reusable inference interfaces, benchmark harnesses, and failure-analysis tools are usually missing.
Common questions about spine pose estimation and spine motion analysis
This section is meant to make the field easier to navigate for people arriving from search terms such as spine pose estimation, spinal imaging AI, markerless spine tracking, or AI for spine biomechanics.
What is spine pose estimation?
Spine pose estimation refers to predicting spine-related landmarks, curves, segments, or vertebral structures from images or video. Depending on the sensing setup, the target may be 2D spinal keypoints in RGB images, vertebra segmentation in CT or MRI, or posture and curvature estimates from radiographs or surface topography.
What is the difference between spine pose estimation and spine motion analysis?
Spine pose estimation usually refers to recovering structure or posture from a single image or frame, while spine motion analysis focuses on how the spine moves over time. Motion analysis therefore needs temporal consistency, 3D geometry, and ideally biomechanics-aware outputs such as vertebral rotations, curvature changes, or anatomically plausible segment motion.
Which sensing modalities are used for AI in spine biomechanics?
The field spans standing radiographs, biplanar X-ray, CT, MRI, structured light and rasterstereography, RGB and multiview video, and RGB-D or point-cloud sensing for surgical or navigation settings. Each modality exposes different tradeoffs between anatomy, motion, radiation, repeatability, and deployment cost.
Where can I find open datasets and benchmarks for spine motion and spine imaging AI?
Open resources exist, but they are unevenly distributed across the field. VerSe is a strong open CT benchmark, SPIDER represents lumbar MRI segmentation, AASCE covers scoliosis landmarking on radiographs, and my own SpinePose and SIMSPINE pages cover open RGB and biomechanics-aware motion resources.
Representative papers, challenges, and systems
This first version is curated rather than systematic. It is meant to be genuinely useful now, then expanded over time.
Field review
The use of deep learning in medical imaging to improve spine care: A scoping review of current literature and clinical applications
Field-wide review used here to frame validation and deployment gaps.
Radiographs
AASCE 2019 challenge
Representative public benchmark line for scoliosis landmarking and curvature estimation on AP radiographs.
Biplanar radiography
EOS 3D modeling
Reference commercial workflow for weight-bearing 3D spine reconstruction.
CT
VerSe challenge
Key open benchmark for vertebra localization, labeling, and segmentation in CT.
MRI
SPIDER challenge
Representative open challenge for lumbar MRI segmentation and evaluation.
Surface topography
DIERS formetric 4D
Commercial structured-light system for radiation-free posture and scoliosis follow-up.
Surface topography as a screening and follow-up tool
Useful recent review for the strengths and limitations of topography-based monitoring.
RGB-D / surgery
SpineDepth: Fully-automatic, marker-less and real-time spinal anatomy tracking for spinal navigation
Representative RGB-D line for anatomy-aware surgical tracking and registration.
RGB biomechanics
Help expand the map
Missing a public benchmark, a clinically important commercial system, or a spine-motion resource that should be here? Send it over. This page is designed to become a durable reference, not a static announcement.