Dense Orientation Recovery via Lightweight Graph Attention for Real-Time Avatar Animation

Muhammad Saif Ullah Khan , Chen-Yu Wang , Tim Prokosch , Michael Lorenz , Bertram Taetz , Didier Stricker

German Research Center for Artificial Intelligence (DFKI), Kaiserslautern, Germany | Rhineland-Palatinate Technical University (RPTU), Kaiserslautern, Germany | International University of Applied Sciences (IU), Erfurt, Germany

Poster Abstract Dataset arXiv Poster

Abstract

Markerless human tracking systems commonly provide 3D joint positions, whereas real-time avatar animation requires joint orientations for skeletal skinning. This position-to-orientation problem is underconstrained, especially because twist around bone axes is not determined by joint locations. We present IK-GAT, a lightweight graph-attention model that recovers fullbody joint orientations from root-centered 3D joints in a single forward pass. Instead of regressing rig-specific local rotations directly, IK-GAT predicts canonical bone-aligned world rotations defined from the rest-pose skeleton. These predictions are analytically and exactly converted to standard parent-relative local rotations using the known kinematic tree, making the output directly usable by animation rigs or SMPL-like body models. The network uses skeletal graph attention, a continuous 6D rotation representation, and geodesic supervision on SO(3). On AMASS, IK-GAT reduces test MPJAE from 9.06◦ for a dense Transformer to 7.43◦ while using 8.6× fewer parameters. On an Unreal Engine mannequin benchmark, it reduces local-space MPJAE from 15.61◦ to 12.83◦ and directly drives production-rig avatars without iterative fitting or runtime retargeting.

Teaser

IK-GAT can create animations (right) from 3D joint positions as input (left) in real time. The positions can come from any upstream tracking system.

Demos

Poster

To be presented at SCA 2026 in Barcelona, Spain.

BibTeX

@inproceedings{10.2312:sca.20261002,
  booktitle = {SCA 2026: ACM SIGGRAPH / Eurographics Symposium on Computer Animation - Posters},
  editor = {Sugimoto, R.},
  title = {{Dense Orientation Recovery via Lightweight Graph Attention for Real-Time Avatar Animation from Markerless Tracking}},
  author = {Khan, M. S. U. and Wang, C.-Y. and Prokosch, T. and Lorenz, M. and Taetz, B. and Stricker, D.},
  year = {2026},
  publisher = {The Eurographics Association},
  ISBN = {},
  DOI = {10.2312/sca.20261002}
}

Maintained by saifkhichi96 on GitHub.