Pose Fusion Project

sellProject sellDFKI sellAugmented Vision sellDeep Learning sellHuman Pose Estimation sellKnowledge Distillation

Unified 2D Human Pose Estimation (Project)

Dhavalkumar Bharatkumar Limbachiya, Muhammad Saif Ullah Khan
Project Report Code

Abstract

The key objective of human pose estimation is to detect the set of keypoints on the human body for understanding the orientation and position of humans. It has many downstream applications in fitness, autonomous driving, surveillance, etc. The research on human pose estimation has progressed and produced numerous state-of-the-art model architectures with the availability of multiple pose estimation datasets. However, each neural network architecture is trained on a particular dataset, which performs adversely when tried on different datasets. As a result, a network that performs well on the majority of datasets is uncommon. In this work, we use the concept of knowledge distillation to train a network architecture capable of efficiently estimating poses across datasets.

Topic

There are many datasets for 2D human pose estimation, each with its unique set of annotated body joints. Typically, a network is trained for each dataset separately. This approach often results in networks that excel on their specific dataset but struggle when tested on others. In this project, we will use a group of pre-trained teachers to help create a single, adaptable student network capable of efficiently estimating poses across datasets.

Tasks

  1. Implementing a top-down pose estimation network.
  2. Running inference on pre-trained models from MMPose.
  3. Training the network using knowledge distillation with multiple teachers and ground-truth datasets.

Expected Skills

  1. Python and PyTorch
  2. Deep learning basics
  3. (Prefered) Familiarity with MMPose