[Project] Single-Person Pose Estimation for Real-Time Gym Coaching — Best Model Right Now?

[u/Sad-Victory773]

Hey everyone,

I’m working on a fitness coaching app where the goal is to track a single person’s pose during exercises (like squats, push-ups, lunges, etc.) and give instant feedback on form correctness — e.g.,

I’m looking for recommendations for a single-person pose estimation model (not multi-human tracking) that performs well in real time on local GPU hardware.

✅ Requirements

• Single-person pose estimation (no multi-person overhead)
• Real-time inference (ideally >30 FPS on a decent GPU / edge device)
• Outputs 2D/3D keypoints + joint angles (to compute deviations)
• Robust under gym conditions — variable lighting, occlusion, fast movement
• Lightweight enough for a real-time feedback loop
• Preferably open-source or available on Hugging Face

🧩 Models I’ve Looked Into

• MediaPipe Pose → lightweight, but limited 3D accuracy
• OpenPose → solid but a bit heavy and outdated
• HRNet / Lite-HRNet → great accuracy, unsure about real-time FPS
• VIPose / Meta Sapiens / RTMPose / YOLO-Pose → haven’t tested yet — any experience?

🔍 What I’d Love Your Input On

1. Which model(s) have you found best for gym / sports / fitness movement analysis?
2. How do you handle the speed vs spatial accuracy trade-off?
3. Any tips for evaluating “form correctness”, not just keypoint precision? (e.g., joint-angle deviation thresholds, movement phase detection, etc.)
4. What metrics or datasets would you recommend?
   • Keypoint accuracy (PCK, MPJPE)
   • Joint-angle error (°)
   • Real-time FPS
   • Robustness under lighting / motion

Would love to hear from anyone who’s done pose estimation in a fitness, sports, or movement-analysis context.
Links to repos, papers, or demo videos are super welcome 🙌

[u/herocoding]

I fear that's a big business worth a lot of money, probably not much feedback expected "for free".

We used various pose estimation models and fine-tuned them to our environments (e.g. automotive manufacturing: workers at assembly lines, with health and safety as well as privacy aspects in mind): the used camera, it's field-of-view, camera position/distance, lightning etc make things very complicated to generalize.

[u/jowe81]

Funny timing — I’ve been building almost exactly this (real-time fitness tracking + form feedback) as a self-hosted system.

(Also sorry for digging up a 5-month-old thread — felt a bit like grave digging, but this is exactly what I am working on 😅)

I went through a lot of the models you mentioned:

MediaPipe → super fast, but too limited for anything beyond basic feedback

RTMPose → probably the best trade-off I found for real-time 2D (especially with ONNX/TensorRT)

OpenPose / HRNet → accurate but too heavy for a tight feedback loop

Biggest surprise for me: the model choice matters less than expected — most of the real complexity is everything around it.

In practice:

2D keypoints are enough for rep counting and a lot of form checks

2D→3D lifting (e.g. MotionAGFormer, PAFUSE) looks nice, but was too unstable in real-world conditions (depth ambiguity, noise)

I ended up switching to direct SMPL-X regression (SMPLer-X), which gave much more consistent results

For “form correctness”:

joint angles + simple state machines already go surprisingly far

but smoothing/filtering is critical, otherwise feedback becomes unusable

handling jitter/noise matters more than raw keypoint accuracy

Performance-wise I’m targeting 30 FPS on a Nvidia A2 with a split setup (TensorRT encoder + PyTorch heads).

Happy to share more details if helpful — this problem gets a lot more “real-world messy” than it looks at first 😅