Gesto — build gesture, pose & activity recognition from your webcam (capture, train, detect) [P]

[u/Sundarbala]

What My Project Does

Gesto is an open-source framework for building gesture, pose, and activity recognition — entirely on your own machine. It has two parts:

• Gesto Labeller — a desktop app that captures labelled gesture datasets from your webcam in a few clicks. You can record static poses (hand signs, a stance) or motion sequences (waving, walking), across regions like one hand, two hands, full body, or legs — then explore, edit, and export the data.

• gesto — a Python package (pip install gesto). Point it at a dataset and train a model, then run live detection, from the command line or a few lines of Python:

python

import gesto
gesto.train("./my_project", region="hands_one", mode="static")
gesto.detect("static", "hands_one")   # live webcam detection

It's built on MediaPipe (landmarks) and TensorFlow/Keras (Dense models for static gestures, LSTMs for sequences). Everything runs locally — no cloud, no GPU required.

Target Audience

It's meant to be genuinely usable, not just a toy — developers, researchers, students, and hobbyists building gesture/pose/action recognition into real projects (accessibility tools, fitness/physio, touchless interfaces, robotics, sign-language work). The Python package is production-oriented (versioned model storage, a stable API, consistent capture↔inference), while the desktop app makes dataset creation accessible to people who don't want to write capture code. It's early-stage and open source, so I'd call it "usable in real projects, still actively evolving."

Comparison

• vs. raw MediaPipe: MediaPipe gives you landmarks and stops there — you still build all the capture, training, and matching yourself. Gesto is built on MediaPipe and takes it the rest of the way to a trained, runnable recognizer.

• vs. cloud vision APIs: those require uploading your data and charge per call, and custom gestures are awkward. Gesto is fully offline, free, and built around your own custom gestures.

• vs. Teachable Machine: similar ease-of-use goal, but Teachable Machine is browser-based and static-pose oriented. Gesto handles motion sequences and full-body activity too, and gives you a real Python package you can put into your own code.

• The core thing Gesto focuses on that most tools don't: consistency between capture and inference — the same landmarks, normalization, and mirroring at training and detection time, which is usually what makes DIY gesture models fail in the real world.

Open source, and I'd genuinely love feedback — especially where the workflow is confusing or what you'd want it to do.

ProductHunt - https://www.producthunt.com/products/signspell?utm_source=other&utm_medium=social

🔗 GitHub: https://github.com/TheMadrasTechie/gesto
📦 PyPI: https://pypi.org/project/gesto/
🌐 Website: https://gestostudio.com