Monkey Gesture Detector

The story

A browser experiment that reads your face and hands frame by frame with MediaPipe, then swaps meme states in real time when you stick your tongue out, make a fist, or open your mouth. Most of the work went into making it feel solid instead of jittery: confidence thresholds, gesture-history smoothing, and priority rules so states don't flicker when the model hesitates. Runs entirely client-side, no server.

What I learned

1. 01How to run a multi-model CV pipeline in the browser (face + hand) without killing responsiveness.
2. 02How to turn raw landmarks into practical gesture rules for tongue-out, fist, middle finger, mouth-open, and two-hand detection.
3. 03How much temporal smoothing matters: gesture history + no-hand frame thresholds made the UX way more stable.
4. 04How to tune real-world detection with confidence gating, fallback logic, and live debug overlays.
5. 05How to structure experimental code so it is still clean, modular, and easy to iterate on quickly.