Monkey Gesture Detector

The story

A browser experiment that reads your face and hands frame by frame with MediaPipe, then swaps meme states in real time when you stick your tongue out, make a fist, or open your mouth. Most of the work went into making it feel solid instead of jittery: confidence thresholds, gesture-history smoothing, and priority rules so states don't flicker when the model hesitates. Runs entirely client-side, no server.

What I learned

1. 01Running a face model and a hand model at once in the browser is mostly a budgeting problem, because the moment either one falls behind the whole thing stops feeling live.
2. 02Raw landmarks are not gestures. Turning them into a usable tongue-out or fist rule took far more fiddling than wiring the models up did.
3. 03Smoothing over gesture history is what made it stable. Without it the model hesitates for a frame and the meme flickers, which reads as broken even when the detection is right.
4. 04Confidence gating and a debug overlay were worth building early, because you cannot tune something you cannot see failing.
5. 05It is a joke project that I still wrote properly, which is the only reason it was fun to keep changing.