Hand gestures from landmarks
A hand model gives you 21 points. Four geometry rules turn them into "palm", "fist", "victory" or "thumbs up", with no training and no camera.
HANDS holds the 21 points of four photos of ours (open palm, fist, victory V and thumbs up), as MediaPipe returned them. Each point is [x, y] between 0 and 1. Write four functions:
fingerExtended(lm, tip, pip): a finger is extended when its tip is farther from the wrist (lm[0]) than its second joint. Compare distances with dist.
thumbExtended(lm): the thumb is out when the distance from its tip (lm[4]) to the wrist is more than 1.1 times the palm size, measured from the wrist to lm[9].
fingersUp(lm): [thumb, index, middle, ring, pinky], five booleans, using FINGERS for the four fingers.
classify(lm): "palma" with all five out, "puño" with none, "victoria" with only index and middle, "pulgar" with only the thumb; any other combination, "otro". The preview draws the skeleton and your label.
Challenges 0/5
- An extended finger and a folded one
- The thumb is out in "pulgar" and "palma", in on "puño"
- Five booleans per hand: palm and victory
- All four photos get their name
- What has no name is "otro"
function fingerExtended(lm, tip, pip) {
// true when the tip is farther from the wrist (lm[0]) than the second joint
return false;
}
function thumbExtended(lm) {
// the tip (lm[4]) is more than 1.1 palm sizes (wrist to lm[9]) from the wrist
return false;
}
function fingersUp(lm) {
// [thumb, index, middle, ring, pinky]
return [false, false, false, false, false];
}
function classify(lm) {
// 'palma', 'puño', 'victoria', 'pulgar' or 'otro'
return 'otro';
}
console.log(Object.keys(HANDS).map((k) => k + ': ' + classify(HANDS[k].landmarks)).join(', '));The idea
A hand model does not tell you "this is a thumbs up". It gives you 21 points with coordinates, and you decide the gesture with rules you can read. That split is useful: the model (heavy, trained by others) does the perception; the business logic (light, yours) decides, and it is tested without a camera, on a file of points.
The rules use ratios, not absolute distances: a hand close to the camera is twice the size of a far one, but the index fingertip is still farther from the wrist than its second joint. That is why the thumb threshold is written in "palm sizes".
What we measured: MediaPipe Hand Landmarker (float16 model, 7.8 MB, Apache-2.0) took between 16 and 38 ms per photo on the DGX Spark CPU on 2026-10-18, and got the hand (left or right) right on all four images, which we generated with FLUX.1-schnell because we have no hand photos we are allowed to publish.
This site blocks the camera and the microphone on every page; that is why the exercise starts from precomputed points. If you take this to production, keep it on the device: a video of hands is personal data, and it has no need to leave the screen.
Run it on your machine
With Python and NumPy, on this exercise's JSON of points (CC BY 4.0):
curl -O https://vorluxai.com/data/labs/gestos-landmarks.json
python3 -c "
import json, numpy as np
hands = json.load(open('gestos-landmarks.json'))
for name, h in hands.items():
if name == 'about': continue
lm = np.array(h['landmarks'])[:, :2]
d = lambda i, j: np.linalg.norm(lm[i] - lm[j])
up = [bool(d(4, 0) > 1.1 * d(0, 9))] + [bool(d(t, 0) > d(p, 0)) for t, p in [(8, 6), (12, 10), (16, 14), (20, 18)]]
print(name, up)
"Go deeper: the MediaPipe reference →
This in production, with your data? Let's talk for 15 minutes →