Webcam Posture Detector Game
Train your own webcam posture classifier with TensorFlow.js transfer learning, then steer a maze game by leaning. Runs in your browser, with no upload.
Still loading. If the tool doesn’t appear, reload this page.
How to play Webcam Posture Detector Game
- Keyboard
- Arrow keys or WASD steer the game; hold Space or Enter on an Add Sample button to record samples.
- Touch
- Hold an Add Sample button with a finger while you hold the posture; use a front camera.
- Mouse
- Hold an Add Sample button to record; use the round Train, Play and Reset buttons.
Tips
- Record 20 to 50 samples per posture and make the postures clearly different.
- Keep the camera still and your head and shoulders inside the square preview.
About this tool
Webcam Posture Detector Game
Teach your webcam four postures with TensorFlow.js transfer learning, then steer a maze game by leaning. Train a real neural network in your browser; nothing is uploaded.
Webcam Posture Detector Game turns your camera into a game controller using a neural network that you train yourself. A frozen MobileNet model converts each camera picture into a 7 by 7 by 256 feature map, and a small dense network on top of it is trained in your browser with TensorFlow.js on the examples you record: sit tall for up, lean left, lean right and lean forward for down. You choose the learning rate, batch size, epochs and hidden units, watch the loss fall during training, and see live probabilities for each posture. Press Play and your postures steer a chomper through a maze of dots while four ghosts chase it. The camera frames and your samples never leave your device, and no account, API key or server is involved.
How to use Webcam Posture Detector Game
- 1
Press Start camera and allow access, then stand where your head and shoulders are in the square preview.
- 2
Hold each Add Sample button for a second or two while you hold its posture (sit tall, lean left, lean right, lean forward) until each shows about 20 or more examples.
- 3
Choose the learning rate, batch size, epochs and hidden units if you want, then press Train Model and watch the loss fall.
- 4
Press Play and move to steer the chomper around the maze; eat every dot and avoid the ghosts. Press Pause or Reset any time.
Why use Webcam Posture Detector Game
Genuine transfer learning
MobileNet is the frozen feature extractor and only the small dense head is trained, on your samples, in your browser. Every prediction on screen comes from that trained network.
Full control of the training
Pick the learning rate, batch size as a fraction of your samples, number of epochs and hidden units, then watch the loss update after every batch and stop early if you like.
Private by design
Frames are processed on your device by TensorFlow.js. The model weights are downloaded once, and your samples and the trained model are discarded when you leave the page.
Works on phones and laptops
The maze scales to fit your screen, the controls stack at full size on narrow screens, the Add Sample buttons work with a finger or a mouse, and the keyboard arrow keys can still steer the game.
Common use cases
- You want to learn how transfer learning works by training a real classifier on your own webcam pictures and seeing the loss, the probabilities and the result on one screen.
- You are teaching machine learning and need a hands-on demo where students record examples, change hyperparameters and watch the model's behaviour change.
- You want a playful way to remind yourself to sit up straight, or a hands-free game controlled by leaning instead of a keyboard.
Frequently asked questions
Is the model really trained in my browser?
Yes. A frozen MobileNet extracts features from each camera picture, and a small dense neural network is trained on those features with TensorFlow.js using the samples you record. The loss and the live probabilities come from that network.
How many samples do I need?
At least 5 samples for each of at least 2 controls, but 20 to 50 per posture works much better. Hold the Add Sample button for a few seconds and move slightly between samples so the model sees some variety.
Why does it confuse two of my postures?
The postures may look too similar to the camera. Make them clearly different, keep the camera still, add more samples, and try more epochs or a higher learning rate before pressing Train again.
What do the training settings do?
Learning rate is how big each update step is, batch size is the fraction of your samples used per step, epochs is how many passes are made over the data, and hidden units is the width of the middle layer of the classifier.
Does any video leave my computer?
No. Camera frames are processed in your browser tab and are never uploaded. Only the MobileNet weights are downloaded, from this site, once.
Can I play without a camera?
Yes. Press Play and use the arrow keys or WASD to steer. Training and posture control need a camera.
The camera does not start. What should I check?
Allow camera access for this site in the address bar, close other apps that use the camera, and make sure the page is secure (https). The error message above the controls explains which of these applies.
Do I have to retrain if I add more samples?
Yes. The tool tells you when samples changed after training. Press Train Model again to include them; the old model keeps working until then.
Is it a copy of the Google demo?
It is an independent tool inspired by the TensorFlow.js Webcam Pac-Man example, with its own maze game. It is not affiliated with Google or with the owners of Pac-Man.
Does it work on a phone?
Yes, with a front camera. Prop the phone up, add samples for each posture by holding the buttons, train, and play.