Project Gameface — Google's Open-Source Facial-Gesture Game Controller Co-Designed With Lance Carr

Project Gameface is a free, open-source application that lets people control a computer cursor using head movement and facial gestures captured by an ordinary webcam, with no additional hardware required.

Google built the tool with quadriplegic gamer and streamer Lance Carr, who lives with muscular dystrophy and had previously relied on an expensive dedicated head-tracking mouse — equipment he lost entirely in a house fire. Working with Carr as a design partner, Google's team used its MediaPipe Face Landmarker model, which detects 52 distinct facial-gesture 'blendshapes' (such as raising an eyebrow or opening the mouth), and mapped a configurable subset of these gestures to mouse clicks, drags and cursor movement.

Google previewed Project Gameface at I/O 2023 for Windows and open-sourced the code on GitHub the same year, inviting developers to extend it. At I/O 2024, Google announced a port to Android and open-sourced the additional code needed for developers to embed hands-free, camera-based control directly into Android games and apps.

As an evidence caveat: Google has not published usage or adoption figures, controlled outcome studies, or independent efficacy research for Project Gameface, so its impact is documented mainly through the origin case study, developer demos and press coverage rather than measured results.

Read the full analysis: https://www.engadget.com/googles-project-gameface-hands-free-mouse-launches-on-android-123029158.html