Google launched an open source project called Gameface. The goal: to be able to use a computer cursor without hands, but using facial movements. The interest is mainly to help people with disabilities of their members so that they can use a computer with the least possible constraints.
Talking about artificial intelligence or the latest Android feature is fine, but if it’s just that, it doesn’t necessarily move society forward, especially when it comes to accessibility. Google presented at its I/O 2023 conference the Gameface project, announced as “ a new open source hands-free gaming mouse “.
A mouse that isn’t really a mouse
In fact, Gameface does not come in the form of a gaming mouse, but of software. Indeed, the latter follows the movements of the user’s head by only needing a webcam. The upside is that there’s no need for expensive hardware, like a disability-specific mouse or a set of motion-tracking sensors.
YouTube linkSubscribe to Frandroid
The movements and gestures performed are recognized by machine learning models connected by a tool called MediaPipe, which allow the cursor to be moved or clicks to be made. The software places 486 points on its user’s face and creates a mesh. For Lance Carr, player of World of Warcraft with muscular dystrophy who helped create Gameface, ” it’s so precise that I can even write in cursive “.
In order to adapt to all users, software parameters make it possible to adjust the triggering of an action of the cursor according to the intensity of their gesture. Gameface, as its name suggests, is especially designed for gamers, the practice of video games may require a certain reactivity and a certain precision, this represents a real need today.
The Gameface tool is still under development, but its source code is available on Github.
Want to join a community of enthusiasts? Our Discord welcomes you, it’s a place of mutual aid and passion around tech.