What if blind people could play ping-pong?

When a bumblebee or mosquito approaches one of your ears, you perceive the intensification of the buzzing, its movement from right to left. Now imagine that a table tennis ball produces a comparable sound and sensation as it travels towards your side of the table. Researchers from the University of Sydney and the start-up ARIA Research had this idea, in order to make sport accessible to blind and visually impaired people. They presented their work at the Acoustics conference held in Sydney in December 2023.

There is a disabled sport inspired by table tennis, called showdown, in which participants try to score points by rolling a ball across a table with edges. Here, hearing is a key factor: rolling the ball on the surface allows you to locate it in order to deliver a precise racket strike.

Add sound to a trajectory

But the project envisioned by recent University of Sydney graduate student Phoebe Peng and her colleagues is different: it’s about playing table tennis in its classic version, therefore returning the ball in the air. “The trajectory of the ball in the air makes no noise, unlike its roll on a table. So we have to find a way to create it”explains the student.

So, to add sound to the movement of the ball in space, the researchers used cameras responsible for capturing its location in space at each moment and speakers to broadcast a sound corresponding to its position.

Film only the movement

The cameras used have nothing to do with the traditional ones which film, frame by frame, the entire scene before them. The cameras used by researchers are called event cameras. “That is to say, only variations in light intensity are captured,” says Phoebe Peng. In other words, only the movement of a scene is filmed, the rest is absent from the images.

The captured image then has nothing of a film scene. It is only binary information, a succession of 0s and 1s, which must be processed by an algorithm to reconstruct the image. From these images, we must extract the position of the ball. “Using these cameras rather than the “classic” ones allows us to process the image more quickly”explains Phoebe Peng.

However, only a two-dimensional image is extracted from a camera, i.e. the position of the ball in two dimensions. “That’s why we use two cameras placed in two different locations in the room. By combining their images, we can reconstruct the position of the ball in three dimensions! » explains Phoebe Peng. This step is called triangulation.

“Until now, it is rather on this part of capturing and processing images to extract the position of the ball that we have focused”, admits Phoebe Peng. The rest of their work was to translate this position instantly into a succession of “beeps”, which audibly describe the movement of the ball. “Each speaker – whose position has been calibrated upstream – plays a more or less intense sound depending on their distance from the ball”she says.

Encouraging, but limited results

For now, success in returning the ball is limited. “What we are proposing is still simplistic and not sufficiently precise, it is still difficult to know precisely the position of the ball », admits Phoebe Peng. Their work was a proof of concept; the researchers wanted to know if such a device was feasible and could ultimately help blind and visually impaired people. “However, what we observed by testing our installation blindfolded is that we get used to it quite quickly thanks to our hearing. So that’s very encouraging for the future! »she rejoices.

Subsequently, the researchers wish to take a greater interest in the perception of sound in humans, in order to refine their translation of the position of the ball: is there a need for continuous noise, is it necessary to play on the frequencies. “For example, it seems that the perception of right and left is finer than the perception of up and down with hearing,” notes Phoebe Peng. They also plan to use the table tennis racket as another information vector, notably through vibrations. And of course, to have the installation tested by blind and visually impaired people.

And once these limits have been crossed, their idea could be exported to other ball sports. It is simply enough that cameras and speakers can be placed around the field.

If you liked this article, you will like the following: don’t miss them by subscribing to Numerama on Google News.