Microsoft trains AI for the visually impaired

Microsoft is now increasingly cooperating with the company Be My Eyes, whose apps and services are intended to make it easier for visually impaired or blind people to perceive their surroundings. To do this, Microsoft’s AI models are fed video data provided by the app’s users. Be My Eyes

Auxiliary app passes on data for AI training

One of the areas in which artificial intelligence (AI) can make people’s lives better, rather than costing jobs or leading to the construction of new nuclear reactors, is physical limitations. The company Be My Eyes tries to help blind or severely visually impaired people with its apps.

The apps used for this run on mobile devices and are intended to turn the cameras of smartphones or tablets into the “eyes” of their visually impaired users. AI is now widely used to evaluate the captured camera images and then provide automatically generated descriptions of what the camera “sees”, which are read out to the user.

Microsoft wants to adapt AI for the visually impaired

Users of the “Be My Eyes” apps can capture what is happening around them or what is around them. In the future, the data collected when using the apps will also be used to train Microsoft’s AI models. In this way, better adaptation for visually impaired users should be possible because the AI can respond better to their needs, it says Blog entry from Microsoft. Microsoft promises that the anonymized video data can improve the accuracy and precision of “scene understanding” and textual description for users.

The company promises that the AI technology will benefit the community of blind and visually impaired people. The data would also help AI gain a “broader understanding of the diversity of language” and expand the number of scenarios in which this technology can provide assistance, Microsoft said. Among other things, the accuracy of the language used by the AI should be increased for special, non-standard areas, especially when it comes to “mainstream” language platforms such as Azure Speech, it is also said.

Through cooperation with Be My Eyes, AI systems for people with disabilities can be improved by incorporating more diverse data and perspectives into the development process. On the Be My Eyes side, the data used for AI training at Microsoft should be anonymized in order to protect the privacy of users. Microsoft asserted that they generally want to rely on transparent communication when it comes to data. Originally from Denmark, Be My Eyes is distributed worldwide and is available free to users in 170 countries and more than 180 languages.