Using Artificial Intelligence to Help Blind People See Facebook

In today’s era of social media, there are over billions of users using social media platforms such as; Facebook, Instagram, Whatsapp, Facebook Messenger, Snapchat…to name a few. Earlier this week, Facebook Messenger announced that it now has 900 million monthly active users on it’s platform. David Marcus who leads the Messenger team at Facebook Inc. said “over 1 billion messages are sent every month between people on Messenger and Businesses/Pages.”

We’re talking about billions of people using these social media platforms, but, How about the blind or visually impaired people! How are they using these platforms? Everyone is entitled to having fun. This is why social media giants, Facebook is making this possible for the blind and the visually impaired to enjoy using their platforms just the same way others enjoy using it. However, not only Facebook, but other tech giants are coming up with different kinds of technology to help the blind and the visually impaired people feel more comfortable about themselves. Japanese automotive manufacturer, Toyota on March 8th, 2016 announced – it’s developing a wearable device to help give the blinds and visually impaired people greater mobility.

Earlier this week, Facebook Inc. announced that it wants to build technology that would help the blind community experience Facebook the same way others are enjoying it. According to the company, every day, people around the world share more than 2 billion photos across Facebook, Instagram, Messenger and WhatsApp.

Automatic Alternative Text from Facebook on Vimeo.

While visual content provides a fun and expressive way for people to communicate online, consuming and creating, it poses challenges for people who are blind or severely visually impaired. With more than 39 million people who are blind, and over 246 million who have a severe visual impairment, many people may feel excluded from the conversation around photos on Facebook.

The social media giants are introducing “automatic alternative text” or “automatic alt text” a new development that generates a description of a photo using advancements in object recognition technology.

People using screen readers on iOS devices will hear a list of items a photo may contain as they swipe past photos on Facebook. They will only hear the name of the person who shared the photo, followed by the term “photo” when they came upon an image in News Feed. For instance, someone (blind) could now hear, “Image may contain three people, smiling, outdoors.”

This is possible because of Facebook’s object recognition technology, which is based on a neural network that has billions of parameters and is trained with millions of examples. Each advancement in object recognition technology means that the Facebook Accessibility team will be able to make technology even more accessible for more people.[related-posts]

The “Automatic Alt Text” will first be launched on iOS screen readers set to English, however, the company also plans to add this functionality for other languages and platforms soon.

“While this technology is still nascent, tapping its current capabilities to describe photos is an important step toward providing our visually impaired community the same benefits and enjoyment that everyone else gets from photos.”