Skip to footer

How Facebook is using artificial intelligence to help the blind 'see'

For blind and visually impaired users, Facebook's timeline can be a bit of a mystery, but new software can help them feel less out of the loop. 

|
Karly Domb Sadof/AP/File
This photo shows the Facebook app icon on an iPhone in New York. A feature rolling out Tuesday on Facebook's iPhone app interprets what's in a photo using a form of artificial intelligence to recognize faces and objects for blind and visually impaired people as they scroll through the pictures posted on the world's largest online social network

Facebook has increasingly become a visual medium, allowing users to share endless streams of photos with their friends. But imagine not being able to participate in the visual experience. That is the reality for more than 39 million people who are blind and more than 246 million others who have severe visual impairment.

But now, Facebook just got much more accessible.

On Monday, the social media giant launched the new "automatic alternative text" feature that uses object recognition technology to provide a basic spoken description of a photo's contents to people who are using screen readers. Previously, the readers automated voice could describe only the name of the person who shared the photo, the text that accompanied the photo, and the word “photo.” That meant that when users posted photos without any text, their blind or visually impaired friends were not be able to find out anything about the images.

Of course, outsourcing captions to an artificial intelligence isn't without potential for snafus. Last year Google’s image recognition technology caused a stir when it identified a black couple as gorillas. At least for now, the automated captions will be relatively simple, to minimize potential for problems. A photo taken outdoors with three people smiling, for instance, may omit other extra details, such as what they are holding.

“We’re ... making it possible for people to feel totally included in the social interaction and be able to feel part of it without having to feel awkward, without having to be annoying to all of your friends, being like, ‘What’s so funny in this photo?’ Nobody wants to do that,” Facebook accessibility specialist Matt King told VentureBeat.

For now the feature isn’t available to everyone, but only people using Facebook's iPhone app, and the built-in screen reader must be turned on for for the feature to work. It will be available in English for now, and later in other languages. The company’s representatives also say the feature will come to other platforms in the near future.

This report contains material from the Associated Press.

You've read  of  free articles. Subscribe to continue.
Real news can be honest, hopeful, credible, constructive.
What is the Monitor difference? Tackling the tough headlines – with humanity. Listening to sources – with respect. Seeing the story that others are missing by reporting what so often gets overlooked: the values that connect us. That’s Monitor reporting – news that changes how you see the world.
editor@csmonitor.com
Subscribe
Mark Sappenfield
Editor

Dear Reader,

About a year ago, I happened upon this statement about the Monitor in the Harvard Business Review – under the charming heading of “do things that don’t interest you”:

“Many things that end up” being meaningful, writes social scientist Joseph Grenny, “have come from conference workshops, articles, or online videos that began as a chore and ended with an insight. My work in Kenya, for example, was heavily influenced by a Christian Science Monitor article I had forced myself to read 10 years earlier. Sometimes, we call things ‘boring’ simply because they lie outside the box we are currently in.”

If you were to come up with a punchline to a joke about the Monitor, that would probably be it. We’re seen as being global, fair, insightful, and perhaps a bit too earnest. We’re the bran muffin of journalism.

But you know what? We change lives. And I’m going to argue that we change lives precisely because we force open that too-small box that most human beings think they live in.

The Monitor is a peculiar little publication that’s hard for the world to figure out. We’re run by a church, but we’re not only for church members and we’re not about converting people. We’re known as being fair even as the world becomes as polarized as at any time since the newspaper’s founding in 1908.

We have a mission beyond circulation, we want to bridge divides. We’re about kicking down the door of thought everywhere and saying, “You are bigger and more capable than you realize. And we can prove it.”

If you’re looking for bran muffin journalism, you can subscribe to the Monitor for $15. You’ll get the Monitor Weekly magazine, the Monitor Daily email, and unlimited access to CSMonitor.com.

Subscribe to insightful journalism

QR Code to How Facebook is using artificial intelligence to help the blind 'see'
Read this article in
https://www.csmonitor.com/Technology/2016/0405/How-Facebook-is-using-artificial-intelligence-to-help-the-blind-see
QR Code to Subscription page
Start your subscription today
https://www.csmonitor.com/subscribe