New App Helps Blind People Know the World Around Them

One of the best uses of technology is to make life easier.

A new product from Microsoft Corporation can do just that for people who have trouble seeing. They may be blind or visually impaired 1.

The product, an app, is currently 2 available for free for Apple’s iPhone and iPad. The app is called Seeing AI. “AI” is short for artificial intelligence, a term for computers with an ability to think and learn like human beings.

Seeing AI uses both a camera and artificial intelligence to identify places, objects and people. The app then announces what the camera sees so users will know what is in front of them.

People can also use Seeing AI to learn words in English. With the app open, you can point your iPhone or iPad at any object and it will say what the object is.

With Seeing AI, users can hear a description of not just objects, but other people. It can even tell you about their emotions. It will not just say that someone is smiling. The app will say that the person is happy. Or surprised. Or angry.

People can use the app when going to a store or supermarket. It can read product bar codes so users will know whether a can is filled with fruit or dog food. Microsoft says Seeing AI will soon be able to identify banknotes so people know whether they are holding a bill that is worth $100 or $1.

At restaurants, diners can use the new app to hear a list of drinks and other menu offerings. It can recognize both food choices and prices.

People can use the app to read signs and get directions, although Microsoft warns against using it for navigation purposes.

One of the more interesting tools of Seeing AI is its ability to read documents. The app can read documents aloud and even help a user position them on the center of the electronic device.

The app looks for the edges of the document so it knows that it is reading all of the wording.

Seeing AI can also identify images in other apps. When you are using another app, hit the Share icon 3 and choose "Recognize with Seeing AI." The app can identify what is in the image, as if it were right in front of you.

How to use Seeing AI

The Seeing AI app has four Channels: Short Text, Document, Product, and Person. After you download it, the app will show video directions for each channel.

With the Document and Person channels, the app takes a picture of the object or person, then examines the image and announces what it is.

The Short Text channel can be used to read things such as signs that have a few words that will fit on a screen. Hold the camera over the text and the app will start reading the wording out loud.

The Documents channel is for longer, printed pages. The app will tell the user how to center the camera so it can read all the text.

Users should put the document on top of something with a different color, such as a white sheet of paper on a dark surface. This helps the app identify the edges of the paper.

The Product channel reads bar codes to help users recognize what the product is. The app may also provide more information about the product, such as cooking directions.

Seeing AI will guide users with sound – short beeps -- to let them know when the camera is getting closer to, and then scanning 4 the bar code.

When the app scans 5 the bar code, it will read the name of the product. If more information is available about the product, a More Information button will appear in the program.

The Person channel helps users identify people, who they are and what they are doing. The app can identify more than one person in a photograph.

Users can teach the app names of people so it can announce who they are by name.

The app will help guide users while taking a photo of people so that their faces are in the center of the image.

This channel also helps identify the emotions of people. The app will tell how the person is feeling, based on their faces. The app will also predict the person's age and sex.

Experimental channels

The app also has an experimental channel called Scene. With that channel, the app guesses what is happening in the photo.

Microsoft also says a Currency channel is coming soon to the app. That area will help users the value of banknotes.

What else you need to know

Seeing AI is a new app and will not be correct 100% of the time. For example, just because someone is smiling does not necessarily mean they are happy.

The app recognizes text, but not handwriting yet. That feature may be available in the future.

Also, Seeing AI needs to be connected to the Internet to work.

For more information about the app, including the informational videos, go to Microsoft's Seeing AI website.

Availability

As we said earlier, Seeing AI is a free app and available for Apple’s iPhone and iPad in the iTunes App Store. Microsoft has not yet said whether it will be available on Android in the future.

The app is currently available in Canada, Hong Kong, India, New Zealand, Singapore and the United States.

I’m Ashley Thompson.

And I'm John Russell.

Words in This Story

visually impaired - adj. eyesight that cannot be corrected to a normal level.

artificial intelligence - n. the ability of a computer to think and learn like a human

bar code - n. a group of thick and thin lines that is placed on a product so that a computer can get the price of the product and other information about it

menu - n. a list of the foods that may be ordered at a restaurant

navigation - n. the act, activity, or process of finding the way to get to a place when you are traveling in a ship, airplane, car, etc.?

icon - n. a small picture on a computer screen that represents a program or function

screen - n. the usually flat part of a television or computer monitor that shows the images or text : the part of a television or computer that you look at when you are using it

frisbee 6 - n. a plastic disc that you throw to someone who tries to catch it as part of a game