Published: Thu, July 13, 2017
Research | By Jennifer Evans

Microsoft's new Seeing AI app describes the world for the blind

Microsoft's new Seeing AI app describes the world for the blind

They can point it at a product and the app will tell them what that is.

It can, for example, recognize friends, guess the emotions of people from their facial expressions, read text, identify bank notes, identify products from their bar-codes, and even recognize images in apps like Twitter ...

In a video demonstrating the app's abilities, the app described a young woman with glasses by saying: '28-year-old female, wearing glasses, looking happy'. Otherwise, all you have to do to start identifying things is select from one of five different categories (the app calls them "channels") to help the app understand what type of object it needs to identify.

The app is now available in just six countries: The US, Canada, India, Hong Kong, New Zealand and Singapore - however, Microsoft says that the app will be available in other countries soon. If you'd like to give it a try yourself, you can download the iOS app here; there's no word on when an Android app may be available. Similarly, when reading documents to users, Seeing AI will provide spoken hints to ensure that all corners of the paper are properly captured.

The app relies on the iPhone camera, backed by Microsoft's machine learning and image recognition algorithms. The app emits a series of beeps to indicate how close the viewfinder is to the barcode, helping users align the camera properly.

The app combines and improves upon the functionality of several existing visual impaired assistant apps.

The app will soon have a feature that identifies currency bills.

The Seeing AI app is very simple to use. The first experimental feature is scene descriptions.

Another great thing about the Seeing AI app is that most of its basic functions are made on the device itself.

Like this: