Seeing AI

Most apps with “AI” in the name are selling a demo. Seeing AI isn’t selling anything. It has no price, no ads and no subscription waiting behind a paywall, and it’s one of the more significant pieces of software Microsoft has put on a phone.

The premise is simple. Point your phone’s camera at something and the app tells you out loud what it is. For blind and low-vision users, that covers a lot of daily friction: the mail on the counter, the can in the pantry, the bill in a wallet, the person who just walked into the room. Microsoft built the app with that community rather than simply for it, and it still calls Seeing AI an ongoing research project. That label explains two things. It’s why the app is willing to ship ambitious experiments like spatial-audio object mapping, and it’s why a few rough edges have lasted longer than they should.

Seeing AI launched on iPhone in 2017, co-led by Microsoft engineer Saqib Shaikh, who is blind. The Android version only arrived in late 2023, a long wait that gave Google’s Lookout years alone on that platform. The category is crowded now. Be My Eyes pairs its volunteer network with a generative describer. Envision sells subscriptions and smart-glasses hardware. Apple has built detection features into its own Magnifier app. Seeing AI competes on breadth. Instead of one general-purpose smart camera, it offers a set of channels, each tuned for one job.

The store rating is 4.3/5 from 631 ratings on the US App Store. That number reflects how many people need an app like this, not how much they like it. In user communities, Seeing AI regularly comes up as one of the most useful apps on the phone.

• Short Text (Read): Speaks text the moment it enters the camera frame, with no shutter press. It’s the app’s signature feature.
• Document Mode: Audio cues help you frame a printed page, the app keeps the original formatting, and you can then ask questions about the document instead of listening to all of it.
• Describe: Takes a photo and returns a detailed description. It supports follow-up questions and lets you explore by touch, moving a finger across the screen to hear where objects sit.
• Products: Beeps guide you toward a barcode or Accessible QR code, then it reads out the product name and package details.
• People: Recognizes saved friends and coworkers and estimates age, gender and facial expression.
• Currency, Color and Light: Identifies banknotes and colors. Light plays a tone whose pitch follows the brightness around you.
• Find My Things: Learns your personal objects (keys, a cane, a specific bag) and guides you back to them.
• World (LiDAR iPhones only): Announces surrounding objects in spatial audio, so each one sounds like it’s coming from where it actually is.
• Share-sheet descriptions: Describes photos and videos sent from Mail, WhatsApp or the camera roll.

The Channel Model

Seeing AI’s biggest design decision is also its most debatable one: each task gets its own channel. You choose Short Text, Products or Currency from a carousel before you point the camera. Critics of this approach say a modern multimodal model should be able to work out what you’re looking at without being told. That’s partly fair. But the separation is a strength on its own terms. A channel that knows it’s looking for a barcode can give you guidance beeps. A channel that knows it’s looking at a banknote doesn’t waste time describing the table under it. Declaring what you want up front makes the output predictable, and for someone who can’t visually check the result, predictability matters more than cleverness.

The cost is mode friction. Switching channels is one more gesture, and moving from “read this label” to “what is this object” happens more often in daily life than the design assumes. Seasoned users can build muscle memory for it. New users will fumble at first.

Reading: Where It Earns Its Keep

Short Text is the reason many people install Seeing AI and the reason they keep it. It needs no capture step: text is spoken as soon as the camera sees it, so scanning an envelope stack or a row of spice jars becomes a quick sweep instead of a series of photos. This kind of instant feedback is hard to build well, and Microsoft has had it since 2017.

Document mode is more deliberate and more impressive. The framing cues tell you when page edges are in view, which addresses the most common failure in blind OCR: a clipped, skewed capture that produces nonsense. Keeping the original formatting means headings and structure survive. The question-and-answer layer changes the job completely. Instead of listening to a two-page utility bill from the top, you can ask what the amount due is. That’s generative AI solving a real accessibility problem, not being bolted on for a press release.

The weak spot is digital displays. Appliance screens, thermostats and segmented LCD readouts are where users report the most inconsistency. Glare, refresh flicker and odd seven-segment fonts all cause trouble, and those are exactly the surfaces that are hardest to read without sight. A microwave’s control panel is a daily accessibility obstacle, and Seeing AI doesn’t reliably solve it yet.

Describe and the Generative Turn

Describe has come a long way from the one-line captions of earlier versions. Descriptions are now detailed, and follow-up questions let you narrow in on what matters, such as the color of a shirt or whether a sign says “open.” The finger-exploration feature is the sharpest idea in the app: you drag across the screen and hear which object is under your fingertip, which turns a flat description into something with a sense of layout. Few competitors offer anything like it.

This channel also has the usual generative-AI risk. A confident description can be wrong, and a user who can’t see the image has no easy way to catch the mistake. Seeing AI’s detail doesn’t change that basic trust problem, and users should treat Describe as very well-informed rather than infallible.

Products and the Grocery Aisle

Products is quietly the most practical channel. The guidance beeps get faster as a barcode comes into view, which solves the “where on this box is the barcode?” problem without asking you to rotate a cereal box blindly. Support for Accessible QR codes matters too: when packaging carries the code, the app can read out richer details. The most common feature request here is expiration dates. That tells you exactly what’s missing. Knowing what a product is only answers half the question in a kitchen.

People, and the Ethics Question

The People channel recognizes saved contacts, which is useful when someone walks into a meeting room without saying who they are. It also estimates age, gender and expression. Those are exactly the inferences Microsoft retired from its public Azure Face API in 2022 on responsible-AI grounds, while carving out an exception for accessibility uses like this one. The exception is defensible: expression cues are social information that sighted people get for free. But the estimates are guesses, gender inference is often wrong, and users should treat this channel as a hint, not a verdict.

Find My Things and World: The Research Lab Shows

Find My Things addresses a real, constant problem: misplaced personal items. Training it on your own objects makes it personal in a way general object detection can’t be. World is the most ambitious feature. On LiDAR iPhones, it places spoken object announcements in spatial audio, so the chair actually sounds like it’s to your left. It’s an impressive concept and potentially transformative for those with the hardware. It’s also clearly a research feature, restricted to a hardware tier most people don’t own.

The Friction Points

The most persistent complaint has nothing to do with AI: settings that reset between launches. Being forced to re-apply your preferred speech rate or channel setup every time you open the app is a basic quality failure, and it hits hardest for users who depend on consistent behavior. For an app this strong in its hard parts, it’s a frustrating failure in an easy one.

Judging Seeing AI’s visual design by normal standards misses the point. Its real interface is audio: sound cues, guidance beeps, the Light channel’s rising pitch, and spoken output. On those terms the design is well judged. The sounds are clear and distinct, and the feedback loops are built to support live hand-eye (or hand-ear) correction.

The visible UI is plain and practical, with large targets, high contrast and a channel carousel that works properly with the phone’s built-in screen reader. It isn’t attractive and doesn’t need to be. It works for low-vision users who still use some sight, which is the right priority.

On performance, Short Text is designed for instant response, which is why it can feel like magic. Describe and document questions take noticeably longer, since the heavier processing appears to happen in the cloud, so a weak connection will slow them down. Running the camera and speech engine continuously also draws heavily on battery, a design consideration for an app someone might keep open through a whole grocery run.

iOS is Seeing AI’s original platform and it shows. The app has had years of tuning against VoiceOver, and it’s the only version with World, since that channel needs the LiDAR sensor on higher-end iPhones. The iPhone’s consistent camera hardware also makes results more predictable across devices. Share-sheet support for Mail, WhatsApp and the camera roll fits naturally into iOS habits.

Android is the newer port, and its slightly lower store rating roughly matches the experience. It works properly with TalkBack and has the core channels, but Android’s hardware variety is a real problem for a camera-first app. Autofocus speed, low-light noise and lens quality differ widely between phones, and OCR accuracy varies with them. Android users also have Google’s own Lookout one download away. Seeing AI’s argument there is its more specialized channels and the document Q&A, not being the only option.

Seeing AI does what a lot of AI marketing only promises. It takes machine perception and makes it serve a specific group of people with specific problems, and it doesn’t charge them for it. Instant text reading, document Q&A and finger exploration are thoughtful features, clearly shaped by the people who use the app every day. The criticisms are real: settings that don’t persist, unreliable reading of digital displays, and a channel model that asks new users for more than it should. For blind and low-vision users, the strengths outweigh those flaws, and this is an easy recommendation for that audience. Others can admire it from a distance.