Kristy Viers demonstrates the iPhone Magnifier combined with VoiceOver to describe the user's surroundings. She shows how the built-in feature reports scenes, faces and lighting.
Why Hacker News points people here: HN recommends it to show what on-device accessibility features can do and how blind users actually use an iPhone.
“It describes the scene, tells you how many faces are visible, whether or not the image is focused, lighting conditions.”HN comment, 2021
“I found this demonstration of camera-to-speech in the iPhone pretty awesome”
Demo of camera-to-speech on iPhone, deepening accessibility empathy.
Full comment
Thanks for the link, super interesting to watch. Over the years as a programmer I've picked up knowledge about web accessibility from a mechanical, standards/implementation focused point of view, but this video is helping me realize that my understanding has been lacking a level of depth and empathy for the real people using these tools. Looking through some of her other videos I found this demonstration of camera-to-speech in the iPhone pretty awesome–I had no idea that this was a feature: this video
RecommendsmimischiMay 2021
“The YouTube channel by Kristy Viers has been helpful in understanding how they use an iPhone and other devices.”
VoiceOver in the Camera app, from a helpful channel.
Full comment
The YouTube channel by Kristy Viers has been helpful in understanding how they use an iPhone and other devices. Here’s a video on VoiceOver in the Camera app: this video
RecommendsaasasdAug 2022
“she uses the ‘magnifier’ app to tell her what is there around her”
Blind woman demonstrates the magnifier app describing surroundings.
Full comment
I'll one-up this a bit: a blind woman called Kristy Viers demonstrates on YouTube how she uses various things, among them iPhone with the onscreen six-finger Braille keyboard. But also she uses the ‘magnifier’ app to tell her what is there around her, i.e. what the camera sees: this video (The videos are filmed by her boyfriend, in case someone has to wonder.) I realize that the vid is not too informative to a blind person, other than a vague demo. IDK where to find more info on the setup, since I'm not a user of iPhones myself. Comments on YouTube say that the magnifier app is in the accessibility options, that it adds a home screen app when enabled, and that it works at least on iPhone 10 since that's what Kristy uses. I would also guess that such usage requires persistent mobile connection and consumes considerable traffic and battery—but again, with modern tech magic I won't bet against it working locally on the phone. On Android, the feature called ‘Lens’ in the Google's camera app can describe some things, more useful to seeing people IMO: e.g. the model of a car—basically the same as Google's image search does, these days. Also purportedly can read and translate text signs. But IDK how far that goes, never heard of it doing stuff like those descriptions by the iPhone magnifier.
Cites itfullstopMay 2021
“It describes the scene, tells you how many faces are visible, whether or not the image is focused, lighting conditions.”
Shows camera describing the scene, faces, focus and lighting.
Full comment
It describes the scene, tells you how many faces are visible, whether or not the image is focused, lighting conditions. this video They can also generate alt text for photos which do not have that information already. Here's a video of the person in the original article describing this feature: link
Cites itjodrellblankJan 2022
“2020 iPhone can do on-device image processing and object recognition and speak a description of what the camera sees.”
iPhone describes what the camera sees, on-device.
Full comment
This this video is a recent thing I found mind blowing; 2020 iPhone can do on-device image processing and object recognition and speak a description of what the camera sees. Just quietly a builtin feature. 2010 iPhone, I was mind-blown by the Word Lens app which could do OCR on text in the camera feed, translate the words into another language, and overlay the results on the image in near-realtime. link These are tasks I had never seen any computer do in any circumstances, things which would be unthinkable on a Java MIDP Blackberry from 2005, or a Pentium II desktop from 1997. Compare this to the wearable augmented reality devices Professor Steve Mann was building through the 80s and 90s[1] and this "describing the image" is so so far ahead in so many ways - processing power, imaging quality, battery life, storage space, size, weight, convenience, reliability, it's just mindblowingly better, neural networks and fast chips and solid state storage is a step change in a way that "faster" isn't enough to really convey. Google tells me "Derivatives have a long history in the United States, dating back to the founding of the Chicago Board of Trade in 1848.", you're telling me people are doing that but with blockchain, why is that interesting at all, why mind blowing? [1] link
