Making the Visual World Tactile with the Wing It App for Monarch

Someone with typical vision can easily view pictures on their phone or computer. Students who are blind or low vision require those images to be made tactile, which takes a lot of paper, supplies, time, and money to produce. To solve this problem, APH created the Wing It app, which allows teachers and students to draw images which appear tactilely on the Monarch’s 10-line by 32-cell braille display. “It’s not just creating the ability to draw, but the ability to draw tactile graphics,” said Jason Martin, APH’s Technical Innovations Product Manager.

This upcoming app can be downloaded from the Apple App Store onto an iPhone or iPad. The Monarch companion app will be available for download as part of a future update. To connect the iOS app to the Monarch, simply open it, enable location and privacy services, and turn on the “Connect” toggle switch. If Bluetooth is enabled on both devices, the iOS app will find and connect to the Monarch. Afterwards, anyone can make a drawing on the iOS app and broadcast it to the Monarch. This feature is also accessible with VoiceOver so individuals who are blind or low vision can participate, too.

The app contains a style menu, similar to Microsoft Paint. Pick from one of the five differentiating fill patterns for an image: no pattern, diagonal, horizontal, or vertical lines, and checkerboard. When a shape is sketched, it will contain that pattern. After users lift their finger and stop drawing, the app automatically sends the image to the Monarch along with the fill pattern in raised dots. “When he made Wing It, Mark Klarer, APH’s Advanced Software Engineer, put in the ability to write on the iOS app, and it converts it into braille on the Monarch,” said Joe Hodge, APH’s Lead Technical Quality Assurance Analyst. This feature is useful when labeling items on a map.

The Wing It app can assist students in and outside of the classroom. Teachers who forgot to adapt an image can create one for a student, so they instantly have the same material as their peers. An orientation and mobility instructor can use the app to draw maps of a student’s school, neighborhood, and other new environments. Now, instead of having to ask where the bathroom is at a new school and getting verbal directions, students can feel a map to tactually learn its location and navigate there, which helps improve their map reading and mobility skills. In schools that utilize 3D printed materials to aid in learning, teachers can draw objects on the app so a student knows what item they will receive from the printer.

Outside of the classroom, sighted peers can illustrate their surroundings so students who are blind or low vision can see what they see. For example, they can draw an eagle flying by so their blind or low vision peers can understand what it looks like. For the first time, the app gives students the ability to participate in people watching. They can learn what others look like and discover the nuances of facial expressions. Friends can reproduce items related to their topic of conversation, such as a video game character, superhero, or celebrity. “Sometimes, connecting with people is hard because visually, we can’t see certain things, and this app helps bridge the gap,” said Joe. Jason commented about how the software could be used during sporting events. He said, “Once we have the ability to save an image, I could do a football lineup of X’s and O’s and then draw plays as they happen in real-time, hit clear, bring back that same football lineup with X’s and O’s, and draw the next play that happens.”

With the Wing It app, users don’t need to be experts to create tactile graphics. They play a critical role by exposing students to more images than they ever had before.

Come see the Wing It app in action at APH’s 156th Annual Meeting on October 2-4, 2024 in Louisville, KY. Stay tuned to the website, APH News, and your email inbox for more information on when this app will be available for Monarch users.