MIT MEDIA LAB HACKATHON

Inventing the Future of Sound

Using Spatial Sounds and Augmented Reality To Help Visually-Impaired People Navigate Their Surroundings

Using Spatial Sounds and Augmented Reality To Help Visually-Impaired People Navigate Their Surroundings

CHALLENGE

CHALLENGE

To discover new ways of environment perception by combining spatial sounds and location-based data.

To discover new ways of environment perception by combining spatial sounds and location-based data.

We were able to create a working prototype solution that ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.

We were able to create a working prototype solution that ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.

How The Internet of Sound Eliminates Billions of IOT Sensors - Networkworld

How The Internet of Sound Eliminates Billions of IOT Sensors - Networkworld

Hacking The Future at MIT - Samsung NEXT

Hacking The Future at MIT - Samsung NEXT

Product, Interaction design, Research, Accessibility design, Prototyping & testing

Product, Interaction design, Research, Accessibility design, Prototyping & testing

ARound brings a new perspective to the possibilities of Augmented Reality. While most of AR today is centered on augmenting one's visual systems, we think that there's immense unrealized potential in augmented one’s sense of hearing. While current-generation AR systems can only augment your visuals in a tiny field of view, spatial audio enhances the user’s perception in all 360 degrees.

We believe that there are huge opportunities to create platforms based entirely on sound. And it seems like our beliefs are coming to fruition with the release of mainstream products such as Bose Frames.

ARound brings a new perspective to the possibilities of Augmented Reality. While most of AR today is centered on augmenting one's visual systems, we think that there's immense unrealized potential in augmented one’s sense of hearing. While current-generation AR systems can only augment your visuals in a tiny field of view, spatial audio enhances the user’s perception in all 360 degrees.

We believe that there are huge opportunities to create platforms based entirely on sound. And it seems like our beliefs are coming to fruition with the release of mainstream products such as Bose Frames.

Before arriving to Boston, I pitched the original idea to the hackathon's Facebook group: "To augment the places around us with the help of generative sounds, using Google Maps API, to create an interactive soundtrack of the world..."

Upon arriving to MIT Media Lab, I met with several candidates who reached out and were interested to take on this challenge. One of them was Sunish Gupta, an accessibility expert who was visually-impaired. That's when all the pieces fallen into place, we could actually make an impact. We ended up having a diverse team with various backgrouds and experiences:

Dhara Bhavsar - back-end engineer 
Sunish Gupta - accessibility expert 
Vedant Saran - unity developer 
Anandana Kapur - project manager and filmmaker
Maxym Nesmashny - product designer

Before arriving to Boston, I pitched the original idea to the hackathon's Facebook group: "To augment the places around us with the help of generative sounds, using Google Maps API, to create an interactive soundtrack of the world..."

Upon arriving to MIT Media Lab, I met with several candidates who reached out and were interested to take on this challenge. One of them was Sunish Gupta, an accessibility expert who was visually-impaired. That's when all the pieces fallen into place, we could actually make an impact. We ended up having a diverse team with various backgrouds and experiences:

Dhara Bhavsar - back-end engineer 
Sunish Gupta - accessibility expert 
Vedant Saran - unity developer 
Anandana Kapur - project manager and filmmaker
Maxym Nesmashny - product designer

First day of the Hackathon we spent entirely on discovery and research sessions in order to understand the needs of our users. We learned a lot about challenges visually-impaired people are facing every day. Activities such as walking around and getting to familiar places like cafes, shops, post office, etc can be an extremely challenging.

Based on our finding we identified 3 main features that seemed to be feasible enough to build during the hackathon.

Discovery Mode, which would use the combination of GPS data and spatial sounds to create an interactive soundscape around the user to navigate in. 

Breadcrumps Mode, to record the path you are taking in order to return using the same path.

Share the Meeting Spot, which would allow a 3rd party user, usually a friend or family of visually impaired, to put a custom sound pin on the map. Upon arrivil to the area, visually-impaired person would be guided directly to the meeting point, again with the help of spatial sound.

First day of the Hackathon we spent entirely on discovery and research sessions in order to understand the needs of our users. We learned a lot about challenges visually-impaired people are facing every day. Activities such as walking around and getting to familiar places like cafes, shops, post office, etc can be an extremely challenging.

Based on our finding we identified 3 main features that seemed to be feasible enough to build during the hackathon.

Discovery Mode, which would use the combination of GPS data and spatial sounds to create an interactive soundscape around the user to navigate in. 

Breadcrumps Mode, to record the path you are taking in order to return using the same path.

Share the Meeting Spot, which would allow a 3rd party user, usually a friend or family of visually impaired, to put a custom sound pin on the map. Upon arrivil to the area, visually-impaired person would be guided directly to the meeting point, again with the help of spatial sound.

We are thrilled to have built a working prototype. We were able to implement directional spatial audio soundscape as well as breadcrumps feature set. We are proud about the idea being user-validated by our team member who is visually impaired.

Our solution ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.

We are thrilled to have built a working prototype. We were able to implement directional spatial audio soundscape as well as breadcrumps feature set. We are proud about the idea being user-validated by our team member who is visually impaired.

Our solution ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.

We are looking to develop our MVP into a full-scale platform that would help blind navigate the environment. We are also excited about the opportunities it holds for mainstream people. From interactive tour experiences to generative soundtracks of the world, so much can be achieved. If you are interested in helping us bring this project to another level, don't hesitate to drop me a message 

We are looking to develop our MVP into a full-scale platform that would help blind navigate the environment. We are also excited about the opportunities it holds for mainstream people. From interactive tour experiences to generative soundtracks of the world, so much can be achieved. If you are interested in helping us bring this project to another level, don't hesitate to drop me a message