Creating The Future of Sound at MIT
Creating The Future of Sound at MIT
Creating The Future of Sound at MIT
MIT MEDIA LAB HACKATHON
MIT MEDIA LAB HACKATHON
Using Spatial Sounds and Augmented Reality To Help Visually-Impaired People Navigate Their Surroundings
Using Spatial Sounds and Augmented Reality To Help Visually-Impaired People Navigate Their Surroundings
CHALLENGE
CHALLENGE
To discover new ways of environment perception by combining spatial sounds and location-based data.
To discover new ways of environment perception by combining spatial sounds and location-based data.
OUTCOME
OUTCOME
We were able to create a working prototype solution that ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.
We were able to create a working prototype solution that ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.
PRESS
PRESS
PROJECT SCOPE
PROJECT SCOPE
Product, Interaction design, Research, Accessibility Design, Prototyping & testing
Product, Interaction design, Research, Accessibility Design, Prototyping & testing
Team: Dhara Bhavsar, Sunish Gupta, Vedant Saran, Anandana Kapur, Maxym Nesmashny
Team: Dhara Bhavsar, Sunish Gupta, Vedant Saran, Anandana Kapur, Maxym Nesmashny
Duration: 4 days
Duration: 4 days
Hackaton Organizer: MIT Media Lab
Hackaton Organizer: MIT Media Lab
What is it about
ARound brings a new perspective to the possibilities of Augmented Reality. While most of AR today is centered on augmenting one's visual systems, we think that there's immense unrealized potential in augmented one’s sense of hearing. While current-generation AR systems can only augment your visuals in a tiny field of view, spatial audio enhances the user’s perception in all 360 degrees.
We believe that there are huge opportunities to create platforms based entirely on sound. And it seems like our beliefs are coming to fruition with the release of mainstream products such as Bose Frames.
ARound brings a new perspective to the possibilities of Augmented Reality. While most of AR today is centered on augmenting one's visual systems, we think that there's immense unrealized potential in augmented one’s sense of hearing. While current-generation AR systems can only augment your visuals in a tiny field of view, spatial audio enhances the user’s perception in all 360 degrees.
We believe that there are huge opportunities to create platforms based entirely on sound. And it seems like our beliefs are coming to fruition with the release of mainstream products such as Bose Frames.
Team formation
Before arriving in Boston, I pitched the original idea to the hackathon's Facebook group: "To augment the places around us with the help of generative sounds, using Google Maps API, to create an interactive soundtrack of the world..."
Upon arriving at MIT Media Lab, I met with several candidates who reached out and were interested to take on this challenge. One of them was Sunish Gupta, an accessibility expert who was visually-impaired. That's when all the pieces have fallen into place, we could actually make an impact. We ended up having a diverse team with various backgrounds and experiences:
Dhara Bhavsar - back-end engineer
Sunish Gupta - accessibility expert
Vedant Saran - unity developer
Anandana Kapur - project manager and filmmaker
Maxym Nesmashny - product designer
Before arriving in Boston, I pitched the original idea to the hackathon's Facebook group: "To augment the places around us with the help of generative sounds, using Google Maps API, to create an interactive soundtrack of the world..."
Upon arriving at MIT Media Lab, I met with several candidates who reached out and were interested to take on this challenge. One of them was Sunish Gupta, an accessibility expert who was visually-impaired. That's when all the pieces have fallen into place, we could actually make an impact. We ended up having a diverse team with various backgrounds and experiences:
Dhara Bhavsar - back-end engineer
Sunish Gupta - accessibility expert
Vedant Saran - unity developer
Anandana Kapur - project manager and filmmaker
Maxym Nesmashny - product designer
Hacking begins
On the first day of the Hackathon, we spent entirely on discovery and research sessions in order to understand the needs of our users. We learned a lot about the challenges visually-impaired people are facing every day. Activities such as walking around and getting familiar with places like cafes, shops, post offices, etc can be extremely challenging.
Based on our finding we identified 3 main features that seemed to be feasible enough to build during the hackathon.
Discovery Mode, which would use the combination of GPS data and spatial sounds to create an interactive soundscape around the user to navigate in.
Breadcrumbs Mode, to record the path you are taking in order to return using the same path.
Share the Meeting Spot, which would allow a 3rd party user, usually a friend or family of visually impaired, to put a custom sound pin on the map. Upon arrival to the area, a visually-impaired person would be guided directly to the meeting point, again with the help of spatial sound.
On the first day of the Hackathon, we spent entirely on discovery and research sessions in order to understand the needs of our users. We learned a lot about the challenges visually-impaired people are facing every day. Activities such as walking around and getting to familiar places like cafes, shops, post office, etc can be extremely challenging.
Based on our finding we identified 3 main features that seemed to be feasible enough to build during the hackathon.
Discovery Mode, which would use the combination of GPS data and spatial sounds to create an interactive soundscape around the user to navigate in.
Breadcrumbs Mode, to record the path you are taking in order to return using the same path.
Share the Meeting Spot, which would allow a 3rd party user, usually a friend or family of visually impaired, to put a custom sound pin on the map. Upon arrival to the area, a visually-impaired person would be guided directly to the meeting point, again with the help of spatial sound.
ARound Mobile User Interface
Around Web Interface
Results
We are thrilled to have built a working prototype. We were able to implement a directional spatial audio soundscape as well as a breadcrumbs feature set. We are proud of the idea being user-validated by our team member who is visually impaired.
Our solution ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.
We are thrilled to have built a working prototype. We were able to implement a directional spatial audio soundscape as well as a breadcrumbs feature set. We are proud of the idea being user-validated by our team member who is visually impaired.
Our solution ended up winning in 3 categories: Best AR/VR for Good, Everyday Mobile Hack, Leadership Award.
What's next
We are looking to develop our MVP into a full-scale platform that would help blind navigate the environment. We are also excited about the opportunities it holds for mainstream people. From interactive tour experiences to generative soundtracks of the world, so much can be achieved. If you are interested in helping us bring this project to another level, don't hesitate to drop me a message
We are looking to develop our MVP into a full-scale platform that would help blind navigate the environment. We are also excited about the opportunities it holds for mainstream people. From interactive tour experiences to generative soundtracks of the world, so much can be achieved. If you are interested in helping us bring this project to another level, don't hesitate to drop me a message
Have a similar project in mind? Let's talk!
Have a similar project in mind? Let's talk!
Have a project in mind? Let's discuss it!
Looking for a similar solution?
MORE PROJECTS
Let's Connect
Get in touch for any full-time or freelance opportunities 
hi@maxym.me
Get in touch for any full-time or freelance opportunities
hi@maxym.me
