Using augmented reality to help artists tell stories behind their art

Using augmented reality to help artists tell stories behind their art

Team: Product Designer, Project Manager, Software Engineer, QA

Team: Product Designer, Project Manager, Software Engineer, QA

App Store Screenshots

App Store Screenshots

Point on this image with Art House App to see it come to life

Point to this with ARt House App

Meet the client

Meet the client

Street Art House is an art curation startup based in Los Angeles. Its team works with street artists around the world to host events, sell merchandise, and connect them with businesses and brands to build unique collaborations.

Street Art House is an art curation startup based in Los Angeles. Its team works with street artists around the world to host events, sell merchandise, and connect them with businesses and brands to build unique collaborations.

One of the artists from outer space (Sara Sandoval) visiting Street Art House

One of the artists from outer space (Sara Sandoval)
visiting Street Art House

Starting with discovery

Starting with discovery

Conducting user research

Understanding users

As a result of user interviews, we created a range of user personas

As a result of user interviews, we created a range of user personas

That helped us to focus on our customers and their stories

That helped us to focus on our customers and their stories

The pattern emerges

The pattern emerges

During the research and understanding phase, we discovered that a lot of the artists resonated with a feeling of not being personally acknowledged for their art. A few of them also voiced concern that true stories behind their art are rarely interpreted in a way it was intended.

The art goers group had mostly different concerns. They enjoyed the modern art scene for its interactivity and blending of different mediums together. A lot of them pointed out to social gatherings and the feeling of community as the main driver for attending art events.

During the research and understanding phase, we discovered that a lot of the artists resonated with a feeling of not being personally acknowledged for their art. A few of them also voiced concern that true stories behind their art are rarely interpreted in a way it was intended.

The art goers group had mostly different concerns. They enjoyed the modern art scene for its interactivity and blending of different mediums together. A lot of them pointed out to social gatherings and the feeling of community as the main driver for attending art events.

Low Fidelity Wireframes

Wireframes overview

Creating the user experience

Creating user experience

Based on key findings from discovery and research sessions, we gathered our findings and moved into crafting the ultimate user experience. Our first step was to write user stories, based on which I moved on to create Information Architecture, User Flows, Sketches, and Wireframes.

Based on key findings from discovery and research sessions, we gathered our findings and moved into crafting the ultimate user experience. Our first step was to write user stories, based on which I moved on to create Information Architecture, User Flows, Sketches, and Wireframes.

Information architecture for ARt House platform

Information architecture for ARt House platform

Art House Sketches

Initial sketches of the app

User Flow

User flow

Content (art) is the king

Content (art) is the king

We believe the user interface of our app shouldn't get between users and art. The content is the king and our app is a comfortable tool that helps people experience it in the best way. Based on that, we made our user interface functional yet minimalistic by eliminating all unnecessary elements and scenarios.

We also set out to learn what type of content works best in AR. It was important to identify it from the start in order to build it into the CMS. Based on our interviews with stakeholders, we identified the following types of content: 

We believe the user interface of our app shouldn't get between users and art. The content is the king and our app is a comfortable tool that helps people experience it in the best way. Based on that, we made our user interface functional yet minimalistic by eliminating any unnecessary elements and scenarios.

We also set out to learn what type of content works best in AR. It was important to identify it from the start in order to build it into the CMS. Based on our interviews with stakeholders, we identified the following types of content: 

• AR Animation - A custom-crafted 2D / 3D animation playing on top of the art piece
• Video Overlay - Timelapse of the process, artist interview, clips, etc
• 360 Artist Studio Tour - A 360 panorama image that allows us to visit the places that are important to the story
• Artist Information - Learn more about the person behind art and give them a follow on social media

• AR Animation - A custom-crafted 2D / 3D animation playing on top of the art piece
  
  
• Video Overlay - Timelapse of the process, artist interview, clips, etc
  
  
• 360 Artist Studio Tour - A 360 panorama image that allows us to visit the places that are important to the story
  
  
• Artist Information - Learn more about the person behind art and give them a follow on social media

Visual overview of Art House app

Visual overview of Art House app

Visual Design: Main Flow

Visual Design: Main Flow

Point camera at Art House powered piece to activate the experience. Slide anywhere to change between different AR Layers. The layers can be set up to represent different types of content: Motion Graphic, Video File, 360 Panorama, or an info card. 

Point camera at Art House powered piece to activate the experience. Slide anywhere to change between different AR Layers. The layers can be set up to represent different types of content: Motion Graphic, Video File, 360 Panorama, or an info card. 

Visual Design: Onboarding

Visual Design: Onboarding

Upon launching the app we present users onboarding flow to familiarize them with capabilities of the app and get their contact information for future promotions. 

Upon launching the app we present users onboarding flow to familiarize them with capabilities of the app and get their contact information for future promotions.

Content Management System

Content Management System

Upon launching the app we present users onboarding flow to familiarize them with capabilities of the app and get their contact information for future promotions. 

Upon launching the app we present users onboarding flow to familiarize them with capabilities of the app and get their contact information for future promotions.

Art House Content Management - Augmentables, Marketing Screens, Users Management

Art House Content Management - Augmentables, Marketing Screens, Users Management

Throughout the development process, we conducted testing to identify pain points and edge cases of current versions at the time. We shared Invision prototypes with stakeholders to get early feedback. We used user pilot applications to share the latest version of the app with the company and the artists. We collected all of the feedback and utilized in further creation of the app.

Throughout the development process, we conducted testing to identify pain points and edge cases of current versions at the time. We shared Invision prototypes with stakeholders to get early feedback. We used user pilot applications to share the latest version of the app with the company and the artists. We collected all of the feedback and utilized in further creation of the app.

App testing in real world exhibition environment. ARt by James Hunt

App testing in real-world exhibition environment. ARt by James Hunt.

Crowds gathering for ARt show where the app was tested

Crowds gathering for ARt show where the app was tested

Artist residency - augmented reality

Artist residency - augmented reality

It is important for us to bring artists up to speed in their understanding of the immersive technologies. For that, we have hosted specialized workshops where we introduced artists to the concepts of AR and VR as well as its use cases in the art creation process. We showed real world examples of how artists already take advantage of that field and introduced them to the best VR software such as Google Tiltbrush and Oculus Quill. After several training sessions artists were able to create professional vr art and had much greater understanding of emerging technologies.

It is important for us to bring artists up to speed in their understanding of the immersive technologies. For that, we have hosted specialized workshops where we introduced artists to the concepts of AR and VR as well as its use cases in the art creation process. We showed real world examples of how artists already take advantage of that field and introduced them to the best VR software such as Google Tiltbrush and Oculus Quill. After several training sessions artists were able to create professional vr art and had much greater understanding of emerging technologies.

Tilt Brush is a groundbreaking app by Google for people to create art in 3D space

Tilt Brush is a groundbreaking app by Google for people to create art in 3D space

Another important topic of discussion that affected design and development was to decide which AR technology would be best suited for the needs of the project. AR/VR is a huge topic in itself that I spent 5 last years researching, so by the start of the project, I was well aware of the limitations and capabilities of the current technologies.

At the time, Vuforia and Wikitude were the leaders on the market of mobile AR tech. That, however, was changing, with Apple introducing their ARkit and Google their ARcore technologies. The problem was that both of those were entirely new and lacked some capabilities as well was not as stable, but it was clear that they are the future of AR nevertheless. Thus the decision was made to develop for the native platforms. As those platforms advance, our product becomes better without the need for too much future development. As a result, one-year later, our product supports ARkit 3.0 natively.

Another important topic of discussion that affected design and development was to decide which AR technology would be best suited for the needs of the project. AR/VR is a huge topic in itself that I spent 5 last years researching, so by the start of the project, I was well aware of the limitations and capabilities of the current technologies.

At the time, Vuforia and Wikitude were the leaders on the market of mobile AR tech. That, however, was changing, with Apple introducing their ARkit and Google their ARcore technologies. The problem was that both of those were entirely new and lacked some capabilities as well was not as stable, but it was clear that they are the future of AR nevertheless. Thus the decision was made to develop for the native platforms. As those platforms advance, our product becomes better without the need for too much future development. As a result, one-year later, our product supports ARkit 3.0 natively.

The feedback we've got given so far from our users was overwhelmingly positive. For most customers, it's their first interaction with augmented reality and they come to love it. At the same time, the business shows positive projections as customers are already showing a bigger interest in buying augmented merchandise and specialized augmented art pieces. Art House was the app powering the Sneakertopia exhibit. The app was launched in December 2018 and is available to download on the App Store and Play Store.

The feedback we've got given so far is overwhelmingly positive. For most customers, it's their first interaction with augmented reality. At the same time, the business shows positive projections as customers are already showing a bigger interest in buying augmented merchandise and specialized augmented art pieces. We launched the platform in December 2018 and it's available for download on the App Store