Training the next generation of automotive technicians
Transforming auto repair training with Mixed Reality to empower the next generation of technicians.
BACKGROUND: The brief was to explore applications of Mixed Reality for positive cultural impact.
OVERVIEW: A Mixed Reality education platform that promotes critical problem-solving & data-driven learning in automotive repair.
TEAM: Hannah Rosenfeld, Leah Jiang , Manjari Sahu & myself.
MY ROLE: User research & synthesis, concept development, AR interface & prototyping.
RECOGNITION: Shortlisted for IxDA Interaction Awards 2018, represented Carnegie Mellon University at Microsoft Design Expo .
As a team, we were interested in education. Mixed Reality was particularly suited for embodied learning. This led us to automotive repair, a field that is changing due to technological advancements and is highly hands-on.
We started by mapping out various stakeholders, education platforms, potential challenges and opportunities for mixed reality in this space.
We interviewed car hobbyists, educators and a number of experts in technology, automotive repair, and related fields, such as aviation and bike repair, to understand the nature of the work and unearth pain points for deeper exploration.
We asked students at Rosedale Technical College (a vocational school in Pittsburgh) to envision and build tools to aid them in their work during a generative workshop. Each student was asked to think of a specific scenario and challenge they faced recently and imagine a tool they wish they had to help them work through it.
The real insight was in the scenarios, challenges, and stories they shared through the process.
We identified the following pain points faced by the students and instructors we spoke to...
The following are key learnings from exploratory research that we needed to account for when designing solutions.
We structured our brainstorm around the following opportunity areas we uncovered from exploratory research:
1. Dynamic learning environments that supported embodied learning
2. Remote support and mentorship to enable collaborative real-world practice and
complement in-classroom learning
3. Smart tools that provided realtime feedback to students
4. Administrative support to automate certain tasks
STORYBOARDING & SPEED-DATING CONCEPTS
From our initial brainstorm, the strongest concepts were further developed into a series of storyboards that explored different combinations of needs, environments, and tools. We speed-dated the storyboards with instructors for feedback.
Our final concept, Dash, is an educational platform that integrates contextually relevant information into Mixed Reality environments to promote critical problem solving and data driven learning in automotive repair education. It consists of the following components...
PROTOTYPING & TESTING
We rapidly prototyped the dashboard UI using paper. The primary objective of this exercise was to get feedback on the information architecture, content and user flow from instructors.
We got valuable suggestions, such as using work orders in place of lesson prompts to help students build the habit of documenting their work.
TABLET DASHBOARD UI
The dashboard is designed to give instructors greater visibility of their students and foster better collaboration with them. View-sharing allows instructors and students to share what they see and make annotations in real time.
A real-time work log uses sensor data from the car to track students’ work flow. Instructors can review students work and provide personalized and granular, feedback. Such features are especially useful when dealing with large class sizes.
Student's Augmented Reality Dashboard
VIDEO PROTOTYPES OF AR INTERACTIONS
For most of our target audience, AR and MR were abstract concepts that were unfamiliar. Through a series of short videos to simulate what interactions with a system in AR might feel like, we learnt about students' and instructors' needs and interaction preferences.
PAPER PROTOTYPES OF DASHBOARD UI
How do you rapidly prototype for AR? Simple. A clear acrylic sheet with printed UI elements and a pair of safety glasses with a laser pointer attached to it (to mimic gaze).
The low-fidelity version of "augmented reality" allowed us to co-design the dashboard with students by enabling the flexibility needed to move elements around quickly as students walked through the features and offered suggestions for improvement. Students were also excited about the potential such a system could have out in the field too.
AR DASHBOARD: MAIN MENU
Dash brings all the information they need for car repair in one location by creating a hybrid physical-digital workspace. This includes, car schematics, real-time sensor data, vehicle specifications and course material.
The goal is to help students focus on problem-solving and making sense of information from the car. The AR menu is, therefore, designed for easy navigation to resources and information students need in the workshop. Students can customize their digital dashboard to ensure frequently accessed information is always kept in view.
AR DASHBOARD: CONTEXTUAL MENU
Contextual menus provide shortcuts for students to quickly access data and information for individual car components.
Given the complexity of field work and the need for context-specific interactions (e.g. when working under a car), Dash is designed to accept different forms of input. Voice, Gaze, Touch, Tap and Gesture can be used at different times and in different combinations depending on the context.
Prototyping played a big role throughout the design process. Here is a video of a high-fidelity interactive prototype of the student dashboard we built using Unity for the Microsoft Hololens.