Predictive AI assistant for airline staff
Role
Team lead
Project manager
Designer
Researcher
Context
Sponsored master's capstone
Duration
Mar-Aug 2024
(6 months)
Skills
Contextual inquiry
User interviews
Workshop facilitation
Evaluative testing
Team
3 designers
1 researcher
Overview
Summary
For my master's capstone project, my team worked with United Airlines to design an AI-powered mobile service that equips employees with real-time insights and resources to better support customers. We presented this to the Strategy team at United Airlines where it was received positively.
"You should be on our strategy team...some of your recommendations are aligned with what we are already thinking about."- Jason Flint (Director, Digital Delivery at United Airlines)
Problem
How might we help customer service representatives anticipate and triage customer needs in the lobby?
My Contributions
Meeting & workshop facilitation
I led and facilitated team meetings, research and design workshops, and stakeholder meetings.
Designing user flows and wireframes
I led the team in identifying core user flows, ensuring the team is aligned and grounded in our research insights.
Project management
With only 10 weeks to turn our research insights into a design solution, I created and enforced project trackers, key milestones, and note-taking templates.
Solution overview
Our solution has 3 touch points that connects the customer and the customer service representative (CSR) working in the lobby.
1
Streamlined kiosk
CUSTOMER FACING
Customers tap their phones on our redesigned kiosk, launching the United Airlines online check-in process.
2
Customer app
CUSTOMER FACING
They can request for help directly from the mobile app at any point.
3
Employee mobile app
CSR FACING
If help is requested, CSRs can learn about the customer through their employee mobile app
The employee mobile app was the focus of our solution; it allows customer service representatives to:
1
Identify customers who need help and what they need help with.
2
Catch up on necessary pieces of context about the customer's situation.
3
Consult relevant air travel policies.
4
Directly send resources to the customer.
Customer profile and check-in status
Real-time audio transcription
Chat interface to edit AI-suggestions
AI suggestions/insights that updates as more context is gathered
Research
CSRs as our target audience
We focused on customer service representatives (CSRs) stationed in the lobby area (pre-security) because it's often the customer's first point of interaction with an airline, in addition to being a stressful environment
Top questions for CSRs
We wanted to learn more about our target audience's:
overall role and key responsibilities
everyday challenges, obstacles, and frustrations
available resources, tools, and support systems
Methodology
In 3 weeks, we conducted:
5 Contextual inquiries
Observed and interviewed 15 CSRs across Seattle-Tacoma & Chicago O’Hare airports
4 SME interviews
Interviewed management and veteran CSRs to learn about the role from an operational POV
Competitive analysis
Observed lobby area of a few competitors (American Airlines, Southwest Airlines, Delta Airlines, and Alaska Airlines) at SeaTac
Analysis
We transferred and categorized our notes into Figjam and saw certain themes emerge, such as:
confusion with United Airlines' open lobby arrangement
difficulties with the customer flow
challenging customer interactions.
Insights
01
CSRs have to adapt to a wider range of interactions because self-service technologies (e.g kiosks, mobile apps, chatbots) have diversified customer scenarios.
02
United Airlines' lobby design in the Seattle-Tacoma airport lacks directional cues, creating a sense of ambiguity and confusion. This forces CSRs to inefficiently go through idle customers one by one.
03
During irregular operations (e.g canceled or delayed flights), customers will seek in-person assistance over self-service tools.
Jobs-to-be-done
We reframed our insights following the Jobs to be Done framework and concluded that our solution should provide CSRs a way to:
Monitor the lobby area
Anticipate customer needs
Ideation
Initial ideas, design goals, and project scope
Next, I led the team through a few brainstorming activities and workshops to explore potential solutions.
Brainstorming
We went through a several rounds of brainstorming both individually and as a group, seeing what themes emerged.
Journey mapping
We mapped out key tasks, thoughts/emotions, and identified areas of opportunity.
Categorizing + dot voting
We categorized potential ideas by level of feasibility (ideas we can implement in the present, near future, and far future) for further alignment.
Selected idea
After a few rounds of ideating, we saw that we had to think about our proposed solution from a service design perspective, ensuring that it will benefit both CSRs and the customers they are helping. The idea we landed on involves 3 touch points:
1
Customers tap their personal devices on a simpler version of a kiosk, launching the already-existing United Airlines online check-in process.
2
As customers go through the check-in process on their phones, they can request for assistance at any point.
3
Meanwhile, on their employee mobile app, CSRs monitor the lobby area and can access the customer's details if help is requested.
Diagram of imagined service
Further refinement
To further flush out this idea, we created storyboards and a service blueprint.
Sketching out our initial idea
Initial service blueprint
Designing the CSR mobile app
Goals
The focus of our project is an AI-powered mobile app CSRs use to monitor the lobby and access customer data on-the-go. We envision this to be integrated with the main mobile app CSRs already use.
User flow and information architecture
I created an information architecture diagram and user flow to align the team on core pieces of information and features to include.
User flow diagram
Information architecture diagram
We wanted this app to give CSRs insights into:
Lobby overview
CSRs can see which kiosks are actively in use and who has requested assistance .
Customer profile
Information about check-in progress, flight status, and basic customer details.
Suggested actions
Using the check-in process and basic customer details, the app will provide CSRs with suggested actions and/or resources based on the context it has gathered.
Low-fidelity and mid-fidelity wireframes
I created wireframes once the team reached alignment on the logic above.
Whiteboard sketches of initial user flows
Sketches of the CSR app
Turning our sketches to mid-fidelity wireframes
Design considerations
Lobby view
Would showing too much information on the lobby overview page introduce bias and discourage CSRs from accepting a request? If the customer's situation is too complicated, would CSRs ignore them?
We decided that CSRs only needed:
the status of each kiosk
customers' statuses
how long customers been waiting at the kiosk
Customer profile
Once a CSR accepts a case, they can see relevant customer details that provide context on the customer's situation. AI is used here to suggest the best course of action based on the information initially collected.
Our research showed that customers got more frustrated when they had to repeat their situation and the steps they've done already. To address this issue, we included a Recent Activity section (outlined) so CSRs can see the steps customers have already tried to avoid suggesting it again.
Transcription view
On this page, CSRs can see their ongoing conversation with the customers they are helping. This gives them additional context for the AI suggestion provided on the homepage.
This section took some realignment because we wanted to visually distinguish customer inputs, CSR responses, and AI suggestions, while ensuring it isn't too confusing or overwhelming to browse.
Transcription view
Customer Profile
7:15 PM
AT&T Wi-Fi
100%
AiR Flow
Listening to Conversation
AI
AiR Flow AI Assistant
Here are relevant policies for delayed flights, and options for re-booking flights.
Suggested Action Plan
General Refund Policy
Go to My Trips to cancel your reservation and start the refund process.
The refund amount will vary based on the situation.
Your refund will be credited back to your original form of payment.
If your ticket was purchased in the last 24 hours, it may qualify for our 24-hour booking policy.
Credit card refunds will be processed within seven business days of the request. All other refunds will be processed within 20 business days of the request.
Basic Economy tickets can’t be changed but can be canceled and refunded within 24 hours of booking as long as it’s one week or more before your scheduled departure.
For more information visit website
Message
AiR Flow
Listening to Conversation
AI
AiR Flow AI Assistant
Here is the policy for rebooking in the event of a cancelled flight.
Rebooking policies
Go to My Trips to cancel your reservation and start the refund process.
The refund amount will vary based on the situation.
Your refund will be credited back to your original form of payment.
If your ticket was purchased in the last 24 hours, it may qualify for our 24-hour booking policy.
Credit card refunds will be processed within seven business days of the request. All other refunds will be processed within 20 business days of the request.
Basic Economy tickets can’t be changed but can be canceled and refunded within 24 hours of booking as long as it’s one week or more before your scheduled departure.
For more information visit website
Ask AI Assistant...
Agent
I’m so sorry to hear that - let’s get you to Chicago today. By tonight... let’s see...
✨ View:
- Rebooking policies
- Flights to Chicago by today, 6pm
Customer Profile
7:15 PM
AT&T Wi-Fi
100%
AiR Flow
Listening to Conversation
AI
AiR Flow AI Assistant
Here is the policy for rebooking in the event of a cancelled flight.
Flights to Chicago today, 6pm
THU, JUL 15, 2028
Seattle to Chicago
Delta 359 • 9:45 am to 11:45 pm • Nonstop • SEA - ORD
Send details to Darcy
Overall Top Pick
THU, JUL 15, 2028
Seattle to Chicago
United 7296 • 3.30 pm to 5:30 pm • Nonstop • SEA - ORD
Send details to Darcy
United Airline Pick
Generate more options
Revise search
Rebooking policies
Ask AI Assistant...
Agent
I’m so sorry sir, I empathize with your situation. It sounds really rough. For now, let’s get you started by looking into relevant cancellation policies.
Customer Profile
7:15 PM
AT&T Wi-Fi
100%
Listening to Conversation
Suggested action
Live transcript
SUGGESTIONS
THU, JUL 15, 2028
Seattle to Chicago
United 7296 • 3.30 pm to 5:30 pm • Nonstop • SEA - ORD
United Airlines pick
Overall top pick
THU, JUL 15, 2028
Seattle to Chicago
Delta 359 • 9:45 am to 11:45 pm • Nonstop • SEA - ORD
Here is some additional information
General Refund Policy
Another Policy
Summary
Issue: Flight cancellation, No new ticket issued yet, unable to check in due to flight being closer to boarding time
Urgency: Needs to be in Chicago by evening for a surprise for his father
Solution: Rebook passenger on new flight
Last refreshed 4:15 am Thu, Jul 22, 2028
Customer Profile
7:15 PM
AT&T Wi-Fi
100%
Listening to Conversation
Suggested action
Live transcript
Paul H.
Nice to meet you! Could we step just over here so I can assist you... Yes, I see that your flight was cancelled, and you have tried to rebook a ticket. Is that what I can help you with today? Rebooking a new ticket?
Paul H.
Nice to meet you! Could we step just over here so I can assist you... Yes, I see that your flight was cancelled, and you have tried to rebook a ticket. Is that what I can help you with today? Rebooking a new ticket?
Darcy L.
Darcy L.
Darcy L.
Yes, it was a mess last night. I was up trying to understand the next steps and get my flight re-booked, but it was really difficult. I really need this flight to make it for my dad’s birthday, it’s his 65th birthday party, I cannot miss this day. I was so stressed.
Yes, it was a mess last night. I was up trying to understand the next steps and get my flight re-booked, but it was really difficult. I really need this flight to make it for my dad’s birthday, it’s his 65th birthday party, I cannot miss this day. I was so stressed.
Yes, it was a mess last night. I was up trying to understand the next steps and get my flight re-booked, but it was really difficult. I really need this flight to make it for my dad’s birthday, it’s his 65th birthday party, I cannot miss this day. I was so stressed.
Version B:
Embedding the suggested actions into expandable sections so everything stays within its original context
Version A:
Separating the suggested actions and live transcript features into separate tabs
Customer Profile
7:15 PM
AT&T Wi-Fi
100%
Listening to conversation
Based on your conversation with Darcy, I figured you might be looking for alternate flights to ORD as soon as possible.
THU, JUL 15, 2028
Seattle to Chicago
Delta 359 • 9:45 am to 11:45 pm • Nonstop • SEA - ORD
Send details to Darcy
Yes, it was a mess last night. I was up trying to understand the next steps and get my flight re-booked, but it was really difficult. I really need this flight to make it for my dad’s birthday, it’s his 65th birthday party, I cannot miss this day. I was so stressed.
Yep.
I’m so sorry! That sounds really rough. Let’s get you on the next flight, ASAP.
Hi! My name is Paul, are you Darcy Lively?
Nice to meet you! Could we step just over here so I can assist you... Yes, I see that your flight was cancelled, and you have tried to rebook a ticket. Is that what I can help you with today? Rebooking a new ticket?
Ask AI Assistant...
Customer Profile
4:00 AM
AT&T Wi-Fi
100%
Darcy may need help with booking an alternative itinerary as check-in for flight UA 4892 is closed.
THU, JUL 15, 2028
Seattle to Chicago
Delta 359 • 9:45 am to 11:45 pm • Nonstop • SEA - ORD
Send details to Darcy
See more
Customer interaction in progress
Darcy Lively
•••• Not checked in • Kiosk 2
XA123456 • 0 miles • Member
Wait time: 5 min
Complete interaction
Travel itinerary
Recent activity
Last refreshed 4:15 am Thu, Jul 22, 2028
Instead of either version, we went with an approach that featured a dynamic card on the home page, prioritizing only needing to stay on one page. The suggestion card on the homepage (left) will update once the AI picks up additional details, either by listening to the conversation or by CSRs manually inputting details like they are talking to chatbot (right).
Designing the rest of the service
Touchpoint 1: Streamlined kiosk
We saw that having an open lobby was too confusing, so we knew we couldn't remove kiosks entirely. Their main purpose is to 1.) act as a visual waypoint and 2.) print boarding tickets when customers check in online.
We wanted the revised kiosk to be sleek and minimal. We created the final version in Spline3D.
UNITED
Stay where you are, a United agent is coming!
UNITED
Tap phone to get started
Touchpoint 2: United Airlines mobile app
After customers tap their phones on the kiosk, their phone will open the United Airlines check-in process through either the web or mobile app if they have it downloaded.
We added an option to the existing online check-in process that allows customers to directly request for in-person assistance.
Evaluation
Concept testing with 4 customers and 2 CSRs
We used our mid-fidelity prototypes to evaluate our concept with 4 customers and 2 CSRs of various airlines. We virtually walked our participants through wireframes and storyboards of the envisioned experience and asked them about their thoughts at each stage, specifically noting concerns with the usage of AI.
What we showed customers
Top row: What customers would see on their mobile application
Bottom row: What customers would see on the kiosk screen in front of them.
What we showed CSRs
Mid-fidelity mockups of CSR mobile application
Discoveries
Our biggest takeaway was that all participants felt that the use of AI in these scenarios were helpful and beneficial in supporting both customers and CSRs. We also saw that:
Customers wanted the ability to provide consent for real-time audio transcription.
When customers request assistance, they want to be clear who to expect.
Before
After
Made more personal, added location indicator to ensure customers don’t leave the lobby.
Added “opt-out” option to be more transparent about use of AI
Final Service
Customer flow
CSR flow
Visual identity
Global color tokens
$neutral.0
#000000
$neutral.1
#2f363f
$neutral.2
#5c697b
$neutral.3
#d6dbe0
$neutral.4
#e7ecef
$neutral.5
#FFFFFF
$blue.dark
#1414D2
$blue.light
#E8E8FD
$orange.dark
#FFC658
$orange.light
#FFFAEE
$yellow.dark
#F5DD01
$yellow.light
#F5DD01
$green.dark
#1B7742
$green.light
#C1F1D5
$red.dark
#FF0000
$red.light
#FFCCCC
$neonGreen
#8BEC4E
$skyBlue
#A4EBFF
$gradientBlue
#A4EBFF
Borders
$color.border.card
$neutral.3
$color.border.card.ai
$gradientBlue
$color.border.seperator
$neutral.3
Labels
$color.label.primary
$neutral.1
$color.label.secondary
$neutral.2
$color.label.active
$blue.dark
$color.label.inactive
$neutral.2
$color.label.button.primary
$blue.dark
$color.label.button.special
$neutral.5
Border radius
$borderRadius.small
16px
$borderRadius.medium
20px
Typography - MAPS
Category
Typeface
Size
Weight
Case
Line ht.
Letter spacing
Heading 1
Helevetica Neue
24
Bold
Sentence
120%
0
Heading 2
Helevetica Neue
18
Bold
Sentence
120%
0
Heading header
Helevetica Neue
16
Bold
Sentence
120%
0
Body base
Helevetica Neue
14
Normal/Bold
Sentence
140%
0
Body small
Helevetica Neue
12
Normal/Bold
Sentence
140%
0
Button large
Helevetica Neue
16
Bold
Sentence
0%
0
Button base
Helevetica Neue
14
Bold
Sentence
0%
0
Design system
Next steps
Further testing regarding information validation
I would like to conduct further concept testing back at the airport with United Airlines employees. I would be curious to know what kinds of information from our solution they actually find useful or if they feel that it's excessive. This was our original plan for concept testing; however, the CrowdStrike outage limited our interactions, making us pivot strategies.
Edge cases and expanded scope of service
After additional validation of our idea, I would want to expand our solution to include certain details that fell out of scope initially. Specifically, I would want to explore the accessibility aspect for users without smartphones, ensuring this solution remains inclusive for all customer types.
Reflection
Prioritizing key metrics
Due to the nature of this project, my team mostly focused on the ideation and storyboarding phases of the project. If I were to redo this project, I would focus on identifying success metrics earlier on in the process so I can leverage them later when evaluating our concept.
Moving forward with assumptions
Because we had limited access to United Airlines research metrics and design systems, we moved forward with our own research insights, focusing on keeping project momentum to meet tight deadlines.
Frequently practicing “vision massaging”
Frequent communication, both internal and external, revealed how specific details resonated with different stakeholders. This helped us distinguish feedback between how we presented our idea versus the idea itself.