2nd Place in Alexa Day Competition
Feb. 2019 - Apr. 2019 (2 months)
VUI Design
Tortu / Lucid Chart / Alexa Dev Tools
Melissa Shi / Ally Liu / Qian Wang / Paopao Ge / Sijia Li / Menghan Zhang / Mengqing Jiang / Wei Wang
VUI Designer, Researcher
Lead the design of the conversation flow.
Responsible for handing over and communicating with the development team.
Conduct research following the research lead.
HMW help the visually impaired apply makeup more easily and independently?
Currently, there are 285 million people in the world who are visually impaired, with two-thirds of whom are female. It’s not easy for them to apply makeup on their own. However, they have been ignored and under-served in the current makeup industry. We believe, “Everyone has the right to wear makeup, be confident and beautiful.”
An Alexa Skill that checks the makeup application for users and provides guidance and tips
With the help of 4 girls with different kinds of visual impairment, we learned that the most challenging step of all is to check if things are well done. So we designed a voice user interface (VUI), an Alexa Skill, that can check users' makeup application and provides guidance and tips.
User Research
Learn from the industry
We researched current technologies in the makeup industry. Although we didn't find any product or services for people with visual impairments. We did found some makeup tutorials on Youtube where the blind blogger is demonstrating how she applies makeup. While we found her techniques amazingly good, we also watched her video again and again to observe and take note of how she does it.
Try it ourselves: eyes closed

To better empathize with the user, we tried applying makeups with our eyes closed. Things we found the hardest are:

1. Knowing how much of the product is poured out.
2. Knowing where the tip of the pen/brush is.
Contextual Interview: Turns out we were very wrong

We interviewed three people with different types of visual impairments (including blindness). We observed them doing their daily makeup routine, and learned:

Key insights

1. Unlike us, our users have no trouble knowing how much product is poured out or where the tip of the pen/brush is, they also know their faces very well that they can apply product at the exact area they want.

2. The No. 1 thing that even experienced users still need everyday from another person is to check if there is no stains etc.

3. Certain type of product containers/tools are easier to use than others

4. Users often prefer more neutral look because they can't easily check the face during the day, and neutral colors are less likely to be messed up after different activities. Another reason some mentioned is that she doesn't want to draw attention on the eyes because of the uncontrollable eye movement.

More online research
Based the insights, we did more online research about fixing issues, applying makeups, and picking products. We concluded a library of tips.
Design Process
We summarized two primary personas. One is an experienced makeup user who needs help mainly for the final check; the other is a novice user who needs more guidance and tips.
We brainstormed together and prioritized ideas from a cost/value matrix..
Without prior experience designing VUI, we prepared ourselves through implementing one
With no experience designing an Alexa Skill, how can we quickly learn about the capabilities and limitations of this platform? I first looked into design guidelines and development documentations, and then designed and built a simple Alexa Skill. This hands-on experience helped me get to know better the framework, useful resources, and best practices to design for it. I was also able to experiment the boundaries of its capabilities.
Dialogue Flow Design

For prototyping the conversational UI, we started with a whiteboard, markers, and post-its, and then digitized them as a flow of intents with Tortu and Lucid Chart. We broke down the flow into several parts and designed anchors to link them. After finalizing the flow, we wrote example utterances with slots for each intent.

We iterated the dialogues with multiple rounds of "Wizard-of-Oz" testing. The main iterating direction is to shorten the responses and make sure we don't provide more than 2 options at a time because user can only process and remember so much through hearing.

Final Product
Technical Solution
Key features
Check if the makeup is done well
Users can ask Beauty+ to check their current makeup at any time. Beauty+ will take a picture and use computer vision technologies to check if the product is blended well or if there is any unexpected color (e.g., not fully covered acne, mascara stains, eyebrow mistakes)
Tailored guidance and tips
Beauty+ can guide users step by step, and provide tailored tips when needed. We gathered 121 tips, and many of them are especially designed for people with no or limited vision.
There are 4 more ideas that we designed but didn't get to complete the implementation