Eavesdrop AI.

A proposal for a new personalized Duolingo feature that puts users in control of their lesson content to match their unique learning motivations, by requesting a conversation topic to eavesdrop on and test their listening comprehension skills by answering questions about.

Project type: Adding a feature to an existing product

Duration: 5 weeks

Project role: UX Research, UX Design

Gimme that TLDR

We all have different motivations when we’re learning a new language, but existing Duolingo lessons projected a universality that does not exist among users. Duolingo acknowledged this in a 2021 article. 

My research showed users struggled to understand spoken language and found Duolingo was doing little to improve those skills. The app taught irrelevant vocabulary, like “dragon” and “princess,” instead words relevant to their unique needs and learning motivations, such as workplace communication.

So I designed Eavesdrop AI: a way to tailor a lesson’s subject matter to match a user’s unique learning goals and motivations.

Scroll for process

*

Scroll for process *

A perceived problema

I’ve used Duolingo everyday for over a year to learn Spanish, but I realized quickly that I wasn’t learning vocabulary that was useful to me. Contrary to what Duo thought, I had no plans to ride horses in L.A. anytime soon, but I can say yo monto mi caballo should the opportunity arise.

At the time of this project, there was no way for me to tailor the content of my lessons. I was stuck with whatever was given to me, whether or not I rode horses.

I wanted to explore how to maximize time spent on Duolingo, so that users, like myself, can feel like they’re spending time constructively while using the app, instead of wasting their time and dropping Duo altogether. 

Was I the only one becoming frustrated and bored with Duolingo? Was I alone in struggling to maintain my streak?

Research

How do we address a perceived gap, in personalization in order to increase trust and retention?

To find out, I needed to test my hypothesis and identify the right user frustrations.

I interviewed 5 people to learn:

  1. Do people indeed learn with a particular motivation and goal in mind?

  2. Is there a perceived gap in personalization?

  3. What drives people to pay for Duolingo, drop their subscription, or never subscribe at all?

  4. What Duolingo features are working well for people? Why?

  5. What makes users feel like they’ve successfully progressed in learning a new language?

Hypothesis confirmed - I was not alone.

The primary pain point was twofold:

Users were frustrated that, despite long-term use, they couldn't understand the language verbally or aurally, and the lesson content was mostly irrelevant to their learning motivations.

I confirmed that users each had unique and narrowly defined learning motivations that were not being met by Duolingo.

0/5 users was aware that Duolingo had any kind of personalization.

5/5 users deemed verbal/audio comprehension as the best way to feel like they’ve gotten better at their language, but no one felt Duolingo provided enough ways to practice those skills.

Bonus insights!

Whatever the new feature would be, it had to be incorporated into the main lesson flow, not as a side quest or embedded in another tab.

  • 4/5 users said they do not veer away from the main screen and linear lessons to seek out additional features

  • Paid users were not even paying to gain access to premium features - they had no idea those were even there. They paid for subscriptions because they hated the disruption of ads.

Duo’s competition had little to be desired in terms of customization.

Of the two options offering some level of customization, Rosetta Stone content was repetitive and boring, not to mention pricey, and Memrise had quality assurance gaps in their user-generated lessons and flashcards. Babbel didn’t have any customization at all.

View an in-depth analysis here

¿Cuál es el problema?

I wrapped up my learnings into a persona to keep our problem statement user centric - meet Hillary.

If Hillary’s main frustrations are:

  1. Inability to tailor content of the lessons to fit her unique learning motivations and

  2. Lack of relevant audio and verbal comprehension practice…

How might we help Hillary practice and improve her audio and verbal comprehension skills in a new language in a way that addresses her personal, unique learning motivations?

Evaluating Ideas

What feature can be added? And can I capitalize on Duolingo’s existing content repository? I had a few ideas, and evaluated each, weighing the trade-offs with the benefits:

I moved forward with Option 4: Eavesdrop. 

Eavesdrop puts users in control of the topic they want to listen to people talk about, at their current comprehension level.

Pros:

  • Addresses audio comprehension concern

  • Addresses unique personalized content concern

  • Potential to bring back a newly revamped version of a feature long-time users miss

Cons:

  • Could be too expensive to remain in regular subscription tier

  • Similar feature of selecting from existing stories in Duo library has been previously deleted

Drawing it out

When sketching ideas, I referred to Duolingo’s brand guidelines and followed existing UX patterns so that the feature would be easy for users to grasp and use.

The basic idea was that users prompt Duolingo to create a conversation to “eavesdrop” on in a topic of their choice.

The conversation that follows plays in the same way the “Stories” lesson does, pausing after every few lines to test the user’s listening comprehension.

I sketched out 4 different ways the feature could flow to test find the optimal flow for the user:

1: Text only by default, no option to hide text

2: Audio only by default, no option to hide text

3: Audio by default, with option to toggle text visibility on/off

4: Text by default, with option to toggle text visibility on/off

Testing the Prototype

Goals of the test:

  • Determine user preference from (1) having the text on screen as default, (2) no text on screen as default, and (3) having the choice to toggle text on and off the screen.

  • Do users understand what the purpose of the feature is by its name?

  • Are the instructions clear?

  • Are they able to complete a prompt?

  • Can users successfully toggle text on or off the screen?

  • Do users feel familiar and comfortable with the flow?

4/5 users preferred the audio forward experience, with the ability to toggle text visibility.

They felt this put them in the right starting position to challenge themselves aurally, with visual support if needed.

Users immediately understood the flow, and were delighted when they could pick a topic they wanted to listen to and learn from.

User flows can be checked out here.

Additional feedback and iterations:

El producto final

Epilogue

Duo’s premiers a new lesson type:

A few months after completing this project, I noticed a new little icon in my Duolingo lesson path…headphones.

Actual Duolingo screens

Duolingo added “DuoRadio” - a listening comprehension lesson to the main path, validating my research.

DuoRadio addresses listening comprehension practice AND the importance that it be a part of the main path instead of behind a paywall. With the exception of personalization, this WAS Eavesdrop AI.

Eavesdrop AI vs. DuoRadio

DuoRadio does not enable users to tailor the radio content to the their unique learning motivations, which I believe will remain a pain-point for most users who will not want to pay for Duolingo Max.

Duolingo Max is priced significantly higher than Super Duolingo, though it is consistent with other language learning tools on the market. 

Eavesdrop AI vs. Duolingo Max’s “Roleplay”

One of Duolingo Max’s main features is “Roleplay,” an AI powered conversational tool - one of my feature ideas mentioned above.

While it addresses personalization, it’s featured as a “side quest,” separate from the main path. This directly contradicts my research that features outside the main path go largely untouched by users. 

Additionally, Eavesdrop is intended to sit within the existing Freemium and Premium subscription tiers.

A year and a half after it was announced, Duolingo Max is now being slowly rolled out to users. I imagine Duolingo Max will continue to evolve with advancements in ChatGPT. Hopefully, as ChatGPT becomes cheaper to use, Duolingo Max will be accessible to more users.

I am unabashedly proud of myself. My research and Duolingo’s research came to almost all the same conclusions not once, but twice, resulting in DuoRadio & Duolingo Max.

The fact that Duolingo and I came to the same conclusions shows that design is so often a science, in addition to being an artform.