Eavesdrop AI.

A proposal for a personalized Duolingo feature that lets users tailor lesson content to their learning goals by selecting conversation topics to eavesdrop on, followed by listening comprehension questions.

Project type: Adding a feature to an existing product

Duration: 5 weeks

Project role: UX Research, UX Design

Gimme that TLDR

We all have different motivations when we’re learning a new language, but existing Duolingo lessons projected a universality that does not exist among users. Duolingo acknowledged this in a 2021 article. 

My research showed users struggled to understand spoken language and found Duolingo was doing little to improve those skills. The app taught irrelevant vocabulary, like “dragon” and “princess,” instead words relevant to their unique needs and learning motivations, such as workplace communication.

So I designed Eavesdrop AI: a way to tailor a lesson’s subject matter to match a user’s unique learning goals and motivations.

Scroll for process

*

Scroll for process *

A perceived problema

I’ve used Duolingo everyday for over a year to learn Spanish, but I realized quickly that I wasn’t learning vocabulary that was useful to me. Contrary to what Duo thought, I had no plans to ride horses in L.A. anytime soon, but I can say yo monto mi caballo should the opportunity arise.

At the time of this project, there was no way for me to tailor the content of my lessons. I was stuck with whatever was given to me, whether or not I rode horses.

I wanted to explore how to maximize time spent on Duolingo, so that users, like myself, can feel like they’re spending time constructively while using the app, instead of wasting their time and dropping Duo altogether. 

Was I the only one becoming frustrated and bored with Duolingo? Was I alone in struggling to maintain my streak?

Research

How do we address a perceived gap, in personalization in order to increase trust and retention?

To find out, I needed to test my hypothesis and identify the right user frustrations.

I interviewed 5 people to learn:

  1. Do people indeed learn with a particular motivation and goal in mind?

  2. Is there a perceived gap in personalization?

  3. What drives people to pay for Duolingo, drop their subscription, or never subscribe at all?

  4. What Duolingo features are working well for people? Why?

  5. What makes users feel like they’ve successfully progressed in learning a new language?

Hypothesis confirmed - I was not alone.

The main pain point was twofold:

Users were frustrated that, despite long-term use, they struggled with verbal and auditory comprehension, and the lesson content felt irrelevant to their learning goals.

I confirmed that users had specific, unmet learning goals that Duolingo didn’t address.

0/5 users were aware of any personalization features.

5/5 agreed that verbal and auditory comprehension was the best indicator of language improvement, but felt Duolingo lacked sufficient practice opportunities in those areas.

Bonus insights!

Any new feature had to be integrated into the main lesson flow, not as a side quest or hidden in another tab.

  • 4/5 users said they rarely stray from the main screen or linear lessons to explore extra features.

  • Even paid users weren’t subscribing for premium features—they were simply avoiding ads.

Duo’s competition had little to be desired in terms of customization.

Of the two options offering some level of customization, Rosetta Stone content was repetitive and boring, not to mention pricey, and Memrise had quality assurance gaps in their user-generated lessons and flashcards. Babbel didn’t have any customization at all.

View an in-depth analysis here

¿Cuál es el problema?

I wrapped up my learnings into a persona to keep our problem statement user centric - meet Hillary.

Hillary’s main frustrations are:

  1. Inability to tailor content of the lessons to fit her unique learning motivations and

  2. Lack of relevant audio and verbal comprehension practice

How might we help Hillary improve her audio and verbal comprehension in a new language, while addressing her unique learning motivations?

Evaluating Ideas

What feature can be added? And can I capitalize on Duolingo’s existing content repository? I had a few ideas, and evaluated each, weighing the trade-offs with the benefits:

I moved forward with Option 4: Eavesdrop. 

Eavesdrop puts users in control of the topic they want to listen to people talk about, at their current comprehension level.

Pros:

  • Addresses audio comprehension concern

  • Addresses unique personalized content concern

  • Potential to bring back a newly revamped version of a feature long-time users miss

Cons:

  • Could be too expensive to remain in regular subscription tier

  • Similar feature of selecting from existing stories in Duo library has been previously deleted

Drawing it out

When sketching ideas, I referred to Duolingo’s brand guidelines and followed existing UX patterns so that the feature would be easy for users to grasp and use.

The core idea was for users to prompt Duolingo to generate a conversation on a topic of their choice to 'eavesdrop' on.

The conversation would play like a 'Stories' lesson, pausing periodically to test listening comprehension.

I sketched out 4 different ways the feature could flow to test find the optimal flow for the user:

1: Text only by default, no option to hide text

2: Audio only by default, no option to hide text

3: Audio by default, with option to toggle text visibility on/off

4: Text by default, with option to toggle text visibility on/off

Testing the Prototype

Goals of the test:

  • Determine user preferences for: (1) displaying text by default, (2) hiding text by default, or (3) allowing users to toggle text on and off.

  • Do users understand the feature’s purpose from its name?

  • Are the instructions clear?

  • Can they complete a prompt?

  • Can they successfully toggle the text display?

  • Do they feel comfortable and familiar with the flow?

4/5 users preferred the audio forward experience, with the ability to toggle text visibility.

They felt this put them in the right starting position to challenge themselves aurally, with visual support if needed.

Users immediately understood the flow, and were delighted when they could pick a topic they wanted to listen to and learn from.

User flows can be checked out here.

Additional feedback and iterations:

El producto final

Epilogue

Duo’s premiers a new lesson type:

A few months after completing this project, I noticed a new little icon in my Duolingo lesson path…headphones.

Actual Duolingo screens

Duolingo added “DuoRadio” - a listening comprehension lesson to the main path, validating my research.

DuoRadio addresses listening comprehension practice AND the importance that it be a part of the main path instead of behind a paywall. With the exception of personalization, this WAS Eavesdrop AI.

Eavesdrop AI vs. DuoRadio

DuoRadio:

  • Personalization Issue: DuoRadio lacks personalization, preventing users from tailoring content to their specific learning goals. This is likely to remain a major pain point for users. Many users may be unwilling to upgrade to Duolingo Max as a result.

Eavesdrop AI vs. Duolingo Max’s “Roleplay”

Duolingo Max & Roleplay:

Duolingo Max includes “Roleplay,” an AI-powered conversational tool, which aligns with one of my earlier feature ideas. However, there are differences and downsides to it:

  • Pricing: Significantly higher than Super Duolingo, though competitive with other language learning tools in the market.

  • Personalization Issue: Although it offers personalization, it is treated as a “side quest,” separate from the main learning path. This contradicts my research showing that features outside the main path are often neglected by users.

  • Subscription Integration: Duo Max is a totally different subscription, while Eavesdrop is designed to fit within the existing Freemium and Premium subscription tiers.

A year and a half after it was announced, Duolingo Max is now being slowly rolled out to users. I imagine Duolingo Max will continue to evolve with advancements in ChatGPT. Hopefully, as ChatGPT becomes cheaper to use, Duolingo Max will be accessible to more users.

I am unabashedly proud of myself. My research and Duolingo’s research came to almost all the same conclusions not once, but twice, resulting in DuoRadio & Duolingo Max.

The fact that Duolingo and I came to the same conclusions shows that design is so often a science, in addition to being an artform.