Independent Redesign

Duolingo is a language-learning app and website that has over 500 million registered learners, according to a November 2020 article.

For 2 of my case studies at CareerFoundry, I conduced research on language-learning apps. So I used the research that I had already conducted as a lens to view for this UX case study for Duolingo's Italian-English dictionary for desktop.

Guiding Questions

• Is the dictionary usable for what learners are looking to accomplish on the page?

• Does the dictionary contain audio files of the words that are looked up by learners?

• Does the dictionary tell you the plural of singular Italian nouns, or vice versa?

• Does the dictionary tell you the gender of an Italian noun?

• What can be done better?

Guiding Questions from a Prior Usability Test on a Language-Learning App

• Does the dictionary contain example sentences of the word?

• Does the dictionary state the word's part of the speech, such as noun, verb, etc.?

Note

• This project was self-initiated, and it was not commissioned or implemented by Duolingo. I was not privy to any of the information, data, and business requirements that Duolingo designers had, so our design ideas may differ.

Redesign Outline

1. I added the IPA pronunciations, along with the audio files, for the singular and plural forms of a noun.

2. I let learners know both the singular and plural forms of a noun when either the singular or the plural form is searched.

3. I let learners know whether a noun has the masculine grammatical gender or the feminine grammatical gender in Italian because exceptions exist.

4. I let learners know the part of speech for the word that is searched.

A/B Test, Round 1

I ran the first round of unmoderated A/B testing from 11 January 2021 to 13 January 2021. Seventeen participants took part in it, with the majority of participants (65%) from the USA. I recruited the participants from CareerFoundry's Slack and on a Discord server on learning Italian.

Test Objectives

1. Find out how learners feel about the Duolingo Italian-English dictionary for desktop

2. Determine learner preferences regarding the positioning of my proposed design elements

Two designs were used for the first preference test:

1. Version 1, Duolingo's original design

2. Version 2, my proposed design

After choosing their preferred design, the participants were provided the option of answering the following 2 questions:

1. Why did you choose this design?

2. Is there anything else that you would like to add?

They were free to skip the questions.

Test Limitations

1. The participants may not had been active learners of Duolingo.

2. The participants may not had been learning Italian.

3. Participants may not had experience studying a language besides their home language.

4. I could not ask follow-up questions since the test was unmoderated.

For the next round of A/B testing

More participants preferred my proposed design than Duolingo's original design, so I am going to have my proposed design be the control in the next found of A/B testing,

I wonder what learners think about the font size in my proposed design, so font size is going to be a variable.

A/B Test, Round 2

I ran the second round of unmoderated A/B testing from 17 January 2022 to 27 January 2022. Thirty-three participants took part in it, with 39% of participants from the USA and 24% from Germany. I recruited the participants from CareerFoundry's Slack.

Test Objective

1. Determine learner preferences regarding font size of my proposed design elements

Two designs were used for the second round of preference test:

1. Version 2, the winner of the first round of preference testing

2. Version 3, like version 2 but with smaller font size for certain info

After choosing their preferred design, the participants were provided the option of answering the following question

1. Is there anything else that you would like to add?

They were free to skip the question.

Test Limitations

1. The participants may not had been active learners of Duolingo.

2. The participants may not had been learning Italian.

3. Participants may not had experience studying a language besides their home language.

4. I could not ask follow-up questions since the test was unmoderated.

For the next round of A/B testing

Since the results are so close to call, I am going to keep the winner of the first round of testing as it is except that I am going to decrease the font size for the part of speech, so there is less conflict with the word's translation, which is more important than the part of speech.

I am going to conduct the final found of testing on an edge case. In Italian, there are nouns, like the Italian word uova (English egg), in one gender while singular and in another gender when plural. I wonder what learners think about the placement of the word's gender information, so the placement is going to be a variable.

A/B Test, Round 3

I ran the third round of unmoderated A/B test on 31 January 2022. I recruited participants from CareerFoundry's Slack, and 3 participants had submitted their answers before I realized that there was a problem with this round of testing.

Two out of the 3 participants left comments that they found the gender's information confusing, and I realized that since they did not know that in Italian, certain nouns have 1 gender when singular and have a different gender when plural. Since the participants were extremely confused, the purpose of this round of testing was defeated.

I decided to stop it prematurely and to improve the test's wording before I conduct another round of testing.

Test Objectives

1. Determine learner preferences regarding the placement of a noun's gender

Two designs were used for the first preference test:

1. Version 4, the logical design from the previous designs

2. Version 5, like version 4 but with the noun's gender information on top between the IPA's pronunciation and information regarding singular vs. plural

After choosing their preferred design, the participants were provided the option of answering the following 2 questions:

1. Why did you choose this design?

2. Is there anything that you would like to add?

They were free to skip the questions.

Test Limitations

1. The participants were confused by the test noun's masculine singular/feminine plural information.

2. The participants may not had been active learners of Duolingo.

3. The participants may not had been learning Italian.

4. Participants may not had experience studying a language besides their home language.

5. I could not ask follow-up questions since the test was unmoderated.

For the next round of A/B testing

I decided to add the following sentences to onboard participants:

'In Italian, there are some nouns that are in one gender when singular and in another gender when plural. Regarding the information presented, which design do you prefer?'

A/B Test, Round 4

I then ran the fourth round of unmoderated A/B testing with the same designs and a short explanation that onboards the participants about the Italian language. The testing ran from 31 January 2022 to 2 February 2022. Seventeen participants took part in it, with 47% of participants from the USA. I recruited the participants from CareerFoundry's Slack.

Test Objective

1. Determine learner preferences regarding the placement of a noun's gender

Two designs were used for the second round of preference test:

1. Version 4, the logical design from the previous designs

2. Version 5, like version 4 but with the noun's gender information on top between the IPA's pronunciation and information regarding singular vs. plural

After choosing their preferred design, the participants were provided the option of answering the following question

1. Why did you choose this design?

2. Is there anything that you would like to add?

They were free to skip the questions.

Test Limitations

1. The participants may not had been active learners of Duolingo.

2. The participants may not had been learning Italian.

3. Participants may not had experience studying a language besides their home language.

4. I could not ask follow-up questions since the test was unmoderated.

For the final iteration

I am going to redesign the page for uccello (bird), so that the spacing matches that of the page for uovo (egg).

What have I learned?

• To make sure that participants for unmoderated A/B tests understand what they are looking at, do introduce unfamiliar concepts to them in the introduction of the tests.

My Recommendations

1. Add the IPA pronunciations and the audio files for the singular and plural forms of an Italian noun.

2. Let learners know both the singular and plural forms of a noun when either the singular or the plural form is searched.

3. Let learners know the grammatical gender of a noun's singular form and the grammatical gender of its plural form when either form is searched.

4. Let learners know the part of speech for the word that is searched.

Future Considerations

1. Duolingo uses the typeface DIN Next Rounded for body text. Since I did not have it during this case study, I used Nunito as a substitute, as recommended by Duolingo's design language for people who do not have DIN Next Rounded. To align more with Duolingo's design language, I would prefer to do future redesigns using the DIN Next Rounded.

2. Solely with participants who use Duolingo on a weekly basis to study Italian, I would like to conduct a round of preference testing using Duolingo's original design and my proposed design from the previous rounds of testing.

3. I would like to find out the reason for which the comments are stylized in 2 different ways on the page.

4. In the bottom section titled 'Related discussions', I would like to show more differentiation between the links and the comments, so learners can tell which link goes with which number of comments at a glance through colour and spacing without having to read. As of now, all the writing is left-indented with equal amounts of vertical spacing among the elements, making it to tell which link goes with which number of comments at a glance.

5. Also in the 'Related discussions' section, the links are inaccessible to learners who are colour-blind and/or visually-impaired. For more information, see my UI analysis on Duolingo's contact us page for desktop: https://shaychu.com/p/shaychu/03638bd1.

6. I would like to do redesigns for the Duolingo desktop dictionary for other languages, like Chinese, French, and Japanese. Each language has its particularities, so the dictionary for each of them should differ somewhat.