Author: Roy Stripling
Disclaimers: This in an opinion piece. The opinions expressed in this review are those of its author (and anyone who happens to agree with him). Other than completing some online courses, the author is not in any way affiliated with DuoLingo.
Note: Images in this post may contain copyrighted material used under the educational and criticism fair use exemptions to the U.S. copyright law.
Disclaimers: This in an opinion piece. The opinions expressed in this review are those of its author (and anyone who happens to agree with him). Other than completing some online courses, the author is not in any way affiliated with DuoLingo.
Note: Images in this post may contain copyrighted material used under the educational and criticism fair use exemptions to the U.S. copyright law.
Duolingo user page
Duolingo lesson page. Gamification is the application of various game features towards non-game uses. It can include using anything you find in a game like scoring points and keeping track of high scores, extra lives, leveling up, unlocking features, using sound effects or music, first-person animations, and so on. In educational applications, gamification is often touted as a way to increase learner interest, engagement, effort, and "stickiness." On the other hand, most educational researchers would caution you against adding bells and whistles that aren't central to your learning objectives. So is gamification good or bad? The research method is an excellent tool for separating what really works from what doesn't, but it is not a fast process. In educational research in particular, you can't really feel confident about conclusions until they've been replicated by several independent studies and labs. So while we wait for researchers to weigh in, we may still be able to learn something by looking at one very successful educational tool that makes use of gamification and consider what it might be doing right (and wrong).
Duolingo is relatively new online/mobile tool for learning a second language that is rapidly growing in popularity. It was selected as Apple's 2013 App of the year, and, according to its Wikipedia page, in a little over two years it went from 300,000 to 12.5 million active users (as of January 2014). Duolingo is free to its learners, which almost certainly helps with it's rapid growth and popularity. But if it didn't work to its learners satisfaction, they would likely not keep coming back. So what is Duolingo doing right?
What are they doing?
First, let's look at how it is structured. I'll focus on its Spanish lessons for Engish speakers with the assumption that Duolingo uses all the same methods and features for its other language courses. Duolingo provides a modest course that includes 64 units, each consisting of a handful of lessons (3-5 typically). When the user selects a unit, they're show the lessons for that unit and are presented the only formal instructional discourse they'll encounter. Typically, this is a short set of tips (if present at all) that are pertinent to the material about to be covered (for example, the difference between formal and informal forms of the word "you"). Each lesson generally contains 15-20 problems for the learner to solve. A problem may ask the learner to translate a word (at lower levels), phrase, or sentence from Spanish to English, from English to Spanish, to transcribe a spoken Spanish phrase or sentence, to repeat a spoken Spanish phrase or sentence. If the learner gets the answer right, they hear a pleasant chime (gamification element: use of sound effect) and they see a visual confirmation (a check mark and "you are correct" message). If they get the answer wrong, they hear a different and slightly less pleasant chime and are shown a red "X" and the correct answer. In either case, they see a grey square in the progress bar at the top of the screen turn green, and they're given access to a discussion thread on that problem. The conversation in the thread may include back and forth on why the answer is what it is, what other answers might also be correct, or it might just discuss something odd about the sentence itself. After the learner is done with the problem and any exploration of the discussion thread they care to take, he or she clicks a button to move to the next problem. Also at the top of the screen are three hearts (gamification element: lives). If you get a problem wrong, you lose a heart. If you lose all of your hearts, the lesson ends and you're not given credit for completing it. (You can retake it from the beginning right away.) If you successfully finish the lesson, you earn points (gamification element: scoring points) which your friends can see (gamification element: competition/high score list). If you finish the lesson without losing any hearts, you earn a "lingot", which you can trade for other items including replacing a lost heart in a later lesson (gamification element: earning "coins" that can be exchanged for other game elements).
On the learner's home page, the course units are organized into a cascading tree of topics, each represented by medalion like icons (the medallion for the Animals unit depicts a whale, while the medallion for the Food unit depicts a hamburger). Rows in the cascading tree imply levels as there are anywhere from one to three units per row and each row is unlocked when all of the units from the previous row are completed (gamification element: leveling up).
Does this work?
Duolingo summarizes one formal evaluation carried out by Roumen Vesselinov (Queens College, City University of New York) and John Grego (University of South Carolina), which found that 34 hours of time spent on Duolingo was equivalent of one semester of an introductory college Spanish language course. My personal experience with Duolingo's Spanish course (having completed about 1/3 of it) is that it is effective at increasing written language comprehension, but spoken language comprehension is not as well supported. Language is used in multiple ways, so being able to read and write (at a very introductory level) is an improvement worth feeling good about. But I'm not seeing much improvement in my ability to understand what's being said on the Spanish channels (even when I tried to watch the Spanish version of Sesame Street), nor do I feel confident enough in my skills to start talking in Spanish with my Spanish-fluent friends. But practicing conversational skill is exactly what it takes to become competent at it, and this is probably the biggest feature that Duolingo is missing.
Yeah, yeah, but what about the gamification part? Does THAT work?
Overall I would say yes. One game feature that had initial appeal, but that quickly became less important to me was scoring points for each lesson completed. I think this quickly became meaningless, because I'm not in this to "win" any point-based competition. If I were a more competitive person, and I was sharing my scores with other friends who I was determined to beat, I might be distracted from the lessons by the drive to accumulate points. Rather than spend time reading the tips at the beginning of the lessons, or reviewing the discussion threads for still more enlightening tips from others, I would be inclined to charge forward or repeat lessons until I managed to just pass them, then never look back -- learning be damned. For me, the point scoring feature is harmless because I don't care about it and it is easy to ignore, but if I did care, it could undermine my progress. I've seen other learning games that offered extra points for quick responses, even though one of the learning objectives was to teach the learners not to make hasty decisions. I think the message here is don't include game features that conflict with or could distract from your learning objectives.
Most of the other game features are used effectively, however. I find earning lingots to be motivating because I can use them to buy an extra heart/life. There have been times when my phone's auto-correct changed my right answer to a wrong one just before I submitted it. Having a way to get my heart back makes this kind of experience much less frustrating. I also find the three hearts/lives motivating as well. It adds some pressure -- I become especially diligent near the end of each lesson as I don't want to lose the lesson at the very end -- without adding too much pressure. After all, it only takes about 3 minutes to repeat the lesson if I do lose it. Leveling up in Duolingo also provides some sense of motivating satisfaction. It feels great to see the progress I've made, especially when I know that it represents an actual real-world skill that exists beyond just this game. This does make me want to stick with the program just a little more. I even find the feedback chimes and "X" or checkmarks motivating. They are immediate and short, so not at all intrusive, while providing meaningful feedback (especially when combined with the presentation of the correct answer if I did not provide it in the first place.
What could they do better?
With regards to gamification, I think they've pretty much nailed it. If I worked at Duolingo, I would push for minimizing the scoring aspect. They're doing a great job at keeping the interface uncluttered. The problem screens in each lesson are almost blank, except for the actual problem and answer field. There is no background music playing, and no characters added to the screen to add charm or color. The screen only contains the material to be learned, the progress bar, your hearts (which tell you what margin of error you have left in this lesson), a button to check your answer, and a "quit" link to get out of the lesson if needed. For some time educational researchers have been saying that less is more when it comes to online and multimedia learning, and Duolingo got the message.
With regards to learning effectiveness, I think the one thing they really could do better is create opportunities for conversational practice. If I worked at Duolingo, I'd test the effectiveness of adding a chat feature that paired Spanish speakers who wanted to learn English with English speakers who wanted to learn Spanish. They could offer it as a reward for completing a level or make it an earned and purchased experience bought with a handful of lingots. With 12.5 million active learners, there is probably almost always a large number of people taking lessons at any given time. Why not cover one of your biggest weaknesses, while give them a chance to get into the language teaching game, too?
Duolingo is relatively new online/mobile tool for learning a second language that is rapidly growing in popularity. It was selected as Apple's 2013 App of the year, and, according to its Wikipedia page, in a little over two years it went from 300,000 to 12.5 million active users (as of January 2014). Duolingo is free to its learners, which almost certainly helps with it's rapid growth and popularity. But if it didn't work to its learners satisfaction, they would likely not keep coming back. So what is Duolingo doing right?
What are they doing?
First, let's look at how it is structured. I'll focus on its Spanish lessons for Engish speakers with the assumption that Duolingo uses all the same methods and features for its other language courses. Duolingo provides a modest course that includes 64 units, each consisting of a handful of lessons (3-5 typically). When the user selects a unit, they're show the lessons for that unit and are presented the only formal instructional discourse they'll encounter. Typically, this is a short set of tips (if present at all) that are pertinent to the material about to be covered (for example, the difference between formal and informal forms of the word "you"). Each lesson generally contains 15-20 problems for the learner to solve. A problem may ask the learner to translate a word (at lower levels), phrase, or sentence from Spanish to English, from English to Spanish, to transcribe a spoken Spanish phrase or sentence, to repeat a spoken Spanish phrase or sentence. If the learner gets the answer right, they hear a pleasant chime (gamification element: use of sound effect) and they see a visual confirmation (a check mark and "you are correct" message). If they get the answer wrong, they hear a different and slightly less pleasant chime and are shown a red "X" and the correct answer. In either case, they see a grey square in the progress bar at the top of the screen turn green, and they're given access to a discussion thread on that problem. The conversation in the thread may include back and forth on why the answer is what it is, what other answers might also be correct, or it might just discuss something odd about the sentence itself. After the learner is done with the problem and any exploration of the discussion thread they care to take, he or she clicks a button to move to the next problem. Also at the top of the screen are three hearts (gamification element: lives). If you get a problem wrong, you lose a heart. If you lose all of your hearts, the lesson ends and you're not given credit for completing it. (You can retake it from the beginning right away.) If you successfully finish the lesson, you earn points (gamification element: scoring points) which your friends can see (gamification element: competition/high score list). If you finish the lesson without losing any hearts, you earn a "lingot", which you can trade for other items including replacing a lost heart in a later lesson (gamification element: earning "coins" that can be exchanged for other game elements).
On the learner's home page, the course units are organized into a cascading tree of topics, each represented by medalion like icons (the medallion for the Animals unit depicts a whale, while the medallion for the Food unit depicts a hamburger). Rows in the cascading tree imply levels as there are anywhere from one to three units per row and each row is unlocked when all of the units from the previous row are completed (gamification element: leveling up).
Does this work?
Duolingo summarizes one formal evaluation carried out by Roumen Vesselinov (Queens College, City University of New York) and John Grego (University of South Carolina), which found that 34 hours of time spent on Duolingo was equivalent of one semester of an introductory college Spanish language course. My personal experience with Duolingo's Spanish course (having completed about 1/3 of it) is that it is effective at increasing written language comprehension, but spoken language comprehension is not as well supported. Language is used in multiple ways, so being able to read and write (at a very introductory level) is an improvement worth feeling good about. But I'm not seeing much improvement in my ability to understand what's being said on the Spanish channels (even when I tried to watch the Spanish version of Sesame Street), nor do I feel confident enough in my skills to start talking in Spanish with my Spanish-fluent friends. But practicing conversational skill is exactly what it takes to become competent at it, and this is probably the biggest feature that Duolingo is missing.
Yeah, yeah, but what about the gamification part? Does THAT work?
Overall I would say yes. One game feature that had initial appeal, but that quickly became less important to me was scoring points for each lesson completed. I think this quickly became meaningless, because I'm not in this to "win" any point-based competition. If I were a more competitive person, and I was sharing my scores with other friends who I was determined to beat, I might be distracted from the lessons by the drive to accumulate points. Rather than spend time reading the tips at the beginning of the lessons, or reviewing the discussion threads for still more enlightening tips from others, I would be inclined to charge forward or repeat lessons until I managed to just pass them, then never look back -- learning be damned. For me, the point scoring feature is harmless because I don't care about it and it is easy to ignore, but if I did care, it could undermine my progress. I've seen other learning games that offered extra points for quick responses, even though one of the learning objectives was to teach the learners not to make hasty decisions. I think the message here is don't include game features that conflict with or could distract from your learning objectives.
Most of the other game features are used effectively, however. I find earning lingots to be motivating because I can use them to buy an extra heart/life. There have been times when my phone's auto-correct changed my right answer to a wrong one just before I submitted it. Having a way to get my heart back makes this kind of experience much less frustrating. I also find the three hearts/lives motivating as well. It adds some pressure -- I become especially diligent near the end of each lesson as I don't want to lose the lesson at the very end -- without adding too much pressure. After all, it only takes about 3 minutes to repeat the lesson if I do lose it. Leveling up in Duolingo also provides some sense of motivating satisfaction. It feels great to see the progress I've made, especially when I know that it represents an actual real-world skill that exists beyond just this game. This does make me want to stick with the program just a little more. I even find the feedback chimes and "X" or checkmarks motivating. They are immediate and short, so not at all intrusive, while providing meaningful feedback (especially when combined with the presentation of the correct answer if I did not provide it in the first place.
What could they do better?
With regards to gamification, I think they've pretty much nailed it. If I worked at Duolingo, I would push for minimizing the scoring aspect. They're doing a great job at keeping the interface uncluttered. The problem screens in each lesson are almost blank, except for the actual problem and answer field. There is no background music playing, and no characters added to the screen to add charm or color. The screen only contains the material to be learned, the progress bar, your hearts (which tell you what margin of error you have left in this lesson), a button to check your answer, and a "quit" link to get out of the lesson if needed. For some time educational researchers have been saying that less is more when it comes to online and multimedia learning, and Duolingo got the message.
With regards to learning effectiveness, I think the one thing they really could do better is create opportunities for conversational practice. If I worked at Duolingo, I'd test the effectiveness of adding a chat feature that paired Spanish speakers who wanted to learn English with English speakers who wanted to learn Spanish. They could offer it as a reward for completing a level or make it an earned and purchased experience bought with a handful of lingots. With 12.5 million active learners, there is probably almost always a large number of people taking lessons at any given time. Why not cover one of your biggest weaknesses, while give them a chance to get into the language teaching game, too?
RSS Feed