Comparative analysis of robot-assisted language learning systems and human tutors in English conversation lessons

Advancements in large language models, robotics, and software such as text-to-speech, have made it possible to develop robots that can understand language, interact physically, and communicate verbally. These breakthroughs have opened up possibilities for robots to be used for educational purposes. However, this raises the question of whether robots are as good as human tutors. While robots offer certain benefits, they cannot replicate the nuanced interactions and personalized feedback human tutors provide.

To determine the suitability of using robots for education, Associate Professor Takamasa Iio from Doshisha University, along with Associate Professor Yuichiro Yoshikawa, Professor Hiroshi Ishiguro from Osaka University, and Associate Professor Kohei Ogawa from Nagoya University, Japan, compared the performance of current Robot-Assisted Language Learning (RALL) systems to human tutors. Their study published in the journal International Journal of Social Robotics on April 11, 2024, explored the effectiveness of each approach in improving students’ English-speaking skills in second language learning. “There has been an active movement to utilize robots in education, particularly in language education, which requires communication. However, it was not clear what benefits there would be in utilizing robots. We began this study with the belief that in order to expand the use of robots in the future, it would be necessary to compare the discussion with the baseline of human tutors,” says Dr. Iio.

The researchers conducted an experiment involving 26 university students whose native language is Japanese. The students underwent a pre-test for an initial assessment of their English-speaking skills. Based on the average scores, the students were divided into two groups: 14 students received instruction from a robot, while the remaining 12 participants received online lessons from English language teachers. Over the course of seven days, both groups engaged in daily 30-minute sessions aimed at improving their English-speaking skills. The robot used was a table-top humanoid called CommU, which students could interact with using a tablet. This robot model could display human-like behaviors. It could nod to agree, slump to show thoughtfulness, and raise its hands to express joy or surprise.

In addition, there were three speaking exercises. The first involved roleplay, where the tutor (human or robot) and the student enacted a conversation in specific situations. The students listened to their scripts from the tutor and then repeated them. In the case of the robot, an audio file containing the speech of a native speaker was played instead of text-to-speech. The second exercise was flashcard practice, where students listened to their tutor deliver the script and repeated it to memorize it. In the final exercise, the students reenacted the conversation with their tutor using the memorized scripts. On the last day, participants underwent tests to assess their speaking errors, fluency, pronunciation, and speech complexity (number of words used in sentences).

All participants underwent a pre-test on the first day, followed by 30 minutes of daily study for 7 days, and three post-tests on the final day. The result indicated that the group taught by the robot made fewer errors and spoke more fluently than the group taught by human tutors. However, no significant difference was observed in outcomes between robots and human tutors for other aspects. The researchers believe that the improvement is because the students could practice the exercises with the robots much more than they could with the human tutor. This level of repetition improved their memory retention and speaking proficiency. Additionally, the expression of the robot may have reduced anxiety, allowing them to speak English without fear of judgment.

“Social robots could be used more in second language learning. They are likely to play an active role in repetitive practice aimed at consolidating basic words, phrases, and grammatical structures in memory,” highlights Dr. Iio.

While current RALL systems are sufficient to provide basic English training, the researchers predict that future systems will become more advanced. They may be able to recognize non-native speaker’s speech, offer corrections, and conduct interactive lessons and open-ended dialogues. However, since the fundamental purpose of language is communication with others, human tutors will remain essential for helping learners feel more comfortable and confident when using a new language in real-life situations.

About Associate Professor Takamasa Iio from Doshisha University, Japan
Dr. Takamasa Iio received the Ph.D. degree from Doshisha University, Kyoto, Japan, in 2012. He has worked at ATR, Osaka University, and the University of Tsukuba. Currently, he is an associate professor at Doshisha University, Kyoto, Japan. His field of expertise is social robotics. He explores the impact of human-robot interaction on cognition and behavior, alongside societal shifts. His current research focuses on social robots that can enhance social capital in human society by mediating relationships between people.

Funding information
This work was supported by JST Moonshot R&D Grant Number JPMJMS2011 (Analysis), JSPS KAKENHI Grant Number 19H05691 (Development of the robot system and the tablet interface), and the Benesse Corporation (Development of the learning materials and assessment of the tests).

Media contact:
Organization for Research Initiatives & Development
Doshisha University
Kyotanabe, Kyoto 610-0394, JAPAN
E-mail：jt-ura@mail.doshisha.ac.jp