Hello everyone. My name is Anara Sandygulova. I would like to present to you our work titled "CoWriting Kazakh: learning a new script with a robot".
In terms of task difficulty, children rated Latin-to-Latin condition as significantly harder than Cyrillic-to-Latin case, which raises a trade-off debate:
And the second hypothesis is It is more effective for the child to perform conversion of Kazakh words mentally and then observe correctly written Latin spelling by the robot.
So as you can see, the robot is writing the Latin-based Kazakh
The pre-test of the RoMan study had a potential confounding factor: when children made mistakes in the pre-test, they were shown the correct answers by the researcher. It provided a means of learning these letters from the pre-test.
In the scenario, the NAO robot plays a role of a peer.
while girls learned more in Latin-to-Latin condition (almost twice more letters in Latin-to-Latin condition).
The system aims to implement an autonomous social robot that would assist children in transition from the old Cyrillic alphabet to a new Latin alphabet.
The robot is introduced to a child as a native English speaker, of approximately his or her age, who wants to learn Kazakh.
This work was conducted at Nazarbayev University in collaboration with EPFL. My co-authors are Wafa Johal, Zhanel Zhexenova, Bolat Tleubayev, Aida Zhanatkyzy, Aizada Turarova, Zhansaule Telisheva, Anna CohenMiller, Thibault Asselborn, Pierre Dillenbourg.
4.06 in comparison to 3.13
The robot says: "Ok, look!"
Then, we calculated the learned letters as the pre- and post-tests difference. And we found that the average number of learned letters was 4.35.
whether it is worth making children work harder and perform mental conversion themselves when a learning gain for all children is not significantly different from the easier condition.
And we implemented two experimental conditions.
This handwriting recognition was trained on over 120,000 samples of the Cyrillic-MNIST dataset, which we collected for the first time for the Cyrillic-based languages.
And indeed when we looked at separate one-way ANOVA tests for each gender group, we found that girls had a significant difference
The difference between conditions is only in wording when the robot asks the child for help to demonstrate how to write Kazakh words using the new Latin alphabet because that is convenient for the robot to read.
We adapted original CoWriter to Kazakh language and also added OCR for automatic conversion of children’s handwriting.
The first condition is Cyrillic-to-Latin demonstration, while the second condition is Latin-to-Latin demonstration.
The time, we conducted the study with 67 children where all participants were assigned to a condition in a between-subject design, with a conversion type as a between-subject variable.
All participants were from the same school, so we cannot generalize and confidently say that the same result will be valid in other schools in Kazakhstan.
Then, we looked at H2. We can conclude that H2 is rejected since the learning gains of children were not affected by the robot condition.
In the first condition, its exactly the same it was in the demo video. The child hears the word, and she writes it in Cyrillic. Then the robot performs the script conversion by writing the same word using the Latin-based Kazakh alphabet. 23 00:00:48,590 --> 00:00:03,990 In Latin-to-Latin condition, the child hears the word to be written and has to write it directly in Latin. Then the robot writes the word in Latin as a corrective feedback. In this condition, the child needs to perform mentally the script conversion.
between robot conditions in comparison to boys who did not have significant difference, even though the difference was still quite big.
This experiment has raised a number of questions and further work is needed in several directions to adequately determine the best scenarios and strategies for learning a new script.
The motivation behind our work is a recent decision of Kazakhstan’s authorities to transition from Cyrillic to Latin alphabet which is a risky endeavor and raises many unique challenges and research questions.
I would like to show you a demo of the system
The Wacom tablet serves as the second monitor. Its pen has pressure sensitivity and tilt recognition. This allows it to acquire the trajectory of handwriting. A humanoid robot NAO is integrated with the tablet using ROS and NaoQi API.
And that's the motivation behind child's trying her best to write in Latin alphabet.
Hello. I am a robot. My name is Mimi
H1 is supported. H2 is rejected.
I study Kazakh language. Can you help me?
Each child interacted with a robot for approximately 20 minutes. Half of the children interacted with the Latin-to-Latin robot condition, while the other half of the children interacted with the Cyrillic-to-Latin condition.
We verified with children’s English teacher that that the level of English understanding fits the vocabulary used by the robot.
However, we found interesting gender and robot condition effect.
We also did not find any significant gender differences: boys learned approximately the same number of letters as girls did.
Considering the need for the current generation of primary school children to be fluent in both scripts, this project aims to assist children via human-robot interaction in learning a new script and its associated handwriting. 5 00:00:58,920 --> 00:00:09,880 The CoWriting Kazakh HRI system is an extension of the existing CoWriter project that was developed at the CHILI lab in EPFL in 2014.
Then, we looked at the results of robot perception and Facial expression analysis
Future work will involve an acceptability and usability study to determine the value of the robot as an engaging peer learner.
How do you say "girl" in Kazakh? How do you write it? 12 00:00:49,810 --> 00:00:04,430 As can be seen from the demo, the system integrates OCR to recognize Kazakh handwritten letters to automatically convert the letters to Latin alphabet.
It can be explained by referring to the literature on gender studies, which often show that girls outperform boys in languages. 52 00:00:52,590 --> 00:00:01,990 Please refer to our paper for a through explanation about this effect.
We have developed two hypotheses. 18 00:00:50,640 --> 00:00:04,090 The first hypothesis is The CoWriting Kazakh learning scenario will result in significant improvement in the number of learned letters, which will serve as a check that the provided intervention results in learning a new script.
The methodology of the experiment was designed and then verified in the prior work with 48 children published in RoMan 2019. 30 00:00:55,450 --> 00:00:06,290 This work was designed as a direct reproduction of the study, where the only difference between the two studies was in the way the pre- and post-tests were performed.
Future work will compare our system with classical approaches (pen and paper, tablet only, teacher, and others).
girls’ expressed happiness was significantly higher than boys’ expressed happiness. 46 00:00:55,110 --> 00:00:12,290 We also found that when children were asked how easy it was for children to learn from the robot: the rating of Cyrillic-to-Latin case was rated as significantly easier than Latin-to-Latin case.
We also had Ethical approval by the NU IREC committee. 35 00:00:57,480 --> 00:00:14,390 From the analysis of the results, we can conclude that H1 is supported, confirming the intervention with the system had a positive effect on the children’s performance in the pre- and post-tests at a highly significant level (p
However, we found a statistically significant interaction between the effects of gender and robot condition. 40 00:00:57,410 --> 00:00:07,890 As can be seen from the graph, Boys learned more in Cyrillic-to-Latin condition: almost five letters in comparison to 3.59 letters on average.
I would like to acknowledge our funding sources.
Thank you so much for your attention. Please email me for any questions.