The present paper introduces a new method for the pronunciation training system.
We believe that foreign language learners should know how to adjust their pronunciation, not an accent.
We will devide it into four parts to illustrate it.
Automatic speech recognition
plays a crucial role in facilitating the language learning system.
For instance,
the TOEFL tests provider applies an ASR system called 'SpeechRater' to score the speaking test part.
In this scenario,
the development of ASR mainly concentrate on scoring users' pronunciation.
However,
could improve the learners' pronunciation through encouraging self-correction by learning to adjust their articulation?
To sum up the problems,
there are three fundamental challenges in this research.
The available application in the market
only send the scores to learners without giving useful Information
about why getting the wrong education.
Therefore,
they don't know how to adjust the correct pronunciation by scores or speech signals.
Why are vowels important?
Vowels are found in every syllable of every word.
In phonetics,
every vowel can distinguish from the other vowel by
categorizing it with two properties
the tongue positions and lip shape.
Similarly,
this is why a photograph asks us to say a word like a vowel /i/
like cheese.
We propose a vowel articulation recognition system implementing on smartphones.
The system simultaneously analyze both speech and ultrasonic signal to capture the tongue position and lip shape.
Finally,
it can gives users instructions on how to adjust this pronunciation.
We take the phonetic rule as a priority task
to map the monophthong vowels
The system has five modules.
First, linear prediction coding is used to etimate formant frequency
to map to the tongue position's height and backness.
Tongue height and backness can be mapped to the vowel diagram
Second,
deriving the ultrasonic signal from the Doppler effect examines the lip shape of
rounded and unwanted vowels.
Finally,
we take the phonetic rules as a priority task to map the monopthong vowels to distinguish each vowel by
tongue and lip.
Finally,
we use machine learning model to extract features
from audible and inaudible signal in order to recognize vowel
Here is our user interface.
User can select which language or vowel to learn.
Our pronunciation system can give users instructions on
how to adjust their mispronunciation after user practice and record.
For example,
the feedback of system like please move your tongue to the front.
We implement a costume-make application into Huawei mate 9
which generates and records
ultrasonic signals for data collection.
In our experiment,
20 international students of the Chinese language
tested a prototype version over 1 week in our college.
The learners are encouraged to be aware of pronunciation
by self-adjusting their misplaced articulation.
The histogram summarizes the pronunciation accuracies
before and after they practiced with the system
90% of the students had their pronunciation accuracy improved
with a noticeable accuracy difference of 10.5.
The impact of following language learning and other types of vowel
is significant in future development protential.
We aim to explore to generate visual graphic feedback of the wrong articulation to the learner
for better understanding and more profound engagement as our prototype.
In aspects of system development,
we tend to design another module,
which could be enable more practical by extracting the initials,
finals, tones and other speech organs in multiple language.
In conclusion,
we designed a pronunciation training system that
can visualize monophthong vowels mispronunciation and speak articulation
according to phonetic rules of tone position and lip shape.
We acquire ultrasonic at 20 k Hz to capture fine-gain signal of the lip shape.
90% of the student have a significant improvement in pronunciation
after using the system with the feedback of tongue and lip.
At last, we proposed a new robust system or pronunciation training which improvement into a commercial, smartphone.
That's all thank you