Current Research

Speech/Spoken Language Processing

Almost all the humans use spoken dialog, which is the most natural communication method. If we can recognize/manage/synthesize speech in computers, this speech can be not only the best method of communication but can also be used as data storage media.
I am engaged in technologies on spoken language.
■Noisy speech recognition
Degradation of speech recognition performance is problematic in practical speech systems. Standard evaluation frameworks for noisy speech recognition are very useful for comparison of many noise reduction methods. I am the leader of the developers' group for the standardized evaluation framework series (CENSREC), which contains data, recognition tools, and evaluation tools, and the frameworks distributed freely in public.
■Large vocabulary continuous speech recognition
Making transcriptions of monologues such as lectures is a very promising research area. We improve acoustic modeling of the human voice using models such as the Hidden Markov Model (HMM) and statistical language modeling (N-gram). We also improve the decoding algorithm.
■Spoken dialog interface (1) -for a friendly interface-
The first impression of a spoken dialog system for novice users is that it is `impassive', because the time-lag between a human utterance and the system reply is too long and as such the user cannot distinguish whether or not the system works. This is one of the reasons why users do not feel that spoken dialog systems can be used in a comfortable, frendly manner.
Thus, we focus on prosodic features like timing and pitch change in a dialog. Our dialog system has begun to speak with appropriate prosodic features considering previous user utterances. When the dialog gets `lively,' the pitch of the system utterances chase the user's pitch.
On the other hand, we also study a semantic dialog strategy. We are now developing a robust and natural response generation method in a system that considers its own misunderstandings.
■Spoken dialog interface (2) -Automatically responding...-
A system that works only when the user would like it to... This system is always silently near the user, but when the user wants to use to talk to it, the system responds naturally and works. To realize such a system, various cues such as user orientation, change of speaking style, and contents of user utterance, etc., are employed.
■Multimodal interface
For a mobile information terminal, we are developing multimodal interfaces using speech/spoken dialog inputs. The combinations of speech, touch-pen, touch panel, and finger pointing, for example, are promising.
■Summarization/Indexing of spoken documents