Current Research

Multimedia and Multi-modal Computer Mediated Communication

OUTLINE
We are promoting research on (1) computer mediated communication, (2) image processing and multimodal interfaces, and (3) wearable and ubiquitous computing for the future era of a computer-symbiotic society. Human-to-human communication is indispensable to any collaborative creative work or productive work. Communication plays an important role in our social activity, too. However, communication is not an easy task, especially in conveying our intentions clearly and in understanding the ideas of other people. Our aim in this research is to use computers as a medium for communication and a supportive tool for each step of communication. This involves recording experience and activity, summarizing it, presenting it in a comprehensive way, and thus reaching mutual understanding. The following three core themes are being tackled.
(1) Media Technology to record, to summarize and to share experiences
We have prototyped a meeting capturing system, a wearable video-blog tool, and a system for capturing sports plays and supporting team discussion. These systems support summarization of events, assist memory recall, and segment the events for annotation with appropriate tags.
(2) Media Processing Technology for context recognition and understanding
Context is a very important attribute of an experienced event. Since the same event can be interpreted differently if context differs, appropriate context information must be attached at the time of memory and be accessible during search steps. We are researching a multi-modal context analysis method for doctor-patient interview dialogs as well as multi-modal interaction pattern mining.
(3) Ubiquitous systems and interface design technology
Based on the systematic recording and analysis of users' contexts and interactions in real-world activities, we can design and provide useful and usable ubiquitous systems. Such interactions are stored and accumulated as an interaction corpus. The interaction corpus will become a knowledge-based social infrastructure.
Our group also belongs to the Information Architecture Group, in the Information Technology Center (ITC). ITC is an R&D and service center of the campus IT services. For example, it has launched a university portal site to serve as a one-stop login for many applications such as course management systems. Furthermore, a universal identification (ID) is being issued to all members of the university. I have served as a working group leader of the MEXT 21^st COE program on Intelligent Media Integration (IMI) since 2002. I am also a project leader of a MEXT-funded project, cc-Society, that has been developing a Ubiquitous Leaning System, uClassroom, since 2004.
Future Work
My research themes are motivated by my background in motion-image processing for user interfaces, such as facial expression and gesture recognition. If we want to achieve better human-machine interface systems, we have to incorporate the capability of understanding user context in order to interpret gestures and events appropriately. Toward the future society, our IT systems will be developed symbiotically and include humanoid robots and wearable computers. These themes will establish future core technologies in practice and in theory.
URL:http://mase.itc.nagoya-u.ac.jp/~mase/