Current Research

Semi-automatic Annotation and Intelligent Application for Multimedia Content

OUTLINE
 Since content-based multimedia processing is extremely difficult, it is hoped that metadata annotation can be associated with multimedia content. Our aim is to be able to semantically structure all content and realize a human-machine interface to enable an intellectual and efficient access over the length and breadth of a piece of content's life cycle. Specifically, we are developing advanced annotation technology by integrating media information analysis based on signal processing and human cognitive processing. We are conducting research on basic speech/image/language processing, semi-automatic content structuring based on meta-information extraction maximizing human cognitive function, and intelligent application of media using annotation.
TOPICS
(1) Offline annotation
 Elemental technology needed for multimedia content annotation depends on the target content and the purpose of use. The important point is a proper balance between achieving the purpose of use and the cost of extracting and structuring necessary information. In the case of low reliability of extracted information owing to immature technology, a mechanism is required for choosing the most appropriate annotation method, considering applications of automatic analysis technologies and manual cost. Offline annotation analyzes content by various machine processings first, then manually modifies extracted information and associates it with meta-information. Though it will be preferable for all information included in content to be extracted by machine processing in the future, we think that a realistic solution is to properly allocate roles between humans and machines in their own game using the semi-automatic annotation method.
(2) Online annotation and community support
 The aim of online annotation is to realize annotation with less work by using many wired content readers with a minimum of machine processing. Online annotation associates content with information available for retrieval such as name and situation descriptions, as well as subjective evaluations such as brief comments and impressions. Accumulating and sharing such information makes it possible to contribute to the formation of a content-centered community.
(3) Semi-automatic annotation and browsing system for meeting content
 We aim to provide an efficient browsing environment for meeting content. To do so, we have been implementing a meeting system that semi-automatically acquires and structures real-world information such as audio-visual information, text information, metadata and so on. Differing from traditional minutes, meeting content includes multimodal information, not just text information, and we are researching a way to produce a seamless will or choice mechanism from among the available meeting information, and methods that allow users to access only needed parts of long-term content within a limited amount of time.
(4) Semantic transcoding
 The annotation-based content transformation is called"semantic transcoding"because we deal with deep semantic content. Semantic transcoding technology allows users to use content according to their preference and environment, based on annotations associated with the requested content.
FUTURE WORK
 Through improving computer performance, spreading broadband communication, and digital broadcasting, it is expected that use of multimedia content will grow rapidly in the future. In the meantime, there are definite temporal and spatial restrictions as well as cognitive limitations to our information processing activities. Therefore, it is important to create an environment in which we can obtain information needed, only the quantity needed, and in the requested format. While studying the above-mentioned themes, we plan to construct a mechanism for annotating multimedia content, and promote the practical use of technology to enable advanced applications of content such as information retrieval, summarization, and transformation.