Current Research

Integration of Semantic Processing in Cognition and Action

OUTLINE
 How does the brain understand　"meaning"? We have been promoting research into the semantic processing in human cognition and action, in which the integration of interactive information is indispensable. Our mission is to break through the impasse of how humans solve the discrepancy between cognition and action, i.e. how the brain manages to bridge between understanding and performance, the so called "mind-body problem."This is important because intention may lose its function if it has no connection with the communicative act.
TOPICS
(1) Semantic processing
 The unpacking of semantic processing is one of the hardest parts of understanding higher-order mental activities in human beings. The question itself has also been called the "symbol-grounding problem" : how do we manage to obtain meaning in daily life ? That general question remains unanswered because semantic processing is not a closed area in linguistic processing but rather consists of wide-ranging mental activities. Furthermore, it requires various types of research in areas such as memory, affection, and motor action. In order to rise to the challenge of the difficult question "what is meaning ?" we have been studying the following topics : reading of ideographic characters and mental lexicon, bilingualism, language and action, communicative gesture and speech, autobiographical memory with emotion, and imitation learning of action.
(2) Behavioral analysis and brain imaging
 As semantic processing varies in the different levels mentioned above, we have been conducting psychological experiments to measure both human performance and brain activities by using a brain imaging method. For instance, in order to comprehend the "meaning of meaning," we are using multi-channel NIRS (near-infrared spectroscopy) to analyze the relative changes in the concentration of oxy-hemoglobin in response to a presented stimulus, and in simultaneous measurements we are examining the cortical activation in participants. We suggest that the cortical study using NIRS detects proficiency in well-grounded skills, which is latent at the performance level.
(3) Interaction of verbal and nonverbal communication
 In classical categorizations, gesture is distinguished from verbal communication. Prevalent and well-accepted dichotomies, such as speech and gesture, language and thought, left and right hemispheres, lead us to a simple question. If the dichotomic concepts differ in each function, how do the sets of functions interact with each other and collaborate in the brain. Hoping to shed light on this question, we have inquired into the hidden binding problem : how do we bind language and action, specifically, speech and gesture ? Our goal is to understand semantic processing and make a bridge through established dichotomies using multiple analysis methods. In order to comprehend meaning, we believe that meaningful and meaningless material (stimulus) must be examined.
FUTURE WORK
 The foundation of learning is formed by the observation and imitation of actions performed by a teacher (demonstrator). We examine semantic processing in a pragmatic situation : How do students imitate their teacher's (demonstrator's) nursing action ? We measure and compare both performance and brain activities in students and their teacher. We focus on the differences between brain activity in novices and junior expert students compared with that of a teacher in order to assess whether the cerebral processes of action observation and imitation are modulated by the existing expertise and motor skill repertoire of the observer. Moreover, by focusing on imitation disability in Alzheimer's disease patients, we are developing a method for a simple bedside test composed of imitation tasks of meaningless movements. In addition, we have already studied imitation of action with an eye towards developing a tele-communication and diagnosis system using computers.