The Catchment Feature Model: A Device for Multimodal Fusion and a Bridge between Signal and Sense

• Main Author: Quek Francis
• Format: Article
• Language: English
• Published: SpringerOpen 2004-01-01
• Series: EURASIP Journal on Advances in Signal Processing
• Subjects: multimodal interaction; gesture interaction; multimodal communications; motion symmetries; gesture space use
• Online Access: http://dx.doi.org/10.1155/S1110865704405101

<p/> <p>The catchment feature model addresses two questions in the field of multimodal interaction: how we bridge video and audio processing with the realities of human multimodal communication, and how information from the different modes may be fused. We argue from a detailed literature review that gestural research has clustered around manipulative and semaphoric use of the hands, motivate the catchment feature model psycholinguistic research, and present the model. In contrast to &#147;whole gesture&#148; recognition, the catchment feature model applies a feature decomposition approach that facilitates cross-modal fusion at the level of discourse planning and conceptualization. We present our experimental framework for catchment feature-based research, cite three concrete examples of catchment features, and propose new directions of multimodal research based on the model.</p>