Inference of Conversation Partners by Acoustic Sensing in Smartphone Networks

• Main Authors: Chen, Ji, 陳驥
• Other Authors: Tseng, Yu-Chee
• Format: Others
• Language: en_US
• Published: 2014
• Online Access: http://ndltd.ncl.edu.tw/handle/66vakd

碩士 === 國立交通大學 === 網路工程研究所 === 102 === Recently, smartphones provide a promising platform for cooperative computation and inference among social groups. Microphone is a common sensor on a smartphone that can be used for acoustic sensing. A lot of personal daily contexts and activities may be inferred by analyzing the recorded acoustic data. In this work, we consider the inference of conversation partners, which is an important context of social interaction. We design an efficient smartphone-based conversation inference system, which can quickly derive the conversation partners of a phone user in a real-time manner. By considering the continuity and overlap of speeches, we propose two novel inference methods to distinguish the conversational relationship among co-located speakers. Via direct wireless communications, smartphones cooperatively conduct speaker turn recognition by processing acoustic data. The speaker turn recognition of a conversation consists of when a user speaks and how long he/she speaks. Furthermore, to improve inference accuracy, our system also derives speakers’ emotion of utterances and uses the consistency of emotions of a conversation group to make inference. To evaluate our system, we collect conversation data from movie clips and real life with the number of speakers ranging from 2 to 9. The result shows that our system achieves promising performance in both quiet indoor and noisy outdoor environments. In addition, we have also demonstrated a prototype on Android smartphones to verify the feasibility of our approach from off-the-shelf devices.