Humanoid Reading Printed Piano Scores Techniques for Robots

• Main Authors: Kai-jay Lu, 盧凱傑
• Other Authors: Chin-shyurng Fahn
• Format: Others
• Language: en_US
• Published: 2009
• Online Access: http://ndltd.ncl.edu.tw/handle/07420692751019545757

碩士 === 國立臺灣科技大學 === 資訊工程系 === 97 === In this thesis, we present a novel technique of document analysis which is distinct from the traditional methods that recognize the images produced from a scanner. To realize this technique, we alternatively employ a camera sold in the market to capture the image of the document in front of the camera and recognize its content. For probing into the recent hot topics on the development of robots, especially in the field of entertainment robots, we apply this technique to recognize printed piano scores. We hope entertainment robots can read the printed piano scores that humans use actually and play the recognition results in real time. For the construction of such a robot vision system, we adopt the PTZ camera used in video surveillance. Although this kind of cameras does not have high resolution like a digital camera, it can pan, tilt, automatically focus, and zoom. Therefore, we may precisely manipulate these functionalities to simulate the human behavior of reading printed piano scores. The reading behavior we understand is not the same as the ordinary manners of document analysis which entirely process the content of an image; to the contrary, it looks at part of a document and recognizes the corresponding content simultaneously in a given reading sequence. To achieve the requirement of real-time reading actual printed piano scores, we implement the aforementioned behavior by use of the functionalities of the PTZ camera to capture and recognize an enlarged partial image properly. After that, we control the PTZ camera to the next position and execute the same actions repeatedly until each part of a printed piano score has been read. Because such a “recognizing while looking” manner is similar to the reading behavior of human being, we call it humanoid reading printed piano scores technique. In the aspect of recognizing printed piano scores, we design an optic music recognition (OMR) module to identify the music symbols within an enlarged partial image. This module is inserted and executed several times ingeniously during the process of the reading. In the experiments, we set the image resolution of pixels to evaluate both the detection and recognition results from the printed piano scores of 10 songs. The OMR module only costs 200~300 milliseconds. Moreover, our system can recognize and play the printed piano scores in real time and the recognition rate is over 87% on an average. To demonstrate the effectiveness of the system, we do the tests on reading and playing actual printed piano scores in three different working environments: an indoor room, an outdoor square, and a concert hall. The experimental results reveal that the performance of our system maintains the same level of recognizing and playing printed piano scores, almost not suffered from the influence of external factors.