Wide-Angle Virtual View Synthesis with Depth-Based Background Modeling

• Main Authors: Liu, Yu-Lun, 劉育綸
• Other Authors: Hang, Hsueh-Ming
• Format: Others
• Language: en_US
• Published: 2014
• Online Access: http://ndltd.ncl.edu.tw/handle/889ms7

博士 === 國立交通大學 === 電子工程學系 電子研究所 === 102 === The 3D virtual view synthesis systems is becoming popular for future multimedia applications. Therefore, the ITU/ISO MPEG joint international standard committee is currently working on the standards for the RGB-D format. Generating (synthesizing) a high quality pictures based on the given RGB and depth information is a very critical component in a 3D virtual view system. The conventional virtual view synthesis mainly focuses on how to synthesize a virtual view on the baseline aligned with two given views. However, in many multimedia applications like video composition, we need to synthesize a new view beyond the baseline to achieve a better subjective quality. But the wide-angle synthesis (with camera tilt and zoom in/out) generally results in many artifacts such as small cracks and large disocclusion regions. How to create a wide-angle virtual view with good subjective quality is our main purpose. To accomplish the aforementioned target, four techniques have been developed and presented in this thesis. They are fast backward depth warping algorithm, backward texture warping with bicubic interpolation, artifact reduction techniques, and depth-based background modeling. Backward depth warping algorithm mainly focuses on creating a warped depth map with nearly no quantization errors. We start from the target view by substituting every possible target depth candidate into the point warping equation and repeat the process until a depth value achieves the minimum difference (in coordinate and depth value) between the warped target view and the reference view. Finally, a depth map small quantization error can be obtained. Depth-based background modeling mainly focuses on creating a global background model of a video sequence using the depth maps together with the RGB pictures. The first key concept is the near objects block the scenes at the back. With the aid of depth information, we can identify the closer moving objects. Secondly, we develop a recursive algorithm that iterates between the depth maps and color pictures. Comparing to the existing schemes, our proposed method can produce better quality background images and it also improves the background depth map at the same time. All the above techniques have been tested on the MPEG test sequences. The results show that they can prove visible subjective quality improvement on the synthesized virtual view videos.