| Summary: | 博士 === 國立臺灣大學 === 電子工程學研究所 === 99 === With the development of TV technology and the progress of broadcasting system, users have the ability to interact with the content provider and even watch the customized video contents. In this dissertation, a next-generation multimedia is proposed with versatile functions: video annotation, visual effect, scalable video and interactive video. For the current methods to present the multimedia, like 3D video, image-based rendering and graphics, can not appropriately implement the versatility of multimedia. Thus, we propose the Video-based Video Rendering, a method to render the video with natural images or videos, to implement the versatility of multimedia. Based on the framework of Video-based Video Rendering, novel applications, algorithms of video processing and architecture designs are presented.
One of the applications extended from Video-based Video Rendering is Tennis Video 2.0, which is a novel presentation method for sports videos with properties of Structure, Interactivity, and Scalability. By the new methods of video analysis---background construction, player segmentation and ball trajectory extraction, the video contents are fully annotated and separated into different layers. Next, we propose the video rendering to integrate the video contents, generate the customized videos, and provide scalable video in the semantic domain. The experiments show that video rendering can generate the videos with vivid players, seamless score insertion and more interesting viewing effects. In addition, the strategy search is a convenient way to search the favorite event by clicking the hitting patterns on the court. From the user studies, the scalable video with the proposed semantic scalability has better viewing quality than that compressed by Scalable Video Coding, and users think they are willing to watch a match video with the functions of Tennis Video 2.0. To the best of our knowledge, Tennis Video 2.0 is the first work to construct the framework of video applications including video annotation, video enrichment, content insertion and scalable video.
Another application of Video-based Video Rendering is Tennis Real Play, which is an interactive game constructed from match videos and is also a novel application combining the topics of video annotation, video rendering and interactive game. As techniques for player model creation, we propose a database normalization process and a 4-state-transition behavioral model of tennis players. For player rendering, we propose clip selection, smoothing transitions, and a framework combining a 3D model with video-based rendering. Experiments show that vivid rendering results can be generated with low computational requirements. Moreover, the player model can adequately record the ability and condition of a player, which can then be used to roughly predict the results of real tennis matches. User studies reveal that subjects identify with increased interaction, immersive experience, and enjoyment from playing Tennis Real Play.
The computation of applications of Video-based Video Rendering is heavy and has the real-time requirements, thus there needs a powerful processor for the computation of video rendering. As for the architecture design, we design the Video Rendering Engine, which is specific for the computation of video rendering and can be embedded in the TV system. The experimental results show that it has the reconfigurable architecture to process the computation of various applications: panorama, concentric mosaics, depth-image-based rendering, Tennis Video 2.0 and Tennis Real Play. Especially, it even has the higher computation ability than a Core2Due 2.83 CPU. It can say that the Video Rendering Engine brings the applications of Video-based Video Rendering into our daily life.
|
|---|
