Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space

• Main Author: Song, Miao
• Format: Others
• Published: 2012
• Online Access: http://spectrum.library.concordia.ca/975072/1/Song_PhD_S2013.pdf; Song, Miao <http://spectrum.library.concordia.ca/view/creators/Song=3AMiao=3A=3A.html> (2012) Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space. PhD thesis, Concordia University.

This major component of the research described in this thesis is 3D computer graphics, specifically the realistic physics-based softbody simulation and haptic responsive environments. Minor components include advanced human-computer interaction environments, non-linear documentary storytelling, and theatre performance. The journey of this research has been unusual because it requires a researcher with solid knowledge and background in multiple disciplines; who also has to be creative and sensitive in order to combine the possible areas into a new research direction. Thus, we summarize the innovative research work surrounding the topic as “Computer-Assisted Interactive Documentary and Performance Arts in Illimitable Space”. This work encompasses a lot of research performed in each of the disciplines. It focuses on the advanced computer graphics and emerges from experimental cinematic works and theatrical artistic practices. Some development content and installations are completed to prove and evaluate the described concepts and to be convincing. More specifically, on one hand, the major research component includes the advanced rendering in real-time of an interactive physics-based softbody object simulation and visualization with OpenGL. Its immediate follow-up work in this thesis extends that system onto an artistic interactive jellyfish simulation controlled with advanced interaction devices, such as Falcon haptics. Some more advanced rendering techniques have been applied, such as stereoscopic effects, GLSL, LOD etc. in order to bring the CG objects to life and to increase the level of realism and speed. On the other hand, the installation work, /Tangible Memories/ transforms the award-winning personal documentary film /I Still Remember/ [1], into a non-linear interactive audience-controlled piece using the new media technologies and devices. The audience and society have already appreciated the personal documentary /I Still Remember/ with the social values resulting in portrayal of immigration, divorce, reunification with the family, and other memories. Turning it into an interactive new media work makes it a much more profound and sensory in-depth storytelling approach that can be educational as well as help the audience to feel the story by interacting with it and making it available via many media sources. Moreover, the audience's participations and feedback are themselves well-preserved in the new "memory bubbles", so that the same documentary project could be eternal and ever evolving, which may determine the concept of tomorrow's documentary film production. Additionally, another audacious approach extended from /Tangible Memories/ installation is for theatrical practice with the same set of technical tools. Theater performers could use their body movements, gestures, and facial expressions to achieve the perceptual and emotional digital effects in sound and images dynamically. To summarize, the resulting work involves not only artistic creativity, but solving or combining technological hurdles in motion tracking, pattern recognition, force feedback control, etc., with the available documentary footage on film, video, or images, and text via a variety of devices (input and output, projection, stereoscopic viewing) and programming, and installing all the needed interfaces such that it all works in real-time. Thus, the contribution to the knowledge advancement is in solving these interfacing problems and the real-time aspects of the interaction that have uses in film industry, fashion industry, new age interactive theatre, computer games, and web-based technologies and services for entertainment and education. It also includes building up on this experience to integrate Kinect- and haptic-based interaction, artistic scenery rendering, and other forms of control. This research work connects all the research disciplines, seemingly disjoint fields of research, such as computer graphics, documentary film, interactive media, and theatre performance together.