You are invited to attend a lecture

By

Asya Aharoni

(MSc. student under the supervision of Prof. Yaroslavsky Leonid)

School of Electrical Engineering, Tel-Aviv University, Tel-Aviv 69978, Israel

Using Temporal and Spatial smoothing of Depth Map for improving 2D to 3D video conversion

The last decade showed rapid improvement in the area of 3D cinema and television technology. New TV sets with 3D display capabilities are released yearly and complex camera systems for 3D content capture are developed and used in the filming industry. But 3D content is still missing. Filming new content doesn't happen at the rate, at which it can be consumed. This motivates intensive research in the area of 2D to 3D video conversion.  The conversion methods range from manual to automatic.

When considering automatic conversion, either pictorial depth cues or motion parallax can be used for generating 3D content. Depth from motion takes advantage of the fact that we have a video sequence thus the inter-frame differences can be used for depth information extraction, since a sequence of frames from the same video scene can be considered as different views of the scene. Thus, if there is motion taking place in the video sequence it can be possible to find for each frame a pair, such that together they can be considered a stereo pair. Following, a depth map can be created from these frames. The depth map then can be used to synthesize a new stereo pair- using one of the original frames and a synthetic image, which then can be converted into some form of 3D representation (for example anaglyph, which is used in this study). This is a simple method of 2D to 3D video conversion. The main obstacle to its implementation and wide use is its vulnerability to image sensor’s noise and image occlusions, which cause irregularities in depth map estimation and worsens 3D visual quality.

This study proposes a way of improving the result of the above conversion method, by applying both spatial and temporal smoothing to the calculated depth maps. The smoothing is based on experimental findings that the resolving power of stereo vision with respect to depth maps is substantially lower than that with respect to images and on the assumption of low temporal variability of depth maps. Correspondingly, two smoothing methods are developed and tested in the present research: spatial (intra-frame) smoothing and temporal (inter-frame) smoothing. The developed methods showed good results on the video samples that were chose.

Wednesday, February 11, 2015, at 16:00

Room 101, Computer and Software Engineering building