|
Video completion is of great importance to many applications such as
video repairing, video editing and movie post-production. We propose a
novel technique to fill in missing
background and moving foreground of a video captured by
a static or moving camera. Different from previous work
which are based on processing in the 3D data volume, we
slice the volume along the motion manifold of the moving
object, and therefore reduce the search space from 3D
to 2D, while still preserving the spatial smoothness and
temporal coherence. In addition to the computational efficiency,
based on geometric video analysis, the proposed
approach is also able to handle real videos under perspective
distortion, as well as common camera motions, such
as panning, translation and zooming. The experimental results
demonstrate that our algorithm performs comparably
to 3D search based methods, while extending the current
state-of-the-art repairing techniques to videos with significant
camera motion and projective effects, as well as lighting
changes.
Some results are shown below by using our video completion method. In
these video, some
persons are removed, leaving the damaged background as well as some
damaged moving foreground, e.g., occluded person. Therefore, the
missing pixels in the background and foreground should be restored.
- Video completion under camera motion:
In this video, the camera is spanning,
and the closer person is removed. Pixels in the background and the
occluded person are recovered. In some frames of the original video,
the person is mostly occluded, e.g., frame 29, however is recovered
correctly in our result. The motion cue in the video is efficiently
used to predict the motion of the occluded person. Click
on images to enlarge.
Download the result video HERE.
- Video completion with perspective distortion:
When the foreground is moving from near to
far, there is perspective distortion in the volume of the foreground.
Therefore, affine rectification and metric
rectification should be applied to the volume before the video
completion. After the
foreground volume is completed, it is post-warpped back and combined
with
the background. The blur effect in the result comes from the
rectification and post-warpping. Click
on images to enlarge.
Download the result video HERE.
- Video completion with periodic illumination change:
Our method can also handle the video with
periodic illumination changes. In the video shown below, the global
illumination is changing due to whirling of the fan. The the walking
person is removed, and the occluded wall and ceiling, which are under
changing illumination, should be completed. Click
on images to enlarge.
Download the result video HERE.
|
