Using the variational approaches to estimate optical flow between two frames, the flow discontinuities between different motion fields are usually not distinguished even when an anisotropic diffusion operator is applied. In this paper, we propose a multi-cue driven adaptive bilateral filter to regularize the flow computation, which is able to achieve the smoothly varied optical flow field with highly desirable motion discontinuities. First, we separate the traditional one-step variational updating model into a two-step filtering-based updating model. Then, employing our occlusion detector, we reformulate the energy functional of optical flow estimation by explicitly introducing an occlusion term to balance the energy loss due to the occlusion or mismatches. Furthermore, based on the two-step updating framework, a novel multi-cue driven bilateral filter is proposed to substitute the original anisotropic diffusion process, and it is able to adaptively control the diffusion process according to the occlusion detection, image intensity dissimilarity, and motion dissimilarity. After applying our approach on various video sources (movie and TV) in the presence of occlusion, motion blurring, non-rigid deformation, and weak textureness, we generate a spatial-coherent flow field between each pair of input frames and detect more accurate flow discontinuities along the motion boundaries.

Related Publications:

Figure 1. The previous results. (a) The first input frame. (b) The second input frame. (c) The estimated optical flow using the traditional variational approach, where the flow of the weak-textured regions are dragged by the high gradient region boundaries. (d) The zoomed image from the blue box in (c). (e) The dense flow field shown in color coded fashion where it is easy to see the dragging around the high gradient boundaries. (f) The color code map where the color represents the orientation of the vector and brightness stands for its magnitude. Note: in (c) and (d), we also draw the flow vector using a line segment which starts from red and ends at green.

Figure 2. (a) The estimated optical flow of Fig.1 using our approach, where the flow of the weak textured regions are not dragged by the high gradient region boundaries any more. (b) The zoomed image from the blue box in (a). Compared to Fig.1.d, the flow vectors at the background region are not dragged by high gradient boundary any more. (c) Dense flow field. (d) The occluded areas in frame 1 (red regions).