Mutual-Structure for Joint Filtering

(Oral presentation in ICCV'15)

Xiaoyong Shen§     Chao Zhou§     Li Xu     Jiaya Jia§

§The Chinese Univeristy of Hong Kong       SenseTime Group Limited


An overview of our mutual-structure for joint filtering framework. Given image pairs in (a) and (b), our method can efficiently get the mutual-structure as shown in (d) and (h).


Previous joint/guided filters directly transfer the structural information in the reference image to the target one. In this paper, we first analyze its major drawback – that is, there may be completely different edges in the two images. Simply passing all patterns to the target could introduce significant errors. To address this issue, we propose the concept of mutual-structure, which refers to the structural information that is contained in both images and thus can be safely enhanced by joint filtering, and an untraditional objective function that can be efficiently optimized to yield mutual structure. Our method results in necessary and important edge preserving, which greatly benefits depth completion, optical flow estimation, image enhancement, stereo matching, to name a few.



RGB/depth Restoration

Download more results on dataset [1]
Rgb Input Noisy Depth Input Rgb Input Noisy Depth Input
Click buttons to show results made by different algorithms

Stereo Matching

Left Image Right Image Bilateral Filter (Error 1.81)
Guided Filter (Error 2.50) Tree Filtering (Error 1.58) Ours (Error 1.42)

Joint Structure Extration and Segmentation

Night image Day image
MCG [2] on Night Image MCG [2] on Mutual-Structure

Matching Outlier Detection

Reference Image Input Image Matched Image [3]
Naive Outlier Mutual-Structure Our Outlier

More results and applications can be found in our supplementary file here.


Snapshot for paper "Mutual-Structure for Joint Filtering"
Xiaoyong Shen, Chao Zhou, Li Xu, Jiaya Jia
IEEE International Conference on Computer Vision(ICCV), 2015

  [Paper (pdf, 3.30MB)] [BibTex]

  [Supplementary file (pdf, 5.90MB)]

 [Matlab Code (zip, 1.3MB)]



[1] S. Lu, X. Ren, and F. Liu. Depth enhancement via low-rank matrix completion. In CVPR, 2014.

[2] P. Arbelaez, J. Pont-Tuset, J. T. Barron, F. Marques, and J. Malik. Multiscale combinatorial grouping. In CVPR, 2014.

[3] L. Xu, J. Jia, and Y. Matsushita. Motion detail preserving optical flow estimation. IEEE Trans. Pattern Anal. Mach. Intell., 34(9):1744–1757, 2012.



Last update: Oct. 4, 2015