Inference of Segmented Color and Texture Description             

Jiaya Jia                

Chi-Keung Tang

Abstract ! A robust synthesis method is proposed to automatically infer missing color and texture information from a damaged 2D image by ND tensor voting (N >3). The same approach is generalized to range and 3D data in the presence of occlusion, missing data and noise. Our method translates texture information into an adaptive ND tensor, followed by a voting process that infers non-iteratively the optimal color values in the ND texture space. A two-step method is proposed. First, we perform segmentation based on insufficient geometry, color, and texture information in the input, and extrapolate partitioning boundaries by either 2D or 3D tensor voting to generate a complete segmentation for the input. Missing colors are synthesized using ND tensor voting in each segment. Different feature scales in the input are automatically adapted by our tensor scale analysis. Results on a variety of difficult inputs demonstrate the effectiveness of our tensor voting approach.

Publications:

BibTex:

@inproceedings{Jia2003repairing,
    author = {Jiaya Jia and Chi-Keung Tang},
    title = {Image Repairing: Robust Image Synthesis by Adaptive ND Tensor Voting},
    booktitle = {CVPR},
    year = {2003},
    pages = {I:
643-650}
}

@article{JIA2004repairing,
    author = {Jiaya Jia and Chi-Keung Tang},
   
title = {Inference of Segmented Color and Texture Description by Tensor Voting},
    journal = {
IEEE Transactions on Pattern Analysis and Machine Intelligence},
    volume = {26},
    year = {2004},
    number = {6},
    month = {June},
    pages = {771-786}

}


Static background repairing in single natural image:

Our method is automatic method to infer the missing pixels in images with the perseverance of the texture continuities. The damaged area can be large and with complex structure.

 

Input images Background Repaired background

 

 

 

 

 

 

 

 

 

 


3D structure inference:

Our method can also be applied in inferring the missing 3D structures through the voting scheme. The continuities and the local geometries can be faithfully maintained.

 


Background restoration in synthesizing novel views:

Using stereo images, we can separate the scenes into background and foreground according to the depth discontinuity automatically. However, it is difficult to interpolate or extrapolate new views using the two images because of the occlusion.  In our method, combining the ability of inferring both the depth values and the missing pixels, we can solve the occlusion problem and create completed new views.

 

  Synthetic views with/out background restoration Zoom-in view 1 Zoom-in view 2
without background restoration

 

with background restoration