More Web Proxy on the site http://driver.im/

research-article

Mathematical Models for Local Nontexture Inpaintings

Authors: Jianhong Shen, Tony F. ChanAuthors Info & Claims

SIAM Journal on Applied Mathematics, Volume 62, Issue 3

Pages 1019 - 1043

https://doi.org/10.1137/S0036139900368844

Published: 01 January 2002 Publication History

Abstract

Inspired by the recent work of Bertalmio et al. on digital inpaintings [SIGGRAPH 2000], we develop general mathematical models for local inpaintings of nontexture images. On smooth regions, inpaintings are connected to the harmonic and biharmonic extensions, and inpainting orders are analyzed. For inpaintings involving the recovery of edges, we study a variational model that is closely connected to the classical total variation (TV) denoising model of Rudin, Osher, and Fatemi [ Phys. D, 60 (1992), pp. 259--268]. Other models are also discussed based on the Mumford--Shah regularity [ Comm. Pure Appl. Math., XLII (1989), pp. 577--685] and curvature driven diffusions (CDD) of Chan and Shen [ J. Visual Comm. Image Rep., 12 (2001)]. The broad applications of the inpainting models are demonstrated through restoring scratched old photos, disocclusion in vision analysis, text removal, digital zooming, and edge-based image coding.

References

[1]

M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, Image inpainting, in Proceedings of SIGGRAPH 2000, New Orleans, LA, 2000.

[2]

P. Blomgren and T. F. Chan, Modular Solvers for Constrained Image Restoration Problems, CAM Report 97‐52, Department of Mathematics, UCLA, Los Angeles, CA, 1997.

[3]

Pierre Brémaud, Markov chains, Texts in Applied Mathematics, Vol. 31, Springer‐Verlag, 1999xviii+444, Gibbs fields, Monte Carlo simulation, and queues

[4]

Vicent Caselles, Jean‐Michel Morel, Catalina Sbert, An axiomatic approach to image interpolation, IEEE Trans. Image Process., 7 (1998), 376–386

Digital Library

[5]

Antonin Chambolle, Pierre‐Louis Lions, Image recovery via total variation minimization and related problems, Numer. Math., 76 (1997), 167–188

[6]

T. F. Chan, S.‐H. Kang, and J. Shen, Euler’s Elastica and Curvature‐based Image Inpaintings, CAM Report 01‐12, Department of Mathematics, UCLA, Los Angeles, CA, 2001;

available online at www.math.ucla.edu/applied/cam/index.html.

[7]

T. F. Chan, S.‐H. Kang, and J. Shen, Total variation denoising and enhancement of color images based on the CB and HSV color models, J. Visual Comm. Image Rep., 12 (2001).

[8]

Tony Chan, Pep Mulet, On the convergence of the lagged diffusivity fixed point method in total variation image restoration, SIAM J. Numer. Anal., 36 (1999), 354–367

Digital Library

[9]

T. F. Chan, S. Osher, and J. Shen, The digital TV filter and nonlinear denoising, IEEE Trans. Image Process., 10 (2001), pp. 231–241.

Digital Library

[10]

Tony Chan, Jianhong Shen, Variational restoration of nonflat image features: models and algorithms, SIAM J. Appl. Math., 61 (2000/01), 1338–1361

Digital Library

[11]

T. F. Chan and J. Shen, Non‐texture inpainting by curvature driven diffusions (CDD), J. Visual Comm. Image Rep., 12 (2001).

[12]

T. F. Chan and L. Vese, A Level‐Set Algorithm for Minimizing the Mumford‐Shah Functional in Image Processing, CAM Report 00‐13, Department of Mathematics, UCLA, Los Angeles, CA, 2000.

[13]

A. Cohen, W. Dahmen, I. Daubechies, and R. DeVore, Tree Approximation and Optimal Encoding, Bericht 174, Institut für Geometrie und Praktische Mathematik, Aachen, Germany, 1999.

[14]

Ingrid Daubechies, Ten lectures on wavelets, CBMS‐NSF Regional Conference Series in Applied Mathematics, Vol. 61, Society for Industrial and Applied Mathematics (SIAM), 1992xx+357

[15]

David Dobson, Curtis Vogel, Convergence of an iterative method for total variation denoising, SIAM J. Numer. Anal., 34 (1997), 1779–1791

Digital Library

[16]

D. L. Donoho, Curvelets, invited talk at the Workshop on Wavelets, Statistics, and Image Processing, Georgia Institute of Technology, Atlanta, GA, 1999;

invited talk at the MSRI workshop on Mathematics of Imaging, Berkeley, CA, 1999.

Beamlets, invited talk at the IMA Workshop on Image Processing and Low Level Vision, University of Minnesota, Minneapolis, MN, 2000.

[17]

G. Emile‐Male, The Restorer’s Handbook of Easel Painting, Van Nostrand Reinhold, New York, 1976.

[18]

S. Geman and D. Geman, Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Machine Intell., 6 (1984), pp. 721–741.

Digital Library

[19]

David Gilbarg, Neil Trudinger, Elliptic partial differential equations of second order, Springer‐Verlag, 1977x+401, Grundlehren der Mathematischen Wissenschaften, Vol. 224

[20]

Enrico Giusti, Minimal surfaces and functions of bounded variation, Monographs in Mathematics, Vol. 80, Birkhäuser Verlag, 1984xii+240

[21]

R. C. Gonzalez and R. E. Woods, Digital Image Processing, Addison‐Wesley, New York, 1992.

[22]

H. Igehy and L. Pereira, Image replacement through texture synthesis, in Proceedings of the 1997 IEEE International Conference on Image Processing.

[23]

K.‐H. Jung, J.‐H. Chang, and C. W. Lee, Error concealment technique using data for block‐based image coding, SPIE, 2308 (1994), pp. 1466–1477.

[24]

G. Kanizsa, Organization in Vision, Praeger, New York, 1979.

[25]

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, Detection of missing data in image sequences, IEEE Trans. Image Process., 11 (1995), pp. 1496–1508.

[26]

A. C. Kokaram, R. D. Morris, W. J. Fitzgerald, and P. J. W. Rayner, Interpolation of missing data in image sequences, IEEE Trans. Image Process., 11 (1995), pp. 1509–1519.

[27]

W. Kwok and H. Sun, Multidirectional interpolation for spatial error concealment, IEEE Trans. Consumer Electronics, 39 (1993).

[28]

S. Mallat, Geometrical Image Representations with Bandlets, invited talk at the IMA Workshop on Image Processing and Low Level Vision, University of Minnesota, Minneapolis, MN, 2000.

[29]

A. Marquina and S. Osher, A new time dependent model based on level set motion for nonlinear deblurring and noise removal, in Scale‐Space Theories in Computer Vision, Lecture Notes in Comput. Sci. 1682, M. Nielsen, P. Johansen, O. F. Olsen, and J. Weickert, eds., 1999, Springer‐Verlag, New York, pp. 429–434.

[30]

D. Marr and E. Hildreth, Theory of edge detection, Proc. Royal Soc. London B, 207 (1980), pp. 187–217.

[31]

S. Masnou and J.‐M. Morel, Level‐lines based disocclusion, Proceedings of the 5th IEEE International Conference on Image Processing, Chicago, IL, 1998, pp. 259–263.

[32]

Bart ter Haar Romeny, Geometry‐driven diffusion in computer vision, Computational Imaging and Vision, Vol. 1, Kluwer Academic Publishers, 1994xxii+439

[33]

D. Mumford, Empirical investigations into the statistics of clutter and the mathematical models it leads to, a lecture for the review of ARO, 1999;

also available online at www.dam.brown.edu/people/mumford/researc

\underline{h}

new.html.

[34]

David Mumford, Jayant Shah, Optimal approximations by piecewise smooth functions and associated variational problems, Comm. Pure Appl. Math., 42 (1989), 577–685

[35]

Rolf Nevanlinna, Analytic functions, Translated from the second German edition by Phillip Emig. Die Grundlehren der mathematischen Wissenschaften, Band 162, Springer‐Verlag, 1970viii+373

[36]

M. Nitzberg, D. Mumford, T. Shiota, Filtering, segmentation and depth, Lecture Notes in Computer Science, Vol. 662, Springer‐Verlag, 1993viii+143

Digital Library

[37]

Stanley Osher, James Sethian, Fronts propagating with curvature‐dependent speed: algorithms based on Hamilton‐Jacobi formulations, J. Comput. Phys., 79 (1988), 12–49

Digital Library

[38]

Stanley Osher, Jianhong Shen, Digitized PDE method for data restoration, Chapman & Hall/CRC, Boca Raton, FL, 2000, 751–771

[39]

P. Perona, Orientation diffusion, IEEE Trans. Image Process., 7 (1998), pp. 457–467.

Digital Library

[40]

L. Rudin and S. Osher, Total variation based image restoration with free local constraints, in Proceedings of the 1st IEEE International Conference on Image Processing, Austin, TX, 1994, pp. 31–35.

[41]

L. Rudin, S. Osher, and E. Fatemi, Nonlinear total variation based noise removal algorithms, Phys. D, 60 (1992), pp. 259–268.

Digital Library

[42]

J. Shen and G. Strang, The asymptotics of optimal (equiripple) filters, IEEE Trans. Signal Process., 47 (1999), pp. 1087–1098.

Digital Library

[43]

J. Shen, G. Strang, A. Wathen, The potential theory of several intervals and its applications, Appl. Math. Optim., 44 (2001), 67–85

Digital Library

[44]

Gilbert Strang, Truong Nguyen, Wavelets and filter banks, Wellesley‐Cambridge Press, 1996xxii+490

[45]

B. Tang, G. Sapiro, and V. Caselles, Color Image Enhancement via Chromaticity Diffusion, Technical report, ECE‐University of Minnesota, Minneapolis, MN, 1999.

[46]

B. Tang, G. Sapiro, and V. Caselles, Direction diffusion, in International Conference in Computer Vision, to appear.

[47]

S. Walden, The Ravished Image, St. Martin’s Press, New York, 1985.

[48]

L.‐Y. Wei and M. Levoy, Fast Texture Synthesis Using Tree‐Structured Vector Quantization, Preprint, Computer Science, Stanford University, Stanford, CA, 2000;

also in Proceedings of SIGGRAPH 2000.

Cited By

Lian WLiu XChen Y(2025)Non-convex fractional-order TV model for image inpaintingMultimedia Systems10.1007/s00530-024-01585-531:1Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s00530-024-01585-5
Li CZhang HXu D(2024)Multi-Scale Hierarchical VQ-VAEs for Blind Image InpaintingProceedings of the 2024 9th International Conference on Multimedia Systems and Signal Processing (ICMSSP)10.1145/3690063.3690065(1-8)Online publication date: 24-May-2024
https://dl.acm.org/doi/10.1145/3690063.3690065
Sun MLi Z(2024)An Efficient Diffusion Depth Image Inpainting Method Based on RGB-GuidedProceedings of the 2024 3rd Asia Conference on Algorithms, Computing and Machine Learning10.1145/3654823.3654848(130-134)Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1145/3654823.3654848
Show More Cited By

Index Terms

Mathematical Models for Local Nontexture Inpaintings

Index terms have been assigned to the content through auto-classification.

Recommendations

Euler's Elastica and Curvature-Based Inpainting

Image inpainting is a special image restoration problem for which image prior models play a crucial role. Euler's elastica was first introduced to computer vision by Mumford [Algebraic Geometry and its Applications, Springer-Verlag, New York, 1994, pp. ...
Infimal convolution type regularization of TGV and shearlet transform for image restoration
Abstract
We propose a novel infimal convolution type functional based on total generalized variation (TGV) and shearlet transform, which can be easily incorporated into image restoration problems. We employ the TGV functional to represent the ...
Highlights
- The new infimal convolution model separates an image to cartoon and texture parts.
Variational Approach for Restoring Blurred Images with Cauchy Noise

The restoration of images degraded by blurring and noise is one of the most important tasks in image processing. In this paper, based on the total variation (TV) we propose a new variational method for recovering images degraded by Cauchy noise and ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image SIAM Journal on Applied Mathematics

SIAM Journal on Applied Mathematics Volume 62, Issue 3

2002

363 pages

ISSN:0036-1399

DOI:10.1137/smjmap.2002.62.issue-3

Issue’s Table of Contents

Copyright © 2002 Society for Industrial and Applied Mathematics.

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 01 January 2002

Author Tags

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

59
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lian WLiu XChen Y(2025)Non-convex fractional-order TV model for image inpaintingMultimedia Systems10.1007/s00530-024-01585-531:1Online publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1007/s00530-024-01585-5
Li CZhang HXu D(2024)Multi-Scale Hierarchical VQ-VAEs for Blind Image InpaintingProceedings of the 2024 9th International Conference on Multimedia Systems and Signal Processing (ICMSSP)10.1145/3690063.3690065(1-8)Online publication date: 24-May-2024
https://dl.acm.org/doi/10.1145/3690063.3690065
Sun MLi Z(2024)An Efficient Diffusion Depth Image Inpainting Method Based on RGB-GuidedProceedings of the 2024 3rd Asia Conference on Algorithms, Computing and Machine Learning10.1145/3654823.3654848(130-134)Online publication date: 22-Mar-2024
https://dl.acm.org/doi/10.1145/3654823.3654848
Banerjee RChatterjee RKar A(2024)Inpainting of coherent regions in binary images based on Coherence Enhancing Cahn-Hilliard equationProceedings of the 2024 6th International Conference on Image Processing and Machine Vision10.1145/3645259.3645272(75-81)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3645259.3645272
Yang XGou TLv ZLi LJin H(2024)Weakly-supervised cloud detection and effective cloud removal for remote sensing imagesJournal of Visual Communication and Image Representation10.1016/j.jvcir.2023.10400698:COnline publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1016/j.jvcir.2023.104006
Li FZuo HJia YQiu J(2024)A developed Criminisi algorithm based on particle swarm optimization (PSO-CA) for image inpaintingThe Journal of Supercomputing10.1007/s11227-024-06099-580:11(16611-16629)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s11227-024-06099-5
Nan CQiao ZZhang Q(2024)Curvature-Dependent Elastic Bending Total Variation Model for Image Inpainting with the SAV AlgorithmJournal of Scientific Computing10.1007/s10915-024-02666-3101:2Online publication date: 12-Sep-2024
https://dl.acm.org/doi/10.1007/s10915-024-02666-3
Schaefer KWeickert J(2024)Regularised Diffusion–Shock InpaintingJournal of Mathematical Imaging and Vision10.1007/s10851-024-01175-066:4(447-463)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1007/s10851-024-01175-0
Lian JZhang JLiu JDong ZZhang H(2024)Guiding image inpainting via structure and texture features with dual encoderThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-03083-740:6(4303-4317)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1007/s00371-023-03083-7
Neena Raj NShreelekshmi R(2023)A secure pixel level self-recovery scheme for digital imagesJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-22124544:3(4481-4493)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.3233/JIFS-221245
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents