Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN)

Liang Nie; Wenxin Yu; Siyuan Li; Zhiqiang Zhang; Ning Jiang; Xuewen Zhang; Jun Gong

doi:10.1007/978-3-030-92310-5_74

Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN)

Liang Nie, Wenxin Yu^*, Siyuan Li, Zhiqiang Zhang, Ning Jiang, Xuewen Zhang, Jun Gong

^*Corresponding author for this work

School of Foreign Languages

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

Abstract

Recently, there have been great advances in many one-stage image inpainting methods. They may have a slight advantage in computation time but lack sufficient context information for inpainting. These inpainting approaches can not inpaint large holes naturalist. This paper proposes a progressive image inpainting algorithm with partial convolution generative networks for solving the above problem. It consists of a generator with partial convolution layers, a fully convolutional discriminative network, and a long short-term memory (LSTM) module. PPCGN has four steps to inpaint the image. Each Step will concentrate on a specific area for inpainting. The final generation results are completed by the cooperation of these four steps, which are connected through the LSTM module. Due to the partial convolution module and LSTM structure characteristics, our method has a good advantage in restoring the images with large holes and achieves better objective results, increasing 1.46 dB and 0.44 dB on the Paris Street View dataset, and CelebA dataset, respectively.

Original language	English
Title of host publication	Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings
Editors	Teddy Mantoro, Minho Lee, Media Anugerah Ayu, Kok Wai Wong, Achmad Nizar Hidayanto
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	640-647
Number of pages	8
ISBN (Print)	9783030923099
DOIs	https://doi.org/10.1007/978-3-030-92310-5_74
Publication status	Published - 2021
Event	28th International Conference on Neural Information Processing, ICONIP 2021 - Virtual, Online Duration: 8 Dec 2021 → 12 Dec 2021

Publication series

Name	Communications in Computer and Information Science
Volume	1517 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	28th International Conference on Neural Information Processing, ICONIP 2021
City	Virtual, Online
Period	8/12/21 → 12/12/21

Keywords

Generative adversarial networks
LSTM
Partial convolution
Progressive image inpainting

Access to Document

10.1007/978-3-030-92310-5_74

Cite this

Nie, L., Yu, W., Li, S., Zhang, Z., Jiang, N., Zhang, X., & Gong, J. (2021). Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN). In T. Mantoro, M. Lee, M. A. Ayu, K. W. Wong, & A. N. Hidayanto (Eds.), Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings (pp. 640-647). (Communications in Computer and Information Science; Vol. 1517 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-92310-5_74

Nie, Liang ; Yu, Wenxin ; Li, Siyuan et al. / Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN). Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings. editor / Teddy Mantoro ; Minho Lee ; Media Anugerah Ayu ; Kok Wai Wong ; Achmad Nizar Hidayanto. Springer Science and Business Media Deutschland GmbH, 2021. pp. 640-647 (Communications in Computer and Information Science).

@inproceedings{4275cafbc1f44353991695c899b94e7e,

title = "Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN)",

abstract = "Recently, there have been great advances in many one-stage image inpainting methods. They may have a slight advantage in computation time but lack sufficient context information for inpainting. These inpainting approaches can not inpaint large holes naturalist. This paper proposes a progressive image inpainting algorithm with partial convolution generative networks for solving the above problem. It consists of a generator with partial convolution layers, a fully convolutional discriminative network, and a long short-term memory (LSTM) module. PPCGN has four steps to inpaint the image. Each Step will concentrate on a specific area for inpainting. The final generation results are completed by the cooperation of these four steps, which are connected through the LSTM module. Due to the partial convolution module and LSTM structure characteristics, our method has a good advantage in restoring the images with large holes and achieves better objective results, increasing 1.46 dB and 0.44 dB on the Paris Street View dataset, and CelebA dataset, respectively.",

keywords = "Generative adversarial networks, LSTM, Partial convolution, Progressive image inpainting",

author = "Liang Nie and Wenxin Yu and Siyuan Li and Zhiqiang Zhang and Ning Jiang and Xuewen Zhang and Jun Gong",

note = "Publisher Copyright: {\textcopyright} 2021, Springer Nature Switzerland AG.; 28th International Conference on Neural Information Processing, ICONIP 2021 ; Conference date: 08-12-2021 Through 12-12-2021",

year = "2021",

doi = "10.1007/978-3-030-92310-5_74",

language = "English",

isbn = "9783030923099",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "640--647",

editor = "Teddy Mantoro and Minho Lee and Ayu, {Media Anugerah} and Wong, {Kok Wai} and Hidayanto, {Achmad Nizar}",

booktitle = "Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings",

address = "Germany",

}

Nie, L, Yu, W, Li, S, Zhang, Z, Jiang, N, Zhang, X & Gong, J 2021, Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN). in T Mantoro, M Lee, MA Ayu, KW Wong & AN Hidayanto (eds), Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings. Communications in Computer and Information Science, vol. 1517 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 640-647, 28th International Conference on Neural Information Processing, ICONIP 2021, Virtual, Online, 8/12/21. https://doi.org/10.1007/978-3-030-92310-5_74

Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN). / Nie, Liang; Yu, Wenxin; Li, Siyuan et al.
Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings. ed. / Teddy Mantoro; Minho Lee; Media Anugerah Ayu; Kok Wai Wong; Achmad Nizar Hidayanto. Springer Science and Business Media Deutschland GmbH, 2021. p. 640-647 (Communications in Computer and Information Science; Vol. 1517 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN)

AU - Nie, Liang

AU - Yu, Wenxin

AU - Li, Siyuan

AU - Zhang, Zhiqiang

AU - Jiang, Ning

AU - Zhang, Xuewen

AU - Gong, Jun

PY - 2021

Y1 - 2021

N2 - Recently, there have been great advances in many one-stage image inpainting methods. They may have a slight advantage in computation time but lack sufficient context information for inpainting. These inpainting approaches can not inpaint large holes naturalist. This paper proposes a progressive image inpainting algorithm with partial convolution generative networks for solving the above problem. It consists of a generator with partial convolution layers, a fully convolutional discriminative network, and a long short-term memory (LSTM) module. PPCGN has four steps to inpaint the image. Each Step will concentrate on a specific area for inpainting. The final generation results are completed by the cooperation of these four steps, which are connected through the LSTM module. Due to the partial convolution module and LSTM structure characteristics, our method has a good advantage in restoring the images with large holes and achieves better objective results, increasing 1.46 dB and 0.44 dB on the Paris Street View dataset, and CelebA dataset, respectively.

AB - Recently, there have been great advances in many one-stage image inpainting methods. They may have a slight advantage in computation time but lack sufficient context information for inpainting. These inpainting approaches can not inpaint large holes naturalist. This paper proposes a progressive image inpainting algorithm with partial convolution generative networks for solving the above problem. It consists of a generator with partial convolution layers, a fully convolutional discriminative network, and a long short-term memory (LSTM) module. PPCGN has four steps to inpaint the image. Each Step will concentrate on a specific area for inpainting. The final generation results are completed by the cooperation of these four steps, which are connected through the LSTM module. Due to the partial convolution module and LSTM structure characteristics, our method has a good advantage in restoring the images with large holes and achieves better objective results, increasing 1.46 dB and 0.44 dB on the Paris Street View dataset, and CelebA dataset, respectively.

KW - Generative adversarial networks

KW - LSTM

KW - Partial convolution

KW - Progressive image inpainting

UR - http://www.scopus.com/inward/record.url?scp=85121898693&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-92310-5_74

DO - 10.1007/978-3-030-92310-5_74

M3 - Conference contribution

AN - SCOPUS:85121898693

SN - 9783030923099

T3 - Communications in Computer and Information Science

SP - 640

EP - 647

BT - Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings

A2 - Mantoro, Teddy

A2 - Lee, Minho

A2 - Ayu, Media Anugerah

A2 - Wong, Kok Wai

A2 - Hidayanto, Achmad Nizar

PB - Springer Science and Business Media Deutschland GmbH

T2 - 28th International Conference on Neural Information Processing, ICONIP 2021

Y2 - 8 December 2021 through 12 December 2021

ER -

Nie L, Yu W, Li S, Zhang Z, Jiang N, Zhang X et al. Progressive Inpainting Strategy with Partial Convolutions Generative Networks (PPCGN). In Mantoro T, Lee M, Ayu MA, Wong KW, Hidayanto AN, editors, Neural Information Processing - 28th International Conference, ICONIP 2021, Proceedings. Springer Science and Business Media Deutschland GmbH. 2021. p. 640-647. (Communications in Computer and Information Science). doi: 10.1007/978-3-030-92310-5_74