More Web Proxy on the site http://driver.im/

research-article

Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions

Authors:

Jiannong CaoAuthors Info & Claims

ACM Computing Surveys (CSUR), Volume 54, Issue 3

Article No.: 63, Pages 1 - 42

https://doi.org/10.1145/3446374

Published: 08 May 2021 Publication History

Abstract

Generative Adversarial Networks (GANs) is a novel class of deep generative models that has recently gained significant attention. GANs learn complex and high-dimensional distributions implicitly over images, audio, and data. However, there exist major challenges in training of GANs, i.e., mode collapse, non-convergence, and instability, due to inappropriate design of network architectre, use of objective function, and selection of optimization algorithm. Recently, to address these challenges, several solutions for better design and optimization of GANs have been investigated based on techniques of re-engineered network architectures, new objective functions, and alternative optimization algorithms. To the best of our knowledge, there is no existing survey that has particularly focused on the broad and systematic developments of these solutions. In this study, we perform a comprehensive survey of the advancements in GANs design and optimization solutions proposed to handle GANs challenges. We first identify key research issues within each design and optimization technique and then propose a new taxonomy to structure solutions by key research issues. In accordance with the taxonomy, we provide a detailed discussion on different GANs variants proposed within each solution and their relationships. Finally, based on the insights gained, we present promising research directions in this rapidly growing field.

Supplementary Material

a63-saxena-suppl.pdf (saxena.zip)

Supplemental movie, appendix, image and software files for, Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions

Download
451.02 KB

References

[1]

Y.-W. Hinton, Geoffrey E. Osindero, and Simon Teh. 2006. A fast learning algorithm for deep belief nets. Neural Comput. 18, 7 (2006), 1527--1554.

Digital Library

[2]

R. Salakhutdinov and G. Hinton. 2009. Deep boltzmann machines. J. Mach. Learn. Res. 5, (2009) 448--455.

[3]

D. P. Kingma and M. Welling. 2014. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations (ICLR’14).

[4]

I. J. Goodfellow et al. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 2672--2680.

[5]

E. Denton, S. Chintala, A. Szlam, and R. Fergus. 2015. Deep generative image models using a Laplacian pyramid of adversarial networks. In Advances in Neural Information Processing Systems. 1486--1494.

[6]

A. Radford, L. Metz, and S. Chintala. 2016. Unsupervised representation learning with deep convolutional generative adversarial networks. In 4th International Conference on Learning Representations (ICLR’16).

[7]

M. Y. Liu and O. Tuzel. 2016. Coupled generative adversarial networks. In Advances in Neural Information Processing Systems. 469--477.

[8]

T. Karras, T. Aila, S. Laine, and J. Lehtinen. 2018. Progressive growing of GANs for improved quality, stability, and variation. In 6th International Conference on Learning Representations (ICLR’18).

[9]

T. Karras, S. Laine, and T. Aila. 2019. A style-based generator architecture for generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 4401--4410.

[10]

C. Ledig et al. 2017. Photo-realistic single image super-resolution using a generative adversarial network. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 105--114.

[11]

C. Spampinato, S. Palazzo, P. D'Oro, D. Giordano, and M. Shah. 2019. Adversarial framework for unsupervised learning of motion dynamics in videos. Int. J. Comput. Vis. (Mar. 2019).

[12]

T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In 34th International Conference on Machine Learning (ICML’17). 2941--2949.

[13]

Y. Hong, U. Hwang, J. Yoo, and S. Yoon. 2019. How generative adversarial networks and their variants work: An overview. ACM Comput. Surv. 52, 1 (2019).

[14]

Z. Wang, Q. She, and T. E. Ward. 2019. Generative adversarial networks: A survey and taxonomy. arXiv Preprint arXiv1906.01529, 2019.

[15]

Z. Pan, W. Yu, X. Yi, A. Khan, F. Yuan, and Y. Zheng. 2019. Recent progress on generative adversarial networks (GANs): A survey. IEEE Access 7 (2019), 36322--36333.

[16]

S. Hitawala. 2018. Comparative study on generative adversarial networks. arXiv Preprint arXiv1801.04271, 2018.

[17]

K. Wang, C. Gou, Y. Duan, Y. Lin, X. Zheng, and F. Y. Wang. 2017. Generative adversarial networks: Introduction and outlook. IEEE/CAA J. Automat. Sin. 4, 4 (2017), 588--598.

[18]

J. Gui, Z. Sun, Y. Wen, D. Tao, and J. Ye. 2020. A review on generative adversarial networks: Algorithms, theory, and applications. arXiv preprint arXiv:2001.06937.

[19]

A. Bissoto, E. Valle, and S. Avila. 2019. The six fronts of the generative adversarial networks. arXiv Preprint arXiv1910.13076, 2019.

[20]

T. Motwani and M. Parmar. 2020. A novel framework for selection of GANs for an application. arXiv Preprint arXiv2002.08641, 2020.

[21]

Y.-J. CAO. 2019. Recent advances of generative adversarial networks in computer vision. IEEE Access 7 (2019).

[22]

A. Jabbar, X. Li, and B. Omar. 2020. A survey on generative adversarial networks: Variants, applications, and training. arXiv preprint arXiv:2006.05132.

[23]

L. Jin, F. Tan, and S. Jiang. 2020. Generative adversarial network technologies and applications in computer vision. Computational Intelligence and Neuroscience.

[24]

M. Wiatrak, S. V. Albrecht, and A. Nystrom. 2019. Stabilizing generative adversarial networks: A survey. arXiv preprint arXiv:1910.00927.

[25]

M. Lee and J. Seok. 2020. Regularization methods for generative adversarial networks: An overview of recent studies. arXiv preprint arXiv:2005.09165.

[26]

S. N. Esfahani and S. Latifi. 2019. A survey of state-of-the-Art GAN-based approaches to image synthesis. In Computer Science & Information Technology (CS & IT) Computer Science Conference Proceedings (CSCP). 63--76.

[27]

N. Akhtar and A. Mian. 2018. Threat of adversarial attacks on deep learning in computer vision: A survey. IEEE Access 6, (2018), 14410--14430.

[28]

A. Chakraborty, A. Manaar, V. Dey, A. Chattopadhyay, and D. Mukhopadhyay. 2018. Adversarial attacks and defences: A survey. arXiv Preprint arXiv1810.00069, 2018.

[29]

F. Huszár. 2015. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? arXiv Preprint arXiv1511.05101, 2015.

[30]

L. Theis, A. Van Den Oord, and M. Bethge. 2016. A note on the evaluation of generative models. In 4th International Conference on Learning Representations (ICLR’16).

[31]

M. Rosca, B. Lakshminarayanan, D. Warde-Farley, and S. Mohamed. 2017. Variational approaches for auto-encoding generative adversarial networks. arXiv Preprint arXiv1706.04987, 2017.

[32]

A. Statnikov, C. F. Aliferis, D. P. Hardin, and I. Guyon. 2011. Support vector clustering. In A Gentle Introduction to Support Vector Machines in Biomedicine, Vol. 1 (Feb. 2011), 136--153.

[33]

T. Le, H. Vu, T. D. Nguyen, and D. Phung. 2018. Geometric enclosing networks. In International Joint Conference on Artificial Intelligence (IJCAI’18). 2355--2361.

[34]

M. Mirza and S. Osindero. 2014. Conditional generative adversarial nets. arXiv Preprint arXiv1411.1784, 2014.

[35]

M. Arjovsky, S. Chintala, and L. Bottou. 2017. Wasserstein generative adversarial networks. In International Conference on Machine Learning. 214--223.

[36]

J. Adler and S. Lunz. 2018. Banach Wasserstein GAN. In Advances in Neural Information Processing Systems. 6754--6763.

[37]

X. Guo, J. Hong, T. Lin, and N. Yang. 2017. Relaxed Wasserstein with applications to GANs. arXiv Preprint arXiv1705.07164, 2017.

[38]

L. Mescheder, S. Nowozin, and A. Geiger. 2017. The numerics of GANs. In Advances in Neural Information Processing Systems. 1826--1836.

[39]

M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems. 6627--6638.

[40]

X. Mao, Q. Li, H. Xie, R. Y. K. Lau, Z. Wang, and S. P. Smolley. 2017. Least squares generative adversarial networks. In IEEE International Conference on Computer Vision. 2813--2821.

[41]

A. Yadav, S. Shah, Z. Xu, D. Jacobs, and T. Goldstein. 2018. Stabilizing adversarial nets with prediction methods. arXiv Preprint arXiv1705.07364, 2018.

[42]

T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. 2016. Improved techniques for training GANs. In Advances in Neural Information Processing Systems. 2234--2242.

[43]

Z. Lin, G. Fanti, A. Khetan, and S. Oh. 2018. PacGan: The power of two samples in generative adversarial networks. In Advances in Neural Information Processing Systems. 1498--1507.

[44]

I. Goodfellow. 2016. Tutorial: Generative adversarial networks. arXiv Preprint arXiv1701.00160, 2016.

[45]

L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein. 2016. Unrolled generative adversarial networks. arXiv Preprint arXiv:1611.02163, 2016.

[46]

T. Che, Y. Li, A. P. Jacob, Y. Bengio, and W. Li. 2019. Mode regularized generative adversarial networks. In 5th International Conference on Learning Representations (ICLR’19).

[47]

D. Warde-Farley and Y. Bengio. 2019. Improving generative adversarial networks with denoising feature matching. In 5th International Conference on Learning Representations (ICLR’19).

[48]

M. Arjovsky and L. Bottou. 2019. Towards principled methods for training generative adversarial networks. In 5th International Conference on Learning Representations (ICLR’19).

[49]

S. Huang, Xun Li, Yixuan Poursaeed, Omid Hopcroft, and John Belongie. 2017. Stacked generative adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 5077--5086.

[50]

Y. Wang, L. Zhang, and J. van de Weijer. 2016. Ensembles of generative adversarial networks. arXiv Preprint arXiv1612.00991, 2016.

[51]

I. Tolstikhin, S. Gelly, O. Bousquet, C. J. Simon-Gabriel, and B. Schölkopf. 2017. AdaGAN: Boosting generative models. In Advances in Neural Information Processing Systems. 5425--5434.

[52]

C. L. Li, W. C. Chang, Y. Cheng, Y. Yang, and B. Póczos. 2017. MMD GAN: Towards deeper understanding of moment matching network. In Advances in Neural Information Processing Systems. 2204--2214.

[53]

H. Kwak and B.-T. Zhang. 2016. Ways of conditioning generative adversarial networks. arXiv Preprint arXiv1611.01455, 2016.

[54]

D. J. Im, C. D. Kim, H. Jiang, and R. Memisevic. 2016. Generating images with recurrent adversarial networks. arXiv Preprint arXiv1602.05110, 2016.

[55]

G. Perarnau, J. van de Weijer, B. Raducanu, and J. M. Álvarez. 2016. Invertible conditional GANs for image editing. arXiv Preprint arXiv1611.06355, 2016.

[56]

A. Jaiswal, W. AbdAlmageed, Y. Wu, and P. Natarajan. 2019. Bidirectional conditional generative adversarial networks. Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 11363. 216--232.

[57]

G. Máttyus and R. Urtasun. 2018. Matching adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition. 8024--8032.

[58]

S. Liu, T. Wang, D. Bau, J.-Y. Zhu, and A. Torralba. 2020. Diverse image generation via self-conditioned GANs. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 14274--14283.

[59]

A. Odena, C. Olah, and J. Shlens. 2017. Conditional image synthesis with auxiliary classifier GANs. In 34th International Conference on Machine Learning (ICML’17). 4043--4055.

[60]

L. Chongxuan, T. Xu, J. Zhu, and B. Zhang. 2017. Triple generative adversarial nets. In Advances in Neural Information Processing Systems. 4088--4098.

[61]

X. Wang, Y. Sun, R. Zhang, and J. Qi. 2018. KDGAN: Knowledge distillation with generative adversarial networks. In Advances in Neural Information Processing Systems. 775--786.

[62]

M. Lee and J. Seok. 2019. Controllable generative adversarial network. IEEE Access 7 (2019), 28158--28169.

[63]

Y. Saatchi and A. G. Wilson. 2017. Bayesian GAN. In Advances in Neural Information Processing Systems. 3623--3632.

[64]

A. Jaiswal, W. AbdAlmageed, Y. Wu, and P. Natarajan. 2019. CapsuleGAN: Generative adversarial capsule network. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 11131. 526--535.

[65]

S. Lloyd and C. Weedbrook. 2018. Quantum generative adversarial learning. Phys. Rev. Lett. 121, 4 (2018).

[66]

A. Zhang, Han Goodfellow, Ian Metaxas, and Dimitris Odena. 2018. Self-attention generative adversarial networks. arXiv Preprint arXiv1805.08318, 2018.

[67]

A. Ghosh, V. Kulharia, V. Namboodiri, P. H. S. Torr, and P. K. Dokania. 2018. Multi-agent diverse generative adversarial networks. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 8513--8521.

[68]

Q. Hoang, T. D. Nguyen, T. Le, and D. Phung. 2017. Multi-generator generative adversarial nets. arXiv Preprint arXiv1708.02556, 2017.

[69]

A. Ghosh, V. Kulharia, and V. Namboodiri. 2016. Message passing multi-agent GANs. arXiv Preprint arXiv1612.01294, 2016.

[70]

H. Ge, Y. Xia, X. Chen, R. Berry, and Y. Wu. 2017. Fictitious GAN: Training GANs with historical models. In Proceedings of the European Conference on Computer Vision (ECCV'18). 119--134.

[71]

S. Arora, R. Ge, Y. Liang, T. Ma, and Y. Zhang. 2017. Generalization and equilibrium in generative adversarial nets (GANs). In 34th International Conference on Machine Learning (ICML’17). 322--349.

[72]

T. D. Nguyen, T. Le, H. Vu, and D. Phung. 2017. Dual discriminator generative adversarial nets. In Advances in Neural Information Processing Systems. 2671--2681.

[73]

I. Durugkar, I. Gemp, and S. Mahadevan. 2019. Generative multi-adversarial networks. In 5th International Conference on Learning Representations (ICLR’17).

[74]

B. Neyshabur, S. Bhojanapalli, and A. Chakrabarti. 2017. Stabilizing GAN training with multiple random projections. arXiv Preprint arXiv1705.07831, 2017.

[75]

G. Mordido, H. Yang, and C. Meinel. 2018. Dropout-GAN: Learning from a dynamic ensemble of discriminator. arXiv preprint arXiv:1807.11346, 2018.

[76]

G. Mordido, H. Yang, and C. Meinel. 2020. MicrobatchGAN: Stimulating diversity with multi-adversarial discrimination. In IEEE Winter Conference on Applications of Computer Vision. 3050--3059.

[77]

T. Chavdarova and F. Fleuret. 2018. SGAN: An alternative training of generative adversarial networks. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 9407--9415.

[78]

A. B. L. Larsen, S. K. Sønderby, H. Larochelle, and O. Winther. 2016. Autoencoding beyond pixels using a learned similarity metric. In 33rd International Conference on Machine Learning (ICML’16). 2341--2349.

[79]

A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey. 2015. Adversarial autoencoders. arXiv Preprint arXiv1511.05644, 2015.

[80]

L. Mescheder, S. Nowozin, and A. Geiger. 2017. Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks. In 34th International Conference on Machine Learning (ICML’17). 3694--3707.

[81]

Y. Pu et al. 2017. Adversarial symmetric variational autoencoder. In Advances in Neural Information Processing Systems. 4331--4340.

[82]

N. T. Tran, T. A. Bui, and N. M. Cheung. 2018. Dist-GAN: An improved GAN using distance constraints. Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 11218. 387--401.

[83]

V. Dumoulin et al. 2019. Adversarially learned inference. In 5th International Conference on Learning Representations (ICLR’19).

[84]

J. Donahue, T. Darrell, and P. Krähenbühl. 2016. Adversarial feature learning. arXiv preprint arXiv:1605.09782, 2016.

[85]

A. H. Li, Y. Wang, C. Chen, and J. Gao. 2020. Decomposed adversarial learned inference. arXiv Preprint arXiv2004.10267, 2020.

[86]

M. Chen and L. Denoyer. 2017. Multi-view generative adversarial networks. In Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 10535. 175--188.

[87]

M. I. Belghazi, S. Rajeswar, O. Mastropietro, N. Rostamzadeh, J. Mitrovic, and A. Courville. 2018. Hierarchical adversarially learned Inference. arXiv Preprint arXiv1802.01071, 2018.

[88]

D. Ulyanov, A. Vedaldi, and V. Lempitsky. 2018. It takes (only) two: Adversarial generator-encoder networks. In 32nd AAAI Conference on Artificial Intelligence. 1250--1257.

[89]

A. Srivastava, L. Valkov, C. Russell, M. U. Gutmann, and C. Sutton. 2017. VEEGAN: Reducing mode collapse in GANs using implicit variational learning. In Advances in Neural Information Processing Systems. 3309--3319.

[90]

D. Bang and H. Shim. 2018. MGGAN: Solving mode collapse using manifold guided training. arXiv Preprint arXiv1804.04391, 2018.

[91]

J. Zhao, M. Mathieu, and Y. LeCun. 2016. Energy-based generative adversarial networks. arXiv preprint arXiv:1609.03126, 2016.

[92]

D. Berthelot, T. Schumm, and L. Metz. 2017. BEGAN: Boundary equilibrium generative adversarial networks. arXiv Preprint arXiv1703.10717, 2017.

[93]

R. Wang, A. Cully, H. J. Chang, and Y. Demiris. 2017. MAGAN: Margin adaptation for generative adversarial networks. arXiv Preprint arXiv1704.03817, 2017.

[94]

G. Zheng, J. Sang, and C. Xu. 2020. MMCGAN: Generative adversarial network with explicit manifold prior. arXiv Preprint arXiv2006.10331, 2020.

[95]

X. Di and P. Yu. 2017. Max-Boost-GAN: Max operation to boost generative ability of generative adversarial networks. In IEEE International Conference on Computer Vision Workshops. 1156--1164.

[96]

Y. Kim, M. Kim, and G. Kim. 2018. Memorization precedes generation: Learning unsupervised GANs with memory networks. arXiv preprint arXiv:1803.01500.

[97]

S. Gurumurthy, R. K. Sarvadevabhatla, and R. V. Babu. 2017. DeLiGAN : Generative adversarial networks for diverse and limited data. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 4941--4949.

[98]

D. Mishra, Prathosh A. P., A. Jayendran, V. Srivastava, and S. Chaudhury. 2018. NEMGAN: Noise engineered mode-matching GAN. arXiv Preprint arXiv1811.03692, 2018.

[99]

X. Di and P. Yu. 2017. Multiplicative noise channel in generative adversarial networks. In IEEE International Conference on Computer Vision Workshops. 1165--1172.

[100]

G. Zhong, W. Gao, Y. Liu, and Y. Yang. 2018. Generative adversarial networks with decoder-encoder output noise. arXiv Preprint arXiv1807.03923, 2018.

[101]

X. Chen, Y. Duan, R. Houthooft, J. Schulman, I. Sutskever, and P. Abbeel. 2016. InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets. In Advances in Neural Information Processing Systems. 2172--2180.

[102]

G.-J. Qi. 2017. Loss-sensitive generative adversarial networks on Lipschitz densities. Int. J. Comput. Vis 128, 5 (2017), 1118--1140.

Digital Library

[103]

S. Nowozin, B. Cseke, and R. Tomioka. 2016. f-GAN: Training generative neural samplers using variational divergence minimization. In Advances in Neural Information Processing Systems. 271--279.

[104]

M. Uehara, I. Sato, M. Suzuki, K. Nakayama, and Y. Matsuo. 2016. Generative adversarial nets from a density ratio estimation perspective. arXiv Preprint arXiv1610.02920, 2016.

[105]

C. Tao, L. Chen, R. Henao, J. Feng, and L. Carin. 2018. X2 generative adversarial network. In 35th International Conference on Machine Learning (ICML’18). 7787--7796.

[106]

T. Salimans, D. Metaxas, H. Zhang, and A. Radford. 2018. Improving GANs using optimal transport. In 6th International Conference on Learning Representations (ICLR’18).

[107]

M. Lin. 2017. Softmax GAN. arXiv Preprint arXiv1704.06191, 2017.

[108]

Y. Li, N. Xiao, and W. Ouyang. 2019. Improved generative adversarial networks with reconstruction loss. Neurocomputing 323 (2019), 363--372.

[109]

F. Juefei-Xu, V. N. Boddeti, and M. Savvides. 2017. Gang of GANs: Generative adversarial networks with maximum margin ranking. arXiv Preprint arXiv1704.04865, 2017.

[110]

Y. Mroueh, T. Sercu, and V. Goel. 2017. McGAN: Mean and covariance feature matching GAN. In 34th International Conference on Machine Learning (ICML’17). 3885--3899.

[111]

N. Park et al. 2018. MMGAN: Manifold-matching generative adversarial networks. In International Conference on Pattern Recognition. 1343--1348.

[112]

M. G. Bellemare et al. 2017. The cramer distance as a solution to biased Wasserstein gradients. arXiv Preprint arXiv1705.10743, 2017.

[113]

I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville. 2017. Improved training of Wasserstein GANs. In Advances in Neural Information Processing Systems. 5768--5778.

[114]

H. Petzka, A. Fischer, and D. Lukovnikov. 2018. On the regularization of Wasserstein GANs. In 6th International Conference on Learning Representations (ICLR’18).

[115]

X. Wei, B. Gong, Z. Liu, W. Lu, and L. Wang. 2018. Improving the improved training of Wasserstein GANs: A consistency term and its dual effect. arXiv Preprint arXiv1803.01541, 2018.

[116]

T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. 2018. Spectral normalization for generative adversarial networks. In 6th International Conference on Learning Representations (ICLR’18).

[117]

K. Roth, A. Lucchi, S. Nowozin, and T. Hofmann. 2017. Stabilizing training of generative adversarial networks through regularization. In Advances in Neural Information Processing Systems. 2019--2029.

[118]

Y. Mroueh and T. Sercu. 2017. Fisher GAN. In Advances in Neural Information Processing Systems. 2514--2524.

[119]

V. Nagarajan and J. Z. Kolter. 2017. Gradient descent GAN optimization is locally stable. In Advances in Neural Information Processing Systems. 5586--5596.

[120]

L. Mescheder, A. Geiger, and S. Nowozin. 2018. Which training methods for GANs do actually converge? In 35th International Conference on Machine Learning (ICML’18). 5589--5626.

[121]

N. Kodali, J. Abernethy, J. Hays, and Z. Kira. 2017. On convergence and stability of GANs. arXiv Preprint arXiv1705.07215, 2017.

[122]

C. Daskalakis, A. Ilyas, V. Syrgkanis, and H. Zeng. 2018. Training GaNs with optimism. 6th International Conference on Learning Representations (ICLR’18).

[123]

H. Prasad, Prashanth L. A., and S. Bhatnagar. 2015. Two-timescale algorithms for learning Nash equilibria in general-sum stochastic games. In International Conference on Autonomous Agents and Multiagent Systems.

[124]

A. Fukui, D. H. Park, D. Yang, A. Rohrbach, T. Darrell, and M. Rohrbach. 2016. Multimodal compact bilinear pooling for visual question answering and visual grounding. In Conference on Empirical Methods in Natural Language Processing. 457--468.

[125]

M. Mathieu, J. Zhao, P. Sprechmann, A. Ramesh, and Y. Le Cun. 2016. Disentangling factors of variation in deep representations using adversarial training. In Advances in Neural Information Processing Systems. 5047--5055.

[126]

Z. Xu, Y. C. Hsu, and J. Huang. 2018. Training shallow and thin networks for acceleration via knowledge distillation with conditional adversarial networks. In 6th International Conference on Learning Representations Workshop (ICLRW’18).

[127]

M. Mirza and S. Osindero. 2014. Conditional generative adversarial nets. arXiv Preprint arXiv1411.1784, 2014.

[128]

T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. 1997. Improved techniques for training GANs. arXiv preprint arXiv:1606.03498.

[129]

Y. Freund and R. E. Schapire. 1997. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci 55, 1 (1997), 119--139.

Digital Library

[130]

I. Albuquerque, J. Monteiro, T. Doan, B. Considine, T. Falk, and I. Mitliagkas. 2019. Multi-objective training of generative adversarial networks with multiple discriminators. In 36th International Conference on Machine Learning (ICML’19). 292--301.

[131]

A. Nguyen, J. Clune, Y. Bengio, A. Dosovitskiy, and J. Yosinski. 2017. Plug and play generative networks: Conditional iterative generation of images in latent space. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3510--3520.

[132]

R. A. Yeh, C. Chen, T. Yian Lim, A. G. Schwing, M. Hasegawa-Johnson, and M. N. Do. 2017. Semantic image inpainting with deep generative models. In 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 6882--6890.

[133]

I. J. Goodfellow. 2015. On distinguishability criteria for estimating generative models. In 3rd International Conference on Learning Representations Workshop (ICLRW’15).

[134]

S. Mohamed and B. Lakshminarayanan. 2016. Learning in implicit generative models. arXiv Preprint arXiv1610.03483, 2016.

[135]

Y. Bengio, N. Léonard, and A. Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv Preprint arXiv1308.3432, 2013.

[136]

Y. Bengio, É. Thibodeau-Laufer, G. Alain, and J. Yosinski. 2014. Deep generative stochastic networks trainable by backprop. In 31st International Conference on Machine Learning (ICML’14). 1470--1485.

[137]

H. De Meulemeester, J. Schreurs, M. Fanuel, B. De Moor, and J. A. K. Suykens. 2020. The Bures metric for taming mode collapse in generative adversarial networks. arXiv Preprint arXiv2006.09096, 2020.

[138]

Y. LeCun, S. Chopra, R. Hadsell, F. J. Huang, and E. Al. 2006. A tutorial on energy-based learning. Predict. Struct. Data 1, 0 (2006).

[139]

Z. Dai, A. Almahairi, P. Bachman, E. Hovy, and A. Courville. 2017. Calibrating energy-based generative adversarial networks. arXiv Preprint arXiv1702.01691, 2017.

[140]

S. Mukherjee, H. Asnani, E. Lin, and S. Kannan. 2019. ClusterGAN: Latent space clustering in generative adversarial networks. In AAAI Conference on Artificial Intelligence. 4610--4617.

[141]

D. P. Kingma and M. Welling. 2014. Auto-encoding variational bayes. In 2nd International Conference on Learning Representations (ICLR’14).

[142]

D. P. Kingma, D. J. Rezende, S. Mohamed, and M. Welling. 2014. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems. 3581--3589.

[143]

B. Cheung, J. A. Livezey, A. K. Bansal, and B. A. Olshausen. 2015. Discovering hidden factors of variation in deep networks. In 3rd International Conference on Learning Representations Workshop (ICLRW’15).

[144]

T. D. Kulkarni, W. F. Whitney, P. Kohli, and J. B. Tenenbaum. 2015. Deep convolutional inverse graphics network. In Advances in Neural Information Processing Systems. 2539--2547.

[145]

W. F. Whitney, M. Chang, T. Kulkarni, and J. B. Tenenbaum. 2016. Understanding visual concepts with continuation learning. arXiv Preprint arXiv1602.06822, 2016.

[146]

F. Huszár. 2017. Variational inference using implicit distributions. arXiv Preprint arXiv1702.08235, 2017.

[147]

T. Karaletsos. 2016. Adversarial message passing for graphical models. arXiv Preprint arXiv1612.05048, 2016.

[148]

D. Tran, R. Ranganath, and D. M. Blei. 2017. Hierarchical implicit models and likelihood-free variational inference. In Advances in Neural Information Processing Systems. 5524--5534.

[149]

Y. Rubner, C. Tomasi, and L. J. Guibas. 2000. Earth mover's distance as a metric for image retrieval. Int. J. Comput. Vis 40, 2, (2000), 99--121.

Digital Library

[150]

I. Goodfellow, Y. Bengio, and A. Courville. 2017. Deep Learning. The MIT Press.

[151]

A. Gretton, O. Bousquet, A. Smola, and B. Scḧlkopf. 2005. Measuring statistical dependence with Hilbert-Schmidt norms. In Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 3734. 63--77.

[152]

K. Fukumizu, A. Gretton, X. Sun, and B. Schölkopf. 2009. Kernel measures of conditional dependence. In Advances in Neural Information Processing Systems.

[153]

A. Smola, A. Gretton, L. Song, and B. Schölkopf. 2007. A Hilbert space embedding for distributions. In Lect. Notes Comput. Sci. (incl. Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinf.), Vol. 4754. 13--31.

[154]

Y. Li, K. Swersky, and R. Zemel. 2015. Generative moment matching networks. In 32nd International Conference on Machine Learning (ICML’15). 1718--1727.

[155]

G. K. Dziugaite, D. M. Roy, and Z. Ghahramani. 2015. Training generative neural networks via maximum mean discrepancy optimization. In 31st Conference on Uncertainty in Artificial Intelligence. 258--267.

[156]

S. Ioffe and C. Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In 32nd International Conference on Machine Learning (ICML’15). 448--456.

[157]

W. Zellinger, E. Lughofer, S. Saminger-Platz, T. Grubinger, and T. Natschläger. 2019. Central moment discrepancy (CMD) for domain-invariant representation learning. In 5th International Conference on Learning Representations (ICLR’19).

[158]

T. Salimans and D. P. Kingma. 2016. Weight normalization: A simple reparameterization to accelerate training of deep neural networks. In Advances in Neural Information Processing Systems. 901--909.

[159]

A. Müller. 1997. Integral probability metrics and their generating classes of functions. Adv. Appl. Probab. 29, 2 (1997), 429--443.

[160]

B. K. Sriperumbudur, K. Fukumizu, A. Gretton, B. Schölkopf, and G. R. G. Lanckriet. 2009. On integral probability metrics, phi-divergences and binary classification. arXiv preprint arXiv:0901.2698.

[161]

J. C. Butcher. 2016. Numerical Methods for Ordinary Differential Equations. John Wiley & Sons, Ltd, Chichester, UK.

[162]

S. I. Amari. 1998. Natural gradient works efficiently in learning. Neural Comput. 10, 2 (1998), 251--276.

Digital Library

[163]

P. Grnarova, K. Y. Levy, A. Lucchi, T. Hofmann, and A. Krause. 2018. An online learning approach to generative adversarial networks. In 6th International Conference on Learning Representations (ICLR’18).

[164]

D. Volkhonskiy, I. Nazarov, and E. Burnaev. 2017. Steganographic generative adversarial networks. arXiv Preprint arXiv1703.05502, 2017.

[165]

W. Zhang. 2018. Generative adversarial nets for information retrieval: Fundamentals and advances. In 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 1375--1378.

Digital Library

[166]

D. Saxena and J. Cao. 2019. D-GAN: Deep generative adversarial nets for spatio-temporal prediction. arXiv Preprint arXiv1907.08556, 2019.

[167]

M. Uricar, P. Krizek, D. Hurych, I. Sobh, S. Yogamani, and P. Denny. 2019. Yes, we GAN: Applying adversarial techniques for autonomous driving. arXiv Preprint arXiv1902.03442, 2019.

[168]

M. Zhang, Y. Zhang, L. Zhang, C. Liu, and S. Khurshid. 2018. DeepRoad: GAN-based metamorphic autonomous driving system testing. In 33rd ACM/IEEE International Conference on Automated Software Engineering. 132--142.

[169]

M. Teichmann, M. Weber, M. Zöllner, R. Cipolla, and R. Urtasun. 2018. MultiNet: Real-time joint semantic reasoning for autonomous driving. In IEEE Intelligent Vehicles Symposium. 1013--1020.

[170]

S. Pascual, A. Bonafonte, and J. Serra. 2017. SEGAN: Speech enhancement generative adversarial network. In Conference of the International Speech Communication Association. 3642--3646.

[171]

Y. Xu, Z. Zhang, L. You, J. Liu, Z. Fan, and X. Zhou. 2020. scIGANs: single-cell RNA-seq imputation using generative adversarial networks. Nucleic Acids Res. 48, 15 (2020), 85.

[172]

M. Lucic, K. Kurach, M. Michalski, O. Bousquet, and S. Gelly. 2018. Are GANs created equal? A large-scale study. In Advances in Neural Information Processing Systems. 700--709.

[173]

Q. Xu et al. 2018. An empirical study on evaluation metrics of generative adversarial networks. arXiv Preprint arXiv1806.07755, 2018.

[174]

A. Borji. 2019. Pros and cons of GAN evaluation measures. Comput. Vis. Image Underst. 179 (2019), 41--65.

Digital Library

Cited By

Shu JYu HLiu GDuan YHu HZhang H(2025)DF-CDM: Conditional diffusion model with data fusion for structural dynamic response reconstructionMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111783222(111783)Online publication date: Jan-2025
https://doi.org/10.1016/j.ymssp.2024.111783
Paul AMallidi S(2025)Enhancing signal-to-noise ratio in real-time LED-based photoacoustic imaging: A comparative study of CNN-based deep learning architecturesPhotoacoustics10.1016/j.pacs.2024.10067441(100674)Online publication date: Feb-2025
https://doi.org/10.1016/j.pacs.2024.100674
Farhan MShah NWang LMuntean GSong H(2025)RDG-TE: Link reliability-aware DRL-GNN-based traffic engineering in SDNExpert Systems with Applications10.1016/j.eswa.2024.125963265(125963)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125963
Show More Cited By

Index Terms

Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Games of GANs: game-theoretical models for generative adversarial networks
Abstract
Generative Adversarial Networks (GANs) have recently attracted considerable attention in the AI community due to their ability to generate high-quality data of significant statistical resemblance to real data. Fundamentally, GAN is a game between ...
A lightweight ensemble discriminator for Generative Adversarial Networks
Abstract
While Generative Adversarial Networks (GANs) have brought immense success in various content-generation tasks, they still face enormous challenges in generating high-quality visually realistic images because of the model collapse or ...
Generative adversarial networks in medical image segmentation: A review
Abstract Purpose
Since Generative Adversarial Network (GAN) was introduced into the field of deep learning in 2014, it has received extensive attention from academia and industry, and a lot of high-quality papers have been published. ...
Graphical abstract

Display Omitted
Highlights
- A systematic review on medical image segmentation based on GAN is provided.
- The ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Computing Surveys

ACM Computing Surveys Volume 54, Issue 3

April 2022

836 pages

ISSN:0360-0300

EISSN:1557-7341

DOI:10.1145/3461619

Editor:
Albert Zomaya
University of Sydney, Australia

Issue’s Table of Contents

Copyright © 2021 ACM.

© 2021 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Accepted: 01 December 2020

Revised: 01 December 2020

Received: 01 April 2020

Published in CSUR Volume 54, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

RGC Collaborative Research Fund
RGC Research Impact Fund

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

271
Total Citations
View Citations
4,301
Total Downloads

Downloads (Last 12 months)1,383
Downloads (Last 6 weeks)126

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shu JYu HLiu GDuan YHu HZhang H(2025)DF-CDM: Conditional diffusion model with data fusion for structural dynamic response reconstructionMechanical Systems and Signal Processing10.1016/j.ymssp.2024.111783222(111783)Online publication date: Jan-2025
https://doi.org/10.1016/j.ymssp.2024.111783
Paul AMallidi S(2025)Enhancing signal-to-noise ratio in real-time LED-based photoacoustic imaging: A comparative study of CNN-based deep learning architecturesPhotoacoustics10.1016/j.pacs.2024.10067441(100674)Online publication date: Feb-2025
https://doi.org/10.1016/j.pacs.2024.100674
Farhan MShah NWang LMuntean GSong H(2025)RDG-TE: Link reliability-aware DRL-GNN-based traffic engineering in SDNExpert Systems with Applications10.1016/j.eswa.2024.125963265(125963)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125963
Azadmanesh MShahgholi Ghahfarokhi BAshouri Talouki M(2025)On the local convergence of ADAM-DPGAN with simultaneous and alternating gradient decent training methodsExpert Systems with Applications10.1016/j.eswa.2024.125646262(125646)Online publication date: Mar-2025
https://doi.org/10.1016/j.eswa.2024.125646
Niño JJohnson O(2025)Data-driven 2D grain growth microstructure prediction using deep learning and spectral graph theoryComputational Materials Science10.1016/j.commatsci.2024.113504247(113504)Online publication date: Jan-2025
https://doi.org/10.1016/j.commatsci.2024.113504
Arafah MPhillips IAdnane AHadi WAlauthman MAl-Banna A(2025)Anomaly-based network intrusion detection using denoising autoencoder and Wasserstein GAN synthetic attacksApplied Soft Computing10.1016/j.asoc.2024.112455168(112455)Online publication date: Jan-2025
https://doi.org/10.1016/j.asoc.2024.112455
Li TMa WZheng YFan XYang GWang LLi Z(2024)A survey on gait recognition against occlusion: taxonomy, dataset and methodologyPeerJ Computer Science10.7717/peerj-cs.260210(e2602)Online publication date: 23-Dec-2024
https://doi.org/10.7717/peerj-cs.2602
Kc BSapkota SAdhikari A(2024)Generative Adversarial Networks in Anomaly Detection and Malware Detection: A Comprehensive SurveyAdvances in Artificial Intelligence Research10.54569/aair.1442665Online publication date: 30-Aug-2024
https://doi.org/10.54569/aair.1442665
MANAA NSERİDİ HMENDJEL M(2024)Advancements in Recommender Systems Through the Integration of Generative Adversarial NetworksInternational Journal of Informatics and Applied Mathematics10.53508/ijiam.14064986:2(35-45)Online publication date: 29-Jan-2024
https://doi.org/10.53508/ijiam.1406498
Al-Khasawneh MMahmoud M(2024)Safeguarding Identities with GAN-based Face AnonymizationEngineering, Technology & Applied Science Research10.48084/etasr.752714:4(15581-15589)Online publication date: 2-Aug-2024
https://doi.org/10.48084/etasr.7527
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Issue’s Table of Contents