literature
Literature
Authors
Jobs
Seminars
Conferences
DataBETA

Guided Image Generation with Conditional Invertible Neural Networks

Jul 4, 2019
e-Print:
View in:
pdf
reference search35 citations

Citations per year

201920202021202220230246810
Abstract: (submitter)
In this work, we address the task of natural image generation guided by a conditioning input. We introduce a new architecture called conditional invertible neural network (cINN). The cINN combines the purely generative INN model with an unconstrained feed-forward network, which efficiently preprocesses the conditioning input into useful features. All parameters of the cINN are jointly optimized with a stable, maximum likelihood-based training procedure. By construction, the cINN does not experience mode collapse and generates diverse samples, in contrast to e.g. cGANs. At the same time our model produces sharp images since no reconstruction loss is required, in contrast to e.g. VAEs. We demonstrate these properties for the tasks of MNIST digit generation and image colorization. Furthermore, we take advantage of our bi-directional cINN architecture to explore and manipulate emergent properties of the latent space, such as changing the image style in an intuitive way.
  • [1]
    Analyzing inverse problems with invertible neural networks. In Intl. Conf. on Learning Representations,. 1, 3
    • L. Ardizzone
      ,
    • J. Kruse
      ,
    • C. Rother
      ,
    • U. Köthe
  • [2]
    Invertible residual networks
    • J. Behrmann
      ,
    • D. Duvenaud
      ,
    • J.-H. Jacobsen
  • [3]
    Large scale GAN training for high fidelity natural image synthesis. In Intl. Conf. on Learning Representations,. 1, 2
    • A. Brock
      ,
    • J. Donahue
      ,
    • K. Simonyan
  • [4]
    Unsupervised diverse colorization via generative adversarial networks. In Joint Europ. Conf. on Machine Learning and Knowledge Discovery in Databases, pages 151-166
    • Y. Cao
      ,
    • Z. Zhou
      ,
    • W. Zhang
      ,
    • Y. Yu
  • [5]
    Comparison of maximum likelihood and GAN-based training of RealNVPs
    • I. Danihelka
      ,
    • B. Lakshminarayanan
      ,
    • B. Uria
      ,
    • D. Wierstra
      ,
    • P. Dayan
  • [6]
    Learning diverse image colorization. In Conf. on Computer Vision and Pattern Recognition (CVPR), pages 6837-6845,. 3, 8
    • A. Deshpande
      ,
    • J. Lu
      ,
    • M.-C. Yeh
      ,
    • M. Jin Chong
      ,
    • D. Forsyth
  • [9]
    A learned representation for artistic style. In Intl. Conf. on Learning Representations,. 2
    • V. Dumoulin
      ,
    • J. Shlens
      ,
    • M. Kudlur
  • [10]
    Understanding the difficulty of training deep feedforward neural networks. In Proc. 13. Intl. Conf. Artificial Intelligence and Statistics, pages 249-256,. 4
    • X. Glorot
      ,
    • Y. Bengio
  • [11]
    Flow-GAN: combining maximum likelihood and adversarial learning in generative models. In Thirty-Second AAAI Conference on Artificial Intelligence,. 3
    • A. Grover
      ,
    • M. Dhar
      ,
    • S. Ermon
  • [12]
    Pixcolor: Pixel recursive colorization
    • S. Guadarrama
      ,
    • R. Dahl
      ,
    • D. Bieber
      ,
    • M. Norouzi
      ,
    • J. Shlens
    et al.
  • [13]
    Zur Theorie der orthogonalen Funktionensysteme
    • A. Haar
      • Math.Ann. 69 (1910) 331-371
  • [14]
    GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems, pages 6626-6637,. 8
    • M. Heusel
      ,
    • H. Ramsauer
      ,
    • T. Unterthiner
      ,
    • B. Nessler
      ,
    • S. Hochreiter
  • [15]
    Arbitrary style transfer in realtime with adaptive instance normalization. In ICCV’17, pages 1501-1510,. 2
    • X. Huang
      ,
    • S. Belongie
  • [16]
    Let there be color! joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification
    • S. Iizuka
      ,
    • E. Simo-Serra
      ,
    • H. Ishikawa
      • ACM Trans.Graph. 35 (2016) 110
  • [17]
    Image-toimage translation with conditional adversarial networks. In CVPR’17, pages 1125-1134,. 1, 2, 3, 6, 8
    • P. Isola
      ,
    • J.-Y. Zhu
      ,
    • T. Zhou
      ,
    • A.A. Efros
  • [18]
    Excessive invariance causes adversarial vulnerability. arXiv preprint
    • J.-H. Jacobsen
      ,
    • J. Behrmann
      ,
    • R. Zemel
      ,
    • M. Bethge
  • [19]
    iRevNet: deep invertible networks. In International Conference on Learning Representations,. 2
    • J.-H. Jacobsen
      ,
    • A.W. Smeulders
      ,
    • E. Oyallon
  • [20]
    Bidirectional conditional generative adversarial networks
    • A. Jaiswal
      ,
    • W. AbdAlmageed
      ,
    • Y. Wu
      ,
    • P. Natarajan
  • [23]
    Improved variational inference with inverse autoregressive flow. In Advances in Neural Information Processing Systems, pages 4743-4751,. 1
    • D.P. Kingma
      ,
    • T. Salimans
      ,
    • R. Jozefowicz
      ,
    • X. Chen
      ,
    • I. Sutskever
    et al.
  • [25]
    CausalGAN: Learning causal implicit generative models with adversarial training
    • M. Kocaoglu
      ,
    • C. Snyder
      ,
    • A.G. Dimakis
      ,
    • S. Vishwanath
  • 1-25 of 44
  • 1
  • 2
  • 25 / page