Predicting novel views using generative adversarial query network

The problem of predicting a novel view of the scene using an arbitrary number of observations is a challenging problem for computers as well as for humans. This paper introduces the Generative Adversarial Query Network (GAQN), a general learning framework for novel view synthesis that combines Generative Query Network (GQN) and Generative Adversarial Networks (GANs). The conventional GQN encodes input views into a latent representation that is used to generate a new view through a recurrent variational decoder. The proposed GAQN builds on this work by adding two novel aspects: First, we extend the current GQN architecture with an adversarial loss function for improving the visual quality and convergence speed. Second, we introduce a feature-matching loss function for stabilizing the training procedure. The experiments demonstrate that GAQN is able to produce high-quality results and faster convergence compared to the conventional approach.

Nguyen-Ha Phong, Huynh Lam, Rahtu Esa, Heikkilä Janne

Publication type:
A4 Article in conference proceedings

Place of publication:
Image analysis : 21st Scandinavian Conference, SCIA 2019, Norrköping, Sweden, June 11–13, 2019, proceedings

Generative Adversarial Query Network, Mean feature matching loss, Novel view synthesis


Full citation:
Nguyen-Ha P., Huynh L., Rahtu E., Heikkilä J. (2019) Predicting Novel Views Using Generative Adversarial Query Network. In: Felsberg M., Forssén PE., Sintorn IM., Unger J. (eds) Image Analysis. SCIA 2019. Lecture Notes in Computer Science, vol 11482. Springer, Cham.


Read the publication here: