VENI: Variational Encoder for Natural Illumination

Paul Walker1, James A. D. Gardner2, Andreea Ardelean1, William A. P. Smith2, Bernhard Egger1,
1Friedrich-Alexander-Universität Erlangen-Nürnberg, 2University of York
Paper arXiv Code
VENI produces a better-behaved latent space compared to the current state of the art illumination prior (RENI++), which we demonstrate by evaluating its uniqueness through optimization of random latent codes and its reconstruction consistency via the correlation between latent space and image space distances.

Abstract

Inverse rendering is an ill-posed problem, but priors like illumination priors, can simplify it. Existing work either disregards the spherical and rotation-equivariant nature of illumination environments or does not provide a well-behaved latent space. We propose a rotation-equivariant variational autoencoder that models natural illumination on the sphere without relying on 2D projections. To preserve the SO(2)-equivariance of environment maps, we use a novel Vector Neuron Vision Transformer (VN-ViT) as encoder and a rotation-equivariant conditional neural field as decoder. In the encoder, we reduce the equivariance from SO(3) to SO(2) using a novel SO(2)-equivariant fully connected layer, an extension of Vector Neurons. We show that our SO(2)-equivariant fully connected layer outperforms standard Vector Neurons when used in our SO(2)-equivariant model. Compared to previous methods, our variational autoencoder enables smoother interpolation in latent space and offers a more well-behaved latent space.

Rotation Equivariance

An important feature of VENI is rotation equivariance. A rotation of the input causes the same rotation in the latent code which causes the same rotation in the output.

Reconstruction Quality

VENI achieves higher reconstruction quality compared to the current state of the art illumination prior (RENI++) and does especially well with low-dimensional latent spaces (27) compared to RENI++.

Acknowledgments

The authors gratefully acknowledge the scientific support and HPC resources provided by the Erlangen National High Performance Computing Center (NHR@FAU) of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). The hardware is funded by the German Research Foundation (DFG).

BibTeX

@misc{walker2026veni,
  author        = {Paul Walker and James A. D. Gardner and Andreea Ardelean and William A. P. Smith and Bernhard Egger},
  title         = {VENI: Variational Encoder for Natural Illumination},
  eprint        = {},
  archivePrefix = {},
  primaryClass  = {},
  year          = {2026},
  month         = {Jan}
}