Item

Abstract

Differentiable rendering allows the application of computer graphics on
vision tasks, e.g. object pose and shape fitting, via analysis-by-synthesis,
where gradients at occluded regions are important when inverting the rendering
process. To obtain those gradients, state-of-the-art (SoTA) differentiable
renderers use rasterization to collect a set of nearest components for each
pixel and aggregate them based on the viewing distance. In this paper, we
propose VoGE, which uses ray tracing to capture nearest components with their
volume density distributions on the rays and aggregates via integral of the
volume densities based on Gaussian ellipsoids, which brings more efficient and
stable gradients. To efficiently render via VoGE, we propose an approximate
close-form solution for the volume density aggregation and a coarse-to-fine
rendering strategy. Finally, we provide a CUDA implementation of VoGE, which
gives a competitive rendering speed in comparison to PyTorch3D. Quantitative
and qualitative experiment results show VoGE outperforms SoTA counterparts when
applied to various vision tasks,e.g., object pose estimation, shape/texture
fitting, and occlusion reasoning. The VoGE library and demos are available at
https://github.com/Angtian/VoGE.