Radiance Caching for Dierentiable Path Tracing
ZIYI ZHANG, École Polytechnique Fédérale de Lausanne (EPFL) and Google, Switzerland
DELIO VICINI, Google, Switzerland
SEBASTIAN WINBERG, Google, Switzerland
STEPHAN GARBIN, Google, United Kingdom
WENZEL JAKOB, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland
Ground truth
(a) (b)
(c) Material rendering under new lighting
Initialization
Reference images
Cache estimator Material estimator
Blending
Ground truth Ours
Hadadan [2023] PRB
Ours Hadadan et al. [2023] PRB
Error
Fig. 1. (a) At each surface interaction, we estimate outgoing radiance by blending a cache estimator (querying a learned radiance cache) and a material
estimator (continuing path tracing through BSDFs). A spatial blending field
𝛼 (
x
)
learns where to trust each estimator. Our training discourages degenerate
decompositions where either estimator is only meaningful inside the blend, enabling reuse of the recovered materials for editing and relighting. (b) Starting
from a rough material initialization (visualized under unknown lighting), we jointly optimize cache, materials, and
𝛼
. (c) We discard cache and
𝛼
and render
material-only under an unseen relighting condition, comparing against Hadadan et al. [2023] and PRB [Vicini et al. 2021].
Dierentiable path tracing oers a principled route to recovering physical
material and lighting parameters, but the combination of high variance and
poor numerical conditioning often makes it too brittle to use in practice.
This is especially the case when lighting is altogether unknown, or when the
scene contains complex light transport eects. Prior work recently showed
that the variance reduction provided by a radiance cache can alleviate these
challenges.
Authors’ Contact Information: Ziyi Zhang, École Polytechnique Fédérale de Lausanne
(EPFL) and Google, Lausanne, Switzerland; Delio Vicini, Google, Zurich, Switzerland;
Sebastian Winberg, Google, Zurich, Switzerland; Stephan Garbin, Google, London,
United Kingdom; Wenzel Jakob, École Polytechnique Fédérale de Lausanne (EPFL),
Lausanne, Switzerland.
© 2026 Copyright held by the owner/author(s).
This is the author’s version of the work. It is posted here for your personal use. Not for
redistribution. The denitive Version of Record was published in ACM Transactions on
Graphics, https://doi.org/10.1145/3811398.
We revisit the combination of inverse rendering and radiance caching
with a twist, by introducing a spatial blending eld that locally interpolates
between the cache and standard unbiased estimators. Recursive applica-
tion of this idea yields a rich design space of evaluation strategies and
inter-estimator consistency losses; we map this space and identify eec-
tive components. A surprising property of the resulting algorithm is that it
can accurately recover material parameters even when the lighting is not
uniquely identiable from the observations. Our experiments demonstrate
signicant improvements in speed and robustness over prior work, making
a strong case for including radiance caching as a standard component of
future physically based inverse rendering systems.
CCS Concepts: • Computing methodologies → Rendering.
Additional Key Words and Phrases: dierentiable rendering, inverse render-
ing, path tracing, radiance caching, material reconstruction
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:2 • Zhang et al.
ACM Reference Format:
Ziyi Zhang, Delio Vicini, Sebastian Winberg, Stephan Garbin, and Wenzel
Jakob. 2026. Radiance Caching for Dierentiable Path Tracing. ACM Trans.
Graph. 45, 4, Article 135 (July 2026), 17 pages. https://doi.org/10.1145/3811398
1 Introduction
Joint physically based recovery of geometry, materials, and lighting
is a long-standing goal in graphics and vision, but the ambiguous
nature of this problem keeps it out of reach of current methods.
We tackle an important subproblem: given reference images and
geometry (e.g., from a sensor, multi-view reconstruction, or a ten-
tative estimate in an iterative pipeline), recover materials under
unknown illumination. Even this reduced problem defeats current
dierentiable path tracing methods, which are fragile and often fail
to recover acceptable solutions.
Three fundamental issues compound to cause this fragility: high-
variance radiance estimates in the path tracer that lead to noisy
gradients, poor numerical conditioning that amplies sensitivity
to noise and initialization, and an optimization landscape that is
riddled with local minima. Increasing the sample count can reduce
Monte Carlo noise, but the cost of reducing it to acceptable levels is
prohibitive and would address only one of the three issues.
Along a light path, multiple unknown materials contribute multi-
plicatively to the nal pixel value, making it dicult for gradient
descent to disentangle their individual contributions. Unknown
lighting worsens this problem because the optimizer can explain
reference images simply by placing emission everywhere, which
technically ts the data but is not the desired material-lighting
decomposition.
A radiance cache can alleviate the noise issue by replacing high-
variance Monte Carlo estimates with a learned cache that approx-
imates the outgoing radiance at surface points. Radiance caches
have proven highly eective in forward rendering; for example,
Müller et al
.
[2021] demonstrated that the overhead of training and
querying a neural radiance cache can be low enough for real-time
path tracing. Despite pioneering work by Hadadan et al. [2023]
for variance reduction, the use of radiance caching for physically
based inverse rendering remains underexplored. We show that this
is a potent combination that can improve the well-posedness of the
problem, and present a general framework of which Hadadan et
al.’s method is a special case.
To build intuition, consider light from a lamp reaching a chair in-
directly after reecting o the ceiling. A standard path tracer would
estimate this radiance by tracing a ray to the ceiling and then either
sampling or evaluating its BSDF to connect to the light source. We
refer to this approach as the material estimator because it simulates
0.0
1.0
Material estimator Cache estimator Blending field
Blended radianceTarget
Lerp
Loss
Fig. 2. Naive blending is ill-posed. Optimizing only the blended radiance
can match the reference image, yet the cache-only estimate (queried at
camera-ray intersections) and the material-only estimate (path tracing with-
out cache) remain incorrect. The resulting parameters are not meaningful
in isolation and cannot be reliably reused for relighting or editing.
the material’s reectance behavior. Alternatively, if we maintain
a radiance cache storing outgoing radiance on surfaces, we could
query it directly at the ceiling intersection and terminate the path.
This cache estimator avoids further path tracing and BSDF evalua-
tion; with an accurate cache, the two estimators agree. In practice,
they exhibit complementary failure modes: the cache estimator is
fast and low-variance but may be unreliable if the cache is uncon-
verged, has insucient capacity, or cannot represent high-frequency
directional variation; the material estimator captures such eects
more easily but produces noisy estimates that can impede or break
the optimization.
A natural idea is to blend these two estimators adaptively. We
introduce a spatially varying blending eld
𝛼 (
x
) ∈ [
0
,
1
]
and dene
a blended estimator
ˆ
𝐿
𝑜
(x, 𝝎 ) =
1 − 𝛼 (x)
ˆ
𝐿
mat
(x, 𝝎 ) + 𝛼 (x)
ˆ
𝐿
cache
(x, 𝝎 ), (1)
where
ˆ
𝐿
mat
and
ˆ
𝐿
cache
are radiance estimates from the material and
cache estimators, respectively. Intuitively,
𝛼 (
x
)
encodes how much
we trust the cache at x: when
𝛼 (x) ≈ 1
, we terminate paths early us-
ing the cache; when
𝛼 (x) ≈ 0
, we fall back to the material estimator
to capture complex directional eects.
The blended estimator is simple to implement with a few addi-
tional lines of code, and it quickly drives the renderer toward the
reference. However, as illustrated in Figure 2, it suers from a funda-
mental decomposition problem: optimizing the blend constrains the
estimators in combination but not individually. Even when the blend
perfectly matches the reference, the material and cache estimators
on their own may be far from the true solution. In other words,
the reconstruction is only valid for the particular spatially varying
blend used during training, which limits its reuse for downstream
tasks such as rasterization, relighting, or material editing.
In the remainder of this paper, we take a step toward a more
comprehensive understanding of radiance caching for inverse ren-
dering. We explore and compare several optimization strategies
beyond naive blending to pursue two goals: to encourage the cache
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:3
and material estimators to remain useful in isolation, and to learn
a blending eld that automatically favors the more reliable estima-
tor at each location. Compared to pure dierentiable path tracing,
our experiments suggest that this joint formulation is particularly
benecial in three respects:
(1)
Robustness to unknown lighting: We obtain stable re-
constructions without known emitters. Light paths need not
reach an emitter; the cache provides reasonable incident radi-
ance at intermediate points, preventing the optimizer from
collapsing to degenerate solutions.
(2)
Spatially adaptive estimator selection: The learned blend-
ing eld
𝛼 (
x
)
allows the optimizer to rely on the cache in
regions requiring long indirect paths, while falling back to
the material estimator near glossy surfaces where the cache
is biased.
(3)
Improved behavior in high-variance regimes: In scenes
with complex global illumination, the cache terminates paths
earlier and reduces variance, making optimization tractable
in cases where pure path tracing struggles.
2 Related Work
2.1 Radiance caching
Early work on irradiance caching [Ward et al
.
1988; Greger et al
.
1998] reduced the cost of global illumination by evaluating low-
frequency indirect irradiance at sparse points and interpolating it
during rendering. Krivanek et al
.
[2005] extended this idea to radi-
ance caching, which stores direction-dependent outgoing radiance.
A key challenge for caching methods is artifact-free interpolation
from sparse samples without blur or light leaks [Křivánek et al
.
2007;
Jarosz et al
.
2008; Marco et al
.
2018; Binder et al
.
2018]. Neural radi-
ance caching [Müller et al
.
2021] replaces explicit interpolation of
precomputed samples with an online-learned, continuous radiance
predictor trained directly from noisy Monte Carlo estimates. We
similarly t a cache online but focus on its use in inverse render-
ing. Recently, variations of radiance caching have seen widespread
use for real-time path tracing [Müller et al
.
2021; Silvennoinen and
Lehtinen 2017; Dereviannykh et al
.
2025; AMD 2025; NVIDIA 2024].
Recent forward-rendering work also studies surface-based cache
representations and explicit path-termination tradeos: Tatzgern
et al
.
[2024] use on-surface radiance caches for real-time global
illumination, while Kandlbinder et al
.
[2024] optimize when paths
should terminate into a cache by explicitly trading bias and variance.
Our setting diers because the cache, materials, and blending eld
are optimized from image observations, so termination choices must
also provide useful inverse-rendering supervision.
2.2 Dierentiable path tracing
There is a rich literature on dierentiable path tracing for inverse
rendering [Gkioulekas et al
.
2016; Li et al
.
2018; Zhang et al
.
2019;
Azinović et al
.
2019; Zhao et al
.
2016; Khungurn et al
.
2015; Velinov
et al
.
2018; Nimier-David et al
.
2019]. Central challenges for dier-
entiable path tracing are visibility discontinuities [Li et al
.
2018;
Loubet et al
.
2019; Zhang et al
.
2019; Bangaru et al
.
2020; Zhang
et al
.
2023a] and the memory use of the backward pass [Nimier-
David et al
.
2020; Vicini et al
.
2021]. Besides the usual Monte Carlo
noise in incident radiance estimates, gradient estimation introduces
additional derivative terms that are often badly matched to primal
sampling strategies; consequently, the variance of gradients can
become arbitrarily large [Zeltner et al
.
2021; Nimier-David et al
.
2022]. High variance can slow convergence or cause the optimiza-
tion to fail, especially with nonlinear loss functions [Nicolet et al
.
2023]. Variance reduction strategies include importance sampling
of local gradient terms [Zeltner et al
.
2021; Belhe et al
.
2024], path
guiding [Fan et al
.
2024], reservoir sampling [Wang et al
.
2023] and
control variates [Nicolet et al
.
2023; Lu et al
.
2025]. The impact of
gradient noise can also be mitigated using preconditioning [Nicolet
et al
.
2021; Weier et al
.
2025] or ltering [Chang et al
.
2024]. We
focus on reducing variance due to complex global illumination and
unknown emitters using radiance caching. This is largely orthogo-
nal and could be combined with many of the above techniques. In
particular, Parameter-space ReSTIR improves sampling eciency by
reusing samples in parameter space, whereas our method changes
the estimator decomposition through a cache/material split and
separated image-space losses; ReSTIR-style sample reuse could in
principle be used inside our material estimator or separated-loss
samples [Chang et al
.
2023]. Adjacent to our work are variance-
aware optimization techniques [Weier et al
.
2021; Yan et al
.
2024].
Accounting for rendering variance explicitly is an interesting future
direction.
2.3 Radiance caching in inverse rendering
Several works have explored the application of radiance caching
for inverse rendering. The most similar to our work is the neural
radiometric prior [Hadadan et al
.
2023], which we discuss in detail
below. We also review the use of radiance caching with radiance
eld representations (e.g., NeRFs) and alternative approaches.
Neural radiometric prior. Hadadan et al
.
[2023] augmented a dif-
ferentiable path tracer with a neural radiance cache, demonstrating
improvements over unbiased dierentiable path tracing [Vicini et al
.
2021]. Their method evaluates the cache at a xed path depth (typ-
ically at the rst surface hit) and only accumulates gradients into
the material at that interaction, even if the path continues further.
Our method generalizes their approach by removing both limita-
tions and introducing a more general family of consistency losses.
Section 4 provides detailed comparisons.
Radiance eld methods. Following the seminal work on neural ra-
diance elds (NeRFs) [Mildenhall et al
.
2020], there has been contin-
ued interest in developing relightable variants of radiance-eld-like
representations. Due to the high evaluation cost of neural radi-
ance elds, most methods in that space try to limit the number of
evaluated ray queries or interactions. Initially, Zhang et al
.
[2021]
augmented NeRF with a direct illumination term. Since NeRFs repre-
sent a scene’s radiance eld, a natural extension is to try to leverage
that eld directly as a radiance cache to render indirect illumina-
tion [Zhang et al
.
2022; Jin et al
.
2023; Sun et al
.
2025]. Gu et al
.
[2025] similarly compute a single indirect bounce on a 2D Gaussian
representation [Kerbl et al
.
2023; Huang et al
.
2024]. Variations of
this idea are neural incident radiance elds [Yao et al
.
2022; Wu et al
.
2023; Zhang et al
.
2023b; Wu et al
.
2025]. These methods represent
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:4 • Zhang et al.
both outgoing and incident radiance elds, which allows them to
avoid recursive ray tracing to evaluate the radiance cache. Similarly,
neural elds have also been used for more accurate emitter modeling
in physically based inverse rendering [Ling et al
.
2024]. Generally,
NeRF-based methods avoid multi-bounce ray tracing due to the
high computational cost. For that reason, the quality of results can
deteriorate if the cache is not expressive enough. Attal et al
.
[2024]
recognized this problem and proposed a few strategies to reduce
bias, but their method also does not trace multiple light bounces. We
instead use multi-bounce path tracing to disentangle materials from
the cache and further reduce bias from the inherent limitations of
radiance caching. Our method thus exposes the classic bias-variance
tradeo.
Alternative approaches. Radiance caching is not the only way to
avoid recursive ray tracing in inverse rendering. Jiang et al
.
[2025]
perform an iterative radiosity solve directly on 3D Gaussian primi-
tives. Poirier-Ginter et al
.
[2025] only path trace specular transport
and cache the diuse component. This improves novel view syn-
thesis but does not directly enable relighting. Finally, Hadadan and
Zwicker [2024] cache dierential quantities at the cost of limiting
reconstruction to low-dimensional parameter spaces.
3 Method
As previously discussed, naive optimization of the blended estimator
is under-constrained, as the model can perfectly match the reference
by arbitrarily blending two individually incorrect estimators. In
this section, we explore several loss formulations to resolve this
ambiguity and furthermore learn the blending eld 𝛼 (x).
Our starting point is the rendering equation, which relates out-
going and incident radiance at a surface point:
𝐿
o
(x, 𝜔) = 𝐿
𝑒
(x, 𝜔) +
∫
S
2
𝐿
𝑖
(x, 𝜔
′
) 𝑓 (x, 𝜔
′
, 𝜔) d𝜔
′⊥
, (2)
where
𝐿
𝑒
(
x
, 𝜔)
and
𝐿
𝑖
(
x
, 𝜔
′
)
denote emitted and incident radiance,
𝑓 (
x
, 𝜔
′
, 𝜔)
is the BSDF, and d
𝜔
′⊥
includes the projected-solid-angle
factor. A path tracer estimates the integral by sampling a direction
𝜔
′
and recursively estimating incident radiance. For now, we assume
the emission is either known or zero. At each surface interaction,
our blended estimator interpolates two approaches:
ˆ
𝐿
𝑜
(x, 𝜔) = (1 − 𝛼 (x))
𝑓 (x, 𝜔
′
, 𝜔) cos 𝜃
′
𝑝 (𝜔
′
)
ˆ
𝐿
𝑖
(x, 𝜔
′
)
| {z }
Material
+𝛼(x) 𝐶 (x, 𝜔)
| {z }
Cache
,
(3)
where
𝛼 (
x
) ∈ [
0
,
1
]
models our trust in the cache at position x,
𝐶
is
a learned cache storing outgoing radiance,
𝜃
′
is the angle between
𝜔
′
and the surface normal, and
𝑝 (𝜔
′
)
is the sampling probability.
In the known-emission discussion below,
𝐶
may be interpreted as
scattered outgoing radiance; in our unknown-emission formulation
in Section 3.2, it stores total outgoing radiance including emission.
The illustration below shows the computation graph of a direct
optimization of this recursive estimator. All terms receive gradients
(gray boxes), but only their weighted combination is constrained,
which leads to an ill-posed problem.
BSDF
… …
Loss
Cache
BSDF
Cache
BSDF
Cache
During training, we render with this blended estimator, but the
novelty of our method lies in how we train the full set of parameters
(cache, materials, and
𝛼
) by imposing additional losses that enforce
the individual correctness of the estimators.
3.1 Consistency losses
We rst present two intuitive consistency-loss formulations to ad-
dress this issue but then show that both are inherently awed.
Depending on the optimized parameter, we can enforce consis-
tency in either direction: by training the cache to match the material
estimator (Section 3.1.1), or vice versa (Section 3.1.2). Both losses
follow directly from the rendering equation, but require care in
choosing which terms to dierentiate and how to sample them to
obtain unbiased gradients.
We assume here that the consistency losses are applied in the
context of minimizing an image loss based on the blended estimator,
which also optimizes the blending eld. If both estimators are useful
approximations of the same outgoing radiance, blending becomes
primarily a bias-variance tradeo:
𝛼
selects whichever estimator is
more reliable at each location.
3.1.1 Cache consistency loss. The rst loss formulation trains the
cache
𝐶
to match the reected-radiance integral in the rendering
equation:
L
cache
(x, 𝜔) =
𝐶 (x, 𝜔) −
∫
S
2
𝑓 (x, 𝜔
′
, 𝜔) cos𝜃
′
𝐿
𝑖
(x, 𝜔
′
)d𝜔
′
2
rel
.
(4)
The loss is parameterized by x and
𝜔
. In practice, it would be used by
accumulating gradients with respect to
L
cache
whenever the path
tracer encounters a vertex x from direction
𝜔
and then samples
a scattered direction
𝜔
′
, which also provides the direction for a
single-sample estimate of the integral in Equation 4. This focuses
the consistency optimization on the spatio-directional subset of ray
space that actually contributes to the rendered image. Imposing this
loss at dierent path depths leads to the following graph structure
(with gray terms receiving gradients while others are held xed).
BSDF
… …
BSDF
Cache
BSDF
Cache
Loss
… …
BSDF BSDF
Cache
Loss
… …
This is the loss used by Müller et al
.
[2021] to train a neural ra-
diance cache in forward rendering, where materials and emitters
are known and the target is therefore a consistent Monte Carlo
estimate. The target is a one-sample Monte Carlo estimate, which is
noisy; following their approach, we use a relative
𝐿
2
loss so that the
cache learns the expected value rather than overtting to individual
samples [Lehtinen et al
.
2018]. In practice, we substitute the blended
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:5
estimate for
ˆ
𝐿
𝑖
, which reduces variance and reuses values already
computed along the path.
3.1.2 Material consistency loss. Conversely, we can train the BSDF
to match the cache. If we use
𝐶
to evaluate incident and outgoing
radiance, the rendering equation provides a training signal for 𝑓 :
L
mat
(x, 𝜔) =
𝐶 (x, 𝜔) −
∫
S
2
𝑓 (x, 𝜔
′
, 𝜔) cos𝜃
′
𝐶 (x
′
, −𝜔
′
) d𝜔
′
2
rel
,
(5)
where x
′
is the surface point visible from x in direction
𝜔
′
. Here,
gradients only propagate through the BSDF (gray box), while the
other terms are held xed. As before, this loss can be imposed at
dierent path depths:
Loss
BSDF
Loss
BSDF
… …
Unfortunately, the simple approach for constructing a one-sample
derivative estimator that worked in Section 3.1.1 now fails and pro-
duces biased gradients. The issue arises from the combination of the
nonlinear
𝐿
2
loss and the fact that we are now propagating deriva-
tives into the integral. Appendix A provides more detail and presents
a decorrelated two-sample estimator that resolves the issue.
3.1.3 Limitations. The consistency losses suer from multiple fun-
damental limitations:
•
Directional mismatch. The cache consistency loss propagates
information from the emitters toward the camera and is there-
fore well-suited to forward rendering, where known emitter
and material parameters are used to infer the observed image.
In inverse rendering, the supervision comes from reference im-
agery at the camera. The loss therefore propagates information
counter to the direction of supervision, making it inherently
less suitable. Along a camera-to-light path
(
x
1
, . . . ,
x
𝑛
)
, cache
consistency trains
𝐶 (
x
𝑖
)
from the sux
(
x
𝑖+1
, . . . ,
x
𝑛
)
. This is
appropriate when emitters/materials are known, but with un-
known emission the reliable signal is the observed pixel at x
1
, so
supervision ows away from the data source.
•
Garbage in, garbage out. Early in optimization, both the ma-
terials and the cache are unreliable (e.g., randomly initialized).
Training with a consistency loss then simply transfers bad data
from one representation to the other.
This is particularly problematic for material consistency: the
cache has limited capacity and cannot represent the radiance
function with high accuracy. High-frequency spatio-directional
variation may be severely distorted or lost entirely, preventing
the cache from ever becoming an authoritative training signal
for materials.
•
Unknown emission. Both losses assume non-emissive surfaces,
yet emission must clearly be present for any radiance to exist.
Generalizing the formulations to include an emission term does
not help: reected and emitted radiance cannot be separated
without additional constraints, so the consistency relationship
no longer denes a meaningful target for either estimator.
BSDF
Loss
Cache
BSDF
Cache
BSDF
Cache
(a) Blended loss
(b)
Separated losses
Loss 1
BSDF
Loss 3
Cache
BSDF
Cache
BSDF
Loss 2
Cache
… …
Fig. 3. Separated losses. (a) A single blended loss supervises only the
recursively mixed estimator, which can leave the cache and material factors
under-constrained in isolation. (b) We instead define pixel losses that force
the path to terminate at a chosen bounce
𝑘
, using the cache value
𝐶
𝑘
as
the endpoint. Each such loss backpropagates to
𝐶
𝑘
and to the BSDF factors
before it, removing the “only-the-blend-is-supervised” ambiguity.
These limitations motivate an alternative approach. We modify
the role of the cache so that it represents total outgoing radiance
(i.e., including both emission and scattering) instead of trying to
separate their individual contributions. On top of this representation,
we introduce a unied family of losses that simultaneously constrain
both estimators with respect to the reference images. This drives
gradients from the camera into the scene, aligning the natural ow
of information with the inverse rendering problem.
3.2 Separated estimators and losses
The cache, material, and blended estimators all aim to recover out-
going radiance 𝐿
o
. At any surface interaction along a path, we can
therefore terminate via a cache lookup or continue recursively, yield-
ing distinct image-space losses. Satisfying these losses discourages
decompositions where either estimator is meaningless outside the
blend.
Let
(
x
𝑗
, 𝜔
𝑗
)
be the
𝑗
-th surface interaction, with
𝑗 =
1 at the
primary hit. As illustrated in Figure 3, instead of matching only the
blended rendering, we sample a forced termination depth; across
iterations, dierent depths supervise dierent combinations of cache
values and material factors. Terminating at depth
𝑘
while using the
blended estimator at earlier vertices gives:
ˆ
𝐿
(𝑘 )
𝑗
=
(1−𝛼
𝑗
)
𝑓
𝑗
cos𝜃
𝑗
𝑝
𝑗
ˆ
𝐿
(𝑘 )
𝑗+1
+𝛼
𝑗
𝐶
𝑗
, 𝑗 < 𝑘,
𝐶
𝑘
, 𝑗 = 𝑘.
(6)
Here
𝐶
𝑗
= 𝐶 (
x
𝑗
, 𝜔
𝑗
)
,
𝛼
𝑗
= 𝛼 (
x
𝑗
)
, and
𝑓
𝑗
is the sampled BSDF
factor. For
𝑘 =
1, the image loss directly trains the primary-hit cache;
for
𝑘 =
2, it trains the rst-bounce BSDF using cached radiance
at the second hit; larger
𝑘
values supervise longer material chains
before cache termination.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:6 • Zhang et al.
Listing 1. Pseudocode of the separated radiance estimator (Equation 6) us-
ing a blending field-aware stopping criterion (Appendix C) with threshold
𝜏
.
The structure of this algorithm resembles standard (dierentiable) path
tracing and only requires minimal changes in existing systems.
1 def render_pixel(pixel_uv, 𝜏):
2 x, 𝝎 = generate_camera_ray(pixel_uv)
3 return Li(x, 𝝎, 𝜏 )
4
5 def Li(x, 𝝎 , 𝜏):
6 𝐿 = 0; 𝛽 = 1; 𝛼
prod
= 1
7 for _ in range(MAX_DEPTH):
8 x = ray_intersect(x, 𝝎 )
9 𝛼, 𝐶 = eval_cache_alpha(x)
10 # Path termination check (separated estimator)
11 if 𝛼
prod
* (1 - 𝛼 ) < 𝜏:
12 return 𝐿 + 𝛽 * 𝐶
13 # Accumulate cache contribution and continue path
14 𝐿 += 𝛽 * 𝛼 * 𝐶
15 𝝎 , 𝛽
bsdf
= sample_bsdf(x, 𝝎)
16 𝛽 *= (1 - 𝛼) * 𝛽
bsdf
; 𝛼
prod
*= (1 - 𝛼 )
17 return 𝐿
Implementation. A naive optimization approach would simulta-
neously compute and optimize renderings for
𝑘 ∈ {
1
, . . . 𝑘
max
}
up to
some maximum path depth
𝑘
max
, but this is too memory-intensive.
Instead, we can sample a dierent termination depth
𝑘
each itera-
tion, keeping memory constant. The distribution over
𝑘
determines
the relative weight of each separated loss.
Termination strategies. One relevant concern is that simple uni-
form sampling may terminate at vertices with a low value of
𝛼
,
propagating errors from an unreliable cache to upstream BSDFs.
In practice, optimizers like Adam are fairly robust to occasional
bad gradients. Nonetheless, an
𝛼
-aware strategy that extends the
path beyond the sampled depth until
Î
(
1
−𝛼)
falls below a threshold
avoids low-
𝛼
endpoints and improves stability by a small margin
(∼0.4 dB PSNR across scenes). We detail options in Appendix C.
Listing 1 shows a pseudocode implementation of the forward
pass. The reverse-mode derivative of this algorithm can be evaluated
using path replay backpropagation [Vicini et al. 2021].
3.3 Optimizing the blending field
The blending eld
𝛼 (
x
)
locally controls how much we rely on the
cache versus the material estimator. Consider setting
𝛼 =
0 ev-
erywhere: paths would then use only the material estimator, ter-
minating using the cache when reaching a maximum depth. This
works but sacrices two benets: blending at earlier vertices reduces
variance, and the existence of a family of losses that probe both
estimators in dierent ways yields a better-posed inverse problem.
Section 4 analyzes these benets in more detail.
We instead optimize
𝛼
jointly with the cache and scene parame-
ters, treating it as a learned condence eld. Since the true outgoing
radiance is unknown during optimization, either estimator may be
more accurate at any given point and training stage. Early on, mate-
rials may be far from correct while the cache already captures much
of the signal. Learning
𝛼
lets the system favor the more reliable
estimator at each location and adapt as both evolve.
Fig. 4. Adaptive estimator selection via a learned blending field. (a)
Optimized blending field
𝛼 (
x
)
. Blue colors indicate higher reliance on the
cache estimator. (b) Rendering using the optimized materials with no cache.
(c) Rendering using only the cache at camera-ray intersection points. The
blending field
𝛼 (
x
)
increases reliance on the cache where indirect transport
is high-variance and reduces reliance where the cache is wrong (e.g., specular
eects).
Bias-driven optimization. We optimize
𝛼
so that it favors the es-
timator with lower bias, which requires decorrelating the loss and
gradient evaluations following standard practice in dierentiable
rendering [Gkioulekas et al
.
2016; Azinović et al
.
2019]. This pro-
vides a graceful fallback when either estimator struggles, which
improves robustness. When the material estimator is unreliable in
a region (e.g., due to high variance, poor initialization, or dicult
transport),
𝛼
naturally increases, and the system becomes more
reliant on the cache. Conversely, where the cache is unreliable (e.g.,
on the mirror in Figure 4),
𝛼
decreases and the system falls back to
path tracing. This adaptive behavior prevents the failure of either
estimator from destabilizing the optimization. A remaining failure
mode is early alpha collapse: if
𝛼 (
x
)
reaches 1 too early in a region,
the material branch there receives little or no gradient. We found
this uncommon but possible. Regularizing
𝛼
away from 1 can miti-
gate the issue, but it introduces a sensitive tuning parameter and
slightly reduced quality in our experiments, so we do not use such
a regularizer in the reported results.
Handling unknown emission. A unique challenge in inverse ren-
dering is that emitter locations are often unknown. In our frame-
work, the cache stores full outgoing radiance including emission,
while the material estimator computes only reected radiance. On
emissive surfaces, setting
𝛼 <
1 would blend these inconsistent
quantities.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:7
💡
💡
Fortunately, the optimization naturally handles this: as shown in
Figure 4a and other results,
𝛼
converges to 1 on emissive surfaces,
since this is the only conguration able to correctly model emission.
Note that
𝛼 =
1 is not a denitive indicator of emission—it can also
occur where material optimization struggles.
Variance-aware optimization. An alternative is to optimize
𝛼
for
both bias and variance by not decorrelating paths, which naturally
favors the estimator with lower variance. This follows the principle
of variance-aware inverse rendering [Yan et al
.
2024], which shows
that lower variance can lead to better optimization. While using
correlated paths for material parameters yields biased gradients [Azi-
nović et al
.
2019; Nimier-David 2022], this is acceptable for
𝛼
as
both estimators target the same quantity. We include a comparison
in Section 4 and leave further exploration for future work.
Alternative formulations. For completeness, we note that
𝛼
can
also be optimized via blended local losses computed at each bounce,
without additional overhead. This idea has been used in Zhang et al
.
[2025] for optimizing blended losses over a surface distribution. We
discuss this alternative in Appendix B.
3.4 Connection to Hadadan et al. [2023]
Hadadan et al
.
[2023] can be seen as a special case of our framework
tailored to single-bounce material optimization. Key dierences are
where cache supervision is applied, how gradients are routed, and
whether the estimator is allowed to adapt spatially and in path depth.
Mapping to our formulation. Hadadan et al. combine three ingre-
dients: (i) a primary-hit material loss where materials at the rst
surface interaction are optimized using cached incident radiance
from the next bounce; (ii) a primary-hit cache loss supervised di-
rectly by reference images; and (iii) a cache-consistency objective
as in neural radiance caching [Müller et al
.
2021] that propagates
radiance information from emitters toward the camera.
In our terminology, (i) corresponds to a separated loss with
𝑘 =
2
that updates materials using cached continuation beyond the rst hit,
without blending and with gradients detached from the cache; (ii)
corresponds to a separated loss with
𝑘 =
1 that directly supervises
cache values at camera-ray intersections, and (iii) corresponds to
a cache-consistency term that is eective for forward rendering
but can become unreliable for inverse rendering, especially when
emission is unknown.
Generalization in our method. Our approach removes the single-
bounce restriction by optimizing a family of separated objectives,
and introducing a learned blending eld
𝛼 (
x
)
that adaptively selects
between cache and material estimators across the scene. This makes
the optimization more robust: when estimating incident radiance,
cache values are only used where reliable, providing more accurate
derivatives for materials along the entire light path.
Hadadan* baseline. In our experiments we also report a variant,
Hadadan*, which disables the cache-consistency loss. This isolates
the limitations of consistency terms in the unknown-lighting setting;
we discuss the resulting behavior in Section 4.1.
4 Results
We evaluate our approach in three stages. First, we present results
on a hard setting: room-scale material recovery under unknown
lighting. Second, we analyze eects of key design choices. Third,
we provide experiments under known lighting to show benets of
our method even in applications where lighting is available.
Evaluation methods. The method labeled "Ours" uses separated
losses to train the cache, materials, and the blending eld
𝛼 (
x
)
op-
timized for low bias (i.e., with decorrelated paths). At evaluation
time, we discard both the cache and the blending eld and path-
trace images using only the optimized material information. This
evaluates whether the recovered materials are physically plausible
and reusable, rather than whether a training-time blend can match
the input images. For all tables, we evaluate metrics by rendering
unseen test views under a scene-specic adjusted relighting con-
dition. All experiments use ground-truth scene geometry during
optimization and evaluation. The reported improvements therefore
isolate material-lighting disentanglement and variance reduction,
not robustness to geometric reconstruction errors.
We do not rely on texture-space error as a primary metric because
we optimize all materials jointly, including many regions that are
occluded (e.g., oor under furniture) or never observed, making
texture space evaluation unreliable.
4.1 Unknown lighting material optimization
When both lighting and materials are unknown, dierentiable path
tracing struggles to disentangle emission from reectance. Gradient
variance compounds the diculty: every surface could be an emitter
and must be sampled as such, leading to low-quality gradients that
exacerbate an already degenerate problem.
Quantitative comparison. Table 1 reports reconstruction quality
with test view material-only renderings. Our method substantially
outperforms PRB and improves over Hadadan [2023]. PRB fails in
this setting because it tries to explain most of the appearance via
optimized emission, leading to poor material estimates that are not
useful on their own.
We also report results for Hadadan*, which disables the cache
consistency loss. This variant can improve numerical scores in some
scenes because cache consistency propagates radiance from emitters
toward the camera, leading to incorrect decompositions when the
emission is unknown.
Qualitative comparison. Figure 5 visualizes representative recon-
structions. For our method and Hadadan, we show the directly
visible cache and the material-only rendering illuminated by the
ground-truth lighting. PRB is shown both with its optimized emis-
sion and with ground-truth lighting. Consistent with the quantita-
tive results, our recovered materials remain plausible, while PRB
frequently produces materials that only explain the training images
when paired with its optimized emission.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:8 • Zhang et al.
Metric Method Bedroom Bathroom2 Kitchen Living-rm Living-rm-2 Living-rm-3 Staircase
PSNR ↑
Ours 24.85 23.43 26.96 22.64 27.56 29.57 20.71
PRB 11.46 4.66 10.09 15.46 10.54 6.15 10.65
Hadadan [2023] 16.47 14.82 23.92 18.50 20.33 30.42 12.15
Hadadan [2023]* 18.26 15.24 24.21 18.89 19.91 31.28 17.33
SSIM ↑
Ours 0.5225 0.8032 0.6724 0.8068 0.8331 0.8769 0.8350
PRB 0.4445 0.3357 0.4028 0.7101 0.5724 0.4464 0.3808
Hadadan [2023] 0.4702 0.7182 0.6573 0.7474 0.8073 0.8821 0.6438
Hadadan [2023]* 0.4904 0.7190 0.6588 0.7519 0.8049 0.8805 0.7401
LPIPS ↓
Ours 0.2190 0.1254 0.2154 0.1257 0.1187 0.0721 0.1184
PRB 0.3113 0.4112 0.2053 0.1927 0.2030 0.2638 0.3098
Hadadan [2023] 0.2698 0.1712 0.2284 0.1530 0.1411 0.0689 0.1910
Hadadan [2023]* 0.2543 0.1581 0.2279 0.1521 0.1427 0.0738 0.1771
Table 1. Unknown-lighting reconstruction quality. Known geometry with unknown lighting and unknown materials. We evaluate by rendering test views
using material-only path tracing (discarding cache and
𝛼
) under a new lighting condition. Hadadan* denotes Hadadan et al
.
[2023] with cache consistency
disabled. Best results per scene/metric are highlighted.
Blending reduces rendering variance. Figure 6 visualizes two ren-
derings of the same optimized scene, one made with the pure ma-
terial estimator (no cache, no
𝛼
), and one made using the blended
estimator, which produces a markedly cleaner image. These pri-
mal variance improvements directly translate into lower-variance
gradients that stabilize the optimization and ultimately produce
higher-quality reectance parameters.
Simple scenes. One notable exception is the scene
Living-room-3
in Table 1, where Hadadan* can achieve higher PSNR. This scene
is dominated by direct illumination and is suciently explained by
single-bounce optimization; longer paths provide little benet and
can introduce additional Monte Carlo noise. We revisit this behavior
in Section 4.2.2 as a limitation of multi-bounce training in simple
transport regimes.
4.2 Design choices
We next analyze how key components contribute to performance
in the unknown-lighting setting.
4.2.1 Importance of the blending field
𝛼 (
x
)
. Table 2 compares our
method against two alternatives: a constant blending eld (in partic-
ular,
𝛼 (x) = 0.5
), and one that disables blending altogether (
𝛼 (x) = 0
).
Learning a heterogeneous
𝛼 (
x
)
yields the best performance across
scenes. A xed blend helps but is consistently worse than an adap-
tive eld, while disabling blending degrades performance severely.
Figure 4 illustrates why: the learned
𝛼 (
x
)
encodes a spatial map of
estimator reliability, favoring the cache where material estimation
struggles and falling back to Monte Carlo sampling where the cache
bias is large. This adaptive routing is crucial under unknown lighting,
where the optimizer must balance variance reduction against the
risk of biased cache predictions.
Table 3 further tests whether a hand-crafted material heuristic
can replace learning
𝛼 (
x
)
. We set
𝛼 (
x
) =
0
.
45
𝑟
e
(
x
)
, where
𝑟
e
is a
BSDF-derived eective roughness; the conservative cap prevents
diuse materials from fully suppressing material-path gradients.
This baseline is viable but consistently worse than learning
𝛼 (
x
)
.
The result indicates that estimator reliability is not determined by
local roughness alone: it also depends on cache accuracy, visibility,
indirect transport, lighting ambiguity, and the current optimization
state.
4.2.2 Maximum path depth. Table 4 studies the inuence of the
maximum path depth parameter
𝑘
max
during training. Depth 2 is of-
ten insucient because it forces cache usage at the second bounce re-
gardless of
𝛼 (
x
)
, which limits the optimizer’s ability to select estima-
tors based on reliability. Increasing to
𝑘
max
=
3 enables
𝛼 (
x
)
to mean-
ingfully inuence termination and generally improves reconstruc-
tion quality. Beyond depth 3, gains often plateau, suggesting most
benchmark scenes are suciently captured by moderate depths.
Consistent with Section 4.1,
Living-room-3
favors shallow depth
because transport is dominated by direct illumination.
4.2.3 Cache capacity. Figure 7 evaluates robustness to cache capac-
ity over a wide range of hash map sizes. Performance remains stable
across capacities, indicating that our approach is not overly sensitive
to the cache representation. Intuitively, when cache approximation
quality degrades due to limited capacity, the learned
𝛼 (
x
)
can reduce
reliance on the cache and fall back to the material estimator.
4.3 Known lighting material optimization
Convergence in a high-variance scene. Figure 8 evaluates opti-
mization in the Veach Ajar scene under known lighting. To isolate
convergence eects in a noisy global-illumination setting, we re-
port RGB-space MSE of the painting texture (rather than full-scene
texture error). Both our method and Hadadan run at 1 spp during
optimization. Our blended estimator converges faster than pure path
tracing at low spp by reducing variance and stabilizing gradients.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:9
Ours
PRB Hadadan et al. [2023]
Ground truth
Error
Rendering
Initialization
Cache rendering
(camera ray hit point)
Material rendering
(ground truth light)
Material rendering
(ground truth light)
Material rendering
(optimized light)
Cache rendering
(camera ray hit point)
Material rendering
(ground truth light)
MAE/RMSE = 0.015/0.024 0.033/0.049 0.053/0.060 0.185/0.217 0.012/0.021 0.135/0.149
0.041/0.146 0.024/0.035 0.046/0.096 0.259/0.340 0.039/0.147 0.131/0.153
0.123/0.156 0.093/0.130 0.222/0.263 0.178/0.248 0.124/0.156 0.415/0.523
0.060/0.074 0.039/0.053 0.058/0.070 0.092/0.125 0.058/0.071 0.184/0.208
Fig. 5. Material reconstruction from unknown lighting. Visual comparison corresponding to Table 1. For ours and Hadadan et al
.
[2023], we show the
cache rendering at camera-ray intersections and a material-only rendering illuminated by the ground-truth lighting (unknown during optimization) to verify
decomposition. PRB optimizes both materials and emission; we show its optimized-light rendering (used during training) and its material-only rendering
under ground-truth lighting.
Rendering using our cache-material blending Rendering using material alone
1 spp 4 spp 16 spp
1 spp 4 spp 16 spp
Fig. 6. Blending reduces rendering variance. We render the same optimized scene at low sample count via (le) our blended estimator and (right) the material
estimator (without cache/blending field). Blending substantially reduces variance at equal sample count, which translates into cleaner
optimization gradients.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:10 • Zhang et al.
Scene
PSNR ↑ SSIM ↑ LPIPS ↓
Ours 𝛼 = 0.5 𝛼 = 0 Ours 𝛼 = 0.5 𝛼 = 0 Ours 𝛼 = 0.5 𝛼 = 0
Bedroom 24.85 23.12 12.07 0.5225 0.5158 0.3955 0.2190 0.2183 0.3653
Bathroom2 23.43 16.54 11.73 0.8032 0.8132 0.7609 0.1254 0.1518 0.2766
Kitchen 26.96 23.58 10.62 0.6724 0.6355 0.4519 0.2154 0.2473 0.4127
Living-room 22.64 18.54 9.76 0.8068 0.7399 0.5095 0.1257 0.1563 0.3283
Living-room-2 27.56 18.75 8.47 0.8331 0.7808 0.6138 0.1187 0.1554 0.3368
Living-room-3 29.57 28.74 14.61 0.8769 0.8750 0.7771 0.0721 0.0728 0.1939
Staircase 20.71 16.52 6.98 0.8350 0.7569 0.4229 0.1184 0.1291 0.4826
Table 2. Eect of optimizing the blending field. Ablation under unknown lighting using the same evaluation as Table 1 (material-only relighting). We
compare learning 𝛼 (x) (ours) to fixing 𝛼 (x) = 0.5 and disabling blending altogether (𝛼 (x) = 0). Learning 𝛼 (x) provides consistent gains.
Scene
PSNR ↑
Learned 𝛼 Roughness 𝛼
Bathroom2 22.99 15.96
Bedroom 24.82 20.71
Kitchen 26.98 19.99
Living-room 24.34 16.37
Living-room-2 27.60 18.24
Living-room-3 29.60 26.77
Staircase 20.72 10.77
Average 25.29 18.40
Table 3. Learned versus roughness-derived blending. We compare our
learned blending field to a fixed local heuristic
𝛼 (
x
) =
0
.
45
𝑟
e
(
x
)
, where
𝑟
e
is an eective material roughness in
[
0
,
1
]
. The roughness heuristic pre-
serves a material-gradient path but cannot model cache reliability, visibility,
illumination, or optimization state; learning
𝛼 (
x
)
is therefore consistently
more eective.
Cache size (hashmap entries)
Normalized metrics
Fig. 7. Sensitivity to cache capacity. Scene-normalized mean performance
across all scenes for PSNR/SSIM/LPIPS (best per scene = 100%) as a function
of cache size (hash map entries, 2
18
–2
23
). Shaded bands indicate bootstrap
confidence intervals. Our method is stable across a wide range of capacities,
indicating robustness to cache representation size.
Limitation of material consistency loss. Figure 9 demonstrates a
limitation of using the cache as a direct training target for materi-
als via the material consistency loss (Equation 5). We optimize the
Scene Metric
Max light path depth
2 3 4 5 6
Bedroom
PSNR ↑ 20.48 23.67 24.71 24.85 24.85
SSIM ↑ 0.5036 0.5169 0.5213 0.5222 0.5225
LPIPS ↓ 0.2403 0.2233 0.2195 0.2191 0.2190
Bathroom2
PSNR ↑ 14.03 16.82 24.57 23.81 23.45
SSIM ↑ 0.8017 0.8091 0.8089 0.8050 0.8033
LPIPS ↓ 0.2043 0.1666 0.1188 0.1232 0.1253
Kitchen
PSNR ↑ 25.45 27.25 27.07 26.97 26.97
SSIM ↑ 0.6506 0.6689 0.6714 0.6719 0.6724
LPIPS ↓ 0.2362 0.2184 0.2164 0.2160 0.2154
Living-rm
PSNR ↑ 22.46 22.60 22.59 22.59 22.64
SSIM ↑ 0.8025 0.8061 0.8060 0.8058 0.8067
LPIPS ↓ 0.1270 0.1271 0.1262 0.1261 0.1257
Living-rm-2
PSNR ↑ 21.44 27.38 27.44 27.51 27.56
SSIM ↑ 0.8124 0.8330 0.8330 0.8331 0.8331
LPIPS ↓ 0.1386 0.1190 0.1187 0.1187 0.1187
Living-rm-3
PSNR ↑ 35.06 33.86 27.44 29.75 29.67
SSIM ↑ 0.8836 0.8813 0.8840 0.8774 0.8771
LPIPS ↓ 0.0672 0.0685 0.0665 0.0718 0.0719
Staircase
PSNR ↑ 15.28 18.98 20.25 20.59 20.67
SSIM ↑ 0.7360 0.8012 0.8300 0.8338 0.8346
LPIPS ↓ 0.1277 0.1333 0.1223 0.1194 0.1188
Table 4. Eect of maximum light-path depth. Unknown-lighting training
with our full method; evaluation follows Table 1. Increasing depth from 2
to 3 typically improves results by enabling
𝛼 (
x
)
-guided termination and
deeper transport, while gains beyond moderate depth oen plateau.
albedo of a painting under known lighting while the cache repre-
sents outgoing radiance. For diuse materials, limited cache capacity
can blur ne details (the cache is trained on the scene scale); for
glossy materials, directional variation is dicult for the cache to
capture, and the resulting bias propagates to the material. This is
why the glossy crops contain structured artifacts even though the
underlying painting texture should remain spatially coherent: the
material-consistency target contains cache approximation errors
caused by view-dependent radiance. This experiment motivates why
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:11
Time (s)
MSE
Optimization results
Ours
Hadadan et al. [2023]
PRB (1 spp)
PRB (4 spp) PRB (8 spp)
PRB (16 spp)
Initialization
Ground truth
Scene setup
Fig. 8. Texture optimization in a noisy scene with known lighting. We track convergence of the painting on the wall while jointly optimizing all scene
materials. Curves show MSE vs wall-clock time for ours, Hadadan et al
.
[2023] (1 spp), and PRB at 1/4/8/16 spp. All runs use 1000 iterations and a relative
𝐿
2
loss to avoid overfiing to noise.
0.035/0.003
MAE/MSE = 0.085/0.015
0.122/0.026
0.036/0.0030.087/0.015 0.125/0.029
Ground truthSeparated lossesMaterial consistency losses
Rendering with diuse layer + clearcoat specular layer painting material
Optimized with simpler
diuse-only rendering
Optimized with diuse + clearcoat rendering
(a)
(b)
(c)
Fig. 9. Limitation of material consistency loss. We optimize the albedo of a painting under known lighting using the material consistency loss (Equation 5),
which uses the cache as the training target. (a) Diuse surface: limited cache resolution blurs fine spatial details, degrading the recovered texture. (b-c) Glossy
surface: the cache struggles with high-frequency view-dependent variation, and bias propagates to the material, causing more artifacts. The artifacts are most
visible in the glossy variants, where the optimized albedo inherits view-dependent cache errors that should instead be explained by directional scaering. We
visualize the optimized albedo to isolate this failure from lighting changes. Bridge over a Pond of Water Lilies, Claude Monet, 1899, public domain.
our main approach relies on separated losses to constrain estimators
without requiring the cache to serve as a universally accurate direct
supervision signal for outgoing radiance.
4.4 Variance-aware optimization of 𝛼 (x)
The results in this article optimize the blending eld
𝛼 (
x
)
to min-
imize bias (i.e., using the decorrelated gradient estimator). Fig-
ure 10 explores an alternative variance-aware objective that ad-
ditionally penalizes variance. On a rough dielectric surface with
spatially varying roughness, variance-aware optimization drives
𝛼 (
x
)
higher overall (favoring the cache more), while still prefer-
ring the material estimator in glossy regions where cache bias is
pronounced. This suggests variance-aware objectives may improve
robustness in extremely noisy settings; we leave a broader evalua-
tion for future work.
4.5 Optimization progress
Figure 11 visualizes how materials, the cache, and the blending
eld evolve during unknown-lighting optimization. We show (top)
a material-only render relit with ground-truth lighting (unknown
during optimization), (middle) cache values at camera-ray intersec-
tions, and (bottom) the learned
𝛼 (
x
)
. As optimization progresses,
both the material-only renders and cache predictions become more
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:12 • Zhang et al.
(b) Optimized blending field
0.0
1.0
(a) Ground truth scene
Variance-aware optimization Decorrelated (default)
mean 0.79
0.71
0.39
0.07
0.06
0.05
Fig. 10. Eect of variance-aware optimization of
𝛼 (
x
)
. (a) We optimize
the blending field
𝛼
on a rough dielectric with spatially varying rough-
ness (le: diuse, right: glossy). (b) Comparison of
𝛼 (
x
)
learned by default
bias-driven (decorrelated) optimization versus variance-aware optimization.
Variance-aware optimization favors the cache more overall while still reduc-
ing cache reliance in glossy regions where cache bias is pronounced.
consistent with the reference, while
𝛼 (
x
)
adapts over time to route
gradients to the more reliable estimator.
4.6 Performance analysis
Our method incurs only a negligible computational overhead com-
pared to PRB. Across all scenes, our method requires an average of
0
.
094 seconds per iteration, compared to 0
.
091 for PRB and 0
.
088 for
Hadadan et al
.
[2023]. Cache evaluation and blending-eld queries
are ecient, and the small overhead is justied by the substantial
improvement in reconstruction quality under unknown lighting.
Detailed per-scene timing is provided in Table 5.
5 Conclusion
Inverse rendering involves a choice between two unsatisfying ex-
tremes. On one side lie radiance eld representations that soak up
data like a sponge, but sidestep the question of what in the world is
actually emitting or reecting. On the other side lie physically based
models that promise semantic, editable parameters, but punish us
with high-variance simulation and an unforgiving loss landscape
that can be impossible to optimize.
By creating a slider that interpolates between these two views,
we have surprisingly not compromised on physics but instead ended
up with something richer: a place to absorb what the model cannot
yet physically explain, which makes the reconstruction task better-
posed and more ecient. We demonstrated this on material recovery
with unknown lighting, a hard problem where inverse path tracing
often fails outright. Our observations suggest that radiance caches
may have a role to play in future inverse rendering systems.
The assumption of known geometry is somewhat articial, as
we will often want to recover it jointly with materials and lighting.
It will be interesting to study how to best introduce a cache and
blending eld if geometry itself can nucleate, evolve, or disappear,
whether represented by triangle meshes, messy triangle-soup ge-
ometry, or heterogeneous volumes. Subsurface scattering is another
interesting application, since it is high-variance and requires long
light paths that could benet from early termination.
Our experimental setup is also synthetic in another sense: all tasks
were designed so that the model can perfectly reproduce the data,
which is never realistic in practice. When attempting to reconstruct
scenes from photos, real-world appearance will inevitably lie outside
the modeling capability of any BSDF, and so there is an unavoidable
source of discrepancy. Investigating whether our framework or
a similar decomposition could help in this case is an interesting
direction for future work.
Acknowledgments
This project has received funding from the European Research Coun-
cil (ERC) under the European Union’s Horizon 2020 research and
innovation program (grant agreement No 948846).
References
Advanced Micro Devices AMD. 2025. AMD FSR Radiance Caching. https://gpuopen.
com/amd-fsr-radiancecaching/. [Online; accessed 14-January-2026].
Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan T Barron,
Matthew O’Toole, and Pratul P Srinivasan. 2024. Flash Cache: Reducing Bias in
Radiance Cache Based Inverse Rendering. In European Conference on Computer
Vision (ECCV). Springer, 20–36.
Dejan Azinović, Tzu-Mao Li, Anton Kaplanyan, and Matthias Nießner. 2019. Inverse
Path Tracing for Joint Material and Lighting Estimation. In IEEE Conference on
Computer Vision and Pattern Recognition (CVPR). doi:10.1109/CVPR.2019.00255
Sai Praveen Bangaru, Tzu-Mao Li, and Frédo Durand. 2020. Unbiased warped-area sam-
pling for dierentiable rendering. ACM Trans. Graph. 39, 6, Article 245 (November
2020), 18 pages. doi:10.1145/3414685.3417833
Yash Belhe, Bing Xu, Sai Praveen Bangaru, Ravi Ramamoorthi, and Tzu-Mao Li. 2024.
Importance Sampling BRDF Derivatives. ACM Trans. Graph. 43, 3, Article 25 (April
2024), 21 pages. doi:10.1145/3648611
Nikolaus Binder, Sascha Fricke, and Alexander Keller. 2018. Fast path space ltering by
jittered spatial hashing. In ACM SIGGRAPH 2018 Talks (Vancouver, British Columbia,
Canada) (SIGGRAPH ’18). Association for Computing Machinery, New York, NY,
USA, Article 71, 2 pages. doi:10.1145/3214745.3214806
Wesley Chang, Venkataram Sivaram, Derek Nowrouzezahrai, Toshiya Hachisuka, Ravi
Ramamoorthi, and Tzu-Mao Li. 2023. Parameter-space ReSTIR for Dierentiable and
Inverse Rendering. In ACM SIGGRAPH 2023 Conference Proceedings (Los Angeles,
CA, USA) (SIGGRAPH ’23). Association for Computing Machinery, New York, NY,
USA, Article 18, 10 pages. doi:10.1145/3588432.3591512
Wesley Chang, Xuanda Yang, Yash Belhe, Ravi Ramamoorthi, and Tzu-Mao Li. 2024.
Spatiotemporal Bilateral Gradient Filtering for Inverse Rendering. In SIGGRAPH
Asia 2024 Conference Papers (Tokyo, Japan) (SA ’24). Association for Computing
Machinery, New York, NY, USA, Article 70, 11 pages. doi:10.1145/3680528.3687606
Mikhail Dereviannykh, Dmitrii Klepikov, Johannes Hanika, and Carsten Dachsbacher.
2025. Neural Two-Level Monte Carlo Real-Time Rendering. In Computer Graphics
Forum. Wiley Online Library, e70050.
Zhimin Fan, Pengcheng Shi, Mufan Guo, Ruoyu Fu, Yanwen Guo, and Jie Guo. 2024.
Conditional Mixture Path Guiding for Dierentiable Rendering. ACM Trans. Graph.
43, 4, Article 48 (July 2024), 11 pages. doi:10.1145/3658133
Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An evaluation of computational
imaging techniques for heterogeneous inverse scattering. In European Conference on
Computer Vision (ECCV). Springer International Publishing, 685–701. doi:10.1007/
978-3-319-46487-9_42
G. Greger, P. Shirley, P.M. Hubbard, and D.P. Greenberg. 1998. The irradiance volume.
IEEE Computer Graphics and Applications 18, 2 (1998), 32–43. doi:10.1109/38.656788
Chun Gu, Xiaofei Wei, Zixuan Zeng, Yuxuan Yao, and Li Zhang. 2025. IRGS: Inter-
Reective Gaussian Splatting with 2D Gaussian Ray Tracing. In IEEE Conference on
Computer Vision and Pattern Recognition (CVPR).
Saeed Hadadan, Geng Lin, Jan Novák, Fabrice Rousselle, and Matthias Zwicker. 2023.
Inverse Global Illumination using a Neural Radiometric Prior. In ACM SIGGRAPH
2023 Conference Proceedings (Los Angeles, CA, USA) (SIGGRAPH ’23). Association
for Computing Machinery, New York, NY, USA, Article 17, 11 pages. doi:10.1145/
3588432.3591553
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:13
Optimization states
Iter 20 Iter 60 Iter 160 Iter 400 Iter 1000
Initialization
Ground truth
0.0
1.0
Material
Cache
Blending field
Initialization
Ground truth
0.0
1.0
Material
Cache
Blending field
Fig. 11. Evolution of materials, cache, and blending field during unknown-lighting optimization. We visualize selected iterations for each scene: (top)
material-only rendering relit with ground-truth lighting (unknown during optimization), (middle) cache queried at camera-ray intersections, and (boom)
learned blending field 𝛼 (x). The blending field adapts over time, increasingly routing gradients to the more reliable estimator as optimization progresses.
Saeed Hadadan and Matthias Zwicker. 2024. Neural Dierential Radiance Field: Learn-
ing the Dierential Space Using a Neural Network. In Advances in Computer Graphics,
Bin Sheng, Lei Bi, Jinman Kim, Nadia Magnenat-Thalmann, and Daniel Thalmann
(Eds.). Springer Nature Switzerland, Cham, 93–104.
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2024. 2D
Gaussian Splatting for Geometrically Accurate Radiance Fields. In ACM SIGGRAPH
2024 Conference Papers (Denver, CO, USA) (SIGGRAPH ’24). Association for Com-
puting Machinery, New York, NY, USA, Article 32, 11 pages. doi:10.1145/3641519.
3657428
Wojciech Jarosz, Craig Donner, Matthias Zwicker, and Henrik Wann Jensen. 2008.
Radiance caching for participating media. ACM Trans. Graph. 27, 1, Article 7 (March
2008), 11 pages. doi:10.1145/1330511.1330518
Kaiwen Jiang, Jia-Mu Sun, Zilu Li, Dan Wang, Tzu-Mao Li, and Ravi Ramamoorthi.
2025. Dierentiable Light Transport with Gaussian Surfels via Adapted Radiosity
for Ecient Relighting and Geometry Reconstruction. ACM Trans. Graph. 44, 6,
Article 210 (December 2025), 25 pages. doi:10.1145/3763305
Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei
Zhou, Zexiang Xu, and Hao Su. 2023. TensoIR: Tensorial Inverse Rendering. In IEEE
Conference on Computer Vision and Pattern Recognition (CVPR).
Lukas Kandlbinder, Addis Dittebrandt, Alexander Schipek, and Carsten Dachsbacher.
2024. Optimizing Path Termination for Radiance Caching Through Explicit Variance
Trading. Proc. ACM Comput. Graph. Interact. Tech. 7, 3, Article 33 (Aug. 2024),
19 pages. doi:10.1145/3675381
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkuehler, and George Drettakis. 2023.
3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph.
42, 4, Article 139 (July 2023), 14 pages. doi:10.1145/3592433
Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner.
2015. Matching Real Fabrics with Micro-Appearance Models. ACM Trans. Graph.
35, 1, Article 1 (December 2015), 26 pages. doi:10.1145/2818648
J. Krivanek, P. Gautron, S. Pattanaik, and K. Bouatouch. 2005. Radiance caching for
ecient global illumination computation. IEEE Transactions on Visualization and
Computer Graphics 11, 5 (2005), 550–561. doi:10.1109/TVCG.2005.83
Jaroslav Křivánek, Pascal Gautron, Greg Ward, Okan Arikan, and Henrik Wann Jensen.
2007. Practical global illumination with irradiance caching. In ACM SIGGRAPH
2007 Courses (San Diego, California) (SIGGRAPH ’07). Association for Computing
Machinery, New York, NY, USA, 1–es. doi:10.1145/1281500.1281617
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika
Aittala, and Timo Aila. 2018. Noise2Noise: Learning Image Restoration without
Clean Data. In Proceedings of the 35th International Conference on Machine Learning
(Proceedings of Machine Learning Research, Vol. 80), Jennifer Dy and Andreas Krause
(Eds.). PMLR, 2965–2974. https://proceedings.mlr.press/v80/lehtinen18a.html
Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Dierentiable
Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6, Article
222 (December 2018), 11 pages. doi:10.1145/3272127.3275109
Jingwang Ling, Ruihan Yu, Feng Xu, Chun Du, and Shuang Zhao. 2024. NeRF as a
Non-Distant Environment Emitter in Physics-based Inverse Rendering. In ACM
SIGGRAPH 2024 Conference Papers (Denver, CO, USA) (SIGGRAPH ’24). Association
for Computing Machinery, New York, NY, USA, Article 39, 12 pages. doi:10.1145/
3641519.3657404
Guillaume Loubet, Nicolas Holzschuch, and Wenzel Jakob. 2019. Reparameterizing
discontinuous integrands for dierentiable rendering. ACM Trans. Graph. 38, 6,
Article 228 (November 2019), 14 pages. doi:10.1145/3355089.3356510
Haolin Lu, Delio Vicini, Wesley Chang, and Tzu-Mao Li. 2025. Vector-Valued Monte
Carlo Integration Using Ratio Control Variates. ACM Trans. Graph. 44, 4, Article
100 (July 2025), 16 pages. doi:10.1145/3731175
Julio Marco, Adrian Jarabo, Wojciech Jarosz, and Diego Gutierrez. 2018. Second-Order
Occlusion-Aware Volumetric Radiance Caching. ACM Trans. Graph. 37, 2, Article
20 (July 2018), 14 pages. doi:10.1145/3185225
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:14 • Zhang et al.
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ra-
mamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields
for View Synthesis. In European Conference on Computer Vision (ECCV).
Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-time
neural radiance caching for path tracing. ACM Trans. Graph. 40, 4, Article 36 (July
2021), 16 pages. doi:10.1145/3450626.3459812
Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large steps in inverse rendering
of geometry. ACM Trans. Graph. 40, 6, Article 248 (December 2021), 13 pages.
doi:10.1145/3478513.3480501
Baptiste Nicolet, Fabrice Rousselle, Jan Novak, Alexander Keller, Wenzel Jakob, and
Thomas Müller. 2023. Recursive Control Variates for Inverse Rendering. ACM Trans.
Graph. 42, 4, Article 62 (July 2023), 13 pages. doi:10.1145/3592139
Merlin Nimier-David. 2022. Dierentiable Physically Based Rendering: Algorithms, Sys-
tems and Applications. PhD Dissertation. École Polytechnique Fédérale de Lausanne
(EPFL), Lausanne, Switzerland. https://merl.in/phd-thesis.pdf Thesis No. 9123.
Merlin Nimier-David, Thomas Müller, Alexander Keller, and Wenzel Jakob. 2022. Unbi-
ased inverse volume rendering with dierential trackers. ACM Trans. Graph. 41, 4,
Article 44 (July 2022), 20 pages. doi:10.1145/3528223.3530073
Merlin Nimier-David, Sébastien Speierer, Benoît Ruiz, and Wenzel Jakob. 2020. Radiative
backpropagation: an adjoint method for lightning-fast dierentiable rendering. ACM
Trans. Graph. 39, 4, Article 146 (August 2020), 15 pages. doi:10.1145/3386569.3392406
Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2:
a retargetable forward and inverse renderer. ACM Trans. Graph. 38, 6, Article 203
(November 2019), 17 pages. doi:10.1145/3355089.3356498
NVIDIA. 2024. NVIDIA RTXGI. https://github.com/NVIDIAGameWorks/RTXGI [On-
line; accessed 14-January-2026].
Yohan Poirier-Ginter, Jerey Hu, Jean-Francois Lalonde, and George Drettakis. 2025.
Editable Physically-based Reections in Raytraced Gaussian Radiance Fields. In
Proceedings of the SIGGRAPH Asia 2025 Conference Papers (SA Conference Papers ’25).
Association for Computing Machinery, New York, NY, USA, Article 201, 12 pages.
doi:10.1145/3757377.3763971
Ari Silvennoinen and Jaakko Lehtinen. 2017. Real-time global illumination by precom-
puted local reconstruction from sparse radiance probes. ACM Trans. Graph. 36, 6,
Article 230 (November 2017), 13 pages. doi:10.1145/3130800.3130852
Jiakai Sun, Weijing Zhang, Zhanjie Zhang, Tianyi Chu, Guangyuan Li, Lei Zhao, and
Wei Xing. 2025. Joint Optimization of Triangle Mesh, Material, and Light from
Neural Fields with Neural Radiance Cache. arXiv:2305.16800 [cs.GR]
Wolfgang Tatzgern, Alexander Weinrauch, Pascal Stadlbauer, Joerg H. Mueller, Martin
Winter, and Markus Steinberger. 2024. Radiance Caching with On-Surface Caches
for Real-Time Global Illumination. Proc. ACM Comput. Graph. Interact. Tech. 7, 3,
Article 38 (Aug. 2024), 17 pages. doi:10.1145/3675382
Zdravko Velinov, Marios Papas, Derek Bradley, Paulo Gotardo, Parsa Mirdehghan, Steve
Marschner, Jan Novák, and Thabo Beeler. 2018. Appearance capture and modeling
of human teeth. ACM Trans. Graph. 37, 6, Article 207 (December 2018), 13 pages.
doi:10.1145/3272127.3275098
Delio Vicini, Sébastien Speierer, and Wenzel Jakob. 2021. Path replay backpropagation:
dierentiating light paths using constant memory and linear time. ACM Trans.
Graph. 40, 4, Article 108 (July 2021), 14 pages. doi:10.1145/3450626.3459804
Yu-Chen Wang, Chris Wyman, Lifan Wu, and Shuang Zhao. 2023. Amortizing Samples
in Physics-Based Inverse Rendering Using ReSTIR. ACM Trans. Graph. 42, 6, Article
214 (December 2023), 17 pages. doi:10.1145/3618331
Gregory J. Ward, Francis M. Rubinstein, and Robert D. Clear. 1988. A ray tracing
solution for diuse interreection. SIGGRAPH Comput. Graph. 22, 4 (June 1988),
85–92. doi:10.1145/378456.378490
Philippe Weier, Marc Droske, Johannes Hanika, Andrea Weidlich, and Jiří Vorba. 2021.
Optimised Path Space Regularisation. Computer Graphics Forum (Proc. EGSR) 40, 4
(2021). doi:10.1111/cgf.14347
Philippe Weier, Jérémy Riviere, Ruslan Guseinov, Stephan Garbin, Philipp Slusallek,
Bernd Bickel, Thabo Beeler, and Delio Vicini. 2025. Practical Inverse Rendering of
Textured and Translucent Appearance. ACM Trans. Graph. 44, 4, Article 117 (July
2025), 16 pages. doi:10.1145/3730855
Haoqian Wu, Zhipeng Hu, Lincheng Li, Yongqiang Zhang, Changjie Fan, and Xin Yu.
2023. NeFII: Inverse Rendering for Reectance Decomposition with Near-Field
Indirect Illumination. In IEEE Conference on Computer Vision and Pattern Recognition
(CVPR). doi:10.1109/CVPR52729.2023.00418
Sean Wu, Shamik Basu, Tim Broedermann, Luc Van Gool, and Christos Sakaridis. 2025.
PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields. In IEEE Conference
on Computer Vision and Pattern Recognition (CVPR).
Kai Yan, Vincent Pegoraro, Marc Droske, Jiří Vorba, and Shuang Zhao. 2024. Dier-
entiating Variance for Variance-Aware Inverse Rendering. In SIGGRAPH Asia 2024
Conference Papers (Tokyo, Japan) (SA ’24). Association for Computing Machinery,
New York, NY, USA, Article 117, 10 pages. doi:10.1145/3680528.3687603
Yao Yao, Jingyang Zhang, Jingbo Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai
Tsin, and Long Quan. 2022. NeILF: Neural Incident Light Field for Physically-based
Material Estimation. In European Conference on Computer Vision (ECCV).
Tizian Zeltner, Sébastien Speierer, Iliyan Georgiev, and Wenzel Jakob. 2021. Monte
Carlo estimators for dierential light transport. ACM Trans. Graph. 40, 4, Article 78
(July 2021), 16 pages. doi:10.1145/3450626.3459807
Cheng Zhang, Lifan Wu, Changxi Zheng, Ioannis Gkioulekas, Ravi Ramamoorthi, and
Shuang Zhao. 2019. A dierential theory of radiative transfer. ACM Trans. Graph.
38, 6, Article 227 (November 2019), 16 pages. doi:10.1145/3355089.3356522
Jingyang Zhang, Yao Yao, Shiwei Li, Jingbo Liu, Tian Fang, David McKinnon, Yanghai
Tsin, and Long Quan. 2023b. NeILF++: Inter-reectable Light Fields for Geometry
and Material Estimation. In IEEE International Conference on Computer Vision (ICCV).
Xiuming Zhang, Pratul P. Srinivasan, Boyang Deng, Paul Debevec, William T. Free-
man, and Jonathan T. Barron. 2021. NeRFactor: neural factorization of shape and
reectance under an unknown illumination. ACM Trans. Graph. 40, 6, Article 237
(December 2021), 18 pages. doi:10.1145/3478513.3480496
Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, and Xiaowei Zhou.
2022. Modeling Indirect Illumination for Inverse Rendering. In IEEE Conference on
Computer Vision and Pattern Recognition (CVPR). doi:10.1109/CVPR52688.2022.01809
Ziyi Zhang, Nicolas Roussel, and Wenzel Jakob. 2023a. Projective Sampling for Dier-
entiable Rendering of Geometry. ACM Trans. Graph. 42, 6, Article 212 (Dec. 2023),
14 pages. doi:10.1145/3618385
Ziyi Zhang, Nicolas Roussel, Thomas Muller, Tizian Zeltner, Merlin Nimier-David,
Fabrice Rousselle, and Wenzel Jakob. 2025. Radiance Surfaces: Optimizing Surface
Representations with a 5D Radiance Field Loss. In Proceedings of the Special Interest
Group on Computer Graphics and Interactive Techniques Conference Conference Papers
(SIGGRAPH Conference Papers ’25). Association for Computing Machinery, New
York, NY, USA, Article 21, 10 pages. doi:10.1145/3721238.3730713
Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling
scattering parameters for rendering anisotropic media. ACM Trans. Graph. 35, 6,
Article 166 (December 2016), 11 pages. doi:10.1145/2980179.2980228
Appendix
A Decorrelated gradient estimator
In Section 3.1.2, we claimed that naively optimizing the material
consistency loss leads to biased gradients. The issue has the same
root as in dierentiating Monte Carlo renderers with noisy ren-
derings, and we adopt a similar decorrelation strategy as used by
Azinović et al. [2019].
Consider the material consistency loss at a single bounce. For
brevity, let
𝐶 = 𝐶(
x
, 𝜔)
be the target outgoing radiance and
𝐶
′
(𝜔
′
) =
𝐶 (x
′
, −𝜔
′
) be the incident radiance from the cache. The loss is:
L =
𝐶 − E
𝜔
′
𝐶
′
(𝜔
′
) · 𝑓 (𝜔
′
) ·
cos𝜃
′
𝑝 (𝜔
′
)
2
, (7)
where
𝐶
and
𝐶
′
are treated as known (from the detached cache),
and we optimize the BSDF
𝑓 (𝜔
′
) ≡ 𝑓 (
x
, 𝜔
′
, 𝜔)
. Let
𝑅 = 𝐶 − E[𝐶
′
·
𝑓 · cos𝜃
′
/𝑝] denote the residual. The true gradient is
∇
𝑓
L = −2 𝑅 · E
∇
𝑓
𝐶
′
· 𝑓 ·
cos𝜃
′
𝑝
. (8)
Naive estimator (biased). If we use the same sample
𝜔
′
1
for both
the residual and the gradient, we obtain
∇
𝑓
L = −2
𝐶 − 𝐶
′
(𝜔
′
1
) · 𝑓 (𝜔
′
1
) ·
cos𝜃
′
1
𝑝
1
· ∇
𝑓
𝐶
′
(𝜔
′
1
) · 𝑓 (𝜔
′
1
) ·
cos𝜃
′
1
𝑝
1
. (9)
The two factors are correlated through the shared sample
𝜔
′
1
, so
E[𝐴 · 𝐵] ≠ E[𝐴] · E[𝐵]
. This estimator is biased: in the extreme one-
sample case, we are asking each sampled direction to individually
match
𝐶
, when in fact only the integral over all directions should
match 𝐶.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:15
Decorrelated estimator (unbiased). To obtain an unbiased gradient,
we sample two independent directions:
𝜔
′
1
for the residual and
𝜔
′
2
for the gradient:
∇
𝑓
L = −2
𝐶 − 𝐶
′
(𝜔
′
1
) · 𝑓 (𝜔
′
1
) ·
cos𝜃
′
1
𝑝
1
· ∇
𝑓
𝐶
′
(𝜔
′
2
) · 𝑓 (𝜔
′
2
) ·
cos𝜃
′
2
𝑝
2
. (10)
Since
𝜔
′
1
and
𝜔
′
2
are independent, the expectation factorizes cor-
rectly:
E
h
∇
𝑓
L
i
= −2 E[𝑅] · E
∇
𝑓
𝐶
′
· 𝑓 ·
cos𝜃
′
𝑝
= ∇
𝑓
L. (11)
B Alpha optimization via blended termination losses
In Section 3.3, we described optimizing
𝛼
by dierentiating the
blended image loss. Here we discuss an alternative approach that op-
timizes
𝛼
more directly, without additional overhead. The idea, used
by Zhang et al
.
[2025], is to dene two losses at each bounce—one
assuming we terminate with the cache, one assuming we continue
with the material—and let 𝛼 blend between them.
At each bounce
𝑖
along a light path, we compare the pixel color
against the radiance estimate obtained by each strategy:
L
cache,𝑖
=
(
𝐼 − 𝑇
𝑖
· 𝐶
𝑖
)
2
, (12)
L
mat,𝑖
=
𝐼 − 𝑇
𝑖
·
𝑓
𝑖
cos𝜃
𝑖
𝑝
𝑖
𝐿
(𝑖+1)
o
2
, (13)
where
𝑇
𝑖
is the path throughput from the camera to bounce
𝑖
, and
𝐼
is the reference pixel color. We then optimize
𝛼
𝑖
to minimize the
blended loss:
L
blend,𝑖
= (1 − 𝛼
𝑖
) L
mat,𝑖
+ 𝛼
𝑖
L
cache,𝑖
. (14)
Intuitively,
𝛼
𝑖
learns to favor whichever estimator yields a smaller
loss at that point.
Optimizing alpha. This formulation gives unbiased gradients for
𝛼
. To see why, observe that the gradient is simply a dierence of
two loss values:
∇
𝛼
𝑖
L
blend,𝑖
= L
cache,𝑖
− L
mat,𝑖
. (15)
Since
𝛼
𝑖
sits outside the nonlinear (squared) part of the loss, there
is no product of correlated terms. Each loss value is an unbiased
estimate of its expectation, so:
E
∇
𝛼
𝑖
L
blend,𝑖
= E
L
cache,𝑖
− E
L
mat,𝑖
. (16)
The gradient pushes
𝛼
𝑖
toward the estimator with lower expected
loss.
Limitations for cache and material. One might hope to use these
losses to optimize the cache or material parameters as well. However,
this leads to biased gradients—even for the cache, which itself has
no estimation variance.
The issue is the path throughput
𝑇
𝑖
. Consider the gradient for the
cache:
∇
𝐶
𝑖
L
cache,𝑖
= −2𝑇
𝑖
·
(
𝐼 − 𝑇
𝑖
· 𝐶
𝑖
)
. (17)
The throughput
𝑇
𝑖
appears twice: once as a direct multiplier and
once inside the residual. Both instances are the same one-sample es-
timate, so they are correlated, and the expectation does not factorize
correctly. This is the same bias issue discussed in Appendix A.
To obtain unbiased gradients, we would need two independent
paths connecting the same pixel to the same point x
𝑖
—one for the
residual and one for the gradient multiplier. This is not ecient
in practice, so we conclude that these blended termination losses
should only be used to optimize
𝛼
, not the cache or material param-
eters.
Bias-only vs. variance-aware. As with the image-space losses (Sec-
tion 3.3), we can optimize
𝛼
for bias alone or for combined bias and
variance.
For variance-aware optimization, we compute
L
cache,𝑖
and
L
mat,𝑖
with a single sample. The expected squared error includes both bias
and variance:
E
(𝐼 − 𝑇 · 𝑅)
2
= bias
2
+ variance, (18)
so
𝛼
is pushed toward the estimator with lower mean squared error.
For bias-only optimization, we split the squared loss into two
terms estimated with independent samples 𝐴 and 𝐵:
L
cache,𝑖
=
𝐼 − 𝑇
𝐴
𝑖
· 𝐶
𝑖
·
𝐼 − 𝑇
𝐵
𝑖
· 𝐶
𝑖
. (19)
Since 𝐴 and 𝐵 are independent, the expectation factorizes:
E
(𝐼 − 𝑇
𝐴
· 𝑅)(𝐼 − 𝑇
𝐵
· 𝑅)
=
(
𝐼 − E[𝑇 · 𝑅]
)
2
= bias
2
, (20)
removing the variance term from the optimization objective.
C Termination strategies for separated losses
The separated loss (Section 3.2) requires choosing where to termi-
nate paths in each iteration. Here we describe strategies in detail.
Uniform depth sampling. The simplest approach samples a ter-
mination depth
𝑘
uniformly from
{
1
, . . . , 𝑘
max
}
each iteration. All
pixels share the same
𝑘
, producing a coherent image for the loss.
This baseline is easy to implement and works well in practice.
𝛼
-aware termination. Uniform sampling may terminate paths
where
𝛼
is low, indicating the cache is unreliable. To mitigate this,
we extend paths beyond the sampled depth deterministically until
reaching a more reliable region:
(1) Sample a minimum depth 𝑘 uniformly at random.
(2)
Starting from bounce
𝑘
, compute the accumulated product
Î
𝑖 ≥𝑘
(1 − 𝛼
𝑖
).
(3)
Terminate at the rst bounce where this product drops below
0.5; no additional random number is drawn.
Intuitively, we continue until enough “trust the cache” has accumu-
lated along the path.
Throughput-based termination. An alternative measures accumu-
lated trust from the path start. We compute the product
(
1
− 𝛼
1
)(
1
−
𝛼
2
) · · · (
1
− 𝛼
𝑘 −1
)
, which represents how much the path has favored
the material estimator. Each iteration, we draw a threshold
𝜏 ∈ [
0
,
1
]
and terminate at the rst bounce where this product drops below
𝜏
.
This naturally biases termination toward high-𝛼 regions.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
135:16 • Zhang et al.
Comparison. In our benchmarks, uniform depth sampling is ap-
proximately 0
.
4 dB lower than the
𝛼
-aware strategies. The
𝛼
-aware
and throughput-based strategies perform comparably, with no dis-
tinguishable dierence between them. We use
𝛼
-aware termination
for all main results.
Computing all losses simultaneously. Rather than sampling one
termination depth, we can compute losses at all depths in a single
pass by storing each
ˆ
𝐼
𝑘
in a separate output buer. The total loss is
then
Í
𝑘
𝑤
𝑘
L
𝑘
for some weights
𝑤
𝑘
. This avoids variance from sam-
pling but requires
𝑂 (depth)
memory per pixel. We use the sampling
approach in all experiments for its lower memory footprint.
Per-ray vs. per-iteration termination. The termination criterion
should be decided per-iteration, not per-ray. If termination were
decided per-ray—for example, terminating at bounce
𝑘
with proba-
bility
𝛼
𝑘
— the expected image over innite samples would recover
the blended estimator from Equation 3, losing the direct supervision
that separated losses provide.
D Additional details and results
Optimized BSDF parameters. We use a script to convert known
scenes to trainable versions, where all materials are replaced with
trainable ones initialized to be neutral gray. We replace reectance-
like slots of non-smooth BSDFs by replacing any existing reectance
(diuse or specular) with trainable textures. IOR and conductor
eta
/
k
remain xed, and smooth BSDFs (e.g., mirror or glass) are
left untouched. In practice, this covers diuse walls/paints, rough
plastic and other plastic variants (wood, fabric, rubber, plastics), and
rough conductors (metals, chrome, gold). Smooth conductors are
skipped.
Cache architecture and defaults. The cache is a scene-level volume
implemented as a position-conditioned neural texture. By default,
we use a hashgrid encoder (base resolution 128, 8 levels, per-level
scale 1.5, per-level hash map size 2
22
, 4 features/level) paired with a
linear decoder storing RGBA values in fp16. Alternatively, a neural
decoder can be used, where the same hashgrid encoding feeds a
small MLP (2 hidden layers of size 64).
Scene material conguration. We replace every reectance pa-
rameter in non-smooth BSDFs with a trainable texture, regardless
of whether the original value was spatially varying or constant.
These textures are parameterized using a Laplacian pyramid [Weier
et al
.
2025] (base resolution 16, scale factor 2.0). While standard tex-
tures could also be used, we found that Laplacian pyramids improve
convergence speed and yield cleaner recovered materials.
Memory footprint. Across 7 scenes, the cache parameters are con-
stant at 240 MB. Scene-side trainable material parameters range
from 44.2 MB up to 255.3 MB; the mean is 133.8 MB.
Because our method is implemented with PRB, optimization re-
quires no more memory than a primal rendering pass.
Per-scene timing. Table 5 reports detailed per-iteration wall-clock
time for each scene. Our method incurs a small overhead compared
to PRB due to cache and blending eld evaluation. Hadadan et al
.
[2023] is marginally faster as it updates materials using only primary
hits and does not simulate light paths.
Scene Ours PRB Hadadan Hadadan*
Bedroom 0.041 0.035 0.032 0.032
Bathroom2 0.039 0.037 0.030 0.030
Kitchen 0.167 0.167 0.169 0.164
Living-room 0.036 0.035 0.037 0.036
Living-room-2 0.091 0.089 0.090 0.090
Living-room-3 0.034 0.031 0.019 0.019
Staircase 0.249 0.240 0.239 0.237
Mean 0.094 0.091 0.088 0.086
Table 5. Per-iteration training time. Average wall-clock time (seconds)
per iteration for optimizing 2
18
pixels (all overhead included). Our method
adds a small overhead over PRB due to cache and
𝛼
evaluation. Hadadan
et al
.
[2023] is faster because it updates only primary-hit materials and does
not simulate full light paths.
Additional progress results. Figure 12 shows additional optimiza-
tion progress visualizations, complementing Figure 11 in the main
paper.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.
Radiance Caching for Dierentiable Path Tracing • 135:17
Optimization states
Iter 20 Iter 60 Iter 160 Iter 400 Iter 1000
Material
Cache
Blending field
Initialization
Ground truth
0.0
1.0
Material
Cache
Blending field
Initialization
Ground truth
0.0
1.0
Material
Cache
Blending field
Initialization
Ground truth
0.0
1.0
Fig. 12. Additional optimization progress. Complementary visualizations to Figure 11, showing (top) material-only relit renders, (middle) cache at
camera-ray intersections, and (boom)
𝛼 (
x
)
over iterations. The bathroom scene’s warm ground-truth illumination makes the neutral-gray initialization
appear tinted when rendered under that lighting.
ACM Trans. Graph., Vol. 45, No. 4, Article 135. Publication date: July 2026.