Exploring Authenticity in NeRF and Gaussian Splatting
Why researchers must explore “authenticity” in realistic VR technologies like NeRF and Gaussian Splatting
The world of immersive digital storytelling and documentation is rapidly evolving, driven by powerful technologies like Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS). These groundbreaking techniques can recreate incredibly realistic 3D environments from just a handful of photographs – drastically fewer than a more “traditional” process like photogrammetry requires.
Resulting 3D reconstructions have serious implications for fields like law, journalism, and history. General consumer audiences may associate “virtual reality” with video games and similar engaging but otherwise low-stakes experiences. In practice, those same VR headsets are already being worn in court by judges in the United States.
On screens around the world, audiences are visiting the “digital twins” of undersea wrecks like the Titanic and Shackleton’s Explorer, or the glittering Doge Palace in Italy and St. Peter’s Basilica at the Vatican.
New opportunities in 3D storytelling and evidence presentation also bring a significant danger: the potential to mistake compelling and believable content for genuine reality. We face the crucial challenge of ensuring people can distinguish between authentic representations and algorithmically generated content.
Three Approaches to 3D Reconstruction: A Reliability Spectrum
Today, we can identify three distinct categories of 3D reconstruction tools, each with different implications for trust and authenticity:
Photogrammetric Processes: These established techniques require hundreds or thousands of images but produce highly reliable – and to an extent, deterministic – results with minimal algorithmic interpolation. Courts, forensic experts, and industry (insurance, etc.) currently rely on these methods to a significant extent due to their verifiable nature and transparent methodology.
Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS): These emerging techniques can create compelling 3D environments from significantly fewer images. While they don't completely fabricate content, they do algorithmically "fill in the gaps" between captured data points, creating a middle ground between documented reality and generation.
Generative AI Video Tools: These technologies can create entirely synthetic environments with minimal or no authentic source material. These approaches can extend a single image into video (popularized by commercial services like Sora, etc.) into the opposite end of the reliability spectrum, producing content that may be visually compelling but risks bearing little relation to documented reality. Selected key frames from these videos can then be fed into NeRF or 3DGS models.
The Double-Edged Sword of Advanced 3D Reconstruction
NeRFs and 3DGS currently occupy a middle position on this spectrum, differing from earlier photogrammetric techniques (based on producing 3D polygons) by representing scenes as point clouds with view-dependent characteristics. This allows them to accurately depict complex visual effects, such as reflections and transparency, much like photographs with depth.
Think of it like taking many photos of a statue from different angles. Traditional methods would stitch these photos together to create a 3D model made of flat surfaces (like a paper model). NeRFs and 3DGS, however, treat the statue as a collection of tiny, colorful dust particles that change how they look depending on your viewpoint, capturing subtle details like how light reflects off the surface.
However, these technologies are evolving at breathtaking speed, with new papers, algorithms, and tools published weekly. The rapid advancement of NeRF and 3DGS techniques presents a concerning trajectory: as they require fewer source images and incorporate more generative capabilities, the boundary between authentic representation and algorithmic fabrication becomes increasingly blurred. There's a significant risk that these tools—or their successors—will soon cross into a "danger zone" where viewers cannot reliably distinguish between documented reality and synthetic creation.
Researching Authenticity in Immersive Environments
Understanding the boundaries between authentic, verifiable reality and algorithmically-generated data is crucial – especially in the context of evidence or historical reconstruction. The relative ease of creating convincing reconstructions thanks to NeRF and 3DGS underscores the need for focused research into how viewers perceive authenticity, how trust is established in immersive media, and the implications of blurred boundaries between the real and synthetic.
In today's models, it's relatively clear at which point the model breaks down. Such failures in realism are easy to spot. Accurate portions of a scene based on authentic 2D photographs are relatively clear, while areas that might be hallucinated are either missing or fuzzy (sometimes blurring into oblivion). Tomorrow's technologies may not present such clear indicators. Unlike generative AI, which can create obvious misdirections, NeRF and 3DGS occupy a more nuanced position that demands careful study.
Above video: Four rendering techniques for the same scene of a burned residential area from the 2025 Los Angeles wildfires. Note in NeRF and Gaussian Splat the tell-tale signs of missing data: blurring, artefacts, wrong colors. Credit: Mike Caronna, for ASU
Our research at the Starling Lab is addressing these very questions, exploring how methods like "authenticity anchors" can serve as tools to verify authenticity. These anchors are cryptographic integrity markers linking elements within the 3D space (such as 2D photographs) back to securely authenticated source data (using industry technical standards like C2PA and a variety of cryptographic techniques). Although they are a practical solution, their greatest value lies in supporting deeper investigations into user interactions with authenticity and the effectiveness of transparency mechanisms in XR environments.
Building Trust in XR: Practical Research Directions
We are seeing 3D reconstruction approaches rapidly evolve at the same time we see them being introduced into high-stakes environments like courtrooms. As a result, we propose the following axes of research demand immediate attention to ensure authenticity infrastructure is integrated:
Perception of Authenticity: Investigating how users perceive and differentiate between authentic and generated content in immersive experiences, and at which point trust breaks down along the spectrum.
Verification Methods: Studying which verification techniques are most effective for each category of 3D reconstruction, from cryptographic proofs for photogrammetry to clearer "authenticity boundaries" for NeRF/3DGS. In addition, the blossoming field of spatial intelligence may help navigate this nuance by providing a way to assess whether the algorithmic "filling in the gaps" is reasonable and aligned with real-world expectations. In other words: if the model's spatial understanding is sound, it increases confidence in the reconstructed environment.
Transparent Interpolation Indicators: Researching methods to visually indicate when and where algorithmic interpolation has occurred in NeRF and 3DGS reconstructions, allowing users to understand the boundary between captured and generated content.
Spatial Intelligence in reconstruction: By enabling models to "understand" the 3D space, the blossoming field of spatial intelligence may enhance the overall reliability of these immersive experiences. If a model can identify and reason about objects within the scene, it suggests a deeper level of processing and a lower likelihood of "hallucinating" or misrepresenting elements of the environment. This directly impacts trust, as users are more likely to believe and rely on a system that demonstrates an understanding of what it's depicting, perhaps even similar to the way a human witness would evaluate a scene.
Building upon this research, our goal is to translate the insights gained into practical, real-world solutions. These solutions will likely encompass a combination of technical authentication, user experience (UX) design considerations, educational programs, and public awareness initiatives.
Looking Ahead
As immersive technologies advance, the critical question of authenticity will become even more prominent. Focused research in this area is essential not just for maintaining trust and transparency but also for safeguarding the integrity of digital storytelling and documentation. By clearly understanding and navigating the fine line between the real and the synthesized, we can harness the extraordinary capabilities of emerging technologies like NeRF and 3DGS responsibly and effectively.
 | A guest post by
|