NARUTO

Neural Active Reconstruction from Uncertain Target Observations

We present NARUTO, a neural active reconstruction system that combines a hybrid neural representation with uncertainty learning, enabling high-fidelity surface reconstruction. Our approach leverages a multi-resolution hash-grid as the mapping backbone, chosen for its exceptional convergence speed and capacity to capture high-frequency local features.

The centerpiece of our work is the incorporation of an uncertainty learning module that dynamically quantifies reconstruction uncertainty while actively reconstructing the environment. By harnessing learned uncertainty, we propose a novel uncertainty aggregation strategy for goal searching and efficient path planning. Our system autonomously explores by targeting uncertain observations and reconstructs environments with remarkable completeness and fidelity. We also demonstrate the utility of this uncertainty-aware approach by enhancing SOTA neural SLAM systems through an active ray sampling strategy.

Extensive evaluations of NARUTO in various environments, using an indoor scene simulator, confirm its superior performance and state-of-the-art status in active reconstruction, as evidenced by its impressive results on benchmark datasets like Replica and MP3D.

The first column displays RGB-D images rendered from the simulator. For enhanced 3D visualization, we present two different viewpoints. The second column illustrates the full trajectory drawn on the final mesh, while the third column depicts the incremental reconstruction progress. The fourth column visualizes the normalized uncertainty 3D map as the agent explores the environment. Black lines indicate the planned path, with white lines representing the rays connecting the current location to selected targets of uncertainty

Upon reaching a keyframe step, HabitatSim generates posed RGB-D images. A select number of pixels from these images are sampled and stored in the observation database.

Utilizing a mixed ray sampling strategy (combining Random and Active methods), a subset of rays is selected from the current keyframe and the database. These rays are then processed through the Hybrid Scene Representation (Map) to deduce the corresponding color, Signed Distance Function (SDF), depth, and uncertainty values.

The predictions derived from this process facilitate uncertainty-aware bundle adjustment, updating both the scene's geometry and reconstruction uncertainty.

Subsequently, the Map is refreshed, and our novel uncertainty-aware planning algorithm is employed to determine a goal and trajectory based on the SDFs and uncertainties. The agent then executes the planned action.

Uncertainty-based aggregation and Planning: The top-k uncertain points are accumulated within the sensing range at each potential goal location. The goal with the greatest level of uncertainty is subsequently selected as the provisional target location. Efficient RRT planning effectively identifies a viable trajectory from the agent’s current position to the designated goal.

Quantitative Results

On Replica and Matterport3D datasets, our method achieves superior active reconstruction results. Our ablation studies on Replica dataset show the effectiveness of our proposed componenets

Qualitative Results

Better watching experience suggestions:
(1) Only 2 models are auto-loaded for better browsing experience.
(2) Change Model Inspector to "Matcap" for better geometry visualization.
(3) Enable Single Sided in Model Inspector for culled model visualization.
(4) Enter fullscreen mode for viewing reconstruction details.