This page contains the accompanying data for the Panoptic Mapping project. The code to run the data and build panoptic maps is available at https://github.com/ethz-asl/panoptic_mapping. The paper is available on IEEE and ArXiv:

Schmid, Lukas, et al. “Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency.” in IEEE International Conference on Robotics and Automation (ICRA), pp. 8018-8024, 2022.

The flat dataset consists of synthetic images, rendered using Unreal Engine 4, of two trajectories in an indoor environment that was subject to change. Structural ground truth point clouds and panoptic annotations are included.

Run 1	Run 2	Changes

The following files are included (# short description). Additional processing and data-to-ROS players are available on github.

Data:

• run<RunNo>
  • <FrameNo>_color.png # Color image of the sequence.
  • <FrameNo>_depth.tiff # Depth image of the sequence [m].
  • <FrameNo>_segmentation.png # Ground truth panoptic labels, each pixel contains a label ID.
  • <FrameNo>_predicted.png # Detectron2 predicted panoptic labels, each pixel contains a label ID.
  • <FrameNo>_labels.json # Dictionary of accompanying information for each Detectron2 prediction.
  • <FrameNo>_pose.txt # Pose of the sensor in Odom frame.
  • timestamps.csv # Timestamps of each frame as recorded in the simulation.

Ground Truth:

• flat_<RunNo>_gt_10000.ply # Structural ground truth pointcloud.

Utility Files:

• groundtruth_labels.csv # Labels of the panoptic ground truth IDs (use with the mapper).
• groundtruth_labels_classes.csv # Names of class IDs in 'groundtruth_labels.csv'.
• detectron_labels.csv # Labels of the panoptic detectron IDs (use with the mapper).
• changes.txt # Explanations of the changed objects with their ground truth label names.
• intrinsics.txt # Camera intrinsics of the images [pixels].

Log Files (You can safely ignore these)

• airsim.yaml # Configuration of the simulator used to create the data.
• infrared_corrections.csv # Infrared corrections when running the simulator.
• waypoints<RunNo>.yaml # Way points traveled to when generating the data.

To run the mapper on real data, two sequences of the RIO dataset [1] were evaluated. The original data set can not be redistributed, please download the data from here. We provide additional data to run Panoptic Mapping for the scenes 466 and 27, in particular, for the scan IDs:

• 0cac7578-8d6f-2d13-8c2d-bfa7a04f8af3
• 20c9939d-698f-29c5-85c6-3c618e00061f
• 2451c041-fae8-24f6-9213-b8b6af8d86c1
• ddc73793-765b-241a-9ecd-b0cebb7cf916
• ddc73795-765b-241a-9c5d-b97744afe077
• f62fd5f8-9a3f-2f44-8b1e-1289a3a61e26

Please download the additional data and add it to the already downloaded RIO sequences.

[1] Wald, Johanna, et al. “RIO: 3D object instance re-localization in changing indoor environments.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.

Data:

• <Sequence Hash>
  • sequence
    • frame-<FrameNo>.panlables.png # Ground truth panoptic labels, each pixel contains a label ID.
    • frame-<FrameNo>.predicted.png # Detectron2 predicted panoptic labels, each pixel contains a label ID.
    • frame-<FrameNo>.detectronlabels.json # Dictionary of accompanying information for each Detectron2 prediction.

Ground Truth:

• <Sequence Hash>
  • gt_10000.ply # Structural ground truth pointcloud for that sequence.
• gt_scene<Scene No>.ply # Combined ground truth pointcloud for a scene.

Utility Files:

• groundtruth_labels.csv # Labels of the panoptic ground truth IDs (use with the mapper).
• detectron_labels.csv # Labels of the panoptic detectron IDs (use with the mapper).

flat_dataset (392.7 MB)

rio_data (87.8 MB)