Paired Image & LiDAR Labeling for Germany (650 Pairs)

Location: Germany (various urban and  suburban)
Data type: 650 paired samples – high‑resolution RGB images + LiDAR point clouds
Output format: 

Output formats (with Unique ID):

• 2D polygons (image segmentation masks)

• 2D bounding boxes (axis‑aligned rectangles)

• 3D bounding boxes (oriented cuboids in LiDAR/world coordinates)

• Per‑point semantic labels (every LiDAR point has a class label)

Project Overview

Nazru’s AI platform was deployed to produce high‑precision, multi‑layer annotations for a large dataset of synchronised camera and LiDAR data. The project required six complementary output types, each with detailed class definitions and attribute sets. The resulting ground‑truth data supports autonomous driving, traffic monitoring, and infrastructure mapping.

The Challenge

The annotation specification covered a wide range of object categories and surface types, with strict geometric and attribute requirements:

• 3D bounding boxes around each traffic participant (humans, vehicles, poles, buildings, guardrails, barriers, traffic signs, traffic lights) in LiDAR space, with position, dimensions, rotation, occlusion level (0–100%), uncertainty flag, and visibility (LiDAR / camera / both).

• 2D cuboids (eight‑point boxes, solid front / dashed back) in camera images, with same classes and attributes plus truncation flag.

• 2D semantic segmentation polygons covering every pixel, including humans, vehicles, objects, vegetation, surfaces (sidewalk, drivable area, road, driving lanes), terrain, sky, and background.

• LiDAR point‑wise labels – each point assigned a semantic class from the same list.

• Driving lanes as polygons (not lines), with directional attributes and relevant lane IDs.

• Lane markings as polygons, with semantic meaning (e.g., yield, priority road, no‑parking, etc.).

Additional attributes for humans included activity type (walking, standing, sitting, pushing, unknown). For vehicles: motion type (moving, crossing, standing, parking, unknown) and light function (blinking left/right, hazard, brake, neutral, unknown).

All annotations had to be cross‑consistent: the same physical object must share a single instance ID across its 3D box, 2D cuboid, 2D polygon, and LiDAR point labels. Geometric tolerances were tight – 3D boxes within 15 cm of nearest LiDAR point, semantic segmentation error less than 3 pixels.

The Nazru Solution

Nazru’s integrated annotation pipeline delivered all six output types simultaneously. For each image‑LiDAR pair, annotators used a custom tool to:

1. Place 3D bounding boxes around every relevant object in the point cloud. Boxes were oriented cuboids with exact fit to the LiDAR points. Attributes were selected from dropdowns (occlusion level, uncertainty, visibility, activity/motion type, light function).

2. Draw 2D cuboids on the camera image, aligned with the projected 3D boxes. Front face drawn with solid line, back face with dashed line. Truncation flag added for objects cut off at image edges.

3. Create 2D semantic polygons for all pixel‑level classes, with special attention to surfaces (sidewalk, drivable area, road, driving lanes) and thin structures (poles, lane markings). Polygons followed visible boundaries; occluded parts were not drawn.

4. Assign LiDAR point labels – each point received a semantic class (same as image segmentation). Points inside 3D boxes inherited the box’s class and instance ID; remaining points were labeled as ground, vegetation, building, etc., via segmentation.

5. Annotate driving lanes as polygons – each lane (including ego lane and same‑direction adjacent lanes) was drawn as a polygon covering the drivable area of that lane. Attributes:

   • direction (straight, right, left, east_to_west, west_to_east, north_to_east, north_to_west, south_to_east, south_to_west, incoming)

   • lane_number (for same‑direction lanes, numbered left to right)

   • opposite_lane (True/False) – opposite lanes have no lane_number

   • relevant_lanes (comma‑separated IDs of lanes that this lane connects to)

6. Annotate lane markings as polygons – each marking (solid, dashed, double, arrow, crosswalk, stop line, etc.) as a polygon. Attribute meaning defined the traffic rule: yield, priority road, no‑stopping, no‑waiting, no‑parking, parking area, barricades, local direction marker, street name sign, pedestrian‑&‑bicycle path, do‑not‑enter, unrecognizable, trams, turn‑right‑ahead, end‑of‑pedestrian‑zone, and others.

All outputs were linked by a unique instance_id per object, allowing direct cross‑modal retrieval.