vis4d.data.transforms.mosaic

Mosaic transformation.

Modified from mmdetection (https://github.com/open-mmlab/mmdetection).

Functions

mosaic_combine(index, center, im_hw, out_shape)

Compute the mosaic parameters for the image at the current index.

Classes

GenMosaicParameters(*[, in_keys, out_keys, ...])	Generate the parameters for a mosaic operation.
MosaicBoxes2D(*[, in_keys, out_keys, ...])	Apply Mosaic to a list of 2D bounding boxes.
MosaicImages(*[, in_keys, out_keys, ...])	Apply Mosaic to images.
MosaicParam	Parameters for Mosaic.

class GenMosaicParameters(*, in_keys=['input_hw'], out_keys=['transforms.mosaic'], sensors=None, same_on_batch=True, **kwargs)

Generate the parameters for a mosaic operation.

Given 4 images, mosaic transform combines them into one output image. The output image is composed of the parts from each sub- image.

mosaic transform

center_x

center_y |----+-------------+-----------|

| cropped | |

|pad | image3 | image4 | | | | | +----|————-+———–+

|

The mosaic transform steps are as follows:

1. Choose the mosaic center as the intersections of 4 images.

2. Get the left top image according to the index, and randomly sample another 3 images from the dataset.

3. Sub image will be cropped if image is larger than mosaic patch.

Parameters:

• out_shape (tuple[int, int]) – The output shape of the mosaic transform.

• center_ratio_range (tuple[float, float]) – The range of the ratio of the center of the mosaic patch to the output image size.

__call__(input_hw)

Compute the parameters and put them in the data dict.

Return type:

list[MosaicParam]

class MosaicBoxes2D(*, in_keys=['boxes2d', 'boxes2d_classes', 'boxes2d_track_ids', 'transforms.mosaic.paste_coords', 'transforms.mosaic.crop_coords', 'transforms.mosaic.im_scales'], out_keys=['boxes2d', 'boxes2d_classes', 'boxes2d_track_ids'], sensors=None, same_on_batch=True, **kwargs)

Apply Mosaic to a list of 2D bounding boxes.

__call__(boxes, classes, track_ids, paste_coords, crop_coords, im_scales)

Apply Mosaic to 2D bounding boxes.

Return type:

tuple[list[ndarray[Any, dtype[float32]]], list[ndarray[Any, dtype[int64]]], list[ndarray[Any, dtype[int64]]] | None]

class MosaicImages(*, in_keys=['images', 'transforms.mosaic.out_shape', 'transforms.mosaic.paste_coords', 'transforms.mosaic.crop_coords', 'transforms.mosaic.im_shapes'], out_keys=['images', 'input_hw'], sensors=None, same_on_batch=True, **kwargs)

Apply Mosaic to images.

__call__(images, out_shape, paste_coords, crop_coords, im_shapes)

Resize an image of dimensions [N, H, W, C].

Return type:

tuple[list[ndarray[Any, dtype[float32]]], list[tuple[int, int]]]

class MosaicParam

Parameters for Mosaic.

mosaic_combine(index, center, im_hw, out_shape)

Compute the mosaic parameters for the image at the current index.

Index: 0 = top_left, 1 = top_right, 3 = bottom_left, 4 = bottom_right

Return type:

tuple[tuple[int, int, int, int], tuple[int, int, int, int]]