vis4d.op.detect.mask_rcnn

Mask RCNN detector.

Functions

positive_mask_sampler(target_masks, ...)

Sample only positive masks from target masks.

Classes

Det2Mask([mask_threshold, no_overlap])

Post processing of mask predictions.

MaskOut(masks, scores, class_ids)

Output of the final detections from Mask RCNN.

MaskRCNNHead([num_classes, num_convs, ...])

Mask R-CNN RoI head.

MaskRCNNHeadLoss(num_classes)

Mask RoI head loss function.

MaskRCNNHeadLosses(rcnn_loss_mask)

Mask RoI head loss container.

MaskRCNNHeadOut(mask_pred)

Mask R-CNN RoI head outputs.

MaskSampler(*args, **kwargs)

Type definition for mask sampler.

SampledMaskLoss(mask_sampler, loss)

Sampled Mask RCNN head loss function.
class Det2Mask(mask_threshold=0.5, no_overlap=False)[source]

Post processing of mask predictions.

Parameters:

  • mask_threshold (float, optional) – Positive threshold. Defaults to 0.5.

  • no_overlap (bool, optional) – Whether to remove overlapping pixels between masks. Defaults to False.

Creates an instance of the class.

__call__(mask_outs, det_boxes, det_scores, det_class_ids, original_hw)[source]

Type definition for function call.

Return type:

MaskOut

forward(mask_outs, det_boxes, det_scores, det_class_ids, original_hw)[source]

Paste mask predictions back into original image resolution.

Parameters:

  • mask_outs (list[torch.Tensor]) – List of mask outputs for each batch element.

  • det_boxes (list[torch.Tensor]) – List of detection boxes for each batch element.

  • det_scores (list[torch.Tensor]) – List of detection scores for each batch element.

  • det_class_ids (list[torch.Tensor]) – List of detection classeds for each batch element.

  • original_hw (list[tuple[int, int]]) – Original image resolution.

Returns:

Post-processed mask predictions.

Return type:

MaskOut

class MaskOut(masks: list[torch.Tensor], scores: list[torch.Tensor], class_ids: list[torch.Tensor])[source]

Output of the final detections from Mask RCNN.

Create new instance of MaskOut(masks, scores, class_ids)

class_ids: list[Tensor]

Alias for field number 2

masks: list[Tensor]

Alias for field number 0

scores: list[Tensor]

Alias for field number 1

class MaskRCNNHead(num_classes=80, num_convs=4, roi_size=(14, 14), in_channels=256, conv_kernel_size=3, conv_out_channels=256, scale_factor=2, class_agnostic=False)[source]

Mask R-CNN RoI head.

Parameters:

  • num_classes (int, optional) – Number of classes. Defaults to 80.

  • num_convs (int, optional) – Number of convolution layers. Defaults to 4.

  • roi_size (tuple[int, int], optional) – Size of RoI after pooling. Defaults to (14, 14).

  • in_channels (int, optional) – Input feature channels. Defaults to 256.

  • conv_kernel_size (int, optional) – Kernel size of convolution. Defaults to 3.

  • conv_out_channels (int, optional) – Output channels of convolution. Defaults to 256.

  • scale_factor (int, optional) – Scaling factor of upsampling. Defaults to 2.

  • class_agnostic (bool, optional) – Whether to do class agnostic mask prediction. Defaults to False.

Creates an instance of the class.

forward(features, boxes)[source]

Forward pass.

Parameters:

  • features (list[torch.Tensor]) – Feature pyramid.

  • boxes (list[torch.Tensor]) – Proposal boxes.

Returns:

Mask prediction outputs.

Return type:

MaskRCNNHeadOut

class MaskRCNNHeadLoss(num_classes)[source]

Mask RoI head loss function.

Parameters:

num_classes (int) – number of object categories.

Creates an instance of the class.

forward(mask_preds, proposal_boxes, target_classes, target_masks)[source]

Calculate losses of Mask RCNN head.

Parameters:

  • mask_preds (list[torch.Tensor]) – [M, C, H’, W’] mask outputs per batch element.

  • proposal_boxes (list[torch.Tensor]) – [M, 4] proposal boxes per batch element.

  • target_classes (list[torch.Tensor]) – list of [M, 4] assigned target boxes for each proposal.

  • target_masks (list[torch.Tensor]) – list of [M, H, W] assigned target masks for each proposal.

Returns:

mask loss.

Return type:

MaskRCNNHeadLosses

class MaskRCNNHeadLosses(rcnn_loss_mask: torch.Tensor)[source]

Mask RoI head loss container.

Create new instance of MaskRCNNHeadLosses(rcnn_loss_mask,)

rcnn_loss_mask: Tensor

Alias for field number 0

class MaskRCNNHeadOut(mask_pred: list[torch.Tensor])[source]

Mask R-CNN RoI head outputs.

Create new instance of MaskRCNNHeadOut(mask_pred,)

mask_pred: list[Tensor]

Alias for field number 0

class MaskSampler(*args, **kwargs)[source]

Type definition for mask sampler.

__call__(target_masks, sampled_target_indices, sampled_targets, sampled_proposals)[source]

Type definition for function call.

Parameters:

  • target_masks (list[Tensor]) – list of [N, H, W] target masks per batch element.

  • sampled_target_indices (list[Tensor]) – list of [M] indices of sampled targets per batch element.

  • sampled_targets (Targets) – sampled targets.

  • sampled_proposals (Proposals) – sampled proposals.

Returns:

sampled masks,

sampled target indices, sampled targets.

Return type:

tuple[list[Tensor], list[Tensor], list[Tensor]]

class SampledMaskLoss(mask_sampler, loss)[source]

Sampled Mask RCNN head loss function.

Initialize sampled mask loss.

Parameters:
forward(mask_preds, target_masks, sampled_target_indices, sampled_targets, sampled_proposals)[source]

Calculate losses of Mask RCNN head.

Parameters:

  • mask_preds (list[torch.Tensor]) – [M, C, H’, W’] mask outputs per batch element.

  • target_masks (list[torch.Tensor]) – list of [M, H, W] assigned target masks for each proposal.

  • sampled_target_indices (list[Tensor]) – list of [M, 4] assigned target boxes for each proposal.

  • sampled_targets (Targets) – list of [M, 4] assigned target boxes for each proposal.

  • sampled_proposals (Proposals) – list of [M, 4] assigned target boxes for each proposal.

Returns:

mask loss.

Return type:

MaskRCNNHeadLosses

positive_mask_sampler(target_masks, sampled_target_indices, sampled_targets, sampled_proposals)[source]

Sample only positive masks from target masks.

Parameters:

  • target_masks (list[Tensor]) – list of [N, H, W] target masks per batch element.

  • sampled_target_indices (list[Tensor]) – list of [M] indices of sampled targets per batch element.

  • sampled_targets (Targets) – sampled targets.

  • sampled_proposals (Proposals) – sampled proposals.

Returns:

sampled masks,

sampled target indices, sampled targets.

Return type:

tuple[list[Tensor], list[Tensor], list[Tensor]]