vis4d.op.base.pointnet

Operations for PointNet.

Code taken from https://github.com/timothylimyl/PointNet-Pytorch/blob/master/pointnet/model.py and modified to allow for modular configuration.

Classes

LinearTransform([in_dimension, ...])	Module that learns a linear transformation for a input pointcloud.
PointNetEncoder([in_dimensions, ...])	PointNetEncoder.
PointNetEncoderOut(features, ...)	Output of the PointNetEncoder.
PointNetSegmentation(n_classes[, ...])	Segmentation network using a simple pointnet as encoder.
PointNetSemanticsLoss(semantic_loss, ...)	Losses for the pointnet semantic segmentation network.
PointNetSemanticsOut(class_logits, ...)	Output of the PointNet Segmentation network.

class LinearTransform(in_dimension=3, upsampling_dims=(64, 128, 1024), downsampling_dims=(1024, 512, 256), norm_cls='BatchNorm1d', activation_cls='ReLU')

Module that learns a linear transformation for a input pointcloud.

Code taken from https://github.com/timothylimyl/PointNet-Pytorch/blob/master/pointnet/model.py and modified to allow for modular configuration.

See T-Net in Pointnet publication (https://arxiv.org/pdf/1612.00593.pdf) for more information

Creates a new LinearTransform.

This learns a transformation matrix from data.

Parameters:

• in_dimension (int) – input dimension

• upsampling_dims (Iterable[int]) – list of intermediate feature shapes for upsampling

• downsampling_dims (Iterable[int]) – list of intermediate feature shapes for downsampling. Make sure this matches with the last upsampling_dims

• norm_cls (Optional(str)) – class for norm (nn.’norm_cls’) or None

• activation_cls (str) – class for activation (nn.’activation_cls’)

__call__(features)

Type definition for call implementation.

Return type:

Tensor

forward(features)

Linear Transform forward.

Parameters:

features (Tensor[B, C, N]) – Input features (e.g. points)

Return type:

Tensor

Returns:

Learned Canonical Transfomation Matrix for this input. See T-Net in Pointnet publication (https://arxiv.org/pdf/1612.00593.pdf) for further information

class PointNetEncoder(in_dimensions=3, out_dimensions=1024, mlp_dimensions=((64, 64), (64, 128)), norm_cls='BatchNorm1d', activation_cls='ReLU', **kwargs)

PointNetEncoder.

Encodes a pointcloud and additional features into one feature description

See pointnet publication for more information (https://arxiv.org/pdf/1612.00593.pdf)

Creates a new PointNetEncoder.

Parameters:

• in_dimensions (int) – input dimension (e.g. 3 for xzy, 6 for xzyrgb)

• out_dimensions (int) – output dimensions

• mlp_dimensions (Iterable[Iterable[int]]) – (Dimensions of MLP layers)

• norm_cls (Optional(str)) – class for norm (nn.’norm_cls’) or None

• activation_cls (str) – class for activation (nn.’activation_cls’)

• kwargs (Any) – See arguments of @LinearTransformStn

__call__(features)

Type definition for call implementation.

Return type:

PointNetEncoderOut

forward(features)

Pointnet encoder forward.

Parameters:

• features (Tensor[B, C, N]) – Input features stacked in channels.

• inputs (e.g. raw point) – [B, 3, N] , w color : [B, 3+3, N], …

Return type:

PointNetEncoderOut

Returns:

Extracted feature representation for input and all applied transformations.

class PointNetEncoderOut(features: torch.Tensor, pointwise_features: torch.Tensor, transformations: list[torch.Tensor])

Output of the PointNetEncoder.

features: Global features shape [N, feature_dim] pointwise Features: Pointwise features shape [N, last_mlp_dim, n_pts] transformations: list with all transformation matrixes that were used.

Shape [N, d, d]

Create new instance of PointNetEncoderOut(features, pointwise_features, transformations)

features: Tensor

Alias for field number 0

pointwise_features: Tensor

Alias for field number 1

transformations: list[Tensor]

Alias for field number 2

class PointNetSegmentation(n_classes, in_dimensions=3, feature_dimension=1024, norm_cls='BatchNorm1d', activation_cls='ReLU')

Segmentation network using a simple pointnet as encoder.

Creates a new Point Net segementation network.

Parameters:

• n_classes (int) – Number of semantic classes

• in_dimensions (int) – Input dimension (3 for xyz, 6 xyzrgb, …)

• feature_dimension (int) – Size of feature from the encoder

• norm_cls (Optional(str)) – class for norm (nn.’norm_cls’) or None

• activation_cls (str) – class for activation (nn.’activation_cls’)

Raises:

ValueError – If dimensions are invalid

__call__(points)

Call function.

Return type:

PointNetSemanticsOut

forward(points)

Pointnet Segmenter Forward.

Parameters:

points (tensor) – inputs points dimension [B, in_dim, n_pts]

Return type:

PointNetSemanticsOut

Returns:

Returns a list of tensors where the first element is the desired segmentation [B, n_classes, n_pts] and the other elements are the linear transformation matrices which have been used to transform the pointclouds @see LinearTransform

class PointNetSemanticsLoss(semantic_loss: torch.Tensor, regularization_loss: torch.Tensor)

Losses for the pointnet semantic segmentation network.

Create new instance of PointNetSemanticsLoss(semantic_loss, regularization_loss)

regularization_loss: Tensor

Alias for field number 1

semantic_loss: Tensor

Alias for field number 0

class PointNetSemanticsOut(class_logits: torch.Tensor, transformations: list[torch.Tensor])

Output of the PointNet Segmentation network.

Create new instance of PointNetSemanticsOut(class_logits, transformations)

class_logits: Tensor

Alias for field number 0

transformations: list[Tensor]

Alias for field number 1