vis4d.eval.shift

SHIFT evaluation metrics.

class SHIFTDepthEvaluator(use_eval_crop=True)

SHIFT depth estimation evaluation class.

Initialize the evaluator.

Parameters:

use_eval_crop (bool) – Whether to use the evaluation crop. Default: True.

__repr__()

Concise representation of the dataset evaluator.

Return type:

str

process_batch(prediction, groundtruth)

Process sample and update confusion matrix.

Parameters:

• prediction (Union[ndarray[Any, dtype[bool_]], ndarray[Any, dtype[float32]], ndarray[Any, dtype[float64]], ndarray[Any, dtype[int32]], ndarray[Any, dtype[int64]], ndarray[Any, dtype[uint8]], ndarray[Any, dtype[uint16]], ndarray[Any, dtype[uint32]]]) – Predictions of shape (N, H, W).

• groundtruth (Union[ndarray[Any, dtype[bool_]], ndarray[Any, dtype[float32]], ndarray[Any, dtype[float64]], ndarray[Any, dtype[int32]], ndarray[Any, dtype[int64]], ndarray[Any, dtype[uint8]], ndarray[Any, dtype[uint16]], ndarray[Any, dtype[uint32]]]) – Groundtruth of shape (N, H, W).

Return type:

None

class SHIFTDetectEvaluator(annotation_path)

SHIFT detection evaluation class.

Initialize the evaluator.

class SHIFTMultitaskWriter(output_dir, submission_file='submission.zip')

SHIFT result writer for online evaluation.

Creates a new writer.

Parameters:

• output_dir (str) – Output directory.

• submission_file (str) – Submission file name. Defaults to “submission.zip”.

evaluate(metric)

No evaluation locally.

Return type:

tuple[Dict[str, Union[float, int, Tensor]], str]

gather(gather_func)

Gather variables in case of distributed setting (if needed).

Parameters:

gather_func (Callable[[Any], Any]) – Gather function.

Return type:

None

process_batch(frame_ids, sample_names, sequence_names, pred_sem_mask=None, pred_depth=None, pred_flow=None, pred_boxes2d=None, pred_boxes2d_classes=None, pred_boxes2d_scores=None, pred_boxes2d_track_ids=None, pred_instance_masks=None)

Process SHIFT results.

You can omit some of the predictions if they are not used.

Parameters:

• frame_ids (list[int]) – Frame IDs.

• sample_names (list[str]) – Sample names.

• sequence_names (list[str]) – Sequence names.

• pred_sem_mask (list[ArrayLike], optional) – Predicted semantic masks, each in shape (C, H, W) or (H, W). Defaults to None.

• pred_depth (list[ArrayLike], optional) – Predicted depth maps, each in shape (H, W), with meter unit. Defaults to None.

• pred_flow (list[ArrayLike], optional) – Predicted optical flows, each in shape (H, W, 2). Defaults to None.

• pred_boxes2d (list[ArrayLike], optional) – Predicted 2D boxes, each in shape (N, 4). Defaults to None.

• pred_boxes2d_classes (list[ArrayLike], optional) – Predicted 2D box classes, each in shape (N,). Defaults to None.

• pred_boxes2d_scores (list[ArrayLike], optional) – Predicted 2D box scores, each in shape (N,). Defaults to None.

• pred_boxes2d_track_ids (list[ArrayLike], optional) – Predicted 2D box track IDs, each in shape (N,). Defaults to None.

• pred_instance_masks (list[ArrayLike], optional) – Predicted instance masks, each in shape (N, H, W). Defaults to None.

Return type:

None

save(metric, output_dir)

Save scalabel output to zip file.

Raises:

ValueError – If the number of samples in each category is not the same.

Return type:

None

class SHIFTOpticalFlowEvaluator

SHIFT optical flow estimation evaluation class.

Initialize the evaluator.

__repr__()

Concise representation of the dataset evaluator.

Return type:

str

class SHIFTSegEvaluator(ignore_classes_as_cityscapes=True)

SHIFT segmentation evaluation class.

Initialize the evaluator.

__repr__()

Concise representation of the dataset evaluator.

Return type:

str

process_batch(prediction, groundtruth)

Process sample and update confusion matrix.

Parameters:

• prediction (Union[ndarray[Any, dtype[bool_]], ndarray[Any, dtype[float32]], ndarray[Any, dtype[float64]], ndarray[Any, dtype[int32]], ndarray[Any, dtype[int64]], ndarray[Any, dtype[uint8]], ndarray[Any, dtype[uint16]], ndarray[Any, dtype[uint32]]]) – Predictions of shape [N,C,…] or [N,…] with C* being any number if channels. Note, C is passed, the prediction is converted to target labels by applying the max operations along the second axis

• groundtruth (ndarray[Any, dtype[int64]]) – Groundtruth of shape [N_batch, …] type int

Return type:

None

class SHIFTTrackEvaluator(annotation_path, mask_threshold=0.0)

SHIFT tracking evaluation class.

Initialize the evaluator.

Modules

vis4d.eval.shift.depth	SHIFT depth estimation evaluator.
vis4d.eval.shift.detect	SHIFT detection evaluator.
vis4d.eval.shift.flow	SHIFT optical flow estimation evaluator.
vis4d.eval.shift.multitask_writer	SHIFT result writer.
vis4d.eval.shift.seg	SHIFT segmentation evaluator.
vis4d.eval.shift.track	SHIFT tracking evaluator.