vis4d.eval.common.binary

Binary occupancy evaluator.

Functions

threshold_and_flatten(prediction, target, ...)

Thresholds the predictions based on the provided treshold value.

Classes

BinaryEvaluator([threshold])

Creates a new Evaluater that evaluates binary predictions.
class BinaryEvaluator(threshold=0.5)[source]

Creates a new Evaluater that evaluates binary predictions.

Creates a new binary evaluator.

Parameters:

threshold (float) – Threshold for prediction to convert to binary. All prediction that are higher than this value will be assigned the ‘True’ label

evaluate(metric)[source]

Evaluate predictions.

Returns a dict containing the raw data and a short description string containing a readable result.

Parameters:

metric (str) – Metric to use. See @property metric

Return type:

tuple[Dict[str, Union[float, int, Tensor]], str]

Returns:

metric_data, description tuple containing the metric data (dict with metric name and value) as well as a short string with shortened information.

Raises:

  • RuntimeError – if no data has been registered to be evaluated.

  • ValueError – if metric is not supported.

process_batch(prediction, groundtruth)[source]

Processes a new (batch) of predictions.

Calculates the metrics and caches them internally.

Parameters:

  • prediction (ArrayLike) – the prediction(continuous values or bin) (Batch x Pts)

  • groundtruth (ArrayLike) – the groundtruth (binary) (Batch x Pts)

Return type:

None

reset()[source]

Reset the saved predictions to start new round of evaluation.

Return type:

None

property metrics: list[str]

Supported metrics.

threshold_and_flatten(prediction, target, threshold_value)[source]

Thresholds the predictions based on the provided treshold value.

Applies the following actions:

prediction -> prediction >= threshold_value pred, gt = pred.ravel().bool(), gt.ravel().bool()

Parameters:

  • prediction (Union[ndarray[Any, dtype[bool_]], ndarray[Any, dtype[float32]], ndarray[Any, dtype[float64]], ndarray[Any, dtype[int32]], ndarray[Any, dtype[int64]], ndarray[Any, dtype[uint8]], ndarray[Any, dtype[uint16]], ndarray[Any, dtype[uint32]]]) – Prediction array with continuous values

  • target (Union[ndarray[Any, dtype[bool_]], ndarray[Any, dtype[float32]], ndarray[Any, dtype[float64]], ndarray[Any, dtype[int32]], ndarray[Any, dtype[int64]], ndarray[Any, dtype[uint8]], ndarray[Any, dtype[uint16]], ndarray[Any, dtype[uint32]]]) – Grondgtruth values {0,1}

  • threshold_value (float) – Value to use to convert the continuous prediction into binary.

Return type:

tuple[ndarray[Any, dtype[bool_]], ndarray[Any, dtype[bool_]]]

Returns:

tuple of two boolean arrays, prediction and target