vis4d.engine.callbacks.evaluator

This module contains utilities for callbacks.

Classes

EvaluatorCallback(*args, evaluator[, ...])

Callback for model evaluation.
class EvaluatorCallback(*args, evaluator, metrics_to_eval=None, save_predictions=False, save_prefix=None, **kwargs)[source]

Callback for model evaluation.

Parameters:

  • evaluator (Evaluator) – Evaluator.

  • metrics_to_eval (list[str], Optional) – Metrics to evaluate. If None, all metrics in the evaluator will be evaluated. Defaults to None.

  • save_predictions (bool) – If the predictions should be saved. Defaults to False.

  • save_prefix (str, Optional) – Output directory for saving the evaluation results. Defaults to None.

Init callback.

evaluate()[source]

Evaluate the performance after processing all input/output pairs.

Returns:

A dictionary containing the evaluation results. The

keys are formatted as {metric_name}/{key_name}, and the values are the corresponding evaluated values.

Return type:

MetricLogs

on_test_batch_end(trainer_state, model, outputs, batch, batch_idx, dataloader_idx=0)[source]

Hook to run at the end of a testing batch.

Return type:

None

on_test_epoch_end(trainer_state, model)[source]

Hook to run at the end of a testing epoch.

Return type:

Optional[Dict[str, Union[float, int, Tensor]]]

setup()[source]

Setup callback.

Return type:

None