pyanno4rt.learning_model.evaluation.metrics

Evaluation metrics module.

The module aims to provide functions to compute different evaluation metrics.

Overview

Function

auc_pr(true_labels, predicted_labels)

Compute the AUC-PR scores.

auc_roc(true_labels, predicted_labels)

Compute the AUC-ROC scores.

f1(true_labels, predicted_labels)

Compute the F1 scores.

kpi(true_labels, predicted_labels, thresholds)

Compute the model KPIs.

Functions

pyanno4rt.learning_model.evaluation.metrics.auc_pr(true_labels, predicted_labels)[source]

Compute the AUC-PR scores.

Parameters:

  • true_labels (ndarray) – Ground truth label values.

  • predicted_labels (tuple) – Tuple of arrays with the training and out-of-folds labels predicted by the machine learning outcome model.

Returns:

scores – Dictionary with the training and out-of-folds AUC-PR scores.

Return type:

dict

pyanno4rt.learning_model.evaluation.metrics.auc_roc(true_labels, predicted_labels)[source]

Compute the AUC-ROC scores.

Parameters:

  • true_labels (ndarray) – Ground truth label values.

  • predicted_labels (tuple) – Tuple of arrays with the training and out-of-folds labels predicted by the machine learning outcome model.

Returns:

scores – Dictionary with the training and out-of-folds AUC-ROC scores.

Return type:

dict

pyanno4rt.learning_model.evaluation.metrics.f1(true_labels, predicted_labels)[source]

Compute the F1 scores.

Parameters:

  • true_labels (ndarray) – Ground truth label values.

  • predicted_labels (tuple) – Tuple of arrays with the training and out-of-folds labels predicted by the machine learning outcome model.

Returns:

scores – Dictionary with the training and out-of-folds F1 scores and the location of the best score.

Return type:

dict

pyanno4rt.learning_model.evaluation.metrics.kpi(true_labels, predicted_labels, thresholds=(0.5, 0.5))[source]

Compute the model KPIs.

Parameters:

  • true_labels (ndarray) – Ground truth label values.

  • predicted_labels (tuple) – Tuple of arrays with the training and out-of-folds labels predicted by the machine learning outcome model.

  • thresholds (tuple, default=(0.5, 0.5)) – Probability thresholds for the binarization of the probability predictions.

Returns:

scores – Dictionary with the training and out-of-folds KPIs.

Return type:

dict