Recommendation System Metrics

batcore.metrics.count_mrr(gt, pred)

Calculates mean reciprocal rank (mrr) of the given predictions wrt ground truth.

Parameters:

  • gt – ground truth

  • pred – predictions

Returns:

mean and std of reciprocal ranks

batcore.metrics.recall(gt, pred)

Calculates recall of the given predictions wrt ground truth.

Parameters:

  • gt – ground truth

  • pred – predictions

Returns:

Recall score

batcore.metrics.precision(gt, pred)

Calculates precision of the given predictions wrt ground truth.

Parameters:

  • gt – ground truth

  • pred – predictions

Returns:

precision score

batcore.metrics.accuracy(gt, pred)

Calculates accuracy of the given predictions wrt ground truth.

Parameters:

  • gt – ground truth

  • pred – predictions

Returns:

accuracy score

batcore.metrics.f1score(gt, pred)

Calculates F1-score of the given predictions wrt ground truth.

Parameters:

  • gt – ground truth

  • pred – predictions

Returns:

F1 score

batcore.metrics.bootstrap_estimation(metric_vals, bootstrap_size=50, bootstrap_repeat=1000)
Parameters:

  • metric_vals – metrics values per data-point

  • bootstrap_prob – probability of the data-point to appear in sub-sample

  • bootstrap_repeat – number of bootstrap iterations

Returns:

real mean and bootstrap variance estimation

batcore.metrics.count_topk_metric(res, top_k, metric, name='metric')
Parameters:

  • res – pd.DataFrame with prediction done by the model. Column ‘rev’ represents ground truth. Column ‘top-k’ represents best k suggestions

  • top_k – list with amount of the best suggestions

  • metric – metric function

  • name – name of the metric

Returns:

dict with mean values and stds for each metric calculated for each of the top-k suggestions

batcore.metrics.count_metrics(res, metrics=None, top_k=None)
Parameters:

  • res – pd.DataFrame with prediction done by the model.

  • metrics – metrics to calculate

  • top_k – list of k-s for top k metrics

Returns:

dict with mean values and variance for each metric