Compute calibration metrics for predictions

Compute calibration metrics for predictions

Usage

compute_calibration(predictions)

Arguments

predictions

Tibble from read_predictions()

Value

List with: brier_score, accuracy, calibration_by_bucket, rolling_brier, n_total, n_resolved

Examples

preds <- tibble::tibble(
  prediction_id = paste0("pred_", 1:5),
  p_success = c(0.9, 0.7, 0.3, 0.8, 0.5),
  outcome = c(TRUE, TRUE, FALSE, TRUE, FALSE),
  outcome_binary = c(1, 1, 0, 1, 0),
  recorded_at = as.character(Sys.time() - (5:1) * 86400)
)
cal <- compute_calibration(preds)
cal$brier_score
#> [1] 0.096