calc_quantile_scores

Calculates quantile-based scores

Usage

calc_quantile_scores(backtest_summary, metrics)

Arguments

backtest_summary

results of backtest call

metrics

the quantile metrics to calculate - see get_quantile_metrics

Value

The weighted interval score for each of the runs in backtest().

Examples

# Load the data
data(denguedat)

# Run a nowcast with very few iterations
# change to method = "sampling" when working and remove the iter = 10 (or set to iter = 2000)
now <- as.Date("1990-10-01")
ncast <- nowcast(denguedat, "onset_week", "report_week", now = now,
  method = "optimization", seed = 2495624, iter = 10)

# Run a backtest for the model checking the model fit for two dates:
btest <- backtest(ncast, dates_to_test = c(as.Date("1990-06-11"), as.Date("1990-06-18")))

# Get the rmse with the scoringutils package
if (requireNamespace("scoringutils", quietly = TRUE)){
  metrics = c("wis","dispersion","bias")
  calc_quantile_scores(btest,metrics=metrics)
}
#>                                    model        now horizon Strata_unified
#>                                   <char>     <Date>   <num>         <char>
#> 1: model_2025-05-16_19h28m26s_1158282656 1990-06-11       0      No strata
#> 2: model_2025-05-16_19h28m26s_1158282656 1990-06-18       0      No strata
#>         wis dispersion  bias
#>       <num>      <num> <num>
#> 1: 4.300000   1.442857   0.9
#> 2: 7.928571   2.071429   1.0