Makes a summary of your backtest_metrics() object — summary.backtest

Makes a tidy summary dataframe of the results of the function backtest_metrics()

Usage

# S3 method for class 'backtest_metrics'
summary(object, ..., metrics = "all")

Arguments

object: A backtest_metrics object generated by backtest_metrics()
...: further arguments passed to the generic summary() function.
metrics: selects the metrics to display. metrics = "all" summarizes all available metrics generated in backtest_metrics()

Value

A summary tibble of the backtest_metrics() results, it specifies: average, standard deviation, standard error, number of "now" dates it also returns the metrics across all the horizons provided ("all" in column "horizon")

Examples

# These examples require the `scoringutils` package
if (requireNamespace("scoringutils", quietly = TRUE)) {
  # Load the data
  data(denguedat)
  # In this example, we will test two models
  now    <- as.Date("1990-10-01")
  ncast1 <- nowcast(denguedat, "onset_week", "report_week", now = now,
                     method = "optimization", seed = 2495624, iter = 10)
  ncast2 <- nowcast(denguedat, "onset_week", "report_week", now = now,
                    method = "optimization", seed = 2495624, iter = 10,
                    dist = "Normal")
  # Run a backtest for each of the models
  btest1 <- backtest(ncast1, dates_to_test = as.Date("1990-06-11"),
                     model_name = "Classic")
  btest2 <- backtest(ncast2, dates_to_test = as.Date("1990-06-11"),
                      model_name = "Normal")
  # Compare the models to select the best model
  comparison <- backtest_metrics(btest1, btest2)

  # Create a summary
  summary(comparison)

  # Create a summary with more metrics:
  summary(comparison, metrics = c("mae", "rmse", "ape", "wis"))
  }
#> # A tibble: 8 × 8
#>   model   Strata_unified horizon metric    avg    sd n_now    se
#>   <chr>   <chr>          <chr>   <chr>   <dbl> <dbl> <int> <dbl>
#> 1 Classic No strata      0       ape     1.92     NA     1    NA
#> 2 Classic No strata      0       mae    11.5      NA     1    NA
#> 3 Classic No strata      0       rmse   11.5      NA     1    NA
#> 4 Classic No strata      0       wis     4.30     NA     1    NA
#> 5 Normal  No strata      0       ape     0.187    NA     1    NA
#> 6 Normal  No strata      0       mae     1.12     NA     1    NA
#> 7 Normal  No strata      0       rmse    1.12     NA     1    NA
#> 8 Normal  No strata      0       wis     1.83     NA     1    NA