tbl.now

Within R, the tidyverse ecosystem (Wickham et al. 2019) provides a widely adopted framework for data analysis. In this paradigm, data is structured such that rows represent individual observations and columns represent variables. This approach is commonly referred to as tidy data (Wickham 2014).

Several tidyverse extensions exist for working with time series data, including tsibble, tibbletime, and timetk (Wang et al. 2020; Wang 2019; Dancho and Vaughan 2023). These packages provide time-aware tibbles and tools that integrate smoothly with the tidyverse. However, epidemiological nowcasting requires two time indices simultaneously: an event time and a reporting time. Classical time-series abstractions assume only a single index and therefore do not fully support this structure.

The tbl.now class and package addresses this gap. The tbl.now extends a regular tibble() to explicitly encode epidemiological event and report dates, allowing consistent data transformation, delay computation, and integration with the diseasenowcasting modeling workflow.

More concretely, tbl.now was designed to:

• Standardize the data inputs required by diseasenowcasting models.

• Preserve tidyverse compatibility so users can continue to apply familiar dplyr operations.

• Diagnose reporting artefacts directly from the data — reporting-delay drift and change points (test_delay_drift(), test_delay_changepoint()) and batch (backlog) reporting (detect_report_batches()).

• Facilitate seamless integration into iterative modeling workflows (Gelman et al. 2020; Wickham et al. 2023):

Data Cleaning -> Modeling -> New Data Cleaning -> New Modeling -> ...

We’ll begin by loading the required packages:

library(dplyr, quietly = TRUE)
library(lubridate)
library(tbl.now)

The tbl.now Framework

In an epidemiological nowcast, we typically observe two key dates:

• event_date: when the underlying event occurred (e.g., symptom onset, exposure, sample collection).

• report_date: when the event was recorded in the data system (e.g., lab result processed, clinical visit documented).

The nowcasting task is:

To estimate, for each past event_date, how many events have already occurred but have not yet been reported as of now. That is, the nowcast will predict how many observations will eventually be observed for the event_dates.

A tbl.now object is therefore a specialized tibble() that:

• Identifies the event_date and report_date columns.

• Stores these as attributes to enable consistent processing.

• Automatically computes auxiliary fields such as delay, numerical indices, and frequency units.

• Ensures the dataset is well-formed for downstream diseasenowcasting models.

Example

Consider the following dataset:

df <- data.frame(
  symptom_onset = c(ymd("2023/12/25"), ymd("2023/12/26"), ymd("2023/12/25"), ymd("2023/12/26")),
  medical_visit = c(ymd("2023/12/26"), ymd("2023/12/26"), ymd("2023/12/27"), ymd("2023/12/27")),
  n = c(10, 2, 5, 11)
)

knitr::kable(df, caption = "Example dataset")

Example dataset

symptom_onset	medical_visit	n
2023-12-25	2023-12-26	10
2023-12-26	2023-12-26	2
2023-12-25	2023-12-27	5
2023-12-26	2023-12-27	11

Here:

• symptom_onset is the event date.

• medical_visit is the report date.

• n is the number of reported cases for each event–report combination.

We can convert this into a tbl_now():

df |>
  tbl_now(event_date = symptom_onset, report_date = medical_visit, case_count = n)
#> ℹ Identified data as <count-incidence> with counts in column "n".
#> # A tibble:  4 × 6
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>   symptom_onset medical_visit       n .event_num .report_num .delay
#>   <date>        <date>          <dbl>      <dbl>       <dbl>  <dbl>
#>   [event_date]  [report_date] [cases]      [...]       [...]  [...]
#> 1 2023-12-25    2023-12-26         10          0           1      1
#> 2 2023-12-26    2023-12-26          2          1           1      0
#> 3 2023-12-25    2023-12-27          5          0           2      2
#> 4 2023-12-26    2023-12-27         11          1           2      1
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2023-12-27 | Event date: "symptom_onset" | Report date: "medical_visit"
#> # ────────────────────────────────────────────────────────────────────────────────

This performs several operations automatically:

• Detects the data type (count-incidence in this example).

• Infers the frequency units of event and report dates (daily).

• Tags the correct columns as event_date, report_date, and case_count.

• Computes .event_num, .report_num, and .delay columns.

• Identifies the appropriate now date (the most recent report date).

The remaining sections describe these features and the broader tbl.now toolkit.

Attributes of a tbl_now()

A tbl_now() stores information about its structure using object attributes, ensuring consistent behavior across transformations and modeling steps. The primary attributes are:

• now: the “current” reporting date used for nowcasting, typically the most recent report_date.

• event_date: the column name storing event dates.

• report_date: the column name storing report dates.

• event_units: the temporal units for event dates (e.g., “days”, “weeks”, “numeric”).

• report_units: the temporal units for report dates.

• data_type: one of the following (see the data types section):

  • linelist
  • count-incidence
  • count-cumulative.

• strata (optional)¹: variables for which the nowcast should be computed separately (e.g., age group, sex).

• covariates (optional): predictor variables that may improve the nowcast (e.g., weather covariates).

• is_censored (optional): identifies cases where some reports arrive in artificial “batches” representing right-censoring rather than true reporting dynamics (e.g. when there is a system error and reports from a lab are not registered until a week after)

• case_count (optional): the column storing case counts when the dataset is aggregated.

• temporal_effects (optional): a lazy specification (list of temporal_effects() configs) stored via add_temporal_effects(). Columns are only materialised when compute_temporal_effects() is called. See the temporal effects section for more details.

You can access any attribute using the corresponding getter, e.g. get_event_date() or get_strata().

Data types

A tbl_now() can represent one of three data structures:

1. Linelist: Each row corresponds to a single reported observation.

Linelist data

patient	event_date	report_date	.event_num	.report_num	.delay
1	2020-09-12	2020-09-12	0	0	0
2	2020-09-12	2020-09-12	0	0	0
3	2020-09-12	2020-09-13	0	1	1
4	2020-09-13	2020-09-13	1	1	0
5	2020-09-13	2020-09-13	1	1	0
6	2020-09-13	2020-09-13	1	1	0

2. Count-incidence: Each row summarizes how many events with a given event_date were reported exactly on that report_date.

Count-incidence data

n	event_date	report_date	.event_num	.report_num	.delay
7	2020-09-12	2020-09-12	0	0	0
1	2020-09-12	2020-09-13	0	1	1
9	2020-09-12	2020-09-14	0	2	2
5	2020-09-13	2020-09-13	1	1	0
0	2020-09-13	2020-09-14	1	2	1
2	2020-09-13	2020-09-15	1	3	2

3. Count-cumulative Each row summarizes how many events with a given event_date had been reported up to and including that report_date. The distinction is crucial for nowcasting models that operate either on daily increments or cumulative totals.

Count-cumulative data

n	event_date	report_date	.event_num	.report_num	.delay
1	2020-09-12	2020-09-12	0	0	0
5	2020-09-12	2020-09-13	0	1	1
8	2020-09-12	2020-09-14	0	2	2
2	2020-09-13	2020-09-13	1	1	0
2	2020-09-13	2020-09-14	1	2	1
4	2020-09-13	2020-09-15	1	3	2

The to_count() function allows you to easily convert between different data-types.

Converting Between Data Types

The to_count() function supports structured transformations:

• Linelist → Count-Incidence: Aggregates by event–report date, counting only cases reported on that date.

df_linelist |>
  to_count(to = "count-incidence")
#> # A tibble:  3 × 6
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   .event_num .report_num       n .delay
#>   <date>       <date>             <dbl>       <dbl>   <int>  <dbl>
#>   [event_date] [report_date]      [...]       [...] [cases]  [...]
#> 1 2020-09-12   2020-09-12             0           0       2      0
#> 2 2020-09-12   2020-09-13             0           1       1      1
#> 3 2020-09-13   2020-09-13             1           1       3      0
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-13 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

• Linelist → Count-Cumulative: Aggregates by event–report date, producing cumulative counts up to each report date.

df_linelist |>
  to_count(to = "count-cumulative")
#> # A tibble:  3 × 6
#> # Data type: "count-cumulative"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   .event_num .report_num       n .delay
#>   <date>       <date>             <dbl>       <dbl>   <int>  <dbl>
#>   [event_date] [report_date]      [...]       [...] [cases]  [...]
#> 1 2020-09-12   2020-09-12             0           0       2      0
#> 2 2020-09-12   2020-09-13             0           1       3      1
#> 3 2020-09-13   2020-09-13             1           1       3      0
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-13 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

Note In the previous example the n counts 3 as it is aggregating the 1 observed at report_date = 2020-09-13 and the 2 observed at report_date = 2020-09-12. This is the difference between the count-incidence that specifies the ones observed exactly on that date and the count-cumulative that specifies the ones observed up until and including that date.

• Count-Incidence → Count-Cumulative: Computes cumulative sums for each event date across report dates.

df_count_inc |>
  to_count(to = "count-cumulative")
#> # A tibble:  6 × 6
#> # Data type: "count-cumulative"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   .event_num .report_num       n .delay
#>   <date>       <date>             <dbl>       <dbl>   <dbl>  <dbl>
#>   [event_date] [report_date]      [...]       [...] [cases]  [...]
#> 1 2020-09-12   2020-09-12             0           0       7      0
#> 2 2020-09-12   2020-09-13             0           1       8      1
#> 3 2020-09-12   2020-09-14             0           2      17      2
#> 4 2020-09-13   2020-09-13             1           1       5      0
#> 5 2020-09-13   2020-09-14             1           2       5      1
#> 6 2020-09-13   2020-09-15             1           3       7      2
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-15 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

• Aggregation within the same type: The to_count() may also be used to re-aggregate datasets that contain duplicate event–report pairs. This is useful when raw surveillance feeds contain repeated entries such as in this case:

df_example <- data.frame(
  n = c(8, 11, 0, 1, 1, 5, 2, 4, 1, 10, 9, 11, 3, 1),
  sex = c(rep("M", 3), rep("F", 4), rep("M", 2), rep("F", 5)),
  event_date = c(
    rep(ymd("2020/09/12"), 3),
    rep(ymd("2020/09/12"), 4),
    rep(ymd("2020/09/13"), 2),
    rep(ymd("2020/09/13"), 5)
  ),
  report_date = c(
    ymd("2020/09/12"), ymd("2020/09/13"), ymd("2020/09/14"),
    ymd("2020/09/12"), ymd("2020/09/13"), ymd("2020/09/14"),
    ymd("2020/09/15"), ymd("2020/09/13"), ymd("2020/09/14"),
    ymd("2020/09/13"), ymd("2020/09/14"),
    ymd("2020/09/15"), ymd("2020/09/16"), ymd("2020/09/17")
  )
)

tbl_example <- df_example |>
  tbl_now(
    event_date = event_date, report_date = report_date,
    data_type = "count-incidence", case_count = n, verbose = FALSE,
    warn_non_uniqueness = FALSE
  )

tbl_example
#> # A tibble:  14 × 7
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>          n sex   event_date   report_date   .event_num .report_num .delay
#>      <dbl> <chr> <date>       <date>             <dbl>       <dbl>  <dbl>
#>    [cases] [...] [event_date] [report_date]      [...]       [...]  [...]
#>  1       8 M     2020-09-12   2020-09-12             0           0      0
#>  2      11 M     2020-09-12   2020-09-13             0           1      1
#>  3       0 M     2020-09-12   2020-09-14             0           2      2
#>  4       1 F     2020-09-12   2020-09-12             0           0      0
#>  5       1 F     2020-09-12   2020-09-13             0           1      1
#>  6       5 F     2020-09-12   2020-09-14             0           2      2
#>  7       2 F     2020-09-12   2020-09-15             0           3      3
#>  8       4 M     2020-09-13   2020-09-13             1           1      0
#>  9       1 M     2020-09-13   2020-09-14             1           2      1
#> 10      10 F     2020-09-13   2020-09-13             1           1      0
#> 11       9 F     2020-09-13   2020-09-14             1           2      1
#> 12      11 F     2020-09-13   2020-09-15             1           3      2
#> 13       3 F     2020-09-13   2020-09-16             1           4      3
#> 14       1 F     2020-09-13   2020-09-17             1           5      4
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-17 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

This dataset intentionally contains repeated event_date–report_date pairs within each stratum. If the user wants to aggregate redundant rows, the to_count() function can be used to collapse duplicates by summing the case_count column.

tbl_example |>
  to_count(to = "count-incidence")
#> # A tibble:  9 × 6
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   .event_num .report_num       n .delay
#>   <date>       <date>             <dbl>       <dbl>   <dbl>  <dbl>
#>   [event_date] [report_date]      [...]       [...] [cases]  [...]
#> 1 2020-09-12   2020-09-12             0           0       9      0
#> 2 2020-09-12   2020-09-13             0           1      12      1
#> 3 2020-09-12   2020-09-14             0           2       5      2
#> 4 2020-09-12   2020-09-15             0           3       2      3
#> 5 2020-09-13   2020-09-13             1           1      14      0
#> 6 2020-09-13   2020-09-14             1           2      10      1
#> 7 2020-09-13   2020-09-15             1           3      11      2
#> 8 2020-09-13   2020-09-16             1           4       3      3
#> 9 2020-09-13   2020-09-17             1           5       1      4
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-17 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

The function ensures that:

• Rows are grouped by event_date, report_date, and any strata, and is_censored.

• The case_count column is summed within each group.

• Attributes are preserved so the resulting object remains a valid tbl_now.

Temporal effects

Often, temporal covariates improve nowcasting performance by helping adjust for systematic changes occurring within the calendar cycle (e.g., day-of-week effects, seasonal effects, or reporting artefacts). The temporal_effects() function creates a specification (recipe) of the features to compute:

library(almanac)

t_eff <- temporal_effects(
  day_of_week  = TRUE,
  week_of_year = TRUE,
  holidays     = cal_us_federal()
)
t_eff
#> 
#> ── Temporal Effects ────────────────────────────────────────────────────────────
#> The following effects are in place:
#> • "day_of_week"
#> • "week_of_year"
#> • "holidays":
#>   1. New Year's Day, US Martin Luther King Jr. Day, US Presidents' Day, US
#>   Memorial Day, US Juneteenth, US Independence Day, US Labor Day, US Indigenous
#>   Peoples' Day, US Veterans Day, US Thanksgiving, and Christmas

Note that the holidays calendar is an rcalendar object from the almanac package.

Lazy evaluation: attach first, compute later

Temporal effects in tbl.now follow a lazy evaluation pattern:

1. Attach the spec with add_temporal_effects() (or via the t_effects argument of tbl_now()). This records what should be computed but adds no columns yet.

2. Materialise the columns with compute_temporal_effects() when you are ready to use them in a model.

data("denguedat")

# Step 1 — create the tbl_now and attach the spec (no columns added yet)
df_now <- denguedat |>
  tbl_now(
    event_date = onset_week, report_date = report_week,
    verbose = FALSE, strata = gender
  )

df_now <- df_now |>
  add_temporal_effects(t_eff)

# The footer shows "T. effects (lazy): ..." — spec is recorded but not computed
df_now
#> # A tibble:  52,987 × 6
#> # Data type: "linelist"
#> # Frequency: Event: `weeks` | Report: `weeks`
#>    onset_week   report_week   gender   .event_num .report_num .delay
#>    <date>       <date>        <chr>         <dbl>       <dbl>  <dbl>
#>    [event_date] [report_date] [strata]      [...]       [...]  [...]
#>  1 1990-01-01   1990-01-01    Male              0           0      0
#>  2 1990-01-01   1990-01-01    Female            0           0      0
#>  3 1990-01-01   1990-01-01    Female            0           0      0
#>  4 1990-01-01   1990-01-08    Female            0           1      1
#>  5 1990-01-01   1990-01-08    Male              0           1      1
#>  6 1990-01-01   1990-01-15    Female            0           2      2
#>  7 1990-01-01   1990-01-15    Female            0           2      2
#>  8 1990-01-01   1990-01-15    Female            0           2      2
#>  9 1990-01-01   1990-01-22    Female            0           3      3
#> 10 1990-01-01   1990-01-08    Female            0           1      1
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2010-12-20 | Event date: "onset_week" | Report date: "report_week"
#> # Strata: "gender"
#> # T. effects (lazy): [event_date] day_of_week, week_of_year, holidays
#> # ────────────────────────────────────────────────────────────────────────────────
#> # ℹ 52,977 more rows

The footer now shows the spec with (lazy) to signal that the columns have not been computed yet. No new columns appear in the tibble at this point.

# Step 2 — materialise the columns when needed
df_computed <- compute_temporal_effects(df_now)

# Columns are now present and annotated [t_effect]
df_computed
#> # A tibble:  52,987 × 9
#> # Data type: "linelist"
#> # Frequency: Event: `weeks` | Report: `weeks`
#>    onset_week   report_week   gender   .event_num .report_num .delay
#>    <date>       <date>        <chr>         <dbl>       <dbl>  <dbl>
#>    [event_date] [report_date] [strata]      [...]       [...]  [...]
#>  1 1990-01-01   1990-01-01    Male              0           0      0
#>  2 1990-01-01   1990-01-01    Female            0           0      0
#>  3 1990-01-01   1990-01-01    Female            0           0      0
#>  4 1990-01-01   1990-01-08    Female            0           1      1
#>  5 1990-01-01   1990-01-08    Male              0           1      1
#>  6 1990-01-01   1990-01-15    Female            0           2      2
#>  7 1990-01-01   1990-01-15    Female            0           2      2
#>  8 1990-01-01   1990-01-15    Female            0           2      2
#>  9 1990-01-01   1990-01-22    Female            0           3      3
#> 10 1990-01-01   1990-01-08    Female            0           1      1
#>    .event_day_of_week .event_week_of_year .event_holiday
#>                 <int>               <int>          <int>
#>            [t_effect]          [t_effect]     [t_effect]
#>  1                  2                   1              1
#>  2                  2                   1              1
#>  3                  2                   1              1
#>  4                  2                   1              1
#>  5                  2                   1              1
#>  6                  2                   1              1
#>  7                  2                   1              1
#>  8                  2                   1              1
#>  9                  2                   1              1
#> 10                  2                   1              1
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2010-12-20 | Event date: "onset_week" | Report date: "report_week"
#> # Strata: "gender"
#> # T. effects: [event_date] day_of_week, week_of_year, holidays
#> # T. effect cols: ".event_day_of_week", ".event_week_of_year", and
#> # ".event_holiday"
#> # ────────────────────────────────────────────────────────────────────────────────
#> # ℹ 52,977 more rows

After compute_temporal_effects():

• The effect columns (.event_day_of_week, .event_week_of_year, .event_holiday) are added.
• get_temporal_effect_cols() lists them.
• The original spec remains accessible via get_temporal_effects(), so you always know which effects were requested even after further dplyr operations.

get_temporal_effects(df_computed) # The spec (list of configs)
#> [[1]]
#> [[1]]$t_effects
#> 
#> ── Temporal Effects ────────────────────────────────────────────────────────────
#> The following effects are in place:
#> • "day_of_week"
#> • "week_of_year"
#> • "holidays":
#>   1. New Year's Day, US Martin Luther King Jr. Day, US Presidents' Day, US
#>   Memorial Day, US Juneteenth, US Independence Day, US Labor Day, US Indigenous
#>   Peoples' Day, US Veterans Day, US Thanksgiving, and Christmas
#> 
#> [[1]]$date_type
#> [1] "event_date"
#> 
#> [[1]]$weekend_days
#> [1] "Sat" "Sun"
get_temporal_effect_cols(df_computed) # The computed column names
#> [1] ".event_day_of_week"  ".event_week_of_year" ".event_holiday"

Around-holiday and around-weekend effects

Reporting often rebounds on the first working day(s) after a holiday or a weekend. To capture that, temporal_effects() has holiday_lags and weekend_lags: each takes a depth N and creates indicator columns ..._holiday_lag_1 … ..._holiday_lag_N (and ..._weekend_lag_k) that flag dates falling exactly k working days after a holiday / weekend. Working days skip weekends and other holidays, so the effect lands on the first day back at work.

# Flag the two working days after a holiday, and the working day after a weekend
after_eff <- temporal_effects(
  holidays     = cal_us_federal(),
  holiday_lags = 2,
  weekend_lags = 1
)
after_eff
#> 
#> ── Temporal Effects ────────────────────────────────────────────────────────────
#> The following effects are in place:
#> • "after-holiday" effect: first 2 working days
#> • "after-weekend" effect: first working day
#> • "holidays":
#>   1. New Year's Day, US Martin Luther King Jr. Day, US Presidents' Day, US
#>   Memorial Day, US Juneteenth, US Independence Day, US Labor Day, US Indigenous
#>   Peoples' Day, US Veterans Day, US Thanksgiving, and Christmas

The mirror image — a slowdown in the days leading up to a break — is a negative depth. ..._holiday_lead_k / ..._weekend_lead_k then flag dates k working days before a holiday / weekend, counting backwards from it, so _lead_1 is the working day closest to the break:

# Flag Christmas Eve (and the eve of every other holiday), plus the Wednesday,
# Thursday and Friday before each weekend
before_eff <- temporal_effects(
  holidays     = cal_us_federal(),
  holiday_lags = -1,
  weekend_lags = -3
)
before_eff
#> 
#> ── Temporal Effects ────────────────────────────────────────────────────────────
#> The following effects are in place:
#> • "before-holiday" effect: last working day
#> • "before-weekend" effect: last 3 working days
#> • "holidays":
#>   1. New Year's Day, US Martin Luther King Jr. Day, US Presidents' Day, US
#>   Memorial Day, US Juneteenth, US Independence Day, US Labor Day, US Indigenous
#>   Peoples' Day, US Veterans Day, US Thanksgiving, and Christmas

To model both sides of the same break, attach one specification per direction:

df_now |>
  add_temporal_effects(temporal_effects(weekend_lags = -1)) |> # the Friday before
  add_temporal_effects(temporal_effects(weekend_lags = 1)) # the Monday after

Event- vs report-date effects

By default effects are derived from the event date and named .event_*. Pass date_type = "report_date" to add_temporal_effects() to derive them from the report date instead (columns named .report_*). Both can coexist on the same tbl_now, and every converter carries both sets of columns.

df_now |>
  add_temporal_effects(temporal_effects(week_of_year = TRUE), date_type = "event_date") |>
  add_temporal_effects(temporal_effects(day_of_week = TRUE),  date_type = "report_date")

Modifying a tbl_now() with dplyr

A tbl_now()’s attributes can be modified using the functions in add_*, change_*, or remove_*. These functions share a consistent interface that allows users to incrementally manipulate strata, covariates, and temporal effects.

The example below demonstrates how to create a tbl_now(), add strata and temporal effects, later modify the strata, and finally remove the temporal effects.

data("mpoxdat")

df_now <- mpoxdat |>
  tbl_now(
    event_date = dx_date, report_date = dx_report_date,
    case_count = n, verbose = FALSE, strata = race
  )

df_now
#> # A tibble:  1,417 × 7
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>    dx_date      dx_report_date race              n .event_num .report_num .delay
#>    <date>       <date>         <chr>         <int>      <dbl>       <dbl>  <dbl>
#>    [event_date] [report_date]  [strata]      [cas…      [...]       [...]  [...]
#>  1 2022-07-08   2022-07-12     Asian             4          0           4      4
#>  2 2022-07-08   2022-07-12     Black             6          0           4      4
#>  3 2022-07-08   2022-07-12     Hispanic          6          0           4      4
#>  4 2022-07-08   2022-07-12     Non-Hispanic…     6          0           4      4
#>  5 2022-07-08   2022-07-13     Asian             2          0           5      5
#>  6 2022-07-08   2022-07-13     Black             3          0           5      5
#>  7 2022-07-08   2022-07-13     Hispanic          8          0           5      5
#>  8 2022-07-08   2022-07-13     Non-Hispanic…     5          0           5      5
#>  9 2022-07-08   2022-07-14     Black             1          0           6      6
#> 10 2022-07-08   2022-07-14     Hispanic          3          0           6      6
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2023-05-19 | Event date: "dx_date" | Report date: "dx_report_date"
#> # Strata: "race"
#> # ────────────────────────────────────────────────────────────────────────────────
#> # ℹ 1,407 more rows

You can see that the strata is race with the corresponding get_*:

get_strata(df_now)
#> [1] "race"

Strata can be modified with the change_* family of functions. The following example adds a new column containing an uppercase version of the existing race variable and sets it as the new strata:

df_now <- df_now |>
  mutate(RACE_UPPER = toupper(race)) |>
  change_strata(RACE_UPPER)

get_strata(df_now)
#> [1] "RACE_UPPER"

To attach a lazy temporal-effects spec, use add_temporal_effects(), then materialise with compute_temporal_effects():

df_now <- df_now |>
  add_temporal_effects(temporal_effects(week_of_year = TRUE))

# Spec is stored (lazy):
get_temporal_effects(df_now)
#> [[1]]
#> [[1]]$t_effects
#> 
#> ── Temporal Effects ────────────────────────────────────────────────────────────
#> The following effects are in place:
#> • "week_of_year"
#> 
#> [[1]]$date_type
#> [1] "event_date"
#> 
#> [[1]]$weekend_days
#> [1] "Sat" "Sun"

# Compute to add columns:
df_now <- compute_temporal_effects(df_now)
get_temporal_effect_cols(df_now)
#> [1] ".event_week_of_year"

Attributes can be removed using the corresponding remove_* functions. remove_temporal_effects() drops both the spec and any computed columns:

df_now <- df_now |>
  remove_temporal_effects() |>
  remove_all_strata()
#> Warning: *Non-unique*: Data has multiple rows for the same event (dx_date) and
#> report(dx_report_date) dates. Consider using `to_count()` to aggregate the data
#> or`distinct()` to remove repeated observations.

df_now
#> # A tibble:  1,417 × 8
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>    dx_date      dx_report_date race                     n .event_num .report_num
#>    <date>       <date>         <chr>                <int>      <dbl>       <dbl>
#>    [event_date] [report_date]  [...]              [cases]      [...]       [...]
#>  1 2022-07-08   2022-07-12     Asian                    4          0           4
#>  2 2022-07-08   2022-07-12     Black                    6          0           4
#>  3 2022-07-08   2022-07-12     Hispanic                 6          0           4
#>  4 2022-07-08   2022-07-12     Non-Hispanic White       6          0           4
#>  5 2022-07-08   2022-07-13     Asian                    2          0           5
#>  6 2022-07-08   2022-07-13     Black                    3          0           5
#>  7 2022-07-08   2022-07-13     Hispanic                 8          0           5
#>  8 2022-07-08   2022-07-13     Non-Hispanic White       5          0           5
#>  9 2022-07-08   2022-07-14     Black                    1          0           6
#> 10 2022-07-08   2022-07-14     Hispanic                 3          0           6
#>    .delay RACE_UPPER        
#>     <dbl> <chr>             
#>     [...] [...]             
#>  1      4 ASIAN             
#>  2      4 BLACK             
#>  3      4 HISPANIC          
#>  4      4 NON-HISPANIC WHITE
#>  5      5 ASIAN             
#>  6      5 BLACK             
#>  7      5 HISPANIC          
#>  8      5 NON-HISPANIC WHITE
#>  9      6 BLACK             
#> 10      6 HISPANIC          
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2023-05-19 | Event date: "dx_date" | Report date: "dx_report_date"
#> # ────────────────────────────────────────────────────────────────────────────────
#> # ℹ 1,407 more rows

get_temporal_effects(df_now) # Empty list — no spec
#> list()
get_temporal_effect_cols(df_now) # character(0) — no computed cols
#> character(0)
get_strata(df_now)
#> NULL

Modifying a tbl_now() with dplyr

tbl_now() objects extend tibble(), and therefore support standard dplyr verbs. The class attempts to preserve and adapt its internal attributes when operations are performed. For example, renaming a strata column will automatically update the stored strata attribute.

library(dplyr, quietly = TRUE)
data(denguedat)

df_now <- tbl_now(denguedat,
  event_date = onset_week,
  report_date = report_week, strata = gender,
  verbose = FALSE
)

# Current strata
get_strata(df_now)
#> [1] "gender"

After renaming the column, the strata attribute updates accordingly:

df_now <- df_now |>
  rename(male_or_female = gender)

get_strata(df_now)
#> [1] "male_or_female"

Certain operations may cause a tbl_now() object to drop back to a standard tibble. This occurs when the operation removes necessary structure—for example, collapsing all data into a single row with summarise():

df_now |>
  summarise(number_males = sum(male_or_female == "Male"))
#> Warning: Dropping `tbl_now` attributes and converting to `tibble`
#> # A tibble: 1 × 1
#>   number_males
#>          <int>
#> 1        26395

Updating a tbl_now()

A tbl_now() object can be updated using the update() method with another data.frame, tibble, or tbl_now() as input. When the column structure is compatible, the update process retains the strata, covariate, and temporal-effect attributes from the original object, and recalculates “now” estimates using the combined data.

Below is an example using an initial dataset:

df <- data.frame(
  patient = 1:6,
  event_date = c(rep(ymd("2020/09/12"), 3), rep(ymd("2020/09/13"), 3)),
  report_date = c(
    ymd("2020/09/12"), ymd("2020/09/13"), ymd("2020/09/14"),
    ymd("2020/09/13"), ymd("2020/09/14"), ymd("2020/09/15")
  )
)

df_now <- tbl_now(df,
  event_date = event_date,
  report_date = report_date, verbose = FALSE
)

And a follow-up dataset containing newly reported cases:

df_new <- data.frame(
  patient = 7:13,
  event_date = c(
    ymd("2020/09/13"),
    rep(ymd("2020/09/14"), 3),
    rep(ymd("2020/09/15"), 3)
  ),
  report_date = c(
    ymd("2020/09/14"), ymd("2020/09/14"), ymd("2020/09/15"),
    ymd("2020/09/16"), ymd("2020/09/15"), ymd("2020/09/16"),
    ymd("2020/09/17")
  )
)

We can update the original object by incorporating the new data:

df_updated <- update(df_now, new_data = df_new)

df_updated
#> # A tibble:  13 × 6
#> # Data type: "linelist"
#> # Frequency: Event: `days` | Report: `days`
#>    patient event_date   report_date   .event_num .report_num .delay
#>      <int> <date>       <date>             <dbl>       <dbl>  <dbl>
#>      [...] [event_date] [report_date]      [...]       [...]  [...]
#>  1       1 2020-09-12   2020-09-12             0           0      0
#>  2       2 2020-09-12   2020-09-13             0           1      1
#>  3       3 2020-09-12   2020-09-14             0           2      2
#>  4       4 2020-09-13   2020-09-13             1           1      0
#>  5       5 2020-09-13   2020-09-14             1           2      1
#>  6       6 2020-09-13   2020-09-15             1           3      2
#>  7       7 2020-09-13   2020-09-14             1           2      1
#>  8       8 2020-09-14   2020-09-14             2           2      0
#>  9       9 2020-09-14   2020-09-15             2           3      1
#> 10      10 2020-09-14   2020-09-16             2           4      2
#> 11      11 2020-09-15   2020-09-15             3           3      0
#> 12      12 2020-09-15   2020-09-16             3           4      1
#> 13      13 2020-09-15   2020-09-17             3           5      2
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-17 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

Visualizing a tbl_now

The autoplot() method gives a quick diagnostic overview of a tbl_now. Using ggplot2 and patchwork, it draws two families of panels. The case-count panels:

1. Empirical delay distribution: a case-count weighted histogram of the reporting delay.
2. Observed epidemic process: the latest reported counts per event_date.
3. Calendar effect: boxplots of the normalized effect (each event date’s cases divided by the overall mean, so 1 is average) by day of week / week of year / month.
4. Holiday effect and holiday lag effect: the same normalized boxplots by day type and by position relative to the nearest holiday — see Holiday effects below.
5. Cycles — a periodogram of the incidence series whose dominant peak suggests a Fourier season length to pass to temporal_effects().

…and the reporting-delay panels, which reveal delay effects (whether the delay itself has a calendar or periodic pattern):

6. Delay calendar effect: boxplots of the mean reporting delay by day of week / week of year / month.
7. Delay holiday effects: the delay twins of panel 4.
8. Cycles (reporting): a periodogram of the mean-delay series (e.g. a weekly reporting rhythm).

The two families are colour-coded the same way everywhere in the package: red for anything reporting-related and green for the epidemic (event-date) process. Every panel also names its process in the subtitle — “Reporting delay process” or “Epidemic (event-date) process” — so a panel lifted out of the grid still says what it is.

Which calendar/delay panels appear depends on the event unit (daily data gets day-of-week and week-of-year; weekly data week-of-year; monthly data month-of-year). Use the panels argument to pick a subset — for example panels = "delay_calendar" for just the delay boxplots, or a single key like panels = "delay_week" (a single panel is returned as a plain ggplot). The aliases are "all" (default), "calendar" and "delay_calendar".

Set by_strata = TRUE to split every panel by stratum: the boxplots become dodged boxes (one per stratum, side by side), the epidemic process and periodograms become one coloured line per stratum, and the delay distribution becomes dodged bars. The boxplots are then normalized per stratum (1 = that stratum’s own average) so the calendar pattern is comparable across strata. By default the object’s strata are used; pass strata = "gender" to group on a subset.

One panel at a time: the plot_*() functions

autoplot() draws the whole grid in one call. When you want a single effect on its own — in its own figure, at its own size — every panel also has a standalone plot_*() twin. They take the same object and return the identical plot, so autoplot(x, panels = "calendar_weekday") and plot_day_of_week_effects(x) are the same figure. Use autoplot() for the overview; reach for a plot_*() when you want to place one effect in a report.

Each calendar/holiday twin takes a type argument choosing the process: type = "epidemic" (the default, green — how the cases vary) or type = "report" (red — how the reporting does).

Function	autoplot() panel
plot_day_of_week_effects(x, type = )	calendar_weekday / delay_weekday
plot_week_of_year_effects(x, type = )	calendar_week / delay_week
plot_month_of_year_effects(x, type = )	calendar_month / delay_month
plot_holiday_effects(x, type = )	calendar_holiday / delay_holiday
plot_holiday_lag_effects(x, type = )	calendar_holiday_lag / delay_holiday_lag
plot_cycles(x, type = )	seasonality / delay_seasonality
plot_delay_distribution(x)	delay_distribution
plot_observed_cases(x)	epidemic

# The same figure, two ways:
autoplot(dengue_now, panels = "calendar_week")
plot_week_of_year_effects(dengue_now)

# The reporting twin (red):
plot_week_of_year_effects(dengue_now, type = "report")

Normalized vs. percentage effects

The day-of-week, week-of-year, month-of-year, holiday and weekend/holiday panels take a measure argument (in autoplot() and in every plot_*() twin):

• measure = "normalized" (the default) is the view shown above: each value divided by its overall mean, so the dashed line at 1 marks an average level.
• measure = "percent" reads as a share instead: the percentage of cases that fall in each group, with the IQR around it. So a weekend box might sit at 10% of cases at the weekend versus 90% on weekdays. The shares add up over a natural block — the seven weekdays within a week, the epidemiological weeks within a year — and the reporting (type = "report") version shares out the reports by report date, answering “what share of the reports arrive at the weekend?”. It needs Date event/report columns.

# What share of the reports arrive on each weekday?
plot_day_of_week_effects(dengue_now, type = "report", measure = "percent")

The strata argument does not have to name a declared stratum — any column of the data works, and it is declared for you, so these two are the same plot:

autoplot(mpox_now, strata = "race", by_strata = TRUE)
autoplot(mpox_now |> add_strata(race), by_strata = TRUE)

library(ggplot2)
library(patchwork)

dengue_now <- tbl_now(denguedat,
  event_date  = "onset_week",
  report_date = "report_week",
  verbose     = FALSE
)

autoplot(dengue_now)

For the weekly dengue data the periodogram peaks near 52 weeks, suggesting an annual cycle. We could capture this with a Fourier term of temporal_effects(seasons = 52).

Holiday effects

Note. This section was written by AI.

The four holiday panels are different from the rest: they describe the temporal_effects() spec you attached, so they appear only once there is one to describe. calendar_holiday / delay_holiday need a holidays calendar or weekend = TRUE; the two _lag panels also need a non-zero holiday_lags. The spec is read directly, so there is no need to call compute_temporal_effects() first.

Holidays are a daily phenomenon, so we switch to daily data — here a synthetic daily series, reports, whose reporting deliberately stalls on weekends and holidays. Attaching the spec is all it takes:

holiday_now <- tbl_now(reports,
    event_date = onset, report_date = reported, case_count = n,
    data_type = "count-incidence", verbose = FALSE
  ) |>
  add_temporal_effects(
    temporal_effects(weekend = TRUE, holidays = cal_us_federal(), holiday_lags = 2)
  )

The holiday effect panel splits the days by type. Which categories you get follows the spec: a calendar plus weekend = TRUE gives Weekday / Weekend / Holiday; a calendar alone gives Non-holiday / Holiday; a weekend effect alone gives Weekday / Weekend. A holiday that falls on a weekend counts as a holiday.

The holiday lag effect panel splits them by position relative to the nearest holiday, showing exactly the days that holiday_lags flags — "1 after", "2 after", and so on, with "Other" (every other day) as the reference to read them against. Negative depths add "1 before", "2 before", … to the left of the holiday, so the axis reads left-to-right as time does. Counting is in working days, so weekends and other holidays are skipped exactly as they are for the ..._holiday_lag_k columns.

autoplot(holiday_now, panels = c("calendar_holiday", "calendar_holiday_lag",
                                 "delay_holiday", "delay_holiday_lag"))

The delay twins (the red panels) are usually the more telling pair. A holiday rarely changes how many cases occur — but it very much changes how long they take to be reported, which is exactly what a nowcast has to model. Here the planted effect comes straight back out: the mean delay is well above average on holidays, above it at weekends, and below it on ordinary working days.

Use these panels to check the spec before you model with it. If the "1 after" box sits on the dashed line at 1, that lag is not doing anything and you can drop it; if "2 before" is the box that moves, your reporting slows down ahead of the break and you want a negative holiday_lags (see ?temporal_effects).

Do delay distributions drift over time?

Note. This section was written by AI. For a human-written treatment of reporting-delay artefacts, see the batch-reporting article.

Reporting delays are not always stable: a surveillance system may speed up or slow down over a season or across years. plot_delay_drift() draws a rolling fan chart of the delay distribution indexed by event date — a solid rolling median, a dashed rolling mean, and 25–75% / 10–90% quantile bands. A rising or falling centre line signals location drift; widening or narrowing bands signal spread drift.

plot_delay_drift(dengue_now)

Because the most recent event dates have not had time to be fully reported, their delays look artificially short; that immature region (after the completeness cutoff) is shaded grey and should not be read as drift.

For a formal answer, test_delay_drift() runs an autocorrelation-robust monotonic-trend test (a modified Mann–Kendall test, via the modifiedmk package) on the per-period delay summaries — a delay series is correlated with itself, so an ordinary trend test would over-state significance. It tests both a location statistic (the median) and a dispersion statistic (the 10–90 spread), on mature data only:

test_delay_drift(dengue_now, stat = c("median", "spread"))
#> Warning: ! `test_delay_drift()` is experimental: results are not guaranteed and the
#>   interface may change.
#> ℹ Interpret a significant result as a potential trend change, not a confirmed
#>   one.
#> This warning is displayed once every 8 hours.
#> # A tibble: 2 × 9
#>   strata stat       n      tau sens_slope statistic p_value method    drift
#>   <chr>  <chr>  <int>    <dbl>      <dbl>     <dbl>   <dbl> <chr>     <lgl>
#> 1 all    median  1090 -0.00918          0    -0.243  0.808  hamed-rao FALSE
#> 2 all    spread  1090 -0.178            0    -2.32   0.0203 hamed-rao TRUE

A significant drift for spread with a non-significant one for median, say, would mean the typical delay held steady while its variability changed.

The trend test looks for gradual change. If instead you suspect an abrupt shift — a reporting-system change on some date — use test_delay_changepoint(), which applies Pettitt’s nonparametric change-point test and returns the estimated change date together with the before/after delay level:

test_delay_changepoint(dengue_now, stat = c("median", "spread"))
#> Warning: ! `test_delay_changepoint()` is experimental: results are not guaranteed and
#>   the interface may change.
#> ℹ Treat a detected change as a potential change point, not a confirmed one.
#> This warning is displayed once every 8 hours.
#> # A tibble: 2 × 10
#>   strata stat       n changepoint statistic  p_value before after  shift
#>   <chr>  <chr>  <int> <date>          <dbl>    <dbl>  <dbl> <dbl>  <dbl>
#> 1 all    median  1090 1998-01-19      38402 2.17e- 3   1.53  1.39 -0.136
#> 2 all    spread  1090 1997-09-01      93409 5.77e-18   2.55  1.91 -0.639
#> # ℹ 1 more variable: changepoint_detected <lgl>

You can mark the detected change point on the fan chart with plot_delay_drift(dengue_now, changepoint = TRUE).

Detecting batch reporting

Note. This section was written by AI. For a human-written treatment of batch reporting, see the batch-reporting article.

Laboratories sometimes withhold results and then release a whole backlog at once — a batch. The key idea is that a batch moves reports along the report axis without creating them, so a window of report dates spanning both the lull and the spike has an unchanged total — whereas a genuine epidemic surge adds cases and inflates it. batch_test() turns this into a per-report-date diagnostic that separates the two:

batches <- batch_test(dengue_now, lookback = 2)

batches |>
  filter(batch) |>
  select(report_date, reported, baseline, everything())
#> # A tibble: 4 × 9
#>   report_date reported baseline stratum deficit  delta p_transport
#>   <date>         <dbl>    <dbl> <chr>     <dbl>  <dbl>       <dbl>
#> 1 1991-08-12        61     50.2 all        37.3 -26.6    0.00200  
#> 2 2007-11-26       152     86.5 all        68.0  -2.50   0.000526 
#> 3 2009-11-16       127     83.5 all        60.5 -17      0.000347 
#> 4 2010-09-13       383    272.  all       132.  -21.0    0.0000639
#> # ℹ 2 more variables: p_transport_bh <dbl>, batch <lgl>

Additional information on dealing with batches and other reporting delay artifacts can be found in the corresponding vignette.

Other functions (utilities)

Convert epidemiological weeks to dates

The function week_2_date() converts epidemiological week/year combinations into an aligned calendar date.

df <- data.frame(
  epidemiological_week = 1:5,
  epidemiological_year = rep(2024, 5)
)

df |>
  week_2_date(
    week_col = epidemiological_week,
    year_col = epidemiological_year
  )
#>   epidemiological_week epidemiological_year       date
#> 1                    1                 2024 2023-12-31
#> 2                    2                 2024 2024-01-07
#> 3                    3                 2024 2024-01-14
#> 4                    4                 2024 2024-01-21
#> 5                    5                 2024 2024-01-28

Reports

The functions get_initial_reported_cases() and get_latest_reported_cases() extract the number of cases first reported for each event date and the most recently reported totals, respectively. These utilities allow users to quantify revisions between initial and final reports.

df_reports <- data.frame(
  n = c(10, 1, 1, 0, 0, 3),
  event_date = rep(ymd("2020/09/12"), 6),
  report_date = c(
    ymd("2020/09/12"), ymd("2020/09/13"), ymd("2020/09/14"),
    ymd("2020/09/15"), ymd("2020/09/16"), ymd("2020/09/17")
  )
)

tbl_reports <- df_reports |>
  tbl_now(
    event_date = event_date, report_date = report_date,
    verbose = FALSE, case_count = n, report_units = "days",
    event_units = "days"
  )

tbl_reports
#> # A tibble:  6 × 6
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>         n event_date   report_date   .event_num .report_num .delay
#>     <dbl> <date>       <date>             <dbl>       <dbl>  <dbl>
#>   [cases] [event_date] [report_date]      [...]       [...]  [...]
#> 1      10 2020-09-12   2020-09-12             0           0      0
#> 2       1 2020-09-12   2020-09-13             0           1      1
#> 3       1 2020-09-12   2020-09-14             0           2      2
#> 4       0 2020-09-12   2020-09-15             0           3      3
#> 5       0 2020-09-12   2020-09-16             0           4      4
#> 6       3 2020-09-12   2020-09-17             0           5      5
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-17 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

The initial reported cases:

get_initial_reported_cases(tbl_reports)
#> # A tibble:  1 × 6
#> # Data type: "count-cumulative"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   .event_num .report_num       n .delay
#>   <date>       <date>             <dbl>       <dbl>   <dbl>  <dbl>
#>   [event_date] [report_date]      [...]       [...] [cases]  [...]
#> 1 2020-09-12   2020-09-12             0           0      10      0
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-17 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

and the latest totals:

get_latest_reported_cases(tbl_reports)
#> # A tibble:  1 × 6
#> # Data type: "count-cumulative"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   .event_num .report_num       n .delay
#>   <date>       <date>             <dbl>       <dbl>   <dbl>  <dbl>
#>   [event_date] [report_date]      [...]       [...] [cases]  [...]
#> 1 2020-09-12   2020-09-17             0           5      15      5
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-09-17 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

Week alignment

The align_weeks() function standardizes dates within the same epidemiological week to a single reference day (for example, the start of the week). This is helpful when computing differences across weekly reporting periods, avoiding fractional time intervals.

df <- data.frame(
  date = c(ymd("2022-10-31"), ymd("2022-11-07"), ymd("2022-11-13")),
  epiweek = c(44, 45, 46)
)

# Align to Sundays
df_aligned <- align_weeks(df, date_col = date)
df_aligned
#>         date epiweek date_aligned
#> 1 2022-10-31      44   2022-10-30
#> 2 2022-11-07      45   2022-11-06
#> 3 2022-11-13      46   2022-11-13

You can verify the resulting weekday using the wday() function from the lubridate package:

df_aligned |>
  mutate(day_label = wday(date_aligned, label = TRUE, abbr = FALSE))
#>         date epiweek date_aligned day_label
#> 1 2022-10-31      44   2022-10-30    Sunday
#> 2 2022-11-07      45   2022-11-06    Sunday
#> 3 2022-11-13      46   2022-11-13    Sunday

Complete zeroes

The complete_zeroes() function fills with zeroes those cases where the event or report weeks have not been observed.

Consider for example the following data with just two observations per date:

ndata <- tibble(
  event_date = c(as.Date("2021/01/12"), as.Date("2021/01/14"), as.Date("2021/01/14")),
  report_date = c(as.Date("2021/01/13"), as.Date("2021/01/15"), as.Date("2021/01/18")),
  case_count = c(10, 5, 1)
)

ndata <- tbl_now(ndata, event_date, report_date,
  verbose = FALSE, case_count = case_count, data_type = "count-incidence"
)

ndata
#> # A tibble:  3 × 6
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>   event_date   report_date   case_count .event_num .report_num .delay
#>   <date>       <date>             <dbl>      <dbl>       <dbl>  <dbl>
#>   [event_date] [report_date]    [cases]      [...]       [...]  [...]
#> 1 2021-01-12   2021-01-13            10          0           1      1
#> 2 2021-01-14   2021-01-15             5          2           3      1
#> 3 2021-01-14   2021-01-18             1          2           6      4
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2021-01-18 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

Notice that there are no observations for 2021/01/13. Furthermore, if we assume that the maximum possible observed delay is of 4, we can fill the unobserved cases with:

complete_zeroes(ndata, max_delay = 4)
#> # A tibble:  14 × 6
#> # Data type: "count-incidence"
#> # Frequency: Event: `days` | Report: `days`
#>    event_date   report_date   case_count .event_num .report_num .delay
#>    <date>       <date>             <dbl>      <int>       <dbl>  <dbl>
#>    [event_date] [report_date]    [cases]      [...]       [...]  [...]
#>  1 2021-01-12   2021-01-13            10          0           1      1
#>  2 2021-01-14   2021-01-15             5          2           3      1
#>  3 2021-01-12   2021-01-12             0          0           0      0
#>  4 2021-01-12   2021-01-14             0          0           2      2
#>  5 2021-01-12   2021-01-15             0          0           3      3
#>  6 2021-01-12   2021-01-16             0          0           4      4
#>  7 2021-01-13   2021-01-13             0          1           1      0
#>  8 2021-01-13   2021-01-14             0          1           2      1
#>  9 2021-01-13   2021-01-15             0          1           3      2
#> 10 2021-01-13   2021-01-16             0          1           4      3
#> 11 2021-01-13   2021-01-17             0          1           5      4
#> 12 2021-01-14   2021-01-14             0          2           2      0
#> 13 2021-01-14   2021-01-16             0          2           4      2
#> 14 2021-01-14   2021-01-17             0          2           5      3
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2021-01-18 | Event date: "event_date" | Report date: "report_date"
#> # ────────────────────────────────────────────────────────────────────────────────

Which looks at all the possible report dates and event dates and sets the counts to zero if they have not been observed.

Censoring extreme delays

The function censor_delays_above() marks all delays above a threshold value (max_delay) as censored. This is useful to indicate extreme delays in some nowcast models:

df <- data.frame(
  onset = as.Date("2020-01-01") + c(0, 0, 1, 2),
  reported = as.Date("2020-01-01") + c(1, 5, 2, 300)
)
tn <- tbl_now(df,
  event_date = onset, report_date = reported,
  data_type = "linelist", verbose = FALSE
)

# the 300-day report becomes censored (an upper bound on its delay)
censor_delays_above(tn, max_delay = 60)
#> ℹ Marked 1 report with delay > 60 days as censored.
#> • This delay is now an upper bound (is_censored).
#> # A tibble:  4 × 6
#> # Data type: "linelist"
#> # Frequency: Event: `days` | Report: `days`
#>   onset        reported      .event_num .report_num .delay .is_censored 
#>   <date>       <date>             <dbl>       <dbl>  <dbl> <lgl>        
#>   [event_date] [report_date]      [...]       [...]  [...] [is_censored]
#> 1 2020-01-01   2020-01-02             0           1      1 FALSE        
#> 2 2020-01-01   2020-01-06             0           5      5 FALSE        
#> 3 2020-01-02   2020-01-03             1           2      1 FALSE        
#> 4 2020-01-03   2020-10-27             2         300    298 TRUE         
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2020-10-27 | Event date: "onset" | Report date: "reported"
#> # Right-censored indicator: ".is_censored"
#> # ────────────────────────────────────────────────────────────────────────────────

Converting to data formats from other packages

NOTE This is still work in progress

tbl.now ships converters that move data between a tbl_now and the data structures used by other nowcasting and delay-estimation packages. They all follow the same naming convention:

• tbl_now_from_*() builds a tbl_now (it wraps as_tbl_now(), so ... is forwarded to tbl_now()).

• tbl_now_to_*() converts a tbl_now into that package’s native object.

Each function accepts a verbose argument that reports the choices it made (the inferred now, the data type, the units, the column mapping, and so on). The supported packages — epinowcast, baselinenowcast, EpiNow2, epidist, data.table and tsibble — are all optional (Suggests).

Package	from	to	Maps to
epinowcast	yes	yes	count-cumulative (reference_date, report_date, confirm); from also accepts a preprocessed enw_preprocess_data or fitted object
baselinenowcast	yes	yes	count-incidence (long data frame or reporting-triangle matrix)
EpiNow2	—	yes	a single date/confirm series for estimate_infections(), or (with model = "estimate_truncation") a list of report-date snapshots for estimate_truncation()
data.table	yes	yes	any data type
epidist	yes	yes	linelist (primary/secondary event dates)
tsibble	yes	yes	a tbl_ts (one date as index, the other date + strata as key)

The object produced by any tbl_now_to_*() can be converted straight back with as_tbl_now(), which has methods for each of these classes:

dengue_now <- tbl_now(denguedat,
  event_date = "onset_week",
  report_date = "report_week", strata = "gender",
  verbose = FALSE
)

# tbl_now -> tsibble -> tbl_now
dengue_ts <- tbl_now_to_tsibble(dengue_now, verbose = FALSE)
#> Warning: tsibble requires unique index/key rows; aggregating linelist to
#> "count-incidence" with `to_count()`.
as_tbl_now(dengue_ts, report_date = "report_week", verbose = FALSE)
#> # A tibble:  8,265 × 7
#> # Data type: "linelist"
#> # Frequency: Event: `weeks` | Report: `weeks`
#>    onset_week   report_week   gender       n .event_num .report_num .delay
#>    <date>       <date>        <chr>    <int>      <dbl>       <dbl>  <dbl>
#>    [event_date] [report_date] [strata] [...]      [...]       [...]  [...]
#>  1 1990-01-01   1990-01-01    Female       2          0           0      0
#>  2 1990-01-01   1990-01-01    Male         1          0           0      0
#>  3 1990-01-01   1990-01-08    Female      13          0           1      1
#>  4 1990-01-08   1990-01-08    Female       1          1           1      0
#>  5 1990-01-01   1990-01-08    Male        11          0           1      1
#>  6 1990-01-08   1990-01-08    Male         1          1           1      0
#>  7 1990-01-01   1990-01-15    Female      16          0           2      2
#>  8 1990-01-08   1990-01-15    Female      17          1           2      1
#>  9 1990-01-15   1990-01-15    Female       2          2           2      0
#> 10 1990-01-01   1990-01-15    Male         7          0           2      2
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2010-12-20 | Event date: "onset_week" | Report date: "report_week"
#> # Strata: "gender"
#> # ────────────────────────────────────────────────────────────────────────────────
#> # ℹ 8,255 more rows

For example, the epinowcast hospitalization data ships in long cumulative format and converts straight into a count-cumulative tbl_now:

library(epinowcast)
data(germany_covid19_hosp)


#Read data from epinowcast
obs  <- germany_covid19_hosp[location == "DE"]

#Remove unused column
obs  <- obs[, location := NULL]

#Pre-process data
pobs <- epinowcast::enw_preprocess_data(obs, max_delay = 40, by = "age_group")

hospitalizations_now <- tbl_now_from_epinowcast(pobs)
  
hospitalizations_now
#> # A tibble:  48,279 × 7
#> # Data type: "count-cumulative"
#> # Frequency: Event: `days` | Report: `days`
#>    reference_date report_date   confirm age_group .event_num .report_num .delay
#>    <date>         <date>          <int> <fct>          <dbl>       <dbl>  <dbl>
#>    [event_date]   [report_date] [cases] [strata]       [...]       [...]  [...]
#>  1 2021-04-06     2021-04-06        149 00+                0           0      0
#>  2 2021-04-06     2021-04-07        289 00+                0           1      1
#>  3 2021-04-06     2021-04-08        350 00+                0           2      2
#>  4 2021-04-06     2021-04-09        402 00+                0           3      3
#>  5 2021-04-06     2021-04-10        438 00+                0           4      4
#>  6 2021-04-06     2021-04-11        446 00+                0           5      5
#>  7 2021-04-06     2021-04-12        456 00+                0           6      6
#>  8 2021-04-06     2021-04-13        468 00+                0           7      7
#>  9 2021-04-06     2021-04-14        493 00+                0           8      8
#> 10 2021-04-06     2021-04-15        508 00+                0           9      9
#> # ────────────────────────────────────────────────────────────────────────────────
#> # Now: 2021-10-20 | Event date: "reference_date" | Report date: "report_date"
#> # Strata: "age_group"
#> # ────────────────────────────────────────────────────────────────────────────────
#> # ℹ 48,269 more rows

Going the other way, tbl_now_to_epinowcast() returns an enw_preprocess_data object ready for epinowcast::epinowcast():

epinowcast_object <- tbl_now_to_epinowcast(hospitalizations_now, verbose = FALSE)
#> Warning: ! Converting a <tbl_now> to epinowcast is lossy: the result is not guaranteed
#>   to be identical to a native epinowcast object.
#> ℹ Some information might be dropped (e.g. covariate columns, maximum delays,
#>   grouping indices and padding rows).
#> ℹ If you have the original data as a `tibble`, `data.frame` or `data.table`,
#>   prefer using that directly over converting from another format.
#> ℹ Silence this warning with `quiet = TRUE`.
epinowcast_object
#> ── Preprocessed nowcast data ─────────────────────────────────────────────────── 
#> Groups: 7 (age_group) | Timestep: day | Max delay: 40 
#> Observations: 198 timepoints x 1386 snapshots 
#> Max date: 2021-10-20 
#> 
#> Datasets (access with `enw_get_data(x, "<name>")`): 
#>   obs                :  49,980 x 8 
#>   new_confirm        :  49,980 x 10 
#>   latest             :   1,386 x 9 
#>   missing_reference  :       0 x 5 
#>   reporting_triangle :   1,386 x 42 
#>   metareference      :   1,386 x 8 
#>   metareport         :   1,659 x 11 
#>   metadelay          :      40 x 5

The same pattern works for the other packages.

References

Dancho, Matt, and Davis Vaughan. 2023. Timetk: A Tool Kit for Working with Time Series. https://doi.org/10.32614/CRAN.package.timetk.

Gelman, Andrew, Aki Vehtari, Daniel Simpson, et al. 2020. “Bayesian Workflow.” arXiv Preprint arXiv:2011.01808.

Wang, Earo, Dianne Cook, and Rob J Hyndman. 2020. “A New Tidy Data Structure to Support Exploration and Modeling of Temporal Data.” Journal of Computational and Graphical Statistics 29 (3): 466–78.

Wang, Yiru. 2019. “Tidy Tools for Supporting Fluent Workflow in Temporal Data Analysis.” PhD thesis, Monash University.

Wickham, Hadley. 2014. “Tidy Data.” Journal of Statistical Software 59: 1–23.

Wickham, Hadley, Mara Averick, Jennifer Bryan, et al. 2019. “Welcome to the tidyverse.” Journal of Open Source Software 4 (43): 1686. https://doi.org/10.21105/joss.01686.

Wickham, Hadley, Mine Çetinkaya-Rundel, and Garrett Grolemund. 2023. R for Data Science: Import, Tidy, Transform, Visualize, and Model Data. O’Reilly Media, Inc.