
Changelog
tbl.now 0.31.0
Breaking: the *_confirmed() counters are gone, replaced by a validated-cases family (#64)
get_latest_confirmed(), get_net_confirmed(), get_initial_confirmed() and get_nth_confirmed() are removed. They answered a version of the question get_latest_reported_cases() already answered, in a different return shape (a plain tibble), with a delay measured from a different anchor – so the two families could not be read against each other.
In their place, the reporting getters have an exact twin on the validation axis:
get_initial_validated_cases(x) # as of the FIRST result back
get_latest_validated_cases(x) # everything settled so far
get_nth_validated_cases(x, delay = 7) # settled within 7 periods
get_latest_validated_cases(x, type = "confirmed") # was get_latest_confirmed()
get_latest_validated_cases(x, type = "net") # was get_net_confirmed()
get_latest_validated_cases(x, type = "by_type") # every outcome, side by side- They return the same
count-cumulativetbl_nowthe reporting getters return, carrying all three dates and the generated numeric columns, rather than a bare tibble. -
type =is new on both families, so the reporting axis can be filtered the same way:"total"(default),"confirmed","retracted","pending","unknown","net", or"by_type"for one row per outcome. On an object with no validation process anything but"total"warns and pools. -
get_nth_validated_cases()counts the delay from the event, so it andget_nth_reported_cases()describe the same period.get_nth_confirmed()measured from the report, which is.validation_delay– a different quantity. Reading the old and new numbers as the same thing is the one migration hazard. - A pending case has no validation date, so it never appears on the validation axis;
type = "pending"is refused there and belongs on the reporting axis. - An empty selection – nothing validated yet, no case with that outcome, no arrival within the delay – is an error naming the reason, rather than a failure inside
tbl_now()about an empty data frame.
The reported-cases getters respect a grouping; to_count() says it does not (#61)
get_latest_reported_cases(), get_initial_reported_cases() and get_nth_reported_cases() (and the three new validated ones) now keep the caller’s grouping and answer by it: the grouping columns join the event date and the strata as keys, and come back on the result.
tn |> dplyr::group_by(hospital) |> get_latest_reported_cases()This is the only way to ask for a count by a covariate – a column that matters without being something you nowcast by. These verbs can do it because they select a point in the process rather than reshaping the object.
to_count() cannot, and now warns rather than dropping the grouping in silence: after aggregating, one row is an (event, report) cell rather than one of the rows that were grouped, so the grouping describes nothing that is left. Declare the column with add_strata() or add_covariates() to keep it out of the sum.
is_tbl_now() is a class check again, not a validation run (#62)
is_tbl_now() used to call validate_tbl_now() inside a tryCatch() that caught errors but not warnings, so the object’s findings escaped from wherever the predicate was called – which is every .assert_tbl_now() in the package. A verb that fixed a problem warned about it twice, after the fix.
It is now a structural check: the class, the attributes a tbl_now cannot do without, and the columns those attributes name. Cheap, and silent.
-
tbl_now_can_reconstruct()suppresses warnings while asking its hypothetical. - An object can be a
tbl_nowand still have datavalidate_tbl_now()warns about. That is the point: the class is a container, and a container is not a claim that what is in it is clean.
Fractional delays are refused where they are created, and reported where they are found (#63)
A calendar has no half-days, so a fractional delay had to become something. It became round() – round-half-to-even, so 2.5 went down and 3.5 went up, silently – while the numeric axis refused the same value outright.
-
censor_reporting_delays(to_delay =),censor_validation_delays(to_delay =)andtbl_now(delay =)now abort on a delay that is not a whole number of the axis’s units, on every axis. Round it yourself if that is what you mean. -
validate_tbl_now()warns when an object’s.delayis fractional; it was adiagnose()-only note. The remaining way in is two date columns on different weekday grids, which is exactly whatalign_weeks()fixes – so this stays a warning rather than an error, and the object you need to hand toalign_weeks()can still be built.
tbl.now 0.30.0
New: coarsen the time grid in one call (#56)
aggregate_time_units() moves a tbl_now onto a bigger time unit – daily to weekly, weekly to monthly, monthly to yearly – and updates the object so that .delay, the converters and the models all count in the new unit:
hai <- hai_bucaramanga |>
tbl_now(event_date = specimen_date, report_date = report_date,
strata = sex, data_type = "linelist", units = "days")
hai |> aggregate_time_units(to = "weeks")- Counts are added up, not merely relabelled.
count-cumulativetotals are de-accumulated first, aggregated as increments and accumulated again on the new grid, because a cumulative total is not additive. -
axes =picks which axes move ("all","event","report","validation"), andlabel =picks whether a period is named by its first or its last day. Uselabel = "end"when you coarsen only a later axis, or a report lands before its own event. - Weeks go through the same epi/ISO machinery as
align_weeks(), sotypeandalign_on_daymean what they mean there. - It only ever coarsens: asking a weekly object for
"days"is an error, not a guess. So is aggregating anumericaxis, which has no calendar. - Weeks do not nest inside months. Aggregating to weeks and then to months is not the same as going straight to months; aggregate once, to the unit you want.
New: censor by condition, and replace the date (#57)
censor_reports() and censor_reporting_delays() take a filter()-style condition and record the matching rows as bounds rather than measurements – optionally replacing the date at the same time. This is the fix for the two dates that are not really dates: the missing one, and the sentinel far in the future.
hai |> censor_reports(is.na(report_date), to_report = Sys.Date())
hai |> censor_reports(report_date == as.Date("2222-02-22"), to_report = Sys.Date())
tn |> censor_reporting_delays(.delay > 60, to_delay = 60)The censoring family is now six verbs, two axes by three ways to select
| by date | by delay | threshold | |
|---|---|---|---|
reporting (is_censored_report) |
censor_reports() |
censor_reporting_delays() |
censor_reporting_delays_above() |
validation (is_censored_validation) |
censor_validations() |
censor_validation_delays() |
censor_validation_delays_above() |
-
censor_delays_above()is renamedcensor_reporting_delays_above()andcensor_delays()(added earlier in this release, never shipped) is renamedcensor_reporting_delays(), so every name says which axis it moves. Their behaviour is unchanged, and the_above()help now says plainly that it considers as censored every delay longer thanmax_delay. -
censor_validations()andcensor_validation_delays()are new: the validation-axis twins ofcensor_reports()andcensor_reporting_delays(). - All six are documented together on
?censoring.
"pending" cases are skipped when a validation date would be written, with a warning saying how many. A pending case is reported and still waiting, so it has no resolution date; writing one would assert a resolution that never happened and make the case look resolved to everything counting arrivals on that axis. Set validation_type to "confirmed" or "retracted" first if the case really was resolved. Flagging without a replacement is unaffected – no date is written, so nothing is contradicted.
-
NAis not a match: a condition that cannot be evaluated on a row is not a condition that row met. - Existing censoring flags are merged, never cleared, and the flag column is created as
.is_censored_reportwhen the object has none. - Replacing a date moves
nowforward when the replacement lands after it, never backwards, and drops any.report_*temporal-effect column that the move has made stale.
New: one units argument instead of three (#58)
tbl_now() gains units, the shared default for event_units, report_units and validation_units:
tbl_now(hai_bucaramanga, event_date = specimen_date, report_date = report_date,
strata = sex, data_type = "linelist", units = "days")Anything given explicitly still wins, so units = "days", report_units = "weeks" reads a daily event date against a weekly report date, and an explicit event_units = "auto" still means infer.
Fixes
-
group_by()(with no grouping variables),summarise()andreframe()copied the event units onto the report axis when rebuilding, so a mixed-unit object silently became a uniform one. They now carryreport_unitsacross. -
infer_units()on a column with a single distinct date warned aboutmin()returningInfbefore aborting with an unrelated message. It now says which column it is, and points atunits. - Censoring a grouped
tbl_nowaborted insideadd_is_censored_report()/add_is_censored_validation(), which refuse agrouped_tbl_now. All four censoring verbs –censor_reports(),censor_reporting_delays(),censor_reporting_delays_above()andcensor_validation_delays_above()– now ungroup, work, and put the grouping back. - The two censoring axes share one implementation of “merge this flag in without un-censoring anything”, rather than a copy each.
-
Demotion is now one operation. Dropping a protected column returns a plain tibble, and that used to be
as_tibble()– which leaves unknown attributes alone on a tibble but rebuilds agrouped_dfand drops them. So a demoted object kept the class’s attributes, or lost them, according to whether the caller had grouped it. It now strips them explicitly, either way. -
align_weeks()failed on a groupedtbl_nowwithColumn "now" not found in data: it readget_now()and nine other attributes off its own input after demoting it, and worked only by the asymmetry above. It now reads them first, and returns the grouping. -
complete_zeroes()aborted on a groupedtbl_now('length = 2' in coercion to 'logical(1)'): every bound it computes is afilter()/distinct()/pull()that a grouping turns into one value per group, so the date grid was built from a vector. The grid is a property of the object, not of how the caller grouped it. -
tbl_now_to_epidist()aborted on a groupedtbl_now; it was the only converter that did. -
DEVELOPMENT_SKILL.mdgains Every new function gets a grouped test (§8) and a line in the definition of done, because the six grouping fixes above are all the same bug;devel/audit_grouped_verbs.Rsweeps every exported function for it. Three verbs drop the grouping deliberately and are left for #61 to decide:to_count(),get_latest_reported_cases()andget_initial_reported_cases().
tbl.now 0.29.0
Breaking: is_censored is now is_censored_report (#54)
There are two censoring axes now, so the unqualified name had to go. The old spelling is removed outright, not deprecated:
| was | is |
|---|---|
tbl_now(is_censored = ) |
tbl_now(is_censored_report = ) |
get_is_censored() |
get_is_censored_report() |
add_is_censored(), change_is_censored(), remove_is_censored()
|
add_is_censored_report(), change_is_censored_report(), remove_is_censored_report()
|
is_censored attribute |
is_censored_report attribute |
.is_censored (the column censor_reporting_delays_above() creates) |
.is_censored_report |
New: is_censored_validation, the validation-axis censoring flag (#53)
The twin of is_censored_report, for models that use censored validation delays. It marks rows whose time from report to resolution is a bound rather than a measurement.
-
tbl_now(is_censored_validation = ),get_is_censored_validation(),add_is_censored_validation(),change_is_censored_validation(),remove_is_censored_validation(). It requires avalidation_date: there is no validation delay to bound without one. - The column is protected, is carried through every
dplyrverb and throughto_count(),update()andalign_weeks(), and joins the grouping keys – a censored resolution and an exact one on the same(event, report, outcome)triple stay two rows rather than being summed into one.
Breaking: censor_validation_delays_above() flags instead of erasing
It used to set the offending rows’ validation_type to "pending" and delete their validation date. That was wrong: a case confirmed after 200 days is still a confirmed case, and the object should say so. It now sets is_censored_validation and leaves the date and the outcome alone, exactly as censor_reporting_delays_above() does on the report axis. get_latest_confirmed() therefore still counts those cases.
New: validation_levels, for data not recorded in English (#54)
validation_type may hold only "confirmed", "retracted", "pending" or NA – that was already enforced, and the error now names the way out. tbl_now(validation_levels = ) is that way out: a named dictionary whose names are the labels in your data and whose values are the canonical four.
tbl_now(casos,
validation_type = desenlace,
validation_levels = c(
confirmado = "confirmed", retractado = "retracted", pendiente = "pending"
),
...
)The column is rewritten to the canonical values; the dictionary is kept on the object and read back with get_validation_levels(). A dictionary that would recode a canonical value into a different one is refused, because it would flip the column on every rebuild.
Fixed: change_now() re-censors instead of erroring (#51)
Moving now backwards is what change_now() is for – it is how a backtest walks through time. On an object carrying a validation process it aborted for every now earlier than the last validation, which is nearly every historical as-of date.
It now masks validations dated after the new now: the validation date becomes NA and the outcome returns to "pending", because a resolution that has not happened yet is not a resolution. change_now() and update_now() gain verbose to silence the report of how many rows were masked.
covid_us carries a validation process (#52)
No shipped dataset had one, so every example fabricated an outcome by row position. covid_us is rebuilt from the same CDC source with the two date columns that were being left on the floor, and it now runs onset -> positive specimen -> registration at CDC:
| was | is |
|---|---|
cdc_case_earliest_dt, cdc_report_dt, n (2020-2021) |
onset_dt, pos_spec_dt, cdc_report_dt, current_status, sex, n (2020) |
cdc_case_earliest_dt is CDC-derived and equals onset_dt for 99.997% of the rows kept, so it is gone as redundant; sex is a stratum, and current_status is the validation outcome – in CDC’s own words, so that mapping it is a worked example of validation_levels. The relationship between outcome and validation delay is real rather than fabricated: probable cases are registered a median of 2 days after the specimen, laboratory-confirmed ones 4 days. CDC does not withdraw cases, so "retracted" does not occur.
tbl.now 0.28.0
Breaking: the confirmation process is now the validation process
The optional third date a tbl_now can carry is called a validation rather than a confirmation, throughout. The old spelling is gone, not deprecated – it had not shipped.
| was | is |
|---|---|
add_confirmation(), change_confirmation(), remove_confirmation()
|
add_validation_date(), change_validation_date(), remove_validation_date()
|
get_confirmation_date(), get_confirmation_type(), get_confirmation_units(), has_confirmation()
|
get_validation_date(), get_validation_type(), get_validation_units(), has_validation()
|
confirmation_counts, confirmation_delay
|
validation_counts, validation_delay
|
censor_confirmation_delays_above(), diagnose_confirmation_delay()
|
censor_validation_delays_above(), diagnose_validation_delay()
|
plot_confirmation_delay(), plot_confirmation_status(), prop_confirmation_type()
|
plot_validation_delay(), plot_validation_status(), prop_validation_type()
|
confirmation_date, confirmation_type, confirmation_units arguments |
validation_date, validation_type, validation_units
|
.confirmation_num, .confirmation_delay columns |
.validation_num, .validation_delay
|
axis = "confirmation" |
axis = "validation" |
"event_to_confirmation", "report_to_confirmation"
|
"event_to_validation", "report_to_validation"
|
The outcome values are unchanged: a case is still "confirmed", "retracted" or "pending". Validation is what the process does; confirmed is one of the things it can conclude.
diseasenowcasting::confirmation_process() is that package’s name and is untouched – model(confirmation = confirmation_process()) still reads exactly as it did.
Documentation: fewer, fuller reference pages
- The validation getters now live on
?nowcast_data_getters, next toget_event_date(), and the validation setters on?add, next tochange_event_date(). Someone asking “what did this object record, and how do I change it” now finds every answer on one page instead of four. -
Describing and diagnosing a tbl_now and Diagnosing reporting batches are now one article, Diagnosing a tbl_now, running structure-first: what is in the data (
summary()), what is structurally wrong with it (diagnose()), and then the statistical testsdiagnose()signposts but refuses to run. - The attribute diagrams in the README now appear on the pkgdown site. They lived in
inst/figures/, which pkgdown does not copy;man/figures/is the directory it publishes, and GitHub renders it just as happily. -
summary()’s"completeness"and"growth"rows are distributions over event dates, so they populatemean/sd/the quantiles (and, for completeness,prop) and leave the scalarvaluecolumn empty. The documented examples selectedvalueand got a column ofNA; they now select the columns that carry the answer.
Fixed: baselinenowcast on a snapshot (“as of”) series
A snapshot stream restates the whole history in every snapshot, so its delay axis is as long as the series itself and the reporting triangle comes out square. baselinenowcast needs more reference dates than delay columns – it spends max_delay of them estimating the delay distribution and keeps two back for the uncertainty model – so it refused, with a message about reference-time arithmetic that mentioned neither the delay axis nor anything to do about it. Three of the six shipped datasets are that shape.
engine_baselinenowcast()gainsmax_delay, the number of delay periods to keep, forwarded totbl_now_to_baselinenowcast().?run_nowcastalready documented it (“max_delaycaps the triangle’s width”); what actually happened is that it fell into...and reached the modelling call, which has no such argument and ignored it.-
A triangle too wide to fit is now refused by
tbl.now, naming the delay axis and a concrete cap – the delay covering 99% of the reported cases:run_nowcast(x, engine_baselinenowcast(max_delay = 21))
Note that a snapshot series must be declared data_type = "count-cumulative". infer_data_type() reads a single downward revision as incidence, by design, and a revised running total has them; left to the inference, every delay carries a whole period’s count instead of an increment and nothing downstream can tell.
New: diagnose() and summary() print as reports
Both still return the tibbles they always returned, and every dplyr verb still works on them. What changed is what you see when you print one.
-
diagnose()– and each of its blocks – prints the errors, warnings and notes in full, each with its hint, and counts the checks that passed, that were deliberately not run, and that could not be assessed.print(x, all = TRUE)spells those out too. -
summary()– and each of its blocks – prints one block per component, dropping the columns that component does not populate. The schema is wide because it holds every block at once; no block fills more than a handful of it. -
tibble::as_tibble()gives the plain table back in both cases.
New: a nowcast prints its value at the now edge
print() on a tbl_nowcast now leads with the number it was fitted to produce – the estimate and interval at the last event date it covers, one line per stratum – before the quantile table, which starts at the oldest event date.
tbl.now 0.27.0
Breaking: a nowcast is specified with an engine()
run_nowcast() and nowcast_backtest() used to take a method name plus a ... (and, for the backtest, a method_args list of lists keyed by label). Both failed the same silent way: an argument that missed its backend simply vanished, and you got a fitted model at its default with nothing on the object to say so.
An engine is one modelling package plus every argument it needs:
run_nowcast(x, engine_nobbs(max_D = 10, moving_window = 64))
nowcast_backtest(x,
engine_baselinenowcast(draws = 1000),
engine_nobbs(max_D = 10),
now_dates = dates, seed = 20260824
)- One constructor per supported package –
engine_diseasenowcasting(),engine_baselinenowcast(),engine_epinowcast(),engine_nobbs(),engine_surveillance(),engine_epinow2()– each naming that package’s own arguments, so they are visible in the signature and a typo is an error at the call rather than a default nobody notices....still carries anything a named argument does not cover. -
engine(method, ...)is the general constructor and works for any registered method, including a backend you wrote yourself. -
The data and
verboseare the only arguments outside the engine.quantile_levelsmoved onto it, because forNobBSit is a fit-time model argument (it lands inspecs$quantiles, and NobBS keeps no draws, so a level it was never asked for cannot be recovered) rather than a way of summarising afterwards. -
nowcast_backtest(x, ...)now takes the engines variadically, or as one list.methodsandmethod_argsare gone. An engine’slabelis its name in the result; labels must be unique, and every engine must report the same quantile levels – the WIS averages over the levels reported, so mismatched engines are not scoring the same quantity. -
nowcast_method()is removed. The engine is the objectnowcast_fit()andnowcast_tidy()dispatch on, so an existing backend needs no change; writeengine("mymodel")where you wrotenowcast_method("mymodel"). - A bare method string is an error that names the constructor to use.
New: min_date, per engine
Every engine takes min_date, saying how much history to fit on:
min_date |
means |
|---|---|
NULL (default) |
the whole series |
a Date
|
keep event dates on or after it |
| a number | keep the last n periods before now, in the object’s own units |
It is per engine on purpose. baselinenowcast and diseasenowcasting take a long series in their stride, while epinowcast and EpiNow2 scale with the number of reference dates and are best given a window – so one global filter() over all of them was the wrong tool.
Prefer the number inside a nowcast_backtest(): now moves between fits, so a fixed calendar cut makes the fitted window grow as the backtest walks forward and the last fit is trained on more data than the first.
min_date trims the event axis, not now, and the trimmed object is what the result carries – so score_nowcast() and autoplot()’s reported counts describe the series the model was actually shown.
Breaking: score_nowcast() / as_scoringutils() take a tbl_now as truth
observed_col is removed, and a plain data frame is no longer accepted. The tbl_now already knows which column holds the observed counts – it is get_case_count(), or the count to_count() produces from a line list – so naming it was a burden on the caller and the old default (“the last column that is neither the event date nor a stratum”) was a guess that could mis-score silently.
score_nowcast(nc, truth = dengue) # the FULL tbl_now, line list or counts
as_scoringutils(nc, truth = dengue)Breaking: argument names made consistent
A documentation audit read every exported function and found the same argument wearing different names in different places. 116 of the 148 exports already took x first; these were the exceptions.
-
nowcast_fit()andnowcast_tidy()takeengine, notmethod. This is the one that affects code outside the package: if you wrote a backend, rename the first argument of your methods.# before nowcast_fit.mymodel <- function(method, x, ..., quantile_levels, verbose) { } nowcast_tidy.mymodel <- function(method, fit, x, ..., quantile_levels) { } # after nowcast_fit.mymodel <- function(engine, x, ..., quantile_levels, verbose) { } nowcast_tidy.mymodel <- function(engine, fit, x, ..., quantile_levels) { }What arrives has always been the engine –
engine()’s own documentation defines an engine as “the objectnowcast_fit()andnowcast_tidy()dispatch on” – and the old name was left over from the removednowcast_method(). The argument is only a dispatch handle, so no method body needed changing;R CMD check’s S3 consistency check will flag yours until you rename it.engine(method = )andlist_nowcast_methods()keep “method”, where it correctly means the name of a backend rather than a configured engine. databecomesxindiagnose_batches(),diagnose_batch_shape(),simulate_batch(),transport_discriminant(),censor_reporting_delays_above()andcensor_confirmation_delays_above(). Positional calls are unaffected. Two internal helpers also nameddatain their error messages, sodiagnose_batches(x = <not a tbl_now>)used to complain about an argument that did not exist.-
quietbecomesverboseincensor_reporting_delays_above()andcensor_confirmation_delays_above(), with the sense inverted and defaulting toTRUE, matching the twenty other functions that control messaging this way. Writeverbose = FALSEwhere you wrotequiet = TRUE.The converters that carry both
verboseandquietkeep both: they are different channels –verboseis the conversion summary,quietis the lossy-conversion warning – and the documentation now says so.
Breaking: align_weeks() numbers weekdays the ISO way
align_weeks(align_on_day = ) counted weekdays from Sunday while is_weekday(weekend_days = ) counted them from Monday. align_weeks() now uses ISO numbering too – 1 = Monday … 7 = Sunday – so the two agree. is_weekday() is unchanged.
The default is unchanged. It becomes 7, which is still Sunday, so align_weeks(x) – and tbl_now(..., align_weeks = TRUE), which is where nearly everyone meets it – behaves exactly as before. Only an explicit align_on_day changes meaning, and the migration is to subtract one, wrapping 1 to 7:
| you wrote | you meant | now write |
|---|---|---|
1 |
Sunday | 7 |
2 |
Monday | 1 |
3 |
Tuesday | 2 |
| … | … | … |
7 |
Saturday | 6 |
New: example_engine(), a toy engine for examples
Every real engine needs its modelling package, so every example that fitted a nowcast sat inside \donttest{} behind a requireNamespace() guard – and none of them ran on a default check. example_engine() needs nothing, is deterministic, and returns in milliseconds, so the examples for run_nowcast(), nowcast_backtest(), nowcast_weights(), score_nowcast() and tidy() on a backtest now show real output.
It is not a nowcasting method. It ignores the reporting delay entirely – reporting the counts that have arrived and putting a spread-wide band around them – so it under-predicts recent dates by construction. That is useful to see and useless to rely on; the examples say so. Its source is also the shortest complete nowcast_fit() / nowcast_tidy() pair in the package, if you are writing a backend.
New: tbl_now() warns on misspelled argument names
tbl_now() keeps unmatched ... names as user metadata, which meant a typo in a real argument name was accepted in silence. case_col = "n" set a useless attribute and left count data typed as a line list – as it had been doing in one of this package’s own examples.
Names close enough to a real argument to be a typo now warn and name the intended one. Deliberate metadata (data_source, citation, population) stays silent: a match needs a shared first letter and an edit distance under a third of the longer name, which is what keeps source from being read as a misspelling of force.
autoplot() on a nowcast draws the reported counts as columns
The cases reported so far were points floating in the middle of the fan, which reads as a second estimate. They are now grey columns under it, so they read as a count measured from zero and the correction the nowcast applies is the visible gap between the top of a bar and the band. The bars are one period wide, taken from get_event_units().
EpiNow2 keeps its draws
nowcast_tidy.EpiNow2() now reads the posterior samples with EpiNow2::get_predictions(format = "sample") instead of the fit’s lower_<pct>/upper_<pct> summary. Before, EpiNow2 could report only a median and the two tails of whatever CrIs it happened to be fitted with – three levels – so quantile_levels could not be honoured, tidy(probs =) was an error, and it could not join a type = "linear_pool" ensemble. It now does all three. The summary path remains as a fallback for a fit get_predictions() cannot read.
This has a visible knock-on: an ensemble containing EpiNow2 now shares all nine of nowcast_quantile_levels() rather than collapsing to three.
Performance: tbl_now() and every dplyr verb on one
No behaviour changed, but the class got substantially cheaper. tbl_now() is about 3x faster and validate_tbl_now() – which runs on every dplyr verb via tbl_now_reconstruct() – about 4x.
Almost all of the cost was building findings that were then discarded. validate_tbl_now() reports at floor = "note", so on a clean object it formatted eleven cli messages and showed one; formatting is the expensive part (a hint interpolating a vector of row numbers costs ~15 ms), and each finding also built its own one-row tibble (~2 ms).
-
.diagnose_text()now returns a template rather than a formatted string, and.diagnose_finalise()filters by the reporting floor before formatting, so only a finding somebody will read is paid for. - Findings are plain lists until
.diagnose_finalise()assembles the one tibble the caller sees.
diagnose() returns exactly the same tibble, and validate_tbl_now() the same conditions.
Documentation
Every reference page was read once, function by function, for an audience of public-health practitioners first and statisticians second.
Eleven defects, most of them found by running examples that had never been run.
tbl_now()documented two attributes that do not exist (repot_num– a typo – andevent_num) and omitted four that do. Thealign_weeksexample passedcase_col =, which...swallowed, building count data as a line list. Thechangeexample referenced an undefined object that only survived because R never forced the promise.update()’s example built from the whole dataset and then “updated” it with rows it already held.Fifteen pages shipped with an empty Description. A block opening with a bare
`r lifecycle::badge()`paragraph gets that badge as its entire@description, pushing the prose into Details;?diagnose_driftand fourteen others showed a badge and nothing else, in the help viewer and in the reference index.Article links.
vignettes/articlesis.Rbuildignored, sovignette("nowcasting-models")andvignette("custom-nowcast-models")resolved to nothing in an installed package. All article references now use URLs.Ten pages merged into five, with aliases preserved so
?changeand existing links still resolve:changeandremoveontoadd;plot_reporting_processontoplot_epidemic_process;names_tbl_nowandmoney_tbl_nowontoassign_tbl;as_scoringutilsontoscore_nowcast;censor_confirmation_delays_aboveontocensor_reporting_delays_above;is_tbl_nowontovalidate_tbl_now;week_2_dateontoalign_weeks;compute_temporal_effectsontoadd_temporal_effects.Every exported topic now has
@seealso,@returnand a runnable example; every internal function carries@noRd. Both@examplesIf FALSEblocks are gone, and nothing inman/containsif (FALSE)or\dontrun{}.?tbl.nowwas the DESCRIPTION text and nothing else. It now lays out the workflow – declare, describe, diagnose, reshape, fit, check – with a link into each step.Three slow examples trimmed:
align_weeksran the whole 452,567-row FluSight table (15.4s to 1.5s),tbl_now_summarycomputedsummary()four times over, and both Stan examples fitted on twenty years of dengue data.vignette("ensemble-nowcasting")gains a figure of the ensemble against each of its members, and a section onmin_dateexplaining why the engines are not all shown the same data.data-raw/ensemble_comparison.Rfits both Stan back-ends with approximate inference (epinowcastthroughenw_pathfinder(),EpiNow2throughstan_opts(method = "pathfinder")), so the article rebuilds in minutes rather than overnight. The article says so, so no member’s band is mistaken for that package’s tuned answer.It also no longer fits three epidemics. It scored every member on mpox and covid as well as dengue and cached the result as
forecasts; no chunk in the article ever read that table, and it was roughly two thirds of the run time.DEVELOPMENT_SKILL.mdrecords why the CRAN test path cannot be measured withtestthat::test_local(), anddevel/TEST_SPEEDUP_BRIEF.mdis a standalone brief on the suite’s runtime with measured per-file timings.
tbl.now 0.26.0
One surveillance line list per stratum
tbl_now_to_surveillance() gains format = "linelist_list", which returns one line list per stratum as a tbl_now_surveillance_list instead of one frame with a pasted strata column. surveillance::nowcast() has no strata argument, so a stratified analysis is one fit per stratum, and the split no longer has to be done by hand:
pieces <- tbl_now_to_surveillance(x, format = "linelist_list")
lapply(pieces, function(piece) surveillance::nowcast(data = piece, ...))It mirrors tbl_now_to_baselinenowcast(format = "triangle_list") throughout: the result is a plain list, so lapply(), [[ and friends work unchanged; it is length one and named "all" when the object declares no strata, so the return type never depends on whether strata happen to be attached; it prints what it is; and as_tbl_now() binds it back into a tbl_now, restoring the original date-column names, the strata and the covariates. Count input comes back as a "linelist" – one row per case, totals unchanged – because that is what a surveillance line list holds.
format = "linelist" remains the default and is unchanged.
Documentation
- The
surveillanceandNobBSsections ofvignette("nowcasting-models")now say that the credible interval is in their figures and is simply too narrow to see: the median band over the plotted window is under 1% of the estimate for both, against 37% forepinowcast. The numbers quoted are computed from the cachedtidy()tables rather than typed. -
EpiNow2gained the nowcast-vs-truth figure every other engine’s section already had. - The
surveillancesection fits its strata through the newformat = "linelist_list".
tbl.now 0.25.0
A vignette on writing your own back-end
vignette("custom-nowcast-models") is the full account of the nowcast_fit() / nowcast_tidy() contract: what a method may assume about the tbl_now it is handed (get the column names from the getters, work on .event_num/.delay rather than the calendar, run the grid to get_now(), remember that a line list cannot hold a zero), how to reuse the tbl_now_to_*() converters and as_tbl_now() instead of reshaping by hand, and what shipping a back-end in a package involves.
The worked example is a delay-ratio nowcast: for each delay it takes the median of the factor by which past mature event dates grew from that delay to their eventual total, and applies the empirical quantiles of that factor to the counts reported so far. It needs no modelling package, so the article runs every line of its own code – including the scoring, the backtest and the ensemble – and it is written twice, once returning predictions and once returning draws, to show both branches of the contract.
vignette("ensemble-nowcasting")’s section 4 now points here instead of carrying its own smaller version of the same material.
Bug fixes
-
autoplot()on atbl_nowcastdrew only the 50% band. The tails of each central interval were matched to the requested width by exact equality, and(1 - (1 - 2 * 0.05)) / 2is not0.05, so every other band came out as anNAribbon and was silently dropped byggplot2. The default nine quantile levels now draw all four bands, andlevels =is matched with a tolerance too.
Documentation
-
?nowcast_tidysaid its...was “available to your own” methods. It is not:run_nowcast()forwards the user’s...tonowcast_fit()only, so anything the tidying step needs has to travel inside the fit object. Both help pages now say so.
tbl.now 0.24.0
diagnose(): a structural health check
summary() describes a tbl_now; diagnose() looks for what is wrong with it. One row is one finding, sorted worst first, and the offending row indices come with it:
findings <- diagnose(dengue_now)
findings |> dplyr::filter(status <= "note")
bad <- findings |> dplyr::filter(check == "ordering")
dengue_now[bad$rows[[1]], ]Ten checks: declarations (attribute types, the columns they name, role collisions, columns the object was never told about, temporal effects added but never materialised), ordering (event <= report <= confirmation, including the transitive leg that a missing report_date would otherwise hide), missing, duplicates, units, negatives, now, truncation, strata and signposts. Each is also an exported function of its own – see ?nowcast_diagnose_components – and diagnose(x) is exactly the dplyr::bind_rows() of them.
status is an ordered factor, worst first, which is why the tibble sorts itself and why status <= "note" reads as “anything worth acting on”: error > warning > note > ok > not_run > skipped.
Four decisions worth knowing about:
-
It runs no statistical test, ever. Whether the reporting delay drifts, and whether reports arrive in batches, are statements about a distribution. Answering them means choosing a method, a maturity window and a multiplicity correction, and
diagnose()has no business choosing those on your behalf. It emitsnot_runrows carrying the call instead –diagnose_drift(x, axis =)anddiagnose_batches(x, axis =). -
Reporting outages are deliberately not detected. A
tbl_nowdoes not carry the zeroes, so an absent row means “nothing was reported” and a quiet Sunday is structurally identical to a three-week outage. Telling them apart requires asking whether a run of zero-arrival dates is improbably long, which is a test. The descriptive answer iszero_run_summary(); the inferential one isdiagnose_batches(). -
An
NAcount is reported neutrally. In a reporting triangle it means not yet observed – correct data, and the thing that tells a nowcast the cell is still open – sodiagnose_missing()counts it without calling it a defect. AnNAdate is a different matter and stays a warning. -
diagnose_strata()uses no thresholds. “Too small to fit separately” depends on the engine and on the epidemic, so it names the extremes – the smallest stratum, its case count and its share; the sparsest stratum and how much of the event grid it leaves empty – and lets you judge.
validate_tbl_now() is the same engine, presented as conditions
validate_tbl_now() no longer has a check list of its own. It calls the findings engine and re-emits the result as the cli conditions it has always emitted: it aborts on the errors and warns about the warnings. One implementation, two presentations.
What that changes for you:
-
validate_tbl_now()now warns when a confirmation precedes its report. That check existed, but only ran at construction, so an object that acquired the problem later never mentioned it again.tbl_now()no longer runs it separately, so it warns once rather than twice. - Everything else aborts and warns exactly as before, including
warn_non_uniqueness, which staysFALSEthere.diagnose()defaults itTRUE. - A
noteis never emitted as a warning.validate_tbl_now()runs inside everydplyrverb, and turning adiagnose()observation into a warning there would make construction noisy for data the class has always accepted. -
One warning was reworded. The missing-date warning said “N rows have NULL or NA values in column
event_date = "event_date"” – it printed the literal string rather than the column, and a column cannot holdNULL. It now reads “N rows have NA values in the event_date column"onset_week"”.
Breaking: the statistical tests take the diagnose_ prefix
The five tests are named for what they are for rather than for the fact that they are tests. The old names are gone, not deprecated:
| was | is now |
|---|---|
test_delay_drift() |
diagnose_drift() |
test_delay_changepoint() |
diagnose_changepoint() |
test_confirmation_delay() |
diagnose_confirmation_delay() |
batch_test() |
diagnose_batches() |
batch_shape_test() |
diagnose_batch_shape() |
The S3 class batch_test, and with it print.batch_test(), is renamed to diagnose_batches to match.
Documentation and website
summary() and diagnose() are now documented where people actually meet the package:
-
A new article, Describing and diagnosing a
tbl_now, treats the two as one workflow: what the schema means, what the six statuses mean, whyskippedis notok, and whydiagnose()refuses to run a statistical test. -
The worked example article is restructured. It now builds the
tbl_nowbefore cleaning and letsdiagnose()report the defects, rather than checking for them by hand and hoping the list was complete. The hand-written cleaning is still there — it is now the fix for what was reported, and it keeps the one checkdiagnose()deliberately will not do for a line list (deduplicating on a record id). - The README and the introductory vignette gain a compact section on each.
-
The reference index is now explicit.
_pkgdown.ymlgained areference:section grouping every exported topic, sosummary(),diagnose()and their components are findable rather than buried in one alphabetical list. Note for contributors:pkgdownnow fails the build on an exported topic that is not listed.pkgdown::check_pkgdown()catches it without building the site.
Fixed: the light/dark switch never rendered
template: light-switch: true was set and lightswitch.js was being loaded, but the site had no toggle. The control is a navbar component, and _pkgdown.yml named an explicit navbar: structure: right: that replaced pkgdown’s default [search, github, lightswitch] without listing it. The script loaded, the button did not exist, and nothing errored. lightswitch is now listed explicitly.
tbl.now 0.23.0
summary() describes the object the way a nowcaster reads it
summary() on a tbl_now now returns a tibble rather than the column-by-column listing summary.data.frame() produces, which said nothing about the structure the class exists to carry. One row is one statistic of one quantity of one stratum:
It covers the case counts on each of the object’s time axes (event, report and, where there is one, confirmation), the delay distributions between them, the lengths of the runs of zero dates, the compositional shares (censored, per confirmation outcome, per stratum, per categorical covariate level), the lag-1 autocorrelation of each series, the reporting-completeness curve, the totals, the date ranges and now, and how full the reporting triangle is.
Three decisions worth knowing about:
-
The date grids run to
now, not to the last row present. “Cases per event date” is a statement about a calendar; a date with no rows is a zero, not an absence. This is what makesprop_zeroand the zero-run lengths mean anything, and it is why a line list – which cannot represent a zero – summarises to exactly the same numbers as its counts. The grid is global, so a stratum whose cases start late shows its leading zeros and the strata stay comparable. So does the triangle-occupancy denominator. -
Quantiles are the inverse-ECDF (type 1) estimator, not
stats::quantile()’s default:q50is the smallest value whose cumulative weight reaches0.5. This is the estimatorautoplot()andtest_delay_drift()already use, so the table and the figures agree, and it always returns a delay that was actually observed. The mean and standard deviation are the ordinary case-weighted ones, equal to expanding the counts to one row per case. -
Not-yet-observed cells are dropped. An
NAcount means the cell has not been observed yet, unlike a0, which was observed and was zero. Those rows carry no cases and are excluded, rather than turning every total they touch intoNA– which is whatflusightdid to an earlier draft. The"unobserved_cells"coverage row says how many were dropped. -
count-cumulativedata gets no delay rows. A cumulative total is not additive across delays, so a case-weighted delay distribution would be meaningless;delay_summary()refuses it outright and points atto_count(). The new"growth"rows take its place, giving the ratio of each event date’s running total from one delay to the next.
Every block of the summary is its own function
summary() is exactly the bind_rows() of these, and each returns the same schema, so they stack:
cases_per_date(), delay_summary(), zero_run_summary(), prop_censored(), prop_confirmation_type(), prop_strata(), prop_covariate_levels(), case_autocorrelation(), date_ranges(), triangle_occupancy(), reporting_completeness() and cumulative_growth().
delay_summary() names the three delays explicitly – "event_to_report", "event_to_confirmation" and "report_to_confirmation" – because the first two are measured from the event and the last is the laboratory’s own turnaround, measured from the report, and confusing them is a documented hazard.
Internal
One date-grid helper replaces three inlined copies of the same seq(from, to, by = <units>) logic, including the one in complete_zeroes() that only knew about days and weeks.
tbl.now 0.22.0
The back-ends that stratify by ONE column
NobBS::NobBS.strat() takes a single strata column name, EpiNow2::regional_epinow() a single region, and surveillance::nowcast() takes no strata argument at all. A tbl_now may declare several stratifying columns, and their interaction – “nowcast each observed combination separately” – is exactly one stratum to those back-ends. The converters now build that column, so there is an argument to write:
-
tbl_now_to_nobbs()andtbl_now_to_surveillance()gainstrata_col(default"strata") andstrata_sep(default" | "). The declared strata are pasted into that one column, whichNobBS.strat(strata = "strata")takes directly and whichsplit()splits asurveillanceline list on. The original columns ride along unchanged, andstrata_col = NULLopts out. - Pasting is refused rather than fudged when a stratum value already contains the separator: the label could not be split back apart, and a nowcast silently attached to the wrong stratum is worse than a failed conversion. The error names
strata_sep.tbl_now_to_EpiNow2(target = "regional_epinow")gained the same check, which it did not have. - Writing into an existing column is refused too, so a declared covariate called
stratais not overwritten.
Previously tbl_now_to_nobbs() handed back the strata as ordinary columns and nothing else, so there was no way to call NobBS.strat() on a multiply stratified object at all. run_nowcast(x, "NobBS") had its own copy of the pasting logic; it now uses the converter’s column, so the two cannot disagree.
tidy() also learned the last per-stratum shape it did not know: a list of stsNC fits, which is what split()-ing a surveillance line list and looping produces.
tidy() returns the quantiles a NobBS fit was asked for
NobBS keeps no draws, so tidy(fit, probs = ...) refused every probs outright. But NobBS(specs = list(quantiles = c(0.1, 0.5, 0.9))) computes those levels at fit time and puts them in estimates – reading them back is a lookup, not an approximation, and refusing it made the documented workflow (“ask at fit time, then request them with probs”) impossible to complete.
tidy() now returns them. A level the fit was not asked for still aborts, because that one really is unrecoverable, and the message now names the missing levels and the specs = list(quantiles = ...) call that would have produced them.
The two date grids surveillance::nowcast() needs
-
get_surveillance_when(x, length = 30)– the dates to estimate, the most recentlengthsteps ending exactly atget_now(). -
get_surveillance_range(x)– the whole time axis, passed ascontrol$dRange.
Both read the step off the object’s own event units and abort on a "numeric" grid rather than anchoring integer indices at the 1970 epoch. dRange matters more than it looks: left to itself nowcast() infers the axis from the line list it was handed, and a line list cannot express a zero – the quiet days at the now edge have no rows, so the inferred axis stops short of exactly the days being nowcast.
The article now runs the code it shows
vignettes/articles/nowcasting-models.Rmd displayed cached results next to code that a separate script, data-raw/nowcast_comparison.R, kept its own copy of. The two drifted, invisibly, because the article never ran what it printed.
data-raw/nowcast_models_precompute.R replaces it: it knitr::purl()s the article, runs the article’s own chunks with the fits live, and reads the displayed objects back out by name. The code that produced every number is now literally the code printed above it. Renaming an object in the article stops the script with a list of what is missing instead of quietly saving a shorter file.
Fixed along the way, all of it drift the old arrangement hid:
- the Summary figure showed an unnamed grey
NAline, becauseEpiNow2had no entry in the figure’s colour scale and the factor dropped it toNA; - two chunk labels were duplicated and two chunks called
tidy()on objects the article never created, so the article could not be knitted at all; - the
EpiNow2delay section tidied adist_fitthat was never fitted, and theepinowcastseasonal fit was never assigned to a name; -
regional_epinow()was called withouttruncation, which is the one argument that makes it a nowcast – the same trap the pooled section spends a warning box on; -
epidist’s marginal model is used, now that it compiles. It reads the aggregated weights the converter produces instead of expanding 6.1M cases back to one row each, which is why the latent model was there; - the
epinowcastsections filtered to two years of daily reference dates while every other engine used 60 days, and the article claimed that “keeps the Stan fit tractable”. It does not: one chain spent six hours in a bad region of the posterior while the other chain of the same fit finished in sixteen minutes. The cached numbers had come from a 180-day fit that took six minutes, so the article had never run its own window. It is 180 days now, with the discrepancy explained in the text; - the
epinowcastfits were unseeded –epinowcast()does not take R’sset.seed(), so Stan drew its own each run and the same fit took 41 minutes once and six hours the next. Both now passseedthroughenw_fit_opts().
tbl.now 0.21.0
The confirmation process
A tbl_now can now carry a third date. Influenza is the picture to keep in mind: symptoms begin (the event), the patient visits a doctor (the report), and days later a swab comes back positive (the confirmation) or negative (a retraction – reported, but not a case after all). The assumed timeline is event <= report <= confirmation <= now.
-
tbl_now()gainsconfirmation_date,confirmation_typeandconfirmation_units.confirmation_typetakes"confirmed","retracted","pending"orNA; pending means reported and still waiting, so it has no confirmation date, which is a different thing from a result you never recorded (NA). Two columns are derived:.confirmation_num(on the same numeric grid as the other dates) and.confirmation_delay, the laboratory’s turnaround, measured from the report. -
add_confirmation(),change_confirmation(),remove_confirmation(),get_confirmation_date(),get_confirmation_type(),get_confirmation_units()andhas_confirmation(). - A date with no type warns rather than guessing: a date alone cannot say whether the case was confirmed or retracted. A confirmation before its own report warns too.
-
nowis confirmation-aware. A result issued on a date means the system was still being observed then, sonowis never earlier than the last confirmation, and setting one earlier is an error. - The confirmation columns survive
dplyrverbs,update(),align_weeks()(which now aligns all three dates) andto_count()(which groups by the confirmation, so a case is never summed together with its own retraction). - The print footer gains a confirmation line: the column, its units, and how many cases are resolved.
Counting when cases can be undone
get_latest_confirmed(), get_net_confirmed() (confirmed minus retracted), get_nth_confirmed(x, delay) and get_initial_confirmed() – the confirmation mirrors of the report-axis getters. censor_confirmation_delays_above() returns implausibly long confirmations to "pending", which is what they really were.
Diagnostics on the confirmation axis
A laboratory clearing a backlog looks exactly like a surveillance system clearing its inbox, so rather than duplicate every diagnostic, they take an axis = c("report", "confirmation") argument: batch_test(), batch_screen(), batch_shape_test(), transport_discriminant(), plot_reporting_process(), plot_epidemic_process(), plot_reporting_triangle(), plot_delay_profiles(), plot_reporting_hexamap(), plot_scalogram(), plot_delay_drift(), test_delay_drift(), test_delay_changepoint() and diagnostic_plot().
On the confirmation axis, delays are still measured from the event, so the two axes are directly comparable and the gap between them is the time the laboratory adds. Cases still "pending" are excluded – counting them would invent an arrival on a date they do not have.
New in their own right: plot_confirmation_status() (the confirmed / retracted / pending shares over time), and test_confirmation_delay() / plot_confirmation_delay(), which ask whether retractions come back faster than confirmations – a laboratory that rules cases out sooner than it confirms them biases any nowcast that treats the two alike.
Other changes
-
Calendar temporal effects are now factors.
day_of_week,day_of_month,month_of_yearandweek_of_yeararefactors with their full level sets (all seven weekdays, 1-31, 1-12, 1-52) rather than character or numeric columns, so a model gets dummy coding rather than treating “Tuesday” as twice “Monday”, and a level absent from a stratum still exists.weekendstays 0/1 and the Fourierseasonsstay numeric, as both are already correctly numeric. - Fixed: the non-uniqueness warning fired on every confirmed/retracted pair. A case and its own retraction share an (event, report) combination and are still two different rows; the confirmation columns are now part of the key.
-
run_nowcast(x, "diseasenowcasting")passes straight through todiseasenowcasting::nowcast(). The confirmation process belongs to that package’smodel(), not totbl.now, so pass it there.
tbl.now 0.20.0
Bugs found by the new engine test suite
Every one of these was found by writing the tests, not before:
-
count-cumulativedata failed ondiseasenowcastingfor want of a confirmation process.diseasenowcasting::nowcast()auto-detects cumulative data and switches to the signed-increment Skellam / SkNB likelihood, but that likelihood needs aconfirmation_process()– the retraction side of a stream that can revise down – andmodel()’s default isno_confirmation(). Without one the fit reports “Joint fit failed to converge for all init attempts”. Pass one through, asrun_nowcast(x, "diseasenowcasting", model = model(confirmation = confirmation_process())).De-accumulating to incidence first would also “work”, and is wrong: it discards the downward revisions the cumulative likelihood exists to model.
A censored report’s window started before its own event date. For
is_censoredrows,.delay_censoring_windows()bounded the secondary window below by the earliest event in the data rather than by that row’s event, so every censored row implied a possibly-negative delay, and the zero-width guard pushed one strictly negative. refuses it outright (“Assertion ondata$stime_lwrfailed: not >= 0”) andEpiNow2::estimate_dist()would have fitted a delay distribution with mass below zero. The window is now[event_date, report_date], and the zero-width guard widens upward.as_tbl_now(x, verbose = )failed on two classes.as_tbl_now.tbl_now_triangle_list()andas_tbl_now.tbl_now_epinow2_snapshots()passedverboseboth explicitly and through...: “formal argument ‘verbose’ matched by multiple actual arguments”. Both now default it into the dots, so the caller still wins.verbose = FALSEwas suppressing warnings, not just chatter..quietly_if()wrapped every backend insuppressWarnings(), which hid exactly the messages that say what the model actually saw – strata pooled, a censoring flag collapsed, covariates dropped. It now suppresses messages only. This is the same failure mode DEVELOPMENT_SKILL section 9 records forrun_engine().diseasenowcastingandNobBSwere pooling multi-column strata needlessly.diseasenowcastingmodels any number of strata and labels each combination"F|N";tbl.nowonly ever read the one-column case and pooled otherwise.NobBS.strat()takes one column, so several are now joined into their interaction and split back apart. Both take any number of strata, and?run_nowcast’s table says so.
Covariates and censoring are no longer dropped in silence
-
tbl_now_to_baselinenowcast()(matrix and triangle formats),tbl_now_to_epinowcast()andtbl_now_to_EpiNow2()now warn when declared covariates cannot be carried, naming them and saying what to do instead. Materialised temporal-effect columns count as covariates: they are the case where somebody asked for an effect and would otherwise never learn it was ignored. - The censoring collapse already warned; it is now reachable through
run_nowcast(verbose = FALSE)because of the.quietly_if()fix above.
nowcast_truth() removed
Dropped entirely rather than kept internal. It was get_latest_reported_cases() reshaped. score_nowcast() and as_scoringutils() take the tbl_now itself as truth.
covidat removed
covid_us is kept: it is the only shipped dataset that actually exhibits backlog dumps, which vignette("batch-reporting") is about. Measured against a 15-day rolling baseline, covid_us has 21 report days above 2x and 5 above 3x; covid_colombia has one above 2x and none above 3x.
New tests
All skip_on_cran(), all on synthetic fixtures built by tests/testthat/helper-engines.R rather than on shipped data, so one axis can be varied at a time:
-
test-engines-matrix.R– 24 real fits per fast engine ({0,2 covariates} x {0,2 strata} x {days, weeks} x the three data types), plus numeric-grid refusals, weekly-grid preservation, strata labelling for 0/1/2 columns, counts-are-cases, andrun_nowcast()against the hand-written call. -
test-engines-covariates.R– used, or complained about, per converter. -
test-engines-censoring.R– used, or announced, per converter. -
test-converter-roundtrip-all.R– a registry of everytbl_now_to_*()shape and whetheras_tbl_now()brings it back; fails when a converter is missing from it. -
test-coercion-methods.R– every converter must expose the target package’s own coercion generic (as_reporting_triangle(),as_tsibble(), …) as a thin wrapper, or record why that package has none. It re-checks the “has none” claims against the installed package, so we find out if one gains a verb.
Articles
-
vignette("nowcasting-models"): EpiNow2 is now a two-step fit. Given only a reporting delay it does not nowcast at all – its median stayed flat and sat below the already-reported count.delayssays how infections become reports; it does not say the newest days are incomplete. Onlytruncationdoes, and that is what the report dimension of atbl_nowmeasures. Step 1 fits it withestimate_truncation(), step 2 passes it astrunc_opts(). Over the last seven days – about 50% complete – the fit now sits below the reported count on 4 of 21 stratum-days instead of most of them. -
vignette("ensemble-nowcasting"): the three-epidemic comparison is removed; the article is now about how to use ensembles.
tbl.now 0.19.0
Converters no longer make you aggregate first
covid_colombia carries sex. An object built without strata = sex therefore has two rows per (notification_date, diagnosis_date) cell, and a reporting triangle, a tsibble key and an epinowcast observation table each have exactly one slot per cell. Until now that meant tbl_now_to_baselinenowcast() aborted (“duplicate reference_date and report_date combinations”) and tbl_now_to_tsibble() aborted (“a valid tsibble must have distinct rows”), and you had to group_by() |> summarise() before converting.
Both now pool undeclared columns for you, as tbl_now_to_nobbs(), tbl_now_to_surveillance(), tbl_now_to_EpiNow2(), tbl_now_to_epinowcast() and tbl_now_to_data_table() already did. The pooling is to_count(), so case totals are preserved exactly, and it is reported under verbose = TRUE:
i `tbl_now_to_baselinenowcast()`: pooled over 1 undeclared column ("sex");
18195 rows -> 10129.
i Declare it with `add_strata()` to nowcast it separately.
Line lists are left alone: one row is already one case there, and collapsing would destroy the individual records the target package is being handed.
The non-uniqueness warning now names the culprit
It used to say “Consider using to_count() to aggregate the data or distinct() to remove repeated observations”. The distinct() half is wrong whenever the cause is an undeclared column – those rows are distinct, they differ in sex – so it sends you in a circle, and on data with genuine repeats it silently deletes cases. The warning now inspects the object and says which:
- undeclared columns: names them, and points at
strata =orto_count(), adding that the converters pool them for you so this is information rather than a fault; - genuine duplicate rows: says so, and then recommends
distinct().
tbl_now_to_baselinenowcast(max_delay = )
A cap on the delay axis, counted exactly as tbl_now_to_epinowcast() counts it – max_delay = 30 keeps delays 0 to 29, giving a 30-column triangle – so the same number means the same triangle in both. NULL (default) keeps every delay, which is the previous behaviour. This replaces the filter(.delay <= 30) |> idiom the docs used to recommend.
nowcast_truth() is now internal
It was get_latest_reported_cases() with the class stripped, undeclared columns summed away and the count renamed .observed – the values were identical. A second public name for that is a second thing to learn for no gain.
score_nowcast() and as_scoringutils() now accept the tbl_now itself as truth and do the reshaping internally, which is shorter than what it replaces:
score_nowcast(nowcast, truth = dengue) # was: truth = nowcast_truth(dengue)A data frame of observed counts still works, as does NULL.
?run_nowcast says what the models actually are
Three new sections, because “it calls the package with its defaults” is not enough to read the output:
-
Strata – a table of how many each backend can model and how.
baselinenowcast,surveillance,EpiNow2andepinowcasttake any number;diseasenowcastingandNobBStake exactly one and pool with a warning beyond that, because the single array dimension they return cannot be split back into two columns. -
Temporal effects – the converters materialise them into columns, but only
diseasenowcastinguses them automatically.epinowcastneeds them named in a module formula; every other backend carries them and ignores them. -
Censored delays – collapsed with a warning by every backend that goes through a converter;
diseasenowcastingreceives the flag intact.
And a section on how each engine’s default model is specified, with the two that most need saying out loud:
-
epinowcastdefaults to a per-day random effect on the growth rate (a random walk in all but name), a single time-constant lognormal reporting delay, and no day-of-week report effect. -
EpiNow2defaults todelays = delay_opts(), which isFixed(0)– no reporting delay at all – andgeneration_time = gt_opts(), which isFixed(1). Those defaults describe a process with nothing to nowcast, so supply the epidemiology yourself. It also models with a Gaussian process rather than a random walk.
Article fixes
-
vignette("nowcasting-models")now cuts at 2021-04-01, on the rising limb of Colombia’s third wave, instead of 2023-03-03 where the epidemic had subsided and a nowcast had nothing to correct. The line-list engines trim to 60 days (278,000 rows; NobBS 24s, surveillance 6s measured) rather than 180. -
The per-package figures were drawing the wrong quantity.
geom_col()was givenwidth = 5.5on a daily series, so each bar overlapped its neighbours and ggplot2 stacked the overlaps: the grey “reported by now” bars showed sums of about six days. The summary figure at the end of the article usedwidth = 0.8and was correct, which is why the two disagreed. All the panels now usewidth = 0.8. -
The stratified
NobBSexample handed it count rows, which is the exact mistake the article’s own warning box forbids two screens earlier – it counts rows, so it was nowcasting counts as cases. It now goes throughtbl_now_to_nobbs()like the unstratified example. - The stratified
surveillanceexample converted the whole series (2.3M cases) at build time and usedN.tInf.max = 1000against per-stratum daily counts of ~4,000, which silently truncates the posterior. It now trims first and uses the same settings as the unstratified fit. -
vignette("ensemble-nowcasting")gains an experimental badge and note at the top, and a figure showing each member’s median against the ensemble and the eventual truth.
tbl.now 0.18.0
New: one call per model, and ensembles
Until now tbl.now prepared data for six nowcasting packages and normalised what they returned, but running several of them still meant six different calls and six different result shapes to reconcile by hand. This release adds the layer that removes that bookkeeping.
-
run_nowcast(x, method)fits any supported package and always returns atbl_nowcast: an S7 object holding the predictions as one row per (event date, stratum, quantile level), plus the draws where the backend has them, plus the backend’s own untouched fit. Backends ship for"diseasenowcasting","baselinenowcast","epinowcast","NobBS","surveillance"and"EpiNow2", each feeding its package through the matchingtbl_now_to_*()converter rather than building the input by hand.It is called
run_nowcast()and notnowcast()because exportsnowcast(); keeping the names distinct means both can be attached at once. nowcast_ensemble()combines several of them, either by averaging their quantiles level by level (type = "quantile", vincentization – narrower) or by pooling their draws into a mixture (type = "linear_pool"– wider, and refused outright when a member has no draws, rather than silently dropping it).nowcast_backtest(),score_nowcast()andnowcast_weights()score models retrospectively and turn those scores into ensemble weights ("inverse_score","optim"or"equal").as_scoringutils()hands the same object to for its full score suite.nowcast_fit()/nowcast_tidy()are the extension point: two S3 methods, in any package, andrun_nowcast()knows about your model. Seevignette("ensemble-nowcasting").autoplot()for atbl_nowcastdraws a fan chart, in the palette’s green – a nowcast estimates the epidemic process, not the reporting one.
New: tidy() for a nowcast and for a backtest
tidy() already worked on every raw engine fit. It now also works on what run_nowcast() and nowcast_ensemble() return, which is the way round it should always have been.
-
tidy()on atbl_nowcastreturns the package’s standard frame –event_date,stratum,estimate,conf.low,conf.high,level,engine, plusq*columns forprobs.engineis the method (or the ensemble’s name);levelis the width of the widest symmetric pair of quantile levels the object actually carries, and isNA, withNAbounds, when no symmetric pair exists. A guessed 0.95 there would defeat the one column that exists to stop a 90% band being compared with a 95% one.probsis honoured only when the nowcast carries draws, and errors otherwise: a quantile-only nowcast cannot produce a level it was not summarised at.Registered in
.onLoad(), becausetbl_nowcastis S7 andtidy.tbl.now::tbl_nowcastis not a writable S3 method name. tidy()on anowcast_backtestgives one row per (method,nowdate, target), with the internal dot-prefixed columns traded for ordinary ones.
New: reproducible backtests
nowcast_backtest() gains a seed argument. When given, the RNG is seeded immediately before each fit, from the seed and the method and date that fit is for. One set.seed() before the whole backtest only pins anything if every method draws the same random numbers in the same order – which stops being true the moment a method is dropped or one date is refitted. This is the same lesson data-raw/nowcast_comparison.R already records.
nowcast_weights(type = "optim") now falls back to equal weights, with a warning, when the optimiser does not converge on a usable point. It used to return NA weights, which do not fail until much later inside nowcast_ensemble(), as an all-NA nowcast that reads like a modelling problem rather than an optimisation one.
Removed: the nowcaster backend
nowcaster was dropped in 0.16.0 along with its converters, for the reasons recorded there. The run_nowcast() backend for it is not shipped: it called tbl_now_to_nowcaster() and get_nowcaster_strata(), which no longer exist. Neither nowcaster nor INLA is reintroduced to DESCRIPTION.
Other
-
scoringutilsadded toSuggests(CRAN, so noAdditional_repositoriesentry is needed). is deliberately not added: it is GitHub-only and sits in no repositoryR CMD check --as-crancan resolve, so declaring it would trade an undeclared-import warning for a CRAN-incoming NOTE about a dependency that cannot be found.nowcast_fit.diseasenowcasting()therefore looks its entry point up withgetExportedValue()after.need_pkg()has confirmed the package is installed, rather than writing a literaldiseasenowcasting::. -
LICENSE/LICENSE.mdcopyright year updated to 2026. The hand-rolled.wis()is now cross-checked against it in the test suite: two implementations agreeing is worth more than either alone. - New article,
vignette("ensemble-nowcasting"), with the fits precomputed bydata-raw/ensemble_comparison.Rso the build never fits anything. It reports WIS per model and per ensemble across three epidemics, and answers “does the ensemble beat its best member?” and “does performance weighting beat equal weighting?” from the cached numbers rather than by assertion. -
vignette("nowcasting-models")gains arun_nowcast()column in its package table, and a pointer to the new article.
tbl.now 0.17.0
New: support
tbl_now_to_EpiNow2() and tbl_now_from_EpiNow2(), against EpiNow2 1.9.0 (now the minimum in Suggests). EpiNow2 takes four different input shapes, one per entry point, so target names the function the result is passed to and it can be handed over unchanged:
-
"estimate_infections"(default) –data.frame(date, confirm), the series as known atget_now(). Also whatepinow()takes. -
"regional_epinow"– the same plus aregioncolumn built from the object’s strata (" | "-joined for several, matching thetriangle_listconvention). The other targets pool strata with a warning. -
"estimate_truncation"– atbl_now_epinow2_snapshotslist, onedate/confirmsnapshot per report date. This is the one EpiNow2 model that uses the report dimension atbl_nowexists to carry. -
"estimate_dist"– the interval-censored frameEpiNow2::estimate_dist()fits a delay distribution to. New in EpiNow2 1.9.0, and it documents the schema exactly, so it shares.delay_censoring_windows()withtbl_now_to_epidist()rather than growing a second copy.
Three things worth knowing:
-
EpiNow2 models a daily process and has no
timestep. As of 1.9.0 there is notimestep,intervalorperiodargument on any entry point (all four formals checked), so a weekly series passed as one row per week is read as one row per day – no error, just an epidemic seven times too fast. The converter lays it on the daily grid with EpiNow2’s ownaccumulatecolumn instead – built by [EpiNow2::fill_missing()] rather than by hand, because a hand-rolled version put each period’s count on the period’s last day wherefill_missing()leaves it on the date given, shifting every weekly fit six days with no error. Units coarser than a week, and the"numeric"grid, are refused by name rather than approximated.initial_accumulateis passed explicitly rather than inferred: withby, EpiNow2 1.9.0’s inference drops each group’s first observation (a two-region weekly series of 336/167 cases came back as 295/147). Single-series inference is unaffected. The snapshot form has a real inverse. Snapshot k is the series as known at report date k, so differencing consecutive snapshots recovers
count-incidenceexactly.tbl_now_epinow2_snapshotscarries the report dates soas_tbl_now()can do it; a bare list needsreport_dates. Verified againstEpiNow2::example_truncated, which round-trips to the case for the case. (The commented-out draft of this converter asserted no inverse was possible. For a single series that is true; for snapshots it is not.)estimate_secondary()andestimate_delay()get no target. The first models two data streams against each other and onetbl_nowis one stream; the second is superseded byestimate_dist()by EpiNow2’s own help and throws away the censoring atbl_nowcarries.
obs_date and the censoring windows are different quantities, and the converter now treats them as such. [sdate_lwr, sdate_upr) brackets when the report happened – at weekly resolution [W, W + 7), a half-open interval whose upper bound is the end of that week, not a claim that anything happened on day W + 7. obs_date is when observation stopped, which estimate_dist() asserts is >= sdate_upr on every row. A tbl_now’s now labels a period, so the instant observation stopped is the end of it: obs_date = now + w. That makes the assertion hold by construction, and nothing is observed after it. Clamping the windows at now instead was tried and rejected – it moves reports in the final period into an earlier one, which the epidist round-trip test caught.
The nowcasting-models article now covers across all three strata, with its results precomputed into nowcast-comparison.rds like every other engine. Two caveats are stated in the article itself: the delay distributions are ’s shipped examples rather than distributions fitted to the Colombian data, and sampling is lighter than the default (500 draws, 250 warmup, 2 chains) because it is much the slowest engine in the comparison.
data-raw/nowcast_comparison.R now takes engine names (Rscript data-raw/nowcast_comparison.R EpiNow2) and merges them into the existing file, leaving every other engine’s rows and recorded timings alone; with no arguments it rebuilds everything as before. This replaces a second script that re-created the setup by parsing the first one.
Two correctness fixes came out of that. Every engine is now seeded per (engine, stratum) immediately before its fit, rather than relying on a single set.seed() at the top of the script – which only pins results if every engine consumes the same random numbers in the same order, and so does not survive refitting a subset. Refitting baselinenowcast alone had been silently changing its estimates, and one EpiNow2 fit produced a stratum whose upper credible bound sat at 1e8 for all 181 days and would not reproduce. Both now refit to max abs diff == 0. The script also refuses to cache any fit whose scale exceeds 100x the observed maximum for its stratum, since an unconverged Stan or INLA fit returns numbers rather than an error.
tidy() gained methods for estimate_infections, epinow, estimate_truncation and estimate_dist, plus a regional_epinow branch in tidy.list() giving one block per region.
tidy.estimate_dist() reports the fitted distribution’s mean and sd alongside its parameters, so its output is directly comparable with tidy.epidist_fit(). They are derived from the distribution, not from the family’s algebra: each draw’s parameters go back into the fit’s own dist_spec and through [EpiNow2::discretise()], which knows the families, and the moments follow by summation over the PMF. Nothing in this package names a distribution, so a family adds later works as soon as discretise() supports it. Against the closed forms the mean is exact and the sd runs about 1% high – the variance a discrete grid adds – so expect a difference of that order against , which reports continuous-distribution moments.
It also honours probs and takes a level argument, matching tidy.epidist_fit(). (An earlier draft rejected probs with a message claiming the engine keeps no draws. It does: summary.estimate_dist() reads them.)
tbl_now_to_EpiNow2(target = "estimate_dist") warns when it pools strata – estimate_dist() has no grouping argument, so it fits one distribution to everything – and warns when a large share of delays are exactly zero, since a lognormal has zero density there and will inflate its variance rather than fail. The message points at the families that do have positive density at zero ("exp", or "gamma"/"weibull" with shape below 1) rather than at a constant shift, which would silently bias every parameter.
Two more points of care:
-
levelis read off thelower_<pct>/upper_<pct>column names, because EpiNow2’sCrIsis a user argument – a fit made withCrIs = c(0.5, 0.95)has nolower_90at all, and hard-coding0.90would report a width the fit never produced. -
tidy.estimate_dist()returns the delay schema (term,estimate, …), not the nowcast one – the second instance of the documented exception alongsidetidy.epidist_fit(). Note thatsummary()’smean/sdcolumns are the posterior mean and sd of each parameter, while themean/sdrows this method reports are the delay distribution’s moments. Same words, different quantities.
.epidist_drop_unusable_counts() is now .drop_unusable_counts() and shared: EpiNow2::estimate_dist() asserts n >= 1 with the identical message epidist uses, so the same filter applies to both.
Audit of the converters and tidy() against the target packages’ own docs
Every claim the converters and tidy() methods make about diseasenowcasting, baselinenowcast, epinowcast, epidist, NobBS, surveillance, tsibble and data.table was re-checked against those packages’ installed help pages and source. Five defects came out of it, all of them cases where the code was silently plausible rather than wrong-looking.
-
tidy()no longer pools strata under"all".tidy.nowcast()documentsstratumas"all"when the fit is unstratified, so(stratum, event_date)is meant to be a unique key. Two methods broke that:-
tidy.epinowcast()readsummary(fit, type = "nowcast")and ignored both.groupand thebycolumns sitting beside it. A real two-group fit (by = "age_group"ongermany_covid19_hosp, age groups00+and80+) came back as 20 rows all labelled"all", with every one of its 10 reference dates duplicated. It now emits one block perbygroup, and several grouping columns are pasted" | "-separated, matchingtbl_now_to_baselinenowcast(format = "triangle_list"). -
tidy.list()recognised aNobBS::NobBS.strat()fit – it has theestimates/onset_dateshape the detector looks for – but ignored thestratumcolumn thatNobBS.strat()puts there. A two-stratum fit ondenguedatreturned 44 rows labelled"all", 22 of them duplicate keys. It now readsstratumwhen present.
The
probspath forepinowcastwas mispaired in the same way: it split the posterior samples onreference_datealone, so on a stratified fit each date’s quantiles went to whichever stratumsplit()sorted first. The split is now keyed on(stratum, reference_date)and indexed by the summary’s own rows. -
tidy()no longer invents an interval for abaselinenowcastpoint fit.baselinenowcast(output_type = "point")returns one value per reference date and stampsoutput_type = "point"on the result.tidy()ignored that column and took the 2.5%/97.5% quantiles of a single number, reportingconf.low == conf.high == estimatewithlevel = 0.95– a zero-width 95% band. It now returnsNAbounds andNAlevel, and refusesprobsrather than returning the point estimate under a quantile’s name.tbl_now_to_nobbs()prints theunitsstring NobBS accepts. Its verbose summary printed the object’s own"weeks", butNobBS::NobBS()documentsunitsas"1 day"or"1 week"; pasting"weeks"into the call produces-Inf/Infwarnings fromseq()and then an opaquereplacement has 1 row, data has 0. It now prints"1 week", and aborts up front for any grid NobBS cannot model.The line-list back-ends no longer fabricate 1970 dates from a
numericgrid.tbl_now_to_nobbs()andtbl_now_to_surveillance()both coerced the event and report columns withas.Date(). On anumeric-unittbl_nowthose columns are integer indices, so index 1 became 1970-01-02 and the conversion succeeded, silently, with a line list of invented dates. Both now abort naming the units, astbl_now_to_baselinenowcast()andtbl_now_to_epinowcast()already did.tbl_now_to_surveillance()also gained"years"->"1 year"; it previously fell through to"1 week".
The remaining findings were addressed too:
A negative delay now warns instead of silently losing cases. A reporting triangle is indexed by delay from 0, so a report that arrived before its event has no cell: 10 cases in gave a triangle summing to 9, with the affected cell reading
0– an observed zero – rather thanNA. Both triangle formats andtbl_now_to_epinowcast()now warn, naming how many rows and cases go and how to filter them yourself.format = "long"has no delay axis, keeps them, and stays quiet.tbl_now_to_epidist()acceptscount-cumulativedata. epidist assertsn >= 1, and de-accumulating a cumulative series produces a0wherever a report added nothing and a negative on any downward revision – so the conversion died on epidist’s ownAssertion on 'data$n' failedfor essentially any real cumulative input, and for plain incidence data that had been throughcomplete_zeroes(). Rows carrying no case are now dropped before the epidist object is built: a zero contributes nothing to a delay distribution, so that is lossless and only reported underverbose = TRUE; a negative discards a revision, so it warns; and if nothing usable is left the converter aborts saying why.flusight– the oneerrorcell in the article’s converter matrix – now converts.tidy()handles a per-stratum list ofbaselinenowcastfits.?tbl_now_triangle_listrecommendslapply(triangles, baselinenowcast::baselinenowcast), andtidy()on the result used to error and suggestengine = "NobBS". A list whose elements are allbaselinenowcast_dfis now recognised: each is tidied and labelled with its list name (or its position, when the list is unnamed), giving the same one-block-per-stratum table the natively stratified engines return.probspasses through.DEVELOPMENT_SKILL.mdsection 2 corrected. It claimedtbl_now_to_surveillance()setscontrol$dRange. It does not, and its own help page says so:nowand the delay unit are deliberately left to the caller, because the converter cannot know which window you mean to fit.
Behaviour changes
tidy()reportslevel = NAfor a fit instead of0.95.NobBS()’slower/uppercome fromspecs$conf, and its return value islist(estimates, estimates.inflated, nowcast.post.samps, params.post)– nospecs, so the width is genuinely unrecoverable from the fit. A guessed default is worse thanNAin the one column that exists to stop widths being compared blindly. Passtidy(fit, level = 0.95)to fill it in. The assertion intest-tidy.Rthat recorded the old behaviour was updated.tidy.epidist_fit()warns on a delay model with covariates.epidist::predict_delay_parameters()returns one row per draw and observation, and the reported quantiles pool over both. Formu ~ 1every observation shares the draw’s value, so that is exactly the posterior interval; with covariates in the delay model the interval is a mixture across covariate levels, which the docs described simply as “Posterior median”. The method now detects a parameter that varies within a single draw and says so, pointing atnewdatafor a specific covariate combination. The numbers are unchanged – only the silence is.
Tests
New
test-tidy-strata.R: stratifiedtidy()forepinowcastandNobBS.strat(), quantile-to-stratum alignment, the point-fit interval, the per-stratum list ofbaselinenowcast_dffits, and thelevelargument. The fits are mocked from the shape of real ones, so the file needs neither cmdstan nor JAGS.New
test-converter-grids.R: thenumericgrid across every converter, zero / negative / very long delays, gaps in the event grid, a trailing event period with no reports under eachcompletesetting, the negative-delay warning, and epidist’sn >= 1filtering (includingflusight).New
test-converter-strata-shapes.R: several stratifying columns, a factor level with no rows, and label-to-value pairing when the data order is not alphabetical.test-converter-censoring.Rnow coverstbl_now_to_nobbs(), which the converter loop skipped because its package name is not its suffix.-
Removed support:
tbl_now_to_nowcaster(),get_nowcaster_strata(), thetidy()branch for its fits, and its sections in the articles are all gone. The converter worked, but the package around it demanded enough special-casing that keeping it cost more than it returned:-
Dmaxandwdware counted in weeks, whatever grid you hand it. On a daily series, values chosen as days are silently read as weeks:Dmax = 30,wdw = 120asked for a 30-week horizon over a 2.3-year window, which ran for 45 minutes and had INLA reporting the fit diverging. The same fit with week-scaled values took 24 seconds. - It returns weekly estimates from daily data, so its numbers are weekly totals while every other engine reports daily counts – roughly 6x larger on the same axis, and not comparable without re-gridding. Its label is the week start.
-
age_colmust be numeric even though the help calls it a stratum column: a character column errors insidecut(), and a characterbins_agetrips anif (bins_age == "SI-PNI")comparison against a vector. The converter existed largely to encode strata into codes and hand back the matching breaks. -
Results come back as those codes, not labels, so a tidied stratified fit reported
stratumvalues of"1"and"2"rather than the levels. -
It takes its maximum observable time from the last event date, not the last report, so cutting the series anywhere except where
max(onset) == max(report)made it NA-mask genuinely observed cells and nowcast below what had already been reported. - It needs R-INLA, and was itself installable only from GitHub.
That last point has a side benefit: was the only entry in
Remotes:, so removing it drops that field entirely – and with it the reason the package could not be submitted to CRAN as-is. -
New
tbl_now_to_nobbs(), filling a real gap. counts rows, so handing itcount-incidencedata was silently wrong: a table of 1,174 rows carrying 50,160 cases was nowcast as 1,174 cases, with no error. The converter expands counts to one row per case first. The articles previously recommendedas.data.frame(), which is correct only for a line list.Fixed the pkgdown build on CI. The shared “Learning more” fragment was pulled in with a relative child path (
../../man/fragments/...). renders into an intermediates directory undertempdir()and copies relative resources alongside it, so a path containing../..escapes that directory – harmless wheretempdir()is deep, fatal on CI where it sits two levels from the filesystem root (cannot create file '/tmp/RtmpXXXX/../../man/...'). The fragment moved toinst/fragments/and every caller now locates it withsystem.file(), which is path-independent and also works underpkgload::load_all().Dependency fixes for CI.
almanacis used by the package but was declared nowhere – theRemotes:entry for it was inert, sinceRemotesonly says where to fetch an already-declared dependency. It is now inSuggests, with its r-universe added toAdditional_repositories(it was archived from CRAN).nowcasterwas declared but unobtainable from any configured repository; it briefly gained aRemotes:entry, and was then dropped altogether (above).tidy()on a fit now works directly. From 2.1.0 that package re-exports the sharedgenericsgeneric and ships its own method, sotidy(fit)returns the standard nowcast table.tbl.nownow registers its own method fordiseasenowcasting::nowcast_predictiononly when the package does not supply one, so older versions keep working and newer ones are not overridden. The article calls plaintidy(dnc_fit)again.Article fixes so the code on the page reproduces the output shown. Three places displayed results the printed code could not produce: the section tidied the fit rather than
predict(fit), and the section hid the trailing-row trim that keeps the final week from exploding. All three now match the precompute.Documented two
tidy()masking hazards.library(diseasenowcasting)attaches its owntidy()generic, andlibrary(broom)overwritestbl.now’stidy.list()method (which fits dispatch on). Neither errors; both silently return a different table.tbl.now::tidy()disambiguates.New
tidy()method for fits. is the one supported package that does not nowcast – it estimates the reporting-delay distribution – sotidy.epidist_fit()returns a delay-shaped table (term,estimate,conf.low,conf.high,level,engine) with one row per distribution parameter, rather than forcing a delay fit into the per-event-date nowcast schema.probsworks, because the fit keeps its draws. Note thatepidist()returnsc("brmsfit", "epidist_fit")in that order, so a loaded wins dispatch; calltidy.epidist_fit()explicitly if that matters.tidy()now returns ’s credible interval, which it previously discarded.surveillance::nowcast()stores a prediction interval in the returned object’spislot at the widthcontrol$alphanames (95% by default), but the method hard-codedconf.low,conf.highandleveltoNA, so surveillance was the one engine that appeared to report no uncertainty. Reaching for the JAGS-backedbayes.trunc/bayes.trunc.ddcpmethods was never needed to get an interval.-
Censored delays no longer break the converters. A censoring indicator that is a property of the case rather than of the delay – an administrative “this date is only an upper bound” mark, say – splits one
(event_date, report_date)cell into a censored and an uncensored row. A reporting triangle has one slot per cell, sotbl_now_to_baselinenowcast()andtbl_now_to_epinowcast()aborted on duplicate cells, and the converters that expand back to a line list picked the flag up as an unrequested stratifier. The censoring dimension is now collapsed before the conversion, and each route warns:- count data: counts are summed over the flag, so case totals are unchanged;
- line lists: the column is dropped, leaving one row per case.
tbl_now_to_epidist()is deliberately exempt: estimating a delay distribution is the one job that can use the flag. tbl_now_to_baselinenowcast()now handles a line list on its own. It already aggregated to incidence; it now also completes the zero periods out to thenow(newcomplete = TRUEargument). A reporting triangle is a rectangular grid, and an event period with no reports has no rows, so the triangle used to stop short unless you rememberedto_count() |> complete_zeroes()first. Linelist and count-incidence input now produce an identical triangle.tbl_now_to_baselinenowcast()also acceptscount-cumulativedata, which it used to refuse. De-accumulating produces negative increments wherever a total was revised downward, and shipspreprocess_negative_values()for exactly that; the converter applies it and warns.negatives = "error"restores the old refusal.Bug fix:
tidy()ignored the strata of adiseasenowcastingfit. A stratified fit reportsstrata_draws(draws x event times x stratum), but the method read only the pooleddraws, so every row came back withstratum = "all"even when the fit itself said “2 strata”. It now returns one block per stratum, matching what the other engines do.-
Two new test files worth naming, because they exist to stop silent regressions:
-
test-converter-equivalence.R– every converter accepts line-list input, and the triangle/preprocessing targets give the same result from a line list as from the equivalent count-incidence object. -
test-converter-datasets.R– every converter against every dataset the package ships. This is the testthat counterpart of the article’s matrix: the article documents, this one fails.
-
Website: fixed a regression that drew an empty scrollbar track (“a rectangle”) under every code chunk. The no-wrap rules have to apply to
codeas well as its container, butoverflow-xmust apply ONLY to the container – setting it on the inner<code>too made each one a second scroll context that reserved a gutter. Figure captions are now centred, smaller and grey.The nowcasting-models article now shows the real output of every fit and every
tidy()call. The fits are far too slow to run on each build, sodata-raw/nowcast_comparison.Rcaptures what each one prints and whattidy()returns for it, and the article replays that. Each section pairs a copy-pasteabletidy(fit)(shown, not run) with a hiddenhead(5)whose output appears beneath it, so nothing in the visible code has to be trimmed for display. The ad-hoc result extraction each section used to do – pulling$estimatesout of NobBS, building adata.frame()fromepoch()andupperbound()for surveillance – is gone; every section now usestidy().New article section running every converter, plus a nowcast, against every dataset the package ships (
data-raw/converter_matrix.R), recording which combinations work and explaining the ones that do not:count-cumulativecannot become a reporting triangle without inventing negative increments,epidisthas no individual delays to censor in a cumulative series, and atsibbleneeds a unique index/key. Each attempt is time-limited so the matrix is reproducible.The article states plainly that each modelling package is a separate install that
tbl.nowdoes not pull in, with the commands for the ones that are not on CRAN and the note that Stan, JAGS and R-INLA are software outside R.SKILL.mddocumentstidy(), thesurveillanceconverter,format = "triangle_list", the newcomplete_zeroes()behaviour, and the zero-period pitfalls.Every package section in that article now shows how to recover its predictions with
tidy().The comparison precompute falls back gracefully when rejects a triangle whose most recent reference times are all zero. A thin stratum can hit that after the zero weeks are completed out to
now– here the female series has no case in the final week even though the pooled series does – and the fallback completes only as far as the last week holding a case, costing that stratum one week rather than the whole nowcast.-
New
tidy()methods, one shape of answer whatever engine produced the fit. The converters normalise what goes into a nowcasting package;tidy()normalises what comes back out. It returnsevent_date,stratum,estimate,conf.low,conf.high,levelandenginefor fits from , , , and .-
probsadds one column per requested quantile, named after the probability (q5,q50,q2.5, …). Only the engines that keep draws (, , ) can honour it; the others error rather than return an approximation dressed up as a quantile. -
levelrecords the width each engine’s interval actually has – reports a 90% band by default while the others report 95%, and without it the two get compared as if they were the same. - returns an unclassed list, so it is told apart by structure, with an
engineargument to override. -
tidy()deliberately does not re-grid: packages that bin onto their own week starts keep them, because snapping would hide a real difference. - The generic comes from (a new, dependency-free
Imports), so it composes with rather than masking it.
-
The nowcasting-models article now cuts at the second week of July 2002 rather than the turn of the year: a December cut lands on the holiday reporting slump, which says more about December than about the models. Section headings are now just the package name, each package’s figure carries a caption instead of a heading, each gains a Simple nowcast heading, and packages needing an external backend (Stan, JAGS, R-INLA) carry a coloured requirement callout. The overview table gained an Additional requirements column.
Website:
.alert-warningcallouts now use the attenuated red the plots use for intervals, and code blocks are pinned to scroll sideways rather than wrap – pkgdown’swhite-space: pre-wraponcodeinsideprewas overriding the bareprerule and folding long lines onto a second line.Print methods now write to stdout instead of emitting messages.
print.batch_test(),print.transport_discriminant(),print.tbl_now_triangle_list()and thetemporal_effectsprint method used thecli_*()family, whose output is a message – so it vanished undermessage = FALSE,sink()orcapture.output(), which is exactly where a print method is expected to work. They now use cli’scat_*()family. The matching tests were switched fromcli::cli_fmt()tocapture.output().The
epinowcastsection filtered on a hard-coded2008-12-20, left over from the old window; against the new data that matched zero rows and aborted the build. It now trims relative to the series, usingtbl_now_to_epinowcast(preprocess = FALSE)followed byenw_filter_reference_dates()andenw_preprocess_data().tbl_now_to_baselinenowcast()gainedformat = "triangle_list": one reporting triangle per stratum, instead of pooling them into a single matrix. Unlike splitting the long format by hand it takes the delay unit and the strata off the object, so neither has to be restated. With no strata attached the result is still a list — of length one, named"all"— so the return type never depends on whether strata happen to be present.The result is a thin
tbl_now_triangle_listclass: still an ordinary list, solapply()and[[work as before, but with aprint()method. The class earns its place as a guard: ’sestimate_and_apply_delays()also takes a list of triangles, but retrospective snapshots of one series rather than one per stratum, and would silently accept a per-stratum list and treat the strata as points in time.as_tbl_now()gained a method fortbl_now_triangle_list, rebuilding acount-incidencetbl_nowwith the strata recoded onto their column. The strata values are stored on the object rather than parsed back out of the element names, so a stratum containing the name separator still round-trips.Bug fix:
as_tbl_now()aborted on any weekly reporting triangle.tbl_now_from_baselinenowcast()ignored the triangle’s owndelays_unitattribute and read the delay columns as days, so a weekly triangle produced daily report dates against weekly event dates and unit inference contradicted itself (“report_units must be coarser than or equal to event_units”). Both directions now defaultdelays_unit = NULLand resolve it from the attribute; an explicit value still wins.Bug fix in
complete_zeroes(): it was silently deleting real cases. The closing “don’t look into the future” filter compared with<rather than<=, so every row reported on the final report date was dropped — in the function’s own documented example, 5 of 55 cases vanished. A function whose job is to add zeroes was removing data at the boundary.complete_zeroes()now completes out to the object’snow, not merely to the last event date present in the data, and gained anuntilargument to complete to a specific date instead. An event date with no reports at all does not appear in the data, so the old behaviour left a gap exactly at thenowedge — where nowcasting matters. A supplieduntilnever truncates below the data. The line-list error message now explains why a line list cannot hold a zero week and points atto_count().plot_reporting_hexamap()’smax_cellsis now a real bound. It previously took the delay at positionmax_cellsand kept every cell sharing that delay, so a wide band at the cut overshot the documented cap.The nowcasting-models comparison now runs to the
nowfor every engine.baselinenowcastgets there viacomplete_zeroes();surveillancecannot (a zero-count row expands to zero line-list rows, so padding evaporates) and is instead given its grid directly throughcontrol$dRange. The article explains both, including that forcingsurveillanceto estimate a period with no observations is unstable on stratified data.The nowcasting-models article now builds its
tbl_nowfrom the wholedenguedatseries (52,987 cases over 1,091 weeks) instead of a pre-filtered two-year window. Every converter runs on the full object in a few seconds; where a package needs a shorter series to fit, the article now uses that package’s own argument rather than subsetting the data first —moving_windowinNobBS(which is what takes the full-series fit from impractical to about six seconds) andwheninsurveillance.diseasenowcasting(~12 s) andbaselinenowcast(~10 s) take all 1,091 weeks as they are.epinowcastis the one engine with no such argument, since the reporting triangle is already built by the time you hold a preprocessed object; the article showstbl_now_to_epinowcast(preprocess = FALSE)followed byenw_filter_reference_dates()andenw_preprocess_data(), and prints the full and trimmed objects side by side. Each section states which it is doing.Website:
.alert-infocallouts (pandoc::: {.alert .alert-info}fenced divs) are restyled from the Bootstrap default blue into the package’s sage green, with a darker green left rule and heading colour.The nowcasting-models article’s
baselinenowcastfit referred to adengue_triangle2object that no longer existed; it now usesdengue_triangle.The example article was rewritten from scratch around the new
hai_bucaramangadataset and is now a full end-to-end tutorial: cleaning a messy surveillance extract withdplyr+tbl.now(duplicate records, missing dates, and reports dated before the event they describe), reading the data withautoplot()and the standaloneplot_*()diagnostics, testing the reporting delay for drift and change points, attaching only the temporal effects the data justifies, and finally nowcasting withdiseasenowcastingand five other engines. Each modelling choice at the end is traced back to a diagnostic at the beginning. It moved fromvignettes/tovignettes/articles/(the pkgdown URL is unchanged) becausediseasenowcastingis not on CRAN and so cannot be fitted while building a shipped vignette.-
New converters for further back-ends:
-
tbl_now_to_surveillance()builds the individual-level line list [surveillance::nowcast()] works from, renaming the event and report dates to ’s owndHospital/dReportdefaults.format = "sts"instead returns the observed curve as asurveillancestsobject.
It accepts count data as well as line lists, expanding counts back to one row per case (de-accumulating first when the data is cumulative). is a new
Suggests. -
The nowcasting-models article gained a section for it and a closing comparison of every engine on one set of axes — one plot for the unstratified object and one faceted by stratum, with a colour per package, the incomplete data each engine actually saw, and the counts those weeks eventually reached. The comparison deliberately uses an earlier 2002-2003 window rather than the article’s main
dengue_now, because the latter runs to the end ofdenguedatand so has no ground truth to check against. The fits are precomputed bydata-raw/nowcast_comparison.Rand read from a saved file, so editing the prose no longer re-runs Stan, JAGS and INLA.flusightno longer ships duplicate rows (#25). The upstream FluSighttime-series.csvcontains 39,139 exact duplicates, which forced every example to open with adistinct()call; the dataset now goes from 491,706 to 452,567 rows. The removal is lossless — every repeated (as_of,target_end_date,location_name) key carried an identicalobservation, with no conflicting values — so that triple is now a unique key. The help page documents the change, and the FluSight example vignette drops the de-duplication step.New dataset
hai_bucaramanga: 1,423 healthcare-associated infections (IAAS) notified in Bucaramanga, Colombia, 2016-2023, from the Colombian open data portal. Column names and categorical values are translated from Spanish. It is a deliberately unpolished extract and its help page documents the defects in detail — a1900-01-01missing-date sentinel, 88 negative reporting delays, 100 exact duplicate records, and a strongly bimodal delay (3-day median, 92-day 90th percentile) — which makes it a realistic exercise for the delay diagnostics rather than a clean modelling example.test_delay_drift()andtest_delay_changepoint()now document every column of their output, plus new Interpreting the result sections. Thetest_delay_drift()help gained a Choosing a method section explaining why"hamed-rao"is the default (deterministic, no AR(1) assumption, effectively instant) and when to cross-check with"block-bootstrap", which is robust to weekly periodicity but stochastic and thousands of times slower.The Get Started vignette now opens the nowcasting problem with a figure showing observed-to-date cases, the reports still in transit, and the nowcast of the eventual total.
The “Learning more” links live in a single
man/fragments/learning-more.Rmdand are included in the README and at the end of every vignette and article, so they only have to be edited in one place.Website: the “Articles” navbar dropdown was rendering near-black with grey text because the styling targeted
.submenu, which Bootstrap 5 does not use; it now targets.dropdown-menuand matches the pale red of the package plots.pkgdown/extra.cssis also no longer listed underincludes: in_header:, which was pasting raw CSS into<head>where it was ignored.README code blocks no longer wrap mid-tibble: printed output was being split into stacked column blocks by R itself, which no stylesheet could undo.
tbl.now 0.15.0
-
autoplot()panels are now consistently colour-coded by process: red for everything reporting-related (the delay distribution, the delay calendar/holiday effects, the delay periodogram) and green for the epidemic (event-date) process (the observed cases and their calendar/holiday effects). This matches the colours the standalone diagnostic plots (plot_reporting_process()/plot_epidemic_process(),plot_scalogram(), …) already used, so a panel and its standalone twin read the same. - Every
autoplot()panel now says which process it describes in its subtitle — either “Reporting delay process” or “Epidemic (event-date) process” — replacing the per-panel explanatory subtitles. A single panel therefore reads on its own. - The two periodogram panels are renamed “Cycles (periodogram)” (previously “Seasonality” / “Delay periodicity”).
- Every
autoplot()panel now has a standaloneplot_*()twin that draws just that panel (identical data, colours and subtitle):plot_day_of_week_effects(),plot_week_of_year_effects(),plot_month_of_year_effects(),plot_holiday_effects(),plot_holiday_lag_effects()(each takingtype = "epidemic"ortype = "report"), plusplot_cycles(),plot_delay_distribution()andplot_observed_cases(). Useautoplot()for the grid and aplot_*()for one effect on its own. - The day-of-week, week-of-year, month-of-year, holiday and weekend/holiday-lag panels gained a
measureargument (inautoplot()and everyplot_*()twin).measure = "normalized"(the default) is the existing view — each value divided by its overall mean, 1 = average.measure = "percent"instead shows the share of cases falling in each group with its IQR (e.g. “10% of cases at the weekend versus 90% on weekdays”); the reporting version shares out the reports by report date. Percentages needDateevent/report columns. - Vignettes: the Get Started guide documents the
plot_*()twins and themeasureargument, and marks the “Holiday effects”, “Do delay distributions drift over time?” and “Detecting batch reporting” sections as AI-written, pointing readers to the human-written batch-reporting article. The FluSight example analysis is flagged as a work in progress.
tbl.now 0.14.1
Strata are now carried into the model converters that can use them.
tbl_now_to_epidist()keeps the strata as data columns (usable as covariates in an epidist formula), andtbl_now_to_baselinenowcast(format = "long")keeps them so you can build one reporting triangle per stratum. A single reporting-triangle matrix has no strata dimension, soformat = "matrix"now pools the strata with a warning instead of erroring on duplicate cells.tbl_now_to_epinowcast()already passed strata as its grouping (by).The nowcasting-models article was restructured: each package is now shown bare (from
dengue_now) and then enriched — onedengue_seasonalobject carrying a stratum and temporal effects flows through every converter — so the separate “Carrying delay effects into each model” section is gone. It adds a worked per-stratumbaselinenowcastloop (one triangle per stratum). The workflow also had a bug: it used the pluralestimate_and_apply_delays()(which expects a list of retrospective triangles) on a single triangle; it now uses the one-callbaselinenowcast()wrapper for samples and notes the singularestimate_and_apply_delay()for a point nowcast.New
plot_reporting_hexamap(): draws the reporting triangle as an age-period-cohort hexamap (Jalal and Burke, 2020). Event date, report date and delay are the cohort, period and age (report = event + delay); each cell is a hexagon coloured by its report count, and a batch — a single report date — reads as a clean vertical stripe. The number of hexagons is bounded by amax_cellssafety cap (the delay axis is auto-capped, with a message, rather than drawing an unbounded map). Replaces the reporting-V panel in the batch article.Bug fix for issue #33:
autoplot(x, strata = "race", by_strata = TRUE)no longer errors with a strata passed as column name.-
autoplot()gained four holiday panels, which describe the attachedtemporal_effects()spec rather than the event unit:-
"calendar_holiday"/"delay_holiday"— normalized cases / mean reporting delay by day type. The categories follow the spec: aholidayscalendar plusweekend = TRUEgivesWeekday/Weekend/Holiday, a calendar alone givesNon-holiday/Holiday, and a weekend effect alone givesWeekday/Weekend. A holiday falling on a weekend counts as a holiday. -
"calendar_holiday_lag"/"delay_holiday_lag"— the same, by position relative to the nearest holiday ("2 before","1 before","Holiday","1 after", …, plus"Other"as the reference), as asked for byholiday_lags. These show exactly the days the..._holiday_lag_k/..._holiday_lead_kcolumns flag, so you can check a lag is worth modelling before you model it. A date that is both after one holiday and before the next is attributed to the nearer one, ties going to the “after” side.
-
Bug fix:
tbl_now_to_epinowcast()now passes atimestepto , inferred from the object’s report units ("days"->"day","weeks"->"week") and overridable with the newtimestepargument. It previously left on its"day"default whatever the data, so weekly data was laid out on a daily grid.Bug fix:
tbl_now_to_epinowcast()now derives the temporal-effect covariates on ’s completed date grid instead of carrying them throughenw_complete_dates(). That function fills the (reference, report) grid and extends the reference axis into the nowcast horizon, but sets every non-schema column toNAon the rows it adds — so the covariates previously survived only on the original rows. Becasue the effects are functions of a date alone, they are n ow re-derived from the completed grid and cover every row, including the recent horizon dates a nowcast has to predict.
tbl.now 0.14.0
-
holiday_lagsandweekend_lagsintemporal_effects()now accept negative depths, placing the effect before the break instead of after it. A negative depth creates..._holiday_lead_k/..._weekend_lead_kindicator columns that flag dates exactlykworking days before a holiday / weekend, counting backwards from it — so_lead_1is the working day closest to the break.weekend_lags = -1flags the Friday,weekend_lags = -3flags the Wednesday, Thursday and Friday, andholiday_lags = -1flags Christmas Eve. Working days skip weekends and holidays exactly as they do for positive depths, andholiday_lagsstill requires aholidayscalendar for either sign. Use it to capture the reporting slowdown that precedes a break; attach one specification per direction to model both sides of it. Positive depths are unchanged. -
?temporal_effectsgained a “Using a different holiday calendar” section.holidayshas always accepted anyalmanac::rcalendar(), but the docs only showedcal_us_federal(); reporting holidays are local, so the section covers the building blocks (built-inhol_*()rules, customrholiday()rules, weekend observance withhol_observe(), and editing a calendar withcal_add()/cal_remove()), and works through the New York City calendar as an example.
tbl.now 0.13.1
- Fixed style in the batch reporting vignette
- Improved the axis title position in the v triangle to better visualize the dates
tbl.now 0.13.0
Bug fix:
batch_shape_test()no longer errors (“missing value where TRUE/FALSE needed”) on large count data. The standardised rank-sum expands counts to one value per item, so the group sizes could exceed the 32-bit integer range and their product overflowed toNA; the group sizes are now computed as doubles.batch_test()now returns a lean, Benjamini-Hochberg-only result:report_date,stratum,reported,baseline,deficit,delta,p_transport,p_transport_bhand thebatchflag, each documented under?batch_test. The raw per-pointclassificationcolumn (and thep_creation/p_deletion/scale columns behind it) has been dropped: it was not multiplicity-corrected and over-identified, whereasbatchcontrols the false discovery rate. (transport_discriminant()keeps itsclassification.)batch_test()(andtransport_discriminant()) now infer the calendarperiodfrom the object’s temporal effects: a day-of-week effect setsperiod = 7, a week-of-year effectperiod = 52(see [add_temporal_effects()]). Aperiodyou pass still wins, with a note if it disagrees; and if the data is daily with no temporal effect, the function suggestsperiod = 7.The
baseline_methodargument ofbatch_test()andtransport_discriminant()has been removed — the baseline is always the repeated-median local line. The running-median (local-constant) alternative had no advantage: it reduces to the same fit on a flat series and is biased the moment the series trends.New
covid_usdataset: a compact aggregation of the U.S. CDC COVID-19 Case Surveillance Public Use Data, with both event and report dates in 2020-2021 (a self-consistent “as of the end of 2021” snapshot), built to demonstrate batch reporting. Its reporting delay is huge and heavily right-skewed — cases were released to CDC in large backlog dumps — sobatch_test()and the batch plots recover a clear, real signal (and correctly call the biggest December-2021 spikes surges, since they land on the Omicron wave). Prepared with duckdb from the 14 GB source (seedata-raw/covid_us.R).New article, Finding batch reporting in CDC COVID-19 case surveillance data, written for public-health practitioners with no maths. It builds a made-up outbreak with a planted batch to show what each plot looks like (including a novel V reporting triangle – the reporting triangle rotated 45° so a batch is a horizontal slice), rehearses on a real dengue epidemic curve with simulated log-normal reporting and self-planted batches, finds the batches in the real
covid_usdata, adds a wavelet view (window-inner report-vs-event scalograms, via ), and ends with a one-page summary table. A new transport-vs-creation tutorial plants a hold, a batch and a surge in a made-up outbreak and colours each day the same way on the reporting timeline and in the creation/transport plane, so a reader can trace a bar to its dot and see why a batch goes up, a surge goes right, and a hold drifts up-and-left.Every plot function now takes
plotly = TRUEto return an interactive widget (hover, zoom) instead of a static plot:plot_reporting_process(),plot_epidemic_process(),plot_reporting_triangle(),plot_delay_profiles(),plot_delay_drift(),plot_transport_discriminant(),plot_reporting_v(),plot_scalogram(),diagnostic_plot()andautoplot(). Needs the (suggested) package.New
plot_reporting_v(): the reporting “V” – the same data asplot_reporting_triangle()(the same event-date x delay cells) rotated 45° so report date runs up the page and the data opens into a V (left arm = event date, right arm = delay). A batch, a diagonal in the square triangle, becomes a horizontal slice. The whole observable triangle is filled (pale-blue reported zeros + coloured reports).New wavelet scalograms,
plot_scalogram(type = "reporting")andplot_scalogram(type = "epidemic"), plus the pairedplot_reporting_process()andplot_epidemic_process()bar charts. The scalogram splits the count series into fast wiggles (short periods) and slow swings (long periods) and shows the energy at each: a batch lights up as a bright short-period ridge in the reporting scalogram that the epidemic (event) scalogram lacks. These use a window-inner scalogram (,border_effects = "INNER"): computed from observed data only, with no border padding, so nothing is fabricated at the recent (“now”) edge that matters for nowcasting. Reporting views are drawn in red, epidemic views in green.plot_scalogram()defaults to the PAUL wavelet (wname), which localises a batch more sharply; takes aformatargument for the x-axis date labels (default"%d/%b/%y"); and paints the region outside the cone of influence dark grey. The series is analysed on its own integer time grid, so weekly (or monthly) data is handled correctly, and the heat map tiles a uniform index relabelled with dates so it stays gapless even for long series.The conservation monitors —
plot_creation_transport()(the two window scores as stacked panels) together with the cumulative-backlog, reporting-lag, dashboard and transport-minus-creation “batch score” plots — live indevel/conservation_extras.R, kept out of the package: clean on large batches but noisy in general. The transport diagnostics keep their exportedtransport_discriminant()/plot_transport_discriminant().simulate_batch()gains aheld_fractionargument: the fraction of each closed date’s reports actually held back and released later (default1, a full closure). Withheld_fraction = 0.5, roughly half of each day’s reports are held and half report on time – a realistic partial slow-down rather than a total blackout. Supported for"linelist"and"count-incidence"data (a cumulative total cannot be split).The default
lookbackforbatch_test()andtransport_discriminant()is now 7 (a week of daily reporting) rather than 3.The
@detailsof the batch functions (batch_test(),transport_discriminant(),batch_shape_test(),simulate_batch()) and the batch plots were trimmed: the formal theorem / null-distribution derivations were replaced with concise, plain-language explanations.-
New
diagnostic_plot(): a gallery of complementary views of the reporting process for spotting reporting artefacts (above all batch reporting), laid out in two columns. The five panels are the reporting process (reports by report date), the reporting triangle (event date x delay), the per-date delay profiles, the reporting-delay drift (plot_delay_drift()), and the transport discriminant plane. Each is also its own exported function. Choose views withpanels(a single one is returned as a plain plot), and every view is facetted by stratum.by = c("report", "event")switches the profiles panel;...(e.g.period = 7) is routed to whichever panels accept it.- Every panel carries a plain-language, grey caption explaining what it shows and what the colours mean, and legends are labelled in words.
- The reporting process y-axis is capped at the 99th percentile only when a pathological dump (over 30x the median day, e.g. covid’s 1.8M-report day) would otherwise flatten the whole series; an ordinary batch spike – the very thing the plot exists to show – is left to tower.
- The transport discriminant y-axis is limited to the batch region (with default clipping, so points stop at the panel edge) so the deep-negative “hold” dates do not squash the confirmed batches; the shaded region is now labelled “Potential batch region” and each confirmed batch gets a bold, unclipped date label.
- The reporting triangle draws a third axis for report date: evenly spaced dashed diagonals (
report = event + delay) running up-right at 45°, labelled by report date, so event date (x), delay (y) and report date are all readable off one plot (plot_reporting_triangle(report_ticks =), default 6;mark_batches =optionally highlights the biggest batch stripes). It also distinguishes an observable reported zero (muted blue) from a not yet reportable cell (blank), on the full calendar event axis. - The delay profiles draw in a single colour at fixed transparency.
- The transport discriminant colours red only the
batch_test()-confirmed batches (BH-corrected), not the raw per-point classification – which at levelalphapainted 10-20% of points batch/surge/hold by construction, ignoring multiplicity and the heavy autocorrelation of the window statistics. The shaded batch region and the±z*lines are drawn only as a reference for where a batch would sit.
New
transport_discriminant(): exposes the plane behindbatch_test()’s conservation law – for every report date the deficit (the transport axis: reports the preceding window is missing) and the window discriminant (the creation axis: the window total relative to its baseline), with robust standardisedtransport_z/creation_zand the same quadrantclassification. A batch sits top-left (a deficit paid the spike, no net creation); a surge sits bottom-right. Returned as atransport_discriminanttibble and plotted bydiagnostic_plot(panels = "transport").The multi-panel
autoplot()title changed from “Diagnostic plots” to “Automatic plot of effects” (that phrase now titlesdiagnostic_plot()).batch_test(null_model = "auto")is now overdispersion-aware. The exact Poisson/Binomial null assumes Poisson counts and a baseline that captures the mean; real surveillance counts are overdispersed, and the conditional transport test is then badly anti-conservative (on clean but overdispersed Poisson data it can fake dozens of batches).autonow reserves the exact null for non-negative counts with no detected overdispersion (dispersion<= 1.5) and otherwise falls back to the dispersion-corrected robust null; signed (count-cumulative) increments still always use the robust null. This makes the default far more realistic on overdispersed data (e.g. filteredcovid_colombiadrops from ~125 flags to ~18; addperiod = 7for its weekly reporting cadence to reach ~4). Force the old behaviour withnull_model = "poisson"if you need it.autoplot()’s empirical delay distribution panel now adapts tocount-cumulativedata: instead of a histogram of increments it shows the cumulative growth by delay — boxplots (on a log scale, with a dashed reference at1) of the ratio of each event date’s cumulative count at a delay to its count at the previous delay. Ratios above1are upward revisions, below1downward ones, and they converge to1as reporting completes, so you can see the cumulative curve stabilise. The log scale makes a doubling and a halving symmetric about1.linelist/count-incidencedata keep the histogram, and the panel respectsby_strata.tbl_now_to_baselinenowcast(delays_unit = )now defaults toNULLand is inferred from the object’s time units for the"matrix"format: when the event and report units are equal and either"days"or"weeks", that unit is used; otherwise the function errors asking you to supplydelays_unitexplicitly. (The"long"format never uses it.)Added the
covid_colombiadataset fromdiseasenowcastingto here.Fixed several documentation issues that produced “could not resolve link” warnings when building the docs (links to internal helpers / to the un-declared
trendpackage, a[0, 1]mis-parsed as a link, and a mis-ordered internal roxygen block).to_count()now supportscount-cumulative->count-incidenceby de-accumulating the series (increment = cumulative total minus the previous one within each event date and grouping). Because cumulative totals can be revised downward, an increment can be negative. This fixesautoplot()(and the other delay diagnostics) oncount-cumulativedata such as FluSight, which previously errored with “Transformation fromdata_typecount-cumulative to count-incidence not implemented” (#26).Updated
SKILL.md(the AI-agent usage guide) to cover everything added since 0.10.0: reporting-delayautoplot()panels and thepanels/by_strataselectors,plot_delay_drift()/test_delay_drift()/test_delay_changepoint(), the model-free batch detectors (batch_test(),batch_shape_test(),simulate_batch()),get_nth_reported_cases(), the after-holiday/weekend temporal-effect lags,as_tibble()/as.data.frame()coercion, and the newcount-cumulative->count-incidencesupport.
tbl.now 0.12.0
Batch detection, rebuilt around a conservation law
The report-batch detectors were rebuilt on a single, exact principle: a batch moves reports along the report axis without creating them, so a window of report dates spanning both the lull and the release has an unchanged total, whereas a genuine epidemic surge inflates it. The previous heuristic detect_report_batches() / plot_report_batches() (multi-signal robust-z, and the model-based conditional scan) are removed and replaced by three model-free, r lifecycle::badge("experimental") functions. Each derives its mathematics in a “The mathematics” section of its help page.
- New
batch_test()returns, per (report date, stratum), thedeficit(reports missing beforehand — sensitive to a batch) anddelta(the window total minus its expected value — sensitive to a real surge), and classifies each date as"batch","surge","batch_and_surge","hold_or_deletion"or"none". The transport (batch) test conditions on the window total, so its size does not depend on the unknown incidence nor on the quality of the baseline; the baseline itself is refit from report dates outside each candidate window, which makesdeltainvariant to a within-window batch pathwise. It handles all data types, including"count-cumulative"(signed increments), and takes aperiodargument that absorbs a fixed reporting schedule (weekends, holidays). - New
batch_shape_test()tests whether a flagged report date drew on unusually old event dates, by a permutation rank-sum on the reporting delays. It is exactly distribution-free whenever incidence is locally log-linear. - New
simulate_batch()plants a known batch (a deterministic close-and-release) in atbl_now, for validation and teaching. - New Batch detection article, with worked examples on dengue (a planted batch), FluSight (count-cumulative), and a weekend reporting schedule.
tbl.now 0.10.1
-
autoplot()’s reporting-delay calendar panels (delay_weekday,delay_week,delay_month) are now normalized: each event date’s mean delay is divided by the overall mean delay, so1marks an average delay and a dashed reference line is drawn there. Previously the ungrouped panels plotted the raw mean delay while theby_strata = TRUEpanels were already normalized. They now share one scale, matching the case-count calendar panels and making the calendar pattern comparable across strata (y-axis:"Normalized delay"). -
plot_delay_drift()’swindownow defaults to7periods regardless of the time unit — 7 days for daily data, 7 weeks for weekly data. Previously the default was data-dependent (max(5, n_periods / 20)), which produced a very wide window on long series. Passwindow =to smooth a specific series. - Internal: replaced the remaining base-R data-frame subsetting and column assignment (
df[cond, ],df$col <- ...) outside the converters with the equivalentdplyrverbs (filter(),select(),slice(),mutate()). No user-facing behaviour change. The examples and vignettes now likewise usedplyr::filter()rather than[(e.g.dplyr::filter(batches, batch)).
tbl.now 0.10.0
New
get_nth_reported_cases(): the cumulative cases reported for each event date within a given delay.delay = 0gives the initial snapshot,delay = 1adds the delay-1 reports, and so on;delay = Inf(or the maximum delay) matchesget_latest_reported_cases(). Documented alongsideget_initial_reported_cases()andget_latest_reported_cases().Performance:
get_latest_reported_cases(),get_initial_reported_cases()andget_nth_reported_cases()are substantially faster (~3-4x on the bundled data) — the aggregation now runs on a declassed data frame and thetbl_nowis reconstructed once, with identical output.The experimental diagnostic functions (
plot_delay_drift(),test_delay_drift(),test_delay_changepoint(),detect_report_batches(),plot_report_batches()) now carry a lifecycle experimental badge.test_delay_drift()andtest_delay_changepoint()additionally emit acliwarning that they are experimental, their results are not guaranteed and their interface may change. Flagged batches, change points and trend changes are surfaced as potential (e.g. “potential batches”, “potential change point”).New
detect_report_batches()andplot_report_batches()to detect batch reporting — report dates on which a laboratory releases a backlog of many old cases at once. Working on the report-date axis, it flags a report date using up to four selectable robust-anomaly signals (volume,delay,span,gap), AND-ed together. Requiring thedelay(long/dispersed delays) signal alongsidevolumeis what distinguishes a batch from an epidemic peak: a peak also spikes the report volume, but its cases keep the normal short delay distribution, so its delay score stays low.detect_report_batches()returns a per-report-date table with the features, robust scores and abatchflag;plot_report_batches()shows the report-volume and mean-delay timelines with the flagged dates marked.New
test_delay_changepoint()complementstest_delay_drift(): where the latter tests for a gradual monotonic trend, this tests for a single abrupt change point in the per-period delay summaries using Pettitt’s nonparametric test (implemented directly, no extra dependency). It reports the estimated change date, the before/after level of the statistic, the shift and achangepoint_detectedverdict, per stat (median / mean / IQR / 10-90 spread) and per stratum, on mature data only.plot_delay_drift()gained achangepointargument: set it toTRUEto mark the estimated change point of the median delay on the fan chart with a vertical line.-
New
plot_delay_drift()andtest_delay_drift()to answer “do reporting delay distributions drift over time?”.-
plot_delay_drift()draws a rolling fan chart of the count-weighted delay distribution indexed by event date: a solid rolling median, a dashed rolling mean, and 25-75% / 10-90% quantile bands. The recent, not-yet-fully reported region (after thelevelincompleteness cutoff) is shaded grey so the truncation-induced dip is not mistaken for drift. Supportsby_strata. -
test_delay_drift()runs an autocorrelation-robust monotonic-trend test (Hamed-Rao modified Mann-Kendall by default, with Yue-Pilon and block-bootstrap options via the newmodifiedmkSuggests) on the per-period delay summaries, testing both a location statistic (median/mean) and a dispersion statistic (IQR / 10-90 spread), on mature data only. Returns a tidy tibble with the Kendall tau, Sen’s slope, p-value and adriftverdict, per stat and stratum.
-
autoplot()gained aby_strataargument (defaultFALSE). WhenTRUE, every panel is split by stratum: the calendar and delay boxplots become dodged boxes (one per stratum, side by side), the epidemic process and both periodograms become one coloured line per stratum (no area fill), and the delay distribution becomes dodged bars. Boxplots are normalized per stratum (1 = that stratum’s own average) so the calendar pattern is comparable across strata, and strata are coloured with aviridisscale. A companionstrataargument chooses which columns to group on (defaults to the object’sstrata; pass a subset such asstrata = "gender"to override).autoplot()now draws reporting-delay diagnostic panels alongside the case-count ones, so you can see delay effects: the mean reporting delay by day of week / week of year / month (delay_weekday,delay_week,delay_month), and a periodogram of the mean-delay series (delay_seasonality) that reveals periodicity in the delay itself. The delay panels are computed on the complete part of the series (before the incompleteness line) so recent truncation does not bias them.autoplot()gained apanelsargument to choose which panels to draw. It accepts the concrete panel keys, or the aliases"all"(default),"calendar"and"delay_calendar". Selecting a single panel returns it as a plainggplot2object instead of apatchwork. Unknown panels error; panels that do not apply to the data’s time unit are skipped with a warning.New pkgdown article “One dataset, many nowcasts” now also demonstrates that temporal (delay) effect columns are carried into
epinowcast(metareference/metareport),baselinenowcast(long) andepidist, with a table clarifying which target formats can hold covariates and how each model can use them.temporal_effects()gained an after-holiday and after-weekend effect via the newholiday_lagsandweekend_lagsarguments. Each takes a non-negative integer depthN; materialising the spec then adds indicator columns..._holiday_lag_1 … ..._holiday_lag_N(and likewise..._weekend_lag_k) that flag dates falling exactlykworking days after a holiday / weekend. Working days skip weekends and holidays, so the effect lands on the first day(s) back at work — designed to capture the rise in cases just after a holiday or weekend.holiday_lagsrequires aholidayscalendar. The columns are picked up automatically by everytbl_now_to_*()converter (as covariate columns) and bydiseasenowcasting::nowcast().Documented and tested attaching temporal effects to the report date (in addition to the default event date) via
add_temporal_effects(x, spec, date_type = "report_date"). Event- and report-date effects can coexist on the sametbl_now; both sets of columns (.event_*and.report_*) are carried through all converters.
tbl.now 0.9.0
- Added
as_tibble()andas.data.frame()methods fortbl_nowwith an opt-incompute_temporal_effectsargument (defaultFALSE). Passingcompute_temporal_effects = TRUEmaterialises the lazytemporal_effects()spec (holidays, Fourier terms, calendar effects) into columns before returning a plaintibble/data.frame; the inputtbl_nowis left unchanged. The default stays lazy on purpose, becausedplyrrelies on these coercions being cheap, non-materialising declassers internally (e.g.group_by()). - The
tbl_now_to_*()converters now carry the (lazy) temporal-effect columns (holidays, Fourier seasonal terms, day-of-week / calendar effects) into the target format as covariate columns. The spec is materialised on demand viacompute_temporal_effects()at conversion time (the inputtbl_nowis left unchanged), and the columns are passed todata.table,tsibble,baselinenowcastlong format,epidist, andepinowcast(where they appear in the observations andmetareferencetables for use in the reference module). Thebaselinenowcastreporting-triangle matrix still cannot hold them. - Removed the
|>export and changed all the pipes to|> - Refactored
converters.Rfor readability (dplyr column operations instead of base indexing, full variable names, lintr-clean). - The
tbl_now_to_*()converters now keep thecovariatesandis_censoredcolumns wherever the target format can hold them (data.table,tsibble,baselinenowcastlong format,epidistlinelist); the fixed modelling objects (enw_preprocess_data, the reporting-triangle matrix, the EpiNow2 series) still cannot carry them. - Added S3 methods on the other packages’ coercion generics so they accept a
tbl_nowdirectly:as_epidist_linelist_data(),as_reporting_triangle(),as_tsibble()andas.data.table(), each wrapping the matchingtbl_now_to_*(). - Fixed
tbl_now_to_data_table()checking forbaselinenowcastinstead ofdata.table, andtbl_now_to_baselinenowcast(format = "long")no longer requiringbaselinenowcastto be installed.
tbl.now 0.8.0
- Modified the
updateas thet_effectargument was not doing anything. - Fixed bug that errored
complete_zeroeswhenis_censoredwas given. - Removed explicit zeroes from the converters (
tbl_now_from_*) as they are not necessary intbl_now. - Added
censor_reporting_delays_above()to flag reports with an implausibly long delay as censored (their delay becomes an upper bound). - Improved documentation and README
- Documented all internal functions with roxygen (
@keywords internal+@noRd) and ensured every exported function has a@return. - Homogenized
lifecyclebadges. - Brought the
censor_reporting_delays_abovefunction fromdiseasenowcastingtotbl_now. -
tbl_now_from_epinowcast()now accepts not only the raw long input but also a preprocessedenw_preprocess_dataobject or a fittedepinowcastobject (grouping auto-detected), matching the formatepinowcastuses for summaries and plots. -
tbl_now_to_EpiNow2()gained amodelargument:"estimate_infections"(default, the singledate/confirmseries) and"estimate_truncation"(a list of report-date snapshots, the one EpiNow2 model that uses the report dimension). Documentation clarified accordingly. - Fixed two converter
requireNamespace()guards:tbl_now_to_data_table()checked forbaselinenowcastinstead ofdata.table, andtbl_now_to_baselinenowcast(format = "long")no longer requiresbaselinenowcastto be installed.
tbl.now 0.7.5
- Bumped roxygen to version 8.0.0. This also resulted in updated documentation.
- Changed
autoplot()’s default level to 0.95 - Added tests for converters and pillars.
- Throws warning when converting to
baselinenowcastif data is"count-cumulative".
tbl.now 0.7.3
- Added the
update_now()function to make it more intuitive to update the now.
tbl.now 0.7.0
- Added an
autoplot()method fortbl_nowobjects that produces a multi-panel diagnostic overview: the empirical delay distribution, the observed epidemic process with an incompleteness line (controlled bylevel), normalized calendar-effect boxplots (cases relative to the overall mean), and a periodogram to help choose Fourierseasons. Daily data shows both a day-of-week and a week-of-year boxplot panel; weekly data shows week-of-year. Built onggplot2andpatchwork. The x-axis limits of each panel can be set individually (delay_distribution_xlim,event_date_xlim,calendar_effect_xlim,seasonality_xlim), and holidays from the temporal-effects spec are marked with red dots on the epidemic process. - Added converters to and from other packages, all of the form
tbl_now_from_*()/tbl_now_to_*():epinowcast,baselinenowcast,EpiNow2(to only),epidist,data.tableandtsibble. Thetbl_now_from_*()functions wrapas_tbl_now()and forward...totbl_now(); thetbl_now_to_*()functions call into the target package. All accept averboseargument that reports the choices made (the inferrednow, data type, units, and column mapping). -
as_tbl_now()gained methods for the classes produced bytbl_now_to_*()(enw_preprocess_data,reporting_triangle,epidist_linelist_data,tbl_tsanddata.table), so a converted object can be turned straight back into atbl_now. - Documented
autoplot()and the converters in the introduction vignette.
tbl.now 0.6.4
- Fixed dependency on R >= 4.2.0
- Update function now defaults the censoring to FALSE if the update is censored but the original is not.
tbl.now 0.6.2
- Removed warning when using columns for temporal effects that cascaded into
to_count. - Changed DESCRIPTION to fix ortographic error and trigger less messages of unknown words.
tbl.now 0.6.0
- Changed temporal effects to be lazy (as required by #17) so that now its easier to use
dplyrfunctions without compromising them. - Bumped the deprecated dplyr’s
*_atfunctions to useall_of() - Fixed to no warnings during test.
- Users can now pass the
.delaycolumn directly (#6) and it will recalculate the missing column (i.e. event or report) - Added
complete_zeroesto vignette (#13).