Package 'tbl.now'

Description

A tbl_now() remembers which of its columns are the event date, the report date, the strata, and so on. These functions edit that memory after the object has been built – useful when new columns appear part-way through a pipeline, or when you want to nowcast the same data broken down a different way.

There are three verbs, and they differ only in what they do to what is already recorded:

• ⁠add_*()⁠ keeps what is there and adds to it. add_strata(x, age_group) on an object already stratified by gender leaves you stratified by both.

• ⁠change_*()⁠ replaces it. change_strata(x, age_group) on the same object leaves you stratified by age group only.

• ⁠remove_*()⁠ takes it away. remove_all_strata(x) forgets every stratum; remove_strata(x, gender) forgets just that one.

Nothing here touches the data itself. Renaming an attribute does not rename, create or delete a column – it only changes which existing column the object treats as playing that role.

Usage

change_now(x, now = NULL, verbose = TRUE)

update_now(x, verbose = TRUE)

change_event_date(x, event_date)

change_report_date(x, report_date)

change_case_count(x, case_count)

change_is_censored_report(x, is_censored_report)

remove_is_censored_report(x)

add_is_censored_report(x, is_censored_report)

change_strata(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

remove_strata(x, ...)

add_strata(x, ...)

remove_all_strata(x)

change_covariates(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

remove_covariates(x, ...)

add_covariates(x, ...)

remove_all_covariates(x)

change_event_covariates(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

change_report_covariates(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

change_revision_covariates(x, ..., warn_now = TRUE, warn_non_uniqueness = TRUE)

add_event_covariates(x, ...)

add_report_covariates(x, ...)

add_revision_covariates(x, ...)

remove_event_covariates(x, ...)

remove_report_covariates(x, ...)

remove_revision_covariates(x, ...)

remove_all_event_covariates(x)

remove_all_report_covariates(x)

remove_all_revision_covariates(x)

replace_temporal_effects(x, t_effects)

remove_temporal_effects(x)

change_is_censored_revision(x, is_censored_revision)

add_is_censored_revision(x, is_censored_revision)

remove_is_censored_revision(x)

add_revision_date(
  x,
  revision_date,
  revision_type = NULL,
  revision_units = "auto",
  revision_levels = NULL
)

change_revision_date(
  x,
  revision_date,
  revision_type = NULL,
  revision_units = "auto",
  revision_levels = NULL
)

remove_revision_date(x)

Arguments

Details

Columns are chosen with tidy-select, so a bare column name works, and so do the helpers dplyr::starts_with(), dplyr::all_of() and dplyr::where(). See dplyr::select() for the full set.

update_now() deserves a note of its own. The now of a nowcast is the day you are standing on, and it does not move when you filter the data – an object filtered down to 1992 still believes now is 2010, which is usually not what you want. update_now() resets it to the latest date actually present:

data(denguedat)
ndata <- tbl_now(denguedat,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

# `now` is in 2010, because the data runs that far.
get_now(ndata)
#> [1] "2010-12-20"

# Filtering the data does not move it ...
ndata_1992 <- ndata |>
  dplyr::filter(
    onset_week <= as.Date("1992/01/01") & report_week <= as.Date("1992/01/01")
  )
get_now(ndata_1992)
#> [1] "2010-12-20"

# ... but `update_now()` does.
get_now(update_now(ndata_1992))
#> [1] "1991-12-30"

Value

A tbl_now object with the attribute updated. The data are returned unchanged; only what the object records about itself differs.

The revision process, the optional third date

add_revision_date(), change_revision_date() and remove_revision_date() set the third date a surveillance record can carry: after the event happened and after it was reported, somebody decided whether it was real. For influenza that is the laboratory result – and it can come back negative, in which case the case is retracted rather than confirmed.

Attaching one is the only verb on this page that changes more than a name:

• now moves. A revision is an observation, so the as-of moment becomes the latest of the report and revision dates. Revision refuses an object whose now falls before a revision that has already happened.

• Two columns appear. .revision_num is the date on the same numeric anchor as .event_num/.report_num; .revision_delay is the time from report to resolution. Both are protected, like .delay.

• Counting gains a dimension. to_count() groups by the revision date and outcome as well, so a confirmed and a retracted case on the same ⁠(event, report)⁠ pair stay separate rather than being summed together.

• The timeline is checked. ⁠event_date <= report_date <= revision_date⁠; rows that break it are warned about, not silently accepted.

A date on its own cannot say whether the test came back positive or negative, so leaving revision_type out gives every dated row NA and warns.

Two optional pieces travel with the third date. revision_levels is a named dictionary translating the labels in your data into the four values revision_type may hold – c(confirmado = "confirmed", ...) – so the recoding happens once rather than in every script. And add_is_censored_revision() names a logical column marking rows whose revision delay is a bound rather than a measurement, the revision-axis twin of add_is_censored_report(); censor_revision_delays_above() sets it for you.

change_now() is revision-aware in both directions. Moving now forward does nothing to the data; moving it backwards, which is how a backtest asks what was known at an earlier date, returns every revision dated after that moment to "pending" and masks its date. A resolution that has not happened yet is not a resolution.

See Also

tbl_now() for setting these when the object is first built; the getters for reading them back; tbl_now_attributes() to list them all at once; add_temporal_effects() and temporal_effects() for calendar structure; update() for appending new rows rather than editing attributes.

Examples

data(denguedat)
# These verbs only touch what the object records about itself, never the
# rows, so a couple of years stands in for the full twenty-year series.
recent <- denguedat[denguedat$onset_week >= as.Date("2009-01-01"), ]
ndata <- tbl_now(recent,
  event_date = onset_week,
  report_date = report_week,
  strata = gender,
  verbose = FALSE
)

## ---- Strata: add, change, remove ------------------------------------

ndata$age_group <- sample(c("<18", "18-60", "60+"), nrow(ndata), replace = TRUE)

## `add_strata()` keeps gender and adds age group.
get_strata(add_strata(ndata, age_group))

## `change_strata()` replaces gender with age group.
get_strata(change_strata(ndata, age_group))

## `remove_strata()` drops one; `remove_all_strata()` drops the lot.
get_strata(remove_strata(ndata, gender))
get_strata(remove_all_strata(add_strata(ndata, age_group)))

## ---- Covariates behave the same way ---------------------------------

# Covariates influence the nowcast but are not of interest in themselves.
# They should be values known at the fitted object's `now`; do not let
# future realized covariates leak into a backtest snapshot.
ndata$temperature <- rnorm(nrow(ndata), 25, 4)
ndata$humidity <- rbeta(nrow(ndata), 0.6, 0.4)
ndata <- ndata |> add_covariates(temperature, humidity)
get_covariates(ndata)

ndata |>
  remove_covariates(humidity) |>
  get_covariates()

## Covariates can also be tagged by model component.
ndata <- ndata |>
  add_event_covariates(temperature) |>
  add_report_covariates(humidity)
get_event_covariates(ndata)
get_report_covariates(ndata)

## ---- Pointing an attribute at a different column ---------------------

## Suppose onset was recorded a day late and you correct it. `change_event_date()`
# tells the object to use the corrected column instead.
ndata$corrected_onset <- ndata$onset_week - lubridate::days(1)
ndata <- ndata |> change_event_date(corrected_onset)
get_event_date(ndata)

## ---- The censoring indicators ----------------------------------------

## TRUE means the report date is only an upper bound (e.g. a backlog dump).
ndata$is_censored_report <- FALSE
ndata <- ndata |> add_is_censored_report(is_censored_report)
get_is_censored_report(ndata)
ndata <- remove_is_censored_report(ndata)

## ---- `now` -----------------------------------------------------------

# Set it by hand ...
get_now(change_now(ndata, now = as.Date("2011-01-01")))

# ... or snap it back to the latest date actually observed.
get_now(update_now(ndata))

## ---- Count data: which column holds the counts ------------------------

counts <- to_count(ndata, to = "count-incidence")
counts |>
  dplyr::mutate(inflated = round(1.15 * n)) |>
  change_case_count(inflated) |>
  get_case_count()

## ---- The revision process, the optional third date -----------------

data(covid_us)
covid <- covid_us |>
  dplyr::filter(onset_dt >= as.Date("2020-12-01")) |>
  tbl_now(
    event_date = onset_dt, report_date = pos_spec_dt,
    case_count = n, data_type = "count-incidence",
    verbose = FALSE, warn_non_uniqueness = FALSE
  )

## Onset -> positive specimen -> registration at CDC. A date alone cannot say
# how the case resolved, so this warns until an outcome column is supplied.
covid <- suppressWarnings(add_revision_date(covid, cdc_report_dt))
get_revision_date(covid)

## CDC's own labels are not this package's four, which is what
# `revision_levels` translates.
covid <- change_revision_date(covid, cdc_report_dt,
  revision_type = current_status,
  revision_levels = c(
    "Laboratory-confirmed case" = "confirmed", "Probable Case" = "pending"
  )
)
table(covid[[get_revision_type(covid)]])
get_revision_levels(covid)

## A revision delay you refuse to believe is a bound, not a measurement.
covid <- censor_revision_delays_above(covid, 45, verbose = FALSE)
get_is_censored_revision(covid)

## Dropping the third date leaves an ordinary two-date object.
has_revision(remove_revision_date(covid))

## ---- Temporal effects --------------------------------------------------

# Recorded lazily: `replace_*` swaps the specification, `remove_*` forgets it.
ndata <- ndata |>
  add_temporal_effects(
    t_effects = temporal_effects(week_of_year = TRUE, month_of_year = TRUE)
  )
get_temporal_effects(ndata)

ndata |>
  replace_temporal_effects(t_effects = temporal_effects(seasons = 52)) |>
  get_temporal_effects()

ndata |>
  remove_temporal_effects() |>
  get_temporal_effects()

Description

These are the second and third steps of using calendar structure in a nowcast. First you write down which patterns you want with temporal_effects(); then:

• add_temporal_effects() records that request on the object. Nothing is computed and no columns appear – the specification is stored lazily, so it survives filtering and joining without going stale.

• compute_temporal_effects() materialises it, building one column per effect from the object's dates.

The split matters because the columns depend on the data. If you computed them first and then filtered, or changed the event-date column, the columns would silently describe the wrong rows. Recording the request and computing it at the end avoids that.

Call add_temporal_effects() more than once to accumulate several specifications on the same object; compute_temporal_effects() builds all of them.

Usage

add_temporal_effects(x, t_effects = NULL, overwrite = FALSE, ...)

## S3 method for class 'data.frame'
add_temporal_effects(
  x,
  t_effects = NULL,
  overwrite = FALSE,
  ...,
  date_col = NULL,
  numeric_col = NULL,
  name_prefix = paste0(".", date_col),
  weekend_days = c("Sat", "Sun"),
  units = "days"
)

## S3 method for class 'tbl_now'
add_temporal_effects(
  x,
  t_effects = NULL,
  overwrite = FALSE,
  ...,
  date_type = "event_date",
  weekend_days = c("Sat", "Sun")
)

compute_temporal_effects(x, overwrite = FALSE)

Arguments

Value

add_temporal_effects() returns the object with the specification recorded. For a tbl_now no columns are added; for a plain data.frame, which has nowhere to record a specification, the columns are computed immediately.

compute_temporal_effects() returns a tbl_now with one column per effect appended. The specification is kept, so it still prints; the names of the columns just created are available from get_temporal_effect_cols().

See Also

temporal_effects() to build the specification, and for what each effect means; replace_temporal_effects() and remove_temporal_effects() to swap or drop it; get_temporal_effects() and get_temporal_effect_cols() to read back the request and the columns; calendar_effect_plots to see the patterns; add() for the other attribute setters.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week",
  report_date = "report_week",
  strata = "gender",
  verbose = FALSE
)

# Step 1-2: say you want a week-of-year effect, and record it.
dengue <- dengue |>
  add_temporal_effects(t_effects = temporal_effects(week_of_year = TRUE))

# The request is stored, but no column has been built yet.
get_temporal_effects(dengue)
get_temporal_effect_cols(dengue)

# Step 3: materialise it.
computed <- compute_temporal_effects(dengue)
get_temporal_effect_cols(computed)
head(computed[[get_temporal_effect_cols(computed)[1]]])

# Specifications accumulate, so you can add a second pattern ...
both <- dengue |>
  add_temporal_effects(t_effects = temporal_effects(month_of_year = TRUE))
get_temporal_effect_cols(compute_temporal_effects(both))

# ... swap the whole specification for another ...
dengue |>
  replace_temporal_effects(t_effects = temporal_effects(seasons = 52)) |>
  get_temporal_effects()

# ... or forget it entirely.
dengue |>
  remove_temporal_effects() |>
  get_temporal_effects()

Description

Daily surveillance data is often too sparse to nowcast: most (event date, report date) cells hold a zero or a one, and the delay distribution is mostly noise. The usual fix is to work in weeks instead. aggregate_time_units() does that in one call – it moves every date onto the coarser grid, adds the counts up, and returns a tbl_now that knows it is now weekly, so .delay, the converters and the models all count in weeks from then on.

Usage

aggregate_time_units(
  x,
  to = "weeks",
  axes = "all",
  label = c("start", "end"),
  align_on_day = 7,
  type = "epi",
  verbose = TRUE
)

Arguments

Details

Each date is replaced by the start of the period it falls in: the Sunday that begins its epidemiological week, the first of its month, the first of January of its year (label = "end" names the period by its last day instead). Weeks go through the same epi/ISO machinery align_weeks() uses, so type and align_on_day mean exactly what they mean there.

What happens to the rows depends on the data type:

• linelist – one row is still one case; only the dates move.

• count-incidence – rows that land in the same (event, report) cell are summed.

• count-cumulative – cumulative totals are not additive, so the series is de-accumulated to increments first, aggregated, and accumulated again on the new grid.

Two consequences worth knowing:

• Weeks do not nest inside months. Aggregating daily data to weeks and then to months is not the same as going straight to months: the second pass sees only the week's label, so a week beginning 31 December lands in December even though most of its cases happened in January. Aggregate once, to the unit you actually want.

• An NA count means not yet observed, so a period containing one has an unknown total, not a total that quietly leaves it out. Use complete_zeroes() first if the NAs are really zeroes.

Value

A tbl_now on the coarser grid, with event_units, report_units and revision_units updated for the axes that were aggregated, and now moved onto the new grid.

What happens to the temporal effects

Any temporal-effect columns that were materialised by compute_temporal_effects() are dropped, because a day-of-week term computed on daily dates is meaningless once those dates are weeks.

The lazy temporal_effects() specification moves onto the new grid with everything else, and the effects the new grid cannot express are dropped from it rather than silently rebuilt on dates that cannot carry them:

• day_of_week, weekend, day_of_month, holiday_lags and weekend_lags are properties of a day, so they survive only to = "days".

• week_of_year survives "weeks"; month_of_year survives "months".

• seasons are rescaled. A Fourier period is a length, not a position in the calendar, so it converts: a 365-day season becomes a 52.14-week one, and ⁠seasons = 52, season_length = 7⁠ becomes seasons = 52 in weeks. A period that ends up two units or shorter is dropped – it is at or below the new grid's Nyquist limit, so it can no longer be told from a constant or from a longer wave.

• holidays are kept. On a coarser grid the holiday column stops being a 0/1 indicator and becomes the share of the period's days that the calendar marks – 1/7 for a week containing Christmas Day – which is the same number as the indicator when the period is one day long.

A specification with nothing left is removed. verbose = TRUE says what was dropped and what was rescaled.

Aggregating one axis only

axes exists because the two axes do not always move together: a system may record the day a specimen was taken but only publish weekly report batches. Two things follow:

• tbl_now() requires the report axis to be at least as coarse as the event axis, so aggregating only the event axis of a daily object is refused. Aggregate the report axis too, or use axes = "all".

• A week named by the day it starts sits before every date inside it, so coarsening a later axis alone – the report against a daily event, or the revision against a daily report – with label = "start" produces negative delays, and validate_tbl_now() warns. Use label = "end": a report that arrived somewhere in week W is known by the end of W, which is the honest bound. When the axes move together the labelling cancels out and either choice gives the same delays.

See Also

align_weeks(), which snaps weekly dates to a common weekday without changing the units or the counts; to_count() to change data type without touching the dates; complete_zeroes() for the cells the coarser grid still leaves empty; tbl_now()'s units argument to declare the units up front; temporal_effects() and compute_temporal_effects() for the specification this coarsens.

Examples

# A sparse daily line list: one or two cases a day.
df <- data.frame(
  onset = as.Date("2024-01-01") + c(0, 1, 3, 8, 9, 15),
  reported = as.Date("2024-01-01") + c(2, 2, 5, 9, 12, 16),
  sex = c("F", "M", "F", "M", "F", "M")
)
daily <- tbl_now(df,
  event_date = onset, report_date = reported, strata = sex,
  data_type = "linelist", units = "days", verbose = FALSE
)
get_event_units(daily)

# The same cases, on a weekly grid: every date moves to the Sunday that
# starts its epidemiological week, and the delays are whole weeks.
weekly <- aggregate_time_units(daily, to = "weeks", verbose = FALSE)
get_event_units(weekly)
weekly[[get_event_date(weekly)]]
weekly$.delay

# Count data is added up rather than merely relabelled.
counts <- to_count(daily, to = "count-incidence")
sum(counts$n)
sum(aggregate_time_units(counts, to = "months", verbose = FALSE)$n)

# A weekly grid cannot carry a day-of-week effect, so it is dropped from the
# specification; the 365-day season is rescaled to 52.14 weeks instead.
spec <- daily |>
  add_temporal_effects(
    temporal_effects(day_of_week = TRUE, seasons = c(7, 365))
  )
get_temporal_effects(aggregate_time_units(spec, to = "weeks", verbose = FALSE))

Description

Weekly surveillance data is rarely as tidy as it looks. The same series may be stamped with a Wednesday one year and a Thursday the next, or event dates may fall on a Sunday while reports fall on a Saturday. When that happens the delay between the two stops being a whole number of weeks – you get delays of 2.86 weeks – and most nowcasting models, which count in whole periods, either refuse the data or quietly round it.

align_weeks() snaps every date to the same weekday, so week differences come out as integers. week_2_date() solves the neighbouring problem: you have epiweek (or ISO week) numbers rather than dates, and need real dates to build a tbl_now from.

Usage

align_weeks(.data, align_on_day = 7, type = "epi", ...)

## S3 method for class 'data.frame'
align_weeks(
  .data,
  align_on_day = 7,
  type = "epi",
  ...,
  date_col,
  new_date_col = NULL
)

## S3 method for class 'tbl_now'
align_weeks(.data, align_on_day = 7, type = "epi", ...)

week_2_date(
  .data,
  week_col,
  year_col,
  align_on_day = 7,
  week_fun = lubridate::epiweek,
  year_fun = lubridate::epiyear,
  date_col_name = "date"
)

Arguments

Details

Applied to a data.frame, align_weeks() adds an aligned copy of the column you name. Applied to a tbl_now, it aligns the event and report dates together and recomputes the delay, so the object stays coherent.

Epi weeks and ISO weeks disagree about where a year starts, so type picks which convention to use: "epi" uses lubridate::epiweek() / lubridate::epiyear(), "iso" uses lubridate::isoweek() / lubridate::isoyear().

Value

align_weeks() returns its input with an aligned date column added (data.frame method), or a tbl_now whose dates have been aligned and whose .delay has been recomputed.

week_2_date() returns the input data.frame with a new date column appended.

Note

Useful whenever week boundaries differ between systems or between years, which is the normal state of affairs for epiweek and ISO week data.

See Also

tbl_now(), whose align_weeks = TRUE argument does this at construction time; is_weekday(), which numbers weekdays the same way; complete_zeroes() for filling the weeks where nothing was reported; temporal_effects() for using week-of-year as a model term.

Examples

## ---- Plain data frames ------------------------------------------------

# Three dates falling on different weekdays.
df <- data.frame(date = as.Date(c("2022-11-02", "2022-11-07", "2022-11-13")))
weekdays(df$date)

# Snap them all back to the Sunday that starts their week.
aligned <- align_weeks(df, date_col = date)
aligned
weekdays(aligned$date_aligned)

# Or to Tuesday. Weekday numbers are ISO: 1 = Monday, so Tuesday is 2.
align_weeks(df, date_col = date, align_on_day = 2)

## ---- A tbl_now: making the delays whole numbers -------------------------

data(flusight)

# One state is enough to see the problem.
texas <- flusight[flusight$location_name == "Texas", ]
flutbl <- tbl_now(texas,
  event_date = "target_end_date",
  report_date = "as_of", case_count = "observation",
  strata = "location_name", verbose = FALSE
)

# `as_of` is sometimes a Saturday and sometimes a Wednesday, so some delays
# land between whole weeks.
mean(flutbl$.delay != round(flutbl$.delay))

# After aligning, every delay is a whole number of weeks.
flutbl <- align_weeks(flutbl)
mean(flutbl$.delay != round(flutbl$.delay))

## ---- Week numbers instead of dates --------------------------------------

# Data reported as "week 1 of 2024" and so on, with no usable date column.
df <- data.frame(
  epidemiological_week = 1:5,
  epidemiological_year = rep(2024, 5)
)

## week_2_date() turns those into the Sunday that starts each epiweek.
week_2_date(df,
  week_col = epidemiological_week,
  year_col = epidemiological_year
)

Description

Convert a supported object into a tbl_now. For a plain data.frame / data.table (or an existing tbl_now) you supply the event_date and report_date columns yourself. For objects produced by other packages the conversion is delegated to the matching ⁠tbl_now_from_*()⁠ converter, which already knows how to map that format – so those methods do not take event_date / report_date.

Usage

as_tbl_now(object, ...)

## S3 method for class 'tbl_now'
as_tbl_now(object, event_date, report_date, ...)

## S3 method for class 'data.frame'
as_tbl_now(object, event_date, report_date, ...)

## S3 method for class 'enw_preprocess_data'
as_tbl_now(object, ...)

## S3 method for class 'reporting_triangle'
as_tbl_now(object, ...)

## S3 method for class 'epidist_linelist_data'
as_tbl_now(object, ...)

## S3 method for class 'epidist_aggregate_data'
as_tbl_now(object, ...)

## S3 method for class 'tbl_ts'
as_tbl_now(object, report_date, event_date = NULL, ...)

## S3 method for class 'data.table'
as_tbl_now(object, event_date, report_date, ...)

## S3 method for class 'tbl_now_epinow2_snapshots'
as_tbl_now(object, ...)

## S3 method for class 'tbl_now_triangle_list'
as_tbl_now(object, ...)

## S3 method for class 'tbl_now_surveillance_list'
as_tbl_now(object, ...)

Arguments

Details

Package-specific inputs forward to a dedicated converter. See that converter for the extra arguments it accepts (e.g. strata, max_delay, format, delays_unit, ...), for which columns are carried over, and for the transformation notes / round-trip caveats in its Round-trip section:

• enw_preprocess_data (or a fitted epinowcast object) -> tbl_now_from_epinowcast()

• reporting_triangle (baselinenowcast) -> tbl_now_from_baselinenowcast()

• epidist_linelist_data / epidist_aggregate_data (epidist) -> tbl_now_from_epidist()

• tbl_ts (tsibble) -> tbl_now_from_tsibble()

• data.table -> tbl_now_from_data_table()

Anything passed through ... is forwarded to the underlying converter (and on to tbl_now()), so options such as event_units, now or verbose can be supplied here too.

Value

A tbl_now object.

See Also

tbl_now() to build one from scratch; the converters this dispatches to – tbl_now_from_epinowcast(), tbl_now_from_baselinenowcast(), tbl_now_from_epidist(), tbl_now_from_tsibble(), tbl_now_from_data_table() – and the ⁠tbl_now_to_*()⁠ functions that go the other way. The One dataset, many nowcasts article shows the round trip against each modelling package.

Examples

## For a plain data.frame this is a synonym for tbl_now(): you name the
# columns yourself.
data(denguedat)
as_tbl_now(denguedat, event_date = "onset_week", report_date = "report_week")

# For an object built by another nowcasting package you often do not name them,
# because that format already fixes which column is the event date and which
# is the report date. Here we send a tbl_now out to tsibble and bring it back.
if (requireNamespace("tsibble", quietly = TRUE)) {
  ndata <- tbl_now(denguedat,
    event_date = onset_week, report_date = report_week, verbose = FALSE
  )
  ts <- suppressWarnings(tbl_now_to_tsibble(ndata, verbose = FALSE))

  # Bare names and strings both work.
  as_tbl_now(ts, event_date = onset_week, report_date = report_week)
}

Description

tibble::as_tibble() and as.data.frame() methods for tbl_now. They drop the tbl_now class and metadata, returning a plain tibble / data.frame.

Set compute_temporal_effects = TRUE to materialise the lazy temporal_effects() specification with compute_temporal_effects() first, so the holiday / Fourier / calendar columns are present in the result. The input tbl_now is left unchanged (the spec is materialised on a copy).

The default is FALSE on purpose: dplyr uses these coercions internally as cheap declassers (for example inside group_by() and the data mask behind mutate() / filter() / slice()). Materialising there would both break the lazy temporal_effects contract and recurse, so materialising is strictly opt-in. To get a modelling-ready frame with the effects computed, either pass compute_temporal_effects = TRUE here or call compute_temporal_effects() on the tbl_now beforehand.

Usage

## S3 method for class 'tbl_now'
as_tibble(x, ..., compute_temporal_effects = FALSE)

## S3 method for class 'grouped_tbl_now'
as_tibble(x, ..., compute_temporal_effects = FALSE)

## S3 method for class 'tbl_now'
as.data.frame(x, ..., compute_temporal_effects = FALSE)

## S3 method for class 'grouped_tbl_now'
as.data.frame(x, ..., compute_temporal_effects = FALSE)

Arguments

Value

A tibble (for as_tibble()) or a data.frame (for as.data.frame()); it carries the temporal-effect columns when compute_temporal_effects = TRUE.

See Also

compute_temporal_effects(), temporal_effects()

Examples

data(denguedat)
df_now <- tbl_now(denguedat,
  event_date  = onset_week,
  report_date = report_week,
  t_effects   = temporal_effects(week_of_year = TRUE),
  verbose     = FALSE
)

## Plain coercion leaves the spec lazy (no temporal-effect columns):
".event_week_of_year" %in% names(tibble::as_tibble(df_now)) # FALSE

# Opt in to materialise the holiday / Fourier / calendar columns:
as_tibble(df_now, compute_temporal_effects = TRUE)
as.data.frame(df_now, compute_temporal_effects = TRUE)

Description

Turns a fitted nowcast into an ordinary tibble you can plot, join or write out: one row per event date and quantile level by default, or one row per posterior draw with type = "draws".

Not every engine keeps draws. When the backend returned only summarised quantiles, type = "draws" has nothing to give you.

Usage

## S3 method for class 'tbl_nowcast'
as_tibble(x, ..., type = c("quantiles", "draws"))

Arguments

Details

Registered in .onLoad() rather than assigned with ⁠S7::method(as_tibble, tbl_nowcast) <- ⁠, which is what the neighbouring print() and as.data.frame() methods use. Assigning an S7 method onto an imported generic copies that generic into this namespace, and tibble::as_tibble() carries rownames = pkgconfig::get_config(...) as a default argument – so the copy makes ⁠R CMD check⁠ report pkgconfig as an undeclared :: import of a package this one never uses. Plain S3 registration dispatches on class(x), which for an S7 object is "tbl.now::tbl_nowcast", and copies nothing.

The registered class string cannot be spelled as a method name – there is no writable as_tibble.tbl.now::tbl_nowcast – but the function itself is still named for the method it implements, so that ⁠R CMD check⁠ can resolve this topic's usage section back to an object that exists.

Value

A tibble. For type = "quantiles", one row per event date, stratum and quantile level, with the quantile level in .quantile_level and the predicted count in .value. For type = "draws", one row per draw.

See Also

tidy() for the same content with the engine's metadata attached; autoplot() to plot it; score_nowcast() to score it against observed counts.

Examples

predictions <- data.frame(
  onset_week = as.Date("2020-01-05"),
  .quantile_level = c(0.5, 0.9), .value = c(10, 14)
)
nc <- tbl_nowcast(predictions = predictions, method = "toy", event_date = "onset_week")
tibble::as_tibble(nc)

Description

A tbl_now() is a tibble with extra attributes recording which column is the event date, which is the report date, and so on. These methods make sure those attributes survive ordinary base-R manipulation, so that x[1:10, ], names(x) <- ... and x$new <- ... give you back a tbl_now rather than a bare data frame.

You never call them directly – they are what makes the operators work.

Usage

## S3 method for class 'tbl_now'
x[...]

## S3 method for class 'grouped_tbl_now'
x[...]

## S3 replacement method for class 'tbl_now'
names(x) <- value

## S3 replacement method for class 'grouped_tbl_now'
names(x) <- value

## S3 replacement method for class 'tbl_now'
x$name <- value

## S3 replacement method for class 'grouped_tbl_now'
x$name <- value

Arguments

Details

When an operation leaves the object unable to describe a nowcast – because it dropped or renamed the event-date column, say – the class cannot honestly be kept. In that case the result is demoted to a plain data frame (with a warning), rather than pretending to still be a tbl_now. Attributes that are still meaningful are preserved on the way down.

The same applies to dplyr verbs, through dplyr_reconstruct().

Value

A tbl_now object, or a plain data frame when the operation inrevised the class.

See Also

tbl_now() for the attributes being preserved; tbl_now_attributes() to check what survived; as_tibble() to drop the class on purpose; validate_tbl_now() to confirm the result is still well formed.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = onset_week, report_date = report_week,
  strata = gender, verbose = FALSE
)

# Subsetting rows keeps the class and everything it knows.
small <- dengue[1:10, ]
class(small)[1]
get_event_date(small)

# So does adding a column with `$<-`.
dengue$season <- ifelse(
  lubridate::month(dengue$onset_week) %in% 6:11, "wet", "dry"
)
class(dengue)[1]

# And renaming an unimportant column with `names<-`.
renamed <- dengue
names(renamed)[names(renamed) == "season"] <- "period"
get_event_date(renamed)

# But dropping the event date leaves nothing a nowcast could use, so the
# object is demoted to a plain tibble instead of lying about itself.
demoted <- suppressWarnings(dengue[, c("report_week", "gender")])
class(demoted)[1]

Description

Produces a multi-panel diagnostic overview of a tbl_now using ggplot2::ggplot() and patchwork. The gallery is a matrix with one column per process: the case counts on the left, the reporting delay next to them, and — when the object declares a revision axis — the revision process on the right. Each row asks the same question of every process, so an object with two dates comes out two columns wide and one with three dates three columns wide. You choose which panels to draw with the panels argument, and the number of columns follows: a selection covering one family only (panels = "calendar") is one column.

A row a process cannot answer — weekly data has no day-of-week panel — leaves that cell empty rather than closing the gap, so the columns keep their meaning all the way down.

Case-count panels

• "delay_distribution" — a (case-count weighted) histogram of the reporting delay (.delay). For count-cumulative data this panel instead shows the cumulative growth by delay: boxplots (on a log scale, with a dashed reference at 1) of the ratio of each event date's cumulative count at a delay to its cumulative count at the previous delay. A ratio above 1 is an upward revision, below 1 a downward one, and the boxes converge to 1 as reporting completes.

• "epidemic" — the latest reported case counts per event_date, with a dashed vertical line marking where the data become incomplete (less than level of the delay distribution has arrived). Holidays from the attached temporal_effects() spec are marked with dots.

• "calendar_weekday", "calendar_week", "calendar_month" — boxplots of the normalized case effect (each event date's cases divided by the overall mean, so 1 is average) by day of week, epidemiological week, or month.

• "calendar_holiday" — the same normalized boxplots by day type, titled "Weekend and/or holiday effects" because that is what the day types are. The categories follow the attached temporal_effects() spec: a holiday calendar and a weekend effect together give Weekday / Weekend / Holiday, a calendar alone gives Non-holiday / Holiday, and a weekend effect alone gives Weekday / Weekend. A holiday falling on a weekend counts as a holiday.

• "calendar_holiday_lag" — the same normalized boxplots by position relative to the nearest holiday, as asked for by holiday_lags (see temporal_effects()): "2 before", "1 before", "Holiday", "1 after", ..., plus "Other" for every other day as the reference. It shows exactly the days the ..._holiday_lag_k / ..._holiday_lead_k columns flag — weekends and other holidays are skipped when counting working days — so you can see whether the lags you asked for are the ones that matter.

• "seasonality" — a cycles periodogram of the incidence series whose dominant peak suggests a Fourier season length for temporal_effects().

Reporting-delay panels (to inspect delay effects)

• "delay_weekday", "delay_week", "delay_month" — boxplots of the normalized mean reporting delay (each event date's mean delay divided by the overall mean delay, so 1 is average) by day of week, epidemiological week, or month; these reveal whether the delay itself has a calendar pattern. Normalizing keeps them on the same scale as the case-count calendar panels and makes them comparable across strata.

• "delay_holiday", "delay_holiday_lag" — the reporting-delay twins of the two holiday panels above: the normalized mean delay by day type and by position relative to the nearest holiday. These are often the more telling pair — a holiday usually does not change how many cases occur, but it very much changes how long they take to be reported.

• "delay_seasonality" — a cycles periodogram of the mean-delay series, whose peak marks a cycle in the reporting delay (e.g. a weekly reporting rhythm).

Revision-process panels (only when the object declares a revision axis)

• "revision_distribution" — the reporting-delay histogram's twin on the revision axis: a (case-count weighted) histogram of .revision_delay, the time from a report to its resolution. A case still "pending" has no resolution and so no revision delay, and does not appear.

• "revision_weekday", "revision_week", "revision_month", "revision_holiday", "revision_holiday_lag", "revision_seasonality" — the same calendar and periodogram questions asked of the dates resolutions arrived on.

Every panel is colour-coded by the process it describes — red for the reporting-delay panels, green for the case-count (epidemic) ones — and says which one it is in its subtitle, so a single panel still reads on its own.

Which panels are available depends on the object. The calendar/delay panels follow the event unit: daily data offers day-of-week and week-of-year panels, weekly data week-of-year, monthly data month-of-year. The four holiday panels describe the attached temporal_effects() spec, so they appear only when there is one to describe: "calendar_holiday" / "delay_holiday" need a holidays calendar or a weekend effect, and the two lag panels additionally need a non-zero holiday_lags. Requesting a holiday panel without the matching effect warns and skips it. The spec is read directly, so you do not need to call compute_temporal_effects() first.

The delay panels are computed on the complete portion of the series (event dates on or before the incompleteness line) so the recent reporting truncation does not bias them.

Usage

## S3 method for class 'tbl_now'
autoplot(
  object,
  ...,
  panels = "all",
  by_strata = FALSE,
  strata = NULL,
  measure = c("percent", "normalized"),
  by_revision_type = FALSE,
  level = 0.95,
  plotly = FALSE,
  size = 1,
  linewidth = 1,
  palette = .tbl_now_palette(),
  delay_distribution_xlim = NULL,
  event_date_xlim = NULL,
  calendar_effect_xlim = NULL,
  seasonality_xlim = NULL
)

Arguments

Value

A patchwork object combining the selected panels, one column per process, or — when a single panel is selected — that panel as a ggplot2 object.

See Also

diagnostic_plot() for the companion gallery, which looks at the reporting process – when reports arrived and whether any of it is artificial – rather than at the case counts; the panels here as standalone functions: plot_observed_cases(), plot_delay_distribution(), plot_cycles(), calendar_effect_plots and plot_delay_drift(); summary() and diagnose() for the same information as tables. The Diagnosing a tbl_now article reads the panels one at a time.

Examples

data(denguedat)
# A few recent months keep the example fast; the panels look the same on
# twenty years of data, they just take longer to draw.
recent <- denguedat[denguedat$onset_week >= as.Date("2010-11-01"), ]
dengue <- tbl_now(recent,
  event_date = "onset_week",
  report_date = "report_week", strata = "gender", verbose = FALSE
)
autoplot(dengue)

# A single panel comes back as a plain ggplot, so it can be selected, split
# by stratum and then treated like any other plot. `panels` takes any of the
# names listed above; `by_strata = TRUE` works on the whole gallery too, but
# one panel keeps the example quick.
autoplot(dengue, panels = "delay_week", by_strata = TRUE)

# Panel-specific arguments compose with it: zoom the delay panel to 0-10 weeks
autoplot(dengue, panels = "delay_distribution", delay_distribution_xlim = c(0, 10))

Description

Draws a fan chart of a tbl_nowcast: the counts reported so far as grey columns, one shaded band per central prediction interval over them, and the median as a line, so that the size of the correction the model is applying is visible as the gap between the bars and the fan.

Usage

## S3 method for class 'tbl_nowcast'
autoplot(object, ..., levels = NULL,
  show_reported = TRUE, colour = NULL, linewidth = 1, date_lim = NULL,
  ylim = NULL, palette = .tbl_now_palette())

Arguments

Details

A named function registered in .onLoad(), like the neighbouring tidy() and as_tibble() methods and for the same two reasons.

⁠S7::method(autoplot, tbl_nowcast) <- ⁠ would be shorter, but ⁠method<-⁠ is a replacement function, so R rewrites the call as an assignment back to autoplot and leaves a copy of ggplot2's generic in this namespace. That copy is what ⁠R CMD check⁠ sees when it decides autoplot is a generic this package owns and exports, which in turn makes every ⁠autoplot.*⁠ function here look like an S3 method that was never registered. Plain registration copies nothing.

The function is also named for the method it implements, rather than being assigned anonymously into the generic, so that ⁠R CMD check⁠ can resolve this topic's usage section back to an object that exists.

Value

A ggplot object.

See Also

run_nowcast(), nowcast_ensemble()

Examples

predictions <- tidyr::expand_grid(
  onset_week = as.Date("2020-01-05") + seq(0, 28, by = 7),
  .quantile_level = c(0.05, 0.25, 0.5, 0.75, 0.95)
)
predictions$.value <- 10 + 30 * predictions$.quantile_level
nc <- tbl_nowcast(predictions = predictions, method = "toy", event_date = "onset_week")

autoplot(nc)

# Zoom on to the corrected weeks without dropping the rows that build the fan.
autoplot(nc, date_lim = c(as.Date("2020-01-19"), as.Date("2020-02-02")))

Description

Joins nowcast_backtest() results that were run separately into one, so that an expensive backtest never has to be repeated:

• Different models, same dates. Backtest each engine on its own (perhaps on different machines, or as it becomes available) and combine them to compare their scores, derive nowcast_weights() or build a nowcast_ensemble().

• Same model, different dates. Backtest last year's dates once, and next year backtest only the new dates and add them to the old result.

Combining backtests of the same data, dates and engines gives the object a single call would have – the tables have the same rows in the same order:

bt1 <- nowcast_backtest(x, engine_a, now_dates = my_dates)
bt2 <- nowcast_backtest(x, engine_b, now_dates = my_dates)
backtest_combine(bt1, bt2)
# is the same as
nowcast_backtest(x, engine_a, engine_b, now_dates = my_dates)

(Only the elapsed_seconds of the timings differ.)

Usage

backtest_combine(..., only_common_dates = FALSE)

Arguments

Details

The backtests must agree on the event date, the strata, truth_axis, truth_type, keep_draws and the quantile levels; otherwise they are not measuring the same thing and the call aborts.

Two backtests must not both have a successful fit of the same method at the same now date: that would be two answers to one question, so the call aborts. A fit that failed in one and succeeded in another is fine – the success is kept, which lets you re-run only the failures and combine.

Scores are kept as they were computed. When the backtests were run at different times the data may have been revised in between, so each was scored against its own truth. The combined truth is the one from the backtest with the latest now date, extended with any event dates only the others have; the call warns when an event date it shares has a different observed count in the others. Re-run the earlier backtest if you need every score against the same truth.

Methods that cover different dates are allowed (that is how you add dates to one model only), and comparisons – nowcast_weights(), the print method – then use only the targets every method scored, and say so. Use only_common_dates = TRUE to drop the dates that not every method has.

Value

A nowcast_backtest; see nowcast_backtest().

See Also

nowcast_backtest(), whose checkpoint_file argument resumes an interrupted backtest, nowcast_weights() and nowcast_ensemble().

Examples

data(denguedat)
recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)
dates <- as.Date(c("2010-10-04", "2010-11-15"))

# Different models, same dates: backtest separately, then combine.
narrow <- nowcast_backtest(dengue,
  example_engine(spread = 0.1, label = "narrow"),
  now_dates = dates, verbose = FALSE
)
wide <- nowcast_backtest(dengue,
  example_engine(spread = 0.5, label = "wide"),
  now_dates = dates, verbose = FALSE
)
both <- backtest_combine(narrow, wide)
both$methods

# Same model, new dates: add them to what you already have.
later <- nowcast_backtest(dengue,
  example_engine(spread = 0.1, label = "narrow"),
  now_dates = as.Date("2010-11-22"), verbose = FALSE
)
backtest_combine(narrow, later)$now_dates

Description

One panel of autoplot(), drawn on its own. Each function shows the same boxplots the corresponding autoplot() panel does, for one calendar grouping:

• plot_day_of_week_effects() — by day of week (daily data only).

• plot_week_of_year_effects() — by epidemiological week.

• plot_month_of_year_effects() — by month (monthly data only).

• plot_holiday_effects() — by day type (Weekday / Weekend / Holiday, following the attached temporal_effects() spec).

• plot_weekend_effects() — the same panel, on an object that may not carry a spec yet: it attaches temporal_effects(weekend = TRUE) when there is no weekend effect already, so the weekend boxes appear without a separate add_temporal_effects() call. The spec goes on a copy — your object is not modified — and a calendar already attached still contributes its Holiday box. Daily data only, since a weekend is a property of the day.

• plot_holiday_lag_effects() — by position relative to the nearest holiday ("1 before", "Holiday", "1 after", ..., plus "Other").

type picks which process to describe: "epidemic" (green — how the cases vary by calendar group), "report" (red — how the reporting does), or "revision" (ochre — how resolved cases arrive on revision dates).

The three day-type / holiday-lag functions have no measure argument: they are always normalized. Their categories are not equal-sized parts of a calendar block — the weekend is two days in seven — so a percentage share would mostly restate the calendar rather than the data ("29% of the cases at the weekend" is average, not low). The day-of-week, week-of-year and month-of-year functions keep both measures.

Use these when you want one effect, in its own figure, at its own size; use autoplot() when you want the diagnostic grid in one call. Everything else is the same: autoplot(x, panels = "calendar_weekday") and plot_day_of_week_effects(x) return the identical plot.

Usage

plot_day_of_week_effects(
  x,
  type = c("epidemic", "report", "revision"),
  measure = c("percent", "normalized"),
  ...
)

plot_week_of_year_effects(
  x,
  type = c("epidemic", "report", "revision"),
  measure = c("percent", "normalized"),
  ...
)

plot_month_of_year_effects(
  x,
  type = c("epidemic", "report", "revision"),
  measure = c("percent", "normalized"),
  ...
)

plot_holiday_effects(x, type = c("epidemic", "report", "revision"), ...)

plot_weekend_effects(
  x,
  type = c("epidemic", "report", "revision"),
  weekend_days = c("Sat", "Sun"),
  ...
)

plot_holiday_lag_effects(x, type = c("epidemic", "report", "revision"), ...)

Arguments

Value

A ggplot2 object (or a plotly widget when plotly = TRUE).

See Also

autoplot.tbl_now(), plot_cycles(), plot_delay_distribution(), plot_observed_cases(); temporal_effects() and add_temporal_effects() for the specification the day-type and holiday-lag panels describe.

Examples

data(denguedat)
# First few years only, to keep the example quick; the full data works the same.
dengue_now <- tbl_now(denguedat[1:2500, ], onset_week, report_week, verbose = FALSE)

# How the cases vary by epidemiological week
plot_week_of_year_effects(dengue_now)

# The weekend on its own, on daily data, with no spec to attach first
days <- seq(as.Date("2021-01-01"), as.Date("2021-06-30"), by = "day")
daily_now <- tbl_now(
  data.frame(event_date = days, report_date = days + 1),
  event_date, report_date, verbose = FALSE
)
plot_weekend_effects(daily_now)

# By month, on monthly-unit data. `type` picks the process and `measure`
# picks the scale; both compose, and work the same way on every calendar
# function here. (The day-type and holiday-lag panels are always
# normalized, so they take `type` but not `measure`.)
monthly_now <- tbl_now(
  data.frame(
    event_date  = seq(as.Date("2018-01-01"), as.Date("2021-12-01"), by = "month"),
    report_date = seq(as.Date("2018-02-01"), as.Date("2022-01-01"), by = "month")
  ),
  event_date, report_date,
  event_units = "months", report_units = "months", verbose = FALSE
)
plot_month_of_year_effects(monthly_now, type = "report", measure = "normalized")

if (requireNamespace("almanac", quietly = TRUE)){

  ## By day type (weekday / weekend / holiday), once a holiday calendar is attached
  holiday_now <- dengue_now |>
   add_temporal_effects(temporal_effects(weekend = TRUE, holidays = almanac::cal_us_federal()))
  plot_holiday_effects(holiday_now)

  # By position relative to the nearest holiday
  holiday_lag_now <- dengue_now |>
    add_temporal_effects(temporal_effects(holidays = almanac::cal_us_federal(), holiday_lags = 2))
  plot_holiday_lag_effects(holiday_lag_now)

}

Description

Surveillance data is full of dates that are not really dates. A case with onset in March turns up in December; a report date is missing altogether; a system codes "never reported" as 2222-02-22. Deleting those records throws away real cases, and believing them drags the estimated delay distribution to the right until the nowcast thinks reporting is far slower than it is.

These functions censor instead: the object keeps the case, but records its delay as "at least this long" rather than "exactly this long".

There are two axes to censor and three ways to say which rows, so there are six verbs. On the reporting axis (event date to report date, the is_censored_report flag):

• censor_reports() – rows matching a condition get a replacement report date (the missing ones become now, say) and the flag.

• censor_reporting_delays() – the same, said as a delay rather than a date; with no replacement it only sets the flag.

• censor_reporting_delays_above() – considers as censored every delay longer than max_delay, in the object's event units, and leaves every other row alone. This is the one you want when the threshold is the rule: "anything that took more than 60 days is a lower bound, not a measurement".

On the revision axis (report date to resolution, the is_censored_revision flag), the same three:

• censor_revisions() – rows matching a condition get a replacement revision date and the flag.

• censor_revision_delays() – the same, said as a delay from the report.

• censor_revision_delays_above() – considers as censored every revision delay longer than max_delay, in the object's revision units: a laboratory result that took months is a case you have stopped believing the turnaround of.

Usage

censor_reporting_delays_above(x, max_delay, verbose = TRUE)

censor_reports(x, condition, to_report = get_now(x), verbose = TRUE)

censor_reporting_delays(x, condition, to_delay = NULL, verbose = TRUE)

censor_revisions(x, condition, to_revision = get_now(x), verbose = TRUE)

censor_revision_delays(x, condition, to_delay = NULL, verbose = TRUE)

censor_revision_delays_above(x, max_delay, verbose = TRUE)

Arguments

Details

The reporting delay is read from the generated .delay column (report date minus event date, in the object's event units); the revision delay from .revision_delay (revision date minus report date, in revision units). Existing censoring flags are merged rather than overwritten, so a delay that was already censored stays censored, and the flag column is created (as .is_censored_report / .is_censored_revision) when the object has none.

The threshold functions keep the case and its date. Nothing is deleted and no outcome is rewritten: the flag says the delay is a bound rather than a measurement, and it is up to the model to use that. A case that was confirmed after 200 days is still a confirmed case, and get_latest_revised_cases() still counts it.

condition is evaluated inside the data, like a dplyr::filter() expression, so it can name any column – including the generated .delay. Rows where it comes out NA are not censored: a condition that cannot be evaluated is not a condition that was met.

The four verbs that take a condition can move a date, and then they rebuild the object: the generated numeric and delay columns are recomputed, and now moves forward when a replacement lands after it, never backwards – now is where you are standing, not the last date in the data. Nothing stops a replacement from landing before the date it is measured from; that is a negative delay, and validate_tbl_now() says so.

Any temporal-effect column materialised on the report date (⁠.report_*⁠, from compute_temporal_effects()) is dropped when the report date moves, because it describes a date that has just changed; run compute_temporal_effects() again to rebuild it. The ⁠.event_*⁠ ones are kept – the event dates never move – and neither is touched by the revision verbs.

Value

A tbl_now with that axis's censoring column updated, creating it when absent (.is_censored_report or .is_censored_revision), and with the dates replaced where a replacement was asked for. The three reporting verbs touch is_censored_report and the report date; the three revision verbs touch is_censored_revision and the revision date. Neither rewrites revision_type, and nothing is ever deleted.

Pending cases are skipped

"pending" means reported and still waiting, so a pending case has no revision date – that is the whole difference between it and a resolution that was never recorded. Writing a date onto one would assert a resolution that never happened, and make the case look resolved to everything counting arrivals on the revision axis.

So censor_revisions() and censor_revision_delays() skip pending rows when they would write a date, and say how many they skipped. To censor a case that really was resolved but whose date is missing, make sure its revision_type says so first. Flagging without a replacement is not affected: no date is written, so there is nothing to contradict.

See Also

add_is_censored_report() and change_is_censored_report() to set the flag by hand, and add_is_censored_revision() for the revision axis; diagnose_revision_delay() and plot_delay_distribution() to find the threshold worth using; diagnose_truncation() for the delays that are missing rather than long; complete_zeroes() for the opposite problem.

Examples

# Four cases, one of which took 300 days to be reported.
df <- data.frame(
  onset = as.Date("2020-01-01") + c(0, 0, 1, 2),
  reported = as.Date("2020-01-01") + c(1, 5, 2, 300)
)
tn <- tbl_now(df,
  event_date = onset, report_date = reported,
  data_type = "linelist", verbose = FALSE
)
tn$.delay

# Anything slower than 60 days is recorded as a lower bound, not a fact.
censored <- censor_reporting_delays_above(tn, max_delay = 60)
censored[[get_is_censored_report(censored)]]

# The same rule written by hand, and capped at 60 days as well, so the
# 300-day outlier stops dominating the delay distribution.
capped <- censor_reporting_delays(tn, .delay > 60, to_delay = 60, verbose = FALSE)
capped$.delay

## ---- Reports that never arrived ---------------------------------------

# A missing report date, and a system that codes "never" as a date in 2222.
messy <- data.frame(
  onset = as.Date("2020-01-01") + 0:3,
  reported = as.Date(c("2020-01-03", NA, "2222-02-22", "2020-01-06"))
)
messy_now <- suppressWarnings(tbl_now(messy,
  event_date = onset, report_date = reported,
  data_type = "linelist", units = "days", verbose = FALSE,
  now = as.Date("2020-01-10")
))

# Both are "not reported yet", so both become `now` and are flagged censored.
# Wrapped in suppressWarnings because the object keeps warning about the
# dates being fixed.
fixed <- suppressWarnings(censor_reports(messy_now,
  is.na(reported) | reported > as.Date("2100-01-01"),
  verbose = FALSE
))
fixed[[get_report_date(fixed)]]
fixed[[get_is_censored_report(fixed)]]

## ---- The revision counterpart ----------------------------------------

cases <- data.frame(
  onset = as.Date("2021-01-04") + 0:4,
  visit = as.Date("2021-01-05") + 0:4,
  result = as.Date("2021-01-05") + 0:4 + c(1, 2, 1, 90, 2),
  outcome = rep("confirmed", 5)
)
flu <- tbl_now(cases,
  event_date = onset, report_date = visit,
  revision_date = result, revision_type = outcome,
  data_type = "linelist", verbose = FALSE
)

# That one is flagged; all five stay confirmed, and the date is kept.
flagged <- censor_revision_delays_above(flu, 30, verbose = FALSE)
flagged[[get_is_censored_revision(flagged)]]
table(flagged[["outcome"]])

# The condition form: cap that turnaround at 30 days from the report, which
# moves the revision date to match.
capped_lab <- censor_revision_delays(flu, .revision_delay > 30,
  to_delay = 30, verbose = FALSE
)
capped_lab$.revision_delay

## A pending case has no resolution date, so there is nothing to censor --
# it is skipped rather than given a date it never had.
waiting <- flu
waiting[["outcome"]][2] <- "pending"
waiting[["result"]][2] <- as.Date(NA)
waiting <- change_revision_date(waiting, "result", "outcome")
out <- censor_revisions(waiting, is.na(result), verbose = FALSE)
out[["result"]][2] # still NA

Description

Surveillance data records what happened, not what didn't. If no dengue case with onset on 3 January was reported on 5 January, there is simply no row for that combination – which is not the same as a row saying zero, even though it means the same thing.

Most nowcasting models need the difference spelled out. They work on a complete rectangle of (event date x report date) cells, and a missing cell is ambiguous: it could be a genuine zero, or a delay so long the report has not arrived yet. complete_zeroes() writes the genuine zeros in explicitly, for every stratum, leaving only the not-yet-reported cells absent.

Usage

complete_zeroes(x, max_delay = NULL, until = NULL)

Arguments

Details

Zeros are only filled where a report could have arrived: cells with a report date on or before the event date's now, and within max_delay. Filling beyond that would invent observations from the future.

Rows with a missing date

A row whose event or report date is NA has no cell on the rectangle, so it takes no part in the grid: the bounds (max_delay, the first and last event date, the last report date) are all computed ignoring it. It is still a case, though, so it is carried through unchanged rather than dropped – use censor_reports() to give it a bound, or dplyr::filter() to remove it, if you would rather it were on the grid or gone. Only an object in which every row is missing one of the two dates is refused, because then there is no grid to complete at all.

Value

A tbl_now object with the same columns as x, plus the rows that were implicitly zero, carrying 0 in the case_count column. Explicit missing counts in the input remain NA; only cells created by complete_zeroes() are filled. The data type is preserved, as are any computed temporal-effect columns (recomputed over the new rows).

See Also

to_count() for the data shapes this operates on; censor_reporting_delays_above() for the opposite problem, delays that are too long; diagnose_missing() and diagnose_truncation() to find the gaps first; plot_reporting_triangle() to see the rectangle being filled.

Examples

ndata <- dplyr::tibble(
  event = rep(c(
    as.Date("2020/01/01"), as.Date("2020/01/01"),
    as.Date("2020/01/02"), as.Date("2020/01/04"),
    as.Date("2020/01/04")
  ), 2),
  report = rep(c(
    as.Date("2020/01/01"), as.Date("2020/01/02"),
    as.Date("2020/01/02"), as.Date("2020/01/04"),
    as.Date("2020/01/05")
  ), 2),
  n = rpois(10, lambda = 5),
  sex = c(rep("Male", 5), rep("Female", 5))
)
ndata <- tbl_now(ndata,
  event_date = event, report_date = report,
  verbose = FALSE, strata = sex, case_count = n, data_type = "count-incidence"
)

# Nothing happened on 2020-01-03, so the data has no row for it at all.
sort(unique(ndata$event))

## complete_zeroes() writes that absence down as an explicit zero, for every
# stratum, so a model can tell "no cases" from "not reported yet".
filled <- complete_zeroes(ndata)
sort(unique(filled$event))
nrow(ndata)
nrow(filled)

# Also works for count-cumulative
ndata |>
  to_count("count-cumulative") |>
  complete_zeroes() |>
  dplyr::arrange(event, sex, report)

Description

Daily case counts of COVID-19 from Colombia's national epidemiological surveillance system (INS), aggregated by notification date, diagnosis date, and sex. Each row is a unique combination of these three variables together with the number of cases n.

Usage

covid_colombia

Format

A data frame with 35,501 rows and 4 variables:

notification_date

Date. The event date – when the case was notified / symptom onset was recorded.

diagnosis_date

Date. The report date – when the laboratory diagnosis was registered in the national system.

sex

character. Biological sex of the case: "Female" or "Male".

n

integer. Number of cases with this (notification_date, diagnosis_date, sex) combination.

Details

In the nowcasting context the event date is notification_date (when the case symptom onset was recorded in the system) and the report date is diagnosis_date (when the laboratory result was entered). The delay between the two reflects the time from symptom onset to laboratory revision and data entry – the quantity the nowcasting model estimates and corrects for.

The dataset covers the full first three years of the Colombian epidemic (2020-03-02 to 2023-03-03) and was used in the NobBS comparison study to benchmark the diseasenowcasting / diseasenowcast2 engine.

Source

Instituto Nacional de Salud (INS), Colombia. Data accessed via the SIVIGILA open-data platform and pre-processed for the diseasenowcasting benchmarking study.

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(covid_colombia)

# A stratified tbl_now: event = notification, report = diagnosis.
tn <- tbl_now(
  covid_colombia,
  event_date  = notification_date,
  report_date = diagnosis_date,
  strata      = sex,
  case_count  = n,
  data_type   = "count-incidence",
  verbose     = FALSE
)
tn

Description

A compact aggregation of the U.S. CDC's individual-level COVID-19 case surveillance database. It is the package's worked example for two different things: batch reporting, and the revision process – the optional third date a surveillance record can carry.

Usage

data(covid_us)

Format

A data frame with 192,953 rows and six variables:

onset_dt

Date. The event date – symptom onset.

pos_spec_dt

Date. The report date – collection of the first positive specimen.

cdc_report_dt

Date. The revision date – when the case was registered at CDC.

current_status

character. CDC's classification, either "Laboratory-confirmed case" or "Probable Case". Map it with revision_levels (see above).

sex

character. "Female", "Male", "Other", "Unknown" or "Missing".

n

integer. Number of cases sharing that combination.

Details

Each row is a unique (onset date, specimen date, CDC report date, status, sex) combination with the number of cases n.

The three dates

The source file carries four date columns. cdc_case_earliest_dt is derived by CDC as the earliest of the others, and equals onset_dt for 99.997% of the rows kept here, so it is dropped as redundant. The three that remain are the only chain that runs forward in time, and they map onto the three roles a tbl_now() knows about:

onset_dt

the event – symptoms begin.

pos_spec_dt

the report – the first positive specimen is collected, which is when the surveillance system first sees the case.

cdc_report_dt

the revision – the case is registered at CDC with a status.

current_status and revision_levels

current_status is kept in CDC's own words rather than recoded, because translating it is exactly what tbl_now(revision_levels = ) is for:

revision_levels = c(
  "Laboratory-confirmed case" = "confirmed",
  "Probable Case"             = "pending"
)

A probable case is one that met the clinical and epidemiological criteria without meeting the laboratory-confirmed definition. Every row here has a positive specimen, so "probable" means the specimen was collected and the case was never laboratory-settled – "pending" in this package's vocabulary. Note what is not there: CDC does not withdraw cases, so "retracted" does not occur in this dataset. It is a two-outcome revision process, and code that needs a retraction has to look elsewhere.

The relationship between the outcome and the revision delay is real rather than fabricated: probable cases are registered a median of 2 days after the specimen, laboratory-confirmed ones 4 days.

What was kept

Cases where all three dates are present, correctly ordered (⁠onset_dt <= pos_spec_dt <= cdc_report_dt⁠) and falling entirely within 2020 – a self-consistent "as of the end of 2020" snapshot. Rows out of order are data-entry errors; rows missing a date cannot be placed on the chain at all. See data-raw/covid_us.R for the exact duckdb aggregation of the 14 GB source file.

The reporting delay is enormous and heavily right-skewed: cases reached CDC not smoothly but in large backlog dumps – a textbook batch-reporting pattern that diagnose_batches() and transport_discriminant() recover.

Source

Centers for Disease Control and Prevention (CDC), COVID-19 Response. COVID-19 Case Surveillance Public Use Data (version date: June 21, 2024). https://data.cdc.gov/Case-Surveillance/COVID-19-Case-Surveillance-Public-Use-Data/vbim-akqf/about_data. COVID-19 case surveillance data are collected by jurisdictions and reported voluntarily to CDC.

See Also

tbl_now() to declare the date columns; add_revision_date() to attach the third one to an object that has none; revised_cases to count the outcomes; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia and hai_bucaramanga.

Examples

library(dplyr)

data(covid_us)

# The three dates with CDC's labels translated to this package's vocabulary.
covid_us <- covid_us |>
  filter(onset_dt <= as.Date("2020-02-01"))

tn3 <- tbl_now(
  covid_us,
  event_date       = onset_dt,
  report_date      = pos_spec_dt,
  revision_date    = cdc_report_dt,
  revision_type    = current_status,
  revision_levels = c(
    "Laboratory-confirmed case" = "confirmed",
    "Probable Case"             = "pending"
  ),
  case_count = n,
  strata     = sex,
  data_type  = "count-incidence",
  verbose    = FALSE
)

Description

Surveillance data from CDC Division of Vector-Borne Diseases. 1990-2010 case reporting data included.

Usage

data(denguedat)

Format

A data frame with 52,987 rows (one per case) and 3 variables:

onset_week

Date. The week symptoms began – the event date.

report_week

Date. The week the case reached the surveillance system – the report date. Always on or after onset_week.

gender

character. "Male" or "Female". Synthetic; see the note.

Details

Each row represents a case with the columns indicating the following:

• onset_week: the week of symptom onset.

• report_week: the week of case report.

• gender: the gender of the infected individual (randomly assigned with 0.5:0.5 probability of "Male"/"Female").

Note

Data originally from the NobBS package. While onset_week and report_week correspond to actual observed data the gender was constructed exclusively for the examples of NobBS. It is a synthetic (simulated) variable and does not correspond to any reality.

References

MCGOUGH, Sarah F., et al. Nowcasting by Bayesian Smoothing: A flexible, generalizable model for real-time epidemic tracking. PLoS computational biology, 2020, vol. 16, no 4, p. e1007735.

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(denguedat)
head(denguedat)

# The two dates every nowcast needs. Weekly data, twenty years of it.
range(denguedat$onset_week)

# Declaring them turns the data frame into a tbl_now.
dengue <- tbl_now(denguedat,
  event_date = onset_week, report_date = report_week,
  strata = gender, verbose = FALSE
)
dengue

# Most cases arrive within a week or two of onset; a few take much longer.
summary(as.numeric(dengue$.delay))

Description

diagnose() is a structural health check. It looks for the things that make a nowcast wrong before any model is fitted – dates out of order, missing values, repeated rows, units that disagree, data after now, event dates too recent to be complete – and returns them as a tibble of findings, sorted worst first.

It is deterministic and runs no statistical test. Whether the reporting delay drifts, and whether reports arrive in batches, are questions about a distribution, not about the object's structure, so diagnose() leaves them to diagnose_drift() and diagnose_batches() rather than quietly running a test whose method, window and multiplicity correction you did not choose.

Every block is also available on its own – see nowcast_diagnose_components – and diagnose() is exactly the dplyr::bind_rows() of those pieces.

Usage

diagnose(x, ...)

## Default S3 method:
diagnose(x, ...)

## S3 method for class 'tbl_now'
diagnose(
  x,
  ...,
  checks = NULL,
  by_strata = NULL,
  strata = NULL,
  warn_non_uniqueness = TRUE
)

Arguments

Value

A tibble with the columns described above, sorted worst first.

The columns

Every function in this family returns the same schema, so results can be stacked with dplyr::bind_rows() and filtered with dplyr::filter().

check

Which block the row belongs to: "declarations", "ordering", "missing", "duplicates", "units", "negatives", "now", "truncation" or "strata".

scope

What the row is about: a column name, a time axis, a pair of axes, or "all".

stratum

Which subset of the data the row describes: "all" for the pooled rows, or the stratum label otherwise.

status

An ordered factor, worst first, so the tibble sorts itself: error > warning > note > ok > skipped. See the section below.

n_affected

How many rows (or cases, or dates) the finding is about.

n_total

How many were considered.

prop

n_affected / n_total.

message

One human sentence, already formatted.

hint

What to do about it, or NA.

rows

A list-column of offending row indices, so x[result$rows[[1]], ] goes straight to the bad rows. Empty when the finding is not about particular rows, or when it was computed on a de-accumulated view whose rows are not the object's own.

What the statuses mean

error

validate_tbl_now() aborts on this. The object is not a usable tbl_now.

warning

validate_tbl_now() warns about this.

note

A diagnose()-only observation worth your attention. It is deliberately never promoted to a warning: validate_tbl_now() runs on every dplyr verb, and a new warning there would turn a quiet construction into a noisy one for data that has always been accepted.

ok

The check ran and found nothing.

skipped

Could not be assessed – no revision process, the wrong data type, or an optional package that is not installed.

See Also

nowcast_diagnose_components for the individual blocks; summary() for the descriptive counterpart – what is in the data rather than what is wrong with it; validate_tbl_now() for the same findings raised as errors and warnings; diagnostic_plot() for the picture version. The Diagnosing a tbl_now article goes through the findings one at a time.

Examples

data(denguedat)
# The last five years. The full twenty-year series gives the same shape
# of answer, it just takes longer to compute.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
ndata <- tbl_now(recent,
  event_date = "onset_week",
  report_date = "report_week",
  strata = "gender",
  verbose = FALSE
)

# Everything, worst first
diagnose(ndata)

# Only what needs acting on
diagnose(ndata) |> dplyr::filter(status <= "note")

# One block on its own
diagnose(ndata, checks = "units")

## `diagnose()` never stops your pipeline -- it hands back a table for you to
## read. Use validate_tbl_now() when you want a broken object to be an error.
nrow(diagnose(ndata))

Description

Detects batches: report dates at which a stalled reporting system releases a backlog. A batch moves reports along the report axis without creating them, so it shows up as a spike preceded by a deficit, while the total over a window spanning both is unchanged. diagnose_batches() is completely model-free – it needs only a tbl_now(), not a fitted model – which makes it the right tool for exploratory data analysis before any nowcasting model is chosen.

Usage

diagnose_batches(
  x,
  lookback = 7L,
  baseline_window = NULL,
  period = NULL,
  null_model = c("auto", "poisson", "robust"),
  axis = c("report", "revision"),
  alpha = 0.05,
  drop_censored = TRUE
)

Arguments

Details

The idea is that a batch moves reports along the report axis without creating them. Over a window of report dates that spans both the lull and the release, the total is therefore unchanged – every report you would have seen in the window you still see, just on a different day. A genuine surge instead adds reports and inflates the window total. So two quantities separate the two cases: the deficit (how many reports the days just before the spike were missing) picks up transport, and the window total (relative to a baseline) picks up creation. A batch has a large deficit but a conserved window total; a surge has an inflated window total but no deficit.

The baseline for each candidate window is refit from report dates lying strictly outside that window, using a robust local line (Siegel's repeated median). Smoothing through the episode instead would let the deficit drag the baseline down and the batch would mask itself as a surge.

The batch flag is the trustworthy verdict: it compares dates within one window (insensitive to the overall level) and is Benjamini-Hochberg corrected across every (report date, stratum) pair, so it controls the false discovery rate rather than firing on every point that crosses a raw threshold. A per-point creation ("surge") label is deliberately not returned: it only compares the window total against the baseline, so on a steeply curved epidemic curve it fires on ordinary growth. If you need genuine surges, fit a model. For "count-cumulative" data the increments are signed and reported can be negative (a down-revision).

A reporting system that is always closed at weekends produces every batch symptom, every week, so diagnose_batches() needs the length of any scheduled cycle. It reads that from the object's temporal effects when it can: a day-of-week effect sets period = 7, a week-of-year effect period = 52 (see add_temporal_effects()). Pass period yourself to override; if the data is daily and carries no temporal effect, the function suggests period = 7. With a period set, the baseline is corrected by per-phase medians across cycles, so an irregular batch reads as an excursion relative to the schedule.

Value

A tibble of class diagnose_batches, one row per (report date, stratum), with a print() method that summarises the flagged dates. Columns:

report_date

The report (registration) date the row describes.

stratum

The stratum label, or "all" when the data is unstratified.

reported

Reports recorded on report_date (a signed increment for "count-cumulative" data, so it can be negative).

baseline

The robust expected number of reports on report_date under "no batch", from the leave-window-out local line.

deficit

How many reports the lookback days before report_date were missing relative to baseline – the transport signal. Large and positive when a stall preceded a spike.

delta

The window total minus its baseline mean – the creation signal. Near zero for a pure batch (mass only moved), large for a surge.

p_transport

One-sided p-value that the deficit is larger than noise (the raw, per-point transport test).

p_transport_bh

p_transport after a Benjamini-Hochberg correction across all rows; the flag below thresholds this.

batch

Logical verdict: TRUE when p_transport_bh < alpha and the window is not still depleted (a hold). This is the column to trust.

See Also

diagnose_batches2() for the complementary test on which event dates a flagged report date drew from; transport_discriminant() for the two coordinates behind the test, without the hypothesis test on top; simulate_batch() to plant a known batch and check the screen finds it; plot_reporting_process() and plot_reporting_triangle() to see it; add_is_censored_report() to record a batch once you believe it. The Diagnosing a tbl_now article works through a real example.

Examples

data(denguedat)

dengue_tbl <- tbl_now(
  denguedat,
  event_date  = onset_week,
  report_date = report_week,
  data_type   = "linelist",
  verbose     = FALSE
)

# Scan every report date for an unusually large arrival.
screened <- diagnose_batches(dengue_tbl, lookback = 2)
head(screened)

# The dates it flagged, strongest evidence first. Treat these as candidates to
# look into, not as confirmed backlog releases. `reported` against `baseline`
# says how much bigger the arrival was than the surrounding days led you to
# expect.
flagged <- screened[screened$batch, ]
nrow(flagged)
flagged[order(flagged$p_transport_bh), c("report_date", "reported", "baseline")]

Description

A complement to diagnose_batches(), which sees only report volumes. This test asks whether the reports that arrived on a candidate date came from systematically older event dates – the signature of a released backlog – by comparing their delays with those of neighbouring report dates. It is model-free and, under the conditions below, exactly distribution-free.

Usage

diagnose_batches2(
  x,
  at,
  neighbours = 3L,
  guard = 1L,
  permute = c("items", "blocks"),
  n_permutations = 999L,
  axis = c("report", "revision"),
  drop_censored = TRUE,
  seed = NULL
)

Arguments

Details

The delays of the reports arriving on at are compared with the pooled delays of the reports arriving on nearby dates, using a one-sided rank-sum (Wilcoxon) statistic directed at longer delays on at. The p-value comes from a permutation, so no asymptotic approximation is used.

The test is model-free: as long as the epidemic curve is locally smooth, neighbouring report dates share one common delay profile, so their delay labels are exchangeable and the permutation test is (essentially) distribution-free – it needs neither the delay distribution nor the epidemic curve. With Poisson counts permute = "items" is exact; if the counts are overdispersed (neighbouring report dates share event dates, so individual items are not exchangeable) use permute = "blocks", which permutes whole report dates.

at need not be a date that carries rows. A line list cannot represent a zero, so a report date on which nothing arrived has no rows at all; that is the observation "no arrivals", not a missing one, and the test reports zero arrivals for it instead of aborting. Only a date off the object's report grid is an error.

The guard argument omits report dates immediately adjacent to at from the comparison set: if a batch is present, its own deficit dates sit right beside the spike and would contaminate the reference group. For "count-cumulative" data only positive increments carry a meaningful delay; negative increments (down-revisions) are dropped with a message.

Value

A tibble, one row per stratum, with stratum, n_at, n_reference, mean_delay_at, mean_delay_reference, statistic (standardised rank-sum) and p_value (one-sided: longer delays on at).

See Also

diagnose_batches(), which finds the report dates worth passing to at; simulate_batch() to plant a batch of known shape and check it is recovered; plot_delay_profiles() to see the delay profile this tests.

Examples

data(denguedat)

dengue_tbl <- tbl_now(
  denguedat,
  event_date  = onset_week,
  report_date = report_week,
  data_type   = "linelist",
  verbose     = FALSE
)

# Pick a report date to interrogate. A real workflow takes this from
## diagnose_batches(); here we simply name one.
diagnose_batches2(dengue_tbl, at = as.Date("1990-06-25"), n_permutations = 99)

# `n_permutations` sets the resolution of the p-value: 99 keeps the example
## fast, but use the default (999) for anything you intend to report.

Description

Complements diagnose_drift(). Where that tests for a gradual monotonic trend, this tests for a single abrupt shift (e.g. a reporting-system change on some date) in the per-period delay summaries, using Pettitt's nonparametric change-point test. As with diagnose_drift() it works on both a location statistic (median / mean) and a dispersion statistic (IQR / 10-90 spread), on mature data only, and — being rank-based — it is robust to the skew and serial dependence of a delay series.

Usage

diagnose_changepoint(
  x,
  ...,
  stat = c("median", "spread"),
  by_strata = FALSE,
  strata = NULL,
  mature_only = TRUE,
  level = 0.95,
  alpha = 0.05,
  axis = c("report", "revision")
)

Arguments

Value

A tibble with one row per requested stat per stratum, and the following columns:

strata

character. The stratum the row refers to. When by_strata = FALSE (the default) there is a single stratum labelled "all"; otherwise one level per observed combination of strata.

stat

character. Which delay summary was tested — one of "median", "mean", "iqr" or "spread". As in diagnose_drift(), the first two are location statistics and the last two dispersion statistics.

n

integer. Length of the tested series: the number of event dates contributing a non-missing value after the mature_only filter — periods, not cases. Series shorter than 8 periods, or with zero variance, are not tested and return NA throughout.

changepoint

Date. The event date of the last period before the estimated change; the shift is taken to occur immediately after it. NA when the series was too short to test. Note this is reported even when changepoint_detected is FALSE — Pettitt's test always returns the most extreme candidate split, so this field is only meaningful once the p-value supports it.

statistic

numeric. Pettitt's K, the maximum absolute value of the rank statistic U_t over all candidate split points. Larger means a cleaner separation between the two sides. It is not standardised, so it grows with n and is not comparable across series of different lengths.

p_value

numeric. Two-sided p-value for the null of no change point, from the standard approximation $2\exp(-6K^2 / (n^3 + n^2))$, capped at 1. This approximation is known to be conservative for small n.

before, after

numeric. The mean of the statistic on each side of changepoint, in the object's delay units. These are plain means of the per-period summaries, so they describe the two regimes directly.

shift

numeric. after - before: the estimated size and direction of the jump, in delay units. Positive means delays got longer after the change point. This is the number to judge operational relevance by.

changepoint_detected

logical. The verdict: TRUE when p_value < alpha. NA p-values give FALSE.

Interpreting the result

Judge shift first and p_value second. A statistically detected change point with a shift far smaller than the day-to-day noise in the delay series is not worth acting on; a large shift is, even at a marginal p-value.

Two structural caveats matter in practice:

• Pettitt's test assumes exactly one change point. Given several, it returns the most prominent and silently ignores the rest. If you suspect more, re-run on each side of the first changepoint to search recursively.

• A slow monotonic drift will often trip this test too, with the change point landing near the middle of the series. Running diagnose_drift() alongside disambiguates: a genuine step shows up here and not necessarily there, while a gradual drift shows up in both.

A confirmed change point usually has an operational explanation — a new laboratory information system, a change in case definition, a reporting mandate, a holiday backlog being cleared. Where it lands is a strong hint about the cause, and about how far back a nowcasting model can safely be fitted: data before the change point comes from a different reporting regime.

Unlike diagnose_drift(), this test has no third-party dependency and no meaningful runtime cost, so it is cheap to run routinely.

See Also

diagnose_drift() for a gradual trend rather than a single break; plot_delay_drift() to see the series and where the break was found; diagnose_batches() for a one-day spike rather than a lasting shift.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week", report_date = "report_week", verbose = FALSE
)
diagnose_changepoint(dengue)

Description

Runs an autocorrelation-robust monotonic-trend test on the per-period, count-weighted delay summaries, to answer "do delays drift over time?" in a way that respects the fact that a delay series is correlated with itself.

Usage

diagnose_drift(
  x,
  ...,
  stat = c("median", "spread"),
  method = c("hamed-rao", "yue-pilon", "block-bootstrap"),
  by_strata = FALSE,
  strata = NULL,
  mature_only = TRUE,
  level = 0.95,
  alpha = 0.05,
  axis = c("report", "revision")
)

Arguments

Details

For each requested stat (and each stratum) it builds the per-event-date series of that statistic and tests it for a monotonic trend with the modifiedmk package, which corrects the Mann-Kendall variance for serial autocorrelation. A plain Mann-Kendall (or an OLS slope) would be anti-conservative here, because positive autocorrelation shrinks the effective sample size.

By default the test uses only mature event dates (those on or before the level incompleteness cutoff), because the recent, not-yet-fully-reported dates would otherwise inject a spurious downward trend.

Value

A tibble with one row per requested stat per stratum, and the following columns:

strata

character. The stratum the row refers to. When by_strata = FALSE (the default) there is a single stratum labelled "all"; otherwise one level per observed combination of strata.

stat

character. Which delay summary was tested — one of "median", "mean", "iqr" or "spread". "median"/"mean" are location statistics (are delays getting longer?); "iqr"/"spread" are dispersion statistics (are delays getting more erratic?).

n

integer. Length of the tested series, i.e. the number of event dates contributing a non-missing value after the mature_only filter. This is a count of periods, not a count of cases. Series with n < 10 (or with zero variance) are not tested and return NA for every test column.

tau

numeric in ⁠[-1, 1]⁠. Kendall's rank correlation between the statistic and time — the effect size. Positive means delays are growing, negative means they are shrinking. Roughly, ⁠|tau|⁠ below 0.1 is a negligible trend even when p_value is small.

sens_slope

numeric. Sen's slope: the median pairwise rate of change, expressed in delay units per period — so for weekly data with delays measured in weeks, "weeks of delay gained per week elapsed". Multiply by n for the total drift implied across the series. Unlike an OLS slope this is robust to outlying periods.

statistic

numeric. The autocorrelation-corrected Mann-Kendall Z score. Under the null it is standard normal, so ⁠|Z| > 1.96⁠ corresponds to p_value < 0.05.

p_value

numeric. Two-sided p-value for the null hypothesis of no monotonic trend, after the serial-correlation correction implied by method. NA when the series was too short or constant.

method

character. The method actually used, echoed back so the result is self-documenting when several runs are bound together.

drift

logical. The verdict: TRUE when p_value < alpha. NA p-values give FALSE, so a FALSE means "no drift detected" and not necessarily "no drift".

Interpreting the result

Read tau and sens_slope before p_value. On long surveillance series a tiny, operationally irrelevant trend will still be highly significant, so drift = TRUE on its own is not a reason to act. The question to ask is whether sens_slope * n — the total drift implied over the observed window — is large relative to the delays themselves.

The location and dispersion statistics answer different questions and can disagree, which is informative rather than contradictory:

• median drifting up, spread flat: reporting is uniformly slower.

• median flat, spread drifting up: the typical case is unaffected but the tail is getting worse — often a subset of reporting sites degrading.

• both drifting up: broad deterioration in reporting timeliness.

A detected drift means a nowcasting model fitted on a fixed delay distribution will be biased, because it is averaging over delay regimes that are not exchangeable. Consider a model with a time-varying delay, or fitting only to the recent, homogeneous stretch of data.

Because this is a trend test it will not find an abrupt one-off shift; a step change can even cancel out to a non-significant monotonic trend. Pair it with diagnose_changepoint(), which is built for exactly that case.

Choosing a method

All three options are Mann-Kendall tests that correct for the serial correlation of a delay series; they differ in what they assume about that correlation, and in cost.

"hamed-rao" (default)

Inflates the Mann-Kendall variance using all significant autocorrelation lags of the detrended ranks. It makes no AR(1) assumption, is deterministic, and is effectively instantaneous, so it is the sensible default for a routine diagnostic. Its variance correction is known to be unstable on short series — treat results with n below roughly 30 as indicative only.

"yue-pilon"

Trend-free pre-whitening, which effectively assumes the series is AR(1). That assumption is a poor fit for daily reporting delays, which carry strong day-of-week periodicity, and pre-whitening is known to remove part of the very trend being tested. Offered for comparability with the hydrology literature; rarely the right choice here.

"block-bootstrap"

Resamples contiguous blocks, so it accommodates arbitrary dependence within a block — including weekly periodicity, if the block length covers it. Statistically the most defensible for daily data, and the best cross-check when a hamed-rao result is borderline. Two caveats: it is stochastic, so call set.seed() first for a reproducible p-value, and it is thousands of times slower — it scales at roughly the square of the series length, so a multi-year daily series can take many minutes per statistic. Reduce nsim (passed through ...) or restrict to a shorter window before reaching for it.

When a decision matters, run the default first and confirm a borderline result with method = "block-bootstrap" on a restricted window.

See Also

diagnose_changepoint() for an abrupt shift rather than a gradual trend; plot_delay_drift() to see the series being tested; censor_reporting_delays_above() once you decide some delays are not to be believed; diagnose() for the structural checks, which this test deliberately sits outside of.

Examples

data(denguedat)
dengue <- tbl_now(denguedat,
  event_date = "onset_week", report_date = "report_week", verbose = FALSE
)
diagnose_drift(dengue)

Description

Lays out a gallery of complementary views of a tbl_now's reporting process, all aimed at spotting reporting artefacts – especially batch reporting. Each view is also available on its own (see See also); diagnostic_plot() picks the ones named in panels and combines them with patchwork. Selecting a single panel returns it as a plain plot. Every view is facetted by stratum when the tbl_now declares strata.

Usage

diagnostic_plot(
  x,
  panels = "all",
  by = c("report", "event"),
  max_delay = NULL,
  ...,
  plotly = FALSE,
  axis = c("report", "revision"),
  size = 1,
  linewidth = 1,
  grid_linewidth = 0.3,
  palette = .tbl_now_palette()
)

Arguments

Value

A patchwork object, or a single plot when one panel is selected (or a plotly widget when plotly = TRUE).

See Also

Every panel is also a function of its own: plot_reporting_process() and plot_epidemic_process() (when reports arrived, versus when cases happened), plot_reporting_triangle() (the full event-by-delay grid), plot_delay_profiles() (each date's delay curve), plot_delay_drift() (whether delays are getting longer), plot_transport_discriminant().

Examples

data(denguedat)
# The two and a half years around the 1996 and 1997 backlog dumps: enough
# for the transport panel to have something to flag, quick enough to draw.
window <- denguedat[
  denguedat$onset_week >= as.Date("1995-06-01") &
    denguedat$onset_week <= as.Date("1998-01-01"),
]
dn <- tbl_now(window, onset_week, report_week, verbose = FALSE)
diagnostic_plot(dn, panels = c("triangle", "transport"))

Description

An engine is one modelling package plus every argument that package needs. It is what run_nowcast() and nowcast_backtest() take, and it is the object nowcast_fit() and nowcast_tidy() dispatch on.

engine() is the general constructor and works for any registered method, including one you wrote yourself. The ⁠engine_*()⁠ functions are its package-specific counterparts: each names the arguments of the modelling function it drives, so the ones that matter are visible in the signature and at ?engine_nobbs rather than buried in a ... that silently swallows a typo. Anything a named argument does not cover still goes through ....

Usage

engine(
  method,
  ...,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

Arguments

Value

An object of class c(method, "nowcast_engine").

See Also

run_nowcast(), nowcast_backtest(), list_nowcast_methods(), and the Adding your own nowcasting model article for writing a backend of your own.

Examples

engine("baselinenowcast", draws = 500)

# Built-in names are matched case-insensitively
engine("nobbs", max_D = 10)

# The same package twice, told apart by `label`
engine("diseasenowcasting", label = "default")

Description

A deliberately naive nowcasting engine that needs no modelling package. It exists so that the examples in this package can actually run: every real engine depends on epinowcast, NobBS, EpiNow2 or another optional package, and an example that cannot run teaches nothing.

Do not nowcast with this. It does not model the reporting delay at all – it reports the counts that have arrived so far and puts a fixed percentage band around them. Because late reports are exactly what it ignores, it under-predicts recent dates by design, which is a useful thing to see and a terrible thing to rely on. For real work use one of the engines – engine_baselinenowcast(), engine_epinowcast(), engine_nobbs() and the rest – or write your own.

Usage

example_engine(
  ...,
  spread = 0.2,
  min_date = NULL,
  quantile_levels = nowcast_quantile_levels(),
  label = NULL
)

## S3 method for class 'example'
nowcast_fit(
  engine,
  x,
  ...,
  spread = 0.2,
  quantile_levels = nowcast_quantile_levels(),
  verbose = TRUE
)

## S3 method for class 'example'
nowcast_tidy(engine, fit, x, ..., quantile_levels)

Arguments

Details

For each event date (and stratum) it takes the cumulative count reported by now, from get_latest_reported_cases(), and reports that as the median. The other quantile levels are that median scaled linearly by spread, so the 2.5% and 97.5% levels sit at roughly ⁠1 -/+ spread⁠ times it.

No random numbers are involved, so it gives the same answer every time and does not disturb the RNG stream.

Value

A nowcast_engine object, as engine() returns, that run_nowcast() and nowcast_backtest() accept.

See Also

nowcast_engines for the engines you would actually nowcast with; engine() for the general constructor; nowcast_fit() and nowcast_tidy(), the two methods this implements – read its source for the shortest possible complete backend. The Adding your own nowcasting model article walks through writing a real one.

Examples

data(denguedat)
recent <- subset(denguedat, onset_week >= as.Date("2010-06-01"))
dengue <- tbl_now(recent,
  event_date = onset_week, report_date = report_week, verbose = FALSE
)

# It is an ordinary engine, so it goes where a real one goes.
example_engine()

nc <- run_nowcast(dengue, example_engine(), verbose = FALSE)
nc

## `spread` controls how wide the (made-up) interval is.
run_nowcast(dengue, example_engine(spread = 0.5), verbose = FALSE)

Description

FluSight's weekly hospital admission prediction targets based on the 'total number of new hospital admissions of patients with confirmed influenza captured during the reporting week' reported through CDC's NHSN (the dataset formerly known as HHS-Protect), Weekly Hospital Respiratory Data. Data was downloaded on November 12th 2025.

Usage

data(flusight)

Format

A data frame with 452,567 rows and 4 variables:

as_of

Date. The date this row's value was published – the report date. The same week appears many times, once per publication.

target_end_date

Date. The week being reported on – the event date.

location_name

character. US state or territory.

observation

numeric. Hospital admissions reported for that week as of that publication date.

Details

Data represents how many cases were considered influenza during the week of target_end_date given the information known until week as_of. Note that as_of is always one week ahead of target_end_date.

This is count data with 452,567 rows and 4 columns:

• as_of: The report date – the date the snapshot was taken, i.e. what was known as of that week.

• target_end_date: The event date – the week the admissions occurred.

• location_name: State, district or territory (53 levels).

• observation: Case counts for those dates. NA for 1,152 rows.

Together, as_of, target_end_date and location_name form a unique key.

Duplicate rows removed

The upstream FluSight time-series.csv ships exact duplicate rows – 39,139 of them in this snapshot. They were dropped with dplyr::distinct() before the dataset was saved (issue #25), taking it from 491,706 to 452,567 rows.

The removal is lossless: every repeated (as_of, target_end_date, location_name) key carried an identical observation, with no conflicting values anywhere in the file, so no information was discarded and the key became unique. If you download the upstream file yourself you will still see the duplicates and should distinct() them before use.

References

Target data from Flusight. Online: https://github.com/cdcepi/FluSight-forecast-hub/blob/main/target-data/time-series.csv

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(flusight)
head(flusight)

## This is count data: one row per (week, publication date, state).
nrow(flusight)
length(unique(flusight$location_name))

# One state is enough to see the reporting process.
texas <- flusight[flusight$location_name == "Texas", ]
flu <- tbl_now(texas,
  event_date = target_end_date, report_date = as_of,
  case_count = observation, verbose = FALSE
)
flu

# `as_of` is not always the same weekday, so some delays are not whole weeks.
## `align_weeks()` fixes that.
mean(flu$.delay != round(flu$.delay))

Description

The same event date has more than one count, depending on when you look. A week of dengue onsets might show 12 cases the day reporting starts, 40 a week later, and 47 once everything has arrived. These functions let you pick which of those numbers you want.

Usage

get_latest_reported_cases(x, type = "total")

get_initial_reported_cases(x, type = "total")

get_nth_reported_cases(x, delay, type = "total")

Arguments

Details

• get_initial_reported_cases() – the count as first seen: the earliest report for that event date. This is what a dashboard would have shown you at the time, and it is always an undercount.

• get_latest_reported_cases() – the count as latest seen: the most recent report. This is the current best estimate of what really happened, and it is what you score a nowcast against.

• get_nth_reported_cases() – the count accumulated within a given delay. delay = 0 gives the cases reported on the event date itself, delay = 1 adds those reported one period later, and so on. delay = Inf is the same as get_latest_reported_cases().

The gap between the first and the latest count is the reporting delay problem that nowcasting exists to solve.

Value

A count-cumulative tbl_now with one row per event date (and stratum, and grouping column), containing:

• the event-date column – when the cases happened. Its numeric version is .event_num.

• the report-date column – the report that was selected for that event date. Its numeric version is .report_num.

• n – the number of cases reported for that event date at the selected point.

• .delay – the delay of the selected report.

• any strata, covariate, censoring indicator and temporal-effect columns the object carried, plus the caller's grouping columns.

The revision columns are not carried: the count pools over many revision dates, so the result has no single one and does not pretend to. type = "by_type" is the exception – it keeps the outcome column, declared as a covariate, because that is the whole point of the call and an undeclared column is one to_count() would pool away. Use get_latest_revised_cases() when you want the third date on the result.

Grouping is respected

Unlike to_count(), these functions keep the caller's grouping and answer by it: the grouping columns join the event date and the strata as keys, and come back on the result. That is what lets you ask for the latest count by a covariate – a column that matters but is not something you nowcast by – which grouping is the only way to express.

They can do this because they select a point in the process rather than reshaping the object: one row in is still one case (or one cell) out. to_count() cannot, and warns instead.

See Also

get_latest_revised_cases() and friends for the same idea on the revision process; to_count() for the underlying data shapes; score_nowcast(), which uses the latest counts as truth.

Examples

data(denguedat)
# The last five years. The counters work the same on the full twenty-year
# series, they just have more weeks to walk.
recent <- denguedat[denguedat$onset_week >= as.Date("2006-01-01"), ]
dengue <- tbl_now(recent,
  report_date = "report_week",
  event_date = "onset_week",
  strata = "gender",
  verbose = FALSE
)

# What the surveillance system showed the very first time it reported each
# week -- an undercount, because the late reports had not arrived yet.
first <- get_initial_reported_cases(dengue)
first

# What it shows now, after all the corrections.
latest <- get_latest_reported_cases(dengue)
latest

# The difference between them is what a nowcast tries to predict.
sum(latest$n) - sum(first$n)

# Everything known within two weeks of onset.
get_nth_reported_cases(dengue, delay = 2)

# A grouping is answered by, not dropped.
dengue |>
  dplyr::group_by(gender) |>
  get_latest_reported_cases() |>
  dplyr::group_vars()

Description

A line list of healthcare-associated infections (IAAS, Infecciones Asociadas a la Atencion en Salud) notified in the municipality of Bucaramanga, Santander, Colombia, between January 2020 and January 2023 as reported by March 19th 2026. Each row is one notified infection: a specimen taken from a hospitalised patient, the laboratory result, and the isolated microorganism.

Usage

hai_bucaramanga

Format

A tibble with 1,423 rows and 13 variables:

id

integer. The source's Orden autonumber. Documented as unique but repeated for 100 records (see above), so it is not a reliable key.

specimen_date

Date. The event date – when the sample was taken from the patient. NA for 318 records.

received_date

Date. When the laboratory received the sample. NA for 1,194 records and absent after 2019-12-01.

report_date

Date. The report date – when the laboratory result was issued. NA for 583 records.

specimen

factor, 6 levels. Sample type: "Whole blood", "Urine", "Secretions", "Other sterile fluids", "Sputum", "Bronchoalveolar lavage". NA for clinically-confirmed cases.

test

factor, 7 levels. Laboratory test performed, e.g. "Blood culture", "Urine culture". NA for clinically-confirmed cases.

microorganism

character, 85 distinct values. The isolated organism, in conventional binomial form, e.g. "Klebsiella pneumoniae". NA for clinically-confirmed cases. Where the source recorded only a genus the value is "<Genus> spp.".

sex

factor. "Female" or "Male".

age_group

⁠ordered factor⁠, 11 levels from "<1" to "70+".

case_type

factor. "Laboratory-confirmed" (1,090) or "Clinically-confirmed" (333).

final_condition

factor. Patient status on discharge: "Alive" or "Dead". NA for 8 records.

icu_type

factor. Intensive-care unit where the case occurred: "Adult", "Paediatric" or "Neonatal".

institution

integer, 1-10. Anonymised code of the reporting health institution.

Details

In the nowcasting context the event date is specimen_date (when the sample was taken from the patient) and the report date is report_date (when the laboratory result was issued). The delay between the two is the specimen-to-result turnaround that a nowcasting model would estimate and correct for. A third date, received_date, records when the laboratory received the sample and splits the delay into a transport and a processing leg – but see the warnings below before relying on it.

The column names and all categorical values have been translated from the original Spanish. The dataset has been trimmed to the columns relevant to a delay or nowcasting analysis; see data-raw/hai_bucaramanga.R in the package sources for the full translation tables and the list of dropped columns.

Data quality – read this first

This is a real, unpolished open-data extract, and it is included partly because it is messy: it is a realistic exercise for the delay diagnostics in this package (diagnose_drift(), diagnose_changepoint(), plot_delay_profiles()). Nothing below has been silently repaired.

Missing dates

The source uses 1900-01-01 as an undocumented missing-date sentinel. It has been converted to NA here – left in, it produces delays of about -45,000 days. After conversion specimen_date is present for 1,105 records (77.7%), report_date for 840 (59.0%) and received_date for only 229 (16.1%).

received_date is largely unusable

Beyond being 84% missing, it stops entirely at 2019-12-01, so the three-date chain exists only for the first third of the study period. Prefer the specimen_date -> report_date pair.

Negative delays

88 records (10.7% of the otherwise-valid pairs) have a report_date before their specimen_date, by up to 331 days. These are kept exactly as recorded. Filter them out before fitting anything.

Exact duplicates

100 records are byte-identical duplicates of another record, sharing the id that the source documents as a unique autonumber. All 1,423 rows are shipped for fidelity; use dplyr::distinct() to reduce to the 1,323 unique records.

Clinically-confirmed cases

For the 333 cases confirmed on clinical grounds rather than by laboratory, the source wrote the literal string "CONFIRMADO POR CLINICA" into the muestra, ⁠nombre prueba⁠ and microorganismo fields. Those are NA here; the information is preserved in case_type.

Sparsity

Only 738 records (51.9%) support a non-negative specimen_date -> report_date delay, spread over 488 distinct event dates – roughly 1.5 cases per event date. That is thin for fitting a nowcasting model, though ample for delay diagnostics. Aggregating to weeks or months is usually necessary.

The delay distribution is strongly bimodal: the median is 3 days but the 90th percentile is 92 days, which makes this a useful test case for delay diagnostics that assume a unimodal delay.

Source

Secretaria de Salud y Ambiente de Bucaramanga, via the Colombian national open-data portal https://www.datos.gov.co/Salud-y-Protecci-n-Social/48-Infecciones-asociadas-a-la-atenci-n-en-salud-IA/w4zx-wbff/about_data. Retrieved 2026-08-17. Column names and categorical values translated from Spanish; see data-raw/hai_bucaramanga.R for the mapping.

See Also

tbl_now() to declare the date columns, and add_revision_date() for the third one this dataset has; diagnose() and diagnose_drift(), which this dataset is deliberately messy enough to exercise; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia and covid_us.

Examples

data(hai_bucaramanga)

# The source ships 100 exact duplicates and a number of unusable rows.
# Reduce to unique records with a valid, non-negative reporting delay.
iaas_clean <- dplyr::distinct(hai_bucaramanga) |>
  dplyr::filter(
    !is.na(specimen_date), !is.na(report_date),
    report_date >= specimen_date
  )
nrow(iaas_clean)

# Roughly 1.5 cases per event date, so aggregate to weeks before building a
# tbl_now for anything model-shaped.
iaas_now <- tbl_now(
  iaas_clean,
  event_date  = "specimen_date",
  report_date = "report_date",
  verbose     = FALSE
)
iaas_now

# The delay is strongly bimodal: a 3-day median with a long secondary mode.
quantile(iaas_now$.delay, c(0.5, 0.75, 0.9, 0.99), na.rm = TRUE)

Description

Tests whether an object is a nowcasting engine – the specification built by engine() that says which modelling package to use and how to configure it. A bare package name is not an engine, which is what this is usually used to check.

Usage

is_nowcast_engine(x)

Arguments

Value

A single TRUE or FALSE.

See Also

engine() to build one; nowcast_engines for the engines that ship with the package; run_nowcast(), which takes one.

Examples

## Built by engine(): yes.
is_nowcast_engine(engine("baselinenowcast"))

# The name of a package on its own: no.
is_nowcast_engine("baselinenowcast")

Description

Tests whether an object is a fitted nowcast – the thing run_nowcast() returns – rather than the data a nowcast is fitted to (for which see is_tbl_now()).

Usage

is_tbl_nowcast(x)

Arguments

Value

A single TRUE when x is a tbl_nowcast, FALSE otherwise.

See Also

tbl_nowcast for the class itself; run_nowcast() which produces one; is_tbl_now() for the input side.

Examples

# A number is not a nowcast.
is_tbl_nowcast(1)

## The object run_nowcast() returns is.
predictions <- data.frame(
  onset_week = as.Date("2020-01-05"),
  .quantile_level = c(0.5, 0.9), .value = c(10, 14)
)
nc <- tbl_nowcast(
  predictions = predictions, method = "toy", event_date = "onset_week"
)
is_tbl_nowcast(nc)

Description

Reporting almost always slows down at the weekend, which is one of the strongest and most predictable patterns in surveillance data. This tells you which days are which, and lets you say what "weekend" means – it is Friday and Saturday in much of the Middle East, and Sunday alone in some countries.

Usage

is_weekday(date, weekend_days = c("Sat", "Sun"))

Arguments

Value

A logical vector, TRUE where the date is a weekday and FALSE where it falls on the weekend.

See Also

temporal_effects() and add_temporal_effects(), which use this to build the day-of-week and weekend terms a model can fit; plot_day_of_week_effects() to see the effect in the data; align_weeks(), whose align_on_day uses this same ISO numbering.

Examples

is_weekday(as.Date("2020-04-22")) # TRUE (Wed)
is_weekday(as.Date("2020-04-19")) # FALSE (Sun)

## Middle East weekend (Fri - Sat)
is_weekday(as.Date("2020-04-17"), weekend_days = c("Fri", "Sat"))

# Weekend only on Friday
is_weekday(as.Date("2020-04-17"), weekend_days = "Friday")
is_weekday(as.Date("2020-04-18"), weekend_days = "Friday")

## Weekend on Sun - Mon (numeric: 7 = Sun, 1 = Mon)
is_weekday(as.Date("2020-04-20"), weekend_days = c(7, 1))

## Day names of the session's own locale are understood too
locale_weekend <- format(as.Date(c("2020-04-18", "2020-04-19")), "%a")
is_weekday(as.Date("2020-04-18"), weekend_days = locale_weekend)

Description

Scans the S3 methods registered for nowcast_fit() in every loaded namespace, so any backend you (or another package) defined shows up here as soon as it is loaded.

"example" in the list is example_engine(), the toy used to keep this package's examples runnable. It is not a nowcasting method – ignore it when choosing one.

Usage

list_nowcast_methods(installed_only = TRUE)

Arguments

Value

A character vector of method names, suitable for passing to engine().

See Also

engine() to turn one of these names into a configured model; nowcast_engines for the engines that ship with the package; nowcast_fit() and nowcast_tidy(), the two functions a new method must define. The Adding your own nowcasting model article walks through writing one.

Examples

## What can this installation actually fit right now? ("example" is the toy
# engine, not a method you would nowcast with.)
list_nowcast_methods()

# Including methods whose modelling package is not installed
list_nowcast_methods(installed_only = FALSE)

Description

Surveillance line list data provided by the New York City (NYC) Health Department at https://github.com/nychealth/mpox_nowcast_eval, to accompany a nowcasting performance evaluation (doi: 10.2196/56495). Patients with a confirmed or probable mpox diagnosis or illness onset from July 8 through September 30, 2022 were included. The original dataset was aggregated and pre-processed as described in the note below.

Usage

data(mpoxdat)

Format

A data frame with 1,417 rows and 4 variables:

dx_date

Date. Specimen collection date of the first positive result – the event date.

dx_report_date

Date. When the Health Department received that result – the report date.

race

character. Synthetic; see the note.

n

integer. Number of cases with that combination.

Details

This is count data: each row holds the number of cases sharing a diagnosis date, a report date and a race. The columns are as follows:

• dx_date: is the specimen collection date of the first positive mpox laboratory result,

• dx_report_date: is the date the report of first positive mpox laboratory result was received by the NYC Health Department,

• n: the case count of individuals within those dates.

• race: the race corresponding to those cases. Race was randomly assigned with probabilities "Non-Hispanic White" = 0.309, "Hispanic" = 0.283, "Black" = 0.202, "Asian" = 0.156, and "Other" = 0.05 which follow what has been reported for the US Census.

Note

While dx_date, dx_report_date and n correspond to actual observed data the race was constructed exclusively for the examples of this package. It is a synthetic (simulated) variable and does not correspond to any reality.

References

ROHRER, Rebecca, et al. Nowcasting to Monitor Real-Time Mpox Trends During the 2022 Outbreak in New York City: Evaluation Using Reportable Disease Data Stratified by Race or Ethnicity. Online Journal of Public Health Informatics, 2025, vol. 17, no 1, p. e56495.

See Also

tbl_now() to declare the date columns; summary() and diagnose() to inspect the result; the package's other datasets – denguedat, mpoxdat, flusight, covid_colombia, covid_us and hai_bucaramanga.

Examples

data(mpoxdat)
head(mpoxdat)

# Count data, and daily rather than weekly -- unlike denguedat.
mpox <- tbl_now(mpoxdat,
  event_date = dx_date, report_date = dx_report_date,
  case_count = n, strata = race, verbose = FALSE
)
mpox

# A short, sharp outbreak: about three months of data.
range(mpoxdat$dx_date)
sum(mpoxdat$n)

Description

Walks back through time: for every date in now_dates, the tbl_now is truncated to the reports that were available then, each method is refitted on that snapshot, and the resulting nowcast is scored against the resolved truth defined by truth_axis and truth_type (reported totals by default). This is what turns a set of models into ensemble weights (see nowcast_weights() and nowcast_ensemble()).

Be aware that this refits every model once per date: with Bayesian backends and a long now_dates it is genuinely expensive.

Usage

nowcast_backtest(
  x,
  ...,
  now_dates = NULL,
  horizon = 4,
  n_dates = 4L,
  seed = NULL,
  keep_draws = FALSE,
  on_error = c("warn", "abort"),
  verbose = TRUE,
  truth_axis = c("report", "revision"),
  truth_type = "total",
  parallel = FALSE,
  checkpoint_file = NULL
)

Arguments