NA observations (if present
implicitly or explicitly) will be treated as missing instead of zero
when using the default obs_opts(). Options to treat
NA as zeros or accumulate them are also provided in the
message. By @jamesmbaazam in #774 and reviewed by
@sbfnk.A release that introduces model improvements to the Gaussian Process models, alongside a number of other improvements and bug fixes.
rt argument has been expanded
in the case where rt = NULL to make explicit the settings
that are applied in that case. By @jamesmbaazam in #779 and reviewed by
@sbfnk.cmdstanr has also be
surfaced. By @jamesmbaazam in #753 and reviewed by
@seabbs.tolerance argumentgp_opts() function has gained three more
options, “periodic”, “ou”, and “se”, to specify periodic and linear
kernels respectively. By @seabbs in #742 and reviewed by @jamesmbaazam.sd of phi in obs_opts() from 1 to
0.25. By @seabbs in
#742 and reviewed by @jamesmbaazam.list(adapt_delta = 0.95, max_treedepth = 15) to
list(adapt_delta = 0.9, max_treedepth = 12) due to improved
performance and to reduce the runtime of the default parameterisations.
By @seabbs in #742 and
reviewed by @jamesmbaazam.dist_fit() can now accept any number of
samples without throwing a warning when
samples < 1000 in #751 by @jamesmbaazam and reviewed by @seabbs and @sbfnk.epinow() now returns the “timing” output in a “time
difference”” format that is easier to understand and work with. By @jamesmbaazam in #688
and reviewed by @sbfnk.{cli} R package to signal
errors, warnings, and messages. By @jamesmbaazam in #762 and reviewed by
@seabbs.fix_dist() has been renamed to
fix_parameters() because it removes the uncertainty in a
distribution’s parameters. By @sbfnk in #733 and reviewed by @jamesmbaazam.plot.dist_spec now uses color instead of line types to
display pmfs vs cmfs. By @jamesmbaazam in #788 and reviewed by
@sbfnk.{progressr} package for displaying
progress bars is now optional, as is the use of {future}
and {future.apply} for parallelisation. By @sbfnk in #798 and reviewed
by @seabbs.forecast_secondary() did not work
with fixed delays. By @sbfnk in #717 and reviewed by @seabbs.stan_sampling_opts() to add that the dots are passed to
cmdstanr::sample(). By @jamesmbaazam in #699 and reviewed by
@sbfnk.generation_time_opts() has been shortened to
gt_opts() to make it easier to specify. Calls to both
functions are equivalent. By @jamesmbaazam in #698 and reviewed by
@seabbs and @sbfnk .update_rt(). By @seabbs in #747 and reviewed
by @jamesmbaazam.A patch release to further fix an issue with the date in the package
citation. This has now been addressed by removing
inst/CITATION.
A patch release to fix an issue with the date in the package citation.
This release comes with a change of maintainer, from @seabbs to @sbfnk. This is to reflect who will handle the upcoming CRAN submission, but is not expected to lead to a change in workflows.
dist_spec() users now
specify distributions using functions that represent the available
distributions, i.e. LogNormal(), Gamma() and
Fixed(). See ?EpiNow2::Distributions.
Uncertainty is specified using calls of the same nature, to
Normal(). More information on the underlying design can be
found in inst/dev/design_dist.md By @sbfnk in #504 and reviewed by @seabbs.sample_approx_dist(),
report_cases(), and adjust_infection_reports()
have been deprecated as the functionality they provide can now be
achieved with simulate_secondary(). See #597 by @jamesmbaazam and
reviewed by @sbfnk.update_list() has been deprecated
in favour of utils::modifyList() because it comes with an
installation of R. By @jamesmbaazam in #491 and reviewed by
@seabbs.fixed argument to dist_spec has been
deprecated and replaced by a fix_dist() function. By @sbfnk in #503 and reviewed
by @seabbs.get_dist(),
get_generation_time(), get_incubation_period()
have been deprecated and replaced with examples. By @sbfnk in #481 and reviewed
by @seabbs.init_cumulative_fit() has been deprecated.
By @jamesmbaazam
in #541 and reviewed by @sbfnk.weigh_delay_priors argument has been
deprecated in favour of delay-specific prior weighting using
weight_priors. See generation_time_opts(),
delay_opts(), and trunc_opts(). By @sbfnk in #630 and reviewed
by @jamesmbaazam.data argument to pass data. The
existing reported_cases, reports, and
obs arguments are deprecated and will be removed in v2.0.0.
By @jamesmbaazam
in #638 and reviewed by @sbfnk.estimate_infections() so that rather than
imputing missing data, it now skips these data points in the likelihood.
This is a breaking change as it alters the behaviour of the model when
dates are missing from a time series but are known to be zero. We
recommend that users check their results when updating to this version
but expect this to in most cases improve performance. By @seabbs in #528 and reviewed
by @sbfnk.simulate_infections has been renamed to
forecast_infections in line with
simulate_secondary and forecast_secondary. The
terminology is: a forecast is done from a fit to existing data, a
simulation from first principles. By @sbfnk in #544 and reviewed by @seabbs.simulate_infections function has been added that
can be used to simulate from the model from given initial conditions and
parameters. By @sbfnk
in #557 and reviewed by @jamesmbaazam.init_cumulative_fit() has been deprecated.
By @jamesmbaazam
in #541 and reviewed by @sbfnk.dist_spec() users now
specify distributions using functions that represent the available
distributions, i.e. LogNormal(), Gamma() and
Fixed(). Uncertainty is specified using calls of the same
nature, to Normal(). More information on the underlying
design can be found in inst/dev/design_dist.md By @sbfnk in #504 and reviewed
by @seabbs.get_parameters(),
get_pmf(), and get_distribution() have been
added to extract elements of a sample_approx_dist(),
report_cases(), and adjust_infection_reports()
have been deprecated as the functionality they provide can now be
achieved with simulate_secondary(). See #597 by @jamesmbaazam and
reviewed by @sbfnk.forecast_infections function. By @sbfnk in #460 and reviewed by @seabbs.inst/CITATION and added a GitHub
Actions workflow to auto-generate citation.cff so that the
two citation files are always in sync with DESCRIPTION. By
@jamesmbazam in
#467, with contributions from @Bisaloo, and reviewed by @seabbs and @sbfnk.data argument in
estimate_infections() and confirm column in
the obs argument of estimate_truncation() to
allow numeric types, not just integer. See
#594, by @jamesmbaazam, and reviewed by @sbfnk.epinow(),
regional_epinow(), and estimate_infections()
using the truncation argument. By @jamesmbaazam in #644 and reviewed by
@sbnfk.purrr::transpose() with
purrr::list_transpose() because the former is superseded.
By @jamesmbaazam
in #524 and reviewed by @seabbs.estimate_infections(),
estimate_secondary(), estimate_truncation(),
simulate_infections(), and epinow().
check_reports_valid() has been added to validate the
reports dataset passed to these functions. Tests are added to check
check_reports_valid(). As part of input validation, the
various *_opts() functions now return subclasses of the
same name as the functions and are tested against passed arguments to
ensure the right *_opts() is passed to the right argument.
For example, the obs argument in
estimate_secondary() is expected to only receive arguments
passed through obs_opts() and will error otherwise. By
@jamesmbaazam in
#476 and reviewed by @sbfnk and @seabbs.estimate_truncation() has been
simplified. The package now ships with a dataset
example_truncated, which is used in the
estimate_truncation() example and tests. The steps for
creating the example_truncated is stored in
./data-raw/estimate-truncation.R. By @jamesmbaazam in #584 and reviewed by
@seabbs and @sbfnk.simulate_secondary() was added to simulate
from parameters of the estimate_secondary model. A function
of the same name that was previously based on a reimplementation of that
model in R with potentially time-varying scalings and delays has been
renamed to convolve_and_scale(). By @sbfnk in #591 and reviewed by @seabbs.simulate_infections() gained the argument
seeding_time to change the seeding time. Additionally, the
documentation was improved. By @sbfnk in #627 and reviewed by @jamesmbaazam.cmdstanr backend has been added. By @sbfnk in #537 and #642 and
reviewed by @seabbs.phi
to be phi = 1 / sqrt_phi ^ 2 rather than the previous
parameterisation phi = 1 / sqrt(sqrt_phi) based on the
suggested prior here
and the performance benefits seen in the epinowcast package
(see here).
By @seabbs in #487 and
reviewed by @sbfnk.na argument to obs_opts() that
allows the user to specify whether NA values in the data should be
interpreted as missing or accumulated in the next non-NA data point. By
@sbfnk in #534 and
reviewed by @seabbs.log I(t) - log I(t - 1). Originally by @seabbs in #213, finished by
@sbfnk in #610 and
reviewed by @seabbs.filter_leading_zeros and
zero_threshold to estimate_secondary() and
estimate_truncation() to allow the user to specify whether
to filter leading zeros in the data and the threshold for replacing zero
cases. These arguments were already used in
estimate_infections(), epinow(), and
regional_epinow(). See ?estimate_secondary and
?estimate_truncation for more details. By @jamesmbaazam in #608
and reviewed by @sbfnk.This release contains some bug fixes, minor new features, and the initial stages of some broader improvement to future handling of delay distributions.
dist_spec() function. This comes with a range of benefits,
including optimising model fitting when static delays are used (by
convolving when first defined vs in stan), easy printing (using
print()), and easy plotting (using plot()). It
also makes it possible to use all supported distributions everywhere
(i.e, as a generation time or reporting delay). However, while for now
backwards compatibility has been ensured this update will break most
users’ code eventually as the interface has changed. See the
documentation for dist_spec() for more details. By @sbfnk in #363 and reviewed
by @seabbs.ifelse() and
data.table::fifelse() in the code base with
data.table::fcase(). By @jamesmbaazam in #383 and reviewed by
@seabbs.calc_backcalc_data() to call
calc_backcalc_data() instead of
create_gp_data(). By @jamesmbaazam in #388 and reviewed by
@seabbs.tune_inv_gamma(). By @sbfnk in #394 and reviewed
by @seabbs.remotes::install_github("epiforecasts/EpiNow2") if
pak fails and if both fail, raise an issue.dist_fit()’s samples argument now takes a
default value of 1000 instead of NULL. If a supplied
samples is less than 1000, it is changed to 1000 and a
warning is thrown to indicate the change. By @jamesmbazam in #389 and reviewed by
@seabbs.estimate_truncation to allow it
to be applied to time series that are shorter than the truncation max.
By @sbfnk in #438 and
reviewed by @seabbs.estimate_truncation to use the
dist_spec interface, deprecating existing options
max_trunc and trunc_dist. By @sbfnk in #448 and #452 and
reviewed by @seabbs.weigh_delay_priors argument to the main
functions, allowing the users to choose whether to weigh delay priors by
the number of data points or not. By @sbfnk in #450 and reviewed by @seabbs.EpiNow2 to the case studies vignette. By @seabbs in #441 and reviewed
by @sbfnk.This is a minor release to resolve issues with the recent CRAN
requirement to make use of a C++ 17 compiler which has been causing issues with the
rstantools package.
This release focusses on bug fixes and package infrastructure updates along with a few quality of life improvements such as enabling the use of fixed delays and generation times.
Thanks to @seabbs, and @sbfnk and for the South African Centre for Epidemiological Modelling and Analysis (SACEMA) for hosting @seabbs whilst some of the development work on this release was being done.
trunc_opts() now takes a single argument
(dist) which defines the truncation delay rather than a
arbitrary list of arguments (which was previously used to define the
distribution).frac_obs parameter has restricted with
an upper bound of 1 to reflect its name and description. This may impact
a small number of edge case uses with the majority being models fit
using estimate_secondary(). By @sbfnk in #340.simulate_secondary() for simulating
secondary observations under the generative process model assumed by
estimate_secondary. Unlike
forecast_secondary() which uses a stan model
to simulate secondary cases (which shares code with the
estimate_secondary model) this new function is implemented
entirely in R and is therefore useful to sense check any results from
the stan implementation.seq_along() rather than
1:length() in all package code.regional_runtimes().rstan and rstantools.pkgnet for package dependency
visualisation.styler.RcppParallel.report_cases to work with the new
delay_opts helper function.report_cases though note this
function will likely be deprecated in future releases.linewidth in plot_CrIs rather
than size to avoid issues with ggplot2
3.4.0.get_ distribution functions to return the
distribution as well as summary parameters.touchstone.estimate_infections().simulate_infections() where passing a
custom number of samples would cause the input vector of R values to be
replicated in a column-wise fashion meaning that the intended R
trajectory was not simulated.estimate_infections() deconvolution
model where the generation time was not correctly being reversed.This release adds a range of new minor features, squashes bugs, enhances documentation, expands unit testing, implements some minor run-time optimisations, and removes some obsolete features.
Thanks to @Bisaloo, @hsbadr, @LloydChapman, @medewitt, and @sbfnk for contributing to this release.
Thanks to @sbfnk,
@pearsonca, and
@nicholasdavies for regression
testing this release against 1.3.2.
simulate_infections so that a
data.frame of R samples can be passed in instead of a
vector of R values. By @seabbs.estimate_infections. By @seabbs.zero_threshold to users allowing for control
over when zeros or NAs in count data are treated as true zeros versus as
reporting errors that require some smoothing. By @seabbs.obs_opts()). This allows, for example, fitting to data
with cases only reported every 3 days. By @seabbs.plot_estimates() and higher level
functions to choose which estimate type to plot. By @seabbs.plot) to
regional_summary to allow plotting to be optional. Closes
#250. By @seabbs in
#317obs_opts()). This allows, for example, fitting to data
with cases only reported every 3 days.target likelihood definition approach when required and in
the use of fmax and fmin over using if
statements. By @seabbs.phi argument of obs_opts
with the default of a normal distribution with mean 0 and standard
deviation of 1 truncated at 0 remaining unchanged. By @seabbs.estimate_truncation
model to deal with zeroes. By @sbfnk.epinowcast by
Adrian Lison and Sam Abbott. By @seabbs in #320.contributing.md to guide contributors and added
pre-commit support to check new contributions styling. By
@seabbs.cowplot::theme_cowplot() to
ggplot2::theme_bw(). This allows the removal of
cowplot as a dependency as well making plots visible for
users saving as pngs and using a dark theme. By @seabbs.epinow and downstream functions remove
leading zeros. Now this is optional with the new
filter_leading_zeros option. Thanks to @LloydChapman in
#285.estimate_secondary(), forecast_secondary(),
and estimate_truncation(). By @seabbs in #315.adjust_infection_to_report. By @seabbs in #316.rstantools to manage compiler flags.future::multisession() from
future::multiprocess() as the latter is being depreciated
in the future package. By @seabbs and @sbfnk.simulate_cases() and simulate_infections()
have been deprecated and have been removed. These functions depend on
EpiSoon which itself is archived and near equivalent
functionality is available within EpiNow2 and in other
packages (@seabbs).simulate_infections() (i.e in `epinow()) have been removed
along with the arguments that supported them (@seabbs).global_map(), country_map(), and
theme_map() have all been deprecated and have been removed.
These functions were used to support reporting of reproduction number
estimates and are considered out of scope for EpiNow2. If
finding useful contacting the EpiNow2 developers (@seabbs).simulate_infections and
forecast_secondary which meant that a Poisson observation
model used for estimation would lead to a error. By @seabbs.use_rt = FALSE did not properly
cancel user settings. By @sbfnk.estimate_truncation where phi was not
initialised. By @sbfnk.zero_threshold was being ignored and
so no post-processing was happening. To maintain backwards compatibility
the default has been changed to Inf (i.e. no zero
threshold). By @LloydChapman in #285.obs_opts(return_likelihood = TRUE) fails. By @sbfnk in #333.In this release model run times have been reduced using a combination
of code optimisation and testing to reduce the likelihood of long
running edge cases. Model flexibility has also been increased,
particularly for the back calculation approach which now supports an
increased range of prior choices. A significant development in this
release is the edition of the experimental
estimate_secondary model (and supporting
forecast and plot functions). This allows a
downstream target to be forecast from an observation. Example use cases
include forecasting deaths from test positive cases and hospital bed
usage from hospital admissions. This approach is intended to provide an
alternative to models in which multiple targets are estimated
jointly.
prior, to
backcalc_opts(). This allows the use of different priors
for the underlying latent infections when estimating using
deconvolution/back-calculation rather than the package default of using
a generated Rt model (enable this option by setting
rt = NULL). The default prior remains smoothed mean delay
shifted reported cases but optionally no prior can now also be used (for
scenarios when the data is very untrustworthy but likely to perform
extremely poorly in real time).In addition, the previously estimated
infections can be used (i.e infections[t] = infections[t-1] * exp(GP))
with this being an approximate version of the generative Rt model that
does not weight previous infections using the generation time.prior = "reports" to be a
partial centred moving average rather than a right aligned moving
average. This choice means that increasing the prior window
does not alter the location of epidemic peaks as when using a right
alighted moving average.init_fit to
stan_opts() that enables the user to pass in a
stanfit to use to initialise a model fit in
estimate_infections(). Optionally
init_fit = "cumulative" can also be passed which first fits
to cumulative data and then uses the result to initialise the full fit
on incidence data. This approach is based on the approach used in epidemia
authored by James Scott. Currently stan warnings from this
initial fit are broadcast to the user and may cause concern as the short
run time and approximate settings often lead to poor convergence.estimate_secondary and
forecast_secondary along with a plot method
and a new option function (secondary_opts()). These
functions implement a generic model for forecasting a secondary
observation (such as hospital bed usage, deaths in hospital) that
entirely depends on a primary observation (such as hospital admissions)
via a combination of convolving over a delay and adding/subtracting
current observations. They share the same observation model and optional
features used by estimate_infections and so support data
truncation, scaling (between primary and secondary observations),
multiple log normal delays, a day of the week effect, and various error
models. stationary_opts() allows for easy specification of
the most common use cases (incidence and prevalence variables). See the
documentation and examples for model details.discretised_gamma_pmf (discretised truncated
Gamma PMF) and discretised_lognormal_pmf (discretised
truncated lognormal PMF) to limit/clip the values of the parameters by
prespecified lower and upper bounds.estimate_infections by reducing all standard deviations
used by a scaling factor of 0.1 in
create_initial_conditions.gt_mean (the mean of the
generation time) to reject samples with a mean greater than
gt_max (the maximum allowed generation time). Adds boundary
checking to reject standard deviations that are negative. Adds a
boundary check on R values to reject them if 10 times greater than the
mean of the initial prior. In some scenarios this will require users to
supply a prior not is not completely misspecified (i.e if the prior has
a mean of 1 and the posterior has a mean of 50).update_rt (an internal stan
function found in inst/stan/functions/rt.stan) to be
vectorised. This change reduces run times by approximately 1- ~ 20%
(though only tested on a small subset of examples) and opens the way for
future model extensions (such as additive rather than multiplicative
random walks, and introducing covariates).This release focusses on model stability, with a functional rewrite
of the model implementation, finalising the interface across the
package, and introducing additional tooling. The additional tooling
includes: support for adjusting for and estimating data truncation,
multiple approaches for estimating Rt (including the default generative
Rt approach, de-convolution coupled with Rt calculation, and
EpiEstim like estimation on observed cases but with a
robust observation model), optional scaling of observed data, and
optional adjustment of future forecasts based on population
susceptibility. The examples have also been expanded with links out to
Covid-19 specific work flows that may be of interest to users. The
implementation and model options are now considered to be maturing with
the next release planned to contain documentation on the underlying
approach, case studies, validation, evaluation the various supported
options, and tools for dealing with secondary reports that are dependent
on a primary report (i.e hospital admissions and hospital bed usage). If
interested in contributing to any of these features please contact the
package authors or submit a PR. User contributions are warmly
welcomed.
estimate_infections to be
divided into calls to _opts() functions. Options are now
divided by type for delays (delay_opts()), Rt
(rt_opts()), backcalculation
(backcalc_opts()), the Gaussian process
(gp_opts()), and stan arguments (stan_opts()).
This has resulted in a larger number of the arguments from
estimate_infections being folded into the related
_opts() function. Please see the function documentation and
examples for details._opts() function in regional_epinow
using the helper functions opts_list and
update_list or alternatively by constructing a named list
with an entry for each region to be estimated.{EpiEstim} (here with a default window of 1 but this can be
updated using backcalc_opts(rt_window)) but this approaches
incorporates uncertainty from all inputs in a single estimate.epinow and
estimate_infections.simulate_infections function from @sbfnk which allows the
simulation of different Rt traces when combined with estimates as
produced by estimate_infections. This function is likely to
form the basis for moving all forecasting out of
estimate_infections which may improve model stability.gp_opts(). The length scale is now defined using a log
normal truncated prior with a mean of 21 days and a standard deviation
of 7 days truncated at 3 days and the length of the data by default.
This prior is an area of active research and so may change in future
releases.1 / sqrt(half_normal(0, rho_prior)) based on this
advice and as the over dispersion being measured is in reports and not
infections and hence a priori there is not strong evidence for over
dispersion (which may be the case for infections) so the previous prior
was overly weighted towards this.obs_opts(). This removes
week_effect and family from the main argument
list which will allow for future extensions. Also adds a new argument
scale which controls the uncertain fraction of cases that
are eventually observed (defined as normally distributed). Setting this
parameter will not impact Rt estimates.rt, using rt_opts(), this includes the
initial prior,use_breakpoints, and future.
Adds a new helper argument rw which enables easy
parameterisation of a fixed length random walk. These changes also help
make it clear that these arguments only impact the Rt generative model
and not the back calculation model.{epidemia}
when Rt is fixed into the future (set by passing a population to
rt_opts(pop = initial susceptible population). Note this
only impacts case forecasts and not output Rt estimates and only impacts
estimates at all beyond the forecast horizon as those based on data
already account for population susceptibility by definition. The impact
of this assumption can be explored using
simulate_infections (by updating est$arg$pop
in the example).truncation as a new argument to
estimate_infections and higher level functions. This takes
output from trunc_opts() and allows for internally
adjusting observed cases for truncation. A new method
estimate_truncation has also been added to support
estimating a log normal truncation distribution from archived versions
of the same data set though this method is currently experimental.estimate_delay as a user friendly wrapper around
bootstrapped_dist_fit.epinow and in regional_epinow
for large batch runs.regional_epinow) to be based on whether or not usage is
interactive.burn_in argument of
estimate_infections as updates to model initialisation mean
that this feature is likely no longer needed. Please contact the
developers if you feel you have a use case for this argument.convert_to_logmean and
convert_to_logsd).expose_stan_fns that exposes the
internal stan functions into R. The enables unit testing, exploration of
the stan functionality and potentially within R use cases for these
functions.warmup to be 250 samples and the
default adapt_delta to be 0.98.{lifecycle} badges
to all functions to indicate development stage.This release introduces multiple breaking interface changes. Please
see the README for examples of the new interface. It adds a range of
quality of life improvements including updating the stan
interface to support fitting each chain independently and offering
variational inference as an alternative, experimental, fitting option.
Notably it also adds support for nesting logging and a parallel enabled
progress bar via the progressr package. Minor bugs have
been fixed in the core model implementation focussing on stability and
several already implemented features have been extended. Major model
developments are planned for the next release of
EpiNow2.
method = "exact") or Variational inference
(method = "approximate").get_dist, get_generation_time,
get_incubation_period based on ideas from @pearsonca. (This leads
to breaking changes with the removal of
covid_generation_times and
covid_incubation_periods).setup_logging to enable users to specify the
level and location of logging (wrapping functionality from
futile.logger). Also added
setup_default_logging to give users sensible defaults and
embedded this function in regional_epinow and
epinow.setup_future to making using nested futures
easier (required when using future = TRUE).progressr.regional_epinowregional_epinowregional_runtimes that summarises
the run time across regions.estimate_infections interface and expanded
the range of options for the future_rt argument. Users can
now choose to set Rt from any time point referenced to the forecast
date.plot_summary.discretised_gamma_pmf (discretised truncated
Gamma PMF) to constrain gamma shape and (inverse) scale parameters to be
positive and finite (alpha > 0 and
beta > 0).readLines incomplete final line warnings.fit_chain function where an interaction between
rstan and timing out may have introduced an exception that
caused whole regions to fail. This did not show on current unit tests or
exploration using examples indicating a gap in testing.create_stan_args with the option to override using
stan_args. This leads to breaking changes - see the
examples for details of the new interface.samples argument to
get_regional_results to make loading in samples optional.
This also allows samples to be dropped when using
regional_epinow which reduces RAM usage.global_map and country_map
expanded by @ellispglobal_map from @ellispregional_summary now saves input reported cases data
reported_cases.csv.