---
title: "weightflow in production (GSBPM 5.6)"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{weightflow in production (GSBPM 5.6)}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(weightflow)
```

In the Generic Statistical Business Process Model (GSBPM), the construction of
analysis weights is sub-process 5.6 ("calculate weights") and the production of
estimates and their variances is 5.7 ("calculate aggregates"). In a statistical
office this step is not a one-off script: it is a governed, auditable and
repeatable part of the production line, run every wave, reviewed, and archived.
This article shows how weightflow supports that workflow. Every piece below
already exists in the package; the point is to use them together.

## The recipe is the artifact

The core idea is that the whole weighting process is a single declarative
object. You define it once, estimate it with `prep()`, and everything else
(diagnostics, variance, the report) reads from that one object. The recipe is
what you version, review in a pull request, and archive alongside the estimates.

```{r recipe}
rec <- weighting_spec(sample_survey, base_weights = pw) |>
  step_nonresponse(respondent = responded, method = "weighting_class",
                   by = "region") |>
  step_calibrate(method = "raking", id = "calib_main",
                 margins = list(region = c(table(population$region)),
                                sex    = c(table(population$sex))))
fit <- prep(rec)
summary(fit)
```

Because the definition is separated from the execution, the same recipe object
is the specification, the documentation and the input to the variance machinery.
Each step has a stable id (here `calib_main`, otherwise a derived one), so a step
can be referenced from a script long after the run.

## A programmatic quality gate

Production pipelines need a machine-readable pass or fail, not a human reading a
report. `prep()` records every quality incident in `$alerts`, readable with
`weighting_alerts()` and `has_alerts()`. That is the hook for a continuous
integration check: fail the build when the recipe raises an incident.

```{r gate}
if (has_alerts(fit)) {
  # in CI: stop() here so the pipeline fails and the run is not published
  weighting_alerts(fit)
} else {
  "no quality incidents"
}
```

`?weightflow-alerts` catalogues the incidents `prep()` can raise, with the
trigger and the remedy for each. For a hard threshold that must hold (a maximum
design effect, a minimum effective sample size), add a `step_assert()`: it
errors at that point of the cascade if the condition fails, so a recipe that
violates it never produces weights at all.

```{r assert, eval = FALSE}
rec |> step_assert(max_deff = 2.5, min_n_eff = 500)
```

## Reproducibility

The replication functions take a single `seed`, and draw the whole resampling
pattern from it up front, so a parallel run is bit-identical to the serial one.
Fix the seed, and record the exact package and R versions used (for example in a
lockfile), because a flexible learner such as a random forest can change between
versions of its engine. The report's reproducibility card records the versions
at run time.

```{r seed}
boot <- bootstrap_weights(fit, replicates = 100, strata = "region",
                          psu = "psu", seed = 20260601, progress = FALSE)
boot_mean(boot, "income")
```

## The report is the quality document

`report_weighting()` writes a self-contained HTML report: the cascade in prose,
the target and achieved control totals, the design effect and effective sample
size, the fieldwork outcomes and response rates when the recipe has eligibility
and nonresponse steps, and a reference-metadata header aligned to the ESS SIMS
concepts and GSBPM 5.6. Archive that HTML next to the released estimates; it is
the artifact a reviewer reads.

```{r report, eval = FALSE}
report_weighting(
  fit, replicates = boot, file = "weights_2026.html", lang = "en",
  metadata = list(
    survey           = "Continuous Household Survey",
    reference_period = "2026",
    producer         = "National Statistical Office",
    frame            = "Population and housing census 2023",
    totals_source    = "Population projections 2026",
    version          = "1.0"))
```

## Variance and dissemination

The recipe-aware bootstrap and jackknife re-run the whole recipe on each
replicate, so the replicate weights carry the variability of every adjustment.
Many offices release replicate weights alongside public-use microdata so that
external users can compute design-consistent variances without the full design.
`collect_replicate_weights()` returns the point weight and every replicate weight
as ordinary columns, ready to write out and to load into `survey` or `srvyr`.

```{r disseminate, eval = FALSE}
pub <- collect_replicate_weights(boot)          # point + replicate weights
# survey / srvyr read them directly:
des <- as_svrepdesign(boot)
```

The point weights and replicate weights also flow into `survey` through
`as_svydesign()` / `as_svrepdesign()`, so downstream estimands and domains use
the design-based machinery the office already trusts.

## Where it sits

The cascade maps onto the total survey error framework: unknown-eligibility
redistribution and dropping out-of-scope units address coverage error, the
within-household selection restores the design, the nonresponse step addresses
nonresponse error, and calibration reduces coverage bias and improves precision.
Placing the whole process in one auditable, re-runnable object is what the
quality frameworks ask for: the UN Fundamental Principles of Official Statistics
and the European Statistics Code of Practice both require sound methodology,
transparency and reproducibility. weightflow does not replace the parts of 5.6
and 5.7 that belong to other tools (small-area estimation, editing, imputation);
it is the reproducible spine that documents and computes the weights, and bridges
to those tools honestly.

## See also

`vignette("variance-estimation")` for the replicate methods, `vignette("quality-report")`
for the report, `vignette("inspecting-auditing")` for the programmatic
quality-control accessors, and `?weightflow-alerts` for the alert catalogue.