synth() regenerates every column from a model fitted on the real data, using a
freshly drawn skeleton — so its output is fully synthetic in the Reiter (2003)
sense. Run the synthesiser twice and the analysis estimates can move. Analysing
only one synthetic dataset does not quantify that between-synthesis variation;
depending on the estimand, synthesis model, sample size, and analysis, its usual
standard errors can therefore be misleading.
This is why synth() can emit m > 1 datasets. Fit the analysis on each and
combine the results using a published fully-synthetic rule. flexsynth provides
two such rules; their large-sample calibration still relies on the assumptions
of the synthesis and analysis models.
n <- 800
real <- data.frame(
id = seq_len(n),
age = round(rnorm(n, 62, 11)),
sex = sample(c("F", "M"), n, replace = TRUE, prob = c(0.45, 0.55))
)
real$sbp <- round(0.6 * real$age + ifelse(real$sex == "M", 5, 0) +
rnorm(n, 90, 10))
res <- synth(real, structure = ~ id, m = 10, seed = 1)
res
#> <synth_result>
#> track : A (high-utility; NOT differentially private)
#> datasets (m) : 10
#> rows each : 800 (input: 800 )
#> synthesised : age, sex, sbp
#> unit-level : age, sex, sbp (once per unit)
#> method : cart
#>
#> Get the data with as.data.frame(x) or x$syn[[i]]
We asked for m = 10 synthetic datasets. Increasing m reduces Monte Carlo
variation in the pooled result; m >= 5 is a practical starting point, not a
universal adequacy threshold.
synth_glm() — the common caseFor a linear or generalised-linear model, synth_glm() fits it on every
synthetic dataset and pools the result in one call.
pooled <- synth_glm(res, sbp ~ age + sex)
pooled
#> <flexsynth_pool> pooled inference from 10 synthetic dataset(s)
#> rule: synthpop population inference (n = 800, k = 800)
#> term estimate std.error statistic p.value conf.low conf.high
#> (Intercept) 87.20280 2.31110 37.732 1.47e-311 82.67313 91.73247
#> age 0.64033 0.03574 17.915 9.03e-72 0.57027 0.71038
#> sexM 5.74845 0.83329 6.898 5.26e-12 4.11523 7.38168
Compare that with the model fitted on the real data — the estimand we are trying to recover:
summary(lm(sbp ~ age + sex, data = real))$coefficients[, 1:2]
| Estimate | Std. Error | |
|---|---|---|
| (Intercept) | 86.199 | 2.128 |
| age | 0.655 | 0.033 |
| sexM | 5.694 | 0.748 |
In this simulation, the pooled estimates should be close to the generating
coefficients (an age slope of 0.6 and a male offset of 5). The reported
standard errors apply the selected combining rule to the within- and
between-synthesis quantities; they are not a guarantee of calibrated coverage
for every dataset or model.
By default the standard error targets the population the real data was drawn
from. If instead you want the standard error an analyst of the original sample
would have reported, set population_inference = FALSE.
pool_synth() — any estimatorsynth_glm() is a thin wrapper. For anything else — a quantile, a survival
model, a hand-rolled statistic — pass pool_synth() a function that runs on one
synthetic data.frame and returns either a fitted model (with coef() /
vcov() methods) or an explicit list(estimate =, variance =).
# A proportion and its variance, supplied directly.
pool_synth(res, function(d) {
p <- mean(d$sex == "M")
list(estimate = c(prop_male = p),
variance = c(prop_male = p * (1 - p) / nrow(d)))
})
#> <flexsynth_pool> pooled inference from 10 synthetic dataset(s)
#> rule: synthpop population inference (n = 800, k = 800)
#> term estimate std.error statistic p.value conf.low conf.high
#> prop_male 0.55912 0.0184 30.391 7.26e-203 0.52307 0.59518
Two combining rules are available via rule =: the default "synthpop" rule
matching Raab, Nowok & Dibben (2016), whose implemented variance estimate is
non-negative, and the "reiter" fully-synthetic rule from Reiter (2003), which
needs m >= 2 and can produce a negative variance estimate (reported as NA).
Choose the rule to match the release and inferential target rather than by the
size of its standard error.
compare_estimates() — analysis-specific utilitydiagnose() (see vignette("getting-started")) asks a general question: can a
model tell real from synthetic records? compare_estimates() asks the question
an analyst actually cares about: does my regression come out the same? It fits
the analysis on both sides — pooling the synthetic side automatically — and
reports, per term, the confidence-interval overlap (Karr et al. 2006; 1 means
identical intervals, 0 means they just touch, negative means disjoint) and the
standardised difference of the estimates.
compare_estimates(real, res, function(d) lm(sbp ~ age + sex, data = d))
#> <flexsynth_utility> real-vs-synthetic estimate comparison (synthetic pooled)
#> CI overlap (1 = identical, <0 = disjoint) at 95%; std_diff = |est_real - est_syn| / se_real
#> term est_real est_syn overlap std_diff
#> (Intercept) 86.1992 87.2028 0.886 0.472
#> age 0.6549 0.6403 0.893 0.443
#> sexM 5.6942 5.7485 0.949 0.073
#> mean overlap: 0.909
Larger interval overlap and smaller standardised differences indicate better agreement for this chosen analysis. They are utility diagnostics, not evidence that every analysis will agree or that the synthetic data is safe to release.
Reproducing an analysis well is a utility win, but the same key-to-target
structure an analyst exploits is what an attacker exploits. disclosure_risk()
gains a target = argument for exactly this: the Target Correct Attribution
Probability (TCAP; Taub, Elliot et al.). An attacker who knows a real record’s
quasi-identifier keys looks up the synthetic records sharing those keys and reads
off the conditional distribution of the sensitive target. TCAP averages the
synthetic probability of the true value over real records whose keys occur in
the synthetic data. The reported baseline evaluates a marginal-only attacker
on those same covered records, so lift is the excess attribution the keys
buy. Inspect coverage as well: TCAP can otherwise look favourable simply
because difficult records have no synthetic key match. The
baseline_unconditional value provides context across all scored real records.
TCAP is meant for categorical keys and target, so we use a small categorical dataset where a sensitive status depends on the quasi-identifiers.
m <- 800
clinic <- data.frame(
region = sample(c("N", "S", "E", "W"), m, replace = TRUE),
agecat = sample(c("40s", "50s", "60s", "70s"), m, replace = TRUE)
)
# hypertension risk rises steeply with age, with a small regional bump.
prob_htn <- c("40s" = 0.12, "50s" = 0.40, "60s" = 0.75, "70s" = 0.93)
reg_bump <- c("N" = 0, "S" = 0, "E" = 0.03, "W" = 0.05)
p <- pmin(0.98, prob_htn[clinic$agecat] + reg_bump[clinic$region])
clinic$hypertension <- ifelse(runif(m) < p, "yes", "no")
clinic$id <- seq_len(m)
cres <- synth(clinic, ~ id, m = 1, seed = 1)
Name the sensitive column as target; the remaining quasi-identifiers become the
attacker’s keys.
disclosure_risk(clinic, cres,
quasi = c("region", "agecat", "hypertension"),
target = "hypertension",
seed = 1)
#> <flexsynth_disclosure>
#> rows : real 800 synthetic 800
#> quasi-identifiers: region, agecat, hypertension
#>
#> Replicated uniques (identity risk):
#> real sample-uniques : 3
#> reproduced in syn : 3 (100.00% of uniques, 0.38% of real rows)
#> syn rows copying a real row: 800 (100.00% of syn)
#>
#> Distance to closest record (Gower, 0 = exact copy):
#> syn->real : median 0.0000 5th pct 0.0000 exact copies 100.00%
#> real->real : median 0.0000 (baseline)
#> median syn distance is not smaller than the real-neighbour baseline
#> descriptive only: inspect lower-tail distances and exact copies; this is not a safety guarantee
#>
#> Membership inference: not run (supply `holdout` of non-training records).
#>
#> Attribute disclosure (TCAP, target = hypertension):
#> TCAP 0.707 baseline 0.506 lift +0.200 (coverage 100.0%, keys: region, agecat)
#> key-conditioning attributes the target well above the covered-record margin -> inspect
Read the Attribute disclosure block: a lift well above zero means the
synthetic data lets an attacker attribute the true hypertension status
meaningfully better than the marginal rate alone — here because agecat carries
real signal about it. That is a governance signal, not a verdict: some attribute
association is the whole point of useful data. Weigh the lift against how
sensitive the target is and who receives the release. A lift near zero means
the keys add almost nothing beyond the margin.
Track A output must never be described as differentially private. The tools here
provide evidence for a documented release review, but the release decision must
also consider data sensitivity, plausible attackers, access controls, and
independent governance review. For a formal privacy guarantee instead, see
vignette("differential-privacy") and verify all of its stated domain and
budget-accounting conditions. Pooled inference (pool_synth(), synth_glm())
is a Track A tool — inference from a differentially private release must also
account for DP noise and is out of scope here.