Releasing confidential microdata as
synthetic data
Statistical offices and survey organisations hold microdata — census
records, labour force or household budget surveys — that cannot be
published because individuals could be identified and their confidential
answers disclosed. A synthetic version of the file, generated from a
model fitted to the confidential data, can be released instead: it
reproduces the statistical structure, but its records are not real
persons.
This vignette goes through such a release step by step, and shows how
to choose how much noise is introduced, by looking at
disclosure risk and analytical utility together.
library(SynthesizerPlus)
library(ggplot2)
The confidential file
census_sim is a simulated census-like file with
5000 persons (it contains no real data, so the example can be shared).
It has a direct identifier, demographic variables that an intruder could
know about a person (key variables or
quasi-identifiers), and confidential variables.
str(census_sim)
#> 'data.frame': 5000 obs. of 8 variables:
#> $ person_id : chr "P00001" "P00002" "P00003" "P00004" ...
#> $ age : int 61 31 53 49 37 20 36 32 53 41 ...
#> $ sex : Factor w/ 2 levels "male","female": 2 2 2 2 1 1 1 2 1 1 ...
#> $ region : Factor w/ 8 levels "Nord-Est","Sud-Est",..: 3 8 4 1 8 2 3 8 5 1 ...
#> $ education : Ord.factor w/ 4 levels "primary"<"lower secondary"<..: 3 3 2 3 3 4 2 3 4 3 ...
#> $ employment : Factor w/ 4 levels "employed","unemployed",..: 1 1 1 3 1 1 1 1 1 3 ...
#> $ household_size: int 2 3 3 2 2 4 1 2 3 3 ...
#> $ income : num 5760 2080 4200 580 6540 ...
keys <- c("age", "sex", "region", "household_size") # what an intruder may know
sensitive <- c("income", "education") # what must be protected
Step 1: how risky would releasing the real data be?
disclosure_risk() measures the two classical risks.
- Identity disclosure: records that are
unique on the key variables can be recognised.
replicated_uniques is the share of these unique key
combinations that also occur exactly once in the released file.
- Exact copies: synthetic records identical to a real record
on all variables except the direct identifier
(
ignore = "person_id").
- Attribute disclosure: an intruder who knows a person’s keys
looks up the matching records in the released file and guesses the
sensitive values (most frequent category, or median for numeric
variables, correct if within 10%). This is the correct
attribution probability (CAP). It is compared with a
baseline guess that uses no microdata at all.
risk_real <- disclosure_risk(census_sim, census_sim, keys = keys, target = sensitive,
ignore = "person_id")
risk_real
#> <sp_disclosure> keys: age, sex, region, household_size
#> real records unique on the keys : 33.6% (1679 combinations)
#> replicated uniques : 100.0% of real uniques
#> synthetic records = real record : 100.00%
#>
#> Attribute disclosure (correct attribution probability):
#> target match_rate cap cap_baseline cap_uniques
#> income 1 0.488 0.100 1
#> education 1 0.744 0.403 1
A third of the persons are unique on four harmless-looking variables,
and for them the intruder would learn income and education with
certainty.
Step 2: a synthetic file
The synthesizer is fitted to the confidential file. The direct
identifier is declared in id_cols (it is replaced by new,
meaningless identifiers), and the model is stratified by employment and
sex, because income depends strongly on employment status.
fit <- fit_synthesizer(census_sim, id_cols = "person_id",
by = c("employment", "sex"), knots = 500)
syn <- generate(fit, seed = 1)
head(syn)
#> person_id age sex region education employment
#> 1 syn0001 48 male Vest upper secondary employed
#> 2 syn0002 83 female Bucuresti-Ilfov lower secondary inactive
#> 3 syn0003 61 male Nord-Est primary inactive
#> 4 syn0004 48 male Bucuresti-Ilfov lower secondary employed
#> 5 syn0005 39 male Nord-Est lower secondary employed
#> 6 syn0006 22 male Sud-Est upper secondary employed
#> household_size income
#> 1 3 4598
#> 2 2 1230
#> 3 1 437
#> 4 1 2736
#> 5 4 3804
#> 6 3 5358
disclosure_risk(census_sim, syn, keys = keys, target = sensitive,
ignore = "person_id")
#> <sp_disclosure> keys: age, sex, region, household_size
#> real records unique on the keys : 33.6% (1679 combinations)
#> replicated uniques : 29.1% of real uniques
#> synthetic records = real record : 0.04%
#>
#> Attribute disclosure (correct attribution probability):
#> target match_rate cap cap_baseline cap_uniques
#> income 0.678 0.081 0.100 0.08
#> education 0.678 0.289 0.403 0.31
The correct attribution probabilities fall to — or below — the
baseline: knowing a person’s age, sex, region and household size and
looking them up in the synthetic file tells the intruder no more about
their income or education than general statistics do. Even the
replicated uniques are mostly coincidences, since synthetic key
combinations are generated at random; no synthetic record is a copy of a
real one.
The argument knots = 500 stores each continuous
distribution through 500 quantiles instead of all 5000 observed values,
so that the fitted model itself contains no individual values.
Step 3: near-copies and the amount of noise
Fully synthetic data already protect against attribute disclosure.
The remaining risk is that some synthetic records happen to be very
close to a real person — close enough to be taken as that person.
dcr() measures it: close_share is the share of
synthetic records that are closer to a real record than 95% of the real
persons are to their nearest real neighbour. If synthetic records are no
closer to real persons than real persons are to each other,
close_share is about 5%; clearly larger values signal
near-copies.
conf <- census_sim[-1] # without the identifier
dcr(conf, syn[-1], max_rows = 1500, seed = 1)$close_share
#> [1] 0.04866667
For the model above, the share is at the chance level: no noise is
needed. Near-copies appear when the model is too
detailed for the data, for example when it is stratified into
many small groups. The model below splits the 5000 persons into 64
strata and treats all numeric variables with up to 60 distinct values as
discrete:
detailed <- fit_synthesizer(census_sim, id_cols = "person_id",
by = c("region", "employment", "sex"),
min_stratum_size = 5, discrete_threshold = 60)
The amount of noise is set with closeness (noise level =
1 − closeness): lower closeness smooths numeric values, re-draws some
categories and weakens the dependence. To see what it buys, we compute
for several noise levels the risk (close_share) and the
utility of a typical analysis — a regression of income on demographic
variables, compared with the same regression on the real data through
the overlap of the confidence intervals (mean overlap, negative values
counted as 0; 1 = identical inference). Results are averaged over four
synthetic files per setting; the utility estimates still vary by about
±0.03 from one set of files to another.
model <- income ~ age + sex + education + employment
risk_utility <- function(fit, closeness, seeds = 1:4) {
v <- sapply(seeds, function(sd) {
s <- generate(fit, closeness = closeness, seed = sd)
c(dcr(conf, s[-1], max_rows = 1500, seed = 1)$close_share,
mean(pmax(ci_overlap(census_sim, s, model)$overlap, 0)))
})
data.frame(noise = 1 - closeness, risk = mean(v[1, ]), utility = mean(v[2, ]))
}
ru <- do.call(rbind, lapply(c(1, 0.95, 0.9, 0.85, 0.8, 0.7), function(cl) {
cbind(model = "detailed (64 strata)", risk_utility(detailed, cl))
}))
ru <- rbind(ru, cbind(model = "stratified by employment x sex", risk_utility(fit, 1)))
ru[-1] <- round(ru[-1], 3)
ru
#> model noise risk utility
#> 1 detailed (64 strata) 0.00 0.074 0.741
#> 2 detailed (64 strata) 0.05 0.066 0.505
#> 3 detailed (64 strata) 0.10 0.063 0.414
#> 4 detailed (64 strata) 0.15 0.049 0.380
#> 5 detailed (64 strata) 0.20 0.041 0.305
#> 6 detailed (64 strata) 0.30 0.036 0.234
#> 7 stratified by employment x sex 0.00 0.052 0.719
ggplot(ru, aes(risk, utility, colour = model)) +
geom_path(data = ru[ru$model != ru$model[nrow(ru)], ], colour = "#c3c2b7") +
geom_point(size = 3) +
geom_text(aes(label = paste("noise", noise)), vjust = -1, size = 3,
colour = "#52514e", show.legend = FALSE) +
geom_vline(xintercept = 0.05, linetype = "dashed", colour = "#898781") +
scale_x_reverse(expand = expansion(mult = 0.08)) +
scale_y_continuous(expand = expansion(mult = c(0.08, 0.15))) +
scale_colour_manual(values = c("#eb6834", "#2a78d6"), name = NULL) +
labs(x = "disclosure risk: share of near-copies (axis reversed: right is safer)",
y = "utility: mean CI overlap", title = "Risk-utility map") +
theme_minimal(base_size = 10) +
theme(legend.position = "top", panel.grid.minor = element_blank())
This risk–utility map (Duncan and Stokes, 2004) is the basis
of the release decision: right is safer, up is more useful. It shows two
things.
- For the over-detailed model, noise works as
intended: as the noise level increases, the share of
near-copies falls towards the chance level (dashed line) — and the
utility falls with it.
- The well-specified model is the better starting
point: without any noise it is already at the chance level of
near-copies, with about the same utility as the detailed model without
noise, and clearly more utility than the noisy versions of the detailed
model that reach the same level of risk. Pushing the detailed model
slightly below the chance level is possible, but costs most of
its utility. The first lever for privacy is therefore the model (not too
many small strata,
knots, a sensible
discrete_threshold); noise is the second lever, to be used
in small, calibrated amounts.
Step 4: set the noise to meet a risk threshold
Suppose the release rule says that at most 6% of synthetic
records may be near-copies of real persons, and the detailed model
has to be used (for instance because analysts need region-by-employment
detail). calibrate_closeness() accepts any risk measure as
a function and finds the smallest amount of noise that meets the
rule:
near_copies <- function(real, synthetic) {
dcr(real[-1], synthetic[-1], max_rows = 1500, seed = 1)$close_share
}
cal <- calibrate_closeness(detailed, census_sim, target = 0.06, metric = near_copies,
grid = c(0.7, 0.8, 0.9, 1), refine = 2)
cal
#> <sp_calibration>
#> metric : custom (target 0.06)
#> closeness : 0.875 (achieved custom = 0.060)
#> use: generate(fit, closeness = <value>)
release <- generate(detailed, closeness = cal$closeness, seed = 2026)
The noise level to report with the release is
1 - cal$closeness.
Step 5: check the released file
Before release, verify that the file still supports the intended
analyses and that the risk is acceptable for a new random file:
ci_overlap(census_sim, release, model)[, c("term", "estimate_real",
"estimate_synthetic", "overlap")]
#> term estimate_real estimate_synthetic overlap
#> 1 (Intercept) 2154.33821 3040.70529 -1.2929795
#> 2 age 60.15464 41.31217 -1.5128657
#> 3 sexfemale -555.35019 -510.85137 0.8040922
#> 4 education.L 2388.34816 1850.05810 -1.0155016
#> 5 education.Q 421.78719 367.22721 0.7699952
#> 6 education.C 189.66863 105.18320 0.5995130
#> 7 employmentunemployed -4182.94003 -4115.48143 0.8736580
#> 8 employmentinactive -4235.49214 -4113.36661 0.6195757
#> 9 employmentretired -2546.45769 -2310.42109 0.2124536
disclosure_risk(census_sim, release, keys = keys, target = sensitive,
ignore = "person_id")
#> <sp_disclosure> keys: age, sex, region, household_size
#> real records unique on the keys : 33.6% (1679 combinations)
#> replicated uniques : 31.9% of real uniques
#> synthetic records = real record : 0.00%
#>
#> Attribute disclosure (correct attribution probability):
#> target match_rate cap cap_baseline cap_uniques
#> income 0.666 0.078 0.100 0.068
#> education 0.666 0.306 0.403 0.291
dcr(conf, release[-1], max_rows = 1500, seed = 2)$close_share
#> [1] 0.04066667
plot_marginals(conf, release[-1], vars = c("age", "income", "household_size"))
The marginal distributions and the employment effects survive, but
the effects of age and education on income are clearly attenuated. This
is the price of the noise shown on the risk–utility map; if these
effects matter for the users of the file, the better choice is the less
detailed model, which meets the same risk threshold without noise.
Targeted noise
Noise does not have to be the same for all variables. With
perturbation() it can be concentrated where it is needed —
for instance on the confidential income and on the key variables age and
household size — while the other variables and the dependence structure
are left untouched. (Variables used in by define the strata
and cannot be perturbed.)
targeted <- perturbation(noise = c(income = 0.5, age = 0.3), swap = c(household_size = 0.2))
rel2 <- generate(detailed, perturb = targeted, seed = 1)
c(risk = near_copies(census_sim, rel2),
utility = mean(pmax(ci_overlap(census_sim, rel2, model)$overlap, 0)))
#> risk utility
#> 0.04533333 0.53948555
Good practice
- Choose a model that does not over-fit before adding noise; check the
risk on the risk–utility map.
- Remove or replace direct identifiers (
id_cols).
- Release the synthetic data, never the fitted model:
unless
knots is used, the model stores the observed values
of continuous variables.
- Avoid very small strata (
min_stratum_size), which can
reproduce small groups too faithfully.
- Report the noise level and the risk and utility measures.
- Synthetic data generated this way have no formal privacy
guarantee such as differential privacy. The measures above are
empirical; they depend on the chosen key variables and on what an
intruder may know. Review them with the disclosure control experts of
your organisation.
References
Duncan, G. T. and Stokes, S. L. (2004). Disclosure risk vs. data
utility: the R-U confidentiality map as applied to topcoding.
Chance, 17(3), 16–20.
Nowok, B., Raab, G. M. and Dibben, C. (2016). synthpop: Bespoke
creation of synthetic data in R. Journal of Statistical
Software, 74(11), 1–26.
Taub, J., Elliot, M., Pampaka, M. and Smith, D. (2018). Differential
correct attribution probability for synthetic data: an exploration. In
Privacy in Statistical Databases, LNCS 11126, 122–137.