---
title: "DecisionDrift: Auditing Temporal Drift in Repeated Decision Systems"
author: "Subir Hait"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{DecisionDrift: Auditing Temporal Drift in Repeated Decision Systems}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse  = TRUE,
  comment   = "#>",
  fig.width  = 7,
  fig.height = 4
)
library(DecisionDrift)
set.seed(42)
```

## Overview

`DecisionDrift` provides tools for detecting, decomposing, and stress-testing
**temporal drift** in repeated binary decision systems. It is designed for
settings in which an institutional or algorithmic system issues a binary
decision — approve or deny, flag or pass, intervene or not — repeatedly over
time for the same or overlapping set of units.

The package does not estimate causal effects and does not require full
specification of the underlying decision rule. Instead, it treats the decision
system as a **time-indexed stochastic process** and asks whether that process
changed over time — and if so, how, when, and for whom.

---

## Formal framework

### 1. The decision system as a stochastic process

Let:

- $i = 1, \ldots, N$: units (individuals, cases, applications)
- $t = 1, \ldots, T$: time points (waves, periods, rounds)
- $D_{it} \in \{0, 1\}$: observed binary decision at time $t$ for unit $i$
- $G_i$: group membership (optional)

The decision system is defined as the stochastic process:

$$S = \{ D_{it} : i = 1, \ldots, N;\ t = 1, \ldots, T \}$$

Critically, $S$ is treated as a **generator of decisions** over time, not as
a parametric model. This makes the framework applicable to hybrid human--AI
systems where the decision rule $f_t$ is only partially observed.

### 2. The human--AI decision mechanism

In many applied settings, decisions are produced by a joint human--AI process:

$$D_{it} = f_t(X_{it},\ H_{it})$$

where $X_{it}$ are inputs (features, signals) available to the system,
$H_{it}$ represents human influence (override, discretion, intervention),
and $f_t$ is the time-varying decision mechanism emerging from the
interaction of algorithmic and human components.

Drift in this system means $f_t \neq f_{t'}$, or equivalently:

$$\Pr(D_{it} = 1 \mid X_{it}, H_{it}) \text{ changes over time}$$

`DecisionDrift` audits changes in $f_t$ **without requiring its full
specification**. This is the core methodological contribution.

### 3. Definition of temporal drift

**Marginal drift** (level shift): a change in the marginal distribution of
decisions across waves.

$$\text{Drift exists if } \exists\ t_1 \neq t_2 \text{ such that }
P_{t_1} \neq P_{t_2}$$

where $P_t(d) = \Pr(D_{it} = d)$.

**Transition drift** (dynamic instability): a change in the conditional
distribution given the prior decision.

$$\text{Transition drift exists if } P_t(d_t \mid d_{t-1})
\text{ varies with } t$$

The distinction matters. A system can have **stable prevalence but unstable
dynamics** (transition drift without marginal drift), or **shifting prevalence
with stable dynamics** (marginal drift without transition drift).
`DecisionDrift` separates these two phenomena.

---

## The four drift indices

### Decision Drift Index (DDI)

The DDI estimates the linear trend in wave-level decision prevalence:

$$p_t = \frac{1}{N} \sum_{i=1}^{N} D_{it}$$

$$\text{DDI} = \frac{\hat{\beta}}{\text{SD}(p_t)}$$

where $\hat{\beta}$ is the OLS slope of $p_t$ on $t$. Standardising by
$\text{SD}(p_t)$ makes the DDI scale-invariant and comparable across datasets.

**Properties:**

- $\text{DDI} > 0$: system became more permissive over time
- $\text{DDI} < 0$: system became more restrictive over time
- $\text{DDI} = 0$: no linear trend in decision rate
- Independent of $N$; stable under random reordering of units
- Does not capture nonlinearity or abrupt shifts — this motivates the
  change-point module

### Transition Drift Index (TDI)

Define the time-indexed transition probabilities:

$$\pi_t^{ab} = \Pr(D_{it} = b \mid D_{i,t-1} = a),\quad a, b \in \{0, 1\}$$

The TDI measures average absolute change in persistence and activation
probabilities:

$$\text{TDI} = \frac{1}{T-1} \sum_{t=2}^{T}
\frac{|\pi_t^{11} - \pi_{t-1}^{11}| + |\pi_t^{01} - \pi_{t-1}^{01}|}{2}$$

**Properties:**

- $\text{TDI} = 0$ under stationarity (constant transition probabilities)
- Sensitive to policy or model updates that alter switching dynamics
- Independent of marginal prevalence — captures dynamics, not just levels
- Captures change in the **decision rule itself**, not just its outputs

### Group Differential Drift (GDD)

Let $\text{DDI}^{(g)}$ denote the DDI estimated within group $g$. For groups
$A$ and $B$:

$$\text{GDD}_{AB} = \text{DDI}^{(A)} - \text{DDI}^{(B)}$$

GDD measures **inequality in system evolution**, not static disparity. A
positive $\text{GDD}_{AB}$ indicates the system became more permissive for
group $A$ faster than for group $B$. A value near zero indicates parallel
drift regardless of any baseline disparity.

### Cumulative Drift Burden (CDB)

$$\text{CDB} = \sum_{t=2}^{T} |p_t - p_{t-1}|$$

CDB captures total accumulated volatility in the decision rate, regardless
of direction. Unlike the DDI, CDB is non-zero even when drift oscillates
(e.g., the system alternates between permissive and restrictive periods).
A high CDB with a near-zero DDI indicates instability without a directional
trend.

---

## Modules

The five core modules each estimate one aspect of drift:

| Module | Stochastic object estimated | Index |
|---|---|---|
| `dd_prevalence()` | $\Pr(D_t = 1)$ over time | DDI |
| `dd_transition()` | $\Pr(D_t \mid D_{t-1})$ over time | TDI |
| `dd_entropy_trend()` | Complexity of path distribution | --- |
| `dd_group_drift()` | Heterogeneity in $\Pr(D_t)$ across $G$ | GDD |
| `dd_changepoint()` | Structural break in process | --- |

Together they answer: **Did the system change?** The robustness and
sensitivity modules then ask: **Is that conclusion credible?**

---

## Worked example

```{r data}
# Simulate a system that became more permissive after wave 4.
# Group A drifts faster than Group B.

n_units <- 80
n_waves <- 8

probs_A <- c(rep(0.20, 4), rep(0.60, 4))  # sharp increase at wave 5
probs_B <- c(rep(0.25, 4), rep(0.40, 4))  # modest increase at wave 5

dat <- data.frame(
  id       = rep(seq_len(n_units), each = n_waves),
  time     = rep(seq_len(n_waves), times = n_units),
  decision = c(
    rbinom(40 * n_waves, 1, rep(probs_A, 40)),
    rbinom(40 * n_waves, 1, rep(probs_B, 40))
  ),
  group    = rep(c("A", "B"), each = 40 * n_waves)
)

dp <- dd_build(dat, id, time, decision,
               group      = group,
               event_time = 5L)
print(dp)
```

### Module 1 — Prevalence drift

```{r prevalence}
prev <- dd_prevalence(dp)
print(prev)
plot(prev)
```

The DDI is positive and the slope is statistically significant, confirming
that the overall decision rate increased over time.

### Module 2 — Transition drift

```{r transition}
tr <- dd_transition(dp)
print(tr)
plot(tr)
```

The TDI captures the instability in switching and persistence dynamics that
accompanies the structural break at wave 5.

### Module 3 — Entropy and stability trend

```{r entropy}
et <- dd_entropy_trend(dp, window = 3L)
print(et)
plot(et)
```

### Module 4 — Group-differential drift

```{r group}
gd <- dd_group_drift(dp)
print(gd)
plot(gd)
```

Group A experienced faster drift than Group B (positive GDD). The gap
trajectory shows divergence over time.

### Module 5 — Change-point detection

```{r changepoint}
cp <- dd_changepoint(dp)
print(cp)
plot(cp)
```

Both the CUSUM and segmented regression methods identify a structural break
near wave 5, consistent with the known event time.

### All four indices in one call

```{r indices}
idx <- dd_indices(dp)
print(idx)
```

### Full audit

```{r audit}
aud <- dd_audit(dp, include_robustness = TRUE, verbose = FALSE)
print(aud)
```

---

## Robustness and sensitivity

A key claim of the `DecisionDrift` framework is that observed drift is not
an artefact of analytic choices or data problems. The robustness and
sensitivity modules test this claim directly.

**Robustness** (`dd_robustness()`) tests whether the DDI sign and magnitude
are stable across:

- Balanced vs. unbalanced panels
- Leave-one-period-out (LOPO)
- Leave-one-group-out (LOGO)
- Alternative minimum follow-up requirements

A **fragility index** summarises the proportion of analytic variants in which
the slope changes sign. Values near zero indicate robust conclusions.

**Sensitivity** (`dd_sensitivity()`) probes vulnerability to plausible data
problems:

- Decision miscoding
- Missing-wave attrition
- Threshold shifts (when decisions derive from scores)
- Subgroup composition shifts

A **tipping-point estimate** identifies the smallest perturbation level at
which more than 50 per cent of resampled conclusions flip sign.

```{r robustness}
rob <- dd_robustness(dp, variants = c("lopo", "min_waves"))
print(rob)
plot(rob)
```

---

## Conceptual positioning

`DecisionDrift` is designed to occupy a specific and currently underserved
position in the audit toolbox:

> We define temporal drift as a change in the decision-generating process
> over time and introduce a set of summary indices that quantify level
> shifts, dynamic instability, subgroup divergence, and cumulative change.
> By implicitly treating decision systems as evolving stochastic processes,
> the framework enables longitudinal auditing of hybrid human--AI systems
> without requiring full specification of the underlying decision rule.

This positions the package as **semi-parametric auditing of dynamic
systems** — distinct from full Markov modelling, time-series estimation,
and cross-sectional fairness testing.

---

## Relationship to `decisionpaths`

`decisionpaths` constructs and summarises longitudinal decision paths,
answering *what happened*. `DecisionDrift` tests whether the system that
generated those paths changed over time, answering *did the system change*.
A third package in this ecosystem, `AIBias`, will ask whether inequality
accumulated or amplified over repeated decisions.

---

## References

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of
tests. *Psychometrika*, 16(3), 297--334.

Hait, S. (2025). Artificial intelligence as decision infrastructure:
Rethinking institutional decision processes. Preprint.

Shannon, C. E. (1948). A mathematical theory of communication. *Bell System
Technical Journal*, 27(3), 379--423.