irtsim: Monte Carlo Simulation-Based Sample-Size Planning for Item Response Theory

Description

Provides a pipeline application programming interface (API) for Monte Carlo simulation-based sample-size planning in item response theory (IRT). Implements the 10-decision framework from Schroeders and Gnambs (2025) doi:10.1177/25152459251314798 as a three-step workflow: specify the data-generating model with irt_design(), add study conditions with irt_study(), and run simulations with irt_simulate(). Supports one-parameter logistic (1PL), two-parameter logistic (2PL), and graded response models with missing-completely-at-random (MCAR), missing-at-random (MAR), booklet, and linking missingness mechanisms. Results include mean squared error (MSE), bias, root mean squared error (RMSE), standard error (SE), and coverage criteria with summary and plot methods.

Author(s)

Maintainer: Stephen Ward stephen_ward+irtsim@abhome.co

See Also

Useful links:

• https://github.com/sward1/irtsim

• Report bugs at https://github.com/sward1/irtsim/issues

Internal Criterion Registry

Description

Internal Criterion Registry

Usage

.get_criterion_registry()

Value

A named list of criterion configurations.

Internal Missing Mechanism Registry

Description

Internal Missing Mechanism Registry

Usage

.get_missing_registry()

Value

A named list of missing mechanism configurations.

Internal Model Registry

Description

Internal Model Registry

Usage

.get_model_registry()

Value

A named list of model configurations.

Apply Missing Data Mechanism (Internal)

Description

Takes a complete response matrix and introduces missingness according to the study specification.

Usage

apply_missing(data, study, seed = NULL, theta = NULL)

Arguments

Value

Numeric matrix of same dimensions as data, with NA values introduced according to the missingness mechanism.

Build Criterion Plot (Internal)

Description

Shared plotting engine used by both plot.irt_results() and plot.summary_irt_results().

Usage

build_criterion_plot(
  summary_obj,
  criterion = "rmse",
  param = NULL,
  item = NULL,
  threshold = NULL
)

Arguments

Value

A ggplot2::ggplot object, returned invisibly.

Build NA Item Results for Non-Converged Iterations (Internal)

Description

Build NA Item Results for Non-Converged Iterations (Internal)

Usage

build_na_item_results(true_params, iteration, sample_size)

Arguments

Value

Data frame matching item_results schema with NA estimates.

Build True Parameter Data Frame (Internal)

Description

Creates a data frame of true parameter values from the design, used for populating the true_value column and for building NA rows on non-convergence.

Usage

build_true_params(design)

Arguments

Value

Data frame with columns: item, param, true_value.

Build True Parameter Data Frame for Estimation Model (Internal)

Description

Creates a data frame of true parameter values adjusted for the estimation model, accounting for misspecification. When generation and estimation models differ, some parameters may need to be filled with defaults (e.g., discrimination = 1 for Rasch when estimating 1PL from 2PL data) or dropped entirely (e.g., discrimination when estimating 1PL from 2PL).

Usage

build_true_params_for_estimation(design, estimation_model)

Arguments

Value

Data frame with columns: item, param, true_value, matching the schema of the estimation model.

Compute Performance Criteria for a Single Parameter (Internal)

Description

Given a vector of per-iteration estimates and the true parameter value, computes bias, empirical SE, MSE, RMSE, coverage, and Monte Carlo SEs following Morris et al. (2019).

Usage

compute_criterion(estimates, true_value, ci_lower = NULL, ci_upper = NULL)

Arguments

Value

A named list with elements:

bias

Mean estimate minus true value.

empirical_se

Sample standard deviation of estimates (n-1 denominator).

mse

Mean squared error: mean((estimate - true_value)^2).

rmse

Root mean squared error: sqrt(mse).

coverage

Proportion of CIs containing the true value, or NULL if CIs not provided. NAs in CIs are excluded from the denominator.

mcse_bias

Monte Carlo SE of bias: empirical_se / sqrt(K).

mcse_mse

Monte Carlo SE of MSE: sd((est - true)^2) / sqrt(K).

References

Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102. doi:10.1002/sim.8086

Convert Mirt's d Parameter to IRT Difficulty b

Description

Applies the delta method to convert d (intercept) to b (difficulty) under the 2PL/GRM parameterization: b = -d / a. Propagates standard errors and confidence intervals correctly.

Usage

convert_d_to_b(a_info, d_info)

Arguments

Details

The delta method variance for b = -d/a is:

\text{Var}(b) = \frac{\text{Var}(d)}{a^2} + \frac{d^2 \text{Var}(a)}{a^4}

CI bounds are transformed directly: b_ci = -d_ci / a.

Value

A list with elements est, se, ci_lower, ci_upper for the converted difficulty parameter b, using delta method variance propagation.

Extract a Single Parameter Estimate with SE and CI

Description

Internal helper to extract estimate, standard error, and confidence interval bounds from a single column of a mirt coefficient matrix.

Usage

extract_one_param(mat, col_name)

Arguments

Value

A list with elements:

est

The parameter estimate (from "par" row).

se

The standard error derived from CI width, or NA if CI rows absent.

ci_lower

Lower CI bound (from "CI_2.5" row), or NA if absent.

ci_upper

Upper CI bound (from "CI_97.5" row), or NA if absent.

Extract Item Parameter Estimates from a Fitted mirt Model (Internal)

Description

Pulls point estimates, SEs, and CIs from a fitted mirt object and returns them in long format matching the item_results schema.

Usage

extract_params(
  mod,
  design,
  estimation_model,
  iteration,
  sample_size,
  true_params,
  true_params_lookup,
  se = TRUE
)

Arguments

Details

Uses the list-based coef(mod) output (one matrix per item, keyed by item name). Default coef() returns rows ⁠par, CI_2.5, CI_97.5⁠; SE is derived from CI width: SE = (upper - lower) / (2 * z_0.975).

Value

Data frame with item_results columns.

Extract Theta Recovery Summary from a Fitted mirt Model (Internal)

Description

Computes EAP theta estimates and summarizes recovery via correlation and RMSE against true theta.

Usage

extract_theta_summary(mod, theta_true, iteration, sample_size)

Arguments

Value

Single-row data frame with theta_results columns.

Fit an IRT Model (Internal)

Description

Wraps mirt::mirt() with error and convergence handling.

Usage

fit_model(data, model, se = TRUE)

Arguments

Value

A list with elements model (fitted mirt object or NULL) and converged (logical).

Generate IRT Response Data (Internal)

Description

Wraps mirt::simdata() to produce a response matrix from an irt_design specification. Handles the b-to-d parameterization translation and theta generation.

Usage

generate_data(design, n, seed = NULL, theta = NULL)

Arguments

Value

A numeric matrix with n rows and design$n_items columns.

Generate Discrimination Parameters

Description

Internal helper to generate discrimination (a) parameters under a specified distribution. Currently supports log-normal.

Usage

generate_discrimination(n_items, a_dist, a_mean, a_sd)

Arguments

Value

A numeric vector of length n_items containing discrimination values.

Generate Theta Values from a Distribution Specification (Internal)

Description

Generate Theta Values from a Distribution Specification (Internal)

Usage

generate_theta(theta_dist, n)

Arguments

Value

Numeric vector of length n.

Get a Criterion Configuration from the Registry (Internal)

Description

Retrieves the metadata for a specified criterion and validates that it exists.

Usage

get_criterion_config(criterion)

Arguments

Value

A named list with criterion-specific metadata: direction (char: "lower_is_better" or "higher_is_better"), use_abs (logical), and display_label (char).

Get a Missing Mechanism Configuration from the Registry (Internal)

Description

Retrieves the metadata for a specified missing data mechanism and validates that it exists.

Usage

get_missing_config(mechanism)

Arguments

Value

A named list with mechanism-specific metadata: requires_rate (logical), requires_test_design (logical), test_design_key (character or NA), print_label (character), print_style (character: "plain", "rate", or "design"), and design_unit (character or NA).

Get a Model Configuration from the Registry (Internal)

Description

Retrieves the configuration for a specified model and validates that it exists.

Usage

get_model_config(model)

Arguments

Value

A named list with model-specific functions and metadata.

Create an IRT Design Specification

Description

Define the data-generating model for an IRT simulation study. This captures decisions 1–3 from the Schroeders & Gnambs (2025) framework: dimensionality, item parameters, and item type.

Usage

irt_design(model, n_items, item_params, theta_dist = "normal", n_factors = 1L)

Arguments

Value

An S3 object of class irt_design (a named list) with elements model, n_items, item_params, theta_dist, and n_factors.

See Also

irt_study() to add study conditions, irt_params_2pl() and irt_params_grm() to generate item parameters.

Examples

# 1PL (Rasch) design with 20 items
design_1pl <- irt_design(
  model = "1PL",
  n_items = 20,
  item_params = list(b = seq(-2, 2, length.out = 20))
)

# 2PL design
design_2pl <- irt_design(
  model = "2PL",
  n_items = 30,
  item_params = list(
    a = rlnorm(30, 0, 0.25),
    b = seq(-2, 2, length.out = 30)
  )
)

Compute Required Monte Carlo Replications

Description

Uses the Burton (2003) formula to determine the minimum number of simulation replications needed to achieve a desired level of Monte Carlo precision.

Usage

irt_iterations(sigma, delta, alpha = 0.05)

Arguments

Details

The formula is:

R = \lceil (z_{\alpha/2} \cdot \sigma / \delta)^2 \rceil

where \sigma is the empirical standard error of the estimand, \delta is the acceptable Monte Carlo error, and z_{\alpha/2} is the critical value for the desired confidence level.

Value

An integer: the minimum number of replications.

References

Burton, A., Altman, D. G., Royston, P., & Holder, R. L. (2006). The design of simulation studies in medical statistics. Statistics in Medicine, 25(24), 4279–4292. doi:10.1002/sim.2673

See Also

irt_simulate() for running the simulation with the computed number of replications.

Examples

# How many replications for MC SE of bias < 0.1
# when empirical SE of the estimand is 0.5?
irt_iterations(sigma = 0.5, delta = 0.1)

# Tighter tolerance with 99% MC confidence
irt_iterations(sigma = 0.5, delta = 0.05, alpha = 0.01)

Generate 2PL Item Parameters

Description

Creates a list of discrimination (a) and difficulty (b) parameters suitable for passing to irt_design().

Usage

irt_params_2pl(
  n_items,
  a_dist = "lnorm",
  a_mean = 0,
  a_sd = 0.25,
  b_dist = "normal",
  b_mean = 0,
  b_sd = 1,
  b_range = c(-2, 2),
  seed = NULL
)

Arguments

Value

A named list with elements a (numeric vector) and b (numeric vector), each of length n_items.

See Also

irt_params_grm() for GRM parameters, irt_design() to use the generated parameters.

Examples

# Default 2PL parameters for 30 items
params <- irt_params_2pl(n_items = 30, seed = 42)

# Evenly-spaced difficulty
params <- irt_params_2pl(n_items = 20, b_dist = "even", b_range = c(-3, 3))

Generate GRM Item Parameters

Description

Creates a list of discrimination (a) and threshold (b) parameters suitable for passing to irt_design() with model = "GRM".

Usage

irt_params_grm(
  n_items,
  n_categories,
  a_dist = "lnorm",
  a_mean = 0,
  a_sd = 0.25,
  b_mean = 0,
  b_sd = 1,
  seed = NULL
)

Arguments

Value

A named list with elements:

a

Numeric vector of length n_items.

b

Numeric matrix with n_items rows and n_categories - 1 columns. Thresholds are ordered within each row.

See Also

irt_params_2pl() for 2PL parameters, irt_design() to use the generated parameters.

Examples

# GRM parameters: 15 items, 5 response categories
params <- irt_params_grm(n_items = 15, n_categories = 5, seed = 42)

Run an IRT Monte Carlo Simulation

Description

Execute a Monte Carlo simulation study based on an irt_study specification. For each iteration and sample size, data are generated, missing values applied, the IRT model is fitted, and parameter estimates are extracted and stored.

Usage

irt_simulate(
  study,
  iterations,
  seed,
  progress = TRUE,
  parallel = FALSE,
  se = TRUE,
  compute_theta = TRUE
)

Arguments

Details

The returned irt_results object stores raw per-iteration estimates. Use summary.irt_results() to compute performance criteria (bias, MSE, RMSE, coverage, etc.) and plot.irt_results() to visualize results.

Parallelization

When parallel = TRUE, the Monte Carlo loop over iterations is parallelized via future.apply::future_lapply(). Each parallel task processes one iteration across all sample sizes sequentially.

Important: This function does NOT configure a future plan. Users must set their own plan before calling with parallel = TRUE:

library(future)
plan(multisession, workers = 4)  # or your preferred backend
results <- irt_simulate(study, iterations = 100, seed = 42, parallel = TRUE)

Without an explicit plan, future defaults to sequential execution (no parallelism).

Reproducibility contract

Reproducibility is guaranteed within a given dispatch mode, not across modes:

• Serial mode (parallel = FALSE) uses deterministic per-cell seeds under the session's default RNG kind (Mersenne-Twister). Re-running with the same base seed reproduces identical results bit-for-bit.

• Parallel mode (parallel = TRUE) delegates RNG management to future.apply::future_lapply(..., future.seed = TRUE), which assigns each iteration a formally independent L'Ecuyer-CMRG substream. Re-running with the same base seed reproduces identical results bit-for-bit across parallel runs, including across different worker counts.

• Across modes, numerical results will differ because the two paths use different RNG algorithms and different seeding strategies. Both are statistically valid; the parallel path has the stronger formal guarantee of independent substreams, which is the standard for Monte Carlo work.

Progress messages are suppressed in parallel mode (workers cannot stream to stdout safely). Set progress = FALSE in serial mode to suppress messages (they appear every 10% of iterations).

Value

An S3 object of class irt_results containing:

item_results

Data frame with per-iteration item parameter estimates (columns: iteration, sample_size, item, param, true_value, estimate, se, ci_lower, ci_upper, converged).

theta_results

Data frame with per-iteration theta recovery summaries (columns: iteration, sample_size, theta_cor, theta_rmse, converged).

study

The original irt_study object.

iterations

Number of replications run.

seed

Base seed used.

elapsed

Elapsed wall-clock time in seconds.

se

Logical flag indicating whether SEs and CIs were computed.

compute_theta

Logical flag indicating whether theta recovery metrics were computed.

See Also

irt_study() for specifying study conditions, summary.irt_results() and plot.irt_results() for analyzing output, irt_iterations() for determining the number of replications.

Examples

# Minimal example (iterations and sample sizes reduced for speed;
# use iterations >= 100 and 3+ sample sizes in practice)
design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)
summary(results)
plot(results)

Define Study Conditions for an IRT Simulation

Description

Add study-level conditions to an IRT design specification. This captures decisions 4–5 from the Schroeders & Gnambs (2025) framework: sample sizes and missing data mechanism.

Usage

irt_study(
  design,
  sample_sizes,
  missing = "none",
  missing_rate = NULL,
  test_design = NULL,
  estimation_model = NULL
)

Arguments

Value

An S3 object of class irt_study (a named list) with elements design, missing, missing_rate, sample_sizes, test_design, and estimation_model.

See Also

irt_design() for the design specification, irt_simulate() to run the simulation.

Examples

# Simple study with no missing data
d <- irt_design(
  model = "1PL", n_items = 20,
  item_params = list(b = seq(-2, 2, length.out = 20))
)
study <- irt_study(d, sample_sizes = c(100, 250, 500))

# Study with MCAR missingness
study_mcar <- irt_study(d, sample_sizes = c(200, 400),
                        missing = "mcar", missing_rate = 0.2)

# Model misspecification: generate 2PL, fit 1PL
d_2pl <- irt_design(
  model = "2PL", n_items = 15,
  item_params = list(a = rlnorm(15, 0, 0.25), b = rnorm(15))
)
study_misspec <- irt_study(d_2pl, sample_sizes = c(100, 300),
                           estimation_model = "1PL")

Plot IRT Simulation Results

Description

Visualize performance criteria across sample sizes from an irt_simulate() result. Calls summary.irt_results() internally, then plots the requested criterion by sample size.

Usage

## S3 method for class 'irt_results'
plot(x, criterion = "rmse", param = NULL, item = NULL, threshold = NULL, ...)

Arguments

Value

A ggplot2::ggplot object, returned invisibly.

See Also

summary.irt_results() for the underlying criteria, recommended_n() for sample-size recommendations.

Examples

design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)
plot(results)
plot(results, criterion = "bias", threshold = 0.05, param = "b")

Plot Summary of IRT Simulation Results

Description

Visualize performance criteria from a summary.irt_results() object. This is a convenience method for users who already have a summary; plot.irt_results() is the primary interface.

Usage

## S3 method for class 'summary_irt_results'
plot(x, criterion = "rmse", param = NULL, item = NULL, threshold = NULL, ...)

Arguments

Value

A ggplot2::ggplot object, returned invisibly.

See Also

plot.irt_results(), summary.irt_results()

Examples

design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)
s <- summary(results)
plot(s, criterion = "rmse", threshold = 0.15)

Print an IRT Design

Description

Display a compact summary of an irt_design object, including model type, number of items, theta distribution, and parameter ranges.

Usage

## S3 method for class 'irt_design'
print(x, ...)

Arguments

Value

x, invisibly.

See Also

irt_design()

Examples

d <- irt_design("1PL", 10, list(b = seq(-2, 2, length.out = 10)))
print(d)

Print an IRT Simulation Result

Description

Display a compact summary of an irt_simulate() result, including model, items, sample sizes, iterations, convergence rate, and elapsed time.

Usage

## S3 method for class 'irt_results'
print(x, ...)

Arguments

Value

x, invisibly.

See Also

irt_simulate()

Examples

design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)
print(results)

Print an IRT Study

Description

Display a compact summary of an irt_study object, including model, items, sample sizes, and missing data mechanism.

Usage

## S3 method for class 'irt_study'
print(x, ...)

Arguments

Value

x, invisibly.

See Also

irt_study()

Examples

d <- irt_design("1PL", 10, list(b = seq(-2, 2, length.out = 10)))
s <- irt_study(d, sample_sizes = c(100, 500))
print(s)

Print Summary of IRT Simulation Results

Description

Display item parameter criteria and theta recovery statistics from a summary.irt_results() object.

Usage

## S3 method for class 'summary_irt_results'
print(x, ...)

Arguments

Value

x, invisibly.

See Also

summary.irt_results()

Examples

design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)
s <- summary(results)
print(s)

Find the Minimum Sample Size Meeting a Criterion Threshold

Description

Given a summary.irt_results() object, find the smallest sample size at which a performance criterion meets the specified threshold for each item and parameter combination.

Usage

recommended_n(object, ...)

## S3 method for class 'summary_irt_results'
recommended_n(object, criterion, threshold, param = NULL, item = NULL, ...)

Arguments

Details

For criteria where smaller is better (bias, empirical_se, mse, rmse, mcse_bias, mcse_mse), the threshold is met when the criterion value is at or below the threshold. For bias, the absolute value is used. For coverage (where higher is better), the threshold is met when coverage is at or above the threshold.

Value

A data frame with columns:

item

Item number.

param

Parameter name.

recommended_n

Minimum sample size meeting the threshold, or NA if no tested sample size meets it.

criterion

The criterion used (echoed back for reference).

threshold

The threshold used (echoed back for reference).

See Also

summary.irt_results() for computing criteria, plot.irt_results() for visualization.

Examples

design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)
s <- summary(results)

# Minimum N for RMSE <= 0.20 on all items
recommended_n(s, criterion = "rmse", threshold = 0.20)

# Minimum N for 95% coverage on difficulty parameters only
recommended_n(s, criterion = "coverage", threshold = 0.95, param = "b")

Summarize IRT Simulation Results

Description

Compute performance criteria for each sample size, item, and parameter combination from an irt_simulate() result. Criteria follow Morris et al. (2019) definitions. Optionally, users can provide a custom callback function to compute additional item-level performance criteria (e.g., conditional reliability, external criterion SE).

Usage

## S3 method for class 'irt_results'
summary(object, criterion = NULL, param = NULL, criterion_fn = NULL, ...)

Arguments

Value

An S3 object of class summary_irt_results containing:

item_summary

Data frame with one row per sample_size × item × param combination, containing the requested criteria plus n_converged and any custom columns from criterion_fn.

theta_summary

Data frame with one row per sample_size, containing mean_cor, sd_cor, mean_rmse, sd_rmse, and n_converged.

iterations

Number of replications.

seed

Base seed used.

model

IRT model type.

References

Morris, T. P., White, I. R., & Crowther, M. J. (2019). Using simulation studies to evaluate statistical methods. Statistics in Medicine, 38(11), 2074–2102. doi:10.1002/sim.8086

See Also

irt_simulate() for running simulations, plot.irt_results() for visualization, recommended_n() for sample-size recommendations.

Examples

# Minimal example (iterations reduced for speed; use 100+ in practice)
design <- irt_design(
  model = "1PL", n_items = 5,
  item_params = list(b = seq(-2, 2, length.out = 5))
)
study <- irt_study(design, sample_sizes = c(200, 500))
results <- irt_simulate(study, iterations = 10, seed = 42)

s <- summary(results)
s$item_summary
s$theta_summary

# Only bias and RMSE for difficulty parameters
summary(results, criterion = c("bias", "rmse"), param = "b")

# Compute custom criterion: relative bias
custom_fn <- function(estimates, true_value, ci_lower, ci_upper, converged, ...) {
  valid_est <- estimates[!is.na(estimates)]
  rel_bias <- (mean(valid_est) - true_value) / true_value
  c(relative_bias = rel_bias)
}
summary(results, criterion_fn = custom_fn)

# Multiple custom criteria
multi_fn <- function(estimates, true_value, ci_lower, ci_upper, converged, ...) {
  valid_est <- estimates[!is.na(estimates)]
  c(mean_est = mean(valid_est), sd_est = sd(valid_est))
}
summary(results, criterion_fn = multi_fn)

Get All Valid Criteria (Internal)

Description

Returns the names of all criteria in the registry.

Usage

valid_criteria()

Value

Character vector of criterion names, in registry order.

Get All Valid Missing Mechanisms (Internal)

Description

Returns the names of all mechanisms in the registry.

Usage

valid_missing_mechanisms()

Value

Character vector of mechanism names, in registry order.

Validate a Binary Design Matrix

Description

Internal helper to validate that a matrix meets standard requirements for booklet and linking design matrices: is a matrix, has correct column count, and contains only binary values.

Usage

validate_design_matrix(mat, n_items, matrix_name)

Arguments

Value

Invisibly returns NULL if all checks pass. Throws an error (via stop()) if any check fails.