scindex: Strategic Convergence Index for Inter-Rater Reliability
Inter-rater reliability analysis for binary classification
tasks involving two or more raters within a signal
detection-theoretic framework. User-supplied rating data are
standardised into a common long-format structure. The package
automatically computes Cohen's kappa for two raters and
Fleiss' kappa for multiple raters. When ground-truth labels
are available, rater-specific hit rates, false-alarm rates,
sensitivity, specificity, and decision thresholds are
estimated from observed classification responses using
standard signal detection-theoretic transformations
(DeCarlo, 1998)
<doi:10.1037/1082-989X.3.2.186>. The package implements the
Strategic Convergence Index (SCI; Gianeselli, 2026)
<doi:10.1177/00131644261417643>, defined as
SCI = 1 - [Var(t_i) / Var_max], where Var(t_i) denotes the
variance of rater-specific decision thresholds and Var_max
denotes the reference variance under maximal threshold
dispersion. SCI quantifies convergence in rater decision
criteria beyond observed agreement alone and complements
classical agreement coefficients by distinguishing agreement
in observed categorical outcomes from convergence in latent
decision thresholds under an explicit signal
detection-theoretic model of categorical judgment. The
package provides structured summaries and threshold-based
diagnostics for applications in which similar agreement
coefficients may reflect substantively different underlying
decision criteria across raters.
Documentation:
Downloads:
Linking:
Please use the canonical form
https://CRAN.R-project.org/package=scindex
to link to this page.
[aut, cre]