SCAtools 0.5.0 was not released on CRAN, so this is the release that brings the 0.5.0 changes below to CRAN users of 0.4.3. Read Upgrading from 0.4.3 first; it collects in one place every change that can break a script written for that release.
frontier again0.5.0 renamed the column that names the frontier technique from
frontier to frontier_technique. It is
frontier again, in sca_table(),
sca_thresholds(), sca_compare(),
sca_locationshift(), sca_scope_sensitivity()
and sca_resample(), which is the name it had in 0.4.3 – so
for this column a script written against the CRAN release needs no
change at all.
The problem 0.5.0 set out to solve was a column that held
"ce_fdh" under a name that the same table also used for the
boundary itself. frontier does not have that problem: the
value identifies which fitted frontier the row reports, and
frontier_accuracy beside it is that frontier’s accuracy.
boundary_type still says whether the boundary is a ceiling
or a floor. The plot legend reads “Frontier” again, and
frontier is no longer listed by
sca_legacy_names().
The arguments ceilings and ceiling are
unchanged.
empty_zone_area is kept beside
empty_space_area0.5.0 renamed empty_zone_area to
empty_space_area. NSCA 0.4.5, the release on CRAN, reads
the old name from sca_table(), and fails without it. The
table therefore carries empty_zone_area as an exact copy of
empty_space_area until a release of NSCA that reads the new
name has replaced 0.4.5; it will then be removed. New code should read
empty_space_area.
NSCA 0.4.5’s own test suite passes against this release.
The method is Sufficient Condition Analysis. 0.5.0 changed
the title, and this release finishes the job in the places it missed:
citation("SCAtools"), the README and the help page of
sca().
Description in DESCRIPTION, the text on the CRAN page,
now says directly what the method is, what it is for and how it helps.
Where the method comes from, with its authors and references, is on the
package help page, ?SCAtools, which is no longer marked
internal and so appears in the reference manual. It now covers the
method, what it is for, how to use it and how to read the result.sca_four_configurations, and install from CRAN. Version
history now lives only in this file, and help pages no longer recount
it.sca(), whose point is the default
1000 permutations, is wrapped in \donttest{}.
sca_power() had lost the example it had in 0.4.3 when the
help pages were regenerated, and has one again.citation("SCAtools") and the help pages cite the two
papers that set out the method: Chan (in press), From Necessity to
Sufficiency and Necessity-and-Sufficiency: Extending Condition Analysis
in Degree, and Chan (in press), Mean Effects Are Not
Sufficiency Effects: How Sufficient Condition Analysis Complements
Regression Analysis, both University of Leeds.
sca_four_configurations is the data of the second.sca_legacy_names() said that sca_extract()
accepts retired names with a warning and that retired arguments still
work. Neither has been true since 0.5.0: sca_extract()
stops and names the replacement, and boundary,
bottleneck.x and bottleneck.y were removed.
The page now says so, and lists the 0.5.0 renames as well as those of
0.4.0.tests/testthat/Rplots.pdf is no longer
shipped.Everything that can make a script written for 0.4.3 fail or give a different number, with the section of 0.5.0 below that explains it.
empty_zone_area is now empty_space_area.
The old name is still present in sca_table() for now, and
legacy = TRUE appends it to both tables.sca() runs a permutation test by default
(test.rep = 1000), so it takes noticeably longer than
sca_analysis().bottleneck.x and bottleneck.y are removed
from sca_analysis(), and boundary from
sca_thresholds() and sca_rescale(). Use
threshold.x, threshold.y and
inequality.sca_extract() stops at once on a retired column name.
In 0.4.3 such a call warned and then failed anyway, so only the message
changes.sca_reference() returns one row per condition and
reference line, with line, tau and
drawn columns.sca_thresholds(), sca_test_plot() and
sca_output() default to the first frontier actually
estimated instead of "ce_fdh".test.p_threshold outside (0, 1), a negative
test.rep, a reversed scope, a
scope with no width, and the same variable as condition and
outcome.Imports gains quantreg, which NCA already
imports.Not released on CRAN; its changes reached CRAN in 0.5.1.
A correctness and consistency release. Four of the changes below alter numbers that earlier versions reported, and two remove arguments, so this is not a patch release. Read Breaking before upgrading a running analysis.
Dul (2016)’s bands for an NCA effect size, and the label a value falls in:
sca_magnitude() # the table
sca_magnitude(c(0.04, 0.18, 0.35)) # "small" "medium" "large"sca_compare() and nsca_compare() gain a
magnitude column.
Why this reverses an earlier position. The package
was already applying Dul’s lower edge as the default
d_cut and Cohen’s as the default beta_cut,
with both conventions written out in a code comment where no user could
see them – while the joint table printed “magnitude benchmarks are not
established”. A convention applied and not disclosed is worse than
either applying it openly or not applying it at all.
They are a convention to cite, exactly as a reader
of a regression cites Cohen. Reporting “d = 0.35, large by the
conventional bands (Dul, 2016)” is a citation and a normal one. The
table says so in a source column, not only in the help
page, because a table gets pasted into a script and a help page does
not.
beta_cut = 0.30 is Cohen (1988), and it was in a code
comment for exactly the same reason Dul’s was. Both are out now:
sca_magnitude(scale = "beta") # Cohen's bands for a standardised coefficientsca_compare() and nsca_compare() gain
beta_magnitude beside magnitude.
The two conventions share their edges and shift their labels by one band. Both use .10, .30 and .50. Cohen’s medium runs .30 to .50; Dul’s large runs .30 to .50.
value as a coefficient (Cohen) as an effect size (Dul)
0.05 below small small
0.15 small medium
0.35 medium large
0.60 large very largeThis is why the two defaults differ – beta_cut = 0.30
and d_cut = 0.10 are each the lower edge of medium
in their own tradition, and those are different numbers even though the
edges are not. A reader comparing a beta and a d in the same row of
sca_compare() needs that stated, and a comment in the
source was not stating it. ### The joint indices still get none, and
that is a different judgement
weakest_effect, balanced_joint_effect and
joint_empty_space_coverage are this package’s own
constructions. Citing somebody else’s published convention and inventing
one for your own index are not the same act, and the second is a claim
with nothing behind it. print() now distinguishes the two
cases instead of denying benchmarks for both.
A band says where a number sits on a conventional scale. It does not
say how much of the number the geometry would have produced anyway –
effect_size_null answers that. A d_nec of 0.10
labelled medium at a floor of 0.055 is a different finding from
the same 0.10 at a floor of 0.007, and both numbers are now in the same
row of nsca_compare().
A grey line ran through the middle of the cloud and nothing on the figure said what it was, except one sentence in the caption. Reported by someone reading a plot, which is where this kind of defect gets found.
Keeping it out of the Component legend was right – colouring an average effect as a component would put it in the same key as an empty-space frontier, and they are not the same kind of object. But “not in that key” had been built as “no key at all”.
It is now labelled on the line itself: OLS, or
QR tau=0.5 for a quantile reference. ggplot2 allows one
scale per aesthetic and both are taken, so a third legend would mean
bending colour or linetype into exactly the conflation the original
decision was avoiding. A label where the reader is already looking
avoids both problems.
The labels are staggered along x, because an OLS line and a median line nearly coincide on symmetric data and two labels in one place is one label with something hidden under it.
Both packages.
Type I error, measured rather than assumed. Independent X and Y, n =
50, test.rep = 199, nominal alpha 0.05, 2500 simulations
(SE 0.00436):
necessity reject .0520 95% CI [.0433, .0607] KS p = .27
sufficiency reject .0528 95% CI [.0440, .0616] KS p = .32
IUT (joint) reject .0080 95% CI [.0045, .0115]Both component p-value distributions are uniform, not merely correct at the cut. A first run at 500 simulations put sufficiency at .0740 (z = +2.46), which looked like it might be a real anti-conservative bias; at 2500 it is .0528. It was noise.
The intersection-union test rejects well below alpha and fails a
uniformity test. That is the design: it rejects only when both
components reject, so its level is at most alpha by construction.
joint_support is therefore conservative, and a null result
from it should be read with that in mind.
The script and both results ship at the repository root as
calibration-0.5.0.R and
calibration-0.5.0-result.txt.
cr_fdh and the other line techniques were drawn across
the full x range with no clipping, so near a corner the line crossed the
grey dashed scope boundary and kept going.
The areas were never wrong – the band integral clips the frontier to
the scope, which is why the identity closed and fit and
frontier_accuracy were 1. But a figure in which the
frontier leaves the box whose area is being reported reads as an error
to anyone checking the number against the picture, and “it is only
cosmetic” is not something a reader can verify.
The drawn line is now clipped the same way the integral clips it, so it is the boundary of the region actually measured.
Found by rendering the figures and looking at them, which the test
suite cannot do – it only ever asked whether a plot was a
ggplot.
Title, subtitle and caption were all wrapped at 95 characters. But a title is bold and about half again the size of a caption, so on a long variable name the first line overflowed the panel and was clipped mid-word:
NSCA: High Hours of structured professional development training per year
is necessa|nsca_shared_plot() was worse – its title was not wrapped
at all and simply ran off. Titles now wrap at a width chosen for their
own size, found by rendering rather than by arithmetic on point
sizes.
nsca_powerplot() drew the joint curve last, on top of
the two components it is built from. At high power all three coincide,
which is the ordinary case – so the figure showed one line above a
legend naming three.
The joint curve is now drawn first, underneath, with the two component curves dashed and dotted over it, and it leads the legend for the same reason it is drawn first: it is the curve a study is powered on. The x axis also breaks at the sample sizes actually simulated; it used to break at 50/75/100 for a study run at n = 30/60/120.
It said Frontier, which is now the name of the boundary
itself. The tables call the estimator frontier_technique;
the figures now agree.
sca_compare() and nsca_compare() gain
effect_size_null and effect_size_excess.
A pure-noise condition at n = 70 came back as configuration II,
necessity-dominant, on d_nec = 0.104 against the
conventional cut of 0.10 – while its independence floor was 0.055, so
half of that effect size was the envelope fitting sample extremes. The
label was defensible and the reading was still misleading.
condition component effect_size effect_size_null effect_size_excess label
Training necessity 0.36825 0.07451 0.29374 mean and necessity
Luck necessity 0.10418 0.05464 0.04955 necessity-dominant
Luck sufficiency 0.03725 0.08698 -0.04974 neitherLuck’s excess is below the cut on one side and
negative on the other. The cut is unchanged – moving it to the
excess would be inventing a criterion – but the number that qualifies it
is now in the same row.
It was hand-maintained, so a new @export had to be added
to it by hand and sca_resample() nearly shipped unexported.
The tags and the file were checked to be exactly in sync – 31 exports
and 5 S3 methods, nothing dropped, nothing missing – before roxygen was
given ownership.
before: The length of scope needs to be equal to the length of x
or
after: 'scope' has 6 values. With 3 conditions it must have 4 --
c(xmin, xmax, ymin, ymax), one box for all of them -- or 12,
four per condition in the order the conditions were given.A list of the wrong length, or with an entry that is not four numbers, is now refused the same way.
Found by the adversarial pass. Eight conditions, one of them real,
seven noise, test.rep = 50. Unadjusted, the real one is
supported. Under BH, nothing is – and not because the evidence was
weak.
The arithmetic is hard. A permutation p-value cannot fall below
1 / (test.rep + 1), and the most significant of
k conditions is multiplied by k under both
Bonferroni and Benjamini-Hochberg. So no condition can reach
alpha unless
k / (test.rep + 1) <= alpha, that is test.rep >= k / alpha - 1At alpha = 0.05 that is 2 conditions on 50 replications,
5 on 100, 10 on 200, 50 on 1000. A researcher who adjusts eight
conditions on a low replication count and sees nothing has been told a
fact about test.rep, not about their data – and nothing in
the output said so.
Warning: With 8 conditions and test.rep = 50, the smallest p-value the
permutation test can produce is 0.0196, and 'BH' multiplies the most
significant of 8 by 8. No condition can reach alpha = 0.05 however strong the
evidence, so an absence of support here is arithmetic rather than a finding.
Use test.rep >= 159.A warning rather than a refusal: the adjusted p-values are still correct, and a caller may want them. What they must not do is read the absence of a rejection as evidence. Silent when nothing is adjusted, and when no test was run.
An adversarial pass over 0.5.0 fed both packages degenerate data, hostile scales and arguments of the wrong shape. Sixteen bad inputs; seven were accepted, and three of those produced a fit that misreported itself — the failure a user cannot catch by reading the output.
The worst of them.
sca_analysis(data, "X", "Y", ceilings = "banana") # used to workNothing checked the name. "banana" was passed to the
engine, which fell back to its own defaults and fitted ce_fdh
and cr_fdh — while the metadata still recorded
"banana", so print() showed
frontiers banana above a table of two techniques nobody
had asked for. A misspelled "ce_fhd" did the same thing,
silently.
Now refused, with the available names in the message. The list is pinned against the engine’s own by a test, so an NCA release that adds or drops a technique breaks the suite rather than being quietly accepted.
test.p_threshold = 2 was accepted and stored. Every row
came back significant = TRUE and the Settings
block printed alpha 2 as though it were a considered
choice.
test.rep = -5 was accepted and stored, and printed in
the Settings block beside a p_value of
NA.
scope = c(1, 0, 1, 0) was accepted. The engine repairs
it silently — it widens whatever it is given until the data fits — so
the analysis was correct while sca_scope() reported
declared_xmin 1, declared_xmax 0, which is not a scope
anybody could put in a methods section. c(0.5, 0.5, 0, 1)
was likewise widened back to the observed range.
scope = c(0, NA, 0, 1) is not affected:
NA means “no theoretical bound on this side”, which is the
documented way to declare half a box.
sca_analysis(data, "X", "X") returned
d = 0.53. A variable against itself is a perfect diagonal,
whose empty corner is half the scope whatever the data contains — a
plausible-looking number that is an artefact of the geometry. In NSCA it
is worse: both frontiers land on the same diagonal.
frame[x] raised R’s own
undefined columns selected, which mentions neither the
argument nor the name that was not found. A caller with eight conditions
had to bisect to find the one they had misspelled.
before: undefined columns selected
after: 'x' names 'Trainng', which is not a column of the data.
Available: Training, Experience, Support, Performance.Both packages now give the same message for the same mistake, so a caller who learns one does not have to learn a second set.
The Settings block added in this release discloses the
multiplicity; this is the means to act on it.
sca_table(fit, p.adjust = "BH")Adds p_value_adjusted and p_adjust_method
beside p_value, and re-derives significant,
supported and support_status from the adjusted
values. An adjusted number printed beside a verdict that ignored it
would be worse than not adjusting at all. relevant is
untouched: an effect size does not shrink because more tests were
run.
"none" is the default and leaves the table
byte-identical to before. The package does not pick a method, because
which correction fits depends on what the family is, and that is a
property of the design rather than of the data.
The family is the conditions, within one frontier
technique. Two techniques are two estimates of one claim, not
two hypotheses, so adjusting across them would inflate k
with re-measurements and cost power for nothing. Choosing a technique
after seeing both p-values is a real problem, but it is a garden of
forking paths rather than a family-wise rate, and the
Settings block names it separately.
sca_table() gains effect_size_null and
effect_size_excess.
The FDH envelope is built from the sample’s own extremes, so it
reports a positive empty area even when the condition and the outcome
are independent: with n points there is always
some corner nothing happens to occupy. The overstatement
shrinks with n, which is why an effect size from 40 cases
and one from 400 are not on the same scale — and nothing in the output
said so.
This is not modelled, it is read. The permutation test already builds the distribution of the effect size under random pairing and then discards all of it except the tail probability. Under independence the true empty area is zero, so the mean of that distribution is the overstatement, at this sample size, this scope and this technique, measured on the data in hand rather than taken from an asymptotic constant.
n= 40 d=0.476 null=0.104 excess=0.372
n=120 d=0.397 null=0.063 excess=0.335
n=400 d=0.369 null=0.054 excess=0.315effect_size_excess is a comparison, not a
corrected estimator. Subtracting the null mean does not give an
unbiased effect size, because the bias under independence is not the
bias under the alternative. What it gives is the part of the number that
random pairing alone does not account for. A condition that is pure
noise can show a positive effect_size and a
negative effect_size_excess, and that is the
finding.
Both are NA when test.rep = 0: with nothing
to compare against, a zero would be a claim.
Refits one condition on random subsamples of several sizes and reports the distribution of the effect size at each size.
n m fraction reps effect_size subsample_median subsample_p05 subsample_p95
150 38 0.2533 120 0.4185 0.4777 0.4539 0.5015
150 75 0.5000 120 0.4185 0.4430 0.4281 0.4672
150 112 0.7467 120 0.4185 0.4287 0.4200 0.4372
150 150 1.0000 1 0.4185 0.4185 0.4185 0.4185The spread at one size is the sampling variability. The trend down
the m column is the bias, visible rather than modelled:
fewer points leave more of the corner empty by accident.
Why no interval. Two reasons, both specific to frontier estimators rather than general caution.
The ordinary n-out-of-n bootstrap is inconsistent here. Resampling with replacement duplicates observations, and a duplicated extreme cannot push the envelope further out than the original did, so the resampled frontier lies inside the fitted one far more often than chance; the resampling distribution does not converge to the sampling distribution. This is the familiar result for DEA and FDH efficiency scores.
Subsampling without replacement, which is what this function does,
restores consistency but leaves a bias with a known direction. Look at
the table above: at m = 38 the 5th–95th percentile range is
[0.454, 0.502], which does not even contain the full-sample
estimate of 0.4185. A percentile interval read straight off
a subsample distribution sits above the quantity it claims to cover.
Correcting for that needs the estimator’s convergence rate, which
depends on the technique and the dimension, and which this package will
not assert.
So the table reports the distribution and stops there — the same restraint as [sca_scope_sensitivity()], which reports the effect size on every box and refuses to subtract two of them for you.
The scope is pinned to the one the model used, so only the frontier moves. The caller’s random stream is restored, as everywhere else here.
frontier_techniquefrontier held "ce_fdh" — the name of the
estimator, not a frontier. The word was already doing a second
job in the same table, where boundary_type takes the values
"ceiling" and "floor", and a third in the
glossary, where frontier is defined as “the boundary as
estimated from data”. One word, three meanings, two of them in adjacent
columns.
sca_table() frontier -> frontier_technique
sca_thresholds() frontier -> frontier_technique
sca_compare() frontier -> frontier_technique
sca_locationshift() frontier -> frontier_technique
sca_scope_sensitivity() frontier -> frontier_techniquefrontier now means only the estimated boundary and
ceiling only “Y bounded from above”. The name also matches
the prose, which has always said frontier technique, and
matches the argument it comes from: ceilings = "ce_fdh"
produces frontier_technique = "ce_fdh".
The arguments ceilings and
ceiling are unchanged. They are NCA’s names, this package
passes them through, and renaming them is a separate question.
frontier is carried by [sca_legacy_names()], so
sca_table(legacy = TRUE) and
sca_thresholds(legacy = TRUE) append it as a duplicate
column. sca_extract(param = "frontier") refuses the retired
name and says what replaced it, which is what every other retired column
here does: a caller who is handed an answer never learns that the name
moved.
sca_outliers() keeps its ceiling column,
which is NCA’s own and means something else entirely: "X"
marks a removed observation that was a peer. Note that for
cols, qr, c_lp and
custom the engine sets it for every row — those techniques
have no peers to test against — so it carries no information there.
It took data, x, y and
direction, which meant restating four things the model
already knew – and therefore being free to restate them differently. A
diagnostic run on direction = "LH" against a model fitted
on "HH" produced a perfectly plausible table about a corner
the analysis never looked at.
fit <- sca_analysis(data, x = c("A", "B"), y = "Y", direction = "HH")
sca_outliers(fit)Every condition is examined unless x selects some.
Passing y, direction or scope
alongside a model is an error rather than an override, and it is an
error even when the value agrees with the fit: the test is whether the
argument was named, not whether it differs. A rule that accepted the
call that happened to agree and refused the identical call that did not
would be harder to explain than a flat one.
The data-frame form is unchanged apart from the column below.
Both forms gain a leading condition column. The model
form needs it, because it examines several conditions and the rows would
otherwise be indistinguishable; the data form gains it so that one
function does not return two shapes. It is the name or the column
position the caller supplied, which the function has always known and
never reported.
beta is documented as the standardised least-squares
coefficient, and the configuration III and IV readings say “on average”.
The function asked sca_reference() for whatever line the
model was fitted with, so a model fitted with
reference = "qr" put the median slope in that column under
the OLS label – and, since the configuration classification thresholds
beta, could land a condition in a different configuration
than the same data fitted with reference = "ols".
The line is now pinned. Reference lines are computed in this package rather than by the engine, so asking for one the model did not request costs nothing.
A model with k conditions runs k
permutation tests, and nothing in the output said so. The
Settings block now reports the count, states that the
p-values are unadjusted, and gives the arithmetic:
multiplicity 3 conditions = 3 p-values, none adjusted
were the 3 tests independent, P(at least one false positive)
at alpha 0.05 would be 0.14Disclosure, not adjustment. Which correction fits depends on what the
family is – whether the conditions were pre-specified, whether one
significant condition would be reported as the finding or all of them
together, whether the analysis is confirmatory at all. Those are
properties of the design rather than of the data, and a package that
guessed at them would be making the methodological claim on the
researcher’s behalf. stats::p.adjust() on the
p_value column is one line once the family is known.
Several frontier techniques are a different problem and get a different line: two techniques are two estimates of one claim, not two hypotheses, so there is no family-wise rate to report. What there is instead is the temptation to pick after seeing both, which the block names.
Silent when there is nothing to disclose – one condition, one technique, or no permutation test – so that a block that says something always means something.
sca() runs a permutation test by default –
test.rep = 1000 – and had no way to pin the result, so it
produced a reportable-looking p-value that was different on every run.
It now takes test.seed, and print() says so
beneath the results when the p-values are unpinned rather than only in
the settings block above them.
The default stays at 1000. A quick result should still carry statistical evidence; what it should not do is carry evidence that cannot be reproduced without saying so.
Koenker and Machado’s R1 for the "qr" line,
NA for "ols", beside an r_squared
that is NA for "qr". Quantile regression
minimises a different loss, so there is no residual variance to take a
share of. Reporting R1 in the r_squared column would be one
column carrying two meanings; leaving it out entirely reported a
difference in kind as a missing value. A pseudo_r1 and an
r_squared are not on a common scale and must not be
compared.
reference = "qr" draws a quantile regression beside the
frontier. reference.tau sets its quantile and defaults to
0.5: the median line, the central-tendency summary that
does not move when one observation does, which is the reason to draw it
beside OLS rather than instead of it.
It is separate from qr.tau, and the separation is the
feature. A qr frontier sits at the edge of the
cloud and uses qr.tau; a qr reference
sits in the middle of it and uses reference.tau. Both can
now be asked for in one analysis, which the engine cannot express:
NCA::nca_analysis() takes one qr.tau per
call.
Reference lines are therefore computed by this package rather than
requested from the engine. sca_reference() already computed
the OLS line itself with stats::lm(), so this makes the two
consistent and removes the second engine run that a reference-only line
would otherwise have cost.
sca_reference() returns one row per condition
and line, with line, tau and
drawn columns. Code that assumed one row per condition
needs a filter.r_squared and beta are NA for
"qr". Quantile regression minimises a different loss, so
there is no residual variance to take a share of, and the pseudo-R1
sometimes reported for it is not comparable with an OLS R-squared. An
empty column beats one that looks the same and means something
else.sca_plot() draws reference lines unkeyed – OLS dotted,
qr long-dashed – and names them in the caption with the
quantile, because a median line and a 95th-percentile line look
identical on paper. They stay out of the frontier legend: they are not
frontiers, and a shared key said they were.qr remains a valid frontier. It is now in both sets, so
the rule that moves a central-tendency line out of ceilings
had to be narrowed to the lines that can only be references,
which is ols alone.Refits one condition on a series of boxes and reports the effect size
each one produces. The scope is the denominator and, unlike the
frontier, it is chosen rather than estimated, so a reported
d is a claim about the data and a claim about the box at
once. This separates them.
The default boxes are the observed range widened by a fraction of the observed span on each side, because that is a quantity a reader can judge: “a box half as wide again as the data” means something where a pair of numbers does not. When the model was fitted on a declared scope, that box is added as a final row so the caller can see where their own choice sits.
No column reports a change from a baseline. Which row is the baseline
is the caller’s question – the observed range and the scope the model
was fitted on answer different ones – so the table reports the effect
sizes and leaves the subtraction to the reader. The data does not change
between rows: every difference down the effect_size column
is the box.
sca_analysis() now records qr.tau in its
metadata, which this needs in order to rebuild a qr
frontier at the tau that produced the numbers being tested rather than
at the default.
A power table is the number that decides a sample size, and it was
different on every run. NCA::nca_power() has no seed
argument, but it does not need one: it draws a single value off the
caller’s stream – offset <- sample.int(2^24, 1) – and
derives every replication’s seed from it, exactly as the permutation
test does. Fixing the stream before the call therefore fixes the whole
simulation, so sca_power() now takes test.seed
and wraps the engine call in the same .sca_with_seed() that
sca_analysis() has always used. No change to NCA was
needed.
The restore matters more here than elsewhere.
nca_power() calls set.seed() inside its own
loop, so without it the session would be left seeded from an arithmetic
expression of the loop counters.
This was the last place in either package where a reported number could not be reproduced.
A constant outcome has been refused since 0.4.x: it leaves the scope
with no height, so every empty area would be measured against zero. A
constant condition does exactly the same thing on the other
axis and was not checked. It returned a fit in which
scope_area was 0, every effect size was
NaN and every p-value NA – the same
degeneracy, refused on one axis and reported as a number on the
other.
It is now refused, with the escape hatch named: a theoretical
scope with distinct xmin and xmax
gives the box a width and the areas are defined again, and the message
says that the result then depends on where the constant sits inside that
scope.
The refusal is only for the case where every
condition is flat, which is the case where the analysis has nothing to
say at all. One flat condition among several leaves the others
estimable, and stopping would throw those away; those rows keep their
coverage warning and their NaN, which is the honest answer
for a row and not for the call.
Found by a calibration and stress pass over properties the test suite
does not check – stress-0.5.0.R, which also confirmed that
the permutation test is correctly sized.
na.action was applied in the function that computes the
axis bounds, the coverage and the data the plots draw, and the full data
frame was then handed to the engine. The engine drops incomplete pairs
per condition inside its own loop (p_create_loop_data():
tmp[complete.cases(tmp), ]), so every effect size was
fitted on that condition’s own rows however na.action was
set.
"listwise" therefore changed the denominators and not
the numerators, which is the disagreement it exists to prevent, and the
documented behaviour – every condition fitted on the same rows – was not
what happened. Complete data was never affected, which is why it went
unnoticed.
The rule is now applied to the frame before the engine sees it, on
the selected columns only: a missing value in a column that is not in
the analysis is not the analysis’s business, and dropping its row would
silently shrink the sample. Listwise deletion that leaves no complete
row is an error naming "pairwise" as the alternative,
rather than an engine failure.
"pairwise", the default, is unchanged.
scope may be one c(xmin, xmax, ymin, ymax),
a vector of 4 * k values, or a list of k such
vectors. The per-condition forms exist for one reason: conditions
measured on different scales need different X ranges. The outcome half
is a repetition the format requires, not a setting – there is only ever
one outcome column – so writing it k times only creates
k chances to write it differently.
When the Y halves do differ, nothing downstream can represent it. The
threshold table is one table, one row per outcome level and one column
per condition, and a row cannot mean y = 2 in one column
and y = 20 in the next. NCA resolves this by reading the
level grid from the first segment alone (p_low_high():
scope[[1]][3], computed once before the per-condition loop)
while each effect size stays an area over its own box, so the table and
the effect sizes end up measured against different rectangles. NCA warns
and carries on – correct there, since its per-condition analyses are
independent and the table is the only shared result.
SCAtools now raises an error instead, naming the condition, the bound and both values. The fix costs nothing: estimation was never joint, so one call with several conditions is a loop over several independent single-condition analyses, and a condition that genuinely needs a different outcome scope can be fitted in its own call. The only thing a separate call gives up is the shared threshold table – which is exactly the thing that cannot hold two outcome ranges.
NA counts as “not declared”. A bound given for one
condition and left NA for another is the quiet form of
this: NA means “use the observed range”, so the two
conditions land on different boxes without anyone writing two different
numbers. The 4 * k vector form is the one to watch, because
NCA’s own p_validate_scope_vector() does not run the check
that p_validate_scope_list() does, so that form warns
nowhere.
A single c(xmin, xmax, ymin, ymax) is replicated to
every condition and can never trigger this.
sca() exists so that three arguments produce a reported
analysis, which means the package picks the frontier techniques, the
number of replications, the significance level, the reporting scales and
the scope on the caller’s behalf. Nothing in the output used to say so,
and a reader could not tell which numbers a researcher had chosen and
which had been chosen for them – the one distinction a methods section
has to make.
print() now ends with a Settings block
listing every one of them, with (default) against each
value the call did not name:
Settings
frontiers ce_fdh, cr_fdh (default)
test.rep 1000 (default)
alpha 0.05 (default)
seed not fixed (default)
relevance none pre-specified (default)
scope observed range of each condition and the outcome (default)
na.action pairwise (default)
thresholds condition on percentage of range, outcome on percentage of range (default)
convention absolute (default)The marks come from the recorded call, so they are what was actually
passed rather than a comparison against formal defaults.
sca() now records its own call for this purpose: it
forwards all of its arguments to sca_analysis() by name, so
sca_analysis()’s own match.call() reported
every one of them as supplied even when the caller had written
three.
sca_output() ends with the same block. It assembles a
report, and a report has to say what it was run with; a caller who
reaches for it instead of print() should not silently lose
that.
The old Threshold reporting: header line is gone; it
said the same thing without saying whether anyone had chosen it.
Every effect size is an empty area divided by the area of the X-Y
scope, so the scope is half of every number reported.
sca_scope() returns the declared bounds, the bounds the
engine used, the resulting area and whether the two differ – one row per
condition.
sca_table() gains scope_area and
scope_adjusted beside empty_space_area, so
that effect_size = empty_space_area / scope_area is
checkable in the table rather than a claim in the documentation. Code
that selects columns by position rather than by name will need
updating.
p_loop_data.R builds the scope it divides by as the
declared scope widened to hold every observation, so a declared bound
that excludes a data point is replaced by the data point. The
denominator of every effect size in the analysis therefore changes while
the declared scope – the thing a researcher writes down and reports –
does not.
The engine does say so, once per analysis: “Theorectical scope
has been adjusted to include all observations”. That sentence names
neither the condition nor the axis nor either bound, and a flag
suppresses it after the first, so an analysis of six conditions in which
four were widened produced one anonymous line. SCAtools now warns per
condition and per axis with the declared bound and the one that replaced
it, and records the same in metadata$scope_adjustment, in
the Settings block and in sca_scope().
The engine writes that suppression flag into the caller’s global
environment as scope_warnings_nca and does not remove it,
so it turned up in ls() and in save.image()
after any analysis. It is now removed on exit, unless it was already
there before the call.
annotate = FALSE
removes it entirely.sca_test_plot() gains a caption stating what the null
is, and the same wrapping and alignment.The documentation no longer implies that a declared theoretical
scope is required for an effect size to be interpretable. It is not, and
the engine says so: NCA::nca_analysis() defaults to the
observed range. The observed range and a declared scope answer different
questions and neither is privileged. What is hard to defend is a
declared scope with no stated justification, because the researcher then
chose the denominator and a box much wider than the data flatters the
number – unobserved space is empty by default, which is what the new
coverage warning is about. The claim that a declared scope makes studies
comparable is also gone: comparability follows the denominators, and two
studies declaring different theoretical scopes are no more comparable
than two fitted on their own observed ranges. Declaring one makes the
denominator explicit and checkable, which is a different and smaller
claim. sca_four_configurations’ scope section is retitled
accordingly: it documents the box those particular numbers assume rather
than prescribing one.
qr.tau now documents the inversion. The engine fits
tau for an upper empty corner and 1 - tau for
a lower one, so the default 0.95 is the 95th-percentile line where the
empty space lies above and the 5th-percentile line where it lies below.
Sufficiency in the "HH" direction has its empty space at
corner 4 and therefore fits the 5th percentile: a floor, not a ceiling.
It also records that frontier_accuracy for qr
is qr.tau by construction rather than evidence of fit, and
that an NA fit is ordinary for qr
because the engine returns NA whenever a technique’s empty
area exceeds CE-FDH’s.
.sca_from_scale() no longer claims to be the inverse
of .sca_to_scale(). It is on the four affine scales and is
not on percentile, where the forward map is the empirical
CDF and the reverse is stats::quantile(); the forward map
is not injective, so no inverse exists.
sca_corner_map() and the README now state that the
four directions assume a monotone relationship, which is what makes the
expected empty space a single corner. A curvilinear or U-shaped
relationship has no single expected corner, so none of the four states
the claim – and the package will still fit a frontier and return a
number for whichever direction is named.
The 0.4.1 entry on OLS is amended. It said that being first in
the default ceilings made ols the line
sca_extract() reached for; in fact
NCA::nca_summary() drops ols from the
parameter matrix before returning it, so the consequence was a silent
NA, not a wrong number.
sca_power() no longer returns the engine’s
NULL. The engine announces a refusal by printing a line and
returning NULL, so the reason used to be lost and the failure surfaced
later as sca_powerplot() complaining about missing columns.
The printed reason is now captured and re-raised as the error.
.sca_check_slope() stays as a convenience that names the
caller’s own direction; the return check covers everything it does not
know about.sca_plot(),
sca_thresholds() and nsca_plot().
NULL already means every condition, so
character(0) could only come from a filter that matched
nothing – and it used to reach the end, where it produced “incorrect
number of dimensions”.empty_space and relationship were stored as
sentences containing the literal characters X and
Y, filled in by two successive
sub(..., fixed = TRUE) calls. sub() replaces
only the first match, so once the condition name had been substituted, a
condition whose own name contained an upper-case Y
swallowed the second replacement:
.sca_empty_space_text(4, "SYSTEM", "score")
"High X with low Y" -> "High SYSTEM with low Y" -> "High SscoreSTEM with low Y"Yield, Year, YEARS,
QUALITY, SUPPLY and SYSTEM all
trigger it. The two columns appear in sca_table(),
sca_thresholds(), print() and the plot
subtitles, so the corrupted text was reaching whatever the reader copied
out. The templates are now sprintf format strings: a
%s placeholder cannot be hit by the data being substituted
into it. The reference tables sca_corner_map() and the NSCA
equivalents were never affected, because they never substitute
names.
Every quantity SCAtools reports is read out of the NCA result
by name – "Effect size",
"Ceiling accuracy", "p-value" and six others.
When a name was not found, .sca_parameter() returned
NA_real_, so a renamed engine parameter would have emptied
a whole column without an error. That failure is now separated into
three cases that are genuinely different:
NA, quietly.NA – the engine computed
it and there is nothing there. A step frontier has no slope and no
intercept by construction, so this must stay quiet or every default
analysis would warn twice per row..sca_condition_data() bounded the condition axis with
that condition’s complete pairs but the outcome axis with every row
having a finite outcome, whether or not the condition was observed
there. That is neither pairwise nor listwise deletion but a third rule
nobody chose, and since every effect size is an area over the scope box,
the denominator came from more rows than the numerator. Complete data
was never affected, which is why it went unnoticed.
The new na.action argument states the rule:
"pairwise" (the default, and what the old code intended) or
"listwise". Under either, both axes of a condition are
bounded by the same rows.
relevance, and four support columnssca_analysis() takes a pre-specified practical-relevance
threshold, as NSCA::nsca_analysis() has for both its
components since 0.4.0. sca_table() gains
significant, relevant, supported
and support_status, which mirror the sufficiency side of
NSCA’s joint table exactly. Significance and relevance are kept as
separate columns on purpose.
test.seed, and the caller’s random stream is given
backThe engine derives its entire permutation sequence from one draw
taken from the caller’s stream, so set.seed() before a call
already made the p-values reproducible. What it did not do was leave the
stream alone afterwards: NCA calls set.seed(NULL) when the
test finishes, which reseeds from the clock. A single analysis therefore
replaced the stream the caller was using. test.seed now
fixes the sequence and restores the previous stream on exit.
sca() runs a permutation test by default
(test.rep = 1000), matching NSCA::nsca().bottleneck.x and bottleneck.y are removed
from sca_analysis(), and boundary from
sca_thresholds() and sca_rescale(). Use
threshold.x, threshold.y and
inequality.sca_extract() no longer accepts retired column names.
Retired names remain reachable through legacy = TRUE on the
tables, which is the one mechanism that cannot give a different answer
from the current name – it copies the same vector. The promise that
sca_extract() accepted them “with a warning” could never be
kept: all three retired names are threshold-table columns, which are
defined per outcome level and have no single value to return, so the
call warned and then failed.sca_thresholds(), sca_test_plot() and
sca_output() default to the first frontier actually
estimated instead of hard-coded "ce_fdh".
sca_analysis(ceilings = "cr_fdh") followed by
sca_thresholds(fit) used to fail on a model that had been
fitted perfectly well.sca_power() refuses a non-finite slope. The previous
check used anyNA(), which does not catch Inf,
while its own message promised finite numbers.inst/NOTICE said SCAtools 0.1.0, the last
survivor of the 0.1.0 cleanup.Authors@R named a placeholder,
SCAtools Contributors <maintainer@example.org>, which
CRAN rejects: the maintainer must be a real person at a working address.
Both are now Young Chan
youngchanresearcher@gmail.com.
inst/CITATION was missed by that pass and still returned
the placeholder author with R package version 0.1.0, so
citation("SCAtools") contradicted both
DESCRIPTION and the package guide. It now names the same
person and reads the version from DESCRIPTION, so it cannot
fall behind again.
URL pointed at the CRAN page of NCA, the computational
engine, which reads as if it were this package’s home. It now points at
this package’s repository, and BugReports is set alongside
it:
URL: https://github.com/youngchanresearcher/SCAtools
BugReports: https://github.com/youngchanresearcher/SCAtools/issuesNCA’s authorship and licence are stated in Description,
inst/NOTICE and the README, which is where that attribution
belongs.
A frontier line bounds only two of the four corners: a rising line
the upper-left and lower-right (HH, LL), a
falling line the upper-right and lower-left (LH,
HL). A horizontal line is accepted for any direction.
sca_random() inherited the engine’s error for the
impossible combination, but nca_power() only warned and
returned NULL. sca_power(direction = "LH")
with the default slope = 1 therefore produced no result at
all, and the failure was silent by the time the NULL
reached sca_powerplot(). sca_power() now
checks the sign itself and stops with the direction, the corner and the
offending value named. Every combination that worked before still works:
the check refuses exactly what the engine already refused.
legacy = TRUE; corrected there.SCAtools_0.1.0.tar.gz, and both READMEs
still asked the reader to replace a placeholder maintainer that no
longer exists.A packaging release. No code, argument, column or default changed.
README_zh-TW.md moves from the top level of the package
to inst/docs/README_zh-TW.md. R CMD check
counts any unrecognised top-level file as a NOTE; from
inst/ the translation is still shipped and is now reachable
from R:
file.show(system.file("docs", "README_zh-TW.md", package = "SCAtools")).R CMD check --as-cran on 0.4.1 was already clean for
this package – 0 errors, 0 warnings, 505 tests passing. This removes the
one remaining avoidable NOTE.
ceilings = "ols" asked the NCA engine for an ordinary
least-squares regression of the outcome on the condition, fitted to all
the data: the line lm(y ~ x) returns. It is a
central-tendency summary, not a frontier. It estimates no empty space,
so it yields no ceiling zone, no effect size, no permutation p value and
no threshold table – and it stood as the first entry of the
default ceilings, so the package’s own default put a
central-tendency line at the head of a list of frontiers.
Amended in 0.5.0. This entry originally continued “which made it the line
sca_extract()reached for when no ceiling was named”. That was wrong about the mechanism.NCA::nca_summary()dropsolsfrom the parameter matrix before returning it –data2[names(data2) != "ols"], atnca_summary.Rlines 146-147 in NCA 5.0.2 – soolswas never a column that any extractor could resolve, and asking for it returnedNArather than a regression coefficient. The default was wrong for the reason given above; its consequence was a silentNA, not a wrong number reported as a frontier.
Section 4.3 of the condition analysis in degree framework states the difference: a frontier is fixed by the most extreme observations rather than by the central tendency. Section 6.1 is why the line is still worth drawing. Average-effect models and condition analysis answer different questions about the same relationship and are most informative side by side, not merged into one number.
ceilings = c("ce_fdh", "cr_fdh").
Frontiers only.reference argument for central-tendency lines drawn
beside the frontier. reference = "ols" is the supported way
to ask for the regression line."ols" in ceilings still works: it is moved
to reference with a warning. On its own it is now an error,
because no empty space is left to estimate. This matches NSCA, which has
always refused it as a component.sca_reference() reports the line’s intercept,
slope, R-squared and the observations it was fitted to. It is computed
here rather than read back from the engine, and is available whether or
not the line was drawn.sca_table() reports frontiers only, even where an
engine returns a column for a reference line.sca_plot() titles the legend “Line” rather than
“Frontier” when a central-tendency line is among those drawn.sca_terms() gains a central tendency row.Nothing else changed. No column, argument or status value was renamed.
This release adopts the vocabulary of condition analysis in degree, the framework that sets SCA and NSCA beside NCA. Terminology follows the article; the package’s own names are kept only where the article has nothing to say. Nothing was removed: every retired name still works.
Section 3.2 of the framework uses five terms in sequence. The scope is the region of the X-Y space bounded by the theoretical or observed extremes. An expected empty space (or empty zone) is the part of that scope which should hold no observations if the hypothesised relation holds. A boundary is the theoretical line separating that space from the compatible region. A frontier is the boundary estimated from data. Ceiling and floor name the direction in which Y is bounded.
The package already used frontier for the estimated
line, which is correct. It also used boundary for something
else entirely – whether a reported rule uses > or
>= – and that name is now needed for the theoretical
line.
sca_thresholds() and sca_rescale() rename
the boundary argument to inequality. The old
argument still works and warns; supplying both keeps
inequality and warns again.boundary output column of
sca_thresholds() is renamed inequality.For sufficiency the fitted boundary is a floor
whenever the outcome level is high (HH, LH) and a
ceiling whenever it is low (HL, LL). The package
estimated the right thing but never said which it was, because the
argument is called ceilings in every direction. Three
columns now state the geometry in the framework’s own words:
boundary_type – "ceiling" or
"floor".empty_space – what the expected empty space contains,
in the wording of Table 1 and with the user’s own variable names filled
in, for example "High training with low score".relationship – the direction as a claim about the
theorised X-Y relationship, for example
"Higher training for higher score". This is a claim about
the relationship, not about the shape of the estimated frontier, which
may be a step function or a straight line under any of the four
types.All three appear in sca_table(),
sca_corner_map() and the printed mapping;
boundary_type and relationship also appear in
sca_thresholds(). They are derived from the empty corner,
which remains the single source of truth.
Section 4.3 names the quantity a sufficiency analysis reports: the lowest X at which the estimated sufficiency frontier reaches a stated outcome target.
condition_threshold becomes
sufficiency_threshold.condition_threshold_actual becomes
sufficiency_threshold_actual.This also makes the two packages agree: NSCA already used these names for the same quantity, and previously had to read SCAtools’ differently named column.
sca_legacy_names() returns the map from retired names
to their replacements.sca_table() and sca_thresholds() gain
legacy = TRUE, which appends the retired names as duplicate
columns. Both retired columns belong to the threshold table, so
in practice the argument adds columns in sca_thresholds()
only; sca_table() accepts it for symmetry and returns the
table unchanged.sca_extract() accepts a retired name with a warning.
Asking it for a threshold-table column now explains that such a column
is defined per outcome level and points to
sca_thresholds(), instead of failing with a message about
an unknown NCA parameter.sca_terms() is a new machine-readable glossary linking
each term of the framework to the name this package uses for it.sca_rescale() was exported in 0.2.1 but never
documented. It now has a help page, on ?sca_scales.sca_output() gains inequality, which it
passes to sca_thresholds(). Previously the strictness of
the printed rules could not be controlled from sca_output()
at all.sca_rescale(), the scale conversion used by
sca_thresholds(), as a public function. Other packages need
it to express their own quantities on the same scale: a threshold from
another source is only comparable with an SCA threshold when both went
through the same direction-aware mirroring and the same percentile
strictness rule. Reaching into the internal function instead would break
whenever this package changes.Renames the reporting arguments of sca_analysis()
from bottleneck.x and bottleneck.y to
threshold.x and threshold.y. A bottleneck is a
necessity concept: it names the minimum level of a condition required to
reach an outcome. A sufficiency statement has no bottleneck, so the term
is no longer used. The old arguments still work and emit a deprecation
warning.
Replaces the engine’s NN and NA markers
with a status column stated in sufficiency terms. The two
markers invert under contraposition, so copying them would reverse the
reader’s conclusion:
| Engine cell | Necessity reading | Sufficiency reading | status |
|---|---|---|---|
NN |
no minimum is required | no attainable condition value reaches this outcome level | no_threshold |
NA |
the requirement lies outside the scope | the threshold falls outside the scope, so the condition restricts nothing | always_satisfied |
The rule text for these rows describes the condition side only. Asserting that the outcome holds for every observation would be stronger than the row supports, because a case sitting exactly on the boundary can still fail it.
Makes percentage.range, percentage.max
and percentile reach the rules themselves, and adds
sd. Previously the requested scale never reached the rule
text: sca_thresholds() always wrote the inequality in
actual units, and the requested scale appeared only in separate display
columns that carried the engine’s conventions unchanged. Asking for
percentages therefore produced actual-unit rules beside percentage
columns that could not be read against them.
SCAtools now requests thresholds from the engine in actual units and applies every scale conversion itself. This fixes three defects that follow from the engine’s own conventions, all of which affected SCAtools 0.1.0:
HH and LH directions. The engine reverses
the actual outcome levels when the corner is flipped but leaves the
displayed percentages ascending, so a row labelled 0%
carried the highest outcome value. Both H-outcome
directions were affected, and percentage.range was the
default.HH and HL the reported percentile was the
share of cases at or above the threshold, so a value of 70
meant the 30th percentile.Adds a convention argument, and
sca_scales() to list what is available. Under
"absolute" (the default) 0 sits at the low end
of every axis and the inequality carries the direction, so numbers are
comparable across directions. Under "directional"
0 sits at the least sufficient end, the axis is mirrored
for a low-level direction, and every rule reads as “more of the
sufficient thing”.
Percentile strictness now follows the rule’s inequality. A
percentile is a step function, so pairing a strict <
rule with P(X <= t) dropped the observation nearest the
threshold once the data were expressed on the same scale. Each
combination of direction, convention and boundary is now
truth-preserving, and is covered by
tests/testthat/test-scales.R.
sca_thresholds() gains scale,
outcome_scale and convention arguments.
Because actual values are retained, a fitted model can be re-expressed
on any scale without refitting.
sca_analysis() stores the per-condition data and axis
bounds it used, after the engine’s complete-case filtering, so
conversions match the sample the frontier was fitted on.sca_thresholds() warns instead of failing silently if
the engine does not preserve exact threshold values, and states that its
fallback is rounded to three decimals.stats and utils to
Imports, which were already in use.sufficiency-logic.md from inst/doc/
to inst/docs/. inst/doc is reserved for
vignette build output, so R CMD build empties it and
devtools::check() deletes its contents from the source
tree.ggplot2 graphics.