--- title: "Upgrading from DesignLibrary 0.1: what changes, what does not, and why" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Upgrading from DesignLibrary 0.1: what changes, what does not, and why} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set(collapse = TRUE, comment = "#>") if (dir.exists("../inst/designs")) { options(DesignLibrary.root = normalizePath("..", winslash = "/")) } ``` # The short version DesignLibrary 2.0 replaces the designer functions of 0.1 with a library of self-contained design files, read through `make_design()`. It requires DeclareDesign, fabricatr, estimatr, and randomizr 2.0. All fifteen 0.1 designers keep their names and their argument names, with two exceptions: `block_cluster_two_arm_designer()` loses six arguments, and three arguments (`label_E1` and `label_E2` of `process_tracing_designer()`, and `args_to_fix` of every designer) are accepted but ignored with a warning. Each now builds a library design, so `two_arm_designer(N = 40)` returns the same design as `make_design("two_arm_flexible", N = 40)`. Most 0.1 scripts that call a designer and diagnose the result run unchanged. There are two types of break: * Removed functions and removed arguments stop with an error that names the replacement. * A handful of changes do not error. These aries if there is a changed default, renamed inquiries, or renamed estimator terms or different step names. See [Porting an existing script](#porting-an-existing-script) for more. # What does not change A 0.1 script like this one runs as written. As in 0.1, `library(DesignLibrary)` also attaches DeclareDesign, fabricatr, estimatr, and randomizr. ```{r, message = FALSE} library(DesignLibrary) design <- two_arm_designer(N = 100, ate = 0.25) set.seed(343) draw_estimates(design) ``` What carries over: - The fifteen designer names: `two_arm_designer()`, `two_arm_attrition_designer()`, `two_arm_covariate_designer()`, `pretest_posttest_designer()`, `randomized_response_designer()`, `mediation_analysis_designer()`, `multi_arm_designer()`, `two_by_two_designer()`, `factorial_designer()`, `block_cluster_two_arm_designer()`, `regression_discontinuity_designer()`, `spillover_designer()`, `cluster_sampling_designer()`, `binary_iv_designer()`, and `process_tracing_designer()`. - Their argument names and defaults, except for `block_cluster_two_arm_designer()` and the default `N` of `multi_arm_designer()`. - Inquiry and estimator labels, except for the cases listed below. - The designs themselves. Seven of the designers became new library files (`two_arm_covariate`, `regression_discontinuity_polynomial`, `spillover`, `cluster_sampling`, `binary_iv`, `process_tracing_bayes`, and `factorial`) flattened from their 0.1 code, and at the same arguments and seed each draws what its 0.1 designer drew. # What changes, and why ## Removed functions Four aliases that 0.1 had already deprecated are gone, along with two helpers. In each case you get an informative message. ```{r, error = TRUE} simple_two_arm_designer(N = 50) ``` | Removed | Use instead | |---|---| | `simple_two_arm_designer()` | `two_arm_designer()`, same arguments | | `simple_factorial_designer()` | `two_by_two_designer()`, same arguments | | `simple_iv_designer()` | `binary_iv_designer()`, same arguments | | `simple_spillover_designer()` | `spillover_designer()`, same arguments | | `get_design_code()` | `get_code("")` | | `match.call.defaults()` | `match.call()` and `formals()` | In 0.1 every designer pasted its own source code into an attribute of the design it returned, and `get_design_code()` read that attribute back. In 2.0 the source of a design is a file in the package, so there is nothing to paste: `get_code()` prints the file. Each designer's help page names the id of the file it builds. ```{r, error = TRUE} get_design_code(two_arm_designer()) ``` ```{r} get_code("two_arm_flexible") ``` The `description`, `definitions`, and `shiny_arguments` attributes went with `code`. Now `get_args()` and `design_info()` describe a design's parameters. ## `block_cluster_two_arm_designer()` takes new arguments The 0.1 designer derived its level sizes from `N` and its three shock standard deviations from one total `sd`, through defaults that referred to each other. Its defaults also produced a design with no blocking and no clustering (one block of 100 single-unit clusters) and no effect. The 2.0 designer takes the level sizes and the shocks directly, and its defaults are a blocked and clustered design with an effect to find: 20 blocks of 4 clusters of 10 units, each shock with standard deviation 0.577, and `ate = 0.2`. The 0.1 arguments `N`, `sd`, `sd_i_1`, `rho`, `control_mean`, and `treatment_mean` are errors that say what to set instead. ```{r, error = TRUE} block_cluster_two_arm_designer(N = 200, rho = 0.5) ``` ```{r} design <- block_cluster_two_arm_designer( N_blocks = 5, N_clusters_in_block = 4, N_i_in_cluster = 10, ate = 0.1 ) ``` `assignment_probs` takes one probability rather than one per block, and `verbose` is ignored with a warning. **Because the defaults changed, `block_cluster_two_arm_designer()` with no arguments is a different design in 2.0, and nothing warns about it.** ## `multi_arm_designer()` has a larger default sample As in 0.1, the designer declares an inquiry for every pair of arms and estimates each by a difference in means between its two arms, under labels like `"DIM (Z_2 - Z_1)"`. The default `N` went from 30 to 90, so that each of the three arms has 30 units rather than 10. ```{r} design <- multi_arm_designer(m_arms = 3) set.seed(343) draw_estimates(design)[, c("estimator", "term", "estimate", "inquiry")] ``` ## Two renamings that change no values `mediation_analysis_designer()`'s inquiries `Indirect_0` and `Indirect_1` are now `Mediator_Effect_0` and `Mediator_Effect_1`. Each is the effect of the mediator on the outcome with treatment held at 0 or at 1. ```{r} draw_estimands(mediation_analysis_designer()) ``` `process_tracing_designer()` fixes its estimator labels at `"Straw in the Wind"` and `"Smoking Gun"`. ```{r} design <- process_tracing_designer(label_E1 = "Hoop") ``` ## `args_to_fix` is ignored 0.1 designers could fix arguments into the returned design so that `expand_design()` could not vary them. In 2.0 every parameter stays redesignable on the design object, and a non-empty `args_to_fix` warns. ```{r} design <- two_arm_designer(args_to_fix = "N") ``` ## Designs are now all DeclareDesign 2.0 designs DeclareDesign 2.0 has no separate potential outcomes or reveal steps: potential outcomes live in `declare_model()` and are revealed in `declare_measurement()`. Some step names differ as a result. `factorial_designer()`'s inquiry step, for example, was `"ATE"` and is now `"inquiry"`. ```{r} names(factorial_designer()) ``` Inquiry and estimator labels are otherwise those of 0.1. **Code that picks a step by name or by position (`design[[3]]`) needs checking**, since the position of a step can change even where its name does not. ## Sweeping a parameter `expand_design()` still works but is deprecated in DeclareDesign 2.0. Pass a vector to `redesign()` instead, which needs no designer function at all. ```{r} designs <- redesign(two_arm_designer(), N = c(50, 100)) length(designs) ``` One caution: `process_tracing_designer()` checks that `cor_E1E2_H` and `cor_E1E2_not_H` are compatible with the clue probabilities, and `redesign()` does not. An incompatible value passed through `redesign()` fails only when the design is run, with "negative probability" from the measurement step. # Porting an existing script Search the script for each pattern in the left column. | Search for | Replace with | If you do not | |---|---|---| | `simple_two_arm_designer`, `simple_factorial_designer`, `simple_iv_designer`, `simple_spillover_designer` | `two_arm_designer`, `two_by_two_designer`, `binary_iv_designer`, `spillover_designer` | error naming the replacement | | `get_design_code(` | `get_code("")` | error naming the replacement | | `match.call.defaults(` | `match.call()` and `formals()` | error | | `block_cluster_two_arm_designer(` with `N`, `sd`, `sd_i_1`, `rho`, `control_mean`, or `treatment_mean` | `N_blocks`, `N_clusters_in_block`, `N_i_in_cluster`, `sd_block`, `sd_cluster`, `sd_i`, `ate` | error naming the replacement | | `block_cluster_two_arm_designer()` relying on defaults | state the 0.1 sizes and `ate = 0` explicitly | **silently a different design** | | `multi_arm_designer(` without `N` | `N = 30` for the 0.1 default | **silently a larger sample** | | `Indirect_0`, `Indirect_1` | `Mediator_Effect_0`, `Mediator_Effect_1` | **filter returns no rows** | | `design[[` with a number, or step names such as `"ATE"` | `names(design)` to find the step | **wrong step, or error** | | `args_to_fix` | drop it | warning | | `expand_design(` | `redesign(designer(), arg = values)` | deprecation warning | | `make_design(design = ` | `make_design("")` | error | The rows in bold are the ones to search for deliberately, since nothing will point you to them. # What is new The big change is that the key files contained in the library are simple design scripts rather than "designer functions." This makes it much easier to inspect a design and to contribute a design. In addition the designs make use of the the new `declare_parameters` functionality from DeclareDesign 2.0 which makes it easier to contribute modifiable designs. The library holds `r nrow(as.data.frame(list_designs()))` designs, including the declarations from *Research Design in the Social Sciences*. * `list_designs()` browses the library * `make_design()` builds a design and changes its parameters * `get_args()` lists what can be changed * `get_code()` prints the source * `designs_shiny_app()` opens the library in a browser * `vignette("DesignLibrary")` gives you a tour.