DMAR 1.1.0 names the total sample size, the number of predictors, a planner's planning value, and the desired width one way, and names an argument for the quantity it holds rather than for a symbol. The DMAR 1.0.x names listed here are not accepted: a call that gives one stops with R's "unused argument" error. The exceptions, all of them, are listed under Old Names That R Still Matches, after the table. There is no deprecation period and no function that maps an old name to a new one. Change the call. Three functions are removed, with no alias; the call that replaces each is given under One Function for Each ANOVA Effect Size.
var_partial_r(), var_semipartial_r(), expected_partial_r(),
ss_aipe_partial_r(), ss_aipe_semipartial_r(),
ss_aipe_partial_r_sensitivity(), and
ss_aipe_semipartial_r_sensitivity() took J, the number of variables
partialed out. They take p, the total number of predictors in the
regression of Y on X and the variables partialed out, in the place J
held, and p = J + 1. A DMAR 1.0.x call that gave the count by
position still runs, and now describes one fewer variable partialed out.
var_partial_r(.3, 100, 2) was a partial correlation given two other
variables and is now a partial correlation given one;
ss_aipe_partial_r(.3, 2, .2) planned for two variables partialed out and
now plans for one. Add 1 to the count in every such call:
var_partial_r(.3, 100, 3), ss_aipe_partial_r(.3, 3, .2). A call that
named the count (J = 2) stops with "unused argument" and becomes
p = 3. p must be a whole number of at least 2; with nothing partialed
out the correlation is the simple correlation, which var_r(),
expected_r(), ss_aipe_r(), and ss_aipe_r_sensitivity() cover.
eta_squared_partial(), omega_squared_partial(), and
ci_eta_squared_partial() are removed. eta_squared(),
omega_squared(), ci_eta_squared(), and ci_omega_squared() take
estimand = c("all", "total", "partial") and report the total and the
partial value of each effect in columns of their own (see Total and
Partial Eta Squared and Omega Squared). There is no alias. Replace
| DMAR 1.0.x call | DMAR 1.1.0 call |
|---|---|
| eta_squared_partial(fit) | eta_squared(fit, estimand = "partial") |
| eta_squared_partial(F_value = , df_effect = , df_error = ) | eta_squared(F_value = , df_effect = , df_error = ) (the partial value; estimand = "partial" leaves out the NA total column) |
| omega_squared_partial(fit) | omega_squared(fit, estimand = "partial") |
| omega_squared_partial(F_value = , df_effect = , df_error = , N = ) | omega_squared(F_value = , df_effect = , df_error = , N = , estimand = "partial") |
| ci_eta_squared_partial(fit) | ci_eta_squared(fit, estimand = "partial") |
| ci_eta_squared_partial(F_value = , df_effect = , df_error = , N = ) | ci_eta_squared(F_value = , df_effect = , df_error = , N = , estimand = "partial") |
The columns of the four tables are named for the estimand. A
script that read $eta_squared, $omega_squared, $ci_lower, or
$ci_upper from these tables reads the new name. The values under the
old names were the partial values, and the new argument does not change
them.
| function | DMAR 1.0.x column | DMAR 1.1.0 column |
|---|---|---|
| eta_squared() | eta_squared | eta_squared_partial |
| omega_squared() | omega_squared | omega_squared_partial |
| ci_eta_squared() | eta_squared, ci_lower, ci_upper | eta_squared_partial, eta_squared_partial_ci_lower, eta_squared_partial_ci_upper |
| ci_omega_squared() | omega_squared, ci_lower, ci_upper | omega_squared_partial, omega_squared_partial_ci_lower, omega_squared_partial_ci_upper |
With estimand = "all", the default, the columns of the total
(eta_squared_total, omega_squared_total, and for the intervals
eta_squared_total_ci_lower and so on) come before those of the
partial value; estimand = "partial" gives the table of 1.0.x with the
new names. The effect, stratum, F_value, df_effect, df_error,
and N columns are unchanged.
estimand is the second argument of eta_squared(),
omega_squared(), ci_eta_squared(), and ci_omega_squared(), after
object. A call that gave F_value by position (as the second
argument) stops; name it.
tidy() of these tables gives one row per effect and estimand, the
rows of the total before those of the partial value, with term naming
the estimand ("eta_squared_total", "eta_squared_partial") where it
named "eta_squared". glance() of a table with more than one row of
tidy() gives the estimands in term, separated by a comma.
An argument is named for the quantity it holds and for the role of the
value. A population value that a planner or a sensitivity function takes
begins with true_, as do those of the analysis of variance, analysis of
covariance, and regression data generators; the value a sensitivity
function plans the design from begins with supposed_ (estimated_ or
est_ in DMAR 1.0.x, which suggested an estimate from data, where the
value is a planning assumption); and a null value begins with null_.
The quantity is named in words: a correlation is cor (true_cor,
true_cor_YX, cor_lower), a contrast contrast and a standardized one
std_contrast, a regression coefficient reg_coef and a standardized one
std_reg_coef, a squared multiple correlation R2, a mean mean, a
standard deviation sd (true_sd, and true_sd_X and true_sd_Y when
the variable is named), a population variance true_var_ followed by
what it is the variance of (true_var_within, true_var_error,
true_var_trend), a covariance matrix cov_matrix, a factor loading
factor_loading, and a residual variance residual_var. A function that
computes the same quantity from population values or from estimates
(cohen_f(), ci_srsnr(), covmat_from_cfa(), var_ete(),
var_indirect_effect()) names its arguments without a role. A sample size
given so that the
power or the precision at that size is returned is specified_N (the
total, or the one group) or specified_n_per_group. The table gives the
name each argument had in DMAR 1.0.x.
| family | function | DMAR 1.0.x | now |
|--------|----------|------------|-----|
| the chi square statistic | ci_nc_chisq(), convert_chisq_F() | chi_square | chisq (as in sem_fit_measures(), ecvi(), and equivalence_rmsea(); a row or column of a table is still chi_square) |
| a per-group sample size given for evaluation | ss_power_contrast(), ss_aipe_c_sensitivity(), ss_aipe_sc_sensitivity(), ss_aipe_sc_ancova_sensitivity(), ss_aipe_smd_sensitivity(), ss_aipe_equivalence_smd_sensitivity(), ss_aipe_pcm_sensitivity() | n_per_group | specified_n_per_group (also in ss_power_c(), ss_power_c_ancova(), ss_power_sc(), ss_power_composite_ancova_2group(), and ss_power_split_plot_anova(), listed below) |
| a population standard deviation | ss_seq_c_sensitivity() | true_sigma | true_sd |
| | power_equivalence_md(), power_density_equivalence_md(), power_equivalence_md_plot() | sigma; power_sigma in power_density_equivalence_md() | true_sd; sd_values |
| a calculator given population values or sample summaries | cohen_f() | mu, sigma_squared, sigma_m, sigma | means, var_within, sd_means, sd_within |
| | ci_srsnr() | sigma_squared | var_within |
| | covmat_from_cfa() | lambda, psi_squared | factor_loading, residual_var |
| | var_ete() | sigma2, sigma2_Z, beta_1, beta_2, mu_Z | error_variance, var_Z (either may be a population value or an estimate, as type says, so neither is named true_), reg_coef_1, reg_coef_2, mean_Z |
| a size per cell or per arm given for evaluation | ss_power_factorial_anova(), ss_power_factorial_ancova(), ss_power_composite_anova(), ss_power_composite_ancova(), ss_power_composite_factorial_anova(), ss_power_composite_factorial_ancova(), ss_power_composite_factorial_ancova_het() | n_per_cell | specified_n_per_cell |
| the estimated contrast and its equivalence bounds | equivalence_c() | psi_hat, delta_lower, delta_upper | contrast, contrast_lower, contrast_upper |
| the variance of the predictor in a two-level model | ss_aipe_mixed_effects_sensitivity() | true_sigma2_x, estimated_sigma2_x | true_var_X, supposed_var_X |
| partial and semipartial correlation | var_partial_r(), var_semipartial_r(), expected_partial_r() | n, J | N, p (p = J + 1) |
| | expected_partial_r() | rho | true_cor |
| | ss_aipe_partial_r() | rho, J | true_cor, p (p = J + 1) |
| | ss_aipe_semipartial_r() | r_sp, J | true_r_sp, p (p = J + 1) |
| | ss_aipe_partial_r_sensitivity(), ss_aipe_semipartial_r_sensitivity() | J | p (p = J + 1) |
| | ss_aipe_partial_r_sensitivity() | true_rho, estimated_rho | true_cor, supposed_cor |
| | ss_aipe_semipartial_r_sensitivity() | estimated_r_sp | supposed_r_sp |
| correlation | ci_r(), var_r(), expected_r(), equivalence_r() | n | N |
| | var_r(), expected_r() | rho | true_cor |
| | equivalence_r() | rho_lower, rho_upper | cor_lower, cor_upper |
| | ss_aipe_r() | rho | true_cor |
| | ss_aipe_r_sensitivity() | true_rho, estimated_rho | true_cor, supposed_cor |
| | ss_power_r() | rho, rho_0, N | true_cor, null_cor, specified_N |
| | ss_aipe_equivalence_r() | population_r | true_cor |
| | ss_aipe_equivalence_r_sensitivity() | true_r, estimated_r, rho_lower, rho_upper | true_cor, supposed_cor, cor_lower, cor_upper |
| coefficient of variation | ci_cv(), var_cv() | n | N |
| | var_cv() | cv | true_cv, as in ss_aipe_cv() |
| | ss_aipe_cv() | C_of_V, mu, sigma | true_cv, true_mean, true_sd |
| | ss_aipe_cv_sensitivity() | mean, estimated_cv | true_mean, supposed_cv |
| squared multiple correlation | ss_aipe_R2() | population_R2, g | true_R2, G_initial |
| | ss_power_R2() | population_R2, cohen_f2, print_progress | true_R2, true_f2, print_iter |
| | var_R2(), expected_R2() | population_R2 | true_R2, as in ss_aipe_R2() and ss_power_R2() |
| | ss_aipe_R2_sensitivity() | w, estimated_R2, rho_yx, rho_xx | width, supposed_R2, true_cor_YX, true_cor_XX |
| | ss_power_R2_sensitivity() | estimated_R2, rho_yx, rho_xx | supposed_R2, true_cor_YX, true_cor_XX |
| regression coefficient | ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity(), ss_aipe_src_sensitivity() | w, estimated_var_Y, estimated_cov_YX, estimated_cov_XX | width, supposed_var_Y, supposed_cov_YX, supposed_cov_XX |
| | ss_power_reg_coef_sensitivity() | estimated_var_Y, estimated_cov_YX, estimated_cov_XX | supposed_var_Y, supposed_cov_YX, supposed_cov_XX |
| | ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity(), ss_power_reg_coef_sensitivity() | standardize (default FALSE) | removed: these report the unstandardized coefficient; ss_aipe_src_sensitivity() reports the standardized one |
| | ss_aipe_src_sensitivity() | standardize (default TRUE) | removed: it reports the standardized coefficient |
| | ci_reg_coef(), ci_rc() | b_j, SE_b_j, s_Y, s_X | reg_coef_j, se_reg_coef_j, sd_Y, sd_X |
| | ci_src() | beta_j, SE_beta_j, b_j, SE_b_j, s_Y, s_X | std_reg_coef_j, se_std_reg_coef_j, reg_coef_j, se_reg_coef_j, sd_Y, sd_X |
| | ss_aipe_reg_coef() | b_j, rho2_Y_X, rho2_j_X_without_j, sigma_Y, sigma_X, rho_XX, rho_YX | true_reg_coef_j, true_R2_Y_X, true_R2_j_X_without_j, true_sd_Y, true_sd_X, true_cor_XX, true_cor_YX |
| | ss_aipe_rc() | b_j, rho2_Y_X, Rho2_j_X_without_j, sigma_Y, sigma_X_j, rho_XX, rho_YX | true_reg_coef_j, true_R2_Y_X, true_R2_j_X_without_j, true_sd_Y, true_sd_X, true_cor_XX, true_cor_YX |
| | ss_aipe_src() | beta_j, rho2_Y_X, Rho2_j_X_without_j, sigma_Y, sigma_X_j, rho_XX, rho_YX | true_std_reg_coef_j, true_R2_Y_X, true_R2_j_X_without_j, true_sd_Y, true_sd_X, true_cor_XX, true_cor_YX |
| | ss_power_reg_coef() | beta_j, rho2_Y_X, rho2_Y_X_without_j, rho2_j_X_without_j, sigma_X, sigma_Y, rho_XX, rho_YX, cohen_f2, print_progress | true_reg_coef_j (the coefficient on the scale true_sd_X and true_sd_Y describe), true_R2_Y_X, true_R2_Y_X_without_j, true_R2_j_X_without_j, true_sd_X, true_sd_Y, true_cor_XX, true_cor_YX, true_f2, print_iter |
| | ss_power_rc() | beta_j, rho2_Y_X, rho2_Y_X_without_j, Rho2_j_X_without_j, sigma_X, sigma_Y, rho_XX, rho_YX, cohen_f2, print_progress | true_reg_coef_j, true_R2_Y_X, true_R2_Y_X_without_j, true_R2_j_X_without_j, true_sd_X, true_sd_Y, true_cor_XX, true_cor_YX, true_f2, print_iter |
| contrast among means | ss_aipe_c() | error_variance, MSwithin, SD | true_var_error (the square of SD) |
| | ss_aipe_c_ancova() | error_var_ancova, error_var_anova, rho | true_var_error_ancova, true_var_error_anova, true_cor |
| | ss_aipe_c_ancova_sensitivity() | true_error_var_ancova, true_error_var_anova, est_error_var_ancova, est_error_var_anova, rho, est_rho, mu_y, sigma_y, mu_x, sigma_x | true_var_error_ancova, true_var_error_anova, supposed_var_error_ancova, supposed_var_error_anova, true_cor, supposed_cor, true_mean_Y, true_sd_Y, true_mean_X, true_sd_X |
| | ss_aipe_c_sensitivity() | true_error_variance, estimated_error_variance, true_psi | true_var_error, supposed_var_error, true_contrast |
| | ci_c(), ci_c_ancova(), ci_sc(), ci_sc_ancova() | psi | contrast (the estimate row is contrast in ci_c() and ci_c_ancova() and std_contrast in ci_sc() and ci_sc_ancova()) |
| | ss_power_c() | psi, sigma, n | true_contrast, true_sd, specified_n_per_group |
| | ss_power_contrast() | psi, mu, sigma_squared, print_progress | true_contrast, true_means, true_var_error, print_iter |
| | ci_c(), ci_c_ancova(), equivalence_c(), ss_seq_c() | n | n_per_group |
| | ss_power_c_ancova() | psi, sigma, rho, n | true_contrast, true_sd, true_cor, specified_n_per_group |
| standardized contrast | ss_aipe_sc() | psi_standardized | true_std_contrast |
| | ss_power_sc() | psi_standardized, n | true_std_contrast, specified_n_per_group |
| | ci_sc(), ci_sc_ancova() | n | n_per_group |
| | ss_aipe_sc_ancova() | psi_standardized, psi, rho, sigma_anova, sigma_ancova, ratio | true_std_contrast, true_contrast, true_cor, true_sd_anova, true_sd_ancova, true_sd_ratio |
| | ss_aipe_sc_sensitivity() | true_psi, estimated_psi, desired_width | true_std_contrast, supposed_std_contrast, width |
| | ss_aipe_sc_ancova_sensitivity() | true_psi, estimated_psi, desired_width, rho, mu_x, sigma_x | true_std_contrast, supposed_std_contrast, width, true_cor, true_mean_X, true_sd_X |
| standardized mean difference | ss_aipe_smd() | delta | true_smd |
| | ss_aipe_smd_sensitivity() | true_delta, estimated_delta, desired_width | true_smd, supposed_smd, width |
| | ss_power_smd() | smd | true_smd |
| | ss_aipe_equivalence_smd() | population_smd | true_smd |
| | ss_aipe_equivalence_smd_sensitivity() | estimated_smd, delta_lower, delta_upper | supposed_smd, smd_lower, smd_upper |
| | equivalence_smd() | delta_lower, delta_upper | smd_lower, smd_upper |
| | expected_smd(), var_smd() | delta | true_smd |
| | var_smd_trimmed() | population_smd_trimmed | true_smd_trimmed, as var_smd() takes true_smd |
| | convert_delta_lambda() | delta | smd (a population value or an estimate; the function name keeps delta) |
| | cles(), nnt_from_smd(), proportion_of_superiority() | smd_lower, smd_upper (confidence limits on d) | smd_ci_lower, smd_ci_upper |
| | ss_power_composite_ancova_2group() | smd, rho, sigma, sd_cov, n | true_smd, true_cor, true_sd, true_sd_X, specified_n_per_group |
| noncentral t and F | convert_lambda_delta(), convert_lambda_R2() | lambda | ncp (the function names keep lambda) |
| | ci_nc_t() | t_value (a second name for ncp) | ncp |
| analysis of variance | ss_aipe_omega_squared() | population_omega_squared | true_omega_squared |
| | ss_aipe_omega_squared_sensitivity() | estimated_omega_squared | supposed_omega_squared |
| | var_omega_squared() | population_omega_squared | true_omega_squared, as in ss_aipe_omega_squared() |
| | ss_power_one_way_anova() | f, eta_squared, N | true_f, true_eta_squared, specified_N |
| | ss_power_rm_anova() | f, eta_squared, rho, epsilon, n | true_f, true_eta_squared, true_cor, true_epsilon, specified_N |
| | ss_power_split_plot_anova() | f, partial_eta_squared, rho, epsilon, n | true_f, true_eta_squared_partial, true_cor, true_epsilon, specified_n_per_group |
| | ss_power_factorial_anova() | f, partial_eta_squared | true_f, true_eta_squared_partial |
| | ss_power_factorial_ancova() | f, partial_eta_squared, covariate_R2 | true_f, true_eta_squared_partial, true_covariate_R2 |
| composite power in the analysis of variance and covariance | ss_power_composite_anova(), ss_power_composite_factorial_anova() | means, sigma | true_means, true_sd |
| | ss_power_composite_factorial_ancova() | means, sigma, covariate_R2 | true_means, true_sd, true_covariate_R2 |
| | ss_power_composite_factorial_ancova_het() | means, correlations, sigma, sd_cov | true_means, true_cor, true_sd, true_sd_X |
| | ss_power_composite_ancova() | means, sigma, covariate_R2, correlations, sd_cov | true_means, true_sd, true_covariate_R2, true_cor, true_sd_X |
| | cohen_f(), is_orthogonal_set(), variance_components_mls() | n | n_per_group |
| equivalence of a contrast and of a mean difference | power_equivalence_c() | n, sigma, delta_lower, delta_upper, true_psi | n_per_group, true_sd, contrast_lower, contrast_upper, true_contrast |
| | ss_power_equivalence_c() | sigma, delta_lower, delta_upper, true_psi | true_sd, contrast_lower, contrast_upper, true_contrast |
| | power_equivalence_md(), power_density_equivalence_md(), power_equivalence_md_plot() | n | n_per_treatment |
| | power_equivalence_md() | ltheta1, ltheta2, ldiff, nu | lower_bound, upper_bound, true_effect, df_error (ratios on the original scale when logscale = TRUE, differences otherwise, as before) |
| | power_density_equivalence_md() | theta1, theta2, diff, nu | lower_bound, upper_bound, true_effect, df_error (on the analysis scale, as before) |
| | power_equivalence_md_plot() | theta1, theta2, nu | lower_bound, upper_bound, df_error |
| | plot_equivalence() | delta_lower, delta_upper | effect_lower, effect_upper (bounds on the scale of whatever estimates are drawn) |
| sequential estimation with minimum risk | mr_smd() | n | n_per_group |
| intraclass correlation and coefficient alpha | ss_aipe_icc() | rho, k | true_icc, n_raters |
| | ss_aipe_icc_sensitivity() | true_rho, estimated_rho, k | true_icc, supposed_icc, n_raters |
| | var_icc() | rho, n, k | icc, N, n_raters |
| | var_alpha() | n, p_items | N, n_items |
| two-level models | ss_aipe_mixed_effects() | sigma2_y, sigma2_x, icc | true_total_var, true_var_X, true_icc |
| | ss_aipe_mixed_effects_sensitivity() | true_sigma2_y, estimated_sigma2_y, estimated_icc, true_beta, specified_K | true_total_var, supposed_total_var, supposed_icc, true_reg_coef, specified_n_clusters |
| | ss_power_mixed_effects() | d, n, rho, J | true_smd, cluster_size, true_icc, specified_n_clusters_per_arm |
| polynomial change models | ss_aipe_pcm() | variance_trend, error_variance, variance_true_minus_estimated_trend | true_var_trend, true_var_error, true_var_trend_error |
| | ss_power_pcm() | beta, tau, level_1_variance, N | true_std_reg_coef (beta under the default standardized = TRUE) or true_reg_coef (beta with standardized = FALSE), true_var_trend, true_var_error, specified_n_per_group (the size of each group, half of the total that N held) |
| | ss_power_pcm() | standardized | removed: the argument given, true_std_reg_coef or true_reg_coef, says the scale |
| | ss_aipe_pcm_sensitivity() | true_variance_trend, true_error_variance, estimated_variance_trend, estimated_error_variance | true_var_trend, true_var_error, supposed_var_trend, supposed_var_error |
| cluster randomized designs | ss_aipe_crd_both_fixed_budget(), ss_aipe_crd_both_fixed_width(), ss_aipe_crd_n_clusters_fixed_budget(), ss_aipe_crd_n_clusters_fixed_width(), ss_aipe_crd_n_individuals_fixed_budget(), ss_aipe_crd_n_individuals_fixed_width() | tau_Y, sigma2_Y, total_var, icc_Y, R2_between, R2_within, num_predictors, clus_cost, indiv_cost | true_var_between, true_var_within, true_total_var, true_icc, true_R2_between, true_R2_within, n_covariates, cluster_cost, individual_cost |
| | ss_aipe_crd_es_both_fixed_budget(), ss_aipe_crd_es_both_fixed_width(), ss_aipe_crd_es_n_clusters_fixed_budget(), ss_aipe_crd_es_n_clusters_fixed_width(), ss_aipe_crd_es_n_individuals_fixed_budget(), ss_aipe_crd_es_n_individuals_fixed_width() | es, icc_Y, icc_Z, R2_between, R2_within, num_predictors, nrep, clus_cost, indiv_cost, multicore, num_proc | true_es, true_icc, true_icc_covariate, true_R2_between, true_R2_within, n_covariates, G, cluster_cost, individual_cost, ncpus (multicore = TRUE, num_proc = 4 is ncpus = 4) |
| | ss_aipe_crd_n_clusters_fixed_budget(), ss_aipe_crd_n_clusters_fixed_width(), ss_aipe_crd_es_n_clusters_fixed_budget(), ss_aipe_crd_es_n_clusters_fixed_width() | n_individuals | cluster_size |
| indirect (mediated) effect | ss_aipe_indirect_effect() | a, b, n_max | true_std_reg_coef_a, true_std_reg_coef_b, max_N |
| | ss_aipe_indirect_effect_sensitivity() | true_a, true_b, estimated_a, estimated_b | true_std_reg_coef_a, true_std_reg_coef_b, supposed_std_reg_coef_a, supposed_std_reg_coef_b |
| | ss_power_indirect_effect() | a, b, c_prime, N | true_std_reg_coef_a, true_std_reg_coef_b, true_std_reg_coef_direct, specified_N |
| | var_indirect_effect() | a, b, var_a, var_b, cov_ab | reg_coef_a, reg_coef_b, var_reg_coef_a, var_reg_coef_b, cov_reg_coef_a_b (population values or estimates, raw or standardized, so neither true_ nor std_) |
| structural equation models | ss_aipe_composite_sem() | Sigma, pop_model, mu, desired_width, N | true_cov_matrix, true_model, true_mean, width, specified_N |
| | ss_power_composite_sem() | Sigma, pop_model, mu, N | true_cov_matrix, true_model, true_mean, specified_N |
| | ss_aipe_sem_path() | Sigma, desired_width | true_cov_matrix, width |
| | ss_aipe_sem_path_sensitivity() | est_Sigma, true_Sigma, desired_width, N | supposed_cov_matrix, true_cov_matrix, width, specified_N |
| | ss_aipe_rmsea() | RMSEA | true_rmsea |
| | ss_aipe_rmsea_sensitivity() | Sigma, N | true_cov_matrix, specified_N |
| | ss_power_sem() | RMSEA_null, RMSEA_true, F_ML, RMSEA_full, RMSEA_res, F_full, F_res, df_res | null_rmsea, true_rmsea, true_F_ML, true_rmsea_full, true_rmsea_restricted, true_F_full, true_F_restricted, df_restricted |
| reliability | ss_aipe_reliability() | S, i, cor_est, lambda, psi_square, initial_iter, final_iter, start_ss | true_cov_matrix, n_items, true_item_covariance, true_factor_loading, true_residual_var, G_initial, G, start_N |
| | ss_aipe_reliability_sensitivity() | estimated_reliability, i | supposed_reliability, n_items |
| measurement models and item response theory | dmacs() | lambda_reference, lambda_focal, nu_reference, nu_focal | factor_loading_reference, factor_loading_focal, intercept_reference, intercept_focal |
| | simple_structure() | Lambda | factor_loading_matrix |
| | irt_information() | a, b, theta | discrimination, threshold, trait_value (item and grm keep their names, and so do the columns a and b of an irt_grm() table given as grm) |
| standardized mean | ss_aipe_sm() | sm | true_sm |
| | ss_aipe_sm_sensitivity() | estimated_sm, desired_width | supposed_sm, width |
| Cliff's delta | ss_aipe_cliff_delta() | delta | true_delta |
| | ss_aipe_cliff_delta_sensitivity() | estimated_delta | supposed_delta |
| Cohen's h | cohen_h() | p1, p2 | p_1, p_2 |
| consequences of a design | design_consequences() | w | width |
| one sample and paired data | bayes_one_sample_t(), bayes_paired_t() | n | N |
| | ci_proportion() | n | N |
| the null value of a mean or a mean difference | bayes_one_sample_t(), ci_mahalanobis() | mu_0 | null_mean |
| | welch_t(), summary_t_test() | mu | null_mean_diff |
| eigenvalue of a covariance matrix | ci_eigenvalue() | n, k | N, which_eigenvalue |
| expected cross-validation index | ecvi() | n | multiplier |
| critical values and simultaneous intervals | dbryant_paulson(), pbryant_paulson(), qbryant_paulson() | num_covariates, num_groups | n_covariates, n_groups |
| | cv_bryant_paulson() | covariates, groups | n_covariates, n_groups |
| | cv_tukey_hsd() | groups | n_groups |
| | ci_c_ancova_bp() | n, num_covariates | n_per_group, n_covariates |
| congruence of factor loadings | procrustes_phi() | n_perm | n_resamples |
| data generators | simulate_anova_data() | mu, sigma, n | true_mean, true_sd, n_per_group |
| | simulate_ancova_data() | mu_y, mu_x, sigma_y, sigma_x, rho, n | true_mean_Y, true_mean_X, true_sd_Y, true_sd_X, true_cor, n_per_group |
| | simulate_ancova_factorial_data() | mu_y, mu_x, sigma_y, sigma_x, rho_y_x, rho_x_x, n | true_mean_Y, true_mean_X, true_sd_Y, true_sd_X, true_cor_YX, true_cor_XX, n_per_cell |
| | simulate_regression_data() | rho_YX, rho_XX, mu_Y, mu_X, sigma_Y, sigma_X | true_cor_YX, true_cor_XX, true_mean_Y, true_mean_X, true_sd_Y, true_sd_X |
| | simulate_longitudinal_polynomial(), simulate_longitudinal_logistic(), simulate_longitudinal_gompertz(), simulate_longitudinal_negative_exponential(), simulate_longitudinal_richards() | n, occasions | n_per_group, n_occasions |
R matches an argument name by its leading characters, and a function that
passes ... to another program does not refuse a name itself. The table
lists 602 old names, a name counted once for each function that had it.
Of these, 536 stop with "unused argument". These 66 do not:
w = in ss_aipe_R2_sensitivity() and design_consequences()
(matched to width); df_res = in ss_power_sem()
(df_restricted); n = in ci_c(), ci_c_ancova(),
ci_sc_ancova(), equivalence_c(), cohen_f(),
is_orthogonal_set(), variance_components_mls(),
simulate_anova_data(), simulate_ancova_data(), ss_seq_c(),
mr_smd(), and power_equivalence_c() (matched to n_per_group, the
per-group size it always was; in the five power planners that took n
and take specified_n_per_group, ss_power_c(), ss_power_sc(),
ss_power_c_ancova(), ss_power_split_plot_anova(), and
ss_power_composite_ancova_2group(), n = now stops with "unused
argument"); and n = in power_equivalence_md(),
power_density_equivalence_md(), and power_equivalence_md_plot()
(matched to n_per_treatment, the same quantity, now that the degrees
of freedom are df_error and no other argument begins with n: a
1.0.x call that gave the degrees of freedom by position runs, and one
that named nu stops for nu). Write the full name: the match is R's,
it is not a name these functions have, and it ends the day one of them
gains another argument that begins with the same letters.es = in the six ss_aipe_crd_es_*() functions is matched to
es_type when es_type is left out, and the call stops where the effect
size is needed, with a message about es_type or about the missing
effect size; a fixed-budget call that does not ask for the expected width
never used the effect size and still runs. a = in
ss_power_indirect_effect() is matched to alpha_level, so a 1.0.x call
stops for the unused b. n = in ecvi() is matched to npar when
npar is left out, and the call stops for the missing multiplier; a
call that also names npar stops with "unused argument". f = in
ss_power_factorial_anova() and ss_power_factorial_ancova() is
matched to factor_levels when the levels are given by position, and
the call stops because a value of Cohen's f is not a vector of
levels; a call that names factor_levels stops with "unused argument".w = in
ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() stops with "argument matches multiple formal
arguments" (which_predictor, width), as does n = in
ci_c_ancova_bp() (n_per_group, n_covariates), in the five
simulate_longitudinal_*() functions (n_per_group, n_occasions), in
ci_sc() (n_per_group, ncp), and in
simulate_ancova_factorial_data() (n_per_cell, n_covariates).
d = in ss_power_mixed_effects() stops with the same message when
desired_power and directional are both left out; when one of them is
named, d is matched to the other and the call stops for the missing
true_smd....; the function reads the name and stops.
n = in var_icc() would be matched to n_raters, and in
var_alpha() to n_items, so a call that named only the sample size
and gave the count by position, var_alpha(.80, n = 100, 10) or
var_icc(.60, n = 30, 4), would run with the two values exchanged:
100 items and 10 respondents, or 30 raters and 4 subjects. n = in
bayes_one_sample_t() would be matched to null_mean, so a call that
named the sample size n would take it as the comparison value for the
mean. The three functions read the names in the call and refuse n
with a condition of class dmar_argument_invalid that says to write
N = . A 1.0.x call of var_icc() or var_alpha() that named every
argument also named k or p_items (and rho in var_icc()) and stops
with "unused argument"; n = in bayes_paired_t(), which has no
argument that begins with n, stops with "unused argument" too.
Four names would reach a ...: t_value = in ci_nc_t(), which
was a second name for ncp, would pass to the root finder; mu_0 =
in ci_mahalanobis() would reach the root search of ci_nc_F();
psi = in ci_sc() would reach ci_nc_t() and stop inside its root
finder; and psi = in ci_c_ancova() would be dropped without a word
under procedure = "t" and passed to ci_c_ancova_bp() under
"bryant_paulson". Each function refuses the name with the same class
and says to write ncp = , null_mean = , or contrast = .... to lavaan, beside the new
arguments; the function reads the name and stops. Sigma = ,
pop_model = , mu = , desired_width = , and N = in
ss_aipe_composite_sem(); Sigma = , pop_model = , mu = , and
N = in ss_power_composite_sem(); Sigma = and N = in
ss_aipe_rmsea_sensitivity(); Sigma = and desired_width = in
ss_aipe_sem_path(); and est_Sigma = , true_Sigma = ,
desired_width = , and N = in ss_aipe_sem_path_sensitivity()
(true_Sigma was that function's 1.0.x name for the generating matrix,
now true_cov_matrix). Each of the five functions checks the names in
... against the arguments and options of lavaan before anything is
fitted, and a name that is neither is refused with a condition of class
dmar_argument_invalid that gives the name as it was typed and points
to the function's Renamed arguments section.... for its root search; the call
stops, but not with "unused argument". chi_square = in
ci_nc_chisq() goes to ..., and the call stops at once because
chisq is missing (class dmar_argument_invalid). sigma_squared =
in ci_srsnr() with means stops the same way because var_within is
missing; with F_value, it reaches the noncentral F search, which
stops with "unused argument (sigma_squared = ...)" inside a condition
of class dmar_computation_failed.ci_cv(), ss_aipe_c(), ss_aipe_cv(), ss_aipe_R2(), ss_aipe_sc(),
ss_aipe_sc_ancova(), ss_aipe_sc_ancova_sensitivity(), ss_aipe_sm(),
and ss_power_R2() took ... and used it for nothing, so a name they did
not have was dropped without a word and the call ran on the default of the
argument that was meant: ss_power_R2(true_R2 = .3, p = 3, N = 50)
planned for the default power and returned 36. They no longer take ...,
and any name they do not have, old or misspelled, is an "unused argument".
The two counts of the simulation that ss_aipe_R2(verify_ss = TRUE) runs,
which were read from ... as g and G, are the arguments G_initial
(the replications for each candidate N of the coarse search) and G
(the replications of the final pass); g and G differed only in case.
ci_c_ancova_bp() and the three Bryant-Paulson distribution functions
took a ... they did not use and no longer take it, for the same reason:
ci_c_ancova_bp(num_covariates = 2) would otherwise have been computed
for the default of one covariate without a word. So did ci_c(),
ci_reg_coef(), plot_randomization_test(), and R2_mixed_effects(),
and ci_rc() and ci_src(), which passed theirs to ci_reg_coef(),
where nothing read it; none of the six takes ... now. No exported
function is left with a ... that its body does not use.
covmat_from_cfa() read only tol_det from its ..., so a misspelled
name, or a third value given by position, was dropped without a word;
tol_det is now an argument of its own, after factor_loading and
residual_var, and the function takes no ....
In ss_power_r() the name N is gone and no other argument took it:
ss_power_r(true_cor = .3, N = 50) stops, and the call is
ss_power_r(true_cor = .3, specified_N = 50). The null value was rho_0
and is null_cor, as the other null values are null_R2 and
null_rmsea; the 1.0.x planning value rho then matches nothing and
stops.
ss_aipe_c() takes the error variance one way, as true_var_error,
the name in ss_aipe_c_sensitivity() and ss_power_contrast(). MSwithin
was a second name for it and SD was its square root; both are gone, with
the three checks that the values agreed, so a 1.0.x call that gave
SD = s gives true_var_error = s^2. A population contrast is
true_contrast in every planner and an observed one contrast; the
standardized contrast is true_std_contrast, where the two standardized
contrast sensitivity functions called it true_psi, the 1.0.x name of
the unstandardized contrast elsewhere.
In the planners for the analysis of variance, Cohen's f is a planning
value and is true_f, beside true_eta_squared and
true_eta_squared_partial, in ss_power_one_way_anova(),
ss_power_rm_anova(), ss_power_split_plot_anova(),
ss_power_factorial_anova(), and ss_power_factorial_ancova(); the row of
ss_power_contrast() that holds the f of the contrast is true_f too
(effect_size_f). Cohen's f squared is true_f2 in ss_power_R2(),
ss_power_R2_change(), ss_power_reg_coef(), and ss_power_rc(), the
argument and the row. The
correlation among the repeated measures and the sphericity value of
ss_power_rm_anova() and ss_power_split_plot_anova() are true_cor and
true_epsilon, and the proportion of within-cell variance the covariates
explain in ss_power_factorial_ancova() is true_covariate_R2. The five
planners report the noncentrality parameter in the row
noncentral_f_parm, the name in ss_power_R2(), and the degrees of
freedom in df_effect and df_error, the names in
ss_aipe_omega_squared() and ci_eta_squared(). The number of subjects of
ss_power_rm_anova(), each measured on every occasion, is the total
sample size, so it is specified_N when given and the row necessary_N
when planned.
cohen_f(), ci_srsnr(), and covmat_from_cfa() are calculators, not
planners: each computes the same quantity from population values or from
sample summaries and estimates, so their arguments carry plain quantity
names without true_. cohen_f() takes means, var_within,
n_per_group, eta_squared, sd_means, and sd_within (mu,
sigma_squared, n, sigma_m, sigma); given population values it
returns the population f, and given sample summaries it returns the
plug-in value from the same calculation, which is not an unbiased
estimate of f. ci_srsnr() takes the observed group means,
var_within (the pooled within-group mean square; sigma_squared), and
n_per_group as an alternative to F_value; with population values in
their place, the interval is a hypothetical one, not an interval from
observed data and not the expected interval of a planned study.
covmat_from_cfa() takes the factor loadings and residual variances,
population values or estimates from a fitted model, as factor_loading
and residual_var (lambda, psi_squared), and returns its matrix as
the element model_implied_cov (population_cov): the matrix is
unchanged, and the new name says what it is whether the loadings and
residual variances are population values or estimates.
covmat_from_cfa(lambda = l, psi_squared = p)$population_cov is
covmat_from_cfa(factor_loading = l, residual_var = p)$model_implied_cov.
The planners for composite power in the analysis of variance and
covariance (ss_power_composite_anova(),
ss_power_composite_factorial_anova(), ss_power_composite_ancova(),
ss_power_composite_factorial_ancova(),
ss_power_composite_factorial_ancova_het(), and
ss_power_composite_ancova_2group()) name their population values as the
single-effect planners do: true_means, true_sd, true_cor (the
covariate correlations, correlations), true_covariate_R2, and
true_sd_X, the standard
deviation of the covariate (sd_cov), which is the name the two ANCOVA
contrast sensitivity functions give the same quantity. The fields of
each element of effects are renamed too: Cohen's f is true_f and
partial eta squared is true_eta_squared_partial (f,
partial_eta_squared), so list(factors = 1, f = 0.25) becomes
list(factors = 1, true_f = 0.25). A field is not an argument and R does
not check it, so the planners now do: an element of effects with a
field they do not read is refused (see Calls That Are Now Refused). The
rows follow: true_f_<label>, noncentral_f_parm_<label>, and
df_error, the names the single-effect planners use. The total sample
size beside the per-cell or per-group answer is the row total_N in both
branches, the rule for an implied total everywhere else in the package
(it was necessary_N beside a planned size and specified_N beside a
size that was given); when the correlations differ in absolute value and
the size was planned, it is approximate_total_N beside
approximate_n_per_cell or approximate_n_per_group (approximate_N).
The attributes the plot() method reads from the tables of
ss_power_composite_factorial_ancova_het() and of
ss_power_composite_ancova(slopes = "heterogeneous") (means,
correlations, sigma, sd_cov) are renamed with the arguments:
true_means, true_cor, true_sd, and true_sd_X.
ss_power_c_ancova() takes true_sd and specified_n_per_group
(sigma, n), as ss_power_c() does.
A size per group is n_per_group and a size per cell is n_per_cell,
in the estimators as in the planners: ci_c(), ci_sc(), ci_c_ancova(),
ci_sc_ancova(), equivalence_c(), cohen_f(), is_orthogonal_set(),
variance_components_mls(), ss_seq_c(), mr_smd(),
simulate_anova_data(), and simulate_ancova_data() took n and take
n_per_group (one number for equal groups, or one per group where the
function allowed it); ss_power_c(), ss_power_sc(), and
ss_power_split_plot_anova(), where the size is given so that the power
at that size is returned, took n and take specified_n_per_group;
simulate_ancova_factorial_data() takes n_per_cell. ci_c() and
ci_sc() had n and N, a per-group size and the total, apart only by
the case of one letter. In power_equivalence_md(),
power_density_equivalence_md(), and power_equivalence_md_plot() the
size is n_per_treatment: the number of subjects who receive each
treatment, which is the size of each group in a parallel design and every
subject in a crossover, the n of the standard error sigma * sqrt(2 / n) in
both. The three functions take the equivalence bounds as lower_bound
and upper_bound, the true difference or ratio as true_effect, and the
error degrees of freedom as df_error (ltheta1, ltheta2, and ldiff
in power_equivalence_md(), theta1, theta2, and diff in the other
two, and nu in all three). The grid attached to the plot keeps its
column labels (n=24); its first column, diff, is true_effect.
power_equivalence_c() takes n_per_group for independent groups; the
number of units of a paired design, new in this release, is N (see
Equivalence and Noninferiority of a Contrast).
The contrast planners name their population values as the sensitivity
functions do. The error standard deviation is true_sd in
ss_power_c(), ss_power_equivalence_c(), and power_equivalence_c()
(sigma), as in ss_seq_c_sensitivity(); ss_power_contrast() takes
true_means and true_var_error (mu, sigma_squared);
ss_aipe_sc_ancova() takes true_sd_anova, true_sd_ancova, and
true_sd_ratio (ratio alone is an allocation ratio elsewhere); and
the values the two ANCOVA contrast sensitivity functions generate their
data from are true_mean_Y, true_sd_Y, true_mean_X, and
true_sd_X. The data generators for the analysis of variance and
covariance and for regression name their population values the same
way: simulate_anova_data() takes true_mean and true_sd (mu,
sigma); simulate_ancova_data() takes true_mean_Y, true_mean_X,
true_sd_Y, true_sd_X, and true_cor (mu_y, mu_x, sigma_y,
sigma_x, rho), and simulate_ancova_factorial_data() the same with
true_cor_YX and true_cor_XX (rho_y_x, rho_x_x); and
simulate_regression_data() takes true_cor_YX, true_cor_XX,
true_mean_Y, true_mean_X, true_sd_Y, and true_sd_X (rho_YX,
rho_XX, mu_Y, mu_X, sigma_Y, sigma_X). mr_smd() and mr_cv()
keep the symbols of the published procedure (A, epsilon, gamma,
m0); their rows now name the sizes they hold: n_1 and n_2 and
pilot_n_per_group in mr_smd(), N and pilot_N in mr_cv().
var_omega_squared() takes the value its formula is evaluated at as
true_omega_squared (population_omega_squared), the name in
ss_aipe_omega_squared(). In the six ss_aipe_crd_es_*() functions
write true_es = , and in ss_power_mixed_effects() true_smd = ; what
R does with es = and d = is under Old Names That R Still Matches.
ss_power_pcm() and ss_aipe_pcm() give the variance of the
individuals' change coefficients and the level one error variance the
same names, true_var_trend and true_var_error (tau and
level_1_variance in ss_power_pcm(), variance_trend and
error_variance in ss_aipe_pcm()), and ss_aipe_pcm_sensitivity()
plans from supposed_var_trend and supposed_var_error. ss_aipe_pcm()
takes the sampling-error variance of one individual's estimated change
coefficient (Kelley & Rausch, 2011, Equation 19) as
true_var_trend_error (variance_true_minus_estimated_trend), and
ss_power_pcm() reports the same quantity in the row of that name
(error_var_of_slopes). ss_power_pcm() took the group difference in the
change coefficient as beta and a flag, standardized, that said
whether beta was divided by the between-subject standard deviation of
the change coefficient. It takes the difference under the name of its
scale: true_std_reg_coef for the standardized difference (beta under
the 1.0.x default) and true_reg_coef for the difference in the units of
the change coefficient (beta with standardized = FALSE). Exactly one
is given; both or neither is refused with a condition of class
dmar_argument_invalid. ss_power_pcm(beta = -.4, tau = .003, level_1_variance = .0262, frequency = 1, duration = 4, N = 238) is
ss_power_pcm(true_std_reg_coef = -.4, true_var_trend = .003, true_var_error = .0262, frequency = 1, duration = 4, specified_n_per_group = 119). The two come first among the arguments, so
a 1.0.x call that gave beta, tau, and level_1_variance by position
gives both and is refused rather than read on the other scale. The table
reports the difference on both scales, in the rows true_reg_coef and
true_std_reg_coef (unstd_coefficient, std_coefficient), whichever
was given, and the power is unchanged. The help of ss_power_pcm() keeps
the symbols of Raudenbush and Liu (2001) in its formulas and says which
argument each one is.
In the planners for an indirect effect each path is a standardized
regression coefficient of the population the plan is made for, so the
paths are true_std_reg_coef_a, true_std_reg_coef_b, and
true_std_reg_coef_direct, and the product of the two paths is the row
true_std_indirect_effect. The power of the indirect effect in
ss_power_indirect_effect() is the row power_ab (actual_power),
followed by power_a and power_b, and tidy() and glance() read it
as the power of the design. var_indirect_effect() computes the same
variances from population values or from estimates, raw or standardized,
so its arguments carry neither true_ nor std_; mediate() keeps its
rows a and b, which are estimates.
In the planners for structural equation models a population covariance
matrix is true_cov_matrix and the matrix a sensitivity study is planned
from is supposed_cov_matrix; the rows of ss_power_composite_sem() that
echo the population value of each parameter are true_<label>, as in
ss_aipe_composite_sem(). In ss_power_sem() df and df_full keep
their names.
In the functions for one sample and for paired data, the size of the one
sample, the number of pairs, and the number of trials are each the total
sample size, so bayes_one_sample_t(), bayes_paired_t(),
ci_proportion(), and ci_eigenvalue() take N; a call that gave the
size by position (ci_proportion(17, 50)) is unchanged. The eigenvalue
that ci_eigenvalue() brackets was chosen with k and is chosen with
which_eigenvalue, as which_predictor and which_path choose theirs. The
per-group sizes of bayes_independent_t(), ci_ratio_of_means(), and
design_consequences() keep n_1 and n_2. In ss_aipe_cliff_delta()
ratio keeps its name (the allocation ratio is a constant of the design,
not a planning value).
A null value is null_ followed by the quantity. bayes_one_sample_t()
and ci_mahalanobis() take the comparison value of the mean as
null_mean (mu_0), and welch_t() and summary_t_test() take the
null value of the difference between the means as null_mean_diff
(mu), beside null_cor in ss_power_r(), null_R2 in
ss_power_R2(), and null_rmsea in ss_power_sem().
The proportion of intervals no wider than the desired width is the row
pct_ci_less_width in every sensitivity function and in
design_consequences() (pct_ci_less_w; every other name in the package
spells the width out), and the rows of ss_aipe_composite_sem() that
hold the same proportion for each parameter are
pct_ci_less_width_<label> (width_within_desired_<label>).
A count is n_<things>. The number of groups is n_groups in
cv_tukey_hsd(), cv_bryant_paulson(), and the three Bryant-Paulson
distribution functions (groups, num_groups), and the number of random
covariates is n_covariates there and in ci_c_ancova_bp() (covariates,
num_covariates); covariates in ancova() and mediate() names
columns and keeps its name. The per-group size of ci_c_ancova_bp() is
n_per_group. The number of random permutations of procrustes_phi() is
n_resamples, the name in randomization_test(),
randomization_test_paired(), and vargha_delaney_A(). The five
simulate_longitudinal_*() functions take n_per_group, the number of
units in each population (one number, or one per population; never a
total), and n_occasions, the number of measurement occasions per unit.
In ecvi() the divisor is multiplier: it is the multiplier with which
the chi square was formed from the fit function, the total sample size
under the normal likelihood and the total less the number of groups under
the Wishart likelihood, so n named it for one case only.
The number of items is n_items, the number of raters n_raters, the
number of clusters n_clusters, and the number of level one units in a
cluster cluster_size, in every function that takes or reports one.
ss_aipe_reliability() and ss_aipe_reliability_sensitivity() took the
number of items as i and var_alpha() as p_items; the row of
reliability(), reliability_alpha(), reliability_omega(),
reliability_omega_categorical(), and reliability_kr20() that holds it
was J and is n_items, the name glance() already gave it, and the row
of ss_aipe_reliability_sensitivity() was items. ss_aipe_icc(),
ss_aipe_icc_sensitivity(), and var_icc() took the number of raters as
k; the help keeps k in the formulas and in the names of the forms
("ICC(1,k)"). ss_power_mixed_effects() took n, rho, and J and
takes cluster_size, true_icc, and specified_n_clusters_per_arm: the number of
clusters in each arm, not the total n_clusters, so its rows are
necessary_n_clusters_per_arm and specified_n_clusters_per_arm, and its
noncentrality parameter is in the row noncentral_t_parm, as in
ss_power_c() and ss_power_smd(). cluster_size has no default and is
checked first.
An intraclass correlation that a planner takes as a planning value is
true_icc: in ss_aipe_icc() (rho) and ss_aipe_icc_sensitivity()
(true_rho; a correlation is true_cor everywhere else, and the value
the sensitivity study is planned from is supposed_icc, which was
estimated_rho), in ss_power_mixed_effects()
(rho), in the twelve ss_aipe_crd_*() functions (icc_Y), and in
ss_aipe_mixed_effects() (icc). var_icc() takes the
intraclass correlation its variance is evaluated at as icc, as in
design_effect().
The total variance of the outcome in a two-level plan is true_total_var:
in ss_aipe_mixed_effects() (sigma2_y) and
ss_aipe_mixed_effects_sensitivity() (true_sigma2_y, with
supposed_total_var for estimated_sigma2_y), and in the
ss_aipe_crd_*() functions (total_var), where true_var_within is the
within-cluster variance alone. Two names that differed by one capital no longer hold two
quantities. The sensitivity function also plans from supposed_var_X and
supposed_icc (estimated_sigma2_x, estimated_icc) and takes the
population slope as true_reg_coef (true_beta), whose simulated
summaries are the rows mean_reg_coef, median_reg_coef, and
sd_reg_coef.
The twelve planners for a cluster randomized design name every population
value with true_: true_var_between, true_var_within, true_total_var,
true_icc, true_R2_between, and true_R2_within, and in the six for
the standardized effect size true_icc_covariate (icc_Z) beside
true_es. The number of covariates beside the treatment indicator is
n_covariates (num_predictors; the value is unchanged, and the total
number of predictors is p = n_covariates + 1). The number of individuals
in a cluster is the argument cluster_size of the four functions that
take it (n_individuals); the functions that solve for it report it in
the row necessary_cluster_size, beside necessary_n_clusters in the
two that solve for both, and a size that was given is not repeated in
the table. The function names keep n_individuals. The four functions
that plan from a width echo it in the row width, as the other AIPE
planners do, and the width column of tidy() and glance() is that
desired width for every AIPE planner (NA for a plan made from a budget
and for a planner that does not echo the request); the expected or
assured width is the glance() column of that name. The number of
processes of the six ss_aipe_crd_es_*() functions is ncpus
(multicore, num_proc), the name in the bootstrap functions. The two costs are cluster_cost
and individual_cost, and the number of simulated data sets of the six
ss_aipe_crd_es_*() functions is G, as in every other function that
simulates (nrep). The row that reports the width no longer carries the
effect or the assurance in its name: it is expected_width without an
assurance and assured_width with one, and the assurance is in a row of
its own, assurance. In 1.0.x the row was, for example,
width_of_unstd_conditions_diff_with_0.8_assurance, a name that changed
with the value asked for.
ss_aipe_reliability() names its population as a planner does:
true_cov_matrix (S, which is a sample covariance matrix in the
estimators), true_factor_loading (lambda), true_residual_var
(psi_square), and true_item_covariance (cor_est:
the help defines it as the common covariance of every pair of items, which
is their correlation only for standardized items, and it is a planning
value, not an estimate). The replications of the two stages of its
assurance search are G_initial and G (initial_iter, final_iter),
the names in ss_aipe_R2(), and an assurance plan reports the rows
assurance, assurance_attained, and G (specified_assurance,
empirical_assurance, final_iter), the names in ss_aipe_partial_r()
and ss_aipe_semipartial_r(). The estimators (reliability_alpha() and
the rest) keep S, data, and items. The reliability
ss_aipe_reliability_sensitivity() plans from is supposed_reliability
(estimated_reliability): a planning assumption, not an estimate from
data.
The measurement model and item response functions name their parameters
for what they are (see the table). irt_information() reports the grid
in the column trait_value and carries the attributes discrimination,
threshold, and trait_value_max_information; the columns a and b of
an irt_grm() table keep their names, and the help of irt_grm() and
irt_information() says what they are. A call of irt_information() that
gives the values by position is unaffected.
Every population value a planner for a regression coefficient takes
carries true_ and is named for its quantity: the coefficient is
true_reg_coef_j in the units of the data and true_std_reg_coef_j when
it is standardized (DMAR 1.0.x used b_j for the first and beta_j for
both), a squared multiple correlation is true_R2_ followed by its
variables, and the standard deviations and correlations are true_sd_Y,
true_sd_X, true_cor_XX, and true_cor_YX. The estimators name what
they are given the same way, without true_ (reg_coef_j,
se_reg_coef_j, sd_Y, sd_X, std_reg_coef_j).
The regression coefficient sensitivity functions lost standardize: the
name of the function says the scale. ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and ss_power_reg_coef_sensitivity() report
the unstandardized coefficient, as their 1.0.x default did, and
ss_aipe_src_sensitivity() reports the standardized one, as its 1.0.x
default did. A call that gave standardize stops with "unused argument".
The default of noncentral is TRUE in ss_aipe_src_sensitivity() and
FALSE in the other two, as in DMAR 1.0.x, so the call that replaces
ss_aipe_reg_coef_sensitivity(standardize = TRUE) or
ss_aipe_rc_sensitivity(standardize = TRUE) is
ss_aipe_src_sensitivity(noncentral = FALSE), and the call that replaces
ss_aipe_src_sensitivity(standardize = FALSE, noncentral = FALSE) is
ss_aipe_rc_sensitivity(). ss_aipe_src_sensitivity(standardize = FALSE), with its default noncentral = TRUE, becomes
ss_aipe_rc_sensitivity(noncentral = TRUE), which is refused when the
sample size is planned from supposed variances that are not 1 (see Calls
That Are Now Refused). With
standardize = TRUE, ss_power_reg_coef_sensitivity() gave the same
power (the t statistic does not depend on the scale), so such a call
drops the argument.
One name has one meaning, and the bounds of an equivalence region are
named for the quantity they bound: smd_lower and smd_upper,
contrast_lower and contrast_upper, cor_lower and cor_upper, and
prop_diff_lower and prop_diff_upper in
equivalence_proportion_paired(), which is new; plot_equivalence(),
which draws whatever estimates it is given, takes effect_lower and
effect_upper. cles(), nnt_from_smd(), and
proportion_of_superiority() used smd_lower and smd_upper for
confidence limits on d, as arguments and as rows; those are now
smd_ci_lower and smd_ci_upper, the names DMAR gives the limits of an
interval named for its estimate, so cles(smd = .5, smd_lower = .2, smd_upper = .8) stops with "unused argument". convert_delta_lambda()
takes smd, and convert_lambda_delta() and convert_lambda_R2() take
ncp; the function names keep their symbols.
A row or column that echoes a renamed argument is renamed with it, a row
that summarizes simulated estimates is named for the quantity
(mean_reg_coef_j, mean_contrast, mean_std_contrast), and the
columns of the per-replication files that the sensitivity functions write
with filename follow the rows they summarize. Code that looks a row or
column up by its 1.0.x name must be changed, because an old name matches
nothing and gives no error: x$value[x$term == "rho"] returns
numeric(0). The columns of eta_squared(), omega_squared(),
ci_eta_squared(), and ci_omega_squared() are under One Function for
Each ANOVA Effect Size.
The limits of every confidence interval are ci_lower and ci_upper.
DMAR 1.0.x named them lower_limit and upper_limit in most functions
and ci_lower and ci_upper in the coefficient tables and the tidy()
results. One pair is now used everywhere, as a row of a long table, a
column of a wide one, and a suffix of a limit named for its estimate
(rmsea_ci_lower, cles_ci_lower). x$value[x$term == "lower_limit"]
becomes x$value[x$term == "ci_lower"], and x$lower_limit becomes
x$ci_lower.
| function | DMAR 1.0.x | now |
|----------|------------|-----|
| every function that reports a confidence interval | rows or columns lower_limit, upper_limit | ci_lower, ci_upper |
| every function whose limits are named for their estimate | <estimate>_lower_limit, <estimate>_upper_limit (for example cles_lower_limit, smd_upper_limit, shift_lower_limit, loa_upper_lower_limit, mean_lower_limit) or <estimate>_lower, <estimate>_upper (for example tau2_lower, indirect_upper, sigma2_b_lower, eta_squared_partial_upper) | <estimate>_ci_lower, <estimate>_ci_upper (cles_ci_lower, smd_ci_upper, shift_ci_lower, loa_upper_ci_lower, mean_ci_lower, tau2_ci_lower, indirect_effect_ci_upper, sigma2_between_ci_lower, eta_squared_partial_ci_upper) |
| combine_p() | row k | row n_studies |
| welch_t(), summary_t_test() | row t_statistic | row t_value |
| pairwise_within(), ci_dunnett(), ci_dunnett_t3() | column t_statistic | column t_value |
| contrast_test() | column t | column t_value |
| contrast_adjusted(), bayes_one_sample_t(), bayes_paired_t(), bayes_independent_t() | row t | row t_value |
| ci_scheffe() | columns contrast_value, F_statistic | columns estimate, F_value |
| ci_c_ancova() | row psi | row contrast, as in ci_c() |
| ci_sc_ancova() | row psi | row std_contrast, as in ci_sc() |
| equivalence_c() | rows psi_hat, delta_lower, delta_upper | rows contrast, contrast_lower, contrast_upper (the lower bound is still signed); plot_equivalence() reads them |
| ci_rc() | rows lower_limit, upper_limit only | rows ci_lower, reg_coef, ci_upper: the coefficient sits between its limits, as in ci_reg_coef() and ci_src() |
| var_r() | row var_r_bonett_wright | removed with kurtosis_x and kurtosis_y (see Calls That Are Now Refused) |
| ss_aipe_partial_r(), ss_aipe_semipartial_r(), ss_aipe_partial_r_sensitivity(), ss_aipe_semipartial_r_sensitivity() | row J | row p, holding p |
| expected_partial_r() | columns n, J, rho | columns N, p, true_cor |
| expected_r() | columns n, rho | columns N, true_cor |
| equivalence_r() | rows n, rho_lower, rho_upper | rows N, cor_lower, cor_upper (the lower one still holds the negative bound) |
| ss_aipe_r(), ss_power_r(), ss_aipe_partial_r() | row rho | row true_cor |
| ss_power_r() | row rho_0 | row null_cor |
| ss_aipe_r_sensitivity(), ss_aipe_partial_r_sensitivity() | rows true_rho, estimated_rho | rows true_cor, supposed_cor |
| ss_aipe_semipartial_r() | row r_sp | row true_r_sp |
| ss_aipe_semipartial_r_sensitivity() | row estimated_r_sp | row supposed_r_sp (beside the new rows true_R2_full and supposed_R2_full) |
| ss_aipe_equivalence_r() | row population_r | row true_cor |
| ss_aipe_equivalence_r_sensitivity() | rows true_r, estimated_r, rho_lower, rho_upper | rows true_cor, supposed_cor, cor_lower, cor_upper |
| ss_aipe_R2_sensitivity(), ss_power_R2_sensitivity() | row estimated_R2 | row supposed_R2 |
| ss_power_R2_sensitivity() | rows mean_F, median_F, sd_F | rows mean_F_value, median_F_value, sd_F_value |
| expected_R2() | row expected_value_population_R2 | row expected_R2, as var_R2() reports var_R2 (the value is unchanged) |
| ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity() | rows mean_b_j, median_b_j, sd_b_j, true_b_j, estimated_b_j | rows mean_reg_coef_j, median_reg_coef_j, sd_reg_coef_j, true_reg_coef_j, supposed_reg_coef_j |
| ss_aipe_src_sensitivity() | rows mean_b_j, median_b_j, sd_b_j, true_b_j, estimated_b_j | rows mean_std_reg_coef_j, median_std_reg_coef_j, sd_std_reg_coef_j, true_std_reg_coef_j, supposed_std_reg_coef_j (the standardized coefficient, as true_std_reg_coef_j in ss_aipe_src()) |
| ss_power_reg_coef_sensitivity() | rows mean_b_j, median_b_j, sd_b_j, mean_se_b_j, median_se_b_j, sd_se_b_j, true_b_j, estimated_b_j, mean_t, median_t, sd_t | rows mean_reg_coef_j, median_reg_coef_j, sd_reg_coef_j, mean_se_reg_coef_j, median_se_reg_coef_j, sd_se_reg_coef_j, true_reg_coef_j, supposed_reg_coef_j, mean_t_value, median_t_value, sd_t_value |
| ci_cv() | rows c_of_v, c_of_v_unbiased | rows cv, cv_unbiased |
| ss_aipe_smd(), ss_power_smd(), ss_power_composite_ancova_2group() | row supposed_smd | row true_smd |
| ss_power_composite_ancova_2group() | rows supposed_rho_group_1, supposed_rho_group_2 | rows true_cor_group_1, true_cor_group_2 |
| ss_aipe_equivalence_smd() | row population_smd | row true_smd |
| ss_aipe_smd_sensitivity() | rows true_delta, estimated_delta | rows true_smd, supposed_smd |
| ss_aipe_equivalence_smd_sensitivity() | rows estimated_smd, delta_lower, delta_upper | rows supposed_smd, smd_lower, smd_upper |
| equivalence_smd() | rows delta_lower, delta_upper | rows smd_lower (still the signed lower bound), smd_upper |
| expected_smd() | column delta | column true_smd |
| cles() | rows cl, cl_lower, cl_upper, smd_lower, smd_upper | rows cles, cles_ci_lower, cles_ci_upper, smd_ci_lower, smd_ci_upper |
| nnt_from_smd() | rows nnt_lower, nnt_upper, smd_lower, smd_upper | rows nnt_ci_lower, nnt_ci_upper, smd_ci_lower, smd_ci_upper |
| proportion_of_superiority() | rows smd_lower, smd_upper | rows smd_ci_lower, smd_ci_upper |
| ss_aipe_sc_sensitivity(), ss_aipe_sc_ancova_sensitivity() | rows mean_psi, median_psi, sd_psi, true_psi, estimated_psi | rows mean_std_contrast, median_std_contrast, sd_std_contrast, true_std_contrast, supposed_std_contrast |
| ss_aipe_sc_ancova_sensitivity() | row rho | row true_cor |
| ss_aipe_c_sensitivity() | rows mean_psi, median_psi, sd_psi, true_psi, true_error_variance, estimated_error_variance | rows mean_contrast, median_contrast, sd_contrast, true_contrast, true_var_error, supposed_var_error |
| ss_aipe_c_ancova_sensitivity() | rows mean_psi, median_psi, sd_psi, true_psi, est_error_var_ancova, rho | rows mean_contrast, median_contrast, sd_contrast, true_contrast, supposed_var_error_ancova, true_cor |
| ss_seq_c_sensitivity() | row true_psi | row true_contrast |
| ss_aipe_omega_squared() | row population_omega_squared | row true_omega_squared |
| ss_aipe_omega_squared_sensitivity() | row estimated_omega_squared | row supposed_omega_squared |
| ancova() | rows eta_squared_partial_lower, eta_squared_partial_upper, omega_squared_partial_lower, omega_squared_partial_upper | rows eta_squared_partial_ci_lower, eta_squared_partial_ci_upper, omega_squared_partial_ci_lower, omega_squared_partial_ci_upper |
| factorial_anova() | columns eta_squared_partial_lower, eta_squared_partial_upper, omega_squared_partial_lower, omega_squared_partial_upper | columns eta_squared_partial_ci_lower, eta_squared_partial_ci_upper, omega_squared_partial_ci_lower, omega_squared_partial_ci_upper |
| ss_power_rm_anova() | rows necessary_n_subjects, specified_n_subjects | rows necessary_N, specified_N |
| ss_power_one_way_anova(), ss_power_rm_anova(), ss_power_split_plot_anova(), ss_power_factorial_anova(), ss_power_factorial_ancova() | row noncentrality | row noncentral_f_parm |
| ss_power_rm_anova(), ss_power_split_plot_anova(), ss_power_factorial_anova() | rows effect_df, error_df | rows df_effect, df_error |
| ss_power_factorial_ancova() | rows f, f_adjusted, covariate_R2 | rows true_f, true_f_adjusted, true_covariate_R2 |
| ss_power_contrast() | row effect_size_f | row true_f |
| variance_components_mls() | rows sigma2_b_lower, sigma2_b_upper, sigma2_w_lower, sigma2_w_upper, icc_lower, icc_upper | rows sigma2_between_ci_lower, sigma2_between_ci_upper, sigma2_within_ci_lower, sigma2_within_ci_upper, icc_ci_lower, icc_ci_upper |
| mr_smd() | rows n1, n2, pilot_ss | rows n_1, n_2, pilot_n_per_group |
| mr_cv() | rows n, pilot_ss | rows N, pilot_N |
| ss_aipe_icc() | rows rho, k | rows true_icc, n_raters |
| ss_aipe_icc_sensitivity() | rows true_rho, estimated_rho, k | rows true_icc, supposed_icc, n_raters |
| ss_power_mixed_effects() | rows necessary_J_per_arm, specified_J_per_arm, noncentrality | rows necessary_n_clusters_per_arm, specified_n_clusters_per_arm, noncentral_t_parm |
| the twelve ss_aipe_crd_*() and ss_aipe_crd_es_*() functions | row exp_width_of_unstd_conditions_diff or exp_width_of_<level>_effect_size | row expected_width |
| the twelve ss_aipe_crd_*() and ss_aipe_crd_es_*() functions | row width_of_unstd_conditions_diff_with_<assurance>_assurance or width_of_<level>_effect_size_with_<assurance>_assurance | rows assured_width and assurance |
| the eight ss_aipe_crd_*() and ss_aipe_crd_es_*() functions that solve for the cluster size (_n_individuals_ and _both_) | row cluster_size | row necessary_cluster_size |
| ss_aipe_reliability() | rows specified_assurance, empirical_assurance, final_iter | rows assurance, assurance_attained, G |
| ss_aipe_reliability_sensitivity() | rows items, estimated_reliability | rows n_items, supposed_reliability |
| reliability(), reliability_alpha(), reliability_omega(), reliability_omega_categorical(), reliability_kr20() | row J | row n_items |
| ss_aipe_mixed_effects() | row icc | row true_icc |
| ss_aipe_mixed_effects_sensitivity() | rows true_sigma2_y, true_sigma2_x, estimated_sigma2_y, estimated_sigma2_x, estimated_icc, true_beta, mean_beta, median_beta, sd_beta; column beta of the file written with filename | rows true_total_var, true_var_X, supposed_total_var, supposed_var_X, supposed_icc, true_reg_coef, mean_reg_coef, median_reg_coef, sd_reg_coef; column reg_coef |
| ss_power_pcm() | rows unstd_coefficient, std_coefficient, true_var_of_slopes, l1_error_var, error_var_of_slopes | rows true_reg_coef, true_std_reg_coef, true_var_trend, true_var_error, true_var_trend_error |
| ss_power_pcm() | row freq | row frequency |
| ss_power_pcm() | row measurement_occasions | row n_occasions |
| ss_aipe_pcm_sensitivity() | rows true_variance_trend, true_error_variance, estimated_variance_trend, estimated_error_variance | rows true_var_trend, true_var_error, supposed_var_trend, supposed_var_error |
| ss_aipe_pcm_sensitivity() | row n_timepoints | row n_occasions |
| ss_aipe_indirect_effect() | rows a, b, ab | rows true_std_reg_coef_a, true_std_reg_coef_b, true_std_indirect_effect |
| ss_power_indirect_effect() | rows a, b, c_prime, indirect_effect, actual_power | rows true_std_reg_coef_a, true_std_reg_coef_b, true_std_reg_coef_direct, true_std_indirect_effect, power_ab (the power to detect the indirect effect, followed by power_a and power_b as before; tidy() and glance() read it as the power of the design) |
| ss_aipe_indirect_effect_sensitivity() | rows mean_ab, median_ab, sd_ab, true_a, true_b, true_ab, estimated_a, estimated_b; column ab of the file written with filename | rows mean_std_indirect_effect, median_std_indirect_effect, sd_std_indirect_effect, true_std_reg_coef_a, true_std_reg_coef_b, true_std_indirect_effect, supposed_std_reg_coef_a, supposed_std_reg_coef_b; column std_indirect_effect |
| mediate() | rows indirect_lower, indirect_upper, direct_lower, direct_upper, total_lower, total_upper | rows indirect_effect_ci_lower, indirect_effect_ci_upper, direct_effect_ci_lower, direct_effect_ci_upper, total_effect_ci_lower, total_effect_ci_upper (the estimates are the rows indirect_effect, direct_effect, and total_effect) |
| ss_aipe_composite_sem() | rows desired_width_<label>, population_<label>, width_within_desired_<label> | rows width_<label>, true_<label>, pct_ci_less_width_<label> |
| ss_power_composite_sem() | rows population_<label> | rows true_<label> |
| ss_power_composite_anova(), ss_power_composite_factorial_anova(), ss_power_composite_ancova(), ss_power_composite_factorial_ancova(), ss_power_composite_factorial_ancova_het() | rows f_<label>, noncentral_parm_<label>, residual_df | rows true_f_<label>, noncentral_f_parm_<label>, df_error |
| the same five planners | rows necessary_N, specified_N (beside the per-cell size); approximate_N | row total_N; approximate_total_N |
| ss_power_composite_ancova(), ss_power_composite_factorial_ancova(), ss_power_composite_factorial_anova() | row covariate_R2 | row true_covariate_R2 |
| ss_power_composite_factorial_ancova_het(), and ss_power_composite_ancova(slopes = "heterogeneous"), which returns its table | attributes means, correlations, sigma, sd_cov | attributes true_means, true_cor, true_sd, true_sd_X (read by the plot() method) |
| ss_power_composite_ancova_2group() | rows sigma, sd_cov, residual_df | rows true_sd, true_sd_X, df_error |
| ss_power_composite_ancova_2group() | rows necessary_N, specified_N (beside the per-group size); approximate_N | row total_N; approximate_total_N |
| ss_aipe_cliff_delta() | row delta | row true_delta |
| ss_aipe_cliff_delta_sensitivity() | row estimated_delta | row supposed_delta |
| ss_aipe_cv_sensitivity() | row estimated_cv | row supposed_cv |
| ss_aipe_sm_sensitivity() | row estimated_sm | row supposed_sm |
| design_consequences() | row target_width | row width |
| ss_aipe_r(), ss_aipe_partial_r(), ss_aipe_semipartial_r(), ss_aipe_icc(), ss_aipe_omega_squared(), ss_aipe_indirect_effect(), ss_aipe_cliff_delta() | row width_target | row width |
| ss_aipe_equivalence_r(), ss_aipe_equivalence_smd(), ss_aipe_mixed_effects() | row ci_width_expected | row expected_width |
| ss_power_R2() | row effect_size | row true_f2 |
| ss_power_reg_coef(), ss_power_rc() | row effect_size | row true_f (Cohen's f, the square root of true_f2) |
| meta_es(), meta_r(), meta_smd(), meta_contrast() | row k | row n_studies |
| meta_es(), meta_r(), meta_smd() | rows tau2_lower, tau2_upper, I2_lower, I2_upper | rows tau2_ci_lower, tau2_ci_upper, I2_ci_lower, I2_ci_upper (prediction_lower and prediction_upper, the limits of the prediction interval, keep their names) |
| meta_es(), meta_r(), meta_smd() | row t under the Hartung-Knapp adjustment, z without it | row test_statistic under either; the reference distribution is in the attributes test_distribution ("t" or "normal") and test_df (n_studies - 1, or NA), and a line under the printed table states it |
| meta_contrast() | row z | row test_statistic (the attribute test_distribution is "normal" and test_df is NA) |
| procrustes_phi() | row n_perm | row n_resamples |
| design_consequences() and the 25 ss_aipe_*_sensitivity() functions of DMAR 1.0.x | row pct_ci_less_w | row pct_ci_less_width |
| design_effect() | row n_total | row N |
| obrien_test() | column n_total | column N |
| contrast_adjusted() | row p | row p_value |
| anova_within_two_way(), simple_effects_AB() | column partial_eta_squared | column eta_squared_partial |
| anova_within() | attribute partial_eta_squared | attribute eta_squared_partial |
| R2_mixed_effects() (ci_method = "boot") | rows R2_marginal_lower, R2_marginal_upper, R2_conditional_lower, R2_conditional_upper | rows R2_marginal_ci_lower, R2_marginal_ci_upper, R2_conditional_ci_lower, R2_conditional_ci_upper (its tidy() still gives ci_lower and ci_upper columns) |
| bayes_one_sample_t(), bayes_paired_t() | row n | row N |
| ci_proportion() | row n | row N |
| cohen_kappa() | column n | column N |
| vargha_delaney_A() | column z_value | column test_statistic |
| mlmr(), mlmr_mv() (the coef_table); cfa_k(), cfa_1(), cfa_2() (the parameter table) | column z_value | column test_statistic |
| dmacs() | columns lambda_reference, lambda_focal, nu_reference, nu_focal (and the same columns of tidy()) | columns factor_loading_reference, factor_loading_focal, intercept_reference, intercept_focal |
| irt_information() | column theta; attributes a, b, theta_max_information | column trait_value; attributes discrimination, threshold, trait_value_max_information |
| covmat_from_cfa() | element population_cov of the returned list | element model_implied_cov |
| power_equivalence_md_plot() | column diff of the power_grid attribute (and of the plotted data) | column true_effect (ratios when logscale = TRUE, as before; the n=<size> columns keep their labels) |
| reliability_H() from loadings | rows reliability_H, lower_limit, upper_limit, var_H | the reliability family table: rows estimate, se, ci_lower, ci_upper, conf_level, N, N_complete, n_items (var_H is the square of se) |
| ss_aipe_rmsea_sensitivity(), ss_aipe_sem_path_sensitivity() | row suc_rep | removed; attr(x, "simulation")$returned |
| ss_aipe_R2_sensitivity() | row num_probs_with_cis | removed; attr(x, "simulation")$failed |
| ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity() | columns b_j, ll_ci_beta_j, ul_ci_beta_j, r_2, se_b_j, t_for_b_j of the per-replication file written with filename | columns reg_coef_j, ci_lower, ci_upper, R2, se_reg_coef_j, t_value |
| ss_aipe_src_sensitivity() | the same six columns of its file | columns std_reg_coef_j, ci_lower, ci_upper, R2, se_std_reg_coef_j, t_value |
| ss_power_reg_coef_sensitivity() | columns b_j, se_b_j, t_stat, r_2 of its file | columns reg_coef_j, se_reg_coef_j, t_value, R2 |
| ss_power_R2_sensitivity() | columns r2, f_stat of its file | columns R2, F_value |
| ss_aipe_R2_sensitivity() | columns r2, lower_width_ci, upper_width_ci, width_ci of its file | columns R2, ci_width_lower, ci_width_upper, ci_width, the names of the rows that summarize them |
| ss_aipe_c_sensitivity() | column psi_hat of its file | column contrast |
| ss_aipe_c_ancova_sensitivity() | columns psi_obs, se_psi, se_psi_restricted of its file | columns contrast, se_contrast, se_contrast_restricted |
| ss_aipe_sc_sensitivity(), ss_aipe_sc_ancova_sensitivity() | column psi_obs of the file | column std_contrast |
| ss_aipe_rmsea_sensitivity(), ss_aipe_sem_path_sensitivity() | columns ci_low, ci_up of the file | columns ci_lower, ci_upper; a file begun by DMAR 1.0.x keeps its header when a run appends to it |
| var_cv() | rows var_cv_mckay, var_cv_vangel | row var_cv_normal, which holds the value that var_cv_mckay held; var_cv_vangel is removed (see Changes to Earlier Results) |
The reasons for each name are in One Notation for the Regression and Correlation Functions, below.
The removed rows suc_rep and num_probs_with_cis held counts of
replications, which every sensitivity function now reports one way, in
the simulation attribute of its table (see Changes to Earlier
Results): suc_rep was the number of replications that gave an
interval, and num_probs_with_cis the number that did not.
reliability_H() returns the same table for every kind of input; the
table of DMAR 1.0.0 is not returned for any call, and there is no
warning period (see Coefficient H From a Fitted Model).
The column that holds a test statistic is test_statistic wherever the
reference distribution depends on an option: under test = "t" in
vargha_delaney_A(), and under a finite wald_df in the multilevel and
confirmatory factor analysis tables, the statistic is referred to
Student's t, so z_value named it wrongly. The meta-analysis
functions named their statistic t under hartung_knapp = TRUE and z
without it (z in meta_contrast()); it is test_statistic in all
four, with the reference distribution in the attributes
test_distribution and test_df and in a line under the printed table,
and the table of meta_contrast() now has the class dmar_meta_contrast,
a dmar_tbl. Otherwise a
column whose name is always true keeps it (z_value in cohen_kappa()
and fleiss_kappa(), t_value in welch_t() and its relatives).
The common language effect size is the row cles wherever it is
reported: cles() named it cl, as randomization_test() did not, and
results_sentence() on a cles() table begins "cles =" where it began
"cl =".
ss_power_R2_change() is new in this release, so its rows true_f2 and
true_partial_R2 have no 1.0.x names; they are named as planning values
like every other row that holds one. The same holds for the other
functions new in this release: ss_aipe_two_phase(),
ss_aipe_two_phase_cost(), and ss_aipe_two_phase_sensitivity() echo
true_cor, and the sensitivity function supposed_sd_Y and
supposed_cor; equivalence_proportion_paired() reports its bounds in
the rows prop_diff_lower and prop_diff_upper;
ss_aipe_mean_sensitivity() echoes supposed_sd;
ss_aipe_crossed_sensitivity() echoes supposed_var_unit,
supposed_var_rater, and supposed_var_residual; and
ss_aipe_mixed_effects_sensitivity() echoes supposed_icc_X.
var_r() no longer takes kurtosis_x and kurtosis_y. DMAR 1.0.x
returned a row var_r_bonett_wright, the normal-theory variance times
1 + rho^2 (kurtosis_x + kurtosis_y) / 4, and credited it to Bonett and
Wright (2000). That article has no such formula, no other source for
it was found, and the help's statement that it is exact for elliptical
distributions was wrong: there the variance is the normal-theory
variance times one factor that does not depend on the correlation. The
large-sample variance under non-normality depends on every fourth-order
cumulant of the joint distribution (Duncan & Layard, 1973, Equation
1.1), which two marginal kurtoses do not supply. A call that gives
either argument stops with "unused argument".
desired_power must be strictly between 0 and 1 in every power
planner. ss_power_R2() accepted 0 and 1 (and returned a sample size
for a power of 1); ss_power_pcm() made no check. Every ss_power_*
function now applies one rule and gives one refusal, of class
dmar_desired_power_invalid, that carries the value given.
The unbiased standardized mean difference on one degree of freedom is
refused. smd(unbiased = TRUE) with n_1 + n_2 = 3 and
smd_c(unbiased = TRUE) with n_C = 2 returned 0, because the
correction factor of Hedges (1981) is 0 at one degree of freedom
whatever the data. They now stop with a refusal of class
dmar_hedges_correction_undefined that says why and what to do
(unbiased = FALSE, or more observations). On no degrees of freedom
(smd_c() from summary statistics with n_C = 1, which returned Inf,
or NaN when the means were equal; smd() with one standard deviation
and n_1 + n_2 = 2) the refusal has the same class and states its own
reason: the correction factor is not defined there, because there is no
standard deviation.
desired_power is checked when a sample size is given too. A power
planner that is given a sample size computes the power and does not use
desired_power; a value outside (0, 1) given with it was accepted
without a word (ss_power_r(true_cor = .3, specified_N = 60, desired_power = 1) returned a table) and is now refused with
dmar_desired_power_invalid.
smd() from summary statistics refuses a group of one that has its
own standard deviation. With s_1 and s_2, n_1 = 1 and n_2 = 2
returned a number and two groups of one returned NaN, where raw data
with a group of one were already refused. Each size must be at least 2;
the refusal has class dmar_argument_invalid.
conf_level, alpha_level, width, and assurance are checked on
entry, one way. conf_level and alpha_level are each a single
number strictly between 0 and 1; width is a single positive finite
number; assurance is NULL or a single number strictly between 0 and
NA, a logical, a character string, a vector of two) reached the
computation in many functions. Fifteen planners searched without end
for a width of 0 or below (ss_aipe_sm(true_sm = 10, width = -1),
ss_aipe_R2(true_R2 = .5, p = 5, width = -1)); others returned a
number for it (ss_aipe_rmsea() 20,000, ss_aipe_reliability() Inf);
ci_c() returned limits of Inf and -Inf for conf_level = -1; and
ss_power_reg_coef() did not return for alpha_level = 0. Each is now
refused at the first line of the function with a condition of class
dmar_argument_invalid (dmar_assurance_invalid for assurance) that
names the argument, carries the value given, and says what to give.
The check is at the head of the 166 functions in which a degenerate
value was found to run or to stop with R's own message; a function
with a narrower range of its own (a width below 2 for a correlation, a
level below .5 for an equivalence test, an assurance above .5) applies
it after. An argument with no default is checked when it is given.A number that is missing, empty, or not a number is refused by
name. An argument that holds one number and was given as NA,
NaN, a character string, a list, or a vector of length 0 stopped
inside the computation with R's message ("missing value where
TRUE/FALSE needed", "non-numeric argument to binary operator",
"argument is of length zero"), which names neither the argument nor
the function. In 79 functions those arguments are now checked on
entry: ci_smd(smd = .5, n_1 = NA, n_2 = 10) stops with "'n_1' must be
a number that is not missing; NA was given", class
dmar_argument_invalid. No call that returned a result is affected.
The interval functions check conf_level before the tail areas are
derived from it. ci_pvaf(), ci_snr(), ci_srsnr(),
ci_eta_squared(), ci_omega_squared(), and ci_sc() refused
conf_level = 1.2 with a message about alpha_lower, an argument the
call did not give. The refusal now
names conf_level. ss_aipe_R2(), ss_aipe_R2_sensitivity(), and
plot_R2() gave an empty message for conf_level = NULL; they give
the same refusal.
ci_c() refuses one tail area given without the other.
ci_c(means = c(1, 0), s_anova = 1, c_weights = c(1, -1), n_per_group = c(20, 20), conf_level = NULL, alpha_upper = .025) (n in 1.0.0)
stopped in DMAR 1.0.0 and the development versions of 1.1.0 with R's
own message, "Non-numeric argument to mathematical function", and with
only alpha_lower given, "arguments imply differing number of rows:
3, 2". It is now refused with class dmar_argument_invalid, arg
c("alpha_lower", "alpha_upper"): "Only 'alpha_upper' was given.
'alpha_lower' and 'alpha_upper' go together: each is the tail area
beyond one limit, so the interval needs both. Give both (for example
alpha_lower = .025 and alpha_upper = .025, or 0 for a one-sided
interval), or neither and a 'conf_level'."
ss_power_c() and ss_power_sc() refuse a contrast of 0 when a
sample size is to be found. A population contrast of 0 is the null
hypothesis; the power is alpha_level at every sample size. The two
functions searched a million sample sizes and stopped with "Failed to
converge within reasonable sample size." They now stop at once with a
refusal of class dmar_argument_invalid that says so. With a sample
size given they return the power at 0, as before. The nine power
planners that had that message for a search that reaches its limit
(ss_power_c(), ss_power_sc(), ss_power_c_ancova(),
ss_power_smd(), ss_power_one_way_anova(),
ss_power_factorial_anova(), ss_power_rm_anova(),
ss_power_split_plot_anova(), ss_power_mixed_effects()) now say what
it means (the effect at the planning values is too small for the power
asked for at any size up to 1,000,000) and what to check; the class is
dmar_computation_failed, as before.
ss_aipe_R2_sensitivity() refuses supposed_R2 together with
specified_N. It ran the simulation at the sample size planned from
the supposed value and ignored specified_N (a call with specified_N = 6
reported total_N 34). The coefficient of variation, Cliff's delta, and
reliability siblings refused the pair already; all four refusals now
have class dmar_argument_invalid.
ss_aipe_sc_ancova_sensitivity() asks for width when it plans.
With supposed_std_contrast and no width the call stopped inside
the planner with "argument is of length zero"; it now stops with a
refusal of class dmar_argument_invalid that names width.
ss_aipe_reliability() checks its two counts of replications.
G_initial and G (initial_iter and final_iter in 1.0.x) each must
be a single whole number of at least 1; a fractional number, 0, a
negative number, NA, or more than one value reached the search
unchecked and is now an error of class dmar_count_invalid, as for
every other G.
ss_power_mixed_effects() refuses a missing or inadmissible
cluster_size, true_icc, or specified_n_clusters_per_arm by class. Each
refusal has class dmar_argument_invalid, carries the name of the
argument, and says what the value must be and why. A cluster size or a
number of clusters that is not finite was not refused in 1.0.x.
An element of effects with a field the composite planners do not
read is refused. Each element of effects in
ss_power_composite_anova(), ss_power_composite_factorial_anova(),
ss_power_composite_ancova(), ss_power_composite_factorial_ancova(),
and ss_power_composite_factorial_ancova_het() is a list of factors,
true_f or true_eta_squared_partial, and label (and type under
heterogeneous slopes). A field under any other name, the f and
partial_eta_squared of 1.0.x among them, or an element with no name,
was not looked at: with the sizes read from true_means it was dropped
without a word. It is now an error of class dmar_argument_invalid
that names the effect and the field and gives the fields that are read.
ss_power_c_ancova() refuses an n_per_group that is not a single
finite number of at least 2 by class (dmar_argument_invalid). A size
below 2 was a plain error in 1.0.x, and a size that is not finite was
not refused.
An argument a function does not have is an error in 20 more
functions. See the paragraph on ... under Old Names That R Still
Matches.
bifactor_indices() refuses a fit with one group factor. A general
factor beside a single group factor is not a bifactor model with
subscales: its indices divide the common variance between those two
factors alone, every item belongs to the one subscale, and PUC is 0.
DMAR 1.0.0 refused only a fit with no group factor and described a fit
with one when every item was listed under that factor (some of them
with loadings fixed at zero). Such a fit now stops with a message that
names the group factor and says what to fit instead, as it does in
fa_bifactor() and ci_bifactor_indices(), which are new in this
release.
The six ss_aipe_crd_es_*() planners refuse a width asked for
without the effect size. The width of the interval is read from data
simulated at true_es. Without true_es the four planners that take
or return a width stopped with a message that did not name the cause
(ss_aipe_crd_es_n_individuals_fixed_width() said that the width
seemed impossible to reach with the number of clusters); they now stop
at once with a refusal of class dmar_argument_invalid that says why
the effect size is needed and what to give. In
ss_aipe_crd_es_n_clusters_fixed_budget() and
ss_aipe_crd_es_n_individuals_fixed_budget() the expected width is
asked for by supplying G: with G and without true_es the call
stopped with "arguments imply differing number of rows: 660, 0", and
with G and without pr_treat or true_icc it returned the table
without the width and without a word.
Both are now refused with the same class, in a message that names every
population value that is missing and says that G = NULL returns the
size the budget buys without a width. A true_es that is not a single
finite number is refused the same way.
With one covariate (n_covariates = 1) the same two functions had no
default for true_R2_within and true_R2_between, where the four
planners that take a width have 0, and did not use true_icc for a
missing true_icc_covariate as the help says: a call with G and a
covariate and without all three stopped with "missing value where
TRUE/FALSE needed" or "arguments imply differing number of rows". The
two proportions now default to 0 in all six functions, a missing
true_icc_covariate is true_icc, and a value of any of the three
that is not a single number in range is refused with class
dmar_argument_invalid, by name.
The twelve cluster randomized planners (ss_aipe_crd_*() and
ss_aipe_crd_es_*()) check their sizes, proportions, and allocation
on entry. n_covariates must be a whole number of 0 or more,
cluster_size a whole number of at least 2 (the cluster size searches
start at 2), n_clusters a whole number of at least 3 + n_covariates (the interval has n_clusters - 2 - n_covariates
degrees of freedom), pr_treat strictly between 0 and 1, and
true_R2_between and true_R2_within in [0, 1), as the two
standardized budget planners already required of a covariate. Each
refusal has class dmar_argument_invalid and names the argument as
arg. DMAR 1.0.0 and the development versions of 1.1.0 planned with
whatever they were given (the calls are written with the 1.1.0 names;
1.0.0 gave the same results under its own):
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8) returned
17.5 clusters with n_covariates = .5, 5 with true_R2_between = 1.1, and 19 with true_R2_within = -.5. Fractional sizes were
planned for and are now refused: cluster_size = 20.5 gave 62
clusters at width = .5, and
ss_aipe_crd_n_individuals_fixed_width(width = .5, n_clusters = 60.5, ...) a cluster size of 23; so is cluster_size = 1, which gave 249
clusters. n_clusters = 2 stopped with "missing value where
TRUE/FALSE needed" (ss_aipe_crd_n_individuals_fixed_width()) or
returned an expected width of NaN
(ss_aipe_crd_n_individuals_fixed_budget()), and pr_treat = 0 or
1 searched without end. A fixed number of clusters that leaves a
condition without a cluster is refused, wherever a width is computed,
with reason "arm_without_clusters" and the fewest clusters that
would do (fewest): ss_aipe_crd_n_individuals_fixed_budget(budget = 1000, n_clusters = 5, pr_treat = .1, ...) returned a cluster size of
200 with an expected width of Inf, since round(5 * .1) = 0 clusters
are treated, and is refused with fewest = 6.
ss_aipe_crd_es_n_individuals_fixed_budget() makes the check only
when G asks for the width, since without G it does not use
pr_treat: with cluster_cost = 0, individual_cost = 1, G = 2, true_icc = .2, true_es = .5 the same design returned a cluster size
of 200 with an expected width of NaN in DMAR 1.0.0 and stopped with
class dmar_computation_failed after simulating it in the development
versions, and without G it still returns 200.
Costs and budgets in the cluster randomized planners must be single
finite numbers of 0 or more, and costs of 0 are refused where they
decide the design. Costs that are both 0 are refused (reason
"costs_all_zero") in the budget planners and the both planners,
and individual_cost = 0 (reason "individual_cost_zero") where
the costs choose the cluster size:
ss_aipe_crd_n_individuals_fixed_budget(),
ss_aipe_crd_both_fixed_budget(), ss_aipe_crd_both_fixed_width(),
and their ss_aipe_crd_es_*() counterparts. Each class is
dmar_argument_invalid, and each message says what to give. A search
over designs that all cost the same has no end. In DMAR 1.0.0 and the
development versions of 1.1.0 all six budget planners searched without
end with both costs 0 (none returned within 4 seconds), among them
ss_aipe_crd_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, cluster_cost = 0, individual_cost = 0); so did, with cluster_cost = 10 and individual_cost = 0,
ss_aipe_crd_n_individuals_fixed_budget(budget = 1000, n_clusters = 20, ...), ss_aipe_crd_es_n_individuals_fixed_budget() with the same
arguments, ss_aipe_crd_both_fixed_budget(budget = 10000, pr_treat = .5, true_var_between = .2, true_var_within = .8, ...), and
ss_aipe_crd_es_both_fixed_budget(budget = 10000, true_es = .5, true_icc = .2, pr_treat = .5, G = 2, seed = 113, ...), which with a
budget of 1000 stopped instead with "'from' must be a finite number".
With the same costs ss_aipe_crd_both_fixed_width(width = .5, ...)
returned a table of NA with a character value column, and
ss_aipe_crd_es_both_fixed_width() stopped with "missing value where
TRUE/FALSE needed". Clusters of a fixed size cost only their clusters,
so ss_aipe_crd_n_clusters_fixed_budget() with individual_cost = 0
and a positive cluster_cost still plans (100 clusters for a budget
of 1000 at cluster_cost = 10). In the four planners that plan from a
width and a fixed size the costs only price the plan, and costs of 0
still give a budget of 0. A negative cost, a vector of costs, and a
negative budget were used, and are refused:
ss_aipe_crd_n_clusters_fixed_width(width = .5, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8, cluster_cost = -1, individual_cost = 1) reported a budget of 1178 (62
clusters at 19 each), ss_aipe_crd_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, individual_cost = c(1, 2)) stopped with
"arguments imply differing number of rows", and
ss_aipe_crd_n_clusters_fixed_budget(budget = -5, cluster_size = 20)
returned NA, in DMAR 1.0.0 and the development versions.
A budget that does not buy the smallest design a cluster randomized
planner can return is refused with the cost of that design. The
refusal has class dmar_argument_invalid, arg "budget", reason
"budget_infeasible", and min_cost, and the budget it names buys a
design the planner returns:
ss_aipe_crd_n_clusters_fixed_budget(budget = 1, cluster_size = 10, cluster_cost = 10, individual_cost = 1) (4 clusters of 10 cost 80)
and ss_aipe_crd_n_individuals_fixed_budget(budget = 1, n_clusters = 10, cluster_cost = 10, individual_cost = 1) (10 clusters of 2 cost
120) returned a table whose size and budget were NA in DMAR 1.0.0
and the development versions of 1.1.0, as did their
ss_aipe_crd_es_*() counterparts;
ss_aipe_crd_both_fixed_budget(budget = 1, cluster_cost = 10, individual_cost = 1, pr_treat = .5, ...) and
ss_aipe_crd_es_both_fixed_budget() with the same budget stopped with
"invalid 'type' (list) of argument" (4 clusters of 2 cost 48). When
the width is asked for (pr_treat, or G in the standardized
planner), the smallest design ss_aipe_crd_n_clusters_fixed_budget()
can return has a cluster in each condition and a degree of freedom:
ss_aipe_crd_n_clusters_fixed_budget(budget = 60, cluster_size = 20, cluster_cost = 0, individual_cost = 1, pr_treat = .1, true_var_between = .2, true_var_within = .8) returned a table of NA in DMAR 1.0.0
and the development versions, and is refused with the cost of 6
clusters, 120 (with n_covariates = 2 and pr_treat = .5, 5
clusters, 100); ss_aipe_crd_es_n_clusters_fixed_budget() with the
same budget and G = 2, true_icc = .2, true_es = .5 stopped with
"NA/NaN argument" in both.
A budget that buys clusters of a fixed size with no finite width is
refused when pr_treat asks for the width.
ss_aipe_crd_n_clusters_fixed_budget(budget = 100, cluster_size = 20, cluster_cost = 0, individual_cost = 1, pr_treat = .1, true_var_between = .2, true_var_within = .8) returned 5 clusters, none of them
treated, with an expected_width of Inf in DMAR 1.0.0 and the
development versions of 1.1.0. It is refused (class
dmar_argument_invalid, reason "arm_without_clusters") with the
budget, the design it buys, and the cheapest design that puts a
cluster in each condition (6 clusters, cost 120); a design with no
degrees of freedom (4 clusters with n_covariates = 2) is refused
with reason "too_few_clusters".
ss_aipe_crd_es_n_clusters_fixed_budget() makes the same check before
it simulates a width.
The six unstandardized cluster randomized planners refuse a
negative, missing, or contradictory variance before searching. Any
two of true_var_between, true_var_within, true_total_var, and
true_icc determine the model. Each one given must be a single finite
number (reason "variance_input_invalid"); a variance must be 0 or
more, the total positive, whether it is given or is the sum of the two
variances, and the intraclass correlation in [0, 1] (reason
"variance_out_of_range"); fewer than two inputs, or a between
variance of 0 with an intraclass correlation of 0, is refused with
reason "underidentified_variances"; and a third or fourth input
that disagrees with the two that determine the model by more than 64
machine epsilons of its scale is refused with reason
"inconsistent_variances", with the value they imply in the message.
With diff_size a within-cluster variance of 0 is refused (reason
"within_variance_zero"), where the budget planners reported a width
of NaN and the others stopped with an error from R. Each refusal has
class dmar_argument_invalid, carries the four inputs as given in
supplied, and says what to give. In DMAR 1.0.0 and the development
versions of 1.1.0 ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = -.2, true_var_within = .8) stopped with "missing value where TRUE/FALSE
needed" after ten warnings, as did true_total_var = 1, true_icc = NA
without the warnings; true_var_between = 0, true_var_within = 0
returned 3 clusters with an expected width of 0; and true_total_var = 1, true_icc = .2, true_var_between = .5 planned 36 clusters with a
between variance of .5 and a within variance of .8, where the pair
(total 1, ICC .2) gives 18; it is now refused with "true_total_var = 1
and true_icc = 0.2 imply true_var_between = 0.2, but 0.5 was given". A
rounded redundant value is refused the same way: true_var_between = 1, true_var_within = 2, true_icc = .333, which DMAR 1.0.0 and the
development versions planned from the two variances (71 clusters),
says that the variances imply true_icc = 0.333333333333333; give two
of the four.
G above 999,999 is refused by the six ss_aipe_crd_es_*()
planners. Each simulated data set takes its own seed, drawn without
replacement from 1 to 999,999.
ss_aipe_crd_es_n_clusters_fixed_budget(budget = 1000, cluster_size = 10, cluster_cost = 0, individual_cost = 1, G = 1000000, pr_treat = .5, true_icc = .2, true_es = .5) stopped with "cannot take a sample
larger than the population when 'replace = FALSE'" in DMAR 1.0.0
(nrep) and the development versions of 1.1.0; it is refused on entry
with class dmar_count_invalid, max 999999, and a message that says
why. The draw of seeds is unchanged, so every G that was accepted
gives the same seeds and the same result.
The cluster randomized planners check assurance, and the
standardized ones width, conf_level, and the covariate's
intraclass correlation, on entry.
ss_aipe_crd_es_n_clusters_fixed_width() and
ss_aipe_crd_es_both_fixed_width() check width, conf_level, and
assurance as the other planners do:
ss_aipe_crd_es_n_clusters_fixed_width(width = -1, cluster_size = 20, true_es = .5, true_icc = .2, pr_treat = .5, G = 2) searched without
end in DMAR 1.0.0 and the development versions of 1.1.0.
ss_aipe_crd_n_clusters_fixed_budget(),
ss_aipe_crd_es_n_clusters_fixed_budget(), and
ss_aipe_crd_es_n_individuals_fixed_budget() refuse an assurance
that is not NULL or a single number strictly between 0 and 1 (class
dmar_assurance_invalid), whether or not a width is computed, as the
other planners do. In DMAR 1.0.0 and the development versions
ss_aipe_crd_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, cluster_cost = 10, individual_cost = 1, pr_treat = .5, true_var_between = .2, true_var_within = .8, assurance = 1.5)
returned 33 clusters with an assured width of NaN, and without the
population values and with assurance = 2 33 clusters and a budget of
990; the two standardized planners with G = 2 and assurance = 2
stopped with "'probs' outside [0,1]", and without G returned the
size the budget buys (50 clusters of 10 at cluster_cost = 10). The
four standardized planners that take a width or search over both sizes
check true_icc_covariate (a single number in [0, 1]) on entry, as
the two budget planners already did with G:
ss_aipe_crd_es_n_clusters_fixed_width(width = 1, cluster_size = 20, true_es = .5, true_icc = .2, pr_treat = .5, n_covariates = 1, true_icc_covariate = 1.5, true_R2_within = .1, true_R2_between = .1, G = 2, seed = 113) stopped with "'data' must be of a vector type, was
'NULL'" in DMAR 1.0.0 and, after simulating three designs, with class
dmar_computation_failed (reason "no_finite_width") in the
development versions; it is refused with class dmar_argument_invalid
and arg "true_icc_covariate".
The cluster randomized planners refuse a diff_size, true_icc, or
es_type that makes the width not a number, and a target that no
design of up to 1,000,000 clusters meets. diff_size must be NULL
or a vector of one or more finite numbers, in all twelve planners.
Wherever a width is simulated, the six ss_aipe_crd_es_*() planners
check es_type (0, 1, or 2) and true_icc, a single number in [0,
1), and above 0 with es_type = 2: at 1 no variance lies within
clusters and the model fitted to each simulated data set has none to
estimate, and with es_type = 2 an intraclass correlation of 0 leaves
the effect without a scale. Each refusal has class
dmar_argument_invalid. In DMAR 1.0.0 and the development versions of
1.1.0 ss_aipe_crd_es_n_clusters_fixed_width(width = 1, cluster_size = 20, true_es = .5, true_icc = 1, pr_treat = .5, G = 2, seed = 113)
stopped with "missing value where TRUE/FALSE needed", as it did with
true_icc = 1.5, true_icc = -.2, and true_icc = 0, es_type = 2;
with true_icc = 1, es_type = 0 it stopped with that message in
1.0.0, and the development versions simulated three designs at which
every fit failed and stopped with class dmar_computation_failed.
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8) stopped
with "missing value where TRUE/FALSE needed" with diff_size = numeric(0), c(Inf, 0), or NA, and with "non-numeric argument to
mathematical function" with diff_size = "a". The searches over the
number of clusters consider at most 1,000,000 clusters, the bound the
power planners use for a sample size, and a target that no design
within it meets is refused with arg "width", reason
"n_clusters_beyond_reach", and the width at the bound as
width_at_max: in DMAR 1.0.0 and the development versions the same
call searched without end with width = 1e-300 and returned 1,639,025
clusters with width = .003, and ss_aipe_crd_both_fixed_width(width = .003, cluster_cost = 10, individual_cost = 1, pr_treat = .5, true_var_between = .2, true_var_within = .8) returned 1,391,996
clusters of 209. A target met within the bound is planned as before
(width = .004: 921,953 clusters). The standardized planners start
from this search and refuse such a target before anything is
simulated. A width that is not a number at a design a search compares
stops it with class dmar_computation_failed and reason
"width_not_a_number"; the checks on entry leave no call that reaches
it.
The twelve cluster randomized planners refuse a nonzero
true_R2_between with n_covariates = 0. true_R2_between is the
share of the between-cluster variance of the outcome that the
cluster-level covariates explain, and n_covariates = 0 says there are
none. Such a call is refused on entry, before any search or simulation,
in all twelve planners, including the budget planners that compute no
width. The refusal has class dmar_argument_invalid, arg
"true_R2_between", and reason "R2_between_without_covariates",
and its message asks for n_covariates or true_R2_between = 0.
true_R2_within may still be nonzero without a cluster-level
covariate. The plans these calls returned are under Changes to Earlier
Results.
Summary statistics given with a fitted model are refused. Through
1.0.x eta_squared(fit, F_value = 3) and ci_eta_squared(fit, N = 50)
ignored the summary statistics; eta_squared(), omega_squared(),
ci_eta_squared(), and ci_omega_squared() now stop and say that a
fitted model and summary statistics were both given. An infinite
F_value is refused on entry by all four; ci_eta_squared() and
ci_omega_squared() let it reach the noncentral root search, which
failed.
ci_mahalanobis() refuses a name in ... that its root search does
not take. tol, maxiter, check.conv, and trace are passed to
the search of ci_nc_F(); any other name, and an argument with no name,
is refused before anything is computed, with a condition of class
dmar_argument_invalid whose arg names it. DMAR 1.0.x passed every
name on, so most stopped inside the search with a message about the
search, and uniroot()'s lower, upper, f.lower, and f.upper
replaced the bracket the search had set.
ss_aipe_reg_coef_sensitivity() and ss_aipe_rc_sensitivity()
refuse noncentral = TRUE when the sample size is planned from the
supposed values and the supposed variances of Y and of the targeted
predictor are not both 1, with a condition of class
dmar_argument_invalid whose message names ss_aipe_src_sensitivity():
the noncentral plan of ss_aipe_reg_coef() is for a standardized
coefficient, and these two report the unstandardized one. With unit
variances, or with specified_N, the call runs.
bifactor_indices() refuses a multiple-group fit, with the error
average_variance_extracted() gives such a fit (class
dmar_argument_invalid, the groups carried as groups): the factors
have indices in each group, those of the model fitted to that group's
data alone. ci_bifactor_indices() already refused such a fit.
ss_aipe_cv() refuses a plan that cannot exist. With
alpha_upper = 0 the interval is a lower confidence bound whose upper
limit is infinite at every sample size; in DMAR 1.0.0 the search ran
its cap of 100,000 steps (about a minute) before stopping with the
error "The sample size search did not converge within 100000
iterations". Such a call is now refused at once
with a condition of class dmar_argument_invalid naming
alpha_upper. With alpha_lower = 0, an upper confidence bound, DMAR
1.0.0 returned no plan for any width: ci_cv() gave that interval a
lower limit of -Inf, so its width was infinite at every sample size
and the search stopped at the same cap. The lower limit is now 0, and
the width is the upper bound itself. With alpha_upper below .5 the
bound stays above true_cv at every sample size the search can reach,
so a width at or below true_cv is refused the same way, naming
width and alpha_lower. A wider target is planned without
assurance (true_cv = .2, width = .25, conf_level = NULL,
alpha_lower = 0, and alpha_upper = .05 give 37) and is refused with
it (see the entry below on assurance together with
alpha_lower = 0).
ss_aipe_sc() and ss_aipe_sc_sensitivity() take the contrast
weights ci_sc() takes: weights that sum to zero and whose positive
values sum to 1, with ci_sc()'s messages and class
dmar_argument_invalid. ss_aipe_sc(true_std_contrast = .5, c_weights = c(.25, .25, -.5), width = .4) returned 41 for a contrast ci_sc()
refuses to estimate. Integer weights such as -1:1 are accepted, as
before.
ss_aipe_sc_ancova() refuses unequal alpha_lower and
alpha_upper (class dmar_argument_invalid, naming both). It plans
for the interval of ci_sc_ancova(), which has equal tails only, and
with unequal tails planned for an interval no DMAR function reports
(alpha_lower = .01 and alpha_upper = .04 returned 215 at
true_std_contrast = .5, c_weights = c(1, -1), and width = .4).
ss_aipe_sc_ancova(divisor = "s_anova") refuses a ratio of the
ANCOVA to the ANOVA error standard deviation outside (0, 1], which no
population has (class dmar_argument_invalid): true_cor = 1 stopped
with "missing value where TRUE/FALSE needed", and true_sd_anova = 1
with true_sd_ancova = 2 returned 508 at true_contrast = 1 and
width = .5.
Every function that takes contrast weights refuses weights that are
all zero (class dmar_argument_invalid; arg is
"c_weights", or the name the function gives the weights: "contrast"
in contrast_adjusted(), "weights" in meta_contrast(), and
"contrasts" in ci_scheffe(), contrast_test(), and
is_orthogonal_set()). Such weights pass the rules that the weights
sum to zero and that the positive ones sum to at most 1, and they define
no contrast. The message is "'c_weights' has no nonzero weight, so it
defines no contrast; give weights such as c(1, -1) or c(1/2, 1/2,
-1).", with the function's own name for the weights in place of
'c_weights' ("'contrast' has no nonzero weight, ..." in
contrast_adjusted()). Where the weights are a set of contrasts (the
rows of a matrix c_weights in ci_c_ancova_bp(), the rows of
contrasts in contrast_test(), the columns of contrasts in
ci_scheffe() and is_orthogonal_set()), a set with one such contrast
is refused, and the message names it ("Row 2 of 'c_weights' has no
nonzero weight", "The contrast 'none' in 'contrasts' has no nonzero
weight"). The functions are ci_c(), ci_c_ancova(),
ci_c_ancova_bp(), ci_sc(), ci_sc_ancova(), ci_scheffe(),
contrast_adjusted(), contrast_test(), equivalence_c(),
is_orthogonal_set(), meta_contrast(), power_equivalence_c(),
ss_aipe_c(), ss_aipe_c_ancova(), ss_aipe_c_ancova_sensitivity(),
ss_aipe_c_sensitivity(), ss_aipe_sc(), ss_aipe_sc_ancova(),
ss_aipe_sc_ancova_sensitivity(), ss_aipe_sc_sensitivity(),
ss_power_c(), ss_power_c_ancova(), ss_power_contrast(),
ss_power_equivalence_c(), ss_power_sc(), ss_seq_c(), and
ss_seq_c_sensitivity(). With c_weights = c(0, 0) (or three zeros
where the design has three groups), DMAR 1.0.0:
ci_c(means = c(1, 2), s_anova = 1, c_weights = c(0, 0), n_per_group = c(20, 20)), from ci_c_ancova()
with either procedure, from ci_c_ancova_bp(), and from
contrast_adjusted(), ci_scheffe(), and contrast_test(), and
[.5, .5] from ci_c(contrast = .5, s_anova = 1, c_weights = c(0, 0), n_per_group = c(20, 20)) and from ci_c_ancova() with contrast = .5; ci_scheffe() and contrast_test() reported a standard error of
0 beside the interval, and contrast_adjusted(), ci_scheffe(), and
contrast_test() a test statistic and p-value of NaN;ss_aipe_c(true_var_error = 1, c_weights = c(0, 0), width = .5), with or without assurance = .9,
and from ss_aipe_c_ancova(), and a summary of zeros (every
simulated contrast and width 0, pct_ci_less_width 1) from
ss_aipe_c_sensitivity() and ss_aipe_c_ancova_sensitivity();Inf from ss_power_c(true_contrast = .5, c_weights = c(0, 0), true_sd = 1), ss_power_c_ancova(), and ss_power_sc(), and a power
of 1 with specified_n_per_group = 10;all_orthogonal 1 from is_orthogonal_set(cbind(c(1, -1, 0), c(0, 0, 0)));ss_aipe_sc(), ss_aipe_sc_sensitivity(), ss_aipe_sc_ancova(),
ci_sc_ancova(), and ss_aipe_sc_ancova_sensitivity() with
supposed_std_contrast, and with "object 'k' not found" in
ss_aipe_sc_ancova_sensitivity() with specified_n_per_group;ci_sc() ("Please use fractions to specify the contrast weights"),
ss_power_contrast(), equivalence_c(), power_equivalence_c(),
ss_power_equivalence_c(), ss_seq_c(), and
ss_seq_c_sensitivity(), and with "All contrast weights are zero
after centering." in meta_contrast().Empty weights, c_weights = numeric(0), have no nonzero weight either,
and they get the same refusal wherever no earlier check of their length
refuses them. DMAR 1.0.0 returned the limits NaN around the contrast
.5 from ci_c(contrast = .5, s_anova = 1, c_weights = numeric(0), n_per_group = 20) and from ci_c_ancova(contrast = .5, s_ancova = .8, c_weights = numeric(0), n_per_group = 20, cov_means = numeric(0), SSwithin_x = 40), the limits NaN around a contrast of 0 with
means = numeric(0) or adj_means = numeric(0) in place of
contrast, and [.5, .5] from ci_c() with N = 40 added.
It stopped with "missing value where TRUE/FALSE needed" in
ss_aipe_c(true_var_error = 1, c_weights = numeric(0), width = .5),
ss_aipe_c_ancova(), ss_aipe_c_sensitivity(),
ss_aipe_c_ancova_sensitivity() (with empty true_mean_Y),
ss_aipe_sc_ancova(), and ss_aipe_sc_ancova_sensitivity(), and a
matrix that holds no contrast stopped with a message from R ("arguments
imply differing number of rows: 1, 0" from ci_c_ancova_bp(adj_means = c(1, 2), s_ancova = .8, c_weights = matrix(numeric(0), 0, 2), n_per_group = 20), and "length of 'dimnames' [2] not equal to array
extent" from ci_scheffe() with contrasts = matrix(numeric(0), 3, 0)), as did a set that holds no contrast in contrast_test()
("length of 'dimnames' [1] not equal to array extent" with contrasts = matrix(numeric(0), 0, 3), and "argument must be coercible to
non-negative integer" with contrasts = list()). In the other
functions DMAR 1.0.0 stopped on empty weights with a message about the
length or form of the weights, the number of groups or contrasts, the
positive weights, or the degrees of freedom; where a check of the length
comes first (as in ss_power_c(), whose weights must have length 2 or
more), DMAR 1.1.0 refuses them with that check's message.
Logical weights that are all FALSE, which are zeros when R computes
with them, are refused too: with the message above, or, in the
functions that require numeric weights, with that rule's message. Those
functions refused them with the same message in DMAR 1.0.0, without a
class, except ci_sc(), ss_aipe_sc(), and ss_aipe_sc_sensitivity(),
which require numeric weights from this release on: there DMAR 1.0.0
stopped with "Please use fractions to specify the contrast weights"
(ci_sc()) and "missing value where TRUE/FALSE needed" (the two
planners). DMAR 1.0.0 returned [0, 0] from ci_c(means = c(1, 2), s_anova = 1, c_weights = c(FALSE, FALSE), n_per_group = 20) and a plan
of 2 per group from ss_aipe_c(true_var_error = 1, c_weights = c(FALSE, FALSE), width = .5).
Weights in which every contrast has a nonzero value are accepted or refused as before, and the results for them are unchanged.
ss_aipe_sc() and ss_aipe_sc_ancova() refuse a width that needs
more than 2^53 per group (about 9.0e15, the largest number up to
which R holds every whole number exactly; class dmar_argument_invalid,
naming width). Past it neighboring sample sizes can be the same number
in R, so no size there is the smallest that reaches width. DMAR 1.0.0
returned such numbers at a standardized contrast of 0:
ss_aipe_sc(true_std_contrast = 0, c_weights = c(1, -1), width = 5e-8)
(psi_standardized in 1.0.x) gave 12,292,668,226,221,188, and
width = 1e-9 gave 30,731,670,565,552,979,968. Those calls are now
refused, both when the start of the search is past 2^53 and when the
search passes it (as at true_std_contrast = .1 with a start of
9,006,298,534,815,513 per group). Below the limit the plans are
returned at every contrast:
ss_aipe_sc(true_std_contrast = 2, c_weights = c(1, -1), width = 1e-6)
gives 46,097,505,846,480, and true_std_contrast = .1 with
width = 5.9e-8 gives 8,839,438,422,537,546; DMAR 1.0.0 raised the
sample size by one from a start about a third below the first and did
not return either within 60 seconds. Past about 1e11 per group the
plans rest on the precision of doubles (Details of ?ss_aipe_sc and
?ss_aipe_sc_ancova give the sizes). The smallest sample sizes here are
for width and the tail areas as R holds them; with the default
conf_level = .95 each tail area, (1 - .95) / 2, is
0.025000000000000022 in doubles. At a contrast of 0, where the width is
computed from the normal quantiles, width = 6e-7 gives
85,365,751,570,981, the smallest sample size that reaches it, where
DMAR 1.0.0 gave 85,365,751,570,976; width = 1e-6 gives
30,731,670,565,553, also the smallest, where DMAR 1.0.0 gave
30,731,670,565,548; and width = 1e-7 gives 3,073,167,056,555,298, 2
below the smallest, where DMAR 1.0.0 gave 3,073,167,056,555,294. At
other contrasts the width is the difference of two limits near the
noncentrality: the plan above at a contrast of 2 and width = 1e-6 is
1,850 below the smallest, 46,097,505,848,330 (see Bug Fixes for the
search, and Changes to Earlier Results for the noncentral t
distribution at many degrees of freedom).
ss_aipe_reg_coef(), ss_aipe_rc(), and ss_aipe_src() refuse a
tail probability of 0 or above .5, and two tails of .5, with a
condition of class dmar_argument_invalid naming alpha_lower and
alpha_upper. With a tail of 0 the interval has a limit at infinity and
is never as narrow as width, and ci_reg_coef(), which reports the
interval, takes no tail above .5. DMAR 1.0.x planned such a call as if
both tails were their average: for two predictors correlated .3,
correlated .4 and .3 with Y, and a width of .3, alpha_lower = 0 with
alpha_upper = .05 gave 157, the plan for tails of .025, and the
noncentral search (noncentral = TRUE, the default of
ss_aipe_src()) did not end; alpha_lower = .6 with alpha_upper = .01 gave 14; and two tails of .5 gave 5 (2p + 1), or 3 with
assurance = .9, which leaves no residual degrees of freedom.
A nonzero tail area below 1e-9 is refused by ci_nc_t() and by the
intervals and plans of a regression coefficient, a contrast, and a
correlation. ci_reg_coef(), ci_rc(), ci_src(), ci_c(),
ci_sc(), ci_sc_ancova(), ci_r(), ci_sm(), ci_cv(),
ss_aipe_reg_coef(), ss_aipe_rc(), ss_aipe_src(), ss_aipe_c(),
ss_aipe_sc(), ss_aipe_sc_ancova(), ss_aipe_r(),
ss_aipe_r_sensitivity(), and ss_aipe_cv() check the floor once
their tails are resolved, and ci_nc_t() checks it for every interval
and plan whose limits come from it, among them ci_smd(),
ci_smd_c(), smd_trimmed(), ss_aipe_smd(), ss_aipe_sm(), and
the sensitivity functions of these planners. Functions that form one
of these intervals at their own conf_level refuse the same levels:
cles(), proportion_of_superiority(), nnt_from_smd(), and
plot_smd() through ci_smd(), ss_aipe_c_sensitivity() through
ci_c(), and ss_aipe_equivalence_r_sensitivity() through ci_r().
A tail of 0, a one-sided interval, is accepted wherever it was. Below
1e-9 the noncentral t limits lose accuracy: the tail probability a
limit is found from carries an absolute error of up to about 1e-12, a
growing share of a smaller tail. The central intervals of a contrast,
a correlation, and a regression coefficient and the plans for them
keep the same floor, so that an interval and the plan for it accept
the same tail areas. DMAR 1.0.0 and the development versions of 1.1.0
returned what they computed. ci_reg_coef(reg_coef_j = .5, se_reg_coef_j = .1, N = 50, p = 2, conf_level = NULL, alpha_lower = .025, alpha_upper = 1e-20, noncentral = TRUE) (b_j and SE_b_j in
1.0.0) gave an upper limit of 1.549 in 1.0.0 and 1.435 in the
development versions, where the limit is 1.554. With noncentral = FALSE the upper limit was Inf, as it was for ci_c(means = c(1, 0), s_anova = 1, c_weights = c(1, -1), n_per_group = c(20, 20), conf_level = NULL, alpha_lower = .025, alpha_upper = 1e-20) (n in 1.0.0).
ci_r(r = .3, N = 50, conf_level = NULL, alpha_lower = .025, alpha_upper = 1e-20) (n in 1.0.0) gave an upper limit of 1, and -1
for the lower limit with the tails exchanged. ci_sc() with the means
and tails of that ci_c() call gave an upper limit of 4.126 in 1.0.0
and 3.782 in the development versions. ss_aipe_reg_coef(true_R2_Y_X = .1, true_R2_j_X_without_j = .2, p = 2, true_reg_coef_j = .1, width = .4, conf_level = NULL, alpha_lower = 1e-20, alpha_upper = .05)
(rho2_Y_X, rho2_j_X_without_j, and b_j in 1.0.0) returned 114 in
1.0.0, the plan for two tails of .025, and stopped in the development
versions with "missing value where TRUE/FALSE needed".
ss_aipe_sc(true_std_contrast = .5, c_weights = c(1, -1), width = .5, conf_level = NULL, alpha_lower = 1e-20, alpha_upper = .025), and
ss_aipe_sc_ancova() with the same contrast and width and both tails
1e-20 (psi_standardized in 1.0.0 for both), stopped with the same
message in 1.0.0 and refused 'width' ("needs more than 2^53 per
group") in the development versions. A level is held to the floor
through its tails, (1 - conf_level) / 2 each, so conf_level = 1 - 1e-12 is refused by every function above, and .999999998 is the
largest level accepted. At conf_level = 1 - 1e-12,
ss_aipe_reg_coef() with the planning values above returned 1,459 in
1.0.0 and the development versions, ss_aipe_c(true_var_error = 1, c_weights = c(1, -1), width = .5) (error_variance in 1.0.0)
returned 1,640, ss_aipe_r(true_cor = .3, width = .2) (rho in
1.0.0) returned 4,189, and ci_smd(smd = .5, n_1 = 50, n_2 = 50)
returned [-0.953, 1.948]. At .999999998 the floor leaves these plans as
they were; ss_aipe_reg_coef() gives 1,032, where 1.0.0 gave 1,033,
through the change in its plans without assurance (see Changes to
Earlier Results). The refusal has class
dmar_argument_invalid, reason "tail_below_floor", arg naming
the tail or tails below the floor (conf_level when the tails came
from it), value, and floor (1e-9). Its message reads, for example,
"'alpha_upper' is 1e-20, a tail area above 0 and below 1e-9, the
smallest nonzero tail area accepted. Below 1e-9 the noncentral t
limits lose accuracy (...), so a limit or a sample size could be off
without notice. The central intervals of a contrast, a correlation,
and a regression coefficient, and their plans, keep the same floor, so
that an interval and the plan for it accept the same tail areas. Give
a tail area of at least 1e-9, or 0 for a one-sided interval." In
ss_aipe_reg_coef(), ss_aipe_sc(), and ss_aipe_sc_ancova(), which
refuse a tail of 0, and ss_aipe_cv(), which refuses one as
alpha_upper, it ends "Give a tail area of at least 1e-9." For a
level the message reads "'conf_level' of 0.999999999999 leaves a tail
area of 5e-13 on each side, below 1e-9, ... Give a conf_level of at
most 0.999999998." At the floor, the noncentral limits in six cases
(t from -3 to 10, 10 to 1,000 degrees of freedom) at 66 tail areas
from 1e-6 down to 1e-9 were within 1.4e-4 of the exact ones (6.0e-6 of
the width of the interval). "Small tail areas" in ?ci_nc_t gives the
numbers.
An assurance above .999999998 is refused by ss_aipe_sc(),
ss_aipe_sc_ancova(), ss_aipe_sm(), and ss_aipe_smd(), and by
their sensitivity functions when they plan with it. The published
assurance step with which DMAR 1.0.x planned them finds noncentral t
limits with tail areas of (1 - assurance) / 2 and 1 - assurance (in
ss_aipe_smd() through ci_smd()), which then fall below the floor of
1e-9 of the entry above. ss_aipe_sc(true_std_contrast = .5, c_weights = c(1, -1), width = .5, assurance = 1 - 1e-10) (psi_standardized in
1.0.0) returned 151 per group in DMAR 1.0.0 and the development
versions of 1.1.0, as did ss_aipe_sc_ancova() at the same values and
ss_aipe_smd(true_smd = .5, width = .5, assurance = 1 - 1e-10)
(delta in 1.0.0), and ss_aipe_sm(true_sm = .5, width = .5, assurance = 1 - 1e-10) (sm in 1.0.0) returned 116. Each came from
limits below the floor. The refusal names assurance, not the tails of
the step, which the caller did not give (class dmar_argument_invalid,
reason "tail_below_floor"): "'assurance' of 0.9999999999 is too
close to 1: the published assurance step, with which DMAR 1.0.x
planned, finds noncentral t limits with tail areas of (1 - assurance) /
2 and 1 - assurance, and (1 - assurance) / 2 is below 1e-9, the
smallest nonzero tail area accepted, below which those limits lose
accuracy. Give an assurance of at most 0.999999998." The check is made
whatever assurance_method is, so whether a call is refused does not
depend on it.
ss_aipe_reg_coef(), ss_aipe_rc(), and ss_aipe_src() refuse an
abbreviation of "Lower" or "Upper", as they refuse those values.
They plan only for the full width, but DMAR 1.0.0 checked which_width
with char.expand() and then tested the value as given, so "L",
"Lo", "U", "Up", and "" were planned for the full width. In
DMAR 1.0.0, ss_aipe_reg_coef(true_R2_Y_X = .3, true_R2_j_X_without_j = .2, p = 5, true_reg_coef_j = .25, width = .15, which_width = "L")
(rho2_Y_X, rho2_j_X_without_j, and b_j in 1.0.x) and ss_aipe_rc()
with the same arguments returned 607, the plan of "Full", and
ss_aipe_src(true_R2_Y_X = .3, true_R2_j_X_without_j = .2, p = 5, true_std_reg_coef_j = .25, width = .15, which_width = "L")
(Rho2_j_X_without_j and beta_j in 1.0.x) returned 626, also the
plan of "Full". These three calls now stop with a refusal of class
dmar_argument_invalid (arg = "which_width") whose message says that
"L" stands for "Lower". "F" and "Fu" still give the plan of
"Full": 606 from ss_aipe_reg_coef() and ss_aipe_rc(), 625 from
ss_aipe_src() (a case below 1.0.0; see Changes to Earlier Results).
ss_aipe_R2(true_R2 = .5, width = .15, which_width = "", p = 5, random_predictors = FALSE) is refused the same way; DMAR
1.0.0 returned NULL for it (population_R2 in 1.0.x).
mr_cv() and mr_smd() check every argument. Each refusal names
the argument and has class dmar_argument_invalid, or
dmar_count_invalid for m0 and n_per_group. In DMAR 1.0.x:
mr_cv() given scores with a negative mean returned the risk and
the decision of the same scores with their signs reversed, beside a
negative coefficient of variation; a data frame stopped with
"is.atomic(x) is not TRUE"; and three observations returned a risk
of NaN and a decision of NA. data must be a numeric vector of
at least 4 finite observations with a positive mean.sampling_cost of -75 gave a negative criterion and a stopping
rule that was met, and a sampling_cost of 0, an epsilon of 0, or
a gamma of -1 gave a pilot sample size of Inf. sampling_cost,
A, structural_cost, and epsilon must each be a single positive
finite number, and a call without sampling_cost is refused by
name.gamma = .5 was accepted. gamma must be strictly between 0 and
1/2, the interval on which the stopping rule is defined
(Chattopadhyay & Kelley, 2016, Equation 23).mr_cv() returned an m0 of 4.5 or of 2 as the pilot sample size.
m0 must be a whole number of at least 4 in both functions.pilot = 1 and pilot = 0 were read as TRUE and FALSE, and
pilot = "yes" stopped with "object 'Outcome' not found". pilot
and verbose must be TRUE or FALSE.mr_smd() given a sample size per group of -5 (n in 1.0.x)
returned a negative risk and a decision of NA, accepted 74.5, and
stopped with "arguments imply differing number of rows" for a vector
there or in d. d must be a single finite number and
n_per_group a whole number of at least 2.mr_cv(pilot = TRUE, A = 1e300, sampling_cost = 1e-10) returned a
pilot sample size of Inf. A divided by sampling_cost must be a
finite number in both functions.ss_aipe_cv() refuses assurance together with alpha_lower = 0,
a one-sided upper confidence bound, with a condition of class
dmar_argument_invalid that names both arguments and advises planning
without assurance. The assurance step of Kelley (2007) was written
for a two-sided interval and does not give the requested probability
for a bound; the numbers are on ?ss_aipe_cv under "Edge cases". DMAR
1.0.0 returned no plan for a call with alpha_lower = 0 (see the
entry above on a plan that cannot exist).
ss_aipe_cv() refuses a planning value or a tail area that is not
one admissible number, each by name (class dmar_argument_invalid).
With conf_level = NULL, a tail area that is left out, NA, or a
vector stopped in DMAR 1.0.0 with an error of R's own ("missing value
where TRUE/FALSE needed" for a tail left out or NA); the refusal now
says that each of alpha_lower and alpha_upper is a single number
at least 0 and below 1. A vector true_cv (C_of_V in 1.0.x) stopped
with "the condition has length > 1". true_mean and true_sd (mu
and sigma in 1.0.x) must each be positive: DMAR 1.0.0 planned a
negative mean with a negative standard deviation as their positive
ratio.
ss_aipe_cv_sensitivity() checks every input that defines its
simulation. DMAR 1.0.0 refused a true_cv that was left out or not
positive and an estimated_cv (now supposed_cv) that was not
positive. It did not check that either was one finite number, and it
did not check mean (now true_mean) or, when specified_N was
given, specified_N, width, or assurance. true_cv,
supposed_cv, and true_mean are now each a single finite positive
number; specified_N is a whole number of at least 4, the smallest
sample size ss_aipe_cv() plans (class dmar_count_invalid), so that
a character string does not turn the value column into text and a
fraction is not echoed as a sample size; width, when given, is a
single finite positive number, also with specified_N; and
assurance is checked on entry (class dmar_assurance_invalid), also
with specified_N, where it is only echoed.
var_cv() refuses an NA or infinite true_cv or N. In DMAR
1.0.x, where the arguments were cv and n, a numeric NA in either
(NA_real_, as a computation returns it) stopped with R's "missing
value where TRUE/FALSE needed", an infinite coefficient returned Inf
in both rows, and an infinite sample size returned 0 and NaN. Each
of these calls now stops with a refusal of class
dmar_argument_invalid that names the argument and states the value
given.
ss_aipe_R2() refuses assurance with a half-width. With
which_width = "Lower" or "Upper" the function plans for the
distance from the expected sample value to one limit, and assurance
given with either stops with a refusal of class dmar_argument_invalid
(arg = c("assurance", "which_width")). In DMAR 1.0.0 every such call
with "Upper" stopped with the R error "unused argument
(random_predictors = random_predictors)", and with "Lower" the
function returned a sample size that did not deliver the probability
asked for. ss_aipe_R2(true_R2 = .30, width = .10, p = 3, which_width = "Lower", assurance = .99) (population_R2 in 1.0.x) returned 253,
the plan without assurance, and at 253 the distance from the sample
squared multiple correlation to its lower limit was no larger than .10
in 52 percent of 20,000 simulated samples. With true_R2 = .10 and
width = .05 it returned 486 for an assurance of .80, again the plan
without assurance (52 percent), and 637 for .99 (95 percent); with
true_R2 = .70 and width = .10 it returned 153 for .80 (72
percent).
ss_aipe_R2() and ss_aipe_R2_sensitivity() refuse a call without
width. ss_aipe_R2(true_R2 = .5, p = 5) (population_R2 in
1.0.x) stopped in DMAR 1.0.0 with R's "missing value where TRUE/FALSE
needed" and now stops with a refusal of class dmar_argument_invalid
that names width (arg = "width"), as a call without p or
true_R2 is refused in the name of the argument left out.
ss_aipe_R2_sensitivity() (where width was w in 1.0.x) stopped
with the same error of R's when w or true_R2 was left out, and
with "invalid 'times' argument" when p was left out beside
specified_N. Each of true_R2, width, and p left out is now
refused in its name, with supposed_R2 and with specified_N.
A condition with a class can be caught or muffled by name, whatever its
wording. Every error and every warning DMAR signals on purpose now has
a class. Every error is a dmar_refusal and carries a code, so
tryCatch(expr, dmar_refusal = function(e) ...) catches whatever DMAR
refused, and an error without that class came from R, from another
package, or from match.arg(). The check of an argument (its type,
length, or range, arguments that cannot be given together, a column that
is not in the data, a model the function does not cover) is a
dmar_argument_invalid, which carries the names of the arguments at
issue as arg (NULL where the refusal is not about a named argument,
and in some checks whose message names the argument only in running
text). Three more general classes cover the errors
that have no class of their own:
dmar_package_required (a package the analysis needs is not installed;
field package), dmar_computation_failed (the arguments were accepted
and a search, a fit, a refit, or a bootstrap made with them gave no
result), and dmar_internal_error (a state DMAR's own code should not
reach). Two cover the warnings that have none: dmar_result_qualified
(the result is
returned with something to know before using it) and
dmar_argument_ignored (an argument given has no effect on the call).
The more specific classes (dmar_not_converged, dmar_no_maximum,
dmar_lavaan_error, dmar_inadmissible, and the others the sections
below name) are dmar_refusals as well; dmar_bootstrap_cancelled and
dmar_progress_error, which come from the caller's progress function,
are not. In 1.0.0 nearly every error was a plain error and no error was a
dmar_refusal. Messages are
worded as before, except that some of the oldest checks now name the
argument and state the rule (for example "'c_weights' must sum to zero."
in place of "The sum of the coefficients must be zero"). The help of
dmar_metadata() lists the classes.
ss_aipe_cv() and ss_aipe_sm() refuse an assurance that is
not NULL or a single number strictly between 0 and 1 with
dmar_assurance_invalid
(the condition carries the value given; the message says why neither end
is accepted). dmar_argument_invalid, which carries the name or names at
issue as arg, is the class of the refusals of ss_power_R2_change()
for a p_restricted not below p_full, for squared multiple
correlations that cannot belong to nested models, and for an effect
stated in more than one way; of the partial and semipartial correlation
functions for a p below 2; and of smd() and smd_c() for
unbiased = TRUE without the sample sizes. One warning joins a class its
siblings already gave: ss_aipe_reliability_sensitivity() reports
replications without an interval with
dmar_replications_without_interval (fields n_without_interval and
G). The count of noncentral F lower-limit clamps that ss_aipe_R2()
reports after its search has the new class dmar_nc_F_clamp_search, with
the field n_evaluations. The wording of each message is unchanged,
except that the refusals of smd() and smd_c() now also say why the
sizes are needed. The warnings about the accurate range of the
noncentral t distribution that ci_nc_t() and the searches built on it
gave in DMAR 1.0.x are gone, since every limit is now computed exactly
(see Exact Noncentral t and Studentized Range Distributions).
which_width that is not a single character string is refused with
a class. In DMAR 1.0.0, ss_aipe_reg_coef(), ss_aipe_rc(),
ss_aipe_src(), and ss_aipe_R2() with which_width = 1 stopped with
the error of base R "'input' and 'target' must be character vectors",
and with which_width = c("Full", "Lower") with "'input' must have
length 1". They now stop with dmar_argument_invalid
(arg = "which_width").Where a check's message opens with the name of the argument and the
check did not store it, arg is read from the message, so
tryCatch(bayes_one_sample_t(mean = .5, sd = 1, N = 30, prior_location = NULL), dmar_refusal = function(e) e$arg) returns "prior_location"
where it returned NULL.
ss_aipe_mixed_effects() plans for a predictor at either level and
for three ways of centering it, and
ss_aipe_mixed_effects_sensitivity() simulates what it plans. In
DMAR 1.0.x the planner was for an individual-level (i.e., Level 1)
predictor centered at its cluster mean, and the sensitivity function
generated a cluster-level (i.e., Level 2) predictor, whose slope has a
far larger variance; its intervals were much wider than the plan and
the comparison tested nothing. Both functions now take
predictor_level (1, the default, or 2), centering
("cluster_mean", the default; "grand_mean"; "none"), and the
intraclass correlation of an individual-level predictor (true_icc_X,
with supposed_icc_X for the planning value in the sensitivity
function), share one set of expressions, and form one interval, the
estimate plus and minus a t quantile times its standard error, on
K(m - 1) - 1 degrees of freedom for an individual-level predictor and
K - 2 for a cluster-level one.
true_total_var = 1,
true_var_X = 1, true_icc = .10, width = .20, and clusters of 20;
88 against 70 with clusters of 5), because it uses the t quantile
and the expected standard error of the interval computed from data,
where 1.0.x used the normal quantile and the total sample size.true_icc_X given for one.expected_width, predictor_level, true_icc_X, and
supposed_icc_X rows.A minor release, and a large one. DMAR 1.0.1 was prepared but never submitted to CRAN; its changes are part of this release, and its section below is kept as the record of them.
Factor analysis gets its own names and new models. fa_confirmatory()
is the 1.0.0 cfa_k() and fa_exploratory() its new exploratory
partner, with cfa_k() and plot_cfa_k() kept as aliases, efa_k() a
new alias of fa_exploratory(), and the 1.0.0 result class kept after
the new one; fa_exploratory() fits
incomplete items by FIML, gives rotation-aware intervals, and rotates to
a target; factor_count() and parallel_analysis() compare numbers of
factors; fa_bifactor() and fa_second_order() fit the bifactor and
second-order models, with delta method intervals for their indices;
factor_scores() gives per-case standard errors and the determinacy of
each factor, beside 2 rho^2 - 1, the lowest correlation two equally
consistent sets of scores can have; and factor_congruence() gives
Tucker's congruence between two solutions. Structural equation models
get measures of fit, precision, and comparison (sem_fit_measures(),
sem_residual_fit(), sem_information_criteria(),
sem_exact_fit_bootstrap(), equivalence_rmsea(), compare_sem())
and sem_sam(), which estimates the structural relations after the
measurement models.
A missing data family states what the missing values cost: the fraction
of missing information for every parameter of a FIML fit, the patterns
and coverage behind it, the maximum likelihood covariance matrix of
incomplete data, and small-sample degrees of freedom with a t reference
distribution for Wald intervals, taught in a new vignette. Two-stage
maximum likelihood is a missing data option for the CFA family,
measurement invariance, and coefficients alpha and omega, and sample
size planning for a reliability coefficient can allow for the missing
values it will be estimated from. The omega of reliability_omega(),
reliability_omega_categorical(), and bifactor_indices() can take
either the observed or the model implied variance as its denominator and
says which it used; categorical omega no longer needs mvtnorm, no longer drops a
correlation the data do not determine, and takes incomplete items;
coefficient H is computed from a fitted model; the bifactor indices get
bootstrap intervals; and the reliability bootstrap counts the fate of
every replication, runs on several processes, refuses a solution that
leaves no coefficient, and takes a progress callback that can cancel it.
New intervals, tests, and plans: Dunnett's T3 (ci_dunnett_t3()); the
difference between two proportions measured on the same units, with
tests of noninferiority and equivalence (ci_proportion_paired(),
equivalence_proportion_paired()); Fieller's confidence set for the
ratio of two means (ci_ratio_of_means(),
ci_ratio_of_means_paired()); nonsuperiority in equivalence_c();
the marker variable adjustment for common method variance with its
interval and test (common_method_adjusted_r()); planning the interval
for a single mean (ss_aipe_mean()) and the assurance of a contrast's
interval when the errors are not normal (true_kurtosis in ss_aipe_c());
a crossed design of units and raters (ci_crossed(),
ss_aipe_crossed()); and two-phase sampling, a costly measurement with
an inexpensive proxy (ci_two_phase(), ss_aipe_two_phase(),
ss_aipe_two_phase_cost(), ss_seq_two_phase()). A new vignette fits
classical tests as path models in lavaan, and a knitted result table
now carries every line the console prints under it. A result table
prints its numbers to three decimal places, where 1.0.x printed three
significant figures (see Changes to Earlier Results). eta_squared(),
omega_squared(), and their intervals report the total and the partial
value of each effect and take sums of squares from a published table;
ss_aipe_smd(), ss_aipe_sc(), ss_aipe_sc_ancova(),
ss_aipe_reg_coef(), ss_aipe_rc(), and ss_aipe_src() can find the
sample size for assurance from the probability that the interval is no
wider than asked for, computed exactly or simulated from the exact
sampling distribution; ss_power_indirect_effect() takes its power from
the joint sampling distribution of the two path tests, with the
predictor measured or fixed by design; and the noncentral t
distribution and the studentized range are computed exactly wherever R's
pt(), qt(), dt(), qtukey(), and ptukey() are not accurate.
Some defaults change results. DMAR 1.1.0 gives no warning when a result
or a default differs from 1.0.x; these notes describe every such change.
The changed defaults are described under Changes to Earlier Results,
each with the argument that restores the 1.0.x analysis where there is
one; ss_aipe_rmsea_sensitivity() also corrects how it forms its
intervals, which no argument restores.
fa_confirmatory() (with its alias cfa_k(), and with
cfa_1() and cfa_2()) fits raw data by full information maximum
likelihood and takes the standard errors that match the estimator
(Huber-White under MLR, where 1.0.x took the standard ones), and reliability_omega(), reliability_alpha(), and
measurement_invariance() fit incomplete data that way (as does
reliability_H(), which takes data for the first time in this
release); reliability_omega() forms the default interval of the model
implied omega on the logit scale; and ci_rmsea(), ss_aipe_rmsea(),
and ss_aipe_rmsea_sensitivity() form and plan a 90 percent RMSEA
interval by default, where 1.0.x used 95 percent.
ss_aipe_semipartial_r() without the new true_R2_full plans for the
largest variance over every full-model R2, which raises the sample size
when true_r_sp squared is above 1/2 (true_R2_full = true_r_sp^2
restores the 1.0.x plan), and ss_aipe_partial_r_sensitivity() and
ss_aipe_semipartial_r_sensitivity() simulate one more variable
partialed out than 1.0.x did for the same model, which no argument
restores.
ss_aipe_partial_r() and ss_aipe_semipartial_r() find the sample size
for assurance from the sampling distribution of the estimate, where
1.0.x multiplied by a chi square factor that does not apply to these
intervals, and ss_aipe_partial_r_sensitivity() simulates the raw-scale
interval it plans, where 1.0.x simulated Fisher's Z interval; no
argument restores the first, and fisher_z = TRUE gives the Fisher
interval, planned and simulated, for the second.
Every function names the total sample size N, the number of predictors
p, a planner's planning value true_X, a sensitivity function's
planning value supposed_X, and the desired width width, and names an
argument for the quantity it holds (true_cor, true_contrast,
true_reg_coef_j, true_cov_matrix) rather than for a symbol; the 1.0.x
names are not accepted, eta_squared_partial(), omega_squared_partial(),
and ci_eta_squared_partial() are removed, and the count of the partial
and semipartial correlation functions is the number of predictors, one
more than the 1.0.x count of variables partialed out (see Breaking
Changes in 1.1.0).
vargha_delaney_A() refers its test statistic to Student's t with
Satterthwaite degrees of freedom, which gives a larger p-value than
the standard normal reference of 1.0.x (test = "z" restores it).
With assurance, ss_aipe_smd(), ss_aipe_sc(), ss_aipe_rc(),
ss_aipe_reg_coef() with its default noncentral = FALSE, and their
sensitivity functions return the smallest sample size, not below the
plan without assurance, at which the exact probability that the
interval is no wider than width reaches the request (the new argument
assurance_method, "analytic" by default), where DMAR 1.0.x returned
published approximations that could attain more or less than the
request. ss_aipe_sc_ancova(), ss_aipe_reg_coef() with noncentral = TRUE, ss_aipe_src(), and their sensitivity functions return the
published approximations by default, and assurance_method = "simulation" returns the smallest sample size at which a simulated
estimate of that probability reaches the request. The four regression
and ANCOVA planners without assurance return the smallest sample size
that meets the width, where 1.0.x could return a case or more above it
(a few more per group in ss_aipe_sc_ancova()), and the approximations,
which start from that plan, move with it. No argument restores
the plans of ss_aipe_r(), ss_aipe_smd(), and ss_aipe_sc() with
assurance, the central plans of ss_aipe_reg_coef() and
ss_aipe_rc() with assurance or with unequal tails, the plans
without assurance of ss_aipe_reg_coef(), ss_aipe_rc(),
ss_aipe_src(), and ss_aipe_sc_ancova(), which can be a case or more
smaller, and the approximations that start from them, the plans
and true values of ss_aipe_src_sensitivity() when the variances of Y
and of the targeted predictor differ, the plans of
ss_aipe_equivalence_smd() and its sensitivity function and of the
standardized cluster randomized planners, the power and plans of
ss_power_indirect_effect(), the rows of var_alpha(), or the
simulated contrasts of ss_aipe_sc_sensitivity(), each corrected in
this release.
fa_confirmatory() on complete data gives the same estimates as 1.0.x,
but under ML its standard errors come from the observed information, the
item intercepts join the table, and AIC and BIC count them; and its
omega, average variance extracted, and H rows get limits formed on the
logit scale, as reliability_H()'s already were, as
described under Exploratory Models and Structural Relations. Its RMSEA
interval stays at the 90 percent of 1.0.0, now with an argument of its own,
rmsea_conf_level: every RMSEA interval in the package, and the SRMR
interval of sem_residual_fit(), defaults to 90 percent, whose limits
decide the tests of close and not close fit at .05, while the other
intervals keep 95 percent (see Every RMSEA Interval at 90 Percent by
Default).
RMSEA and SRMR intervals default to 90 percent, and every output
states the level. A 100(1 - 2 alpha) percent interval for the
RMSEA decides the one-sided tests of close and not close fit at level
alpha (MacCallum, Browne, & Sugawara, 1996, p. 136), and its lower limit
is 0 exactly when the test of exact fit does not reject at alpha, so the
limits of a 90 percent interval decide those tests at .05, the level
conventionally used with them and the default at which ss_power_sem()
plans them; sem_fit_measures() reports the tests' p-values beside the
interval, and the 90 percent limits agree with those p-values compared
with .05. A 95 percent interval decides them at .025.
Every RMSEA interval in the package therefore defaults to 90 percent, as
lavaan's does, and so does the SRMR interval of sem_residual_fit(),
whose test of close fit pairs with its interval in the same way; the
package's other intervals keep 95 percent. Where a function reports both
kinds, the RMSEA interval has an argument of its own. The functions and
their arguments:
ci_rmsea(), ss_aipe_rmsea(), and ss_aipe_rmsea_sensitivity():
conf_level = 0.90, where 1.0.x defaulted to 0.95.
conf_level = 0.95 restores the 1.0.x result of ci_rmsea() and
ss_aipe_rmsea(), and the 1.0.x level of
ss_aipe_rmsea_sensitivity(), whose intervals were also corrected,
with no argument to restore them (see Changes to Earlier Results).sem_fit_measures(): conf_level = 0.90, which also sets the gamma
hat and adjusted gamma hat intervals, mapped from the same
noncentrality limits.sem_residual_fit(): conf_level = 0.90 for the interval of the
unbiased SRMR, lavaan's default too.factor_count(): conf_level = 0.90, the level of its RMSEA intervals
and of nothing else; the Monte Carlo limits of the parallel analysis
reference in its parallel_analysis attribute keep the 95 percent of
parallel_analysis().fa_confirmatory() (and cfa_k()), cfa_1(), cfa_2(),
fa_bifactor(), and fa_second_order(): the new rmsea_conf_level = 0.90 sets the RMSEA interval of the fit rows, which the
rmsea_ci_level row and the glance() column of that name record,
while conf_level = 0.95 sets the intervals of the parameters and
indices and no longer reaches the RMSEA interval. cfa_k(), cfa_1(),
and cfa_2() thereby report the RMSEA interval of DMAR 1.0.0, which
took lavaan's 90 percent whatever conf_level was; a development
version of 1.1.0 made it follow conf_level, and that change is
withdrawn. In fa_confirmatory() (and so cfa_k()), cfa_1(), and
cfa_2(), whose arguments DMAR 1.0.0 fixed in order, rmsea_conf_level
comes after ..., beside wald_df, so a 1.0.0 call that gives its
arguments by position works as before and the new argument is given
by name.fa_exploratory() (and efa_k()): the new rmsea_conf_level = 0.90
sets the RMSEA interval of the fit line and of the fit_measures
attribute, while conf_level sets the intervals of the solution.
factor_count() passes its conf_level to it.irt_grm(): the new rmsea_conf_level = 0.90, passed to lavaan, sets
the RMSEA interval of the fit_measures attribute, which held
lavaan's 90 percent interval before with no way to choose it, and is
recorded as the rmsea_conf_level attribute.equivalence_rmsea() keeps alpha_level, whose 1 - 2 alpha
interval is 90 percent at its default of .05.For 95 percent intervals pass conf_level = 0.95 to ci_rmsea(),
ss_aipe_rmsea(), ss_aipe_rmsea_sensitivity(), sem_fit_measures(),
sem_residual_fit(), and factor_count(), and rmsea_conf_level = 0.95
to the factor analysis functions and irt_grm(). The level is stated
wherever such an interval is shown: the line beneath a printed table
("Confidence level: 90%", and for the factor analysis tables, whose
parameter intervals are at another level, "Confidence level: 95% (90%
for the RMSEA interval)"), the fit line of fa_exploratory() ("RMSEA
with 90% interval"), the line beneath factor_count()'s table,
results_sentence(), the conf_level column tidy() puts on the RMSEA
rows of sem_fit_measures() and beside the RMSEA columns of
factor_count(), and a new rmsea_conf_level field of
dmar_metadata(), which holds the level of a table's RMSEA interval
beside conf_level.
The help of rmsea_conf_level in fa_confirmatory(), fa_bifactor(),
and fa_second_order() (and so in cfa_1() and cfa_2(), which
inherit it) now says where the argument reaches: the table DMAR returns
and its glance(). output = "summary" returns the lavaan summary,
which shows lavaan's 90 percent interval whatever the argument is.
fa_exploratory() and fa_confirmatory() name the exploratory and the
confirmatory factor analysis. fa_confirmatory() is the function DMAR
1.0.0 released as cfa_k(); cfa_k() remains as an alias, as
plot_cfa_k() does for plot_fa_confirmatory(), and its results keep
the 1.0.0 class dmar_cfa_k after dmar_fa_confirmatory, so code
written for 1.0.0 still runs, though on incomplete items its default
analysis has changed (FIML in place of listwise deletion; see Changes to
Earlier Results). efa_k() is the alias of fa_exploratory(),
whose name is DMAR's own so that attaching lavaan, whose efa() it
calls, masks nothing. fa_exploratory() fits common-factor ML or MLR
models directly to incomplete raw observations with FIML, or explicitly
requested WLSMV models to ordered items. Rotated loadings, factor correlations, and communalities carry
rotation-aware confidence intervals. The estimator, missing-data method,
and selected rotation travel with the result. The factor count and
rotation are conditioned on, not treated as known without qualification.
estimator = "ULS" adds the minimum residual (minres) solution of
Harman and Jones (1966): unweighted least squares fitted to the
correlation matrix (lavaan's correlation = TRUE), whose loadings and
communalities equal those of psych::fa(fm = "minres") and of a direct
minimization of the squared off-diagonal residuals. With raw data the
standard errors are lavaan's normal theory sandwich for a least squares
estimator (robust.sem.nt); from a supplied matrix they are NA, since
the sandwich needs the raw data. ULS has no likelihood, so incomplete
data are refused under the default missing = "fiml" rather than
deleted or imputed silently: missing = "listwise" fits the complete
cases, and S takes a correlation matrix estimated from every case (for
example cov_fiml()'s). The ULS statistic, N - 1 times the ULS
discrepancy of the correlations, has no reference distribution, so in
the fit_measures attribute the chi_square row holds it without a
p-value, the RMSEA rows and the CFI, TLI, RNI, and IFI are NA, and the
NFI and the SRMR are reported; sem_fit_measures() says why in the
terms of a correlation structure, in a warning of class
dmar_uls_standard_statistic that fa_exploratory() reads by class. glance()
carries the fitted structure ("correlation" or "covariance").
fa_exploratory() prints the solution, not the parameter list. One
row per item with its standardized pattern loading on each factor and
the loading's interval, its communality with its interval, and
Hofmann's (1978) complexity; above the table the sample, estimator,
missing data method, rotation and its form, and standard error method;
beneath it the variance each factor accounts for (the diagonal of Phi
Lambda' Lambda, as a sum of squared loadings and as proportions of the
total and of the common variance, in the new variance_accounted
attribute), the factor correlations with their intervals, and a fit
line read from the fit_measures attribute with its version lines.
knit_print() and as_kable() show the same display. The table keeps
every parameter in both metrics for tidy(), now with the complexity
rows (op "complexity"), and a subset of it prints as an ordinary
table. The rotation list opens from three criteria to ten, each applied
in the form the help names and refused if lavaan applied it otherwise:
oblique oblimin, quartimin, geomin, simplimax, oblimax, infomax, and
Bentler; orthogonal varimax and quartimax; and the unrotated solution. The result carries the class
dmar_fa_exploratory in front of dmar_factor_analysis, and the help
states the covariance-first rule (Cudeck, 1989) with its two
correlation-structure exceptions, minres and ordered items.
fa_exploratory() reports each factor's determinacy and coefficient
H. The determinacy is the correlation of the regression factor score
with its factor, the multiple correlation of the factor with the items
(Grice, 2001; the factor determinacy index of Ferrando & Lorenzo-Seva,
2018), whose square is the factor's maximal reliability under the model;
H is Hancock and Mueller's (2001) closed form on the factor's
standardized loadings alone, as Rodriguez, Reise, and Haviland (2016)
apply it to one factor of a multidimensional solution. Each carries a
delta method standard error over lavaan's covariance matrix of the
standardized estimates and Wald limits formed on the logit scale
(Browne, 1982), as reliability_H() forms them by default. They are
rows of the table (ops "determinacy" and "H"), the new
factor_quality attribute, and a block of the display under the factor
correlations. With one factor H is the squared determinacy, and its
standard error is the one fa_confirmatory()'s H row carries for the
same model fitted the same way. Beside each determinacy rho the display
and the attribute give 2 rho^2 - 1, the lowest correlation two sets of
factor scores equally consistent with the model can have (Guttman,
1955, as Grice, 2001, presents it), with its delta method standard error
and the determinacy's limits mapped through it; it is zero when rho^2 is
.5 and negative below, and no line is drawn.
fa_confirmatory()'s omega, average variance extracted, and H rows
take one interval form. Each keeps lavaan's delta method standard
error, and its limits are formed on the logit scale and transformed
back to the scale of the coefficient (Browne, 1982; Kelley, 2007), as
reliability_H() and fa_exploratory() form H's, so they lie within
(0, 1) and are asymmetric near one; for H with one factor the three
functions give the same interval for the same fit. A finite wald_df
puts its t quantile on the logit scale; under wald_df = "missing_information" these rows keep the normal reference, as
reliability_omega()'s logistic intervals do, and only the loadings
take the t reference. Through this change the limits were the estimate
plus or minus the normal quantile times the standard error, which can
exceed one.
fa_exploratory() rotates to a partially specified target.
rotation = "target" (oblique; Browne, 1972b) and
rotation = "target_orthogonal" (Browne, 1972a) take the target in the
new target argument, either as fa_confirmatory()'s named list of item
keys, which targets each item at zero on every factor it is not keyed to,
or as a matrix of items by factors with NA for each unspecified
loading; with no items, the items the target names are analyzed, from
raw data or from S alike, and the number of factors is read from the
target. lavaan's rotation checks neither the target nor whether it
identifies the rotation (an unidentified one comes back with finite
standard errors and no warning), so DMAR checks the identification
counts before fitting and the rank condition on the standardized
unrotated loadings (Algina, 1980; Browne, 2001), and refuses a target
that fails with an error of class dmar_target_unidentified (a target
it cannot read, with dmar_target_invalid; both inherit
dmar_refusal). The target's mask is passed explicitly, so nonzero
target values are kept, and no row weights are applied. lavaan reflects
all the factors or none, so when any specified value is nonzero no
factor is reflected: a factor with a nonzero value takes its sign from
the target, and one whose specified values are all zero takes the sign
of the random start that reached the minimum, which seed fixes (the
help says so). Loadings the rotation attains exactly, which lavaan
reports with a standard error of zero and a meaningless test, are shown
as fixed by the target, at the target value and without a standard
error; when every specified loading is attained the solution is the
confirmatory model with those loadings fixed, and the tests check it
against lavaan::cfa(). The rotation line of the display counts the
specified loadings, the target prints beneath the table with a line
mapping lavaan's factor names to the target's, and the method
attribute carries the target and that count. The help summarizes the
simulation of Myers, Jin, Ahn, Celimli, and Zopluoglu (2015) and says
that the intervals condition on a target fixed before the data were
seen.
New factor_count() compares factor counts. It fits the exploratory
model with one factor, two, and so on (by default up to the largest
count that keeps a degree of freedom) and sets beside each count the
chi square test, RMSEA with its interval, CFI, TLI, and SRMR of that
model (from fa_exploratory(), under the same version policy),
lavaan::lavTestLRT()'s test of the previous count against it, the AIC,
BIC, SABIC, and HBIC of sem_information_criteria() with the comparison
compare_sem() draws by a chosen criterion (BIC by default: each
count's gap from the lowest, its relative weight, and the log Bayes
factor of the best count over it), the reduced eigenvalue at
that rank with its parallel analysis reference (parallel_analysis(),
Horn, 1965), and Velicer's (1976) average squared partial correlation
after that many components (the MAP). A count whose model does not
converge or is improper is a row that says so, with its fit withheld and
the counts beyond it still fitted. The lines under the table name the
counts with the lowest criteria, how far the eigenvalues exceed their
reference, and where the MAP is smallest; no cutoff is applied and no
count is chosen. fa_exploratory()'s refusal of a fit without a proper
solution now carries the class dmar_efa_no_solution and a reason
(not converged, improper, or standard errors unavailable), with a
message that says which.
fa_exploratory() takes its fit measures from the unrotated
solution. lavaan's FIML-corrected (robust) RMSEA, CFI, and TLI of a
rotated efa object depend on the rotation, since the rotated
parameterization counts more free parameters than the model has: on six
Holzinger and Swineford items the robust 95 percent RMSEA interval of one
two-factor model was [0, 0.075] unrotated and [0, 0.154] under oblimin, with the
same chi square statistic; the standard indices do not depend on it. The
fit line and the fit_measures attribute now come from the unrotated
fit of the same model, so every rotation reports the same fit, and
factor_count() reports those values.
fa_exploratory() can return the point estimates alone. With the
new point_estimates_only = TRUE (the default FALSE gives the full
analysis, unchanged) the table has the rows and the estimates of the
full analysis, the determinacy, H, variance accounted for, and every
fit measure included, with its standard errors, tests, and limits NA,
because none of them is computed: lavaan is asked for no standard
errors (se = "none"), the rotated fit is made without the test
statistic DMAR never reads from it, and the delta method intervals of
the standardized solution, the determinacy, and H are skipped. The fit
measures, the robust statistic and FIML-corrected indices of MLR and the
scaled and shifted statistic of WLSMV among them, come from the
unrotated fit as before and are identical, and the RMSEA interval keeps
its own level, rmsea_conf_level. A loading a target rotation attains exactly is
recognized by its value and shown at its target, as in the full
analysis. The input, target, convergence, proper-solution, and no
maximum refusals are unchanged. The refusal of a solution without
rotation-aware standard errors, which guards against a solution that
is not identified at its estimates, is kept by inverting the
information of the unrotated fit, the cheap expected information first
(a failure confined to the rotation's part of those errors is not
detected); a FIML fit whose
errors would need unavailable H1 information is not refused, since
none are computed, and its fit measures are withheld as before. The
display says that standard errors and intervals were not computed, and
states no confidence level beside the RMSEA's on the fit line, and the
setting travels as the point_estimates_only attribute and a column of
tidy() and glance(). The time saved is largest for a rotated
solution: about a third on the nine classic Holzinger and Swineford
tests under ML and MLR, more than half under WLSMV, and from 29 s to
17 s for the rotated solution of 18 items with 410 patterns of missing
values under MLR with FIML, where what remains is the unrotated fit's
robust statistic and FIML-corrected fit indices; an unrotated analysis
saves little or nothing, so a program that fits both the rotated and
the unrotated model saves on the rotated fit alone (45 s against 34 s
there).
New factor_congruence() compares the factors of two solutions. It
takes two loading matrices, fa_exploratory() tables, or single-group
lavaan fits (such as fa_confirmatory(output = "fit"); the result
tables of fa_confirmatory(), cfa_1(), and cfa_2() are refused
with a pointer to that route), reads fits in the standardized metric,
and aligns the items by name. The factors of y are paired one to one
with those of x by the assignment that maximizes the sum of the
absolute coefficients (the Hungarian method, with ties kept in the
listed order), and a factor whose match is negative is reflected;
match = "order" pairs them by position instead, signed. Each row
gives the coefficient of a pair and the largest absolute coefficient of
the row's factor with any other factor of the other solution, a surplus
factor of the larger solution has a row of its own, and the full
matrix, the aligned loadings of y, and its reflected factor
correlations are attributes. loadings = "structure" compares the
structure loadings of fa_exploratory() tables and lavaan fits, not of
a matrix. The help describes Lorenzo-Seva and ten Berge's (2006) study
of how analysts judged similarity as a finding about the analysts, and
the table applies no threshold. The coefficients agree with
psych::fa.congruence() to 1e-12, and procrustes_phi(), the
coefficient of one pair of vectors, links to it.
parallel_analysis() uses the same fixed one-factor ML communality
reduction for observed and reference correlation matrices. Normal and
independently resampled-response references remain distinct, with
ordered margins respected when requested. Reference quantile limits
describe Monte Carlo error, not observed-eigenvalue sampling uncertainty
or confidence in a factor count. The output counts failed and defined
boundary reference fits. A complete-case reference size alongside a FIML
correlation matrix is explicitly a benchmark, with no effective-sample-
size or conservativeness guarantee.
fa_exploratory(), factor_count(), and parallel_analysis() read
items with a covariance matrix as they read it with raw data: S is
reduced to the named rows and columns, in the order given, before its
entries are checked. Naming six items of a nine-item S therefore
analyzes exactly what S[items, items] would: a missing entry outside
the six does not refuse the call, nor does a whole matrix that is
indefinite (as one from pairwise deletion can be) while the six-item
submatrix is positive definite. A name that is not an item of S is
refused, with a message that lists the names not found.
factor_count() and parallel_analysis() fit complete ordered
items with missing left out, as fa_exploratory() does. Such items
were refused with the message meant for incomplete ordered items
(factor_count() passed on its own default, "fiml"; a direct call of
parallel_analysis() stopped with "Ordered parallel analysis requires
explicit pairwise or listwise handling"), while missing = "listwise"
ran. The three functions now resolve the default by one rule: with no
value missing, leaving missing out analyzes the items as they stand,
factor_count() uses that method for every fit, correlation, and
parallel analysis, and the method attributes record "listwise". On
six four-category items (400 cases) under estimator = "WLSMV",
leaving missing out gives the table of missing = "listwise" exactly
(chi square 46.972 on 9 df for one factor). Ordered items with a value
missing still require "pairwise" or "listwise", and "fiml" or
"ml" is refused for ordered items; every call that rule refuses is
refused by fa_exploratory() too, with the class
dmar_argument_invalid, before anything is computed or any random
number drawn.
sem_sam() provides local structural-after-measurement estimation for
complete continuous data, using ML/twostep or MLR/twostep.robust. Independent
measurement fits and derivatives of the corrected latent covariance
reproduce the point estimates and normal or robust uncertainty, including
lavaan's optional finite-sample Gamma correction. This is numerical
agreement for the documented scope, not a coverage calibration. Incomplete
data, derived cross-stage effects, and other unverified models are refused
with a pointer to joint latent SEM where appropriate.
dmar_metadata() is new, and refusals carry a class. A program
built on DMAR (SIEVER, a simulation runner) reads a result's estimator,
standard error method, test, missing data method, interval method,
reported version and correction, cases and groups, two-stage row counts,
auxiliary variables (the reliability family's aux record and
fa_confirmatory()'s two-stage record included), eligibility (eligible, or
qualified by an unavailable H1 model or by a two-stage fit that could
not be computed again with every case, with the text of the warning the
function gave), and bootstrap counts from one list with fixed names,
whatever family the table came from, instead of from attributes that
differ by family. Refusals a program handles are classed: an unavailable
unrestricted model (dmar_h1_unavailable, now the class of the error as
well as of the qualifying warning; inherits(cnd, "error") tells them
apart) and a two-stage fit that could not be computed again with every
case (dmar_two_stage_rows_unavailable) can be caught by name, inherit
dmar_refusal, and carry a code and their diagnostic or row counts,
so an application need not match the wording of a message; the wording
of these two did not change. The refusals of the SEM fit family are
classed the same way: an inadmissible solution (dmar_inadmissible),
a two-stage fit conditional on observed exogenous variables
(dmar_two_stage_fixed_x), and the refusals of
sem_exact_fit_bootstrap() (dmar_exact_fit_bootstrap_scope,
dmar_pattern_too_small, and dmar_transformation_undefined), each
with the fields the help of dmar_metadata() lists. The message of an
inadmissible solution names what failed (the variable with a negative
variance estimate, or the matrix that is not positive definite, with
its group and level), and the two-stage refusal says on which rows to
refit to keep the fit's estimates (and, for a fit made with
conditional.x = TRUE, to refit with conditional.x = FALSE as
well). Every other error DMAR signals is a dmar_refusal with its
own class as well; the help of dmar_metadata() lists them.
A misnamed item is refused with a class by fa_exploratory(),
fa_confirmatory(), measurement_invariance(),
measurement_alignment(), and the functions that share their
checks.
fa_exploratory(), factor_count(), and parallel_analysis() refuse
items that cannot be read from the input (a name that is not a column
of data or an item of S, a name given twice, a name that is NA,
fewer than three items, or names that are not a character vector) with
an error of class dmar_items_invalid, a dmar_refusal whose reason
element names the check that failed ("form", "unknown",
"duplicate", "count", or, for raw data whose column names repeat a
name or, with items = NULL, that have a column without a name,
"data_names") and whose items element holds the names at issue. An
item that the factors of fa_confirmatory() name (and so those of
cfa_k(), cfa_1(), cfa_2(), fa_bifactor(), and
fa_second_order()), or that the model of sem_sam() names, and that
the input lacks is the same mistake and is refused with the same class
and the reason "unknown" ("form" for a name that is NA). So is an
indicator that the model of measurement_invariance() names, whether
as a named list of factors or as lavaan syntax, and the items of
measurement_invariance() and measurement_alignment() are refused
with the class and the reasons of fa_exploratory()'s. The messages
and the calls refused are unchanged, with two exceptions:
measurement_invariance() now refuses an item named twice in items
(see Changes to Earlier Results), with the message
measurement_alignment() gives it ("'items' must not repeat an
item."); and in both functions an items that repeats a name and names
fewer than three columns, such as c("x1", "x1"), is refused for the
repeat, where it had been refused for the count. sem_sam(), new in
this release, names an indicator its data lack ("Indicators not found
in 'data': y7.") rather than speaking of an items argument it does
not have. The class is new, and a program catches the mistake by it
rather than by the wording, which differs by function.
A rung of measurement_invariance() that lavaan's optimizer does not
converge on stops with an error that names it. Through 1.0.x the
call ended with lavaan's error from its fit measures ("fit measures not
available if model did not converge"), which named no rung. Each rung
is now checked before anything is read from it, and one that did not
converge stops with an error of class dmar_not_converged, the class
fa_confirmatory() gives such a fit, whose message names the rung and
which carries it as level, with the lavaan code that reproduces the
rung's fit as syntax and lavaan_call. Three items whose
correlations in one group have a negative product, with the factor
scaled by a marker item (lavaan's default), give such a rung.
A sufficient affine-collapse diagnostic replaces the ordinary blanket
FIML case-count refusal. It uses every row observing the candidate support
and distinguishes dimension-guaranteed collapse from numerical affine
instability. EM convergence alone cannot establish a finite global H1
maximum. Analytic alpha, observed-denominator omega, FMI, and two-stage
methods require eligible unrestricted moments. Proper identified
constrained FIML estimates with H0-based errors can remain available when
H1 fails; unavailable fit indices and residual correlations are withheld,
and the reason stays in print, tidy(), and glance(). A separate
distinct-case bootstrap screen remains an explicit estimability policy.
H1-dependent bootstrap points apply the likelihood screen again to the
selected resample. Fixed/conditional-X model diagnostics screen responses
only; conditional design rank and exact Y-on-X relations remain unassessed.
The footers, warnings, and help of a fit by estimator = "ULS" state
the multiplier lavaan uses to form the standard statistic from the ULS
fit function, N - G for G groups, where they said N; the help's sample
size convention already had it. No value changed. The warning of a
ULS fit tested by the standard statistic has the class
dmar_uls_standard_statistic.
Continuous ULS now withholds all ML-derived robust indices, including CFI/TLI/RNI, consistently in fit summaries and invariance ladders. Qualified scaled diagnostics may remain; common-units invariance does not establish validity of an ML correction outside its derivation.
CFA uses estimator-matched SE defaults: standard for ML and Huber-White
for MLR (DMAR 1.0.x passed se = "standard" under every estimator; see
Changes to Earlier Results). Incompatible SE/test choices cannot silently change FIML to
listwise deletion. Its RMSEA interval keeps lavaan's 90 percent, as
DMAR 1.0.0's cfa_k(), cfa_1(), and cfa_2() reported it whatever
conf_level was, and the new argument rmsea_conf_level sets it (see
Every RMSEA Interval at 90 Percent by Default). A development version of
1.1.0 made the interval follow conf_level, .95 by default; that change,
which no release carried, is withdrawn.
Saturated models retain lavaan's RMSEA = 0, CI [0, 0], and TLI = 1 conventions. The explanation states that these are saturated-model descriptions without approximate-fit evidence or a fit-index SE; identified parameter SEs and intervals can still be meaningful.
sem_fit_measures() corrects eligible two-stage fits for auxiliary-only
cases, so estimates, SEs, tests, indices, and sample size use every case
the first stage read. The correction is recorded rather than reporting
a smaller sample size beside first-stage moments based on all rows.
The bootstrap family requires at least half of attempted replications and at least two defined estimates for each target. AVE retains its additional 100-return minimum. Ineligible intervals are withheld while point estimates and counts remain available. Defined improper estimates are retained under the documented policy. These floors do not guarantee nominal coverage. Categorical omega now defaults to 20 imputations inside each bootstrap replication; its separate point-estimate rule already uses at least 20. No new calibration of the 20-imputation default is claimed.
Reliability sensitivity takes the width-target proportion, the widths,
and the coverage over one denominator, the simulated studies that
returned an interval; a study without one is counted in the simulation
attribute (attempted, with an interval, without), where a binomial
interval describes Monte Carlo uncertainty in the width-target
proportion. (DMAR 1.0.0 took the proportions of intervals that missed the true
value over all attempted studies; see the entry on the record of the
simulation.)
Simultaneous-interval sentences use only a matching multiplicity-adjusted
p-value and explicitly name it, or explain that no matching p-value is
available. Equivalence sentences say TOST p. Correlation and coefficient-
of-variation intervals accept supplied tail areas when conf_level was
omitted and refuse an explicitly inconsistent level; one-sided CV bounds
respect the parameter's zero boundary.
The eight score columns of factor_scores() carry the prefix
.derived_ (.derived_regression, .derived_regression_se,
.derived_regression_ci_lower, .derived_regression_ci_upper, and the
same four for .derived_bartlett), so that a score joined to the
observed data cannot be taken for an observed variable or collide with
one. The attribute derived_columns names them by what each holds.
Valid regression conditional means are retained when a factor-disturbance
relation with an observed conditioning variable invalidates Bartlett's
conditional unbiasedness; the Bartlett quantities are then unavailable
with a reason. Ordered models outside the verified score likelihood retain
their explicit refusals.
Every indefinite supplied covariance matrix is refused after unit scaling. A general loading fixed or constrained to zero is treated like an omitted loading by the bifactor-index eligibility rule. General CFA/SEM can fit such a model, but the current bifactor indices do not apply.
Welch simple effects retain their valid F tests and withhold the inherited partial-eta-squared interval: its estimand does not match the reported ordinary variance-ratio estimate. Parallel population plans require the supplied population to satisfy parallel-model restrictions. Congeneric complete-data plans below items + 1 are raised to that feasibility minimum before missing-information inflation, with an assurance-planning warning.
A factor analysis or structural equation model fit whose likelihood
has no maximum stops. lavaan lets an error variance fall below zero,
and for some models and data the likelihood then has no maximum: it
keeps rising, by less and less, as one item's error variance falls
without bound and its loading grows to match, and lavaan's optimizer
stops where its steps stop gaining and reports convergence, at a point
its starting values and tolerance set rather than the data. When an
error variance is negative, DMAR now fits the model again with that
variance held further out (at ten times its estimate, or one model
implied variance below it, whichever is further; with several, all of
them together and then each alone), and when the log likelihood there
is higher, or lower by no more than 1e-6 per case (a likelihood that
flat leaves the estimate as much to where the optimizer stopped as a
ridge does), it stops, for every output (output = "fit" included),
with an error of class dmar_no_maximum
(a dmar_refusal). The message names the function and the item, says
what the fit further out found, says that DMAR fits the model with
lavaan, the estimation engine, and to see lavaan for more details,
where bounds = "pos.var" or a different model are the choices, and
prints the lavaan code that reproduces the fit (the model syntax DMAR
built, less the lines that define its indices, and the call, with a
placeholder for the data that selects the items where lavaan would
otherwise read every variable it is given); a model whose syntax would
carry the message past the 8,170 bytes R prints of an error has the
call printed and the syntax left in the condition. The condition
carries the syntax, the call, the items, their error variances, the
two log likelihoods, the tolerance, and the number of cases. A finite
negative estimate, at a maximum of the likelihood, is reported as
before: the nine-test bifactor model of fa_bifactor()'s help gives
visual perception an error variance of -41, and the likelihood falls on
either side of it, while a fourth group factor on the Woody-McCall,
revised paper form board, and flags tests sends the error variance of
paper form board to -1163 under FIML, where the likelihood is still
rising, and that model now stops. The check covers fa_confirmatory()
(with cfa_k(), cfa_1(), and cfa_2()), fa_bifactor(),
fa_second_order(), fa_exploratory() (whose refusal is also a
dmar_efa_no_solution, with the reason "no maximum", which
factor_count() reports as that count's status),
measurement_invariance() (naming the rung and the group), the
configural fit of measurement_alignment(), and the single-factor fits
of reliability_omega() and reliability_H() (the tau-equivalent fit
of reliability_alpha() has a maximum, since its equal loadings keep
one item's error variance from running off). It applies to maximum
likelihood fits; least squares and WLSMV fits have no likelihood and
are not checked. A solution with no negative error variance costs
nothing more; one with a negative error variance costs a refit, or,
with several, one with all of them held and then one for each until
one shows a ridge (0.03 s for the nine-test model above, and 0.08 s
for the model with the fourth group factor, whose own fit takes 5 s).
Inside a bootstrap (the reliability family's, ci_bifactor_indices()'s,
and that of reliability_H() on a fit), a replication with no maximum
is set aside and counted among the replications that raised an error,
under a message that names the item, and the run goes on. The engine
fits each such replication a second time to recover its message, with
the second fits spread over the same ncpus processes as the
replications, and a fit that runs off is among the slowest lavaan
makes: on one process, 100 replications of the 15-test bifactor model
of fa_bifactor()'s help, 8 of them with no maximum, took 143 s,
against 99 s without the check. The warning of fa_bifactor() about
an improper solution names sampling error, a model that does not
describe the items, and a group factor the data barely separate from
the general factor as the possible causes, where it named the last
alone.
The structural equation models DMAR fits with lavaan are checked the
same way. The fit of the analysis model to the population moments,
from which ss_aipe_composite_sem(), ss_power_composite_sem(),
ss_aipe_sem_path(), and ss_aipe_sem_path_sensitivity() (to
supposed_cov_matrix, and to true_cov_matrix, now fit before any
replication is
drawn) read their population values, stops the planner when it has no
maximum, with the code that fits the model to the matrix given (or to
cov_sem(true_model)$sigma_theta) and the model printed whole, its
:= lines included, since they define the user's own quantities. So
does the lavaan fit from which mediation_mbco() starts its OpenMx
fits. Its code fits the summary statistics directly when they are one
group's (a covariance matrix given as a data frame converted by
as.matrix()); for several groups, or a model whose product terms the
function writes into columns, it fits the data the function builds
from them, which the message describes exactly, seed included, since
the moments of a product column depend on the rows drawn; and a lavaan
fit given as model is refused without code. The message names a
different model as the choice, since the OpenMx model carries no
bounds. A replication of those planners' Monte Carlo with no maximum,
which lavaan reports as converged where its optimizer stopped, is set
aside as one that did not converge, and a fresh sample is drawn in its
place. ss_aipe_rmsea_sensitivity() and sem_exact_fit_bootstrap()
read only the chi square of a fit, and are not checked: along a ridge
the likelihood levels off, so the chi square, unlike the estimates,
does not depend on where the optimizer stopped (in 60 bootstrap
samples of a factor with four weak indicators, the 7 that ran off had
statistics of 0.37 to 3.96 that would move by 0.00005 to 0.0011
further out), and setting such a fit aside would change the reference
distribution, or refuse a population RMSEA that is determined. A
bootstrap sample with no maximum therefore stays among the improper
ones, counted or set aside as improper says. In a structural model,
lavaan carries an observed variable that enters a regression as a
latent variable of its own: an outcome, a covariate, or an indicator
that also enters a regression (with a direct effect on it, for
example). The variance of such a variable is not held and its loading
is not moved, so a ridge on such an indicator, or on an outcome
regressed on a latent variable, is not looked for; on the ridges of
that kind tried, lavaan reported no convergence, where the check does
not apply. A ridge on which lavaan reports no convergence, as it often
does with a marker item fixing the scale, was already a replication
that did not converge. The examples of these pages, none of whose fits
has a negative error variance, take no longer: the check reads each
fit's parameter table, 5 to 15 microseconds a fit (21 fits in each
composite planner's examples, a fraction of a millisecond), and refits
only a fit with a negative error variance, which in a Monte Carlo at a
small N costs one refit for each replication that has one.
reliability_omega_categorical() gains a denominator argument.
"observed" (the default, and the only coefficient the function
computed before) builds the variance of the summed categories from the
unrestricted polychoric correlations, as Kelley and Pornprasertmanit
(2016, Eq. 18) did; it is the categorical counterpart of robust omega.
"model_implied" builds it from the latent-response correlations the
fitted single-factor model reproduces, which is categorical omega as
Green and Yang (2009) defined it: their appendix program takes "the
reproduced (model-based) correlation matrix" as the input to the
denominator. The help page had attributed the unrestricted denominator
to Green and Yang's Eq. 21; it now states which paper used which.
Every interval method applies to both settings, and the bootstrap
recomputes the coefficient with the chosen denominator on each
replication. reliability() forwards denominator for type = "omega_categorical" as it already did for type = "omega". The model
implied setting needs no unrestricted polychoric correlation, but its
loadings rest on the correlations the data do determine. Those
identify the loadings only when they connect every item to the others,
directly or through other items, and include at least one cycle of an
odd number of items (three items whose three correlations are all
determined, for example). When they do not, the loadings are
arbitrary. In 13 of 60 samples of 300 from Green and Yang's mixed-skew
design (the help page's example with N <- 300 and seeds 1 to 60 in
place of 113) they did not, and in 2 of those lavaan's start values
gave loadings of opposite signs and a categorical omega of 0.05 and
0.09, against a population value of 0.716. Both settings then stop
with an error naming the undetermined pairs and either the blocks of
items no determined correlation joins or, within one block, the two
sets of items between which every determined correlation runs. A
bootstrap replication or a completed data set in that state is set
aside and counted as undetermined. The argument changes no existing
result; what the observed setting now does with a correlation the data
do not determine is under Changes to Earlier Results.
bifactor_indices() gains a denominator argument. The default,
"model_implied", divides by the variances the fitted bifactor model
reproduces (Rodriguez, Reise, & Haviland, 2016), now including any
residual covariances the model specifies (see Changes to Earlier
Results); "observed" divides omega, omega_H, and omega_HS by the
composite variances estimated from the data, the denominator of
robust omega carried to the bifactor coefficients. ECV, PUC, and H do
not depend on the setting. The observed setting requires a
single-group fit. For a fit to ordered items both settings describe
the sum of the latent responses behind the categories, not the summed
categories; a new metric attribute ("observed" or
"latent_response") records which, and the printed line names it.
Every omega now names its denominator and reports its local
dependence. The printed tables of reliability_omega(),
reliability_omega_categorical(), and bifactor_indices() state in
one line how the total variance in the denominator was estimated, so a
reader need not know the default to know which coefficient is on the
page. reliability_omega() and reliability_omega_categorical() also
carry a residual_cor attribute (the residual correlations of the
single-factor fit, the largest of them, the item pair it belongs to,
and whether the fit is saturated) and print the largest under the
table. The residual correlations come from a maximum likelihood fit
whatever the interval method (the weighted least squares fit behind
ci_method = "adf" has residuals of its own and is not used). With
auxiliary variables, under FIML or two-stage estimation, the residual
correlations are those of the items alone, under the items' own names.
With three items the single-factor model is saturated and a proper
solution reproduces every correlation, so the attribute's saturated
element is TRUE, no pair is named, and the printed line says so. An
observed denominator protects the denominator, not the numerator: a
doublet or a method factor shared by a few same-signed items raises
their loadings, and omega with them, under either setting while
leaving the two denominators close. The residual correlations are
where such structure shows, although when it sits on the items that
dominate the factor the single factor absorbs it and the misfit
appears among the other items. No row of any table changes; the
additions are attributes and printed lines.
In a bootstrap, the model implied setting sets aside only the
replications whose loadings are not identified or whose solution is
improper. A resample often leaves out the few respondents who
determine a polychoric correlation, so a bootstrap meets undetermined
correlations far more often than the sample does (see Changes to
Earlier Results for the observed setting). denominator = "model_implied" needs no unrestricted polychoric correlation, so it
sets aside a replication only when the correlations left determined do
not identify the loadings or, as the observed setting does, when its
solution is improper. That was 1 of 200 in a percentile bootstrap of
the help page's complete data, of 1,000 respondents (seed = 1). In
the 45 of the 60 samples of 300 above whose point estimate is
identified and proper, a median of 42% of the replications was set
aside (B = 50, seed = 1), and more than half in 15 of them.
Help pages name the denominator each paper used. fa_confirmatory() and
cfa_1() described the categorical omega that fa_confirmatory() reports for
ordered factors as Green and Yang's (2009); it is their assembly with
the unrestricted polychoric denominator of Kelley and
Pornprasertmanit (2016, Eq. 18), the default of
reliability_omega_categorical(), and the pages and the message now
say so. The example of reliability_omega_categorical() now
reproduces Green and Yang's mixed-skew design and sets their
coefficient beside the one Kelley and Pornprasertmanit recommended.
The categorical omega assembly is checked against Green and Yang's (2009) population values. A new test reproduces ten entries of the nonlinear SEM column of their Table 1. Nine agree to the third decimal; the tenth (binary items, two negatively and two positively skewed, loadings of .8; published .591) computes as 0.5904.
The reliability bootstrap refits by maximum likelihood. Every
replication of robust omega, of a bootstrap interval for the model
implied omega, and of the model implied reliability_alpha() refit
the single-factor model by robust maximum likelihood, whose estimates
are those of maximum likelihood; only the standard errors and the test
statistic differ, and a replication reads neither. The replications
now refit by ML, with the same estimates to the last bit, so every
interval is unchanged for a given seed, and the bootstrap is faster:
the scaled test statistic had cost 1.3, 10, and 37 times an ML fit at
9, 24, and 40 items, and a 40-replication bootstrap of robust omega on
24 items now takes 1.5 s rather than 7.1 s. Robust omega also fit the
model twice on every call, once for the estimate and once for the
residual correlations, and now fits it once.
Every replication is counted where parallel workers keep the count,
and the draws are returned. The bootstrap attribute gains
improper_kept, not_converged, too_few_cases, improper_rule,
caution, require_half, ncpus, t (the estimate of every
replication), and status (its fate: proper, improper, error,
non_finite, undetermined, or not_converged), and the result gains the
B_used attribute that the package's other bootstraps carry. No
element changes meaning; not_converged is zero for the reliability
coefficients, whose replications still count a fit that did not
converge among the non-finite ones, and is used by
ci_bifactor_indices(). Through 1.0.1 the counts were kept where a
forked worker could not report them, so a session with
options(boot.parallel = "multicore") lost every count.
ncpus spreads the replications over several processes.
reliability_omega(), reliability_alpha(), reliability_kr20(),
reliability_omega_categorical(), reliability_H(), and
reliability() gain ncpus = 1L: forked workers where the platform
allows them and a socket cluster on Windows, with the jackknife behind
a BCa acceleration on the same workers. boot draws every resample
before any replication runs, so a given seed returns the same
interval, counts, and draws whatever ncpus is.
progress follows a bootstrap and can stop it.
reliability_omega(), reliability_alpha(), reliability_kr20(),
reliability_omega_categorical(), reliability_H(), reliability(),
and ci_bifactor_indices() gain progress = NULL, their last
argument: a function that the bootstrap calls as
progress(done, total), with the replications attempted so far and
B, after each replication when ncpus = 1 and after each block of
ncpus * ceiling(B / (50 * ncpus)) replications (about a fiftieth of
B) on several processes. An application that runs a long bootstrap,
a Shiny app above all, can show how far it has gone and let its user
stop it: when the callback returns FALSE, the bootstrap stops before
the next replication or block with an error of class
dmar_bootstrap_cancelled that carries done and total, and any
other value is ignored. A BCa acceleration comes from the jackknife
when fewer replications than cases return a value or the regression
on the resampling frequencies fails, and while the jackknife runs the
callback is called after each leave-one-out refit, or each block of
them on several processes, with done and total both B, so a stop
is honored there too, and done equal to total does not mean that
the bootstrap has finished; the bootstrap then impute replications of
categorical omega under missing = "mi" report the same way. An error
inside the callback stops the call with an error of class
dmar_progress_error that carries done, total, and the callback's
own condition, its class intact, as parent. The callback runs
outside the random number stream, so a given seed returns the same
interval, counts, and draws with or without it, and the default NULL
runs the bootstrap exactly as before. ?dmar_metadata lists both
classes beside the refusals. sem_exact_fit_bootstrap() and
average_variance_extracted() have no progress argument in 1.1.0:
their bootstraps do not run through this engine, so an application
that calls them cannot follow or stop them this way.
improper = c("keep", "drop") for the single-factor coefficients.
reliability_omega(), reliability_alpha(), and reliability() gain
improper. A bootstrap replication whose solution was improper (a
negative error variance) but still yielded a coefficient was kept and
not counted; it is now counted, kept by default, and set aside under
"drop". Keeping is the default because setting such replications
aside conditions the bootstrap distribution on admissibility and trims
its upper tail; in a small coverage check (model implied one-factor
omega, five items with loadings .95, .70, .60, .50, and .40, N = 60,
120 data sets per condition, B = 300), percentile intervals covered at
.958 when they were kept and at .933 when they were dropped, a
difference near the Monte Carlo standard error of about .02. Under
"drop" an improper point estimate stops when a bootstrap is
requested. When the replications set aside and the improper ones kept
reach the tail area (1 - C)/2 of B, the warning and the printed line
add a caution that the limits depend on how they were treated.
Categorical omega and H are not defined for an improper solution, so
they set such replications aside, and both record improper_rule = "drop".
Replications of three items with no single-factor solution are
counted on their own. The bootstrap attribute carries
no_solution: the replications set aside because the correlations of
three items had a product that was not positive, which no loadings
reproduce. It is a subset of improper, reported under the status
"improper", and the warning and the printed line name these
replications apart from the Heywood cases ("had no single-factor
solution"). reliability_H() from data counts them the same way, and
its warning says that setting aside a Heywood case can trim the upper
tail of the bootstrap distribution while one with no single-factor
solution belongs on neither side. The element is zero for categorical
omega, the bifactor intervals, and H from a fitted model, whose refits
are not checked for it, as the help of
reliability_omega_categorical(), reliability_H(), and
ci_bifactor_indices() says; the categorical page defines returned
and non_finite as reliability_omega()'s page does. The warnings
describe an improper replication by its criterion, an error variance
below zero (a standardized loading past one; on the items' own metric
a loading above one is routine in a proper solution), and the
bootstrap then impute warnings of categorical omega describe an
improper replication or completed data set the same way. A replication
with no single-factor solution is set aside under either rule: on the
estimator's scale it continues the top of the bootstrap distribution,
since one loading, and omega with it, grows without bound toward that
boundary, while its correlations point to low reliability, so it
belongs on neither side. When some are set aside, the caution adds
that the lower limit is likely too high. In a coverage check of the
three-item estimator (three items of unit variance with loadings .70,
.60, .30 or .775, .645, .155; 500 samples of N = 60 drawn after
set.seed(1); percentile intervals from B = 999 resamples), the lower
limit exceeded the population omega in 11.6 and 13.1 percent of the
samples that yielded an estimate (413 and 352 of the 500), against the
nominal 2.5.
The analytic reliability_alpha() records the improper rule it
was given in its bootstrap attribute rather than the engine's
default. It fits no model, so no replication can be improper.
The bootstrap reports no interval when most replications fail the
distinct-case rule. The standard error and limits are NA when
more than half of the replications have too few distinct cases for
the coefficient. Near that rule's bound, the resamples kept are
selected for repeating the fewest rows; at the bound they are only
reorderings of the data and give an interval with no width. The
bootstrap attribute counts these refusals in too_few_cases,
including in its jackknife record. This rule applies throughout the
engine. The separate require_half rule withholds an interval
whenever fewer than half return a value, whatever the reason; it
holds for every caller (the element require_half of the bootstrap
attribute is always TRUE), as the entry on the bootstrap
family's half-return requirement states. The number of replications
a request must ask for is
set by the rule of the next section, One Rule for Bootstrap
Replications. A bias estimate for H is
reported only when its interval is formed.
A bootstrap warning describes limits only when they exist. With fewer than two returned values, or when either rule above withholds the interval, the warning and printed line say that no interval is reported and give no caution about its limits. The caution names an improper solution only when one was kept. A refusal quoted with its count loses its trailing period inside the parentheses.
An unobserved column in a replication is named separately from a
constant one. No observed value leaves its variance and covariances
unidentified; a single value leaves its variance zero. Both are
counted and quoted by name. A bootstrap of reliability_H(fit = )
follows the fit's own missing data method: complete-data and listwise
fits use the rank of the sample covariance matrix, FIML and two-stage
fits use the saturated model's case rule, and pairwise fits use
neither case-count rule. Its messages describe the resample without
claiming that the supplied fit's data were refused.
B
replications and is located by the replications that lie beyond it.
An interval read from quantiles is now run only when B puts at least
10 replications beyond each limit, and never with fewer than 100: 200,
400, and 2000 replications at the 90, 95, and 99 percent levels. A BCa
interval, whose limits the bias correction and the acceleration move
farther into the tails, is run only with 50 beyond each nominal limit:
1000, 2000, and 10000. An interval built from the standard deviation
of the replications ("bootstrap_se", "bootstrap_se_logistic") needs
100, and so does a Monte Carlo p-value. A one-sided bound and a
reference quantile need 10 beyond their one quantile (200 at the 95th
percentile). A B below the number required is refused before
anything is computed, with an error of class dmar_count_invalid
whose message states the number and the count behind it:
mediate(d, "x", "m", "y", B = 200), the call of the 1.0.x help
example, returned 0.1816 [0.1163, 0.2684] and now stops with "'B' must
be a single whole number of at least 400. A 95% interval read from the
quantiles of the replications has 2.5% of them beyond each limit, and
an interval is reported only when at least 10 replications lie beyond
each limit (10 / 0.025 = 400). To run fewer for exploration or a
demonstration, see ?dmar_bootstrap."; with B = 400 it returns
0.1816 [0.1165, 0.2592], the numbers DMAR 1.0.0 returned for
B = 400. The numbers of the rule were computed for DMAR (the tail
area a limit cuts off has a relative standard deviation of about 30
percent with 10 replications beyond it, whatever the shape of the
bootstrap distribution; at 400 replications a 95 percent limit of a
bell-shaped distribution varies from run to run by about 0.13
bootstrap standard errors); no published source is cited for them,
and the new help page ?dmar_bootstrap gives the reasons.mediate() (percentile, BCa, and Monte Carlo intervals; the Sobel
interval draws nothing and B is not read for it), mediation_mbco()
and plot_mediation_mbco() (Monte Carlo), reliability_alpha(),
reliability_kr20(), reliability_omega(),
reliability_omega_categorical(), reliability(), reliability_H(),
ci_bifactor_indices(), average_variance_extracted(),
fraction_missing_information(), cohen_kappa(), fleiss_kappa(),
krippendorff_alpha(), htmt(), ci_ratio_of_means(),
ci_ratio_of_means_paired(), R2_mixed_effects(),
ci_eta_squared_generalized(), mlmr(), mlmr_mv(),
parallel_analysis(), factor_count(), and
sem_exact_fit_bootstrap(). Before, a B was accepted from 2 in the
reliability family, ci_bifactor_indices(), parallel_analysis(), and
sem_exact_fit_bootstrap(); from 100 in mediate(),
mediation_mbco(), the kappa functions, krippendorff_alpha(),
htmt(), the ratio functions, average_variance_extracted(), and
fraction_missing_information(); and was not checked at all in
R2_mixed_effects(), mlmr(), and mlmr_mv(). A fractional B was
accepted and truncated by cohen_kappa(), fleiss_kappa(), and
average_variance_extracted() (cohen_kappa(B = 250.5) ran 250
replications) and is now refused. mediation_mbco() required a B of
at least 100 whatever the interval and now reads B only for
ci_method = "monte_carlo". ci_eta_squared_generalized() keeps its
own minimum of 1000, which the rule raises to 2000 at the 99 percent
level. htmt() needs 200 at the default level (100 to 199 were
accepted), parallel_analysis() and factor_count() 200 at the 95th
percentile (factor_count(B = 0) still omits the reference), and
sem_exact_fit_bootstrap() 100. The planners that run an interval
inside each simulated study follow the rule too:
ss_aipe_indirect_effect() and
ss_aipe_indirect_effect_sensitivity() hold B to it under
method = "monte_carlo" (400 at the 95 percent level, where 100 was
accepted by the planner and the sensitivity function made no check;
under method = "closed_form", which draws nothing, B need only be
a whole number), and the default B of
ss_aipe_reliability_sensitivity() is now NULL, resolved to the
larger of 200 and one and a quarter times the number the interval
method and the level require: 500 for "percentile", 2500 for
"bca", and 200 for the two bootstrap standard error methods at the
95 percent level, where the default was 200 for all four. The quarter
is room for replications that return no value: the number required
is held to the request, so a simulated study that loses a few
replications still reports its interval, and with the quarter its
limits still rest on about the number required. Its run time is
proportional to G times B, so a percentile check takes about two
and a half times as long by default and a BCa check twelve and a half
times. With the exploratory option the three planners give one warning
for the call, not one for each simulated study, and their results
carry the attributes B, B_used, n_dropped, B_required, and
bootstrap_status, which record the request each simulated study ran.B = 1000 of average_variance_extracted(),
fraction_missing_information(), krippendorff_alpha(),
ci_bifactor_indices(), R2_mixed_effects(), mlmr(), mlmr_mv(),
and of reliability_omega_categorical() under missing = "mi" serves
the 90 and 95 percent levels and not conf_level = 0.99, which needs
2000; the default of ci_bifactor_indices() does not serve
ci_method = "bca" (2000 at 95 percent); and the default B = 2000
of mediate() serves a 99 percent percentile interval exactly and not
a 99 percent BCa interval (10000). In each case the refusal names the
number to pass.mediation_mbco() (a Monte Carlo draw
at which an effect is not a finite number), R2_mixed_effects(),
ci_eta_squared_generalized(), mlmr(), and mlmr_mv() (a refit
that failed or a value that is not finite). They now give one warning
of class dmar_bootstrap_replications_dropped, as the others do.
R2_mixed_effects(), mlmr(), and mlmr_mv() removed a missing
value for one statistic only, so the intervals of one call could rest
on different replications; a replication without a value for any
statistic is now dropped for all of them (see Changes to Earlier
Results). The warnings that were unclassed now carry the class, and
every one carries the fields B, B_used, n_dropped, B_required,
share_dropped, caution, and status. When the dropped
replications number at least the share of B beyond one limit (2.5
percent at the 95 percent level) the warning adds a caution, the rule
the reliability family already applied, because they could then
decide by themselves where that limit falls.B is held to the count before anything
is computed, and that is the whole of the count rule. A request that
meets it and then loses replications still reports its interval: the
warning states how many were dropped, and when those that remain are
fewer than the number required it says so and says to request more
("The 400 required are counted on the number requested (400), not on
the number that remained; the 397 that remained are fewer, so request
more bootstrap replications where some can fail, enough that at least
400 remain."). The two rules about shares then decide: more than half
dropped withholds the interval, and dropped replications that reach
the share of B beyond one limit bring the caution. A constrained
bifactor model of six tests of holzinger_swineford with
ci_bifactor_indices(B = 400, seed = 63) and one improper replication
set aside reports the intervals of its 15 indices from the 399 that
return a value. Where replications can fail, ask for more than the
minimum; the defaults do at the 95 percent level.reliability_alpha(), reliability_kr20(),
reliability_omega(), reliability_omega_categorical() (with and
without missing = "mi"), reliability_H(), and
ci_bifactor_indices() share one bootstrap engine, whose warnings
began with the count ("3 of 400 bootstrap replications were set aside
...") and now begin with the function ("reliability_omega(): 3 of 400
bootstrap replications were set aside ..."), as the warnings of the
other functions under the rule do. The condition carries the name in
the field fun. reliability() passes its call on, so its warning
names the function that ran the bootstrap.average_variance_extracted(),
parallel_analysis(), and sem_exact_fit_bootstrap() name the
function too. These three compose their own account of a run (the
factors, the reference data sets, the reasons a sample was set aside),
and it began with the count ("2 of 200 bootstrap replications were set
aside ..."). It now begins with the function
("average_variance_extracted(): 2 of 200 ..."), so every warning under
the rule says where it came from. A note that factor_count() prints
from its parallel analysis begins "parallel_analysis(): ".mediation_mbco() and plot_mediation_mbco() draw the estimates from
their estimated sampling distribution. Their caution and their note on
an exploratory run spoke of "replications" and of "the bootstrap
distribution"; they now speak of draws and of the distribution of the
draws, as the rest of each message does.NA, the estimate and every other row are returned, and the warning
has the new class dmar_bootstrap_insufficient. mediate(),
cohen_kappa(), fleiss_kappa(), krippendorff_alpha(), htmt(),
and the two ratio functions stopped with an error when fewer than 100
replications remained, and fraction_missing_information() when fewer
than two did. With a predictor that is nonzero in one of 20 cases,
mediate(B = 150, seed = 113) stopped with "Only 91 of 150 bootstrap
replications returned a finite indirect effect"; the same call with
B = 400 now returns the indirect effect 0.5959 with limits from the
240 replications that remain and the warning that 240 is fewer than
the 400 required, and with B = 2000 it
returns [0.1319, 0.9613] from 1267 replications, as before.B (requested), B_used (the number the interval
rests on), n_dropped, B_required, and bootstrap_status
("complete", "dropped", "exploratory", "insufficient_count",
or "insufficient_share") are on every such result;
cohen_kappa(), fleiss_kappa(), htmt(), krippendorff_alpha(),
and average_variance_extracted() had no B attribute, and
mediation_mbco(), R2_mixed_effects(),
ci_eta_squared_generalized(), parallel_analysis(), and
factor_count() recorded no counts under these names. mlmr() and
mlmr_mv() carry them as elements of the fitted object, and print()
and summary() state them in one line. The rows B and B_used of
mediate() and the row B_used of krippendorff_alpha() are kept.
The bootstrap attribute of the reliability family keeps every
element and gains required, bootstrap_status, and B_used;
sem_exact_fit_bootstrap() gains B_required and bootstrap_status
in its sem_fit list. dmar_metadata() returns required and
status in its bootstrap element. The line under a table of the
ratio functions reads "Bootstrap: 2000 replications, 1283 used (717
dropped; at least 400 must be requested). A dropped replication had a
denominator of zero"; mediation_mbco() prints the corresponding line
for its Monte Carlo interval, and the tables of mediate(),
cohen_kappa(), fleiss_kappa(), krippendorff_alpha(), htmt(),
R2_mixed_effects(), and ci_eta_squared_generalized() print
"Bootstrap: 400 replications, 400 used (0 dropped; at least 400 must
be requested)" beneath them when a bootstrap was run.B puts fewer than 10 replications beyond one of
them, no BCa interval is reported (limits NA, class
dmar_bootstrap_insufficient, with the positions as positions and
a suggested B as B_suggest). This applies to mediate(), the
kappa functions, the ratio functions, and the reliability family. In
the probe behind the rule this happened in at most 14 percent of data
sets at B = 2000 (an indirect effect with 50 cases and small paths)
and in 0 to 3 percent of the other conditions; the positions asked for
more than 10,000 replications in about 1 percent of the data sets of
that hardest condition (4 of 400, the largest count 32,909) and in
none of the others. The count the positions take is what the result
records as B_required and what the warning carries, in every family;
the reliability family recorded the 2,000 of the request beside a
message that stated the larger count, and printed no reason under the
table, and now records and prints the count at the positions
("5105 are required where the bias correction and the acceleration
place the limits, so no BCa interval is reported").ci_ratio_of_means() or
ci_ratio_of_means_paired() that the rule withholds prints
"Percentile bootstrap interval: no interval is reported." in place of
a line that called the missing interval "the whole line", and its
confidence_set attribute and tidy() column are NA. The messages
call a one-sided bound a bound and the reference quantile of
parallel_analysis() and factor_count() a reference quantile; a BCa
interval with unequal tail areas is refused with a sentence that names
the smaller nominal tail area, where it named two positions the
request did not have; factor_count() gives the exploratory mark
under its own name; parallel_analysis(percentile = 50) and
factor_count(percentile = 50) stop with a message that names
percentile, where it named conf_level. A warning about a run in
which nothing was dropped (in the reliability family, a BCa interval
replaced by the percentile interval, or replicates that are all the
same value; in average_variance_extracted() and
parallel_analysis(), the function's account of improper
replications kept or of reference fits that warned) was unclassed and
has the class dmar_bootstrap_note. The B attribute of the
reliability family is an integer, as the other counts are (it was a
double when B was given as one). In an exploratory run below the
required number the reliability family no longer passes on
"extreme order statistics used as endpoints" from boot::boot.ci()
beside its own warning, which already says how few replications each
limit rests on, and mlmr() and mlmr_mv() no longer pass on
lavaan's count of bootstrap runs that failed beside the classed
warning that reports the same replications with the counts.
plot_mediation_mbco() returns a plot, which carries no attributes,
and its warnings no longer say that the counts are recorded in the
result.cohen_kappa() and fleiss_kappa() refuse a BCa interval that is
not defined. The acceleration needs kappa with each case left out.
A leave-one-out kappa that is undefined (chance agreement 1) was
removed without a word and the acceleration computed from the rest.
The call now stops with an error of class dmar_bca_undefined (a
dmar_refusal) that names the cases, as mediate() does; the same
class is given when every bootstrap estimate falls on one side of the
sample estimate or the acceleration is not finite.
ci_method = "percentile" does not need these quantities.options(dmar.bootstrap_exploratory = TRUE) runs fewer
replications, and marks the result. A first look, a help page
example, and a simulation over many data sets do not need a
reportable interval from each run. Under the option a B from 2 up is
accepted; the result records B_required and has
bootstrap_status "exploratory", the line under the table says that
the interval is below the number required for a reported interval,
and every call gives a warning of class dmar_bootstrap_exploratory.
The share rule still applies. The examples of R2_mixed_effects(),
mlmr(), mlmr_mv(), ci_bifactor_indices(), reliability_omega(),
and sem_exact_fit_bootstrap(), whose replications each refit a
model, set the option, run their small B, and restore it; the
examples of mediate() and krippendorff_alpha() (now B = 400),
of parallel_analysis() and factor_count() (B = 200), and the BCa
example of reliability_alpha() (B = 2000) meet the requirement.
The vignettes missing_data (six intervals, B = 200 before) and
DMAR (factor_count(), B = 50 before) run the required numbers.sem_exact_fit_bootstrap() gives one warning about the samples it
sets aside. It gave two when no p-value was formed, one for the
samples set aside and one for the missing p-value; it gives one,
with the class that says which case it is.?dmar_bootstrap. It states the rule and its
reasons, what is recorded and where, the treatment of BCa intervals,
the exploratory option, the conditions with their fields, and a table
of the functions with their kind, default B, and the number required
at the 95 percent level.reliability_H() computes coefficient H from a fitted model, raw
data, or a covariance matrix, and its interval now uses the full
covariance matrix of the estimates. In 1.0.0 the function took
standardized loadings and, for an interval, their marginal standard
errors, the diagonal of the covariance matrix of the loading
estimates. Those estimates are correlated in any fitted factor model,
so that interval could be too narrow or too wide: on one-factor ML and
MLR fits (std.lv = TRUE) of seven blocks of holzinger_swineford
tests (1, 2, 4; 6, 7, 9; 10, 12, 13; 1--4; 5--9; 14--19; and 20--24)
the standard error from the full matrix is 0.75 to 1.29 times the one
from the diagonal under ML and 0.69 to 1.16 under MLR. H is now
computed from the fitted loadings, residual covariances, and factor
variance. It does not depend on how the factor was identified, and
when residuals inside the block are correlated it is the maximal
reliability of Gabler and Raykov (2017). Its delta method variance
runs over every free parameter with the fit's own covariance matrix of
the estimates (standard, robust, FIML, or WLSMV), under either
ceq.simple setting of a fit whose equality constraints come from
labels or group.equal, and it reproduces the H_<factor> row of
fa_confirmatory() and lavaan's own delta method for H defined with :=. The
new arguments come after the 1.0.0 ones: fit (any factor of a lavaan
confirmatory or structural model measured by observed items, named
with factor, and any group, named with group); data (the
single-factor model fitted as reliability_omega() fits it, with
missing and aux); S and N; vcov_loadings; ci_method; B;
seed; and ncpus. The default interval is the Wald interval on the
logit scale, with the robust maximum likelihood standard error for
data, the maximum likelihood one for S, the fit's own for fit,
and for loadings the one their vcov_loadings (or se_loadings)
gives; standardized loadings with neither carry nothing an interval
can be formed from and give the estimate alone (ci_method = "none"),
with a line under the table saying what an interval needs. The four
bootstrap methods refit the model on every replication, and their
bootstrap attribute carries the estimated bias of the sample H,
which runs upward (Aguirre-Urreta, Rönkkö, & McIntosh, 2019). On the
fit path the model is refitted to the fit's own rows before any
resample is drawn, and the call stops unless that refit reproduces the
estimate; every refit starts from the fit's estimates. A fit with
sampling weights or a cluster variable, which a resample of rows cannot
carry, is refused, and the delta method interval, which uses the fit's
own standard error, is named in its place. The refits of a fitted
model are not checked for three items whose correlations admit no
single-factor solution, so such a resample
is counted as improper when lavaan stops at a solution that has run
away and as non-finite when it does not converge; from data it is
counted in improper and in no_solution, as reliability_omega()
counts it. A fit for which H is not defined stops with the remedy
named: an item that also loads on another factor or is regressed on
another variable, a higher-order factor, an observed covariate
predicting the factor, an improper solution, or three items whose
correlations admit no single-factor solution. A loading or regression
fixed or constrained to zero is not one: a model written with
0*item, or with a*item and a == 0 (directly or through a chain
such as a == b with b == 0), gets the H of the model written
without it. A bifactor fit, whose items load on more than one factor,
is refused with a message that names the index bifactor_indices()
reports, its maximal_reliability attribute, and
ci_bifactor_indices(). Every call but the 1.0.0 form (see Changes to
Earlier Results) returns the reliability family table (estimate,
se, ci_lower, ci_upper, conf_level, N, N_complete,
n_items; class dmar_reliability, coefficient "H"). Attributes name the
factor, the group, the metric ("latent_response" for ordered items),
the basis of the standard error, what came with standardized loadings
(vcov_supplied), and the standardized loadings, and the printed
table says which factor of a model it describes. reliability(type = "H") dispatches to it; auto-detection never chooses it. From data, a
fit that lavaan stops with an error is reported with lavaan's message,
as reliability_omega() reports it.fa_bifactor() is new. It fits the bifactor model, a general
factor on every item and two or more group factors, each on its own
items, with every factor uncorrelated with the others and its variance
fixed at 1, described as fa_confirmatory() describes a model: the
items of each group factor are named in factors, the general factor's
name in general, and items on the general factor alone in
general_only. It shares fa_confirmatory()'s engine, interface, and
fit rows: raw data (by full information maximum likelihood, the
default, or after listwise deletion) or a covariance matrix with its
N, ML or MLR, equal_loading, equal_error, and equal_intercept
within a factor, and the five outputs, "fit" returning the lavaan
fit that bifactor_indices(), ci_bifactor_indices(), and
factor_scores() take. Beside the loadings, error variances, and
intercepts the table reports, with a delta method standard error and a
confidence interval each, omega and omega_H for the total score; omega
and omega_HS for each subscale; ECV, H, and the maximal reliability
for every factor; and PUC, which depends on the structure alone and
has no interval. Omega, omega_H, and omega_HS are the reliabilities of
sums of the items as scored, the coefficients fa_confirmatory() and
semTools::compRelSEM() compute, where bifactor_indices() keeps the
standardized metric (on the help page's 15 Holzinger and Swineford
tests, omega_H is 0.552 for the sum of the scores and 0.577 for the sum
of the standardized scores); ECV, H, the maximal reliability, and PUC
are bifactor_indices()'s. The standard errors of the indices lavaan
can define are its own; the maximal reliability's is the delta
method's with numerical derivatives, and every limit is formed on the
logit scale and lies within (0, 1). A structure outside the model, or
one that is not identified, is refused before anything is fit, with an
error of class dmar_bifactor_unidentified (a dmar_refusal) that
names the factors or items and says in its reason element which kind
of refusal it is: fewer than two group factors and an item on two group
factors are outside the model ("scope"; a general factor beside one
group factor, with items on the general factor alone, is identified,
but the subscale indices need two or more); an item listed twice, under
one group factor or under a group factor and in general_only, and
names that clash are "names"; and a group factor of one item or of two
with free loadings, and two group factors whose general loadings the
data fix only up to a scale (two groups of three items each with
nothing on the general factor alone, as on six of the Holzinger and
Swineford tests, where lavaan cannot invert the information matrix),
are "unidentified". Every model with two group factors that is fit
gives a warning of class dmar_bifactor_empirical on empirical
underidentification (Green & Yang, 2018), once the arguments and the
data have passed their checks, so a call that then fails does not get
it. When lavaan's information matrix is singular, whether the observed
information of a FIML fit or the expected information of a listwise fit
or of a covariance matrix, every standard error, test, and interval is
NA, the maximal reliability's included, with a warning; lavaan
inverts such a matrix without complaint, and the maximal reliability's
delta method would otherwise give a standard error of 0. Ordered items
are refused with a pointer to
bifactor_indices() for the latent response metric, and
wald_df = "missing_information", calibrated on one factor models, is
refused as well. An improper solution leaves the maximal reliabilities,
and the H and omega rows it makes undefined, NA with a warning.
bifactor_indices() and ci_bifactor_indices() take items on the
general factor alone. An item that loads on the general factor and
on no group factor, which DMAR 1.0.x
refused, now counts in the general factor's ECV, omega, omega_H, H, and
maximal reliability and in PUC, whose pairs with it are
uncontaminated, and in no subscale, when the fit has two or more group
factors. A fit with one group factor is refused by both (see Changes
to Earlier Results).
fa_second_order() is new. It fits the second-order factor model,
first-order factors on their own items and one or more second-order
factors on their own first-order factors, named in factors and
second_order; several second-order factors are always correlated
(uncorrelated second-order factors are not offered). The model is
identified as lavaan's std.lv = TRUE identifies it, every
second-order factor's variance and every first-order factor's
disturbance variance fixed at 1, and it shares fa_confirmatory()'s
engine, interface, and fit rows: raw data (by full information maximum
likelihood, the default, or after listwise deletion) or a covariance
matrix with its N, ML or MLR, equal_loading, equal_error, and
equal_intercept within a first-order factor and
equal_second_order_loading within a second-order factor, and the five
outputs. The table reports the first-order and second-order loadings,
the error variances, and the intercepts; then, for each second-order
factor, omega and omega_H for the sum of its items and its ECV, H,
maximal reliability, and PUC, and for each first-order factor omega and
omega_HS for its subscale score and the ECV, H, and maximal reliability
of its disturbance, each with a delta method interval whose limits are
formed on the logit scale (PUC has none); then the fit rows. The
indices are those of fa_bifactor() computed from the Schmid-Leiman
loadings the estimates imply (Schmid & Leiman, 1957; Yung, Thissen, &
McLeod, 1999), which the schmid_leiman attribute gives in the
standardized metric, item by item. Structures outside the model or not
identified, and names that clash, are refused before anything is fit
with an error of class dmar_second_order_unidentified (a
dmar_refusal) whose reason element says which, as for
fa_bifactor(): a first-order factor of one item, a second-order factor
of one first-order factor, and one second-order factor over two
first-order factors without equal loadings are "unidentified"; an item
on two first-order factors and a first-order factor under two
second-order factors are "scope"; and an item or factor listed twice
under one factor, a first-order factor under none, and clashing names
are "names". With exactly three first-order factors under one
second-order factor (or two with equal loadings) the second-order
structure is just identified, and a line under the table states that
condition in words and says that the model then fits as the correlated
factors model of fa_confirmatory() does whenever the factor
correlations have a positive product and imply standardized loadings
below one (on nine Holzinger and Swineford tests, a chi square of 85.31
on 24 degrees of freedom under both). Incomplete data under FIML in
which no case observes two first-order factors together are refused
when the pairs observed together do not identify the second-order
loadings (three first-order factors under one second-order factor with
one such pair), and fit when they do (four). In this parameterization a
standardized second-order loading reaches one only as its loading runs
off toward infinity, and lavaan's optimizer can stop on the way short of
the maximum, with other second-order loadings drifted far off, or not
converge; so the model is fit a second time in a parameterization whose
boundary is a point (each first-order factor on the scale of its
second-order factor, the variance of its disturbance there bounded below
at zero, every equality constraint of the call imposed exactly), and the
larger likelihood is reported, two log likelihoods within 1e-6 per case
being taken as the same maximum whatever the number of cases. On one
second-order factor over spatial, verbal, and speed tests and t24, t25,
and t26 of holzinger_swineford, the first fit stopped at a chi square
of 251.77, where the maximum is 169.77. A standardized second-order
loading of one at the maximum, a Heywood case at the second order,
leaves that factor's second-order loading and its loadings on its
disturbance without a finite estimate and the rows of its disturbance
undefined, all NA, while the other rows and the fit rows are those of
the maximum (the standard errors holding the loading at one, and
absent, with a warning, under se = "bootstrap"); and a correlation of
two second-order factors beyond one leaves every maximal reliability
NA. Each gives a warning that names it, with the values in the
improper_second_order attribute, as a negative error variance is
handled in fa_bifactor(). Under FIML, output = "summary" of a fit
with a loading at one shows lavaan's robust CFI, TLI, RMSEA, and GFI as
NA, since lavaan cannot form them for a fit computed without the
optimizer, with one warning in place of lavaan's three. Each first-order
factor is reported with
loadings that sum to a positive value, and then each second-order
factor with second-order loadings that do.
The help shows the likelihood ratio test of
the second-order model against the bifactor model in which it is
nested (Yung et al., 1999; Chen, West, & Sousa, 2006), through
lavaan::lavTestLRT() on the output = "fit" objects of
fa_second_order() and fa_bifactor(). The function is named for the
structure it fits because the sources call the orthogonal layered
model of the bifactor family the hierarchical factor model.ci_bifactor_indices() is new. It gives nonparametric bootstrap
confidence intervals for the indices bifactor_indices() reports:
ECV, omega, omega_H, omega_HS, and coefficient H for each factor they
apply to, and the maximal reliability of each factor, which
bifactor_indices() returns beside its table. PUC depends only on
which items share a group factor and has no interval. Each replication
resamples the rows the fit used (under full information maximum
likelihood, every incomplete row with its missing data pattern),
refits the model with the fit's own parameter table and options, and
reads every index from bifactor_indices() on the refit, so the
intervals rest on the same replications (under improper = "keep",
those of the maximal reliability on the proper ones among them);
before any resample is drawn, the call checks that such a refit
reproduces the point estimates. The percentile interval is the default
(Kelley & Pornprasertmanit, 2016) and the BCa interval is available.
B = 1000, seed, and ncpus work as in the reliability family, and
the draws and the fate of every replication travel in the bootstrap
attribute. Improper replications are set aside by default (improper = "drop"), unlike the reliability family's default. A solution, the
fit's or a replication's, is improper when bifactor_indices() flags
it or lavaan's post-estimation check fails, a check made on the
model's own parameters for a fit with auxiliary variables (lavaan's
aux), as sem_fit_measures() makes it: lavaan's own check also
tests the residual covariances of an auxiliary variable, which are not
positive definite once it is related to the general factor strongly
enough, however proper the bifactor solution. On a bifactor model
of 15 Holzinger and Swineford tests (a general factor on tests 1 to 9
and 14 to 19, with spatial, verbal, and memory group factors on tests
1 to 4, 5 to 9, and 14 to 19; seed = 113), 212 of 1,000 replications
were improper and their omega ran as high as 26.5; keeping every one
of them would have put the upper limit of the 95 percent interval for
omega at 13.7 rather than 0.902. improper = "keep" uses an improper
replication only when every index it gives but the maximal reliability
is defined and a proportion. The maximal reliability, which is not
defined for an improper solution, rests on the proper replications
under either rule, so its interval is the same under both, and an
improper point estimate reported under "keep" has none. Coefficient
H is not defined when a standardized loading reaches one (see Changes
to Earlier Results), and such a replication is set aside under either
rule. Every replication set aside is counted and named (improper, not
converged, an error, an ordered item that lost a category in the
resample), and a caution is printed when they reach the tail area of
an interval. As throughout the bootstrap family, an index's interval
requires at least half of the B replications, and at least two, to
return a value for it; otherwise its se and limits are NA, and the
warning and the printed line say so. Multiple-group, clustered, and
weighted fits, fits to a covariance matrix, and fits that did not
converge are refused. So are the fits bifactor_indices() refuses,
among them a fit with an item on two group factors. The H it
bootstraps is the index of Rodriguez, Reise, and Haviland (2016), not
the maximal reliability of the factor, which has rows of its own (on
the help page's model run with B = 50 in place of its 10 and
seed = 113, the general factor's maximal reliability is 0.678
[0.646, 0.734] beside the index's 0.769 [0.741, 0.808]). The sample
values of both run upward (Aguirre-Urreta, Rönkkö, & McIntosh, 2019),
so the bootstrap attribute carries the bootstrap estimate of each
index's bias (the mean of the replicates its interval uses minus the
estimate), and the printed table gives it for H and the maximal
reliability. On the same example with B = 200 (the same seed) it is
0.031 for the first group factor's maximal reliability and 0.019 for
its H, against 0.005 for its omega. The printed table says what ECV
and omega mean for a group factor. The table has one row per index and
factor, with estimate, se, ci_lower, and ci_upper; tidy()
and glance() give one row per index and one row of counts and
settings.Categorical omega no longer needs mvtnorm.
reliability_omega_categorical() and the per-factor categorical omega
of fa_confirmatory() sum a bivariate normal probability over every pair of
thresholds of every pair of items (Green & Yang, 2009, Eq. 21).
Through 1.0.1 each probability was its own call to
mvtnorm::pmvnorm(), so categorical omega stopped for a user without
mvtnorm, a suggested package, and those calls took most of the time
of a bootstrap replication. DMAR now computes the probabilities
itself, all at once, by the method of Drezner and Wesolowsky (1990) as
refined by Genz (2004), the algorithm mvtnorm applies in two
dimensions. The two agree to within 4e-16 on a grid of 2,873 points.
Every categorical omega checked is unchanged to the rounding of its
sums, by at most 9e-16: Green and Yang's ten population designs, six
to 24 Holzinger and Swineford tests cut into two to seven categories
under both denominators, fa_confirmatory()'s per-factor omega, and a seeded
bootstrap. Within about 1e-7 of a correlation of one mvtnorm loses
accuracy, and within about 1e-10 it returns the limit at one (an error
of up to 2.2e-6, at both limits zero); DMAR's computation matches the
exact values there. Only a communality or a polychoric correlation
that close to one reaches that region. At 12 items with 7 categories
the assembly takes 0.002 s rather than 0.37 s, so a bootstrap
replication takes about 0.07 s rather than 0.43 s, and a BCa interval
at the default B = 10000 about 12 minutes rather than 70 on the
machine where this was measured. No argument, default, or output shape
changes, and mvtnorm leaves Suggests.
reliability_omega_categorical() analyzes incomplete items by
pairwise deletion or by multiple imputation. Through 1.0.1 it
analyzed only the complete rows. It gains missing = c("listwise", "pairwise", "mi"), aux, impute = c("pmm", "polr"), and
imputations = 5, appended after ncpus. "listwise" stays the
default and returns the same results as before.
"pairwise" estimates each threshold from the rows that observe its
item and each polychoric correlation from the rows that observe both
items (lavaan's pairwise deletion). With no covariates in the model it
is consistent only when values are missing completely at random, as
listwise deletion is (Asparouhov & Muthén, 2010). Under missingness at
random it is biased (Shi, Lee, Fairchild, & Maydeu-Olivares, 2020),
and the printed table says so.
"mi" imputes the items by chained equations with mice (now in
Suggests): predictive mean matching by default, or the proportional
odds model. Auxiliary variables named in aux are predictors in the
imputation model only, and a row with no observed item is imputed when
an auxiliary variable is observed on it, and N counts it. The
estimate is the mean of categorical omega over the imputations of the
observed data. Its interval is the bootstrap then impute interval of
Schomaker and Heumann (2018): each of B bootstrap samples of the
incomplete rows is imputed imputations times, and the percentile
interval of the replicate means is the confidence interval. BCa is
refused. The items reach mice as plain integer codes, so a column read
with haven, which carries value labels, and a factor whose labels are
whole numbers are imputed as those numbers. Every incomplete item is
imputed, including one observed together with another on only two or
three rows, as in a planned missing design with a small overlap; the
check by which mice sets aside a variable almost perfectly correlated
with another would otherwise leave it with its missing values. An item
with a single observed value is refused, and an item mice still does
not impute stops the call by name. Turning that check off would also
keep an auxiliary variable that duplicates another, so the auxiliary
variables are screened first. One whose multiple correlation with the
columns observed on every row that observes it (the items and the
auxiliary variables kept before it) reaches .999, mice's own threshold
for a pair, carries no information beyond the predictors already in
the model and is left out, with a message. Kept together, two or more
such variables would make mice fall back to a ridge penalty and impute
unstably. When mice still applies its ridge penalty, the warning says
how often (ridge in the imputation attribute).
The point estimate is formed from at least 20 imputations, and more when von Hippel's (2020) quadratic rule asks for more, and averages those that return a value. The rule is applied to the upper limit of the fraction of missing information that the replications themselves estimate. A call with a bootstrap makes the most imputations the rule can ask for (201) and fits every one before the bootstrap runs, so nothing found in a completed data set of the observed data can discard a finished bootstrap.
A completed data set with no categorical omega, an improper (Heywood)
solution above all, is set aside and counted, as a bootstrap
replication is, and the estimate averages the rest. The call stops
only when fewer than half return a value, the criterion of Lim and
Cheung (2022) for a failed analysis. Stopping on any one would fail
with probability 1 - (1 - p)^M, for a share p of improper sets among M
imputations. Imputation produces such solutions when the observed data
have none, if two items are skewed in opposite directions and values
are missing among the few respondents who determine their polychoric
correlation. On the help page's incomplete items under the model
implied denominator (missing = "mi", ci_method = "percentile",
B = 200, seed = 1), 26 of the 201 completed data sets of the
observed data were improper, while the complete data and pairwise fits
were proper (the complete rows leave two polychoric correlations
undetermined and the loadings unidentified, so listwise deletion
stops). An improper set has the largest loadings, so setting it aside
lowers the estimate a little. The warning, the printed lines, and the
imputation attribute (imputations_returned,
imputations_set_aside, status) say how many were set aside and
why.
A bootstrap replication computes the same statistic as the point
estimate: it averages those of its completed data sets that return a
value, and it is set aside only when fewer than half do. A rule
requiring every completed data set to return a value would set a
replication aside with probability 1 - (1 - p)^M for a share p of
sets returning none, which grows with imputations. Under this rule
the share goes to zero as imputations grows when fewer than half of
the sets fail. It does not fall steadily, though: an odd number of
imputations needs one more set to return a value than the even number
below it, so it sets more replications aside. The warning therefore
names an even number when any set was set aside. In the bootstrap of
the help page's incomplete items above, a quarter of the replications'
completed data sets returned no value. Applied to the first two,
three, four, and five imputations of each replication (the
replicate_estimates element of the imputation attribute), the rule
set aside 12.5, 21, 14.5, and 20% of the replications, where requiring
every set would set aside 35.5, 45.5, 48.5, and 52.5%. The replicate
by imputation estimates form an unbalanced one-way analysis of
variance (Searle, Casella, & McCulloch, 1992). The warning and the
imputation attribute (replicate_sets_set_aside) count the
completed data sets set aside within the replications kept. As for
the other treatments, no interval is reported when fewer than half of
the replications return a value.
The result's imputation attribute carries the imputations, the
fraction of missing information, and how much imputation noise widens
the interval. A replication whose imputation fails is set aside and
counted under a new reason, imputation, and a seed gives the same
result for any ncpus. A completed data set whose fit lavaan stops
with an error is set aside under "error", and the warning quotes
lavaan's message.
In a Monte Carlo simulation described on the help page (10 items at n = 300, about a fifth of the values missing at random), predictive mean matching came closest to the complete data estimate, ahead of the proportional odds model, pairwise deletion, and listwise deletion.
B defaults to NULL in reliability_omega_categorical() and
reliability(). For every existing call it means 10000, the default
written before. Under missing = "mi", where each replication imputes
the data and refits on every completed data set, it means 1000. When
no item value is missing, missing = "mi" has nothing to impute and
returns the complete data result, whose bootstrap takes that path's
default of 10000; the message says so.
reliability() forwards each family function's missing data
treatment. missing is now NULL by default and forwarded as
given, so each family function checks it against its own choices
(listwise, FIML, or two-stage for alpha and omega; listwise or FIML
for H; listwise, pairwise, or multiple imputation for categorical
omega). missing, aux, and the new impute and imputations reach
reliability_omega_categorical(). KR-20 still refuses missing and
aux.
CFA help states the standard errors each estimator receives. se = "standard" supplies inverse-information standard errors: normal
theory for ML, MLR, and GLS, and Browne's asymptotically distribution
free ones for WLS. Ordered items request robust.sem in its place.
MLR otherwise needs se = "robust.huber.white" for sandwich standard
errors under FIML. From a covariance matrix, robust.sem with ML
needs the asymptotic covariance of the moments in NACOV. This
corrects the description; no result changes.
sem_exact_fit_bootstrap() is new. It gives the chi square test of
exact fit a reference distribution that does not assume normal data: the
model based bootstrap of Bollen and Stine (1992), which transforms the
data so that the fitted model holds in them exactly, resamples the
transformed rows B times, refits the model to each sample, and reports
the share of bootstrap statistics at least the observed one as
p_bootstrap, with the Wilson interval for its Monte Carlo error (its
limits print to the four decimal places of the p-value) and the
counts of samples used, improper, and failed. With complete data it is
lavaan's bootstrapLavaan(type = "bollen.stine"), resample for resample
when the seed given here is passed to lavaan as its iseed (with lavaan
run serially; lavaan called without iseed draws its own seed from the
caller's random number stream). With incomplete data fitted by FIML it
transforms the rows so that the saturated FIML estimates on the
transformed data are the model implied moments, by the two
transformations of Savalei and Yuan (2009): one matrix for every missing
data pattern (the default) or pattern by pattern with each pattern's own
moments. Both are solved on the variables divided by their saturated
standard deviations, so the result does not depend on the units of the
variables; it differs element by element from the same transformations
solved in raw units (as semTools' bsBootMiss() solves the common one),
though both make the model hold exactly. Every bootstrap
sample keeps the missing values of the rows it draws, so the model is
refitted by FIML. The transformed data are checked: the largest
saturated estimating equation and the largest mean residual at the model
implied moments are recorded, and cov_fiml() on the transformed data
returns the fitted moments. Replications are drawn before any refit, so
a seed gives the same result whatever ncpus is. A sample's solution
is improper when lavaan's post-estimation check finds a negative
variance estimate or a covariance matrix that is not positive definite;
for a fit with auxiliary variables (lavaan's aux) the check is made on
the model's own parameters, as for the fit itself (see the
sem_fit_measures() entry). Fits to summary statistics, ordered
indicators, non-ML estimators, two-level and clustered models, sampling
weights, two-stage fits, fits conditional on observed exogenous
variables, fits whose analysis data keep missing values without FIML
(pairwise deletion), and fits without a standard test statistic are
refused with the reason, as an error of class
dmar_exact_fit_bootstrap_scope (a dmar_refusal) whose reason
element names it (a fit whose cases lavaan removed by listwise deletion
is bootstrapped on its complete cases, as it was fitted, with a
warning); a missing data pattern too small for the pattern
transformation is refused with class dmar_pattern_too_small, which
carries the pattern, its counts, and its group, and a transformation
that is undefined for the data with class
dmar_transformation_undefined.
sem_fit_measures() is new. It reports the fit of a structural
equation model in one table of twenty-four rows: the chi square test
of exact fit; the RMSEA with its confidence interval and the tests of
close fit (Browne & Cudeck, 1993) and not close fit (MacCallum,
Browne, & Sugawara, 1996); CFI, TLI, NFI, IFI, and RNI; the covariance
SRMR; and gamma hat and adjusted gamma hat with their intervals, with
NA where the input does not determine a row. It takes a fitted
lavaan model or the summary statistics a paper reports. With a fitted
model the default version = "auto" gives lavaan's robust measures
for a fit with a corrected test statistic (MLM, MLR, WLSMV, a sampling
weighted fit; Brosseau-Liard, Savalei, & Li, 2012; Brosseau-Liard &
Savalei, 2014; Savalei, 2018, 2021), the FIML corrected indices of
Zhang and Savalei (2023) for a FIML fit (which approximate the
complete data RMSEA and CFI and are not corrections for nonnormality),
and the standard measures for a normal theory ML fit of complete data;
with auxiliary variables (lavaan's aux under FIML) the SRMR is over
the model's own variables, since lavaan's SRMR averages in the
auxiliary variables, whose residuals of about zero make it lower for
the same model;
the printed footer names the version and why it was chosen. A fitted
model keeps lavaan's sample size multiplier, so its values equal
lavaan's (for the scaled version, those lavaan gives under
rmsea.scaled.mplus = TRUE); summary input takes
sample_size_convention = "N-1" by default, the convention of
ci_rmsea() (pass "N" for a statistic from lavaan or Mplus), and
n_groups for Steiger's (1998) multiple group RMSEA. The default
conf_level is 0.90, the conventional level of the RMSEA interval, whose
two limits decide the tests of close and not close fit at .05 that the
table reports beside it; the help page explains why, and
conf_level = 0.95 gives a 95 percent interval. A fitted model that did
not converge is refused with an error of class dmar_not_converged,
and one whose solution is inadmissible with a refusal of class
dmar_inadmissible that names the negative variance estimate, or the
covariance matrix that is not positive definite (that of the latent
variables, of the residuals of the observed variables, or of the
residuals of the structural part of the model), as lavaan's
post-estimation check finds them. With auxiliary variables (lavaan's
aux under FIML) the check is made on the model's own parameters:
lavaan's own check also tests the residual covariances of the
auxiliary variables, which it models as saturated correlates, and that
block is not positive definite once an auxiliary variable is related to
a factor strongly enough (for one factor whose items have uncorrelated
residuals, and an auxiliary variable related to them only through the
factor, a correlation above .33 between the two when the maximal
reliability of the items is .9), however proper the model's solution.
Auxiliary variables are recognized when they are given through lavaan's
aux; saturated correlates written into the model by hand are judged
with the rest of theta. sem_residual_fit(),
sem_information_criteria(), compare_sem(), ecvi(), and
sem_exact_fit_bootstrap() share the check, and
ci_bifactor_indices() judges a fit with auxiliary variables by it.
sem_fit_measures() reports the robust measures of a mean and
variance adjusted statistic, a CFI of 1 for a saturated model, and no
measure of a ULS fit of continuous data that changes with the units of
the variables. In the examples, the three-factor model of nine
Holzinger and Swineford tests is visual perception, cubes, and
lozenges; paragraph comprehension, sentence completion, and word
meaning; and addition, counting groups of dots, and straight and
curved capitals, fit to holzinger_swineford by lavaan::cfa() with
std.lv = TRUE.
estimator = "MLMVS", or test = "mean.var.adjusted"). Savalei (2018) shows that the robust RMSEA
and CFI of Brosseau-Liard, Savalei, and Li (2012) and
Brosseau-Liard and Savalei (2014) do not depend on which corrected
statistic accompanies them. Each subtracts c df from the ML
statistic, with c = tr(U Gamma)/df, which is that statistic's
scaling constant times its degrees of freedom over df. So "auto"
reports the robust version of such a fit, computed by DMAR, with
the interval and close fit tests of Brosseau-Liard et al. (2012,
Eq. 10) and a robust gamma hat. The values equal lavaan's for the
same model fitted by MLM. For the three-factor model by MLMVS the
RMSEA is .091 with 90 percent interval (the default) [.070, .113], the
CFI .932, and gamma hat .958, where the scaled
measures version = "scaled" gives are .089 [.066, .113] and .734.
A saturated fit with this statistic is reported in the robust
version as well. Nothing is computed from its undefined scaling
constant there, and the footer attributes the corrections to
Brosseau-Liard et al. without saying DMAR computed them."auto" reports the standard version, with a warning; a bootstrap
test, which carries no scaled statistic, gets the standard version
without one. The standard version estimates the population values of
the ML measures, as the robust one does and the scaled one does not
(Savalei, 2018).NA, or a value only by rounding.NA. So are those
of the standard version, whose CFI, TLI, RNI, and IFI set the
degrees of freedom against the same statistic and are NA too
(lavaan's standard CFI is .977 for the raw scores and .979 for the
scores divided by 10); its NFI, a ratio of two such statistics, is
reported. When the model restricts the means, the ULS fit function
adds the squared mean residuals to the squared covariance
residuals, which are in other units, and every RMSEA row and
incremental index of every version is NA: with the intercepts of
paragraph comprehension and word meaning constrained equal in the
same model (meanstructure = TRUE), lavaan's robust CFI is .979
for the raw scores and .972 for the scores divided by 10, and its
scaled RMSEA .160 and .181. The chi_square row and the SRMR are
kept. The robust CFI, TLI, and RNI and the scaled measures of a
model with free means do not depend on the units and are reported,
though no cutoffs have been developed for them (Savalei, 2018). So
are the measures of a fit by estimator = "ULS" alone, tested by
Browne's residual statistic, when its estimates do not change with
the units. When restrictions link the means and covariances through
shared parameters, as equal loadings and intercepts do through
latent means, ULS estimates change with the units. Browne's
statistic at those estimates and its indices change too; RMSEA and
incremental indices are withheld. The valid test statistic and SRMR
remain. A warning, the uls element of the sem_fit attribute,
and a line of the printed footer say which rows are NA and why.The scaled version of sem_fit_measures() puts the scaled statistic
in the standard formulas. Its RMSEA interval and its tests of close
and not close fit are computed from the chi_square and df rows of
the same table (Brosseau-Liard, Savalei, & Li, 2012, Eqs. 11 and 13;
Savalei, 2018, Table 1). These are the values EQS and Mplus report,
and the ones lavaan gives under rmsea.scaled.mplus = TRUE. lavaan's
default for a Satorra-Bentler, Yuan-Bentler, or mean and variance
adjusted statistic refers the unscaled statistic to the noncentral chi
square on tr(U Gamma) degrees of freedom, c df with c = tr(U Gamma)/df
as in the previous entry (for a Satorra-Bentler or Yuan-Bentler
statistic, whose degrees of freedom are df, c is the scaling constant
itself): the same point estimate, but an interval and p-values that
the rows beside them do not imply. The two differ little when c is
near 1: for the three-factor model by MLR, c = 0.98, and the 90
percent interval is [0.0728, 0.1150] against lavaan's default [0.0726,
0.1152]. They differ much when c is not near 1: for the same model on
multivariate t data with 4 df (each pupil's centered scores
multiplied by sqrt(4 / rchisq(301, 4)) after set.seed(1)), c =
1.80, the interval is [0.0506, 0.0950] against [0.0562, 0.0892], and
the p-value of close fit .046 against .013. A scaled and shifted
statistic (MLMV, WLSMV) gets lavaan's own values. For an MLMVS fit,
whose scaled measures version = "scaled" reports, the interval is
[0.0659, 0.1125] against lavaan's default [0.0684, 0.1098]; "auto"
reports the robust measures of such a fit, which DMAR computes.
Gamma hat and adjusted gamma hat come with intervals, and gamma hat
has a robust version. The rows gamma_hat_ci_lower,
gamma_hat_ci_upper, adjusted_gamma_hat_ci_lower, and
adjusted_gamma_hat_ci_upper of sem_fit_measures() are the
noncentrality limits of the RMSEA interval mapped to each index
(Steiger & Fouladi, 1997, pp. 251--252, whose 90 percent interval of
.972 to .997 for the adjusted index in their worked example the tests
reproduce). For a robust ML fit of complete continuous data (MLM, MLR,
MLMV, and MLMVS, whose robust measures DMAR computes since lavaan does
not), the robust version reports gamma hat from the robust
noncentrality, with limits from the robust RMSEA interval: 0.956, with
the 90 percent interval [0.935, 0.973] that the default conf_level
gives ([0.931, 0.976] at conf_level = 0.95), for the three-factor
model by MLR. The help page says that
gamma hat is lavaan 0.7-3's gfi_lrt, which lavaan reports only when
asked by name, and not the gfi lavaan prints, which is computed from
Browne's residual based statistic (for the same fit, the standard
gamma hat, lavaan's gfi_lrt, is 0.957 and its gfi 0.959). Only NFI
and IFI have no robust version.
Gamma hat is NA where Steiger's result does not reach. The
result is for covariance structures. A fitted model that restricts the
means gets NA, with the reason recorded in the sem_fit attribute.
Examples are a growth model, or intercepts constrained equal (under
either ceq.simple setting), constrained to a constant, entered in a
linear constraint, tied to a loading, or held at the bound of an
inequality constraint, which lavaan's degrees of freedom do not count.
Whether the means are restricted is read from the derivatives of the
model's moments, with the constraints that bind at the estimates
imposed. Free means change nothing. Summary input whose df exceeds
p(p + 1)/2, which only restricted means can produce, gives NA with a
warning, where the formula would give an adjusted gamma hat above
gamma hat. Below that bound summary input carries nothing that shows
restricted means, and supplying p states that the model is a
covariance structure. A sampling weighted fit gives NA too, since
its statistic is a weighted pseudo likelihood ratio statistic.
A fit without a corrected statistic is described from lavaan's
options. "auto" reports its standard measures under a reason that
names the estimator and the data. Some standard statistics have no chi
square reference distribution: an ordered fit by DWLS or ULS with the
standard test alone (the default test of those estimators, which WLSMV
and ULSMV extend with a corrected one) and a DWLS or ULS fit of
continuous data whose test was set to "standard" (lavaan's default
for continuous data adds Browne's residual based statistic, which is
chi square under normality), to which lavaan gives no p-value, and a
sampling weighted fit whose test was set to "standard", a weighted
pseudo likelihood ratio statistic whose p-value lavaan reports all the
same. For such a fit a warning says that the statistic is not chi
square distributed and names the remedy (a corrected test, or for
continuous data leaving out test = "standard"), and version = "robust" refuses the fit with the same advice. Full WLS for ordered
data is the exception. Its statistic is asymptotically chi square
(Browne, 1984; Muthén, 1984) and lavaan reports its p-value, so its
standard measures are reported without a warning, and a request for
the robust version says that they are the ones to report. A continuous
fit tested by a chi square statistic is described by its estimator (a
normal theory ML or GLS fit, an asymptotically distribution free WLS
fit, or a least squares fit tested by Browne's residual based
statistic) rather than as a normal theory fit.
equivalence_rmsea() is new. It tests whether the population
RMSEA is below a bound chosen in advance: the test of not close fit
(MacCallum et al., 1996) read as an equivalence test by Yuan, Chan,
Marcoulides, and Bentler (2016). It reports the 1 - 2 alpha interval,
whose upper limit is their T-size RMSEA, with the p-value and a
verdict, and takes n_groups and sample_size_convention (default
"N-1").
sem_residual_fit() is new. It reports the unbiased SRMR of
Maydeu-Olivares (2017) with its standard error and confidence
interval (90 percent by default, as lavaan's, which pairs with the test
of close fit at .05), the test of exact fit (with the standard error that holds
under exact fit only), an optional test of close fit, the SRMR of the
mean residuals, and the largest standardized residual
(Maydeu-Olivares, Shi, & Rosseel, 2018). The residual covariance it
used (normal theory, or distribution free for MLM and MLR) is recorded
on the result. A FIML or pairwise fit of complete data gives the
residual inference of a listwise fit with the same mean structure and
information setting; missing values, two-stage fits, and fits with
auxiliary variables (which change nothing in the model's fit when no
value is missing, but lower lavaan's SRMR) are refused, and the
message says which.
sem_information_criteria() is new. It reports AIC (Akaike,
1974), BIC (Schwarz, 1978), the sample size adjusted BIC (SABIC;
Sclove, 1987; lavaan's bic2), Haughton's (1988) HBIC, and the IBIC
and SPBIC that Bollen, Ray, Zavisca, and Harden (2012) proposed and
Bollen, Harden, Ray, and Zavisca (2014) evaluated for structural
equation models, then the log likelihood, the number of parameters,
and N, in one table in which a criterion not requested is NA. It
takes a fitted lavaan model, or as summary input the log likelihood,
the parameter count, N, and (for IBIC and SPBIC) the information
matrices, so that a published comparison can be reproduced; all four
rows of Table 6 of Bollen et al. (2014) are reproduced to within
0.015. A fit whose fit function is not the normal log likelihood is
refused with the reason (ordered indicators, an estimator other than
ML, the Wishart likelihood, or sampling weights, whose pseudo log
likelihood has no justification for these penalties; Asparouhov,
2005), a two-level model gives AIC only, and automatic IBIC and SPBIC
are refused for fits whose information matrices are not the ones the
criteria are defined on. The help page explains that IBIC depends on
the units of the observed variables and on how the latent variables
are identified, and that SPBIC under its default prior depends on the
identification; it also notes that the worked example of Bollen et
al. (2012, Table 1) counts log N twice in IBIC.
compare_sem() is new. It ranks models fitted to the same data by
one criterion (BIC by default) and reports each model's difference
from the best, its relative weight within the set (Burnham & Anderson,
2004), and, for the BIC family, the approximate log Bayes factor of
the best model over it. It first checks that the log likelihoods are
comparable: the same cases and data values (or the same summary
statistics), the same observed variables in any order, the same
groups, missing data method, and mean structure, and the same
exogenous observed variables (with fixed.x = TRUE, the default of
lavaan::sem(), the log likelihood is conditional on them, and two
saturated models of the same three variables that condition on
different variables differ by hundreds of BIC points). A refusal names
the two models and what differs. The two two-stage settings count as
one missing data method, since they fit the same EM moments by the
same likelihood and differ only in their standard errors and test,
which no criterion uses; the comparison attribute records each
setting used. For IBIC, and SPBIC under its default prior, a refusal
states the rule: one identification for every latent variable of every
model, which two models that each mix the two identifications in the
same way do not meet. ?sem_information_criteria says the same.
Weighted fits are refused; multiple group models (configural against
metric invariance, for example) are compared by AIC, BIC, SABIC, and
HBIC. tidy() and glance() methods come with it.
results_sentence() reads an interval held in <estimate>_ci_lower
and <estimate>_ci_upper rows, and gains term. A
sem_fit_measures() table holds the RMSEA's interval in rows named
for the estimate, and gamma hat's and adjusted gamma hat's the same
way. results_sentence() writes the one term names or, when the
table holds no interval in another form, the first such estimate, with
no p-value: the table's p-values are the tests of exact, close,
and not close fit. The level is the interval's own when the table
records it, and otherwise the table's. For the three-factor model of nine
Holzinger and Swineford tests (visual perception, cubes, and lozenges;
paragraph comprehension, sentence completion, and word meaning;
addition, counting groups of dots, and straight and curved capitals)
fit by lavaan::cfa() with its defaults,
results_sentence(sem_fit_measures(fit), digits = 3) is "rmsea =
0.092, 90% CI [0.071, 0.114]", at the default level. term = "rmsea"
also reaches the RMSEA of a fa_confirmatory() table, whose
rmsea_ci_level row records the level of its RMSEA interval (its
rmsea_conf_level, .90 by default), and gives the same sentence for the
same model. The level is also read from the table's
rmsea_conf_level attribute, so a selection of the rmsea,
rmsea_ci_lower, and rmsea_ci_upper rows, which leaves the
rmsea_ci_level row out, keeps the level of its interval:
for the model above the selection reads "rmsea = 0.092, 90% CI
[0.071, 0.114]", as the full table does.
The same holds for the tables of fa_bifactor() and
fa_second_order(). A term that names no estimate with
such rows is refused with the names of those that have them. term is
appended after digits, and reading such rows changes no sentence
written before.
A vignette, "SEM Fit and Model Comparison"
(vignette("sem_fit_measures")), teaches these measures on one
running example, the nine tests of Jöreskog (1969) in
holzinger_swineford: the chi square test of exact fit; the RMSEA
with its interval, the tests of close and not close fit, and why the
RMSEA is reported with a 90 percent interval; the equivalence test and
the T-size RMSEA; the sample size convention and the multiple group
RMSEA; the incremental indices and gamma hat; the standard, scaled,
and robust version of each measure and which one a robust or FIML fit
reports; the unbiased SRMR and reading the residuals; the information
criteria, including a worked example in which rescaling one variable
reverses an IBIC verdict while BIC, HBIC, and SPBIC do not move;
ranking candidate models with compare_sem(); and the ECVI. Every
number in its text is computed from the fits.
fraction_missing_information() is new. For each free and each
defined parameter of a full information maximum likelihood fit (a
lavaan fit with missing = "ml" or "ml.x", for example from
fa_confirmatory(..., missing = "fiml", output = "fit"), or an mlmr() or
mlmr_mv() fit) it reports the fraction of missing information, 1 -
SE_C^2 / SE_O^2 (Savalei & Rhemtulla, 2012), the proportion of the
sampling variance of the estimate that is due to the missing values,
with the effective N, N(1 - FMI), the target N, N / (1 - FMI), and the
interval width inflation factor, 1 / sqrt(1 - FMI), which state in
AIPE terms the precision the missing values cost. Both standard errors
come from refits at the fit's estimates with the optimizer off and one
information type for both, the complete data side fit to the EM
(unrestricted) moments with the same N and no (N - 1)/N rescaling of
the covariance matrix, as lavaan computes them. information chooses
among the three estimators of Chen and Savalei (2021):
"unstructured", the default, which cannot be negative and broke down
least in their simulations; "structured"; and "hessian", lavaan's
default, which agrees with lavaan's parameterEstimates(fmi = TRUE)
to about 1e-7 when the fit runs lavaan's EM as far (em_h1_args = list(tol = 1e-10)), and otherwise only to the precision of lavaan's
EM (see the first example below). The default therefore differs from
lavaan's. An MLR fit gets the normal theory FMI at its estimates,
which is the ML fit's. Multiple groups are supported, each row's N
following its group and its equality constraints, whether a constraint
names labels or defined parameters (ind1 == ind2 with ind1 := a1*b1 and ind2 := a2*b2 gives the rows it ties the summed N, as
a1*b1 == a2*b2 does), and the omega, H, and AVE that fa_confirmatory()
defines get their FMI through their delta method variances. A
percentile bootstrap interval is opt-in (ci_method = "percentile",
with B and seed), and the printed table says that a sample FMI is
imprecise unless N is several hundred (Chen & Savalei, 2021). Fits
with ordered items, an estimator outside the ML family, listwise,
pairwise, or two-stage missing data handling, conditional.x = TRUE,
more than one level, sampling weights, or no convergence are refused
with the reason, as are lavaanList objects and rotated EFA solutions.
tidy() and glance() methods come with it. The examples below use
the data sets of tests/testthat/test-fraction_missing_information.R
in the source package.
set.seed(11), then
x5[runif(301) < plogis(-1 + x4)] and x9[runif(301) < 0.25] set
to NA). lavaan's own FMIs move by up to 5e-6 when its EM is run to
1e-14 rather than 1e-5, and those of the same model by school, whose
solution has a negative residual variance, by up to .0014. With the
fit's estimates held (the fit to the rescaled data made at the
rescaled estimates with optim.method = "none"), the FMIs of the
three estimators agree with those in the original units to within
1e-9 in one group and 5e-8 by school when every variable is
multiplied by .001, .01, 10, 1000, 10,000, or 1,000,000. Take next a
regression of y on three predictors with values missing at random
(200 cases of four standard normal variables correlated .5, drawn
after set.seed(3), with x3[runif(200) < plogis(1 + 2 * x1)] and
x2[sample(200, 3)] set to NA), fit by lavaan::sem("y ~ x1 + x2 + x3", missing = "ml", fixed.x = FALSE). Fit anew to the
rescaled data, its FMIs agree with those at a standard deviation of
1 to within 1.3e-7 when y, x1, or x3 alone, or every variable, is
multiplied by .001, 10,000, or 100,000, when x2 alone is multiplied
by 10,000 or 100,000 (by .001 its standard deviation lies just above
the .001 at which lavaan rescales a fit, and lavaan's fit does not
converge), and when the outcome is multiplied by 100,000,000, where
mlmr(wald_df = "missing_information") gives the degrees of freedom
it gives at 1. lavaan's own parameterEstimates(fmi = TRUE), which
refits in the variables' own metric, is off by up to .93 (by .04 to
.07 for the slopes) with every standard deviation near 100,000, and
returns NA with every standard deviation near 300,000 or with that
of y near 10,000,000.ov_order = "data" and rescaled by lavaan is not the maximum likelihood fit
when the columns of the data are not in the model's order: with x3,
y, and x1 of the regression multiplied by .004, .03, and .5 and the
columns ordered x3, x2, x1, y, the slopes' FMIs are .28, .22, and
.70 in place of .06, .19, and .70. The computation takes one Newton
step in the refit metric. When an estimate moves by more than 1e-4
of the larger of 1 and its size, the FMIs are computed again one
step on. A warning is given only when an FMI changes by more than
1e-4. It names the parameter, likelihood gain, and largest FMI
change. Units and rescale_data = TRUE are named only for a fit
lavaan did not rescale whose standard deviations indicate that
cause; variable order is named only for a fit lavaan rescaled. A
flat small- sample likelihood can trigger the warning without
either cause, and the warning does not prescribe a remedy already
applied. The FIML refit estimates the unrestricted moments in its
own order of the variables, so a fit made with ov_order = "data"
that lavaan does not rescale gets the FMIs of the same fit in the
model's order.mlmr() or
mlmr_mv() fit as its formula does.aux, which fa_confirmatory(..., aux = ) passes on) is supported. Every refit keeps the fit's
estimates whatever lavaan's checks of a solution say of them,
except for the run-away Heywood case described next. This covers an
auxiliary fit, whose augmented residual covariance matrix is not
positive definite by construction, two factors correlated above 1,
and a residual variance a little below zero. lavaan's own
parameterEstimates(fmi = TRUE) shrinks such a covariance to a
correlation below 1 before it refits, and so reports the FMIs of
other estimates: -1.62 against .25 for the covariance of two
factors correlated 1.08 in a sample of 60. The computation stops
with a message that says why when the refit at the estimates moves
them. Besides a rotated solution, this happens with an improper
one: lavaan (0.7-3) will not take as a starting point the estimates
of a Heywood case whose negative residual variance exceeds the
variable's observed variance in size, and restarts from its simple
starting values, and lavaan's own parameterEstimates(fmi = TRUE)
then reports FMIs computed at those values (NA for some rows,
negative for the intercepts), warning only that the covariance
matrix of the estimates is not positive definite. The computation
also refuses data with no unrestricted estimate of their means and
covariances: a pair no case observes together, or a saturated
likelihood without a maximum. The pair or singular subset is named,
and the checks precede every refit. An EM iteration limit where the
maximum exists instead gives a warning in DMAR's words. The
conditions have classes dmar_fmi_never_together and
dmar_fmi_saturated_singular; wald_df = "missing_information"
gives the same reason and names numeric degrees of freedom as the
alternative. Refits do not repeat lavaan's notice of low pairwise
coverage.missing_patterns() and missing_coverage() are new. They
describe the missing values before any model is fit: the distinct
patterns of observed and missing items with their counts (the pattern
with nothing observed kept as a row, since lavaan sets those cases
aside), and the number and proportion of cases that observed each
pair of items, with lavaan's denominator, the cases with at least one
observed item. Both return tables of the same shape whatever the
number of items.
df_missing() is new. A Wald interval from full information
maximum likelihood refers the estimate to the normal distribution; von
Hippel (2016) showed that in small samples with missing values this
reference distribution, not the estimate or its standard error, is
what makes the interval too short, and that a t reference on degrees
of freedom that shrink with the parameter's fraction of missing
information brought his intervals for regression coefficients and
moments near nominal coverage. df_missing() computes both of his
rules from a vector of FMIs or from the table
fraction_missing_information() returns: method = "effective_n",
N(1 - FMI) - k (his Eq. 31), and method = "complete_df",
df_complete(1 - FMI)(df_complete + 1)/(df_complete + 3) (his Eq. 32),
each bounded below at 3, as he recommends (floor). Neither k nor
df_complete has a default,
because the right value depends on the parameter: for a regression
coefficient k counts the slopes and the intercept, and for a factor
loading or omega DMAR's calibration study chose k = 1 (see the
wald_df items below). A clearly negative FMI, which the structured
and Hessian estimators can return, is refused. Its example takes the
FMIs from fraction_missing_information() on an mlmr() fit and sets
the t interval beside the Wald interval it replaces.
df_barnard_rubin() is new. It gives Barnard and Rubin's (1999)
degrees of freedom for an estimate pooled over imputations, with the
total variance, the imputations' estimate of the FMI, the relative
increase in variance, and both components of the formula, vectorized
over parameters. floor = 3 is von Hippel's bound; floor = 0
reproduces mice's barnard.rubin(), and df_complete = Inf gives
Rubin's (1987) degrees of freedom. The between-imputation variance is
passed as between, not B, which DMAR keeps for bootstrap counts.
Both df functions return fixed-column tables with tidy() and
glance() methods, and print beneath the table a line naming the rule
that made the df column (with the m and df_complete used, for
df_barnard_rubin()) and the value it is bounded below at, which a
knitted table carries as well.
wald_df refers Wald intervals to a t distribution.
reliability_omega(), reliability_alpha(), reliability(),
fa_confirmatory() (with cfa_1() and cfa_2()), mlmr(), and mlmr_mv()
gain wald_df = Inf, appended after their existing arguments (after
the ... that fa_confirmatory(), its wrappers, and the mlmr pair forward to
lavaan, so it must be named in full): the degrees of freedom of the
t distribution their Wald intervals refer to. The estimate and its
standard error are unchanged and the normal quantile is replaced by
the t quantile (von Hippel, 2016); in fa_confirmatory() and the mlmr pair
the p-values refer to the same t distribution, so the interval
and the test agree, and the ratio is reported in the column
test_statistic (z_value in 1.0.x; see the breaking changes). The
default reproduces every interval and p-value of 1.0.x exactly
(qt() and pt() at infinite degrees of freedom are qnorm() and
pnorm()): results gain a wald_df attribute and their tidy()
rows a wald_df column, Inf by default; a finite value adds a line
under the printed table. summary() of an mlmr() or mlmr_mv()
fit then heads its coefficient table t value and Pr(>|t|), as
summary.lm() does, with that line beneath the table's significance
codes. A number applies to every Wald interval of the call: the
"ml", "mlr", and "adf" intervals of alpha and of the model
implied omega and their _logistic variants (on the logit scale),
every parameter row of the fa_confirmatory() tables including the
standardized solution, and every coefficient of the mlmr pair. A
finite value where the interval is not a Wald interval (the closed
forms, the profile likelihood, a bootstrap, robust omega,
fa_confirmatory(output = "fit"), se = "none") is an error. mlmr() and
mlmr_mv() also accept wald_df = "missing_information", von
Hippel's (2016) rule for regression coefficients, max(3, N(1 - FMI) -
k) for each coefficient, with the FMI estimated from the fit as
fraction_missing_information() estimates it and k the slopes and
intercept of the coefficient's outcome; with complete data it gives
the N - k of lm(). Its computation does not depend on the units of
the variables. It warns when the fit's estimates are not at the
maximum of the likelihood, as lavaan's can be for variables measured
in small units: with every standard deviation of the regression of
the fraction_missing_information() entry near .003 (mlmr(y ~ x1 + x2 + x3, ci_method = "wald")), the intercept's degrees of freedom
move by .15, and rescale_data = TRUE, passed through ..., is the
remedy. A coefficient whose fraction of missing information cannot be
computed stops the call with the reason rather than receive NA
degrees of freedom. The degrees of freedom and the df_missing()
table they came from are attributes of the fit, and summary() lists
each coefficient's FMI and degrees of freedom. As von Hippel
published it, the interval pairs the maximum likelihood standard
error with the t quantile, so on complete data it is shorter than
the interval of lm() by sqrt((N - k)/N). fa_confirmatory() and
reliability_omega() accept the keyword as well, for the loadings
and omega, on the rule DMAR's calibration study chose (next item);
reliability_alpha() takes a number only.
wald_df = "missing_information" for factor loadings and omega.
Von Hippel's (2016) evidence for his degrees of freedom covers
regression coefficients and moments, not factor loadings or
reliability coefficients, so DMAR ran a calibration study of its own.
It lives in the source repository, methodology/missing_data, with
the plan (the design and the decision rule, fixed before the run),
the script, the results, and the per parameter summary from which
every verdict can be recomputed; it is not in the package.
fa_confirmatory()
writes them: 6 items at N from 30 to 200 and 12 items at N = 50 and
100, normal data fit by ML and nonnormal data by MLR with
Huber-White standard errors, and no missing values or 15 or 30
percent of the values of half the items missing completely at
random or at random, with 2000 replications in each of 60 cells. It
compared the normal interval with t intervals on four of von
Hippel's rules under a decision rule fixed before the run, and a
repetition of his own design reproduced his Tables 4 and 5 to within
.012.fa_confirmatory() (with cfa_1() and cfa_2()) and
reliability_omega() accept the keyword. In fa_confirmatory() the
loadings of the "verbose" table take the t reference, each on its
own degrees of freedom, and every other row keeps the normal
reference, since the study evaluated neither factor correlations,
intercepts, error variances, nor the AVE and H; omega passed on its
own scale, but fa_confirmatory() now forms omega's limits on the
logit scale, where the t reference overcovered (.9645), so omega
keeps the normal reference there, and the "measurement" table,
which has no loading, refuses the keyword. In reliability_omega(denominator = "model_implied") it applies to the "ml" and "mlr" intervals.
With complete data, listwise deletion, or a covariance matrix every
FMI is 0 and the degrees of freedom are N - 1.holzinger_swineford
(visual perception, cubes, lozenges, the second paper form board,
and flags), two of which only the Grant-White pupils took (156 of
the 301 values missing), fa_confirmatory(missing = "fiml", wald_df = "missing_information") gives the loadings of those two tests 139.8
and 143.7 degrees of freedom (FMI .53 and .52), the other three
246.0 to 274.9, and omega 214.6 (FMI .28); omega's 95% interval
moves from [.6946, .8061] to [.6942, .8064]. In a pilot of 60 of the
pupils (set.seed(113); sample(301, 60)), the model implied omega's
FMI under reliability_omega(missing = "fiml", ci_method = "ml", denominator = "model_implied") is .23, its degrees of freedom are
45.5, and its interval [.665, .888] becomes [.662, .891], 2.7%
wider._logistic intervals (on the
logit scale the t reference covered up to .9645 with normal data,
where the normal interval was already within .0044 of .95 on
average), the ADF intervals, robust omega, the standardized
solution, estimators other than ML and MLR, ordered items, pairwise
deletion and two-stage estimation, standard errors other than the
normal theory and Huber-White ones (robust.sem is accepted only
under listwise deletion, or from a covariance matrix with the
required NACOV; it and Satorra-Bentler tests passed through ...
are refused under FIML whether values are missing or not, because
lavaan falls back to listwise deletion), and, under full
information maximum likelihood, standard errors from any
information but the observed information computed from the Hessian,
lavaan's default there and the study's.reliability_alpha() refuses it too, since no rule
passed the study's decision rule for alpha: the t reference
brought alpha's FIML interval barely closer to .95 than the normal
one, which was already close (.0042 against .0043), and covered
.966 in one cell, the one where the study's fit of the saturated
model converged in only 53 percent of the replications. Without
that cell the rule would have passed (.0035 against .0042), but the
rule was fixed before the run and its verdict stands; a number of
degrees of freedom remains available.wald_df attribute,
the df_missing() table they came from is the wald_df_table
attribute, and a line under the printed table names the rule, N, and
the rows it applied to.cov_fiml() returns the maximum likelihood covariance matrix and
means of incomplete continuous data. These are the estimates of the
saturated model fitted by full information maximum likelihood, the
moments the EM algorithm converges to, computed with lavaan's EM, run
on the variables centered at their means and divided by their standard
deviations, with the estimates transformed back. They are the
unrestricted moments lavaan::lavInspect(fit, "sampstat.h1") returns
for any FIML fit to the same variables. They are consistent when
values are missing at random given the variables included, where the
complete case matrix is consistent in general only when values are
missing completely at random. lavaan's EM stops when no mean or
covariance changes by more than 1e-5, a criterion on the absolute
change, so run in the raw metric its estimates depend on the units: on
the regression data of the fraction_missing_information() entry with
every variable multiplied by .001, lavaan::lavCor(missing = "ml")
returns correlations off by up to .16, with no warning that its EM
stopped short (it warns only that the matrix is near singular, its
variances being near zero). Multiplying a variable by a constant
multiplies the mean cov_fiml() returns, and its row and column of
S, by that constant, and the estimates agree with lavaan's own to
about seven digits. A warning lavaan raises about the variables names
them as data does. Columns named in aux enter the saturated model
as auxiliary variables and are left out of what is returned, and a
case observed only on its auxiliaries is used, as the reliability
family uses it under FIML (see Changes to Earlier Results). The result
is a list, like that of cov_sem(), because every use of it takes the
matrix: S (divisor N), mu, N, N_complete, patterns, and
converged. A non-numeric column, a column with fewer than two
distinct observed values, a pair of variables never observed together,
and a pair observed together only in rows where one of them takes a
single value (or in one row only), whose rows in common say nothing
about how the two vary together, are refused by name. cov_fiml() and
the reliability family refuse such pairs in the same words, from
shared checks: the message names the method that needs the covariance,
says what an estimate of it would rest on (lavaan only warns about a
pair never observed together, and its estimate of that covariance is
arbitrary), and advises leaving out one variable of each pair, an
auxiliary variable where the pair has one. The help page explains why
N is not an effective sample size for a method that treats the
matrix as complete data (Enders & Peugh, 2004; Savalei & Bentler,
2009). The matrix is the basis for an exploratory factor analysis or a
parallel analysis of incomplete items.
A vignette, "Missing Data: What the Missing Values Cost"
(vignette("missing_data")), teaches the missing data family on the
six spatial tests of holzinger_swineford, two of which were given
only at the Grant-White school. It covers the patterns of missing
values and the cases behind each covariance; why listwise deletion
needs values missing completely at random and FIML needs them missing
at random given the model's variables, shown on the real data (where
listwise deletion analyzes one school) and on a deletion made missing
at random whose complete data answer is known; auxiliary variables as
saturated correlates through fa_confirmatory(..., aux = ), and what lavaan's
warning about the residual covariance matrix means there; the fraction
of missing information of every parameter, with the effective sample
size, the target sample size, the width inflation factor, the three
estimators of Chen and Savalei (2021), and a bootstrap interval for
the FMI; the EM covariance matrix of cov_fiml() as the basis of an
exploratory factor analysis, and why its N is not an effective sample
size; t reference distributions in small samples through wald_df = "missing_information", df_missing(), and df_barnard_rubin();
two-stage maximum likelihood with its residual based statistic and
corrected fit indices; omega, alpha, and categorical omega (pairwise
deletion and multiple imputation) from incomplete items; factor scores
and their determinacy for each missing data pattern; and a sample size
plan for omega made with the FMI and checked by simulation. Every
number in its text is computed from the analyses.
fa_confirmatory(), cfa_1(), cfa_2(), and measurement_invariance() gain
missing = "two_stage" and missing = "robust_two_stage" (lavaan's
spellings "two.stage" and "robust.two.stage" are accepted too).
The model is fit to the EM estimates of the means and covariances,
computed from every partially observed case and from any auxiliary
variables passed as lavaan's aux, which enter that first stage only
(Yuan & Bentler, 2000; Savalei & Bentler, 2009). A case observed on an
auxiliary variable but on no variable of the model is used too. The
first stage estimates the items and the auxiliaries jointly and sums
its likelihood over every case (Savalei & Bentler, 2009, Eq. 10),
which is what makes the estimates consistent when the items go
missing together at random given the auxiliaries.
lavaan's two-stage estimator reads such a case in its first stage but leaves it out of its N. That N turns the per case quantities of the two stages into:
The fit is therefore computed again with the sample size set to the
cases the first stage read, group by group. lavaan's own formulas then
give every value on that N, in the table and in the lavaan fits of
output = "fit" and of the "fits" attribute.
twostage() gives.summary() of such a fit counts the cases as used, and
lavTestLRT() warns that a difference test between it and a fit
lavaan made of the same data on its own N is invalid.Example: six Holzinger and Swineford tests, a spatial factor of visual
perception, cubes, and lozenges and a verbal factor of paragraph
comprehension, sentence completion, and word meaning, with the whole
block missing for 77 of 301 pupils at random given general
information, the auxiliary (set.seed(113), then the six tests set to
NA in the rows sample(301, 77, prob = plogis(-1.5 + 1.2 * z)),
with z the standardized general information score).
set.seed(2026)), each with the block deleted in the same way, the
intercept's mean error against the resample's complete data estimate
is -0.49 with the cases set aside and -0.01 with them kept.The standard errors are the two-stage sandwich, normal theory or
robust to nonnormality. After normal theory two-stage estimation the
chi_square row is the residual based statistic of Savalei and
Bentler (2009). DMAR computes it from the observed information of the
saturated model over every case the first stage read, and their
simulations found it to hold its rejection rate near the nominal
level. After robust two-stage estimation the row is lavaan's scaled
statistic (Savalei & Falk, 2014). Simple equality constraints that
lavaan keeps under ceq.simple = TRUE, passed through ..., are
read: the residual based statistic of such a model is that of the same
model without ceq.simple.
The cfi and rmsea rows are lavaan's robust indices, the corrected
indices of Zhang and Savalei (2023). They subtract from the naive
statistic an estimate of its expectation, built from the asymptotic
covariance matrix of the EM moments.
set.seed(1), then sample(301, 90) for each in turn), the
RMSEA is .0869 after normal theory and .0859 after robust two-stage
estimation.tli and
nnfi rows and the RMSEA interval are lavaan's extension of the
correction. A line under the table says which version it holds.The rung-to-rung tests of measurement_invariance() are the scaled
difference tests (Satorra & Bentler, 2001). AIC and BIC are the
two-stage criteria of Lai (2021), the values
sem_information_criteria() reports. The printed table says which
statistic, indices, and criteria it reports. fa_confirmatory() frees the item
intercepts under two-stage estimation as it does under FIML; through
bare lavaan() lavaan would fix them at zero. An estimator other than
ML, a robust se, ordered items, and covariance input are errors that
name the option to use. fa_confirmatory()'s table gains a test attribute
naming the statistic in its chi_square row, and
measurement_invariance() a model_test attribute. No existing value
changes.
reliability_omega(), reliability_alpha(), and reliability()
gain missing = "two_stage", appended to the choices; the default is
full information maximum likelihood when a value is missing (see
Changes to Earlier Results). The model implied coefficients take the
normal theory two-stage standard error under ci_method = "ml" and
the robust one under "mlr". Their fits go through fa_confirmatory(), so a
case observed on an auxiliary variable alone is counted as above.
Robust omega divides the squared sum of the two-stage loadings by the sum of the EM covariance matrix, with a bootstrap interval that refits both stages. No delta method interval is offered for it, since none has been studied. Each bootstrap replication fits the second stage to the EM moments of the resample, without the two-stage standard errors and scaled statistic that lavaan computes for every two-stage fit and no replication reads. The bootstrap replications of the model implied coefficients do the same. The estimates are those of a full two-stage fit, to rounding error, wherever the likelihood has a well defined maximum. On a small, heavily incomplete resample, whose likelihood is nearly flat or has no proper maximum, the two optimizers can stop at different points. A replication's value can then differ at the optimizer's tolerance, and a solution that runs away can be counted as not converged rather than improper, or the reverse; under either verdict it is among the replications the bootstrap counts as set aside or improper.
Analytic alpha, a function of the first stage alone, equals its FIML
value and interval. Two-stage estimation uses the cases FIML uses,
every case with an observed item or an observed auxiliary variable,
since its first stage reads a case observed on an auxiliary variable
alone. A message counts such cases, once per call rather than once
per bootstrap replication. When the fit of a model implied
coefficient cannot be computed again with every case, a warning says
that its analytic standard error is too large and that a bootstrap
interval counts every case. On visual perception, paragraph
comprehension, sentence completion, and word meaning, with the block
deleted as in the example above and general information as the
auxiliary, alpha (ci_method = "ml") is .773 (SE .0235) on 301
cases, the FIML value. Setting the 77 aside would give .769 (SE
.0242) on 224.
aux names auxiliary variables of the first stage, and aux without
missing still means FIML. The intervals refused under FIML are
refused here too, the profile likelihood also because no likelihood
of the observed data is maximized.
sem_fit_measures() reports a two-stage fit with the statistic and
indices fa_confirmatory() reports, and compare_sem() compares two-stage fits
by their two-stage criteria. sem_fit_measures() reports the robust
version, the corrected indices of Zhang and Savalei (2023). After
robust two-stage estimation these are their TS V2. After normal theory
two-stage estimation they are its normal theory analog, which lavaan
forms with the inverse of the observed information of the saturated
model where TS V2 takes the sandwich of the observed and first order
information, and which they did not study. The chi_square row is the
residual based statistic, or the scaled statistic after robust
two-stage estimation, and the printed footer names the version of the
indices, as fa_confirmatory()'s does. It refuses version = "standard" (the
naive statistic, which is not chi square distributed) and "scaled".
The residual based statistic is refused, with a message that says what to do instead, for a fit with:
fixed.x = TRUE or
conditional.x = TRUE. Refit with fixed.x = FALSE (and with
conditional.x = FALSE, since lavaan does not take fixed.x = FALSE beside conditional.x = TRUE), which leaves the estimates as
they are when the exogenous variables are complete and otherwise
also keeps the cases lavaan deletes for a missing exogenous variable
when it conditions on them (robust two-stage estimation included),
so the estimates then differ; to keep the fit's estimates, refit on
the rows the fit used, those on which every exogenous variable is
observed, which lavInspect(fit, "case.idx") lists. The refusal is
an error of class dmar_two_stage_fixed_x (a dmar_refusal) that
carries the exogenous variables;Its listwise deletion warning counts only the cases lavaan deletes listwise; the empty cases lavaan drops under FIML or two-stage estimation, which carry no information, draw none.
It warns when a two-stage fit made with lavaan has cases observed on
an auxiliary variable but on none of the model's variables. lavaan's
first stage reads such cases and its N leaves them out. The naive
statistic, the scaled statistic and corrected indices built on it, the
RMSEA multiplier, and the N row then rest on fewer cases than the EM
moments, while the residual based statistic uses every case. The table
reports lavaan's values. The warning points to fa_confirmatory() and
measurement_invariance(), whose fits keep and count such cases, and
to FIML. On the six tests above (lavaan::cfa(model, data, std.lv = TRUE, missing = "two.stage", aux = "t5_general_information")),
lavaan's N is 224 and its robust RMSEA .095; the fit fa_confirmatory(missing = "two_stage") returns for the same model counts 301 cases and gives
.103.
compare_sem() reports no log evidence for two-stage criteria, which
approximate no marginal likelihood, and refuses two-stage fits whose
auxiliary variables differ.
sem_information_criteria() and compare_sem() count every case
the first stage of a two-stage fit read. lavaan forms the two-stage
log likelihood as its N times the complete data log likelihood per
case at the EM estimates, and BIC, SABIC, and HBIC take the same N
into their penalties. A case observed on an auxiliary variable but on
none of the model's variables moves the EM estimates and is left out
of that N. The log likelihood and N are taken instead from the fit
computed again with every case the first stage read.
sem_information
attribute records the counts.On the six tests above, fit by lavaan as in the previous entry,
lavaan's N for the two-factor model is 224 and its log likelihood
-4040.25. That is 224/301 of the -5429.09 reported with N = 301, and
BIC is 10966.61 (8183.32 from lavaan's values). With the two schools
as groups and equal loadings and intercepts (group = "school", meanstructure = TRUE, group.equal = c("loadings", "intercepts")),
the log likelihood is -5393.34 and BIC 10957.89, against -5393.52 and
10958.25 at lavaan's estimates and -4020.79 and 8203.92 from lavaan's
values. A fit fa_confirmatory() or measurement_invariance() returns already
counts the cases and gives the same criteria, and compare_sem()
compares it with a lavaan fit of the same data. A warning names what
rests on lavaan's N only when the fit cannot be computed again.
factor_scores() is new. It returns both standard factor scores
of a fitted lavaan model for every row and factor: the regression
score (the posterior mean) and the Bartlett score, or for ordered
items lavaan's empirical Bayes modal and maximum likelihood scores.
Each comes with a per-case standard error and a conf_level interval,
beside the number of the factor's items the case answered, its missing
data pattern, and, for continuous items in a fit not conditional on
exogenous covariates, the determinacy of each score for that pattern,
in one long table with one row per row of the data and factor. An item
listed under a factor with its loading fixed or constrained to zero
(0*item, or a*item with a == 0) is not one of that factor's
items. print(), tidy(), and glance() methods come with it.
Under FIML each case is scored from its own observed items, and the
determinacy for each pattern shows what a missing item costs; for one
factor its square is coefficient H. Beside the regression score's
determinacy rho, the display and the determinacy and patterns
attributes give 2 rho^2 - 1 (min_score_correlation), the lowest
correlation two sets of scores equally consistent with the model can
have (Guttman, 1955), reported without a line drawn. Rows the fit did not score
(deleted listwise, with no observed item, without a group) stay in the
table with the reason when data is supplied, and a frame that is not
the rows given to the fit is refused. A row deleted listwise from an
ordered fit is told that missing = "pairwise" scores it, since
lavaan has no FIML for ordered items. A row deleted for a missing
exogenous covariate is told, whatever the fit's treatment of missing
values, that missing = "ml.x" scores it (with conditional.x = FALSE for a fit conditional on the covariates), or with ordered items
that no fit lavaan can score keeps it. Without data, where the two
cannot be told apart, the reason in a one-group fit by listwise
deletion names what scores each; in any other fit that deletes such
rows it names missing = "ml.x" and says that supplying data
identifies the reason. A case that answered none of a factor's items
has no Bartlett score and keeps a regression score that comes only
from its other observed variables (the items of correlated factors, a
covariate that predicts the factor, an outcome it predicts). A factor
that a pattern's observed items do not separate from another (a case
that answered only an item two factors share) has no Bartlett score,
rather than the minimum norm solution of lavaan's generalized inverse;
a factor the pattern does identify keeps its score.
A fit of continuous items that keeps the incomplete cases without FIML
is scored as a FIML fit is, each case from its own observed items,
with the fit's estimates. That covers a two-stage fit (lavaan's
missing = "two.stage" or "robust.two.stage", or fa_confirmatory(missing = "two_stage")), whose estimates are consistent when the values are
missing at random, as the FIML estimates are (Savalei & Bentler,
2009). It also covers a fit by pairwise deletion (missing = "pairwise" or "available.cases"), whose estimates require values
missing completely at random. lavaan's lavPredict() scores such a
fit as complete data and leaves every case with a missing value
without a score, so factor_scores() computes these scores itself, by
the closed forms lavPredict() applies pattern by pattern under FIML.
Such a fit conditional on exogenous covariates is refused.
Without a mean structure (the default for one group under listwise or
pairwise deletion), a factor that a covariate predicts is centered
where lavaan and FIML center it, at the slope times the covariate's
mean. In such a fit without an exogenous covariate, lavPredict()
counts twice the mean of an observed variable that lavaan holds among
the latent variables, which shifts by a constant every score whose
weights involve it (the regression score of every factor related to
it, the Bartlett score of a factor that predicts it);
factor_scores() computes those scores itself, and a higher order
factor keeps lavaan's Bartlett score less that constant. An observed
variable regressed on others, such as an item with a direct effect of
a covariate (the MIMIC model of differential item functioning) or an
outcome the factor predicts, is held by lavaan among the latent
variables, with its loadings among the regressions. The scores, their
standard errors and determinacies, the check of the ML score, and the
check of the solution take its loadings and residual variance from
there, as lavPredict() does, so an item with a direct effect enters
its factor's Bartlett score. An observed variable that lavaan holds
the same way but regresses on nothing, such as a criterion that
covaries with a factor (f ~~ z) or a covariate of a factor that also
covaries with another variable, is conditioned on, as an exogenous
covariate is.
Without regressions among the variables lavaan holds as latent, a
variable regressed only on exogenous covariates is conditioned on as
well, also when it covaries with a factor. With such regressions,
lavPredict() gives an observed variable regressed on nothing a row
of zeros among the loadings, leaving its covariances with the factors
out of the scores. factor_scores() scores these fits itself and
refuses one conditional on the covariates. A fit with such
regressions is refused when the disturbance of an observed variable
regressed on others covaries with a factor or a variable the scores
condition on: lavaan's Bartlett score is then not conditionally
unbiased, and where a factor is involved its regression score is not
the conditional mean. The remedy carries the variable as a single
indicator latent variable without fixing its loading, with its
observed residual variance fixed at zero. This preserves the model
under std.lv = TRUE as well. In multiple groups the refusal also
names the intercept and residual variance and covariance equalities
needed to preserve the constraints.
For ordered items a factor has an ML score only where the likelihood
of the case's responses has a finite and unique maximum in it, which
an exact check decides for any loading structure, cross-loadings,
bifactor models, and observed variables regressed on the factors
included (by Farkas' lemma, through nonnegative least squares). Each
of these has an NA ML score with the reason and keeps its EBM
score: a factor whose observed items all sit at the end of their
scales that their loadings point to (all in the lowest category, or
all in the highest, for positive loadings), one whose likelihood
rises without bound together with others, one its items cannot
separate from another, and a higher order factor. Every other factor
keeps its ML score, also one that shares items with a factor whose
likelihood rises without bound: its maximum is then that of the
case's other items, which lavaan's optimizer reaches within its
tolerance. When lavaan's default optimizer stops on the other factor
and returns no score, the score comes from lavaan's nlminb
optimizer. With ordered items, lavaan's EBM and ML scores omit terms
from each case's likelihood or prior. A fit is scored when the
omitted term is constant in the factors: a covariate's effect on an
outcome that depends on no factor directly, or a fully mediated chain
of continuous observed variables. A fit is refused for a direct
effect of a covariate on a variable that depends on a factor
directly, a regression of such a variable on an observed variable, or
a regression on an ordered item. It is also refused when an observed
variable lavaan holds among the loadings has a disturbance covarying
with a factor, or a row lacks a predictor whose path reaches a factor
without an observed mediator cutting it. A fit with exogenous
covariates that is not conditional on them is refused because
lavPredict() cannot score it. A fit of ordered items with residual
covariances is scored with a warning, since lavaan's EBM and ML
scores leave those covariances out of each case's likelihood; the
help page gives the size of these errors in an example. Ordered items
require lavaan 0.7-3 or newer: lavaan 0.7-2 counted the intercepts of
ordered indicators twice in its EBM and ML scores, which were then
wrong in any model where those intercepts are not zero (a multiple
group model with equal thresholds, for one), so factor_scores()
refuses an ordered fit under an older lavaan.
Auxiliary variables carried as saturated correlates are set aside,
and the scores are computed from the indicators with the fit's
estimates, since conditioning on them removes true score variance. A
fit that carries them with ordered items is refused (refit without
them), as is one that carries them and is conditional on exogenous
covariates (refit with conditional.x = FALSE). A higher order
factor has no Bartlett score, and its reason says so, where the
scores are computed from the indicators (auxiliary variables set
aside), where the data have missing values and the scores are
computed here for a two-stage fit or one by pairwise deletion
("pairwise" or "available.cases"), or in a fit in which lavaan
gives an observed variable regressed on nothing a row of zeros among
the loadings, since lavaan's construction of that score is not
repeated there. Elsewhere it keeps lavaan's score (less the constant
above where lavaan counts a mean twice), and its Bartlett standard
error from lavaan is in the pattern table.
An improper solution is scored with a warning: a negative variance (the residual variance of an observed variable regressed on others included), a covariance matrix of the factors that is not positive definite (a factor correlation beyond 1, for one), a residual covariance matrix of the continuous items that is not, or, when none of these holds, a residual covariance matrix of the structural part of the model that is not, the conditions lavaan's post-estimation check flags. For ordered items lavaan computes no score for a solution with a negative residual variance, and the warning and the reasons say so.
The help page says which score belongs where: regression scores for a
predictor and Bartlett scores for an outcome, each from its own
measurement model (Skrondal & Laake, 2001), and one-step SEM or
structural after measurement (lavaan::sam(); Rosseel & Loh, 2024,
which includes Croon's correction) as the route to relations among
factors. Scores under missing data are discussed with Estabrook and
Neale (2013). The help says that the Bartlett score's conditional
unbiasedness and the coverage of its interval hold under missing data
only when whether an item is observed does not depend on the residuals
of the items scored (Rubin, 1976), so not when it depends on the
factor's own observed items, with an example. It also says that the
regression score's posterior interval keeps its coverage when the
missingness depends on the items scored, but not when it runs through
an auxiliary variable set aside.
A loading or regression held at zero, directly or through a chain of plain equalities, is read as absent. It makes no item, does not keep a saturated correlate from being set aside as an auxiliary variable, and gives the scores and reasons of the model written without it.
ss_aipe_reliability() gains fmi and seed. A reliability
coefficient estimated by full information maximum likelihood from
incomplete data has a wider interval at a given N than a complete data
plan assumes. fmi (default 0) is the fraction of missing information
anticipated for the coefficient, supplied by a pilot study or a
comparable data set through fraction_missing_information(); it is
not the proportion of missing values. Missing information multiplies
the sampling variance by 1 / (1 - fmi) (Savalei & Rhemtulla, 2012).
The planner's variance is V / (N - 1), so the plan divides the
complete data N - 1 by 1 - fmi: N = (N_c - 1) / (1 - fmi) + 1, and
an FMI of .5 doubles N - 1, not N. With an assurance, the Monte Carlo
search runs at the complete data (effective) size and its answer is
mapped back the same way, a first-order approximation that the
sensitivity function checks. The table keeps its rows; the attributes
fmi and necessary_N_complete record the plan, and a line under the
table names both. The help page explains where a planning FMI comes
from, why a pilot's FMI is imprecise (Chen & Savalei, 2021), and why
the upper limit of its interval is the conservative planning value.
The plan maps the unrounded complete data solution, so the formula
applied to the printed complete data N can give a case or two more
than the plan (the first example of the help page, six parallel items
that correlate .6 planned for a width of .10, needs 38 complete data
cases and 42 at an FMI of .1, where the formula on 38 gives 43); the
help says so. It also says that a pilot FMI from
fraction_missing_information() refers to the cases the fit used, and
that when the planned N counts every case recruited, those who will
answer no item are folded in as 1 - (N_used / N)(1 - fmi). seed
(default NULL) makes the assurance search reproducible and restores
the caller's random number stream on exit; without it the search draws
from and advances that stream, as before. With fmi = 0 the plan is
the complete data plan and no attribute is added; the planner's other
changes are under Changes to Earlier Results.
ss_aipe_reliability_sensitivity() gains fmi, missing_prop, and
B. fmi is the planning value passed to the planner;
missing_prop deletes each simulated item score completely at random
with that probability and analyzes every replication by full
information maximum likelihood, so a plan made with fmi is judged by
the widths that incomplete data actually deliver. For four parallel
items with reliability .80, a width of .15, and missing_prop = .2,
the FMI of the coefficient is .184 relative to every case and .182
relative to the cases a fit keeps, since a share .2^4 = .0016 of the
cases loses every item (the realized FMI, described below, of one FIML
fit to 200,000 simulated cases:
ss_aipe_reliability_sensitivity(true_reliability = .8, n_items = 4, width = .15, specified_N = 200000, missing_prop = .2, G = 1) after
set.seed(113)). In 1,000 replications, with set.seed(113) before
each call, the complete data plan (N = 74) gave "ml" intervals
averaging .170 wide, and the plan made with fmi = .184 (N = 91) gave
.152, against .153 for the complete data plan with complete data.
Under missing data the default interval is "ml", as with complete
data, and intervals whose standard errors ignore the missing values
are refused. With missing_prop above 0, each replication that
returns an interval also estimates the fraction of missing information
its simulated data realized for the coefficient (the unstructured FMI
of fraction_missing_information(), from alpha's saturated model or
omega's single-factor model). The mean is reported beside the planning
fmi as the attribute mean_realized_fmi, with the number of
replications behind it in n_realized_fmi, a second line under the
table, and a realized_fmi column in the per-replication file. A
single sample FMI is too imprecise at the sizes a plan resolves to
check a plan by (Chen & Savalei, 2021), and the mean over replications
is the figure to carry into fmi. Computing it draws no random
numbers, so the table's rows are those of a run without it; it costs a
fit and two refits per replication. The realized FMI is relative to
every simulated case, the footing of the planner's fmi: lavaan sets
aside a case that loses every item, and the FMI relative to the cases
it kept is folded back as 1 - (n_used / n)(1 - fmi_used). With three
items at missing_prop = .5, a share .5^3 = .125 of the cases loses
every item, so the FMI relative to the cases lavaan keeps understates
the FMI the plan needs. B is the number of bootstrap
resamples behind each replication's interval, which was fixed at 200
whatever the interval; its default is the larger of 200 and the
number the interval method and the level require, and a smaller value
is refused (see One Rule for Bootstrap Replications).
For alpha, with fmi and missing_prop at 0, B at its default, no
assurance, and ci_method = "bonett", the results are those of 1.0.x,
except for replications whose interval lies wholly below zero (see the
entry on clamped limits under Changes to Earlier Results); the default
interval is now "ml" (see the entry there on the interval its plan
was made for).
ss_aipe_rmsea() does not take fmi. RMSEA is a function of the
test statistic, not a model parameter, so no parameter's FMI governs
the noncentrality its interval is built on, and Savalei and
Rhemtulla (2012) leave global measures of missing information to
future research.
ss_aipe_mean() plans the sample size for the confidence interval
for a single mean, the mean of one sample or the mean of paired
differences, which DMAR could not plan before: ss_aipe_sm() plans the
standardized mean, and ss_aipe_c() requires contrast weights that sum
to zero. Without an assurance the plan makes the expected width no
wider than width, and the expected width is exact under normality: it
uses E(S) = c4(N) sigma, where ss_aipe_c() sets S equal to sigma, and
so needs the same N as that rule or one fewer. With an assurance the
realized width is no wider than width with that probability.
true_kurtosis, on the excess scale of kurtosis() so that a pilot's value
can usually be passed as it is, adjusts the assurance through a scaled
chi square with the exact mean and variance of S^2; the expected width
plan does not use it and warns when one is supplied. A pilot whose scores
take nearly two values can give a kurtosis() of -2 or less, which is
refused, and the help page says what to pass instead. With a large
kurtosis the assurance can be met at a very small N, missed over a
stretch above it, and met again, so the plan is the smallest N from which
the assurance holds at every larger N, and the search finds that N
exactly. The answer row is necessary_N, beside the expected width at
that size and the inputs.
ss_aipe_mean_sensitivity() checks such a plan by Monte Carlo,
under a normal population or by resampling a vector of observed scores
(population), which is how a plan meets the tails and skewness of
real data, such as a pilot's difference scores. It follows the
sensitivity family's arguments and rows, reports mean_mean,
median_mean, and sd_mean for the realized means, and takes seed
(default NULL), which makes a run reproducible and restores the
caller's random number stream on exit. For the help page's pilot (the
IQ gains of the control children in pygmalion, excess kurtosis 1.2),
a plan for 90 percent assurance made under normal theory gave intervals
no wider than the target of 4 points in .870 of 1,000 replications, and
the plan made with the pilot's kurtosis, six children larger, in .913.
ss_aipe_c() gains true_kurtosis. With an assurance, the planner
takes the pooled variance to follow the chi square law of normal
theory, which holds only for normal errors. Errors lighter tailed than
the normal, as bounded outcomes such as proportions and rating scale
scores often are, make the pooled variance less variable than that law
says, so a plan (for any assurance above one half) delivers more
assurance than was requested; heavy tails make it deliver less.
true_kurtosis (default 0) is the population excess kurtosis of the
errors, a planning value on the scale kurtosis() reports, and it
replaces the error degrees of freedom df = J(n - 1) in the chi square
quantile with the effective degrees of freedom
2 / (2 / df + true_kurtosis / (n J)), which match the
variance of the pooled variance (Box, 1953). The default reproduces
the plans of earlier releases exactly, and the expected width plan (no
assurance) does not use the argument. An assurance plan made with a
nonzero true_kurtosis keeps its one row, carries the value as the
attribute true_kurtosis, says in a line under the table that its
assurance allows for that excess kurtosis, and reports it as the column
true_kurtosis of glance(); with true_kurtosis = 0 no attribute or
line is added. In a
Monte Carlo check with two groups and an assurance of .80 (10,000
replications per condition), the plans made with the kurtosis supplied
met the width target in .811 of the replications under uniform errors
(excess kurtosis -1.2) and .811 under t errors on 10 degrees of
freedom (excess kurtosis 1), where the normal theory plans met it in
.915 and .776. The help page describes the approximation and its
limits, and says that ss_aipe_c_sensitivity(), which is unchanged,
simulates normal errors.common_method_adjusted_r() gives every marker adjusted correlation
with a confidence interval and the t test of Lindell and Whitney
(2001), over one or more marker correlations.
common_method_marker() works from the correlation matrix alone and
does not take N, so its page left the test to the user. The new
function takes N and returns one row per marker value and variable
pair: the correlation, the adjusted correlation, its interval, and the
t test on N - 3 degrees of freedom. The interval is the Fisher's Z
interval of ci_r() carried through the adjustment, conditional on
the marker correlation. A grid of values for marker_r is how the
dependence of a conclusion on that correlation is read. With marker_r = NULL the smallest positive correlation serves as the proxy marker,
as in common_method_marker(). common_method_marker() is unchanged,
and its page now points to the new function. A lower limit that the
adjustment carries below -1 is set to -1; that happens exactly when
the Fisher's Z lower limit for the correlation falls below 2 r_M - 1,
which with a positive marker correlation a small sample can produce
whatever the sign of the correlation.
The missing-information t reference checks the fitted method.
With missing values, mlmr() and mlmr_mv() refuse the keyword when
lavaan fell back to listwise deletion, including under a
Satorra-Bentler test. Complete data retain N minus the regression
parameter count. fa_confirmatory() refuses robust.sem and Satorra-Bentler
tests under FIML whether values are missing or not, and checks the
fitted method for other fallbacks. The help correctly states that its
default se = "standard" is normal theory for ML and MLR; an MLR
test needs se = "robust.huber.white" for sandwich standard errors
under FIML. With a covariance matrix, robust.sem needs the
asymptotic moment covariance in NACOV. Alpha keeps its calibrated
refusal of the keyword. The loading coverage maximum is qualified by
whether values are missing; complete-data coverage uses t on N minus
one.
Reliability's missing-information diagnostics use the caller's item and auxiliary names. The warning about a fit short of its likelihood maximum names the parameter in words. It recommends dividing every item by one common constant, which preserves omega, and each auxiliary by its own, rather than an unavailable refit option.
Categorical MI diagnostics describe the interval actually formed. Under bootstrap then impute, no widening or imputation-count advice is given for an interval that was not formed. An FMI estimated from the minority of replications that returned a value is labeled as such. An even imputation count is named only for completed data sets that failed, rather than an imputation failure alone.
ci_dunnett_t3() gives Dunnett's T3, the pairwise comparisons SPSS
reports when the variances are not assumed equal. Each pair has its
own separate-variance standard error and Welch-Satterthwaite degrees of
freedom, as in ci_games_howell(), and its critical value is the
studentized maximum modulus for all a(a - 1)/2 comparisons at those
degrees of freedom (Dunnett, 1980), the value cv_smm() returns. The
adjusted p-value is that distribution's tail beyond |t|, so it is below
alpha when the interval excludes zero and not otherwise, to numerical
precision; neither is simulated, and the degrees of freedom are not
rounded. The table has the columns and class of the other post hoc
intervals (contrast, mean_difference, se, df, t_value, the
limits, and p_adjusted, class dmar_post_hoc_ci), and its display
names the adjustment. It reproduces the Dunnett's T3 output of SPSS for
Chapter 12, Exercise 25 of Maxwell, Delaney, and Kelley (2027) and
agrees with PMCMRplus::dunnettT3Test(). The help explains why the
formula that raises the single t probability to the power C is
Tamhane's T2 (Tamhane, 1979), a different procedure whose intervals are
never narrower, and when T3 is preferred to Games-Howell.
ci_games_howell() and cv_smm() link to it.ci_proportion_paired() gives a confidence interval for the
difference between two proportions measured on the same units, and
equivalence_proportion_paired() the score tests of noninferiority and
equivalence on that difference. Two diagnostic procedures applied to
the same patients, or a pass rate before and after instruction for the
same students, give correlated outcomes, and only the discordant pairs
inform the difference, so the methods for independent proportions do
not apply. ci_proportion_paired() takes either the two 0/1 vectors or
the counts n_10, n_01, and N, and returns the score interval of
Tango (1998), which inverts the score statistic of Nam (1997). The
interval stays inside [-1, 1], is defined when no pair is discordant,
and excludes 0 exactly when McNemar's test without a continuity
correction rejects. The Wald interval is available for comparison; the
help shows, from the exact coverage of every possible table, how far
short of its level it can fall when discordant pairs are rare.
equivalence_proportion_paired() reports in the shape of
equivalence_c(), with the flag nonsuperior (the upper limit is
below prop_diff_upper) as the last row and the verdict "Not
relevantly superior only" among its outcomes. prop_diff_upper alone
gives symmetric bounds; prop_diff_lower alone is a noninferiority
test, whose table
leaves the rows that would need an upper bound NA rather than
reading them against a mirrored one. The tests reproduce Nam's
restricted estimates and score statistic (2.795) for the liver scan
data he takes from McNeil, Keeler, and Adelstein (1975), and the
limits match PropCIs::scoreci.mp(), against which a live comparison
joins tools/oracle_checks.R.ci_ratio_of_means() and ci_ratio_of_means_paired() give a
confidence interval for the ratio of two means, for independent groups
and for two variables measured on the same units, from the raw data or
from summary statistics. The default is Fieller's confidence set, which
inverts the t test of mu_1 - rho mu_2 = 0 and is reported as what it
is: a bounded interval, the complement of an interval (whose gap is in
the excluded_lower and excluded_upper rows), or the whole line, with
the denominator's t that decides which beside it. Independent groups
default to unequal variances, inverting the Welch-type test with degrees
of freedom that change with the ratio, so that the interval agrees with
the test; var_equal = TRUE gives Fieller's pooled interval. The paired
function takes cluster for the cluster-robust covariance on Q - 1
degrees of freedom. A baseline argument gives the ratio of two
advantages over a baseline, the retained fraction of a three-arm
noninferiority design, and mean_baseline, sd_baseline, and
n_baseline give it from the three arms' summary statistics;
noninferiority at a fraction f holds exactly when f is below the lower
limit of a bounded interval with a positive denominator, and the help
says so rather than testing chosen fractions. The log-scale interval of
the response ratio, and percentile and BCa bootstrap intervals
(B = 10000, seed = NULL, written in base R), are offered for
comparison; the line under a log or bootstrap interval says when the
Fieller set at the same level is unbounded, where a bounded interval can
undercover. tidy() carries the gap and the form of the set, so a
complement is not read as the whole line, and results_sentence()
writes such a set as what it is ("99.5% confidence set: every value
except those between 0.90 and 147.05") rather than as "99.5% CI [-Inf,
Inf]"; the sentence of every other table is unchanged. The log interval
of a ratio whose terms are not both positive, and a bootstrap from
summary statistics, are refused with conditions of class
dmar_log_ratio_undefined and dmar_bootstrap_needs_data, which
inherit dmar_refusal. The paired page works Fieller's (1954) own
example, whose set takes all three forms at three confidence levels.equivalence_c() reports nonsuperiority. A nonsuperior row, the
last of the table, is 1 when the upper confidence limit is below
contrast_upper: the contrast is then shown to be smaller than the
smallest advantage that would matter, the mirror of noninferior and
the decision of the upper one-sided test (p_upper, which reproduces
emmeans::test() with side = "nonsuperiority"). The verdict gains a
sixth outcome, "Not relevantly superior only", checked after
"Noninferior only" and before "Inconclusive", which now means an
interval that contains both bounds. This is the claim a study needs
when contrast_upper is the improvement that would justify a change.
plot_equivalence() applies the same rule through one shared helper,
draws the sixth verdict in the palette's sixth color, and keeps the
colors of the other five; its legend sets each one-sided verdict beside
the stronger one that implies it.
Noninferiority is planned without an upper bound, and paired
designs are planned. power_equivalence_c() and
ss_power_equivalence_c() require contrast_upper only for side = "equivalence". For side = "noninferiority" they take the margin,
contrast_lower, and a call that gives only contrast_upper, as 1.0.x
required of its delta_upper, still uses it as the margin, so no
earlier call changes but for the names of the bounds.
Both take a new last argument, design = c("independent", "paired"),
and c_weights now defaults to NULL (it is still required for
independent groups). With "paired", true_sd is the standard
deviation of the per-unit contrast scores (for two conditions, the
difference scores), and the standard error is sigma / sqrt(N) on N - 1
degrees of freedom, where N is the number of units. Each design has
its own size, because the two are different quantities:
power_equivalence_c() takes n_per_group for independent groups and
N, the number of units and so the total sample size, for a paired
design, and refuses the size of the other design with a condition of
class dmar_argument_invalid. The planner answers in the same terms:
the rows of an independent plan are necessary_n_per_group,
total_N, and actual_power, as in 1.0.x, and the rows of a paired
plan are necessary_N and actual_power; tidy() reports the planned
size and the power in the same columns for both. At 50 units, a
difference score SD of 1, and a margin of .5, the noninferiority power
is .9672, and 36 units reach .90. Independent designs give the results
they did.
ss_aipe_crossed() plans the numbers of units and raters for a fully
crossed design, every rater scoring every unit, so that the interval
for the mean is narrow enough at the least cost. Units and raters
are both random, so the variance of the grand mean is
var_unit / n_u + var_rater / n_r + var_residual / (n_u n_r), and with
n_r raters no number of units brings the half-width below
z sqrt(var_rater / n_r); the plan reports that rater floor. A design
costs unit_cost per unit, rater_cost per rater, and
judgment_cost per score (by default the number of scores), and the
search prices the smallest feasible number of units for every number
of raters up to max_raters (or for a panel fixed by n_raters).
quantile = "t", the default, plans the interval ci_crossed()
computes, with Satterthwaite (1946) degrees of freedom from the
planning values; quantile = "normal" reproduces planning that uses
the normal quantile. With the t quantile the planned width is not
monotone in the number of units (the degrees of freedom eventually
fall toward n_r - 1, and past a point the larger quantile outweighs
the smaller variance), and the search takes the first feasible
design. An assurance rests on the chi
square approximation to the plug-in variance, and a plan with an
assurance is never smaller than the plan without one, which already
meets the width somewhat more than half the time; an assurance at or
below one half returns that plan. A target no design within the
limits meets is refused with a message naming the limit that binds,
max_units or the number of raters. With judgments as the
only cost the plan lies near the ratio n_u / n_r = var_unit /
var_rater of Marcoulides and Goldstein (1990). The help gives the
mapping to an information retrieval evaluation, in which the units
are topics and the raters are relevance assessors.ss_aipe_crossed_sensitivity() draws complete arrays from the
true components at a planned or specified design and reports the
realized widths, the proportion at or below the target, and the
coverage of the true mean, in the return schema of the sensitivity
family, with seed = NULL and filename = NULL. The planner's
arguments, its search limits included, pass through to it.ci_crossed() gives the interval for the mean of a complete units
by raters array: the ANOVA estimates of the three components, a
negative one set to zero, plugged into the variance of the grand mean,
with Satterthwaite degrees of freedom. Its var_unit, var_rater,
and var_residual rows are the planning values ss_aipe_crossed()
takes. On the four judges and six targets of Shrout and Fleiss (1979)
it reproduces their mean squares and shows how much wider the
interval for the mean over a population of judges is than one that
treats the four judges as fixed.tidy() and glance() read necessary_n_units as the planned size
of a crossed design, with necessary_n_raters (or
specified_n_raters) kept beside it.A family for the mean of a target measurement that is costly to
obtain, when an inexpensive proxy is available on a large pool of
units. The design is two-phase, or double, sampling (Neyman, 1938):
the proxy is measured on the whole pool and the target on a random
subsample of it, the labeled units. ci_two_phase() gives the
interval for the population mean of the target from the regression
estimator, with standard error from the residual variation among the
labeled units and the proxy's variation over the pool, optional
cluster standard errors whose degrees of freedom count the clusters
among the labeled units, and units with a missing value set aside one
by one; it refuses a target that is constant on the labeled units
rather than report an interval of zero width.
method = "prediction_powered" gives the prediction-powered
estimate of Angelopoulos et al. (2023), which fixes the proxy's weight
at 1 where the regression estimator learns it. The table reports the
estimate as mean_Y, named for its estimand, the population mean of
the target, with its standard error, degrees of freedom, and limits,
and the weight the estimate gives the proxy mean of the unlabeled units
as proxy_weight (the slope times N / M for the regression estimate, 1
for the prediction-powered one), beside slope, correlation,
proxy_bias, se_ratio_target_only, n_labeled, and n_pool.
ss_aipe_two_phase()
plans the number of labeled units for a given pool, for the expected
width or with assurance, always leaving at least one unit with the
proxy alone; it reports beside the plan the number needed without the
proxy, which is the plan ss_aipe_mean() makes, and refuses a width
the pool cannot deliver rather than returning a missing value.
ss_aipe_two_phase_cost() chooses both sizes: given the cost of a
proxy measurement relative to a target measurement, it returns the
least costly design in whole numbers whose ci_two_phase() interval
has an expected width no larger than the one asked for. The width is
that of the interval as it is reported, with the t quantile on
n - 2 degrees of freedom, which is the criterion ss_aipe_two_phase()
uses for a given pool, so that planner, given the returned pool,
returns the returned number of labeled units; the alternative of
measuring the target alone is the necessary_n_target_only plan of
that function, and a two-phase design is returned only when it costs
less. The search is exact: for each number of labeled units the
smallest pool follows from the width, a convex lower bound on the cost
(from the normal quantile, which the t quantile is never below) says
which numbers of labeled units can still beat the best design found,
and those are all examined. In 2,200 planning sets drawn at random the
result equaled an exhaustive search over pairs of sizes every time.
The rows are necessary_n_labeled, necessary_n_pool (0 when the
design measures the target alone), relative_cost (the cost of the
returned design over that of the target alone, both in whole numbers),
relative_cost_continuous (the same ratio for fractional sizes under
the normal quantile, which is a lower bound and is 1 at or below
threshold_cor), expected_width, necessary_n_target_only,
threshold_cor, and the inputs. threshold_cor is exact for
fractional sizes only; above it the least costly design in whole
numbers can still be the target alone, and is then returned. Planning
values whose sizes cannot be represented as whole numbers, or whose
search would examine more than 1e8 numbers of labeled units, stop with
an error of class dmar_two_phase_no_design. For a width of
0.20 with a standard deviation of 1, a correlation of .90, and a cost
ratio of .05, the design is 109 labeled units in a pool of 1,011, at a
cost of 159.55 target measurements against 387 for the target alone,
with an expected width of 0.19999.
ss_aipe_two_phase_sensitivity() evaluates a design by
simulation under the family's return schema, from a bivariate normal
population or by resampling observed pairs; it reports the population
target mean as true_mean_Y, the simulated estimates of it as
mean_mean_Y, median_mean_Y, and sd_mean_Y, and the estimate in
its per-replication file as mean_Y. ss_seq_two_phase()
evaluates the sequential stopping rule, which needs no planning
values, in the form of ss_seq_c(); it asks for more labeled units
while the target is constant on them or is an exact linear function of
the proxy on them, where the rule could stop on no evidence. The
vocabulary is that of measurement (target, proxy, labeled units,
pool); each help page says in one sentence how the terms map onto the
evaluation of an AI system's outputs.
ss_seq_c()'s help page says when to use it and how it differs from
the balanced procedure of Kelley, Darku, and Chattopadhyay (2018).
ss_aipe_two_phase_cost() refuses a cost_ratio that is not a single
number above 0 and below 1 with a condition of class
dmar_cost_ratio_invalid, which carries the value given.
The regression, correlation, and coefficient of variation functions now name the total sample size, the number of predictors, a planner's planning values, and the desired width one way. No function is renamed and no result changes for a given model. The DMAR 1.0.x names are not accepted; Breaking Changes in 1.1.0, at the top of this section, lists every old and new name.
p is the number of predictors in the partial and semipartial
correlation family, as it is in ss_power_R2(), ci_R2(), and
ci_reg_coef(). var_partial_r(), var_semipartial_r(),
expected_partial_r(), ss_aipe_partial_r(), ss_aipe_semipartial_r(),
ss_aipe_partial_r_sensitivity(), and
ss_aipe_semipartial_r_sensitivity() took J, the number of variables
partialed out. They now take p, the number of predictors in the
regression of Y on X and the variables partialed out, so p - 1
variables are partialed out, p is at least 2, and p = J + 1: a
partial correlation of Y and X given two other variables was J = 2 and
is p = 3. A categorical variable with a levels counts as a - 1
predictors. The degrees of freedom are then the regression ones: the
raw-scale variance of a partial correlation divides by N - p, Fisher's
Z variance is 1 / (N - p - 2), the squared standard error of the t
test of a semipartial correlation is (1 - R2_full) / (N - p - 1), and
the planners return N = p + ceiling(z^2 V / h^2) for a half-width h.
p holds the place J held, so a 1.0.x call that gave the count by
position must add 1 to it (see Breaking Changes in 1.1.0). In the
notation of a model comparison, q is the number of predictors removed
from the full model to give the restricted model
(q = p_full - p_restricted, the predictors being tested) and the number
of variables partialed out is p - q; a partial or semipartial
correlation concerns one predictor, so q = 1 and p - 1 variables are
partialed out. The help pages of the family and of
ss_power_R2_change() say so, and ss_power_R2_change() reports a row
q. No result changes for a given model.
N is the total sample size. var_partial_r(),
var_semipartial_r(), expected_partial_r(), ci_r(), var_r(),
expected_r(), equivalence_r(), ci_cv(), and var_cv() took n
and take N, as ci_R2(), ci_R(), cv(), and the regression
functions already did; ci_r() and ci_R(), which share a help page,
now document one N. A size per group is n_per_group and a size per
cell is n_per_cell; n alone remains only for one size per study or
per plotted effect (meta_r(n = ) is a vector of study sizes). The
second position is unchanged, so ci_r(.3, 100) runs as it did;
ci_r(r = .3, n = 100) stops with "unused argument". The other
functions that took a total sample size as n take N too
(var_alpha(), var_icc(), ci_proportion(), ci_eigenvalue(), and
the rest of the table in Breaking Changes in 1.1.0). The divisor of
ecvi() was n and is multiplier, because it is the multiplier of
the fit function (N, or N less the number of groups), not always the
total.
A planner names its planning value as its sensitivity function names
the data generating value, true_X. The planner treats the value as
the truth; the sensitivity function shows what the plan delivers when
the truth (true_X) differs from the value planned with
(supposed_X, which was estimated_X).
The planners are ss_aipe_cv(), ss_aipe_R2(), ss_power_R2(),
ss_aipe_r(), ss_power_r(), ss_aipe_partial_r(),
ss_aipe_semipartial_r(), and ss_aipe_equivalence_r(); the
sensitivity function ss_aipe_equivalence_r_sensitivity(), which took
true_r and estimated_r, takes true_cor and supposed_cor. The
planners for a regression coefficient take true_reg_coef_j
(ss_aipe_reg_coef(), ss_aipe_rc(), ss_power_reg_coef(),
ss_power_rc()) or, for the standardized coefficient,
true_std_reg_coef_j (ss_aipe_src()) for the coefficient the plan is
made for, and ss_aipe_cv() takes true_mean and true_sd where it
took mu and sigma; the mean that ss_aipe_cv_sensitivity()
generates data from is true_mean, as in ss_aipe_mean_sensitivity().
ss_aipe_cv(true_cv = .25, width = .10, conf_level = .99) is the worked
example of Kelley (2007, pp. 759--760; the call is printed on p. 765),
which gives 99; with the exact
noncentral t distribution the function returns 101 (see Changes to
Earlier Results). The full-model planning value that
ss_aipe_semipartial_r() gains in this release is true_R2_full.
var_r(), expected_r(), and expected_partial_r(), which evaluate a
formula at a population correlation, take it as true_cor (rho), the
name in the planners, and var_R2() and expected_R2() take the
population squared multiple correlation as true_R2 (population_R2),
as ss_aipe_R2() and ss_power_R2() do; expected_R2() reports its
value in the row expected_R2 (expected_value_population_R2), as
var_R2() reports var_R2. The null value of ss_power_r() was
rho_0 and is null_cor, as null_R2 and null_rmsea are the null
values of their planners.
width is the desired width. ss_aipe_R2_sensitivity(),
ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() took w and take width, as their planners
do.
true_cor_YX and true_cor_XX. ss_aipe_R2_sensitivity() and
ss_power_R2_sensitivity() took rho_yx and rho_xx, the correlations
their data are generated from, and take true_cor_YX and true_cor_XX,
the names in the coefficient planners.
Every population value of a coefficient planner carries true_.
ss_aipe_reg_coef(), ss_aipe_rc(), ss_aipe_src(),
ss_power_reg_coef(), and ss_power_rc() took rho2_Y_X,
rho2_j_X_without_j, rho2_Y_X_without_j, sigma_Y, sigma_X,
rho_XX, and rho_YX beside b_j or beta_j; each now has the prefix
the coefficient has and names its quantity: true_R2_Y_X,
true_R2_j_X_without_j, true_R2_Y_X_without_j, true_sd_Y,
true_sd_X, true_cor_XX, and true_cor_YX, beside
true_reg_coef_j or true_std_reg_coef_j. Cohen's f squared was
cohen_f2 in ss_power_R2(), ss_power_reg_coef(), and
ss_power_rc() and is
true_f2, the name of the row that reports it in ss_power_R2() and
ss_power_R2_change(); the row of the two coefficient planners that
holds its square root, Cohen's f, was effect_size and is true_f.
specified_N in ss_power_r(). The sample size at which power is
wanted was N and is specified_N, the name in ss_power_R2() and
ss_power_reg_coef() and the name of the row the function returns.
The wrappers spell their arguments as the functions they call do.
ss_aipe_rc() and ss_aipe_src() took Rho2_j_X_without_j and
sigma_X_j and take true_R2_j_X_without_j and true_sd_X, the
names in ss_aipe_reg_coef(); ss_power_rc() took Rho2_j_X_without_j
and takes true_R2_j_X_without_j, the name in ss_power_reg_coef().
The row for the full-model planning value of ss_aipe_semipartial_r(),
new in this release, is true_R2_full. A row that echoes an argument has
the argument's name, and one quantity has one row name: the desired width
is the row width (seven planners and design_consequences() called it
width_target or target_width), the expected width is expected_width
(three planners called it ci_width_expected), and the null value of
ss_power_r() is null_cor. necessary_N, specified_N, and total_N
keep their names.
ss_power_R2_change() plans the sample size for the test that
predictors added to a regression model contribute nothing, the model
comparison F test of a restricted model with p_restricted predictors
against a full model with p_full, and returns the power at a
specified_N. DMAR could plan the omnibus test of all predictors
(ss_power_R2()) and, with fixed predictors, the test of one coefficient
(ss_power_reg_coef()), and had no planner for a set of added
predictors or for one added predictor when the predictors are random.
The effect is given as any two of true_R2_full, true_R2_restricted,
and true_R2_change, or as true_f2 alone. The change in R^2 is the
squared semipartial correlation of the added predictor when one is added
and the squared multiple semipartial correlation of the added set when
several are; divided by one minus the restricted model's R^2 it is the
squared multiple partial correlation, and Cohen's f^2 is that partial
value divided by one minus itself, (rho^2 full - rho^2 restricted) /
(1 - rho^2 full). The help page sets these relations out, with the error
degrees of freedom of the two models in the notation of Maxwell,
Delaney, and Kelley (2027).
With fixed predictors (random_predictors = FALSE) the test statistic is
noncentral F with p_full - p_restricted and N - p_full - 1 degrees
of freedom and noncentrality N f^2. With one added predictor that is the
calculation of ss_power_reg_coef() (the two agree to 1e-8 on 81
combinations, and both give 50 for a change from .30 to .40 with five
predictors and power .80), and with no predictors in the restricted model
it is ss_power_R2() (agreement to 1e-12 on 54 combinations, both
predictor models). For five predictors that explain .25 and four added
predictors that raise that to .30, at an alpha of .01, the power at N = 90
is .2413 and the sample size for power .80 is 242, the values the G*Power
3.1 manual prints for that design (.241297 and 242).
With random predictors, the default, the power comes from the
distribution of the sample squared multiple partial correlation, which is
that of a squared multiple correlation with the added predictors alone
and a sample size reduced by the number of predictors in the restricted
model (Gatsonis & Sampson, 1989, p. 518), computed with Lee's (1971,
Section 5.1) approximation as in ss_power_R2(). The same design needs
250, and the power at N = 90 is .2202. Against the exact distribution,
the power by Lee's approximation differed by at most .0043 over the
combinations examined (squared partial correlations up to .95, an alpha
from .001 to .10, and one or more error degrees of freedom) and by at
most .0004 with 25 or more error degrees of freedom; the necessary
sample size differed from the one the exact distribution gives, by one
case, in 20 of 1,500 searches, and in those the power by the exact
distribution at the size returned was within .0011 of desired_power.
The same holds for
ss_power_R2(), which shares the search. In a
Monte Carlo with raw data (20,000 replications in each of twelve
conditions, six with random and six with fixed predictors) the rejection
rate under the null hypothesis was within 2.0 standard errors of
alpha_level and the power within 1.9 standard errors of the value
returned.
Cohen's (1988) noncentrality, f^2 (u + v + 1), is f^2 (N - w) for an
increment with w predictors in the restricted model, where this function
uses N f^2 for fixed predictors; Cohen's value is the fixed-predictor
call with no restricted model at a sample size of N - w, which the last
example of the help page shows (noncentrality 6.071 and power .2225
where N f^2 gives 6.429 and .2413), and the necessary sample size by
Cohen's form is at most w above the fixed-predictor answer (247 against
242 in that design). The result has nine rows: necessary_N or
specified_N, actual_power, noncentral_f_parm (NA with random
predictors), true_f2 (the planning value of Cohen's f^2),
true_partial_R2, true_R2_change, q
(the number of predictors removed from the full model to give the
restricted model, p_full - p_restricted), df_1, and df_2.
A change of zero is refused with a classed condition.
ss_power_R2_change() and ss_power_R2() stop with a refusal of class
dmar_effect_zero when the planning values give an effect of exactly
zero (no sample size gives a test of a true null hypothesis power above
alpha_level); the message says what to give instead.
var_semipartial_r() gives the first-order variance of a semipartial
correlation, which depends on the full-model R2. Under multivariate
normality, N times the variance of the sample semipartial correlation
tends to (1 - rho_sp^2)^2 - q (1 - 2 rho_sp^2), where q is the full-model
R2 minus rho_sp^2, the squared multiple correlation of the outcome with
the variables partialed out. DMAR 1.0.x gave (1 - r_sp^2)^2 / (N - p),
which is that variance only when q = 0. When the variables partialed out
explain some of the outcome, the 1.0.x value is too large if rho_sp^2 is
below 1/2 and too small if it is above: at r_sp = .3 with a full-model R2
of .59 the numerator is .4181 where 1.0.x used .8281, and at r_sp = .8
with a full-model R2 of .94 it is .2136 where 1.0.x used .1296. At
r_sp = 0 the 1.0.x value was 1 / (N - p) whatever the R2, while the
same function with R2_full = .64 returned .36 / (N - p - 1). The new
argument under_null (default TRUE) chooses between the two quantities
that need R2_full: under_null = FALSE returns the first-order variance
with the divisor N - p, the quantity for a Wald interval and for
planning, and the default returns (1 - R2_full) / (N - p - 1) as before.
The help page now describes that default as what it is, the squared
standard error in the t test of X's coefficient, which equals the
first-order variance at rho_sp = 0; it is not a variance for an interval
about a nonzero r_sp. No value an existing call returned has changed. A
call without R2_full returns what it did and now warns (class
dmar_semipartial_r_no_R2_full) that the value holds only when the
variables partialed out explain none of the outcome. An R2_full smaller than r_sp
squared, which no data can produce and which 1.0.x accepted
(the 1.0.x call var_semipartial_r(.9, 100, 1, R2_full = .10) returned
.0093), is
refused with an error of class dmar_semipartial_r_incoherent, a
dmar_refusal, that says what to pass when the two values were rounded.
The help page states the limits of the approximation found in simulation:
it understates the variance when |rho_sp| is large and N - p is small,
and the sample semipartial correlation is biased toward zero by roughly
rho_sp (p - 1) / (2N).
The help pages no longer credit (1 - r_sp^2)^2 to Olkin and Finn (1995)
or to Algina and Olejnik (2003). Olkin and Finn (1995) do not treat the
semipartial correlation; their Model A is the difference between a
squared multiple correlation with two predictors and a squared simple
correlation, from whose delta method variance the result above follows
for one variable partialed out (Alf and Graf, 1999, give the difference
between two squared multiple correlations with any number of
predictors), and the pages now say so. The Yuan and Chan (2011) entry
leaves the references of var_semipartial_r(), since the sentence that
cited it is withdrawn. Algina and Olejnik (2003, p. 317) are cited for what
they state, that for the squared semipartial correlation the required
sample size is largest when the squared multiple correlation of the
restricted model is zero. The sentences that described the 1.0.x form as
Fisher's (1924) result applied by analogy, and as matching the leading
1/N behavior of the variance in Aloe and Becker (2012), are withdrawn;
Aloe and Becker (2012) are cited for the effect size.
ss_aipe_semipartial_r() takes true_R2_full, a planning value of
the full-model R2, and plans from the first-order variance at that
value: N = p + ceiling(z^2 V / w^2), with V as above and w the
half-width. For a planning value of .2, three variables partialed out
(p = 4), and a width of .10, necessary_N is 714 with
true_R2_full = .54, where the plan without it (and DMAR 1.0.x) gives
1421. The result gains a last row, true_R2_full, which is NA when none
was given. Without true_R2_full the plan uses the largest V over every
full-model R2 that is possible with true_r_sp, so the sample size is
sufficient whatever that R2 is (see Changes to Earlier Results for the
cases in which this differs from 1.0.x). The count of predictors must be
a whole number, where the 1.0.x call
ss_aipe_semipartial_r(r_sp = .3, J = 1.5, width = .2) had returned a
necessary_N of 321.5. A plan with true_r_sp = 0 and
true_R2_full = 1 is refused: the first-order variance is zero there.
ss_aipe_semipartial_r_sensitivity() simulates a population in which
the variables partialed out can explain part of the outcome, through
the new arguments true_R2_full (the population's full-model R2; by
default true_r_sp squared, the population 1.0.x simulated) and
supposed_R2_full (the planning value passed to the planner). Under
normality the distribution of the sample semipartial correlation depends
on the population only through the semipartial correlation, the
full-model R2, p, and N, so the two values describe every normal
population. Its result gains the rows true_R2_full and
supposed_R2_full. Each interval is now the Wald interval the planner
plans for (see Changes to Earlier Results). In 20,000 replications of a
plan for a width of .10, the intervals missed the true value in .053 of
samples at r_sp = .8 with a full-model R2 of .94 (N = 331, mean width
.0995) and in .051 at r_sp = 0 with a full-model R2 of .64 (N = 556,
mean width .0998); the Monte Carlo standard error of each rate is .002.
The pages of ss_aipe_semipartial_r() and var_semipartial_r() name
two further conditions in which the interval misses too often: a
full-model R2 near 1 with a small N, at any r_sp (.13 at r_sp = .1,
a full-model R2 of .99, p = 4, and the planned N of 12), and |r_sp|
above about .9 with variables partialed out that explain some of the
outcome, where the misses fall mostly on one side even with several
hundred observations (.064, .051 of it on one side, at r_sp = .95, a
full-model R2 of .97, p = 2, and N = 395).
ss_aipe_partial_r() with assurance returns the smallest sample
size that attains it, computed from the distribution of the sample
partial correlation. At a given sample size the width of either
interval the function plans depends on the sample only through r and
narrows as |r| grows, so the interval is no wider than the target
exactly when |r| is at least a value r* that the function solves for.
The probability of that event is taken from the distribution function
of the sample partial correlation, which under multivariate normality
is that of a simple correlation from N - (p - 1) observations, written as an
integral of a central t probability against a beta density (R's
noncentral t is an approximation above a noncentrality of 37.62,
which plans for narrow intervals reach). The function returns the
smallest sample size, at or above the one planned without assurance,
at which the probability is at least assurance, and two rows are
appended to the table: assurance and assurance_attained, the
probability at the returned sample size. The search does not go below
the plan without assurance, because below it the probability is not
monotone in the sample size. DMAR 1.0.x multiplied the sample size by
qchisq(assurance, df) / df, the step for an interval whose width is
proportional to a standard deviation; here it gave too few cases when
the correlation is large in absolute value and too many when it is
small. With one variable partialed out (p = 2), a width of .10, and
an assurance of .80:
| rho | 1.0.x | now | attained now | |-----|-------|------|------| | 0 | 1586 | 1539 | 1.000 | | .30 | 1318 | 1310 | .807 | | .50 | 902 | 915 | .803 | | .70 | 426 | 448 | .804 | | .85 | 135 | 151 | .807 | | .95 | 23 | 28 | .805 |
The page's example (true_cor = .30, p = 3, width = .20,
assurance = .80) returns 338, where it returned 344. In 16 further
conditions simulated from regression data (20,000 data sets each, the
variables partialed out by least squares; Monte Carlo standard error
.0015 to .0025), the 1.0.x sample sizes delivered from .139 below the
request to .145 above it (.711 for a requested .85 at rho = .80, and
1.000 for a requested .95 at rho = .20), and the sample sizes now
returned from .002 to .012 above it, each within .004 of
assurance_attained. At rho = 0 the plan without assurance already
meets the width in every sample, and assurance adds nothing. No
argument restores the earlier sample sizes.
ss_aipe_semipartial_r() with assurance finds the sample size by
simulation, and takes G and seed. The width of this interval
depends on two statistics, the sample semipartial correlation and the
sample full-model R2, so the function simulates them: for each sample
size tried it draws G (default 20,000) samples from a multivariate
normal population with the planning values, through the entries of the
triangular factor of the Wishart matrix that the two statistics depend
on, and takes the share of intervals no wider than the target. It
returns the smallest sample size found, at or above the one planned
without assurance, at which that share is at least assurance.
Three rows are appended: assurance, assurance_attained (the share
in a fresh set of G draws at the returned sample size, which had no
part in choosing it), and G; the Monte Carlo standard error of the
attained share is the attribute assurance_mc_se. The answer has
Monte Carlo error: without seed two calls can differ by a few cases
(eight unseeded plans for one condition gave 252 to 254; 40 seeded
plans of each of two other conditions gave 470 to 473 and 306 to 310,
and the probability at the smallest of those sizes was .897 and .895
for a requested .90), and assurance_attained can fall below the
request by a few of its standard errors (the standard error is .003 at
the default G and an assurance of .80; in 200 seeded plans the
lowest value was .009 below a request of .90). The sample size
returned is therefore the one at which the probability is about the
request, and assurance_attained is an unbiased estimate of the
probability at it. Without true_R2_full the simulated population is the
one the plan without assurance is made for (variables partialed out
that explain none of the outcome when true_r_sp squared is at most 1/2, a
full-model R2 of 1 otherwise). G and seed are checked and used only
with assurance; G must be at least 100, because the step estimates
a probability from those samples (G = 1 returned an attained
assurance of 0 or 1). DMAR 1.0.x multiplied by the same chi square factor
as the partial correlation planner. With an assurance of .80 (now with
seed = 113):
| true_r_sp | true_R2_full | p | width | 1.0.x | now | attained now (fresh draws) | |------|---------|---|-------|-------|-----|------| | .25 | not given | 4 | .15 | 635 | 621 | .802 | | .30 | .09 | 3 | .20 | 344 | 337 | .814 | | .70 | .49 | 3 | .15 | 197 | 214 | .808 | | .85 | .7225 | 3 | .15 | 65 | 89 | .798 | | .80 | not given | 2 | .10 | 219 | 370 | .824 |
(DMAR 1.0.x had no true_R2_full; it planned every row with variables
partialed out that explain none of the outcome. In the last row, where
true_r_sp squared is above 1/2, this release plans without
true_R2_full for the full-model R2 at which the variance is largest,
which accounts for most of the change from 219.) In 11
further conditions checked from data with
ss_aipe_semipartial_r_sensitivity() (5,000 data sets each; Monte
Carlo standard error .003 to .006), the earlier sample sizes delivered
from .37 below the request to .02 above it (.530 for a requested .90
at a semipartial correlation of .90 and true_R2_full = .81; .574 and
.623 for a requested .80 at .80 and -.70 with true_R2_full equal to
true_r_sp squared), and the
sample sizes now returned delivered .798 to .819 for .80, .854 to .858
for .85, .906 to .908 for .90, and .953 for .95. No argument restores
the earlier sample sizes. The page
still says that a plan whose sample size is small (a large |r_sp| with
true_R2_full near true_r_sp squared) should be checked with the
sensitivity
function, because the first-order variance is too small there.
The count of variables partialed out must be a whole number in the
seven functions of the partial and semipartial correlation family
(var_partial_r(), var_semipartial_r(), expected_partial_r(),
ss_aipe_partial_r(), ss_aipe_semipartial_r(), and the two
sensitivity functions). A fractional count (J = 1.5 in 1.0.x) had been
accepted and carried into the result. The count is now given as the
number of predictors, p, a whole number of at least 2 (see "One
Notation for the Regression and Correlation Functions"). The two
sensitivity functions also check G (a whole number of at least 1) and
specified_N (a whole number of at least p + 3 for the partial
correlation on Fisher's Z scale, whose interval needs N - p - 2 > 0,
and p + 2 for the raw-scale interval and for the semipartial).
ss_aipe_partial_r_sensitivity() simulates the interval that was
planned, and takes fisher_z. The planner plans the raw-scale
interval, r plus or minus z (1 - r^2) / sqrt(N - p), unless
fisher_z = TRUE; the sensitivity function planned that interval and
then formed Fisher's Z interval in every replication, so its
pct_ci_less_width did not describe the plan. It now takes fisher_z
(appended after filename), passes it to the planner, and forms the
same interval in each replication; the raw-scale limits are not
truncated, and the smallest sample size is p + 2 for the raw-scale
interval and p + 3 for Fisher's. The table carries the attribute
interval ("raw" or "fisher_z") and prints a line that names the
interval. For true_cor = supposed_cor = .70, p = 3, width = .20
(N = 103), 2,000 replications under set.seed(7): mean width .203,
pct_ci_less_width .470, and non-coverage .046 in DMAR 1.0.0 (Fisher's
interval at the raw-scale plan, with J = 2), and .200, .516, and
.062 now; with
fisher_z = TRUE the plan is 107 and the three are .199, .521, and
.050. No argument restores the 1.0.x results of this function or of
ss_aipe_semipartial_r_sensitivity().
A full-model R2 that falls below the squared semipartial correlation
only by rounding is accepted, with a warning. A semipartial
correlation reported as .46 has a square of .2116, and the full-model
R2 beside it may be reported as .21. The 1.0.x call
var_semipartial_r(.46, 100, 1, R2_full = .21) (one variable
partialed out, J = 1 by position) returned .00814433 in DMAR 1.0.0.
The pair is checked in this release, since a full-model R2 cannot be
below the squared semipartial correlation. A
pair that two-decimal rounding can produce, R2_full + .005 >= max(0, |r_sp| - .005)^2, passes with a warning of class
dmar_semipartial_r_rounded (fields R2_full and r_sp):
var_semipartial_r(.46, 100, p = 2, R2_full = .21) returns .00814433,
as it did (p = 2 is the 1.0.x count of 1 plus 1). What the warning says
follows what is computed. The squared standard error of the t test
(the default, under_null = TRUE) is (1 - R2_full) / (N - p - 1),
which does not involve r_sp, so the warning says that the value
uses R2_full as given. The first-order variance
(under_null = FALSE), the planner, and the sensitivity function
compute from the part of the full-model R2 that the partialed
variables explain, which is then taken as zero, and their warning
says that the variables partialed out are taken to explain none of
the outcome.
A pair further apart (R2_full = .15 with r_sp = .46) is refused as
before, with the class dmar_semipartial_r_incoherent. The same rule
applies in ss_aipe_semipartial_r() and
ss_aipe_semipartial_r_sensitivity().
The variance on ?var_partial_r and ?ss_aipe_partial_r is named
for what it is, and the variance of the semipartial correlation is
credited to the article that prints it. The pages called
(1 - rho^2)^2 / (N - p) the "Olkin-Finn (1995) asymptotic
variance". Olkin and Finn (1995, Equation 3, p. 157) give the
first-order variance of a simple correlation, (1 - rho^2)^2 / N; the
article has no variance of a partial correlation and no N - p
form. The pages now say "first-order variance", state that the sample
partial correlation given p - 1 variables is distributed as a simple
correlation from N - (p - 1) observations, and say that the divisor
N - p is the package's and differs from that at order 1/N^2; the
message of var_partial_r() for too small an N reads "Need N > p for
the first-order variance of the partial correlation."
?var_semipartial_r and
?ss_aipe_semipartial_r name the first-order variance of Aloe and
Becker (2012, Equation 4, p. 284), which they obtain from Alf and Graf
(1999), with N - p in place of their n, and show that its numerator is
the one the functions compute. No number changes.
vargha_delaney_A(), cliff_delta(), and
probability_of_superiority_paired() read their observations through one
check, and say what they will not order. The three statistics depend on
the data only through the order of the observations. They accept numeric
and logical vectors, and now ordered factors with the same levels in the
same order, which are read through their order: a five-category item
supplied as an ordered factor gives the value its integer codes give, and
the value any increasing rescoring of the categories gives. A one-column
matrix (the result of scale()) and a one-dimensional array (the result
of tapply()) are read as the vectors they are, and Date and POSIXct
vectors are ordered by their numeric values. vargha_delaney_A() did so
before; cliff_delta() and probability_of_superiority_paired() refused
logical vectors and dates ("must both be numeric vectors") and now accept
them. Infinite values have their place in the order in all three
functions. When an infinite value was compared with an equal infinite
value, cliff_delta() stopped ("missing value where TRUE/FALSE needed")
and the paired function returned a table of NA; a single infinite value
was already ordered correctly. Input with no common order is refused
with an error of class dmar_scores_not_ordered, a dmar_refusal (see
?dmar_metadata): an unordered factor, an ordered factor beside anything
else or beside an ordered factor with other levels, two difftime
objects in different units or one beside anything else, a Date beside a
POSIXct, and character, complex, list, and matrix input.
Missing values are removed and counted in all three.
vargha_delaney_A() has always removed them group by group.
cliff_delta() stopped on a missing value with "missing value where
TRUE/FALSE needed", and probability_of_superiority_paired() returned a
table in which every row was NA, with no message. cliff_delta() now
removes them group by group and the paired function removes every pair
with a missing member. Each result carries the numbers removed in its
n_missing attribute: c(group_1 = , group_2 = ) for the two-group
functions (through the formula interface of vargha_delaney_A() as
well), one count of pairs for the paired function. A conf_level that is
missing or not a single number gets the worded error in all three;
cliff_delta(x, y, conf_level = c(.90, .95)) stopped with "'length = 2'
in coercion to 'logical(1)'".
A grouping factor with an unused level is accepted by the formula
interface of vargha_delaney_A(). The two groups are the two levels
observed with an outcome; vargha_delaney_A(y ~ g, data = d) with
levels(d$g) of "a", "b", "c" and no row in "c" stopped with
"Grouping variable must have exactly two levels."
An estimated variance of 0 is reported the same way by all three, with
a classed warning. The estimated variance is 0 exactly when the groups
are completely separated (every observation in one group exceeds every
observation in the other; for pairs, every pair favors the same member)
or every observation has the same value. cliff_delta() already
reported both limits at the estimate with a warning under complete
separation. vargha_delaney_A(c(4, 5, 6), c(1, 2, 3)) returned
A = 1 with limits [1, 1] and no warning, and
probability_of_superiority_paired(c(1, 2, 3, 4), c(4, 5, 6, 7))
returned NaN limits and no warning. All three now report both limits
at the estimate and warn, with class dmar_interval_degenerate and a
reason element ("complete_separation" or "no_variation"), that
the limits are degenerate and are not a confidence interval with the
stated coverage. The warning of cliff_delta() keeps its 1.0.x opening
words through "completely separated (|delta| = 1)" and gains the class;
cliff_delta() and the other two
also warn when every observation has the same value, where the limits
are both at 0 (0.5 for A and the paired probability). Each help page
has a paragraph on the case.
The pairwise comparisons are computed from ranks, not from a
matrix of every pair. vargha_delaney_A() and cliff_delta()
built the matrix of all pairwise comparisons with outer(), about 36
bytes a pair. The placements and the proportions of pairs won, lost, and
tied now come from the mid-ranks, minimum ranks, and maximum ranks of the
pooled and the separate samples. The estimators, the variance
estimators, and the tie conventions are the same, and the results agree
with the earlier ones to rounding (see "Changes to Earlier Results").
With 6,000 observations per group, vargha_delaney_A() took 0.52 s and
about 1,240 MB, and now takes 0.03 s and under 30 MB; cliff_delta()
took 0.57 s and about 960 MB, and now takes 0.02 s and under 30 MB.
vargha_delaney_A() refers its test to Student's t by default, and
offers a studentized permutation test. The new argument
test = c("t", "permutation", "z") chooses the reference distribution
of the statistic W = (A - 1/2) / SE, which is the same under all
three and is reported in the column test_statistic (z_value in
1.0.x, a name that is wrong under the t and permutation references;
see the breaking changes). "t", the default, uses
Student's t with the Satterthwaite degrees of freedom of the two
placement variances, the Brunner-Munzel test as Schüürhuis,
Konietschke, and Brunner (2025, Equations 7 and 8) give it. "z" uses
the standard normal distribution, as DMAR 1.0.x did. "permutation"
recomputes W over reassignments of the pooled scores to the two
groups, with the arguments exact, n_resamples, and seed of
randomization_test() and its rules: every reassignment when there
are at most 50,000, random ones otherwise, and a p-value that is the
share of reassignments at least as extreme in absolute value. The
three arguments are read and checked only under "permutation". Two
columns are added after the nine of 1.0.x, df and test; tidy()
and glance() give them after p_value; the table records the
reassignments in a resampling attribute and prints "Permutation
test: all 252 reassignments" or "Permutation test: 10000 random
reassignments". The function reproduces the statistic 3.1375 on
17.683 degrees of freedom, p = 0.005786, of the pain score example
in the help of lawstat::brunner.munzel.test(), and the estimate 0.8
and p = 0.0239 of Table 2 of Schüürhuis et al. The permutation
p-value equals a brute-force enumeration in 400 small data sets with
and without ties. The normal reference rejects a true A = 1/2 too
often in small samples. At a nominal .05 with normal data (10,000
samples a condition for z and t, Monte Carlo standard error .002;
2,000 for the permutation test, standard error .005) the rates for
z, t, and the permutation test were .086, .071, and .047 with 5
per group; .073, .056, and .047 with 10 per group; .090, .062, and
.063 with 8 and 20 and the larger standard deviation, three times the
other, in the smaller group; and .060, .052, and .046 with 20 per
group. The permutation test is exact when the two distributions are
equal and approximate when they differ in spread with A = 1/2. With
completely separated groups it is the only one of the three that
gives a p-value (2/252 for 5 and 5 untied scores). The help has the
formulas, the table, and a paragraph on why the test and the logit
interval can disagree. See "Changes to Earlier Results".
Each refusal of input with no common order says why, what would have
been computed, and what to supply. The class is still
dmar_scores_not_ordered. For an unordered factor the message names
the argument and shows the levels of the object and the codes R
stores for them ("here the levels are high, low, mid, stored as 1, 2,
3"), says how the statistic would have ranked them, and gives the
call that states the order. A character vector has a message of its
own ("10" sorts before "9"); it was told only that it was of class
character. An ordered factor beside anything else is told what the
other group is and that the comparison would be of positions in a
list of levels with scores. Two ordered factors with different levels
are shown both lists and, where the two share a label under two
codes, that label and its codes. Every message says "Nothing was
computed." The condition of an unordered factor or a character vector
carries the argument at fault as arg. Each help page has a
paragraph "What is refused, and why", and ?vargha_delaney_A and
?cliff_delta an example: for the ratings low, mid, high, high
against low, low, mid, high, the codes of the unordered factors give
A = 0.406 where the ordered levels give 0.656.
The warning of an estimated variance of 0 explains the case, and
states the limit Feng and Cliff (2004) give for it. The class is
still dmar_interval_degenerate, and the reported limits do not
change. The message now says what the two reported limits are, why a
sample without overlap gives a variance estimate of 0, that this
describes the sample and not the population, how large the samples
are, what to report, and that the randomization test remains valid.
For completely separated groups cliff_delta() and
vargha_delaney_A() state the limit of Feng and Cliff (2004, p. 324,
Equations 6 to 8), (n_b - z^2) / (n_b + z^2) for a delta of 1 with
n_b the smaller group size: 0.131 at the 95 percent level with five
in the smaller group, and 0.566 on the scale of A. That limit is
not the one the functions report. The condition carries estimate,
n, and limit_feng_cliff beside reason. The help of each
function has a section "When the estimated variance is zero" that
names every case and what is returned: complete separation, no
variation, separation produced by infinite values, two constant
groups, groups that only touch (an ordinary, wide interval), and, for
pairs, wins with ties and fewer than three pairs.
A table whose limits are not a confidence interval carries the mark
with it. When the estimated variance is 0 the three functions set
the attribute interval_degenerate to "complete_separation" or
"no_variation"; dmar_metadata() returns it in a field of that
name; the printed table adds "Limits are not a confidence interval:
the estimated variance is 0."; and tidy() and glance() have a
logical column interval_degenerate, so that a stack of rows keeps
the mark. cliff_delta() and probability_of_superiority_paired()
return tables of the classes dmar_cliff_delta and
dmar_probability_of_superiority_paired (before dmar_tbl) for
this. results_sentence() reads the mark; see "Changes to Earlier
Results".
The help of vargha_delaney_A(), cliff_delta(), and
probability_of_superiority_paired() is rewritten in full. Each
page states the estimand, the sampling unit, and the kind of interval;
the assumptions; the estimator and the variance estimator with their
formulas and sources; how the interval is formed; what each column or
row is; what is refused; every case of a zero variance; and a reading
of an example in words. ?cliff_delta states that the variance is
the sample version of Feng and Cliff's (2004) Equation 2 (p. 323),
whose printed coefficients (n_1 - 1) and (n_2 - 1) are transposed,
with the argument that shows it, and that their Equation 4 is not
used; it no longer cites Cliff (1996) or says what other R
implementations do. ?vargha_delaney_A no longer calls the DeLong,
DeLong, and Clarke-Pearson (1988) estimator "the standard
nonparametric variance (Brunner & Munzel, 2000)"; it says that
Schüürhuis, Konietschke, and Brunner (2025) state the two estimators
to be identical. The two pages show that their intervals are the same
construction on two scales with different variance estimates: for
1:5 against 3:7, A = 0.18 with limits 0.034 and 0.576, which
are -0.931 and 0.151 on the scale of delta, where cliff_delta()
gives -0.64 with limits -0.935 and 0.177.
?probability_of_superiority_paired derives its variance as that of
a mean of independent scores and no longer attributes it to Brunner
and Munzel (2000).
randomization_test() gives the degenerate-interval warning under its
own name. When the Cliff's delta rows have no variation or the groups
are completely separated, the warning of class
dmar_interval_degenerate began "cliff_delta():", a function the user
did not call. It now begins "randomization_test(): for the Cliff's delta
rows (computed by cliff_delta()),", with the same class and elements.
The examples of vargha_delaney_A(), cliff_delta(),
randomization_test(), and randomization_test_paired() show the
accepted call. The call that is refused is described in a comment and
is no longer run under try().
mediate() codes its covariates once and decides in each resample
whether the indirect effect is defined. The two regressions were
refit in every bootstrap resample from a formula, so the columns of a
factor covariate were recoded from the rows drawn. Three things
followed, and each is repaired.
A covariate category with few cases stopped the call with R's
contrasts can be applied only to factors with 2 or more levels
whenever a resample held no case of it; with 2 of 60 cases in a
category and B = 400 the call failed as a factor or a character
covariate and returned 0.1857 [0.0644, 0.3222] as a 0/1 column. All
four codings now return that table. A resample of this kind leaves
the covariate column redundant, the paths a and b keep unique
values, and the resample is kept; the new attribute
"n_covariate_redundant" counts them.
A resample in which a or b is not estimable (the predictor
constant, or the predictor or the mediator a linear combination of the
other predictors in the rows drawn) has no indirect effect. It is
dropped and counted: the table has a new row B_used after B
(the number of replications the interval rests on; B for the Monte
Carlo interval, NA for Sobel), the attributes "B", "B_used", and
"n_dropped" hold the counts, dmar_metadata() now records the
bootstrap of a mediate() result, and the one warning has the class
dmar_bootstrap_replications_dropped and carries B, B_used, and
n_dropped. When too few replications remain the limits are withheld
(see One Rule for Bootstrap Replications); through 1.0.x the call
stopped when fewer than 100 remained. The interval then describes resamples in which
the effect is defined; that is the package's rule for a replication
that returns no value, not a result that gives the interval its
nominal coverage, and the help page says so.
The decision does not depend on the units or the origin of a
variable: a predictor with a large mean relative to its spread
(x + 1e9) stopped with missing value where TRUE/FALSE needed and
now returns the table of the unshifted predictor.
mediate(ci_method = "boot_bca") refuses the interval when it is not
defined. The acceleration is estimated by the jackknife, which needs
the indirect effect with each case left out in turn. When one of those
was not estimable the value was removed without a word and the
acceleration computed from the rest. The call now stops with an error
of class dmar_bca_undefined (a dmar_refusal) that names the cases,
and likewise when the bias correction or the acceleration is not
finite; ci_method = "boot_percentile" does not need these
quantities. This is DMAR's rule for its jackknife acceleration.
mediate() takes column names as they are. The names given as
x, m, y, and covariates were pasted into a formula, so a name
was read as formula syntax. A column named predictor score
stopped the call with a parse error and one named outcome$score
with object 'outcome' not found; a covariate
column named w^2 beside w was read as w a second time and the
square left out of the model with no message (in the
data of the independent check, N = 120, the path a was 0.831 where
the model with the square gives 0.451); a
column named w:x, I(w), or log(w) was evaluated as an expression
and could use an object w that was not in data; and a column named
. put every analysis column, the outcome included, among the
predictors of the mediator. No name reaches a formula now, and every
such column is the column it names.
mediate() says what it cannot use. A factor or character x,
m, or y, and a logical x or m, stopped with subscript out of bounds; the message now says the column must be numeric and that a
predictor with two categories is supplied coded 0 and 1. A logical
y, which was read as 0 and 1, is refused with the same message (see
Changes to Earlier Results). A one-column matrix, such as scale()
leaves in a data frame, is read as its column, as before. A column named in two roles
(x and m the same column, y among the covariates, a covariate
named twice) stops with the name. An infinite value among the analysis
variables stops with a message, as does a predictor that is constant
or a linear combination of the covariates, and a model that leaves the
outcome regression no residual degrees of freedom. A covariate with a
single value in the complete cases, which stopped the call when it was
a factor or a character column, is left out with a message. A Date,
date-time, or time difference predictor is read as a number, as
before; a mediator or an outcome of these classes, which stopped the
call ("'^' not defined for "difftime" objects"), is now read the same
way. B must be finite: B = Inf was accepted with ci_method = "sobel".
mediation_mbco() gives at most one warning per call for its Monte
Carlo intervals. A call in which several effects lost draws gave one
warning for each effect. It now gives one: of class
dmar_bootstrap_insufficient if the interval of any effect is withheld,
otherwise dmar_bootstrap_replications_dropped if any effect lost
draws, otherwise dmar_bootstrap_exploratory. The message has one line
for each affected effect and then the full account of the effect with
the fewest usable draws, and the condition carries the data frame
effects (term, B_used, n_dropped, formed, status).
eta_squared(), omega_squared(), ci_eta_squared(), and
ci_omega_squared() take estimand = c("all", "total", "partial").
The default, "all", reports every estimand; "total" or "partial"
selects one. These four functions replace eta_squared_partial(),
omega_squared_partial(), and ci_eta_squared_partial(), which are
removed (see One Function for Each ANOVA Effect Size for the call that
replaces each and the columns of the tables).
Total eta squared and total omega squared. eta_squared() reports,
beside the partial value SS_effect / (SS_effect + SS_error), the total
value SS_effect / SS_total, with the corrected total sum of squares, and
omega_squared() reports the total omega squared
(SS_effect - df_effect MS_error) / (SS_total + MS_error) beside the
partial value (SS_effect - df_effect MS_error) /
(SS_effect + (N - df_effect) MS_error) (Olejnik & Algina, 2000, Table
10; Maxwell, Delaney, & Kelley, 2027, Eqs. 7.40 to 7.47). In a one-way
design the two are the same number. For a fitted model the total uses
the sums of squares of anova(); for an aovlist fit the total eta
squared divides by the corrected total over every stratum, and the total
omega squared, which is defined for designs whose factors are all
between subjects, is NA.
Sums of squares from a published table. eta_squared() takes
SS_effect with SS_total (the total) and SS_error (the partial);
omega_squared() takes SS_effect, df_effect, and the error mean
square (MS_error, or SS_error with df_error), with SS_total (the
total) and N (the partial). SS_effect, df_effect, and F_value
may hold several effects of one analysis, named for the effects, and
the table has a row for each: the 2 by 4 table of Kim (2016, Table 3) is
eta_squared(SS_effect = c(LIGHT = 34.716, MATERIAL = 1996.998, "LIGHT:MATERIAL" = 1570.389), SS_error = 2261.612, SS_total = 5863.715, df_effect = c(1, 3, 3), df_error = 72). Each interface takes exactly
its own inputs: sums of squares with F_value, MS_error with
SS_error, and a SS_total smaller than SS_effect + SS_error are
refused, and so are summary statistics given with a fitted model (see
Calls That Are Now Refused).
An estimand the inputs do not determine is NA, with the reason.
An F statistic and its degrees of freedom determine the partial value
only, since the total also depends on the design's other effects. With
estimand = "all" such an estimand is NA, the estimand_notes
attribute gives the reason, and the console prints it beneath the
table. Asked for by name, it is refused with an error of class
dmar_estimand_undetermined, which carries estimand and reason.
The interval functions report the total where an interval exists.
The noncentral F interval from an effect's F is an interval for its
partial value, which in a one-way design is the total, so
ci_eta_squared() and ci_omega_squared() of a fitted one-way model
report it for both. For an effect of a design with other effects, and
for an aovlist fit, the total is reported with NA limits, and
estimand = "total" is refused.
ci_omega_squared() takes an aovlist fit, computing the partial
value of each effect and converting its limits with the size of its
stratum, as ci_eta_squared() and omega_squared() do; through 1.0.x it
took only a single-stratum fit.
results_sentence() reads an interval held in columns named for its
estimate, such as eta_squared_partial_ci_lower and
eta_squared_partial_ci_upper in a row of ci_eta_squared(). A row
with one interval gives its sentence; for a row with two (a one-way
model reports the same interval for the total and the partial value),
term names the one to write. plot_ci() draws each estimate of such
a table that has limits, labeling the rows with the effect and the
estimand when there are two.
The estimand changes no value: every partial eta squared, partial omega squared, and interval is reported as the other entries of this release describe, under the column names of One Function for Each ANOVA Effect Size; the total values are new.
assurance_method in ss_aipe_smd(), ss_aipe_sc(), ss_aipe_rc(),
ss_aipe_reg_coef(), ss_aipe_src(), ss_aipe_sc_ancova(), and their
sensitivity functions. With assurance it says how the planner finds
the probability that the realized interval is no wider than width;
without assurance it changes nothing. "analytic", the default,
computes without simulation.
ss_aipe_smd(),
ss_aipe_sc(), ss_aipe_rc(), and ss_aipe_reg_coef() with its
default noncentral = FALSE, the plan is the smallest sample size, not
below the plan without assurance, at which the exact probability is
at least assurance, and the row assurance_attained reports that
probability. These plans differ from those of DMAR 1.0.x (see Changes
to Earlier Results).ss_aipe_sc_ancova(), ss_aipe_reg_coef() with
noncentral = TRUE, and both intervals of ss_aipe_src(), the plan is
the published approximation: that of Lai and Kelley (2012) for the
ANCOVA contrast, that of DMAR 1.0.x and MBESS::ss.aipe.reg.coef() for
the noncentral interval, and the chi square plan of Kelley and Maxwell
(2003, Equation 3) for the central interval for the standardized
coefficient. It starts from this release's plan without assurance
(see Changes to Earlier Results) and draws no random numbers. It does
not compute the probability it attains, which can be less than the
request, so assurance_attained is NA, and the printed footer says
so."simulation", offered only where the probability has no exact form,
returns the smallest sample size, not below the plan without
assurance, at which the probability estimated from G simulated
samples, with seed, is at least assurance. It reports the estimate
as assurance_attained, with the row G and the Monte Carlo standard
error of the estimate as the attribute assurance_mc_se. The same
samples serve every sample size tried, so the estimate changes
smoothly with the sample size, and the plan can be a case or two from
the one the probability itself would give."simulation" is refused, with or without
assurance, with a condition of class dmar_argument_invalid (arg = "assurance_method") whose message names where it is offered:
ss_aipe_sc_ancova(), ss_aipe_reg_coef() with noncentral = TRUE,
ss_aipe_src(), and their sensitivity functions, among them
ss_aipe_reg_coef_sensitivity() and ss_aipe_rc_sensitivity() with
noncentral = TRUE. Any other value is refused the same way, and an
abbreviation is not matched.assurance_method and assurance_probability
("exact", "simulation", or "not computed") and a line of the
printed footer record how assurance_attained was found. The
sensitivity functions pass the argument to the planner they plan
through. In the planners G and seed are used and checked only with
assurance and assurance_method = "simulation".The exact probability in ss_aipe_smd() and ss_aipe_sc(). The
interval is no wider than width when the noncentral t statistic
falls between the two values at which the width equals the target. In
ss_aipe_smd() the probability of that event is an integral of the
normal distribution function over the chi square, which does not depend
on R's noncentral t; in ss_aipe_sc() it is the noncentral t
probability, computed exactly (see Exact Noncentral t and
Studentized Range Distributions). With equal tails it depends on the
effect only through its magnitude. The plan is found by doubling a step
upward from the plan without assurance and then bisecting. At
true_smd = 0, width = .8, and assurance = .95 it is 50 per group
(.994636), where the published plan of DMAR 1.0.x, 49, attains
.949878, and at true_smd = 1, width = .5, and assurance = .99 it
is 149 (.990260), where the published plan, 150, attains .994302.
ss_aipe_sc() at
true_std_contrast = 0 and c_weights = c(1, -1) gives the same 50. In
45 planning sets of ss_aipe_smd() (|true_smd| of 0 to 2, widths of
.25 to 1, assurances of .80 to .99) and 96 of ss_aipe_sc() (contrasts
of -1 to 2, weights for two, three, and five groups, widths of .5 and
1, equal and unequal tails) the plan is the smallest sample size found
by raising the sample size one case at a time from the plan without
assurance, and the probability does not fall at any sample size from
that plan to ten past the returned one. With assurance, ss_aipe_sc()
adds the rows assurance and assurance_attained to the one row DMAR
1.0.x returned.
The simulated step of ss_aipe_sc_ancova(), assurance_method = "simulation" with G (default 10000) and seed, returns the smallest
sample size per group, not below the plan without assurance, at which
the ci_sc_ancova() interval is no wider than width with at least
that probability, for normal errors and a normal covariate distributed
alike in every group. The probability is simulated from the exact
sampling distribution of the five statistics the width depends on, so
the covariate means term of the standard error and, with divisor = "s_anova", the sampling of s_ancova / s_anova enter it, which the
published plan of Lai and Kelley (2012, pp. 358--359) leaves out. This
is the a priori simulation Lai and Kelley (2012, pp. 360--361) gave for
the ANOVA divisor. A contrast and its negative get the same plan.
Sample sizes per group, with the probability that the interval is no
wider than the width:
| standardized contrast, correlation, weights, width | divisor | assurance | 1.0.x and the default of 1.1.0 | "simulation", seed = 1 |
|---|---|---|---|---|
| .5, .5, c(1, -1), .5 | s_ancova | .80 | 129 (.772) | 130 (.872) |
| the same | s_ancova | .90 | 130 (.872) | 131 (.933) |
| the same | s_anova | .80 | 98 (.547) | 104 (.842) |
| the same | s_anova | .90 | 100 (.658) | 106 (.906) |
| .8, .6, c(.5, .5, 0, -1), .6 | s_anova | .90 | 47 (.691) | 51 (.937) |
The simulated plan can differ by one from the smallest sample size at
which the probability itself reaches the request (in the third row 103
already gives .803). With assurance, ss_aipe_sc_ancova() returns the
rows assurance and assurance_attained (NA for the published plan)
and, for the simulated plan, G; DMAR 1.0.x returned one row. In
ss_aipe_sc_ancova_sensitivity() the draws of the simulated step come
from the stream before the replications, so under set.seed() the
replications differ from those of the default even where the two plans
agree.
The plans of the regression coefficient planners with assurance.
For the central interval for the coefficient in the units of the data,
the plan of ss_aipe_reg_coef() with noncentral = FALSE and of
ss_aipe_rc(), the probability is an exact F probability for random
predictors from a multivariate normal population. For the standardized
coefficient, the noncentral interval of ss_aipe_reg_coef(noncentral = TRUE) and ss_aipe_src() and the central one of
ss_aipe_src(noncentral = FALSE), assurance_method = "simulation"
estimates it from G (default 20000) samples of the exact Wishart
sampling distribution, with seed. Given squared multiple
correlations, which leave the correlation of Y with the predictor
open, the simulated step uses the least favorable value. The plans,
and what they attain, are in Changes to Earlier Results. In
ss_aipe_src_sensitivity(), which now plans for the standardized
coefficient at unit standard deviations, the default plan is the 1.0.x
step only when the supposed variances of Y and of the targeted
predictor are equal (for the noncentral interval, when both are 1;
DMAR 1.0.x refused the noncentral plan otherwise): for true_var_Y = 4, true_cov_YX = c(.8, .6), and unit predictor variances correlated
.3, supposed correctly, width = .3, noncentral = FALSE, and
assurance = .9, it gives 181 where DMAR 1.0.x planned 657 (see
Changes to Earlier Results). DMAR 1.0.x stopped with an error on every
plan of the three sensitivity functions with one predictor, so there
is no 1.0.x plan for such a call to return.
G and seed in ss_aipe_reg_coef() and ss_aipe_src(), for
assurance_method = "simulation"; G must be a whole number of at
least 100.
noncentral in ss_aipe_src(). TRUE by default, the noncentral
interval for the standardized coefficient that ss_aipe_src() planned
in DMAR 1.0.x; FALSE plans the central interval for the standardized
coefficient of Kelley and Maxwell (2003, Equation 1), whose standard
error is formed from the two sample squared multiple correlations: their
plan for the expected width without assurance (Equation 2), with
assurance their chi square plan (Equation 3) by default, and with
assurance_method = "simulation" the smallest sample size that attains
it, simulated as the noncentral step is.
New rows in the result of the three regression coefficient
planners: assurance (the request) and assurance_attained (the
probability at the returned sample size, NA for a published
approximation) whenever assurance is given, and G when the step
simulated.
cohen_f(), ci_srsnr(), and covmat_from_cfa() say what they
compute from each kind of input. Each computes the same quantity from
population values or from sample summaries and estimates, its help page
says what the result is in each case, and its arguments carry quantity
names without true_ (see Renamed Arguments and the paragraph on these
calculators under Old Names That R Still Matches). The values are
unchanged: every call that still runs, with the arguments renamed,
returns what it returned, and ss_aipe_reliability(), which builds its
population covariance matrix with covmat_from_cfa(), plans the same
sample sizes.The noncentral t distribution is computed exactly wherever R's
pt(), qt(), and dt() are not accurate. R's pt() sums a series
up to a noncentrality of 37.62 and returns a normal approximation
beyond it (and beyond 400,000 degrees of freedom), which was off by as
much as .14 with one degree of freedom, .008 with 38, and .001 with
1,000; below one degree of freedom it is not accurate even inside
37.62. DMAR now takes the probability, quantile, or density from
pt(), qt(), or dt() where they are accurate (within 1e-9 of the
exact probability up to a noncentrality of 37.62 with at most 2,000
degrees of freedom, and up to 33 with more) and otherwise from an
integral of the normal distribution function against the distribution
of the chi square (below one degree of freedom, against the standard
deviation raised to the power df). ci_nc_t() and every interval and
planner built on it, the critical values of cv_t(), the individual
intervals of limits_of_agreement(), the power of the ss_power_*()
planners whose test is a t test (of ss_power_indirect_effect(),
only the power of the test of a with X fixed; its other powers average
the noncentral t power over the sums of squares of its predictors)
and of design_consequences(), and
the likelihood of the Bayesian t tests use these routines. Inside the
range of pt() the results are those of pt() and qt() (to 1e-13 for
probabilities and powers). No warning about the range is given, since
no limit is approximate. A limit outside the range takes about ten
times as long to compute as one inside it: ss_aipe_sm(true_sm = 3, width = .3, assurance = .9) took 0.5 seconds and takes 2.1. Below one degree of freedom, should the
integral or a search fail, the call stops with a condition of class
dmar_computation_failed that names df. The values that change are
in Changes to Earlier Results.
The studentized range is computed exactly wherever R's qtukey() and
ptukey() are not accurate. There cv_tukey_hsd(),
ci_tukey_kramer(), ci_games_howell(), contrast_test(adjust = "tukey"), qbryant_paulson(), pbryant_paulson(),
cv_bryant_paulson(), and ci_c_ancova_bp() integrate the probability
integral of the range against the chi square distribution of the error
variance, take each critical value as its root, and take the adjusted
p-values from the same distribution function; elsewhere the values
are those of qtukey() and ptukey(), as before. Where the exact
computation is used and the values that change are in Changes to
Earlier Results.
vignette("anova_as_path_model")), fits the single sample and paired
t tests, the one way and two way analyses of variance (balanced and
unbalanced), and the analysis of covariance as path models, and shows
exactly where they reproduce the classical answers: a factor enters as
a set of code variables tested jointly, and the likelihood divides by
N where the classical test divides by the error degrees of freedom.
It uses the depression_bdi and pygmalion data and draws its path
diagrams with base graphics.holzinger_swineford in place of the first five
mtcars columns, which are not a scale. The tests are scored on
different scales, so their raw-score loadings on one factor are far
from equal and the sum of the raw scores carries that inequality; the
vignette prints the test standard deviations, the loadings, and the
fit with the loadings free and constrained to be equal, then
standardizes the tests and shows alpha and omega nearly agreeing on
the sum of the standardized scores, so the reader sees that
tau-equivalence is a property of the composite as scored. A second
example, labeled as hypothetical, simulates six items that each have
variance 1, three loading .9 on the factor and three loading .1; the
vignette computes the population alpha (0.626) and omega (0.718) from
the loadings and then estimates both from a sample of 300, so the
reader sees a gap larger than the spatial tests' on items that already
share one variance, where standardizing leaves the gap in place. The
vignette no longer draws on a data set from outside the package.vignette("DMAR")) fits its one-way
ANOVA, for omega squared and for the Tukey pairwise comparisons, to
the drinks_trial outcome on its log scale in place of PlantGrowth,
pooling the trial's two cohorts and saying that the pooling is a
simplification, and screens six spatial and verbal tests of
holzinger_swineford in place of attitude, the same six its
measurement model takes up. "Reading DMAR Result Tables"
(vignette("dmar_output")) and "Visualizing Effect Sizes and
Distributions With DMAR" (vignette("effect-size-visualization"))
show their two-factor table, and in the second its forest plot, on
pygmalion, expectancy condition crossed with grade, in place of
ToothGrowth and warpbreaks. Both say that the rows come from
sequential (Type I) sums of squares, which with these unequal cells
differ from Type II or III sums of squares for the earlier terms, and
point to the help page that describes how to supply those statistics.
The visualization vignette also shows the bloomers' advantage grade by
grade, concentrated in grades 1 and 2, which the omnibus interaction
spreads over 5 degrees of freedom.vignette("DMAR")) shows anova_within() on pygmalion in
place of nlme::Orthodont, with the pretest and the two follow-up IQ
scores as three occasions of a within-subjects factor, reports the
partial eta squared for occasion with its confidence interval from
ci_eta_squared(estimand = "partial") on the same analysis fitted by
aov() with an Error() term, and says that the analysis pools the
two expectancy conditions. DESCRIPTION, the package citation
(citation("DMAR")), the README, and the vignettes "Introduction to
DMAR", "Moving From MBESS to DMAR" (vignette("mbess_to_dmar")), and
"Visualizing Effect Sizes and Distributions With DMAR" now describe
DMAR in one phrase, as a more general and greatly expanded reimagining
of MBESS. DMAR 1.0.x called it, in different places, a more modern,
more general, and greatly expanded reimagining, the modern, more
general reimplementation and expansion, and a modern and greatly
expanded reimagining. The help pages of
ci_eta_squared(), ci_omega_squared(), eta_squared(), and
omega_squared(), which suggest
car::Anova(object, type = 3) for Type III statistics, now say that
Type III main-effect tests compare unweighted marginal means only when
the factors are coded with sum-to-zero contrasts (contr.sum, set in
the fit or with options(contrasts = c("contr.sum", "contr.poly"))),
as the help page of car::Anova() warns, and that factorial_anova()
applies that coding itself.seed. seed = NULL is now the last argument of the sensitivity functions that
lacked it (ss_aipe_c_sensitivity(), ss_aipe_c_ancova_sensitivity(),
ss_aipe_equivalence_r_sensitivity(), ss_aipe_icc_sensitivity(),
ss_aipe_indirect_effect_sensitivity(),
ss_aipe_mixed_effects_sensitivity(),
ss_aipe_omega_squared_sensitivity(), ss_aipe_pcm_sensitivity(),
ss_aipe_r_sensitivity(), ss_aipe_rc_sensitivity(),
ss_aipe_reg_coef_sensitivity(), ss_aipe_reliability_sensitivity(),
ss_aipe_rmsea_sensitivity(), ss_aipe_sc_sensitivity(),
ss_aipe_sc_ancova_sensitivity(), ss_aipe_sem_path_sensitivity(),
ss_aipe_sm_sensitivity(), ss_aipe_smd_sensitivity(),
ss_aipe_src_sensitivity(), ss_power_R2_sensitivity(),
ss_power_reg_coef_sensitivity(), and, described in their own
entries, ss_aipe_cliff_delta_sensitivity(),
ss_aipe_partial_r_sensitivity(),
ss_aipe_semipartial_r_sensitivity(), ss_aipe_cv_sensitivity(),
ss_aipe_R2_sensitivity(), and
ss_aipe_equivalence_smd_sensitivity()), of procrustes_phi(), of
ss_aipe_equivalence_r() (whose assurance step is a simulation), and
of the data generators
simulate_ancova_data(), simulate_ancova_factorial_data(),
simulate_longitudinal_polynomial(),
simulate_longitudinal_gompertz(),
simulate_longitudinal_logistic(),
simulate_longitudinal_negative_exponential(), and
simulate_longitudinal_richards(). Where a function ends in ...,
seed follows it and is given by name. With the default every one of
them draws from the caller's stream exactly as before: for each of the
30 functions other than the six with entries of their own, a call under
set.seed() returns the result it returned before the function had
the argument, where no other entry of this release changes that
result, and leaves the stream at the same point. A
number makes the call reproducible, gives what set.seed() with that
number followed by the call gives, and leaves the caller's stream as
it was. A seed that is not a single number is an error. For those
six sensitivity functions the same holds wherever their results did
not change for another reason (1,314 values of four of them compared
under a common set.seed(), largest difference 0, every run of
ss_aipe_cliff_delta_sensitivity() without separated replications,
and, for ss_aipe_equivalence_smd_sensitivity(), whose interval
changed, the mean_smd, median_smd, and sd_smd rows and the
stream after the call in 4 of 4 cases).
In ss_aipe_semipartial_r_sensitivity(),
ss_aipe_sc_ancova_sensitivity(), ss_aipe_reg_coef_sensitivity(),
ss_aipe_rc_sensitivity(), and ss_aipe_src_sensitivity() the seed is
set before the plan, so it also fixes the simulated assurance step of
the planner. With assurance and assurance_method = "simulation",
ss_aipe_sc_ancova_sensitivity(), ss_aipe_src_sensitivity(), and
ss_aipe_reg_coef_sensitivity() and ss_aipe_rc_sensitivity() with
noncentral = TRUE take that step from the caller's stream when seed
is NULL, so the stream after the call is not where 1.0.x left it;
the default, "analytic", draws nothing for the plan.
vargha_delaney_A() takes seed for its permutation test, and
ss_aipe_semipartial_r(), ss_aipe_sc_ancova(), ss_aipe_src(), and
ss_aipe_reg_coef() with noncentral = TRUE for their simulated
assurance steps.tools/release_checks.R fails when an export that can draw random
numbers has no seed = NULL argument. tools/precompute_vignettes.R
optimizes the PNG figures without loss after a knit, with oxipng or
optipng when one is installed and otherwise, when python3 can
import Pillow, with tools/optimize_png.py. That script rewrites a
figure only when the new file decodes, with Pillow and with the
script's own decoder, to the values and channels of the original, and
it copies the iCCP, eXIf, and pHYs chunks byte for byte; it leaves
16-bit images, APNGs, and files with other chunks (gAMA, cHRM, sRGB,
sBIT, and others) as they are, and prints a line for each figure. The
step says so when none of the three is available. The 42 vignette
figures and man/figures/logo.png were passed through it: the figures
went from 2,112,862 bytes as knitr wrote them to 1,380,358, and the
logo from 90,909 bytes to 75,027.
tools/doi_audit.py compares every DOI in the help with the metadata
registered for it; tools/release_gate.R prints the number of
references that disagree and does not fail on it.
tests/testthat.R runs every test file only when NOT_CRAN reads as
true the way testthat's skip_on_cran() reads it
(isTRUE(as.logical(Sys.getenv("NOT_CRAN")))), and the curated CRAN
subset otherwise. In DMAR 1.0.0 any nonempty value selected every
file, so NOT_CRAN = "false" (or "F", "1", "yes") ran the whole
suite while skip_on_cran() still skipped its Monte Carlo blocks.
tools/release_gate.R runs R CMD check on the CRAN path whatever
the shell that starts it has set: NOT_CRAN unset, TESTTHAT_CPUS = 2, and one thread for OpenMP, OpenBLAS, MKL, and vecLib, for the
check's process only. In 1.0.0 the check inherited the caller's
values: from a shell with NOT_CRAN = true and TESTTHAT_CPUS = 8,
the check's tests ran the full suite on eight workers instead of the
curated subset on two, and the gate timed that run.
The release gate fails the full local test suite when a test was
skipped, when a test file ran no test, when no expectation passed, or
when the result cannot be read. A skip at the top of a file (eleven
test files open with skip_if_not_installed() for lavaan or boot)
leaves no result for that file, so the gate counts files as well as
tests and its line says how many of the test files ran. It fails the
oracle harness when a snippet was skipped, when none ran, or when the
harness printed no summary line. In 1.0.0 only failures, errors, and
warnings failed the suite, so an empty suite, a skipped test, and a
file skipped from its top level all passed. Only a nonzero exit status
failed the harness, and the harness exits 0 when an oracle package is
missing ("oracle checks: 2 ok, 1 skipped, 0 failed" passed).
When every gate passes, the release gate checks the copy it hands off
against the sha256 on its record and writes
DMAR_<version>.tar.gz.sha256 beside it, in the form shasum -a 256 -c reads (the hash, two spaces, the file name). A sidecar an earlier
run left there is removed first (emptied where the folder allows no
removal), so a failed delivery leaves no sidecar that vouches for an
older tarball. A copy that fails or does not match, or a sidecar that
cannot be written, ends the run with status 1, prints "DELIVERY
FAILED", and rewrites the record's Result line as "ALL GATES PASSED,
DELIVERY FAILED; the hand-off is not certified". In 1.0.0 the result
of the copy was not read: with an unwritable or missing destination
the gate printed "ALL GATES PASSED" and exited 0 with no tarball
delivered, and no sidecar was written.
The release gate's NOTE gate is named for what it allows: "no NOTE beyond incoming feasibility (new submission, spelling in DESCRIPTION, a URL server error that a re-probe clears) and HTML validation skipped by an old local HTML Tidy". It reads each item of the feasibility NOTE against those three and fails on any other, naming it (an invalid DOI, a Title not in title case, a URL check that could not run, a CRAN URL not in canonical form). It allows the HTML manual NOTE only when its whole text is R's message that the local HTML Tidy is too old. It lists every NOTE with its text, so a PASS shows what it tolerated; it judges a NOTE whose result R printed on a line of its own (after a network warning, say) with the step it belongs to; and a NOTE the Status line counts but the log does not show fails it. In 1.0.0 the gate was "no NOTE beyond new-submission and spelling" and passed the old-Tidy NOTE with no detail. It failed a feasibility NOTE only on five phrases (invalid file URIs, Non-FOSS, Insufficient package version, Size of tarball, Days since last update) or on a URL that stayed bad, so an invalid DOI, a Title not in title case, and a URL check that could not run all passed. It also passed an HTML NOTE that said math rendering was skipped as well, and, unlisted, a NOTE whose result was on a line of its own.
Both R CMD check gates of the release gate (no ERROR or WARNING, and
the NOTE gate) fail when the check printed no Status line. Given a
tarball that does not exist, R CMD check warns ("is neither a file
nor directory, skipping"), prints no Status line, and exits 0. In
1.0.0 a check log with no feasibility step, that one among them,
stopped the gate with the R error "'from' must be a finite number"
before any record was written.
The timing limits of the release gate are raised. The whole check may now take 720/403 (about 1.79) times the accepted release's local time, the ratio that projects 720 s on the r-devel Linux flavor of the CRAN farm, and the CRAN-path tests may take 60 s; the limits were 600/403 (about 1.49) and 30 s. The 1.1.0 candidate checked in 316 s locally (ratio 1.71), with the CRAN-path tests at 46 s; in 284 s (ratio 1.54, a projected 619 s) after the first trimming of the CRAN-path tests and examples; and in 250 s (ratio 1.35, a projected 545 s) after the cuts below, which take the CRAN-path tests to 16 files, prebuild the vignettes, and shorten twenty example pages, three of them for the second time. The anchors of the closed-form planners, added after those measurements, bring the subset to 40 files and add under a second to the CRAN-path tests. The gate still prints the 600 s line of the incoming pretest, which returned the package for overall check time on 5 September 2026, and says when the projection is over it.
On CRAN the curated test subset runs in one process and holds 40 of
the 362 test files. The promised detectors (test-rd_hygiene.R and
test-ss_sensitivity_filename.R) run whole. Fourteen files each carry
a published-value or oracle anchor for one family, which pins the
package's numerics across platforms: the pinned MBESS values, metafor,
multcomp, car, lavaan, Equation 5.14 of Maxwell, Delaney, and Kelley,
the example of Kelley (2007), for which the exact noncentral t gives
the plans of 101 and 142 where the article reports 99 and 141 from R's
approximation, the exact noncentral t against its oracle, SIEVER's
recorded numbers, the published studentized maximum modulus, Yuan et
al. (2016), Shrout and Fleiss (1979), Kelley and Rausch (2011), and
Raudenbush and Liu (2001). Twenty-four more files each keep one block
of a closed-form sample size planner (one that runs no Monte Carlo and
fits no model) on CRAN: the G*Power 3.1 manual for
ss_power_R2_change(), Kelley and Maxwell (2003) for ss_aipe_src()
and ss_aipe_reg_coef(), the pinned plans of the published
approximation of Lai and Kelley (2012) in ss_aipe_sc_ancova(), the
exact probability of an ss_aipe_sc() plan computed apart from the
package, the exact noncentral t past the range of pt() for
ss_power_c(), ss_power_smd(), ss_power_indirect_effect(), and
ss_power_composite_ancova_2group(), and, for the others, a closed
form, a scan, or an integral written in the test file. Closed-form
planners that no file of the subset tests are tested in the full local
suite only, among them ss_aipe_c() against the 1.0.0 search over a
grid (4.5 s) and ss_aipe_R2() against MBESS (0.2 s), whose blocks
take more than 150 ms, ss_aipe_reliability() against lavaan,
ss_power_equivalence_c() against the noncentral t, and
ss_aipe_rc(), which calls ss_aipe_reg_coef(). In each anchor file
but test-numerical-correctness.R, whose pinned values all run on
CRAN except the ss_aipe_R2() search, the anchor comes at or near the
top, and a top-level skip_on_cran() after it skips the rest of the
file there with one skip; a skip_on_cran() inside a block costs
about 8 ms under the check's reporter, more than most blocks take to
run. The per-family smoke files left the subset, and with them the
published-value anchors of other functions (those of combine_p(),
omega_squared(), and fleiss_kappa() among them). They, and every
block the CRAN check skips, run in the full local suite and the
release gate, which are unchanged. In the local check the CRAN-path
tests take 5 to 6 s with 527 expectations, where they took 23 s with
4,678, and 47 s (93 s of CPU on two testthat workers) with 13,578
before the first trim. The \donttest{} detector collects the tags of
the manual in 0.25 s rather than 0.7 s, with the same result on every
page. The tests load lme4 only for the full suite. The release gate's
floor on CRAN-path expectations is the subset's count, 527 (it was
2000), so a file that loses its anchor fails the gate.
The vignettes go to CRAN as prebuilt HTML. Every vignette has been a
precomputed twin since 1.0.0, so the check ran no vignette code, but
its step "checking re-building of vignette outputs" still rendered the
21 twins through rmarkdown and pandoc: 17 to 18 s of the local check,
which win-builder has run at about three times the local time.
tools/release_gate.R now runs tools/asis_vignettes.R on its
archive of the commit before R CMD build. In that copy only, the
script renders each twin by the call R CMD build makes, writes
<name>.html.asis beside the HTML for the R.rsp::asis engine, and
removes the twin and its figures. The check then copies the HTML
instead of rendering it: the step takes 7 s locally (5 to 6 s of CPU)
in place of 17 to 18 s (13 to 14 s of CPU). The HTML is byte for byte
what the check rendered before, and the 21 vignettes keep their
titles. Each vignette's executable source, vignettes/<name>.Rmd.orig,
ships in the tarball beside its .html.asis entry: it is the source
that tools/precompute_vignettes.R knits to the twin from which the
HTML is rendered. No vignette engine claims a file whose name ends in
.orig, so R CMD build and R CMD check neither run nor render it,
and vignette(package = "DMAR") lists the same 21 vignettes. The
tarball is 1.5 MB smaller (7.2 MB against 8.7 MB), because the
figures, and the twins' markdown, no longer ship beside the HTML that
holds them. A vignette's source link in browseVignettes() now shows
its R.rsp::asis entry; the twins are in the GitHub repository. The
source tree keeps the twins, and its .Rbuildignore still leaves out
the HTML and the .Rmd.orig sources, so devtools::build() (with
R.rsp installed) and pkgdown work as before, and no rendered output is
committed. R.rsp joins Suggests and VignetteBuilder. The gate fails in
each of these cases:
vignettes/ holds anything other than one
.html.asis entry and one .Rmd.orig source for each twin, or a
shipped .Rmd.orig is not the committed one byte for byte;inst/doc is not the one rendered from its twin, or does
not record the MD5 of its .Rmd.orig as committed;A tarball built by hand goes through the same script, and the script's header gives the commands.
Twenty-seven help pages run their examples faster than in the first
builds of 1.1.0, every demonstration kept and each comment still
giving the count a reported analysis deserves. They were trimmed in
two rounds, ten pages in the first and twenty in the second (three in
both); six of the twenty-seven are new in 1.1.0, and for the other
twenty-one the earlier counts below are those of 1.0.0. The bootstrap
and Monte Carlo counts are lower:
ci_bifactor_indices() and reliability_omega() with B = 10 (were 50
and 40), mlmr() and sem_exact_fit_bootstrap() with B = 10 (were
20), ss_power_composite_sem(), ss_aipe_composite_sem(),
ss_aipe_pcm_sensitivity(), ss_aipe_sem_path_sensitivity(), and
ss_aipe_rmsea_sensitivity() with G = 10 (were 20),
ci_eta_squared_generalized() with B = 100 under
options(dmar.bootstrap_exploratory = TRUE) (was 1000, the fewest it
accepts otherwise), htmt() with B = 500 (was 1000),
randomization_test() with n_resamples = 2000 (was the default
10,000), ss_aipe_reliability() with G_initial = 20 and G = 100
(were 50 and 200), ss_aipe_cv_sensitivity() with G = 100 (was 200),
and ss_aipe_c_ancova_sensitivity() with G = 25 (was 50); the
permutation test of vargha_delaney_A() draws 2,000 reassignments,
and the FIML check of ss_aipe_reliability_sensitivity() runs G = 3.
Where a page fitted the same model twice to show two settings, one
call now shows both: the per-factor descriptor call of
fa_confirmatory() carries the equal loadings and its parallel call
the tau-equivalent constraints, fa_exploratory() shows the target
rotation on the ULS solution, the covariance matrix call of cfa_1()
also names its items, the raw data figure of plot_smd() carries the
group labels and the title, and the 90 percent limits of
limits_of_agreement() use the individual method. Fits that feed
another function skip what it does not use: point_estimates_only = TRUE in the factor_congruence() example, effect_sizes = FALSE in
the df_missing() example, and a fit to the covariance matrix of the
complete tests in the covmat_from_cfa() example.
plot_trajectories() panels six children (was twelve), and now()
waits a twentieth of a second (was a fifth). The examples took 36 s of
page time, where they took 44 s in the first builds. The comments of
the ss_aipe_rmsea_sensitivity() and ss_aipe_pcm_sensitivity()
examples no longer say that the share of intervals narrower than the
target is about one half and that the mean width is at or under the
target, which their output did not show, and the bias figures of
?ci_bifactor_indices say they are from the example run with B = 200.
The interval and the plans for the coefficient of variation are tested
against values computed apart from the package. ci_cv() is checked
against the interval Kelley (2007, p. 758) prints, against limits
computed to 30 digits, and at each one-sided bound. The probability
that the interval at an ss_aipe_cv() plan is no wider than the width
is computed in base R from the noncentral t distribution, by
integration over the normal variate: .5307 and .9966 at the two plans
of the article's worked example (101 and 142), and .7436 at a plan
that the assurance step leaves below a request of .80 (5, where Table
1 prints 6). tools/kelley2007_table3_row.R reruns the five plans of
the Table 3 row that shows a misprint, which take about six minutes of
processor time (about two and a half minutes on five cores) and are
not in the suite.
Every author-year mention in a help page now has its entry on the same
page. A check of the release tooling parses every built page and finds
the mentions in its text and examples ("Name (1970)", "(Name, 1970)",
with chains of authors, et al., possessives, and particles such as
von). The pages its first run flagged are fixed: ten gained the entries
they name (cfa_1() and cfa_2(), for example, carry the seven
estimator and interval references their inherited arguments cite), and
on the others the mention was reworded instead (example comments that
named the Holzinger and Swineford study now name the
holzinger_swineford data, whose page carries the reference).
The five-group example of the noncentral F interval pages
(ci_pvaf(), ci_snr(), ci_srsnr(), ci_eta_squared(),
ci_omega_squared(), eta_squared(), and omega_squared(), and in
1.0.x omega_squared_partial(), now removed) and of the
one_way_anova_power vignette,
which 1.0.0 credited to Bargmann (1970) with his name spelled as
Venables (1975) spells it, with one n, now spells it Bargmann; Venables,
Fleishman (1980), and Steiger (2004) used the example in turn. Each of
the seven pages gains the Bargmann (1970) entry, with
its editors and pages, and the Venables (1975) entry, neither of which
any of them carried in 1.0.0. The Fleishman (1980) entry gains its DOI,
and eta_squared() and omega_squared() gain the entry itself. In the
vignette the Bargmann entry gains its
editors and pages, and Fleishman and Venables entries are added. A test
pins ci_nc_F() to the .90 limits of Venables (19.381, 71.551) and of
Steiger (19.380, 71.549).
The release tooling fails when a citation form corrected for this release appears again. It keeps a list of the wrong forms, each beside the form that replaced it, and first confirms that the pattern for each form finds examples of the wrong text and passes examples of the right. It then reads the R sources (help sources, comments, and messages), the help pages, the vignette sources, the tests, DESCRIPTION, the package citation, the spelling word list, and the README. The list holds the journal and subtitle of Kelley, Maxwell, and Rausch (2003), the title of the Steiger and Fouladi (1997) chapter, and the editions of Rencher and Christensen (2012) and of Anderson (2003) (all four under Changes to Earlier Results), the spelling of Bargmann above, and, on pages new in 1.1.0, the DOI of the Kiers (1994) entry and of the Saunders (1961) entry, the year and pages of Fieller's (1941) insulin study, the equation number given for the incremental fit index of Bollen (1989), and the description of DMAR as a "reimplementation and expansion" of MBESS, replaced in the package citation, the README, and the MBESS vignette as described under Teaching. The list also holds seven forms corrected on 1.0.0 pages: the DOI of the Owen (1968) entry, the first page of the Phillips (1990) entry, and the pages of the Hofmann (1977) entry (all three under Changes to Earlier Results), the citation of Sokal and Rohlf (1995) for the expected value of the sample coefficient of variation, which is Expression 3 of Sokal and Braumann (1980), and the entries for Kelley, Darku, and Chattopadhyay (2018), for Lachowicz, Preacher, and Kelley (2018), and for Tofighi and Kelley (2020) without their issue numbers.
?ss_power_R2 credits the random-predictor approximation as Lee's
(1971, Section 5.1) three-moment noncentral F approximation. The
help called it a two-moment Patnaik (1949) approximation; the code is
Lee's Equation 25, which fits three moments, and Lee (1971) does not
cite Patnaik. The Patnaik (1949) reference is removed from the page.
The pages of ss_power_R2(), ci_R2(), and ci_correlation() said
the distribution of the sample R^2 under random predictors "is given
by Lee (1971)"; Lee (1971, p. 117) credits it to Fisher (1928), and
the pages now do, with Lee credited for the approximation they use.
?vargha_delaney_A no longer calls the DeLong, DeLong, and
Clarke-Pearson (1988) variance estimator unbiased. The estimator is
consistent; it is not unbiased in general. Its expectation exceeds the
variance of the estimate of A by a nonnegative term of order
1 / (n_1 n_2), which is zero when both variables take only two
values; with three continuous observations per group from one
distribution the variance is 7/108 and the estimator averages 8/108 (a
test enumerates the 20 allocations). The page now says that A = 0.5 is
stochastic equality (Vargha & Delaney, 2000, p. 102), which is weaker
than equality of the two distributions, and that the reported test is of
A = 0.5; cites Equation 14 on p. 107 for the rank formula, where it
cited p. 109; says that the samples must be independent and points to
probability_of_superiority_paired(); and says that the limits of
cliff_delta() and of vargha_delaney_A() do not map onto each other
exactly, since the two use different variance estimators. The sentence
that credited Newcombe (2006a, 2006b) with showing better coverage for
the logit interval is removed with its two references, as the claim was
not verified against the papers; the page cites Pohar Perme and Manevski
(2019, doi:10.1177/0962280218814556), who conclude that the DeLong
variance estimator is reliable and that the interval should be formed on
the logit scale. The Hanley and McNeil (1982)
entry, which no sentence cited, is removed, and the DeLong et al.
reference gains its DOI. ?cliff_delta says that its analytic interval
is "valid in large samples" where it said "exact in the large-sample
limit".
The help of cv() describes the unbiased = TRUE value as Kelley
(2007, Equation 4) does, as a nearly unbiased estimate, and no longer
as unbiased or as a first-order bias correction, and its title is now
"Coefficient of Variation (Sample Value or Nearly Unbiased Estimate)".
The page gives the expected value of the sample coefficient from
Expression 3 of Sokal and Braumann (1980, p. 51), which Kelley (2007)
reproduces, in place of the expansion with the single term 1/(4N) that
1.0.0 credited to Sokal and Rohlf (1995), and it bounds the statement
that the sample coefficient is biased downward: that holds when the
population coefficient is below about one half, and above it the
sample coefficient is too large on average. The multiplier
(1 + 1/(4N)) is credited to Haldane (1955), as both papers credit it.
The page now says what the multiplier leaves (a relative bias of about
kappa^2/N - 9/(32 N^2), the second term derived for the page), gives
the bias of both values at a coefficient of 0.1 and of 0.3 with 10
cases, and says that the corrected value is the more biased of the two
once the coefficient exceeds about 0.35 to 0.4. The sentence that
called the interval of ci_cv() "the McKay (1932) noncentral t
based interval" and the comparison with Vangel (1996) that 1.0.0
attributed to Kelley (2007), which does not cite Vangel, are replaced:
ci_cv() inverts the noncentral t distribution (Johnson & Welch,
1940; McKay, 1932; Kelley, 2007), and the chi square approximations of
McKay (1932) and Vangel (1996) are ones it does not use. Where 1.0.0
pointed to "the corresponding asymptotic variances" in var_cv(), the
page names the one variance that function returns, the first-order
variance of the sample coefficient under normality. The sample
coefficient is written k = s / M, as on ?var_cv and ?ss_aipe_cv,
and the entry for Maxwell, Delaney, and Kelley (2027) no longer points
to Chapter 3. The page
gains the Haldane (1955), Johnson and Welch (1940), and Sokal and
Braumann (1980) entries and drops the Sokal and Rohlf (1995) entry,
which nothing on it cites. ci_cv() describes its cv_unbiased
row (c_of_v_unbiased in 1.0.x) as the nearly unbiased estimate, and
the table of the
mbess_to_dmar vignette describes ci_cv() as the noncentral t
interval. No value changes.
DESCRIPTION no longer says that every estimation, inference, and
planning function returns one data frame. Most do; the regression
functions for incomplete data (mlmr(), mlmr_mv()) and a few others
(correlations_test(), cov_fiml()) return a list suited to their
task, as ?DMAR and the README already said.
?ss_aipe_crd_es opens with what the six planners take and return,
and its width argument is described as the width of the interval for
the standardized difference (the text said unstandardized).
?var_cv says where its variance comes from and how accurate it
is. The 1.0.x page said the first row was McKay's (1932)
noncentral t based approximation, that Vangel (1996) showed the
second gave better coverage, and that the first is what most planning
tables use. McKay (1932) and Vangel (1996) give chi square
approximations to the distribution of a function of the sample
coefficient of variation, which ci_cv() does not use, and the
variance the function returns is not taken from them. The page now
gives the value as the first-order (delta method) variance under
normality, kappa^2 (1/2 + kappa^2) / (N - 1), with its derivation.
Expression 5 of Sokal and Braumann (1980, p. 52) without its
third-order term is kappa^2 / (2 (N - 1)) + kappa^4 / N; the function
has N - 1 under both terms, so its value is larger by
kappa^4 / (N (N - 1)). The page states how close the value is to the
variance of the sample coefficient under normality, computed with the
sample mean kept away from zero: for a population coefficient of .30
or less and 10 or more cases it is too large, and the largest excess
is 3.0 percent to one decimal place; at .50 with 10 cases it is 5.7
percent too small. It also states how far the value can be from the
large-sample variance of Kelley, Darku, and Chattopadhyay (2018,
Equation 30) when the population is not normal: in the limit it is 39
percent too large for a gamma population with a coefficient of
variation of .80 and 56 percent too small for a Laplace population
with a coefficient of .30. The page gains the Sokal and Braumann
(1980) and Kelley, Darku, and Chattopadhyay (2018) entries, and its
entry for Maxwell, Delaney, and Kelley (2027) no longer points to
Chapter 3. Its example prints the variance to six decimal places and
sets its square root beside the standard deviation of simulated
coefficients, for normal data and for gamma data. The argument and
the row have new names and the second row is removed (see Renamed
Arguments, Renamed Output Rows and Columns, and Changes to Earlier
Results).
?ss_aipe_crd has a title and a first paragraph that say what the six
planners take and return ("Sample Sizes for a Cluster Randomized
Design: Accuracy in Parameter Estimation for the Difference Between
Condition Means"), in the words ?ss_aipe_crd_es uses for the
standardized difference. No number changes.
?ss_seq_c credits each part of the procedure to its source. The
stopping rule is that of Chow and Robbins (1965) applied to a contrast,
and the allocation of the groups is the cost-optimal allocation of
Chattopadhyay, Bandyopadhyay, Kelley, and Padalunkal (2025, Theorem 1).
The page credited the whole fixed-width procedure to the 2025 article,
which plans for noninferiority and equivalence tests under a budget and
has no fixed-width interval; ?ss_seq_c_sensitivity is corrected the
same way.
?mr_cv gains the entries for Bao (2009) and for Bao's erratum to
it, which its part on Equation 22 of Chattopadhyay and Kelley (2016)
cites, and it cites Chattopadhyay and Kelley (2017) and Kelley (2007),
which the page listed without citing. ?mr_cv and ?mr_smd give the
issue of Kelley, Darku, and Chattopadhyay (2018), 23(2);
?ss_aipe_indirect_effect and ?var_indirect_effect that of
Lachowicz, Preacher, and Kelley (2018), also 23(2); and those two pages
and ?ss_aipe_indirect_effect_sensitivity that of Tofighi and Kelley
(2020), 25(4). Both minimum risk pages state the range of m0 and of
gamma, and ?mr_smd describes A as the structural cost divided
by the square of epsilon (it said that the maximum allowable
difference was the square of epsilon).
?ci_cv states that its interval assumes a normal population. The
interval is exact for a random sample from a normal population with a
positive mean and is not robust to departures from normality, which a
larger sample does not repair: in simulations of the 95 percent
interval (coefficients of variation of .3 and .5, sample sizes of 30
and 200) coverage was .81 to .85 for Laplace data, .90 to .94 for
lognormal data, and .96 to .97 for gamma data. The page says when the
upper limit is infinite and why that interval has the stated coverage,
and describes the tail columns as the requested tail areas. Its
references gain Kelley, Darku, and Chattopadhyay (2018), whose
large-sample interval the page mentions, and lose the Journal of
Statistical Software article, which the page does not cite.
?ss_aipe_cv writes Equation 3 of Kelley (2007) as the approximation
it is (Sokal & Braumann, 1980, p. 51, Expression 3; for a coefficient
of .25 it is above the expected value of the sample coefficient by 3.1
percent at N = 5 and 0.1 percent at N = 20), and it and
?ss_aipe_cv_sensitivity say what the plans deliver for gamma and
lognormal populations. On the three pages the pointers to Maxwell,
Delaney, and Kelley (2027) name what they point to: Chapter 3 on
confidence intervals for effect sizes, and on
?ss_aipe_cv_sensitivity Sections 3.9.1.5 and 4.9 in place of Section
3.5. No number changes.
Sizes given as R integers no longer overflow. R multiplies two
integers as an integer, and the product becomes NA, with the warning
"NAs produced by integer overflow", once it passes 2^31 - 1 =
2,147,483,647: with 46,341 or more in each of two groups, and also with
unequal groups such as 40,000 and 60,000. ci_smd(), ci_smd_c(),
ci_mahalanobis(), convert_d_r(), convert_r_d(),
convert_delta_lambda(), convert_lambda_delta(), cles(),
nnt_from_smd(), proportion_of_superiority(), meta_smd(),
ss_power_smd(), var_smd(), and bayes_independent_t() multiplied
the two group sizes this way when they were given as integers (such as
50000L, or a count from length()), and plot_smd() and
ss_aipe_smd_sensitivity() pass their sizes to ci_smd(). In DMAR
1.0.0, at 50,000 in each group and at 40,000 and 60,000,
ci_smd(smd = .2, n_1 = 50000L, n_2 = 50000L),
ci_smd_c(smd_c = .2, n_C = 50000L, n_E = 50000L), cles(),
nnt_from_smd(), proportion_of_superiority(), and
plot_smd(smd = .2, n_1 = 50000L, n_2 = 50000L) then stopped with
"missing value where TRUE/FALSE needed", ci_mahalanobis() stopped
with a message that its limits "could not be computed for the observed
F-statistic NA", meta_smd() with "Supply a positive sampling variance
for each effect size.", and bayes_independent_t() with "'from' must
be a finite number", while ci_smd() and ci_smd_c() given ncp, the
four conversion functions, and var_smd() returned NA.
ss_aipe_smd_sensitivity(true_smd = .1, specified_n_per_group = 50000L, G = 3) (true_delta and n_per_group in 1.0.x), which uses its one
size for both groups, stopped with "missing value where TRUE/FALSE
needed", as it did with 46341L (with 46340L it gave its table).
ss_power_smd() overflowed even with sizes given as doubles, since it
turned a given size into an integer, and reported the power as NA.
randomization_test() and plot_smd() count the group sizes of raw
data as integers (randomization_test() with sum() of a logical
vector, plot_smd() with length()), so they stopped with "missing
value where TRUE/FALSE needed" for any data whose two group sizes have
a product above 2^31 - 1 (46,341 scores in each group, or 100,000 and
21,475), whatever the type of the scores. The sizes are now multiplied
as doubles, and the results are those for the same sizes given as
doubles: ci_smd_c(smd_c = .2, n_C = 50000L, n_E = 50000L) gives the
limits 0.188 and 0.212, convert_r_d(.2, n_1 = 50000L, n_2 = 50000L)
gives 0.408, ss_power_smd(.02, n_1 = 50000, n_2 = 50000) gives the
power .885, randomization_test() reports the standardized mean
difference with its interval, plot_smd() draws the interval that
ci_smd() gives for the sizes as doubles, and
ss_aipe_smd_sensitivity(true_smd = .1, specified_n_per_group = 50000L, width = .03, G = 3, seed = 1) gives the table of the same call with
50000 (mean interval width 0.0248). The specified-size paths of
ss_power_reg_coef() and ss_power_R2() turned a size above
2^31 - 1 into NA ("NAs introduced by coercion to integer range"): in
DMAR 1.0.0 ss_power_reg_coef(true_f2 = 1e-9, p = 3, specified_N = 3e9) (cohen_f2 in 1.0.x) returned NA for the power, and
ss_power_R2(true_R2 = 1e-9, p = 3, specified_N = 3e9)
(population_R2 in 1.0.x) stopped with "missing value where
TRUE/FALSE needed". Such a size is now used as given: the first call
gives the power .410, and the second .275.
krippendorff_alpha() gives alpha for 46,342 or more pairable
values. It formed N (N - 1), where N is the number of pairable values
counted with length(), as a product of integers, which becomes NA from
N = 46,342 on (46,342 times 46,341 is 2,147,534,622). Take 23,171 units
rated twice on three nominal categories, drawn after set.seed(7) as
r1 <- sample(1:3, 23171, TRUE) and r2 <- ifelse(runif(23171) < .7, r1, sample(1:3, 23171, TRUE)). In DMAR 1.0.0
krippendorff_alpha(cbind(r1, r2)) summed the distances over all
1.07e9 pairs of values and then stopped with "missing value where
TRUE/FALSE needed", after the warning "NAs produced by integer
overflow". The product is now formed as a double, and the same call,
after the same sum over pairs, gives alpha .698 (D_expected .667), the
values from the frequencies of the ratings.
The Paule-Mandel estimate of tau2 and the Q-profile interval for
tau2 no longer depend on the units of the effect sizes.
meta_es(), meta_smd(), meta_r(), and plot_forest() with
method = "pm" found the PM estimate, and every estimator found the
limits for tau2 and I2, by root searches that stopped within 1.5e-8
of the root in the units of tau2, a distance larger than tau2 for
effect sizes in small units. For y = (-1, 0, 1) with sampling variances
.01 the PM estimate is 0.99 and the interval 0.2611 to 39.49; for the
same effect sizes times 1e-6 (variances times 1e-12) the estimate,
divided by 1e-12, was 10.1 and the interval 101 to 101. The searches
are now made on tau2 divided by the scale of the data, as the REML
search already was, and stop at a distance relative to the root, so
the results for effect sizes multiplied by c are those for the original
ones times c squared (tau2 and its limits) or c (the pooled estimate,
its standard error, and the prediction limits). For the 12 simulated
studies of the package's tests the PM tau2 was 0.0431063198 and is
0.0431063219; metafor at its default tolerance gives 0.04310566, so a
comparison with its default output can differ in the sixth significant
digit.
ss_aipe_smd() no longer passes on the warnings of R's pt() in its
assurance step. For a negative true_smd it asked pt() for the
lower tail of a probability near 1 (15 warnings full precision may not have been achieved in 'pnt{final}' at true_smd = -1, width = .5,
and assurance = .90). Each tail is now computed directly as a small
probability, and those plans give no warning.
equivalence_smd() and ss_aipe_equivalence_smd_sensitivity() give
their intervals when the product of the two group sizes passes
2^31 - 1. The product of the two group sizes, stored as integers,
overflowed R's integer range ("NAs produced by integer overflow").
equivalence_smd() with integer n_1 and n_2, or with raw data,
whose two sizes had a product above 2,147,483,647 (46,341 or more in
each group, or 40,000 and 60,000) returned NA for the t
statistics, the p-values, the limits, and equivalent; it now
returns them (equivalence_smd(smd = .1, n_1 = 46341L, n_2 = 46341L, smd_lower = .2, smd_upper = .2) gives the limits 0.089 and 0.111). The
sensitivity function, whose sizes are integers, stopped there with
"missing value where TRUE/FALSE needed" (at 54,112 per group, the plan
for width = .02 at a 90 percent level) and now returns its table.
ss_aipe_equivalence_smd() and
ss_aipe_equivalence_smd_sensitivity() check their numeric arguments on
entry. A missing, not-a-number, or infinite true_smd or
supposed_smd, a missing or vector-valued equivalence bound, a
specified_n_per_group that is missing, fractional, or below 2, and a
missing width are refused with a condition of class
dmar_argument_invalid that names the argument. DMAR 1.0.0 stopped on
most of these with R's "missing value where TRUE/FALSE needed", refused
an infinite planning value as "Required n exceeded 1e6 per group; check
inputs", and truncated a per-group size of 10.7 to 10 without a word.
reliability_H() with fit checks the covariance matrix of the
estimates as the factor analysis tables do. Where that matrix comes
from an information matrix that is singular in the directions the fit's
equality constraints leave free, the standard error and limits are NA,
with the warning of class dmar_result_qualified that
fa_confirmatory() gives for the same fit; a two-item factor
uncorrelated in the sample with the other factor had H = .853 with a
standard error of 533.8 and the interval [0, 1], with no warning. Where
lavaan computed no standard errors, reliability_H() stopped with
"argument is of length zero" and now returns the estimate with the same
warning (the vcov_basis attribute is then "none").
The check of the information matrix in fa_confirmatory(),
fa_bifactor(), fa_second_order(), and reliability_H() reads a fit
with ceq.simple = TRUE correctly. Parameters that share a label then
share one free number and carry no equality constraint, and the check
took the information of the unconstrained parameters for that of the
model, so models identified by their labels (a two-item factor with
equal loadings, two groups with loadings equal by label, a
group.equal fit) were reported singular.
fa_confirmatory() and fa_bifactor() with output = "standardized"
and se = "none" return the standardized estimates, with the se,
z, pvalue, ci.lower, and ci.upper columns NA; they stopped
with "missing value where TRUE/FALSE needed". fa_second_order(),
which returned the estimates without those columns, has them too.
bifactor_indices() no longer asks lavaan for the standard errors of
the standardized solution, which it does not use, so lavaan's warning
"Could not compute standard errors!" no longer reaches the caller beside
the Heywood warning of an improper fit. The indices are unchanged.
htmt() refuses blocks that are not disjoint sets of distinct items
under distinct names. An item listed twice in a block, an item placed
in two blocks, or two blocks under one name gave a ratio without a
word: for nine simulated items whose blocks A = x1, x2, x3 and
B = x4, x5, x6 give .615, listing x1 twice in A gave .544,
placing x3 in both blocks .944, and naming both blocks A 1.533.
These, and an item whose name belongs to two or more columns of data,
are refused before anything is computed, with an error of class
dmar_htmt_blocks_invalid whose reason element names the check
("repeated", "shared", "construct_names", or "data_names") and
whose blocks and items elements name what is at issue. Disjoint
blocks give the same ratios and bounds as before.
The standardized cluster randomized planners (ss_aipe_crd_es_*())
count and report the simulated data sets that give no usable interval,
and pass on the warnings of their fits. The counts, the reasons, and
the first error message are in the new attribute simulation (also read
with dmar_metadata()) and on the printed line "Replications"; a warning
of class dmar_replications_without_interval gives the counts when data
sets failed, and a design at which no data set gave an interval, or at
which an assurance cannot be reached, is refused with an error of class
dmar_computation_failed, where the width had been NaN or the search
had stopped with "missing value where TRUE/FALSE needed". The planners
wrapped their search in suppressWarnings(); each kind of warning a fit
gives is now given once, as a warning of class dmar_result_qualified
that counts the data sets that gave it. A fit of optimizer status 0 or
1 is used, and any other status counts as a failed data set.
The models are built with mxExpectationRAM(), mxFitFunctionML(), and
mxFitFunctionAlgebra() in place of the deprecated mxRAMObjective()
and mxAlgebraObjective(), with identical limits, and the socket
cluster used on Windows is stopped however the simulation ends.
The searches of the standardized cluster randomized planners end.
The search upward for the cluster size at a fixed number of clusters
had no end when the width leveled off above the target. It now
simulates cluster sizes up to 100 above the size the unstandardized
difference needs (above its own start when that size is beyond
100,000) and never above 500 (a search that would start above 500 is
refused with reason "cluster_size_beyond_reach", whose field
lower_bound says whether the start is a lower bound on the size the
target needs), and a width still above the target is refused with
reason "width_not_reached", the narrowest width found, and its
Monte Carlo interval. Three designs in a row with no finite
width end either search (reason "no_finite_width"), and a call
simulates at most 500 designs (reason "search_budget"); each is an
error of class dmar_computation_failed. The two planners over both
sizes pick the first of tied widths or costs, where a tie stopped them
with an error from if().
The cluster randomized searches meet a budget or a width equal to
the end of a bracket. A cost or a width equal to the target at the
end of a bracket of the coarse search stopped
ss_aipe_crd_n_clusters_fixed_budget(budget = 20000, cluster_size = 20, cluster_cost = 0, individual_cost = 1),
ss_aipe_crd_n_individuals_fixed_budget(budget = 10000, n_clusters = 10, cluster_cost = 0, individual_cost = 1), and the two width
planners at the width of 1000 clusters of 20 (0.1216020) and of 100
clusters of 1000 (0.3557016) with "'from' must be a finite number" in
DMAR 1.0.0 and the development versions of 1.1.0. Each now returns
1000, at a cost equal to the budget or a width equal to the target;
their ss_aipe_crd_es_*() counterparts share the budget searches. In
a grid of 480 budget searches the 51 (number of clusters) and 29
(cluster size) that stopped now return a size, and no other result
changed; in 1,152 cluster size searches from a width, at targets that
are not the width of a design, no result changed. A width equal to the
target inside a bracket also meets it now (see "Changes to Earlier
Results").
ss_aipe_crd_both_fixed_width() returns a numeric value column.
The cluster size search gave the string "> 100000" for a number of
clusters that cannot reach the width, which made the whole column
character whenever one of the numbers of clusters tried could not:
ss_aipe_crd_both_fixed_width(width = .5, cluster_cost = 10, individual_cost = 1, pr_treat = .5, true_var_between = .2, true_var_within = .8) returned the strings "80", "7",
"0.499131775130474", and "1360" in DMAR 1.0.0 and the development
versions of 1.1.0 (the development versions also "0.5", the width
asked for). The search gives Inf internally, and the plan is the
same numbers (80 clusters of 7, budget 1360), now numeric in the table
and in tidy() and glance(): in 720 calls the number of clusters,
the cluster size, and the budget are identical, and the width differs
only where the development versions returned strings, by at most
2.5e-15 relative. Given back as width, the width a plan reports now
gives that plan: ss_aipe_crd_both_fixed_width(width = 1, cluster_cost = 30, individual_cost = 1, pr_treat = .5, ...) plans 20 clusters of
10 for a budget of 800, and the string "0.994337792651072" that DMAR
1.0.0 and the development versions reported for its width is below
that width and, given back, gave 19 clusters of 13 for 817. Every
cluster randomized planner returns a numeric value column.
ss_aipe_crd_n_individuals_fixed_width() refuses a width no cluster
size up to 100,000 reaches with reason "width_not_reachable"
(class dmar_argument_invalid, as in the development versions), the
width the interval falls toward as the clusters grow as width_limit,
and whether a larger cluster size would reach it; the message said
only that the target was "impossible to achieve".
ss_aipe_crd_es_n_individuals_fixed_width() gives its refusal of a
search whose unstandardized start is beyond 100,000 the reason
"cluster_size_beyond_reach", with start = Inf.
The searches over both sizes no longer stop with an error from R.
ss_aipe_crd_both_fixed_budget(budget = 20000, cluster_cost = 1000, individual_cost = 1, pr_treat = .1, true_var_between = .2, true_var_within = .8) stopped with "invalid 'type' (list) of
argument" in DMAR 1.0.0 and the development versions of 1.1.0, because
no number of clusters it tried was within the budget, and pr_treat = .005 stopped with "wrong sign in 'by' argument", because the floor of
two clusters in each condition (400) was above the end of the search
(200). Both are refused with reason "budget_infeasible" (20
clusters of 2 cost 20,040; 400 cost 400,800). In 720 calls of the
budget planner the 92 that stopped this way are refused, and the 628
plans are identical. ss_aipe_crd_es_both_fixed_budget(budget = 1000, cluster_cost = 5, individual_cost = 1, true_es = .5, true_icc = .9, pr_treat = .5, G = 2, seed = 113) simulated one more cluster than the
budget buys, with a cluster size of NA, and stopped with "invalid
'times' argument" in both; that design is no longer simulated, and the
call returns 142 clusters of 2.
The documented variance pairs (between, ICC) and (within, ICC) plan
in the six unstandardized cluster randomized planners.
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_icc = .2) and the same
call with true_var_within = .8, true_icc = .2 stopped with "missing
value where TRUE/FALSE needed" in DMAR 1.0.0 and the development
versions of 1.1.0; both return 18 clusters, as the other pairs do. The
variances are resolved once per call, where they had been resolved at
every number of clusters tried; every call with consistent inputs
gives the same plan (36 calls over the six planners and six pairs,
identical but for the string rounding of
ss_aipe_crd_both_fixed_width() above).
mediation_mbco() refuses a joint hypothesis one of whose expressions
adds no restriction to the others, before any model is fit, with a
condition of class dmar_argument_invalid (arg = "hypotheses") whose
redundant and kept elements name the expressions. The degrees of
freedom of a joint test were the number of expressions, so a repeated
expression, a multiple or combination of others, or one that is 0
wherever the others are counted as a restriction: in the single
mediator model fit to 100 simulated cases (seed 731),
hypotheses = list(h = c("a", "a")) gave the statistic of "a", 4.980,
with 2 degrees of freedom (p = .0829, where "a" gives p = .0256),
and c("a*b", "b"), whose null set is that of "b", gave p = .0316,
where "b" gives .00858. The rank of the Jacobian of the constraints at
the unrestricted estimates, and points of the null set of the other
expressions found branch by branch, find such an expression.
c("a*b", "a*cp"), whose null set is a = 0 or b = cp = 0, is still
tested with 2 degrees of freedom. See Constraints that add no
restriction in ?mediation_mbco.
mediation_mbco() fits badly scaled variables in a rescaled
metric. When the standard deviations of the observed variables differ
by a factor of 1000 or more (the rule of lavaan's rescale_data
option), each variable is divided by a power of 2 near its standard
deviation for the OpenMx fits, and every result is returned in the
original metric; the divisors are in the new "rescale" attribute.
The optimizers had stopped far from the maximum: with the seed-731
data, x multiplied by 1000, and m divided by 1000, the statistic
for "a" was 1316.2 and is 4.980, the least squares value, and that of
the indirect effect 7.56 and now 4.980.
mediation_mbco() fits the null model of a conditional indirect
effect at a probe value branch by branch ((a0 + a1*w)*b is 0 where
b = 0 or where a0 + a1*w = 0): in a first-stage moderated mediation
of 300 simulated cases (seed 3812) the statistic at a moderator value of
1 was 37.294 and is 33.631, the smaller of the two branch statistics.
When every branch fit is valid, the smallest is the constrained minimum
and the constrained searches, which could end just off the null set,
are not run.
mediation_mbco() reports the test of a total effect (and of other
sums and differences of effects) where its null fit had failed
(reported as NA with a warning in 10 of 100 simulated data sets of
150 cases with every coefficient 0). A constraint without branches is
now also started from the full-model estimates moved onto its null set.
?mediation_mbco states the conditions under which the likelihood
ratio statistic has its chi square reference, and that the null
hypothesis of no indirect effect is not regular where both coefficients
are zero. There, in the single mediator model with observed variables,
the test at the .05 level rejects with probability .05 squared, .0025,
in large samples (between .0025 and .05 when the estimates are
correlated), so it is conservative. The page gives this limit beside
the rejection rates of .025 to .075 that Tofighi and Kelley (2020)
report from simulations in which one coefficient of the indirect effect
is zero, and says what each describes.
ss_power_reg_coef() and ss_power_rc() check p on every route:
a single whole number of at least 1. p = 2.5 returned necessary_N =
47.5 (true_f2 = .2), p = 0 and p = -1 returned 47, and the route
through true_reg_coef_j stopped with "argument is of length zero" when
p was left out. Each is now refused with a condition of class
dmar_argument_invalid (arg = "p").
ss_power_reg_coef() and ss_power_rc() plan for a single predictor
given through true_cor_XX and true_cor_YX.
ss_power_reg_coef(true_cor_XX = matrix(1, 1, 1), true_cor_YX = .3, which_predictor = 1) stopped with 'a' is 0-diml and returns
necessary_N = 93, the result of true_f2 = .09 / .91 with p = 1.
which_predictor must be a whole number from 1 to p, true_cor_XX a
correlation matrix, and true_cor_YX possible with it (a squared
multiple correlation below 1); each is checked by name.
ss_power_reg_coef() and ss_power_rc() refuse at once a sample size
for an effect of 0 when desired_power is above alpha_level, with a
condition of class dmar_effect_zero, since the power is then
alpha_level at every sample size; the search ran to N = 10,000,000
(about 40 seconds) and stopped with class dmar_computation_failed. An
effect too small for any sample size up to 10,000,000 to reach
desired_power is refused at once with that class. Both inherit
dmar_refusal.
ss_aipe_reg_coef() plans for one predictor on the covariance
structure path. true_cor_XX = matrix(1) with p = 1 stopped with
"the condition has length > 1" (and, with true_reg_coef_j given,
object 'b_j_tmp' not found); it now returns the plan of the direct
path (327 for a correlation of .4 and width = .2), and p and
which_predictor follow from a 1 x 1 matrix.
ss_aipe_reg_coef() and ss_aipe_rc() check a supplied
true_reg_coef_j on the covariance structure path against the
coefficient on the scale true_sd_Y and true_sd_X describe. With
true_sd_Y = 2 and true_sd_X = 1 the correct coefficient .681 was
refused and the standardized .341 accepted. The plan is unchanged; the
refusal now gives both values.
ss_aipe_rc(), ss_aipe_src(), and ci_src() state the conditions
of the function they call in their own argument names (for example
true_std_reg_coef_j, not true_reg_coef_j, in ss_aipe_src()).
ss_aipe_src() refuses true_sd_Y or true_sd_X other than 1 itself,
saying that it plans for the standardized coefficient and
ss_aipe_rc() for the unstandardized one; the refusal had spoken of
'noncentral=TRUE', which the user had not given.
ci_src() without a coefficient is refused by name, where it
stopped with "argument is of length zero"; it takes the coefficient
from t with its standard error, from t with R2_Y_X and
R2_j_X_without_j, or from the three squared multiple correlations. The
warning that a standardized coefficient is unusually large applies
beyond 1.1 in either direction, not only to positive values. ci_rc()
and ci_reg_coef() take reg_coef_j = NULL by default, for the routes
on which the other inputs determine the coefficient, and ci_rc()
reports that coefficient between its limits.
ss_aipe_reg_coef_sensitivity() refuses a missing true_var_Y
(with noncentral = FALSE it built a covariance matrix of the wrong
size), a which_predictor outside 1 to p, and a specified_N that is
not a whole number of at least p + 2.
ss_aipe_src_sensitivity() differs from DMAR 1.0.0 when the
variances of Y and of the targeted predictor differ. For
true_var_Y = 4, true_cov_YX = c(.8, .6), unit predictor variances
correlated .3, supposed correctly, width = .3, and noncentral = FALSE, DMAR 1.0.0 planned 610 (657 with assurance = .9) and judged the intervals against .681; this release plans 156
(with assurance = .9, 181 by default and 171 with assurance_method = "simulation" and seed = 1) and judges them against .341. A call
with one predictor and specified_N, which DMAR 1.0.0 answered, changes
as well: with true_var_Y = 4, true_cov_YX = .8, true_cov_XX = matrix(1), and specified_N = 100 it judged the intervals against .8,
where this release judges them against .4. With equal variances of Y
and of the targeted predictor (unit ones, or every variance 4),
whatever the variances of the other predictors, and two or more
predictors, the
calls without assurance and those with assurance and the default
assurance_method return the 1.0.0 table, to rounding error, with the
same draws at the same sample size, plan a case below 1.0.0 (156 and
181 here, 157 and 182 in 1.0.0; see Changes to Earlier Results).
ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() plan with the supposed standard deviations,
as 1.0.x did (609 here without assurance and 672, the exact central
plan, with assurance = .9, where 1.0.x planned 610 and 657), and need
no such change; for their plans with one predictor see the next entry.
The regression sensitivity functions plan the calls DMAR 1.0.x
refused. In ss_aipe_src_sensitivity() a noncentral plan (the
default) from a supposed variance of Y or of the targeted predictor
other than 1 was refused by DMAR 1.0.0 ("Since you've specified
'noncentral=TRUE', all variances should be one"); for true_var_Y = 4
and assurance = .9 this release plans 179 (176 with
assurance_method = "simulation" and seed = 1). (DMAR 1.0.0 answered
a call in which only a predictor that is not targeted has a variance
other than 1.)
DMAR 1.0.0 also stopped on every call of
ss_aipe_reg_coef_sensitivity(), ss_aipe_rc_sensitivity(), and
ss_aipe_src_sensitivity() that planned the sample size with one
predictor, for either interval and with or without assurance
("argument is of length zero", or, for a noncentral plan from a
supposed variance of Y other than 1, the refusal above). This release
plans these calls: for true_var_Y = 1, true_cov_YX = .4,
true_cov_XX = matrix(1), width = .3, noncentral = FALSE,
assurance = .9, and seed = 1, ss_aipe_reg_coef_sensitivity() and
ss_aipe_rc_sensitivity() plan 178 and ss_aipe_src_sensitivity()
171 (159 with assurance_method = "simulation"), and the noncentral
plans are 172 (165 with "simulation"). With true_var_Y = 4 and
true_cov_YX = .8, ss_aipe_src_sensitivity() plans the same sample
sizes, and the central plans of the other two functions are 639.
The upper quantiles of the intervals and plans of a regression
coefficient, a contrast, and a correlation are taken in the upper
tail. ci_reg_coef() with noncentral = FALSE, ci_c(), ci_r(),
the central plans of ss_aipe_reg_coef(), ss_aipe_rc(), and
ss_aipe_src() and the exact central assurance step of the first two,
the simulated central step of ss_aipe_src(), ss_aipe_sc(), and
ss_aipe_sc_ancova() formed an upper quantile as qt(1 - alpha, df)
or qnorm(1 - alpha). 1 - alpha rounds a small tail area away, and it
is exactly 1 for a tail of 2^-54 or less, where the quantile is Inf.
That is how ci_c() and the central ci_reg_coef() returned an upper
limit of Inf, and ci_r() a limit of 1 or -1, at a tail of 1e-20 in
DMAR 1.0.0 and the development versions of 1.1.0. Each quantile is now
qt(alpha, df, lower.tail = FALSE) (or the qnorm() counterpart),
exact at any tail. A tail that small is now refused (see Calls That
Are Now Refused); above the floor the change is in the last digits at
the usual levels and in about the eighth or ninth significant digit
near the floor (see Changes to Earlier Results).
The help of the regression coefficient planners describes what they
plan for. In ?ss_aipe_reg_coef, ?ss_aipe_src, and ?ss_aipe_rc,
the plan without assurance was called the plan for the expected
width. It is the plan for the width at the expected values of the two
sample squared multiple correlations, which is not the mean width over
samples, as Details already defined it. On the squared multiple
correlations path, the noncentral and standardized assurance step was
said to take the smallest probability over every value of the
correlation of Y with the targeted predictor that the planning
values allow, "so the sample size suffices whatever it is". It takes
the smallest over 21 equally spaced values across that range, an
approximate minimum: between two of them the probability can be a
little lower, and the plan a little short of the request. With p = 2
the two ends are the only possible values. The help also said the
simulated estimate rises with N. The search assumes that it does not
decrease, which held in every set checked, although single simulated
intervals were wider at N + 1 in 3,208 of 95 million comparisons.
No result changes.
ss_aipe_R2() plans for an abbreviated which_width. The function
checked which_width with char.expand() and discarded the expanded
value, so an abbreviation matched none of the three plans and the
function returned NULL. In DMAR 1.0.0, ss_aipe_R2(true_R2 = .5, width = .15, which_width = "L", p = 5, random_predictors = FALSE)
(population_R2 in 1.0.x) returned NULL, as did "F", "Fu",
"Lo", "U", and "Up". It now gives 117, the plan of "Lower";
"F" gives 264, the plan of "Full", and "Up" gives 13, the plan of
"Upper". An abbreviation is expanded as match.arg() expands it in
the other planners whose which_width lists the three values.
cohen_f() refuses a specification it cannot read as one. It
computes f from means with var_within, from eta_squared, or from
sd_means with sd_within, and an argument of one of these given with
an argument of another was ignored: in DMAR 1.0.x,
cohen_f(eta_squared = .10, mu = c(1, 2)) returned .333, the f of eta
squared. Such a call, one half of a pair given alone, n_per_group
without means, a vector where one number is needed, and a missing
value among means are refused with a condition of class
dmar_argument_invalid that names the arguments.
ci_srsnr() with means refuses df_1, df_2, or N that
disagree with the group sizes, and group sizes that are not whole
numbers. They were replaced without a word: ci_srsnr(means = c(94, 91, 92, 83), var_within = 67.375, n_per_group = 6, N = 100) ran with
N = 24, and n_per_group = c(7.5, 8, 9) ran with 21.5 denominator
degrees of freedom. var_within or n_per_group without means, which
was ignored, is refused, and the refusal of equal group means (an F
of 0) names means, not F_value.
omega_squared() returns a DMAR table of class dmar_anova_es, as
eta_squared() does, so it prints with the package's display and has
tidy() and glance() methods; through 1.0.x it returned a plain
data.frame.
omega_squared() and ci_omega_squared() accept F_value = 0 and
report 0; through 1.0.x they refused it.
plot_ci() draws the estimates of ci_eta_squared_generalized(),
where it plotted the midpoint of each interval.
ci_proportion() keeps every Wilson limit in [0, 1], with a lower
limit of exactly 0 at 0 successes and an upper limit of exactly 1 at N
successes. In floating point those limits came back a few units in
the last place either side of 0 or 1 (as low as -5.6e-17, as high as
1 + 2.2e-16). Every other limit is unchanged. The Wald interval is not
clamped.
ci_games_howell() gives limits and a p-value for a pair whose
Welch degrees of freedom fall below 2, where R's qtukey() and
ptukey() return NaN. With groups a = (1.2, 2.9), b = (3.1, 5.5), and
c = (4.0, 7.9, 6.1), the b - a pair (1.80 df) had NA limits and
p-value and has [-7.516, 12.016] and p = .4499. With two groups the
interval and p-value equal those of t.test(var.equal = FALSE) below
2 df as well. cv_tukey_hsd(), ci_tukey_kramer(),
contrast_test(adjust = "tukey"), and qbryant_paulson(n_covariates = 0) likewise return the critical value below 2 error df
(cv_tukey_hsd(.05, df = 1.5, n_groups = 4) is 10.189), and a critical
value R's qtukey() fails to find (qtukey(.25, 51, 32) stops with
"convergence failed") is computed as the root of the exact distribution
function.
contrast_test() no longer returns NA standard errors when a group
has a single observation, whose variance is NA in R though its
weight in the pooled error variance is zero. For 20 groups of which two
have a second observation (2 error df), the standard error of the
"2 - 1" contrast was NA and is 0.2008, from the residual mean square
of the one-way fit.
format_p() refuses a digits_p that is not a whole number from 1
to 324, with a condition of class dmar_argument_invalid;
format_p(.123456, digits_p = 3.9) returned "0.123". Its help page says
that it formats and does not check its input. A refusal that names the
value given shows it to 15 significant digits when 7 would show a
different number (a digits_p of 1 + 1e-12 was named as "1").
ci_sc() judges contrast weights by their values. It compared the
sum of the positive weights with 1 by identical(), which also compares
the storage type and every bit: c_weights = c(1L, -1L) or -1:1
stopped with "Please use fractions to specify the contrast weights",
and c(.7, .2, .1, -1) was refused while c(.1, .2, .7, -1) was
accepted. ci_sc(), ss_aipe_sc(), and ss_aipe_sc_sensitivity() now
apply one rule to the values of the weights (the positive values sum to
1 within 1e-8), with one set of messages, and weights given as text are
refused with class dmar_argument_invalid (they stopped with "invalid
'type' (character) of argument").
ss_aipe_sc() with assurance plans a negative contrast with unequal
tails as its magnitude with alpha_lower and alpha_upper
exchanged, since the ci_sc() interval at -t is the reflection of
the interval at t with the tails exchanged. It kept the tails as
given, so the interval could be too wide more often than the request
allowed (see Changes to Earlier Results).
ss_aipe_sc_ancova() plans wide targets and correlations near 1.
Its plan without assurance let the sample size fall below 2 per group,
where the error degrees of freedom are 0 or below:
ss_aipe_sc_ancova(true_std_contrast = .5, true_cor = .5, divisor = "s_anova", c_weights = c(1, -1), width = 3.6) did not return, and
width = 5 with divisor = "s_ancova", or true_cor = .999 with
divisor = "s_anova" and width = .5, stopped with "missing value where
TRUE/FALSE needed". The plan is now at least 2 per group, and these
give 2, 2, and 6, the smallest sample sizes that meet the widths (see
Changes to Earlier Results).
ss_aipe_sc() and ss_aipe_sc_ancova() find the plan without
assurance by doubling a step and bisecting, not by raising the
sample size by one, so a narrow target no longer takes seconds or
minutes, and a plan the 1.0.0 search did not finish, such as width = 1e-5 (about 3e11 per group), is returned. The plans of ss_aipe_sc()
are unchanged: ss_aipe_sc(true_std_contrast = .5, c_weights = c(1, -1), width = .01), 316,921 per group, took 3 seconds and takes under a
second. ss_aipe_sc_ancova() starts the search at 2 per group, and
some of its plans are smaller (see Changes to Earlier Results).
ss_power_indirect_effect() returns a sample size that attains
desired_power in data drawn from the model it plans for. It took
the power of each path test with the noncentrality fixed at the
population standard error, as if the sum of squares of X were N and the
residual sum of squares of the mediator given X were its expectation,
and took the joint significance power as the product of the two path
powers. In a study both sums of squares vary from sample to sample (that
of the mediator in every design, since the mediator is measured, and
that of X when X is measured), and the two tests are not independent,
since the residual sum of squares of the mediator is the error sum of
squares of the test of a and the predictor sum of squares of the test
of b. So the reported power was too high: its plan of 65 for paths of
.39 and power .80 has a power of .7675. The power of each path test is
now its noncentral t power averaged over the chi square distributions
of those sums of squares, and the joint significance power is the
probability that both tests reject, computed given the residual sum of
squares of the mediator, given which the two tests are independent; the
plan is 69, with power .8010. The new argument random_predictors,
TRUE by default as in ss_power_R2_change() and ci_R(), makes X
random (normal), as when it is measured; FALSE fixes X by design,
with variance 1 computed with divisor N (two groups of equal size, for
example). The mediator is random under either setting, and the setting
is recorded in the "random_predictors" attribute. The help said that
the tests validated the result against raw-data simulation; the test
then allowed a relative error of 6 percent, and it now requires
agreement within four Monte Carlo standard errors, at the plan and at
one case fewer, for either test and either predictor model.
The Sobel power of ss_power_indirect_effect() is the probability
that the Sobel test rejects. DMAR 1.0.x referred |ab| / se_ab, with
population standard errors, to the normal distribution. The first-order
Sobel statistic that mediate() uses is t_a t_b / sqrt(t_a^2 + t_b^2),
in the t statistics of the two paths; it is not normal with unit
variance, and the approximation erred in either direction: at N = 96
with paths of .39 it gave .8026 where the power is .8818, so its plan
for .80 was 96 where 84 suffice, and at N = 12 with paths of .6 and .5
and a direct effect of .2 it gave .3165 where the power is .1226. The
power is now the probability that 1 / t_a^2 + 1 / t_b^2 is below 1 /
z^2 under the joint sampling distribution of the two statistics.
The probability is an integral over |t_a| that integrate() computes
in pieces, and development versions of 1.1.0, which no release
carried, computed it with three faults that are now fixed. First, they
stopped with an error of class dmar_computation_failed whenever
integrate() returned a piece with a roundoff status, as it does for
pieces next to the critical value z that are worth almost nothing
(less than 1e-45 in this call): ss_power_indirect_effect(.2, .1, specified_N = 10, alpha_level = .999, method = "sobel") stopped with
"The power of the Sobel test at N = 10 could not be computed reliably:
the numerical integral over the distribution of the path t statistics
failed (roundoff error is detected in the extrapolation table)." and
is .9983 (DMAR 1.0.x gave .9990; see Changes to Earlier Results). In a
grid of 5,760 Sobel powers (alpha_level from .000001 to .99999,
paths from .02 to .95, N from 10 to 1,000,000, X random and fixed)
they refused 664, at alpha_level .995 (two, both at N = 1,000,000),
.998, .999, .9995, .9999, and .99999, and a path near 1 brought the
refusal to ordinary levels: ss_power_indirect_effect(.39, .999999, specified_N = 10, alpha_level = .5, method = "sobel") was refused and
is .7457. A piece with either roundoff status ("roundoff error was
detected" or "roundoff error is detected in the extrapolation table")
is now accepted when its estimated absolute error is finite and at
most its share of 1e-9 and its value is at least minus that share, so
the power carries an estimated quadrature error below 1.2e-9; the
largest such piece in the grid was worth 2.9e-19. A piece returned
"OK" is accepted whatever its error estimate, as before. A roundoff
piece over its share is refused with class dmar_computation_failed
and the message "... failed (<status>, with an estimated absolute
error of <e> against a budget of <b> for this piece). The joint
significance test (method = "joint_significance") does not need this
integral."; every other status of integrate() is refused as before.
Second, at alpha_level of .99999 and above, the lower end of the
integral can round to just below z, where the integrand was NaN: 40 of
the 664, among them ss_power_indirect_effect(.95, .3, specified_N = 10, alpha_level = .99999, method = "sobel"), stopped with "... failed
(non-finite function value)." and the warning "NaNs produced". The
Sobel test cannot reject at |t_a| <= z, so the integrand is now 0
there, and that call gives .999991. Third, with X random and |a| near
1, the density of |t_a| has a separate peak at each node of the
distribution of the sum of squares of X, and integrate() stepped
over some of them: ss_power_indirect_effect(.999999, .5, specified_N = 10, alpha_level = .5, method = "sobel") gave .3521 where the power
is .5216, and the plan ss_power_indirect_effect(.999999, .5, desired_power = .52, alpha_level = .5, method = "sobel") stopped at
ten million where it is 10. Once two neighboring peaks stand more than
2 standard deviations apart, which takes |a| above about .9, each peak
gets pieces of its own. No call in the grid is now refused or warns,
and none of 270 plans at alpha_level up to .99999 is refused (126
were). In the grid of 5,760 the powers that both computed agree within
3e-15. In a second grid of 2,520 powers with X random (|a| from .85 to
.999999, b from .02 to .999999, N from 10 to 100,000, alpha_level
from .001 to .999), of which they refused 130, the powers that both
computed agree within 5e-14 at |a| up to .999; at |a| = .9999 they
move by up to 2e-7, and at .99999 and .999999 by up to .23 and .97,
and the plan ss_power_indirect_effect(.99999, .2, desired_power = .80, method = "sobel") moves from 9,418,685 to 9,418,684. The new
values agree with independent integrals that share no code with the
function to 3e-11 or better (alpha_level up to .999999, paths up to
.999999), and with 2,000,000-draw Monte Carlo estimates within 4
standard errors in 360 settings with |a| from .9999 to .999999.
The Bryant-Paulson distribution with covariates is computed at large
error df and with many covariates. From about 4 x 10^7 error df with
one covariate (5.6 x 10^7 with six, 10^8 with 30), pbryant_paulson()
returned about 0 (pbryant_paulson(3.7, n_covariates = 1, n_groups = 4, df = 1e8) was 5.6e-25, where the probability is .956), and
qbryant_paulson(), cv_bryant_paulson(), and ci_c_ancova_bp()
stopped with uniroot()'s "f() values at end points not of opposite
sign" or, at the edge of that range, returned a critical value that was
too large (qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = 4.09e7) was 4.5327, where it is 3.6332). df = Inf stopped with
"non-finite function value". With 40 or more covariates the same
failures began at smaller df: about 3 x 10^6 with 40 covariates, 10^6
with 150, 1.3 x 10^5 with 200, and 1.6 x 10^4 with 300. At large df the
weight of the integral over the covariate shrinkage factor lies within a
few multiples of sqrt(p / df) of one end of its range, where the
quadrature over the whole range did not find it, and with many
covariates the factors of the weight overflowed and underflowed apart.
The integral now stops where all but 1e-15 of the weight lies, the
weight is computed on the log scale, and with df = Inf the functions
return the studentized range at infinite df, as they do with no
covariates.
The help of seed in the standardized cluster randomized planners,
mediate(), plot_mediation_mbco(), ci_eta_squared_generalized(),
mediation_mbco(), ss_aipe_indirect_effect(), and
measurement_alignment() says what seed = NULL does: it draws from
the current random number stream and advances it. The help said that it
left the generator's state unchanged.
Help pages state what the code computes and attribute claims to
their sources. Documentation only; no function, argument, output row,
or value changed. ?ci_src and ?ci_reg_coef say that the standard
error of a standardized coefficient treats the sample standard
deviations as fixed (Kelley & Maxwell, 2003, pp. 315--316), so it is
approximate, and give the coverage of the intervals in simulations;
?ci_c_ancova_bp says how much wider the Bryant-Paulson intervals are
than Tukey's (84 percent with three covariates and 2 error df at
alpha = .05);
?var_alpha attributes the N - 1 form to van Zyl, Neudecker, and Nel
(2000) and the N - 2 form to Bonett (2003), as the rows now do; and
?ss_aipe_pcm, ?irt_grm, ?irt_information, ?mr_cv, ?mr_smd,
?skewness, ?kurtosis, ?descriptives, and the pages of four power
functions define their arguments, columns, and formulas. Statements
the code, the data, or the cited source do not bear out are corrected
or attributed on the pages of correlations_test(),
ci_ratio_of_means(), ci_ratio_of_means_paired(), ci_smd(),
expected_smd(), var_smd(), var_r(), expected_r(),
ci_proportion(), krippendorff_alpha(), gwet_ac(),
equivalence_c(), fa_confirmatory(), obrien_test(),
manova_split_plot(), welch_t(), summary_t_test(),
randomization_test_paired(), teacher_expectancy,
prime_time_achievement, and holzinger_swineford, and ?DMAR and
the pages of more than thirty functions and data sets describe the
package without praise or unsupported claims ("modern", "easy to use",
"the most general", and the like).
Refusal messages state the requirement instead of the user's
error. ci_reg_coef() said "You have not properly specified
'alpha_lower' correctly." and says "'alpha_lower' must be between 0 and
.5."; convert_R2_f() and convert_f_R2() say "Specify both 'df_1' and
'df_2', or omit both and specify 'N' and 'p'."; and
reliability_alpha() and reliability_omega() say that DMAR offers no
ADF interval under that treatment of missing data, rather than that
none exists. The condition classes are unchanged.
The vignettes, the README, and DESCRIPTION are corrected.
Documentation only. Rd markup that pandoc left out of the HTML of four
vignettes is Markdown now. The vignettes and the README no longer say
that every ci_*() function inverts a noncentral distribution, that
every plot shows a confidence interval and the sample size, or that
every result is a data frame of term and value rows; their tables of
argument names give the 1.1.0 names; they and the DESCRIPTION describe
the package and MBESS in factual terms; and the README's suggested
citation gives the version as 1.1.0.
The refusals of ci_cv() name the argument at issue. With cv
and no N, the condition's arg is N, the argument that is
missing. data given together with cv is refused with "Give either
'data', or 'cv' with 'N', not both.", where DMAR 1.0.0 asked for the
sample size. A mean or an sd that is not a single finite number is
refused by name; ci_cv(mean = 5, sd = "a", N = 30) (n in 1.0.x)
stopped with "non-numeric argument to binary operator". A sample mean
that is not positive has one message, whether it was given as mean
or computed from data, which states the value and that the interval
requires a positive mean and a positive standard deviation; it no
longer asks whether taking the absolute value of the mean makes sense.
ss_aipe_cv() plans on the expected value of the sample
coefficient of variation again, as Kelley (2007) does. The method
(Kelley, 2007, p. 758) evaluates the width at
E[k | kappa, N] of the paper's Equation 3 at each candidate sample
size. DMAR 1.0.x evaluated it at the population value multiplied by
(1 + 1/(4N)), the nearly unbiased estimate of Equation 4, which is
above the population value where the expected value is below it. The
cause is inherited: MBESS through 3.3.3 computed Equation 3 inside
cv(unbiased = TRUE), which the planner called; cv() was later
changed to the Equation 4 multiplier and the planner followed it
without being edited. Of 34 expected-width conditions of the paper's
Table 1 the earlier value was the tabled one in 10 and one larger in
24; of 16 conditions with assurance .99 it was one to three larger in
all 16. The function now returns the tabled value in 39 of the 50 (24
of the 34 and 15 of the 16), and 101 and 142 for the paper's worked
example (true_cv = .25, width = .10, conf_level = .99, without
and with assurance = .99), where the paper gives 99 and 141 and DMAR
1.0.x returned 100 and 142. The tables were computed with R's
approximation to the noncentral t distribution, and the function now
plans for the exact interval of ci_cv() (see the entry on that
change below). The help states
how the returned sample size relates to the expected width itself.
ss_aipe_cv_sensitivity(), which plans through ss_aipe_cv() when
no sample size is given, changes with it.
The population coefficient of variation in ss_aipe_cv() is the
argument true_cv (it was C_of_V), the name the data generating
value has in ss_aipe_cv_sensitivity(). C_of_V is not accepted. See
Breaking Changes in 1.1.0.
ss_aipe_cv() with assurance is the published procedure of Kelley
(2007), for any assurance strictly between 0 and 1. Two rows of the
tables of that article show a misprint. Table 1, assurance .80, width
.125, coefficient of variation .40 prints 127, between 69 and 122 in
its row; the procedure gives 93. Table 3, assurance .80, width .010
prints 14,279; 28,321; 50,209; 38,120; and 82,817 for coefficients of
variation of .30 to .50, which do not increase; the procedure gives
14,279; 20,475; 28,321; 38,120; and 50,209 with R's approximation to
the noncentral t distribution, with which the tables were computed.
How the plans of the exact distribution compare with the printed
values is in the entry on the exact interval below. The help page
describes the step as it is computed: the search is run for the value
of the article's Equation 3 at the quantile of the sample coefficient,
and applies Equation 3 to that value again at each sample size. With
R's approximation this computation gives the printed value in 524 of
the 528 assurance cells (the other four are the misprints), where the
search for the quantile itself, as the article's text words the step
(p. 759), gives it in 365. The result has
one row,
necessary_N, with or without assurance, as in 1.0.x. assurance
had to be above .50 and may now be any probability strictly between 0
and 1; a value such as .50 asks that the realized interval be no wider
than width in about half of the studies that could be run. A
malformed assurance (a vector, a character string, NA) or one
outside (0, 1) stops with one refusal, of class
dmar_assurance_invalid, that states the value given, why neither 0
nor 1 is accepted, and what to give, where R's own errors appeared. The
procedure is a close approximation and not an exact search. With the
exact distribution, in the 488 printed cells with an assurance and at
most 5,000 cases, the probability that the realized interval is no
wider than width at the returned sample size is .744 to .902 where
.80 was requested (below .80 in 37 of 244) and .9916 to .9997 where
.99 was requested, and away from the published tables it can fall
below the request (.587 for a requested .60 with true_cv = .22,
width = .09, and a 95 percent interval, where the function returns
57). The shortfall grows as the request falls toward .50: in 105 plans
at 95 percent confidence the probability is below the request by at
most .038 for .50, .036 for .55, .058 for .60, .032 for .70, and .001
for .80, and by nothing for .90 and .99; and a request of .50 returns
one fewer than the plan without assurance in 12 of 15 plans (17
against 18 for true_cv = .05 and width = .0375, where the
probability at 17 is .462). The help gives these numbers, and
ss_aipe_cv_sensitivity() estimates the probability for any plan by
simulation.
ss_aipe_sm() takes any assurance strictly between 0 and 1. The
assurance step plans for the value that the sample standardized mean
exceeds in magnitude with probability 1 - assurance, the one-sample
form of the procedure of Kelley and Rausch (2006, Equations 18 to 20
and 25 to 31), and that value is defined for every probability in
(0, 1). DMAR 1.0.x refused an assurance at or below .50. A value
such as .50 asks that the interval be no wider than width in about
half of the studies that could be run, and a value below about one
half returns a smaller sample size than the plan without assurance:
for true_sm = 1, width = 0.5, and a 95 percent interval the
function returns 93 without assurance, 87 at .20, 93 at .50, 100
at .80, and 105 at .90, where the probability that the interval is no
wider than 0.5, from the noncentral t distribution, is .228, .519,
.812, and .925. An assurance that is not NULL or a single number
strictly between 0 and 1 stops with the refusal of class
dmar_assurance_invalid that ss_aipe_cv() gives. No result for an
assurance above .50 changes for this reason (the exact interval of
ci_nc_t() changes a few plans by one or two; see the entry on the
AIPE planners built on ci_nc_t()).
ss_aipe_cv() plans for the exact interval of ci_cv(), so it no
longer returns every value printed in Kelley (2007). The
noncentrality of the interval, the square root of N over the
coefficient of variation, passes 37.62 whenever the interval is narrow,
and the assurance step also takes the noncentral t quantile exactly
where R's qt() inverts pt() (see Exact Noncentral t and
Studentized Range Distributions). The tables of Kelley (2007) were
computed with R's approximation. Apart from the four misprinted cells
(127 in Table 1; 28,321, 50,209, and 82,817 in Table 3; see the entry
above on ss_aipe_cv() with assurance), the function returns the
printed value in 468 of 788 cells (the procedure computed with the
approximation gives all 788), one to three fewer cases in 232, and one
to three more in 88; it returns every printed value
with a width of .125 or more at 90 and 95 percent confidence and 9 of
the 120 with a width of .075 or less at 99 percent. The worked example
of the article (true_cv = .25, width = .10, conf_level = .99)
gives 101, and 142 with assurance = .99, where the article gives 99
and 141 and DMAR 1.0.x gave 100 and 142: at 99 the interval at the
expected value of the sample coefficient is .1006 wide, and .0999 by
the approximation. For true_cv = .10, width = .10, and 99 percent
the plan is 20 (printed 19); for .05, .05, and 99 percent, 19
(printed 21). The help page's numbers on the assurance step are
recomputed with the exact distribution (see the entry above on
ss_aipe_cv() with assurance). The warning ss_aipe_cv() gave when
its search passed a noncentrality of 37.62 is gone.
ci_cv() no longer warns when its upper limit is infinite, and the
tail columns at that limit hold the requested areas. DMAR 1.0.0
warned "The computed confidence interval does not have the same
coverage as the specified confidence interval." and reported
prob_less 1 and prob_greater 0 on the upper limit. The interval
does have the stated coverage. The upper limit is infinite when the
square root of N over the sample coefficient is at most the
1 - alpha_upper quantile of the central t distribution on N - 1
degrees of freedom, and the infinite limit is part of the exact
interval: in the 19,975 of 20,000 normal samples with a positive mean,
for a coefficient of variation of .8 and N = 6, the upper limit of the
95 percent interval was infinite in 31 percent and coverage was .952,
and with .5 and N = 5 it was infinite in 9 percent and coverage was
.948. ci_cv(cv = 1.2, N = 6) returns 0.571 and Inf without a
warning, with prob_less .975 and prob_greater .025 on the upper
limit. No limit changes. The functions that call ci_cv() are quiet
with it:
ss_aipe_cv(true_cv = .45, width = .10, conf_level = .99) (C_of_V in
1.0.x), whose search starts at a sample size where the limit is
infinite, gave the warning once in DMAR 1.0.0 and gives none, with the
plan unchanged by this; and ss_aipe_cv_sensitivity() gives one
warning of its own for the replications with an infinite limit (see
the entry on the replications it summarizes).
A fit whose likelihood has no maximum now stops, where DMAR 1.0.x
reported a table or stopped with an unrelated error. When an error
variance has run off below zero along a ridge of the likelihood, which
is flat or still rising there, lavaan's optimizer stops where its
steps stop gaining and reports convergence, and DMAR 1.0.x reported
that point's estimates, omega, and fit rows, with a warning about a
Heywood case. fa_confirmatory(), cfa_k(), cfa_1(), cfa_2(),
measurement_invariance(), and measurement_alignment(), and the
single-factor fits behind reliability_omega(), now stop instead, for
every output, with an error of class dmar_no_maximum that prints the
lavaan code of the fit (see "A factor analysis or structural equation
model fit whose likelihood has no maximum stops" above). On three
items whose correlations have a negative product (the sample of the
reliability tests, population correlations .4, .3, and -.05, N = 300),
DMAR 1.0.0's cfa_1(), which deleted incomplete cases listwise by
default, returned the table of the point where lavaan stopped, with an
error variance of -1899 and an omega of 442;
cfa_1(missing = "listwise") now stops. With that sample as one group
and a sample of 300 from a population in which three items correlate
.5 as another, DMAR 1.0.0's measurement_alignment() aligned a
configural fit that had stopped with a loading of 21.8 and an error
variance of -475 in the first group and gave the second a factor
variance of 1282; it now stops. ss_aipe_reliability() counts a
simulated sample with no maximum as an interval wider than width,
and ss_aipe_reliability_sensitivity() sets such a replication aside,
whatever its error variances sum to; DMAR 1.0.x used the interval of
the point where lavaan stopped. A bootstrap replication with no
maximum, whose estimate DMAR 1.0.x used, is set aside and counted
among the replications that raised an error. A finite negative error
variance, at a maximum of the likelihood, is reported as before.
ss_aipe_composite_sem(), ss_power_composite_sem(),
ss_aipe_sem_path(), ss_aipe_sem_path_sensitivity(), and
mediation_mbco() stop the same way when the fit they read their
result from has no maximum: a planner's fit of the analysis model to
the population moments, or the lavaan fit from which
mediation_mbco() starts its OpenMx fits. DMAR 1.0.x went on from the
point where lavaan stopped. Given a lavaan fit (std.lv = TRUE,
meanstructure = TRUE) in which those three items measure a mediator
between two other variables, mediation_mbco() reported an indirect
effect of -0.0000008 with a likelihood ratio statistic of 0; and for a
single factor with free loadings fit to their covariance matrix,
ss_aipe_sem_path() stopped with R's "incorrect number of dimensions"
and the composite planners said that the loading "gives no sampling
variability", errors that did not name the cause. A replication of the
Monte Carlo of ss_aipe_composite_sem(), ss_power_composite_sem(),
or ss_aipe_sem_path_sensitivity() with no maximum, which DMAR 1.0.x
kept, is set aside as one that did not converge.
ss_aipe_rmsea_sensitivity(), which reads only the chi square of its
fits, is not checked, and its results change only as its own entries
below describe (its default level and how its intervals are formed).
ss_aipe_sem_path_sensitivity() passes the arguments in ... to
every fit, and ss_aipe_sem_path() adjusts its own fit's plan for
assurance. The arguments in ... go to lavaan::sem(). DMAR 1.0.x
passed them to the planning fit to supposed_cov_matrix alone, so the fit
to
true_cov_matrix, which gives the population value of the path, and the
fit to each simulated sample used lavaan's defaults: with
std.lv = TRUE, the path of the help's population was planned in the
metric that argument sets, where it is 0.577, and simulated and
covered in the metric of a marker item, where it is 0.5. With an
assurance, ss_aipe_sem_path() adjusted a sample size it read from
a second fit of the model made without ..., so the plan under
std.lv = TRUE was adjusted from the plan of the default metric; it
now adjusts the plan of the fit it reports, with one population fit
fewer. Results change only when ... is used, and an argument such as
bounds = "pos.var", which the refusal of a fit with no maximum names
as a choice, now reaches every fit.
measurement_invariance() refuses an item named twice in items.
DMAR 1.0.x built the model f =~ y1 + y1 + y2 + y3 from items = c("y1", "y1", "y2", "y3"), which lavaan (0.7-3) reads as the
three-item model f =~ y1 + y2 + y3, so the ladder described one item
fewer than the call named. The call now stops with an error of class
dmar_items_invalid (reason "duplicate"), as
measurement_alignment() and fa_exploratory() refuse such items.
Only items is checked for a repeat, since DMAR writes the model from
it. An indicator repeated in lavaan syntax given as model is read by
lavaan as one indicator, as before, and a named list given as model
that names an item twice is refused as before, without the class.
fa_confirmatory(), cfa_k(), cfa_1(), and cfa_2() take the
standard errors that match the estimator by default. se now
defaults to NULL, which gives Huber-White (robust) standard errors
under estimator = "MLR" and "robust.sem" under "WLSMV"; DMAR
1.0.x defaulted to se = "standard" and passed it to lavaan under every
estimator but with ordered items, so its MLR standard errors were the
normal theory ones beside a robust test (for the second loading of a
verbal factor of three Holzinger and Swineford tests, 0.249 then and
0.213 now). se = "standard" gives the 1.0.x standard errors (checked
against DMAR 1.0.0's cfa_k(): identical).
fa_confirmatory() fits by full information maximum likelihood by
default, as fa_exploratory() does, and cfa_k(), cfa_1(), and
cfa_2() follow it. missing = "listwise" gives the listwise deletion
that DMAR 1.0.x applied by default, and fa_exploratory() accepts it as
well. With incomplete items the default now uses every case with an
observed item, so the estimates, their standard errors, and N change.
With complete data the estimates are unchanged,
and for estimator = "ML" so are the chi square, CFI, TLI, and RMSEA;
the standard errors come from the observed information, which lavaan
uses under FIML and Savalei (2010) recommends with incomplete data,
rather than the expected information, and moved by at most about 2
percent on the two factors of the examples. The item intercepts (nu
rows) join the table, and AIC and BIC count them. With estimator = "MLR" FIML on complete data leaves the standard errors as listwise
deletion gives them (DMAR 1.0.x took other ones under MLR; see the
entry on se below), and the robust CFI and RMSEA
are the corrected version 3 indices of Zhang and Savalei (2023), named
under the table. Ordered items, the least squares estimators, se = "robust.sem", and the Satorra-Bentler kind of test, which FIML cannot
accompany, are fitted as before when no item value is missing; when
values are missing they ask for missing = "listwise" (or "pairwise"
for ordered items) rather than deleting cases unasked, and
fa_exploratory() now treats ordered items the same way. The srmr
row is the SRMR of the covariance residuals alone whenever the model
has a mean structure (lavaan's srmr_bentler_nomean, as
sem_fit_measures() reports it), so the intercepts, whose residuals
are zero, do not lower it. With auxiliary variables (lavaan's aux
under FIML) the srmr row is over the items alone: lavaan averages
the auxiliary variables in, whose residuals are also about zero, and
for the three-factor model of the nine Holzinger and Swineford tests
with 40 values missing per item the SRMR of .068 fell to .062 with
one auxiliary variable and .058 with two. A fit whose observed
information matrix is singular reports no standard errors, with a
warning, as a listwise fit
already did; lavaan inverts such a matrix without complaint and gives
finite standard errors for parameters that are not identified.
fa_exploratory() and factor_count() record the FIML method as
"fiml" rather than lavaan's "ml".
cv_smm() is right at very small degrees of freedom. In DMAR 1.0.0,
with df below 1 at conventional levels (below about .8 at an
alpha_level of .05, and below about .6 at .10) or with df of about 2
or less and an alpha_level of .02 or less, its integral over the chi
squared variate missed a sliver of the integrand by the origin: the call
stopped with "the integral is probably divergent", or, at df = 2 and
alpha_level = .001, returned a critical value whose tail area was 1.5
to 5 times alpha_level (32.80 rather than 40.43 for two comparisons).
Each root is now checked against the upper tail computed by conditioning
on the maximum modulus instead, and found on that tail when the two
disagree. Every value that was right is returned unchanged, bit for bit.
design_consequences(), sd_unbiased(), and var_ete() are right
at a very large df or N. Each uses
E(S)/sigma = sqrt(2/df) Gamma((df + 1)/2) / Gamma(df/2), the mean of a
standard deviation on df degrees of freedom relative to sigma, or the
variance 1 - (E(S)/sigma)^2 that goes with it. E(S)/sigma is below 1
at every df and about 1 - 1/(4 df) for a large one, but each function
computed it as a difference of two lgamma() values that nearly cancel
there. design_consequences()
multiplies the width it would have with the standard error known by
E(S)/sigma for mean_ci_width and expected_half_width, and by the
square root of the variance for sd_ci_width. The factor was off by
4e-11 at df = 1e5, first reached 1 at df = 10810066, and was 1 or
more at about half of all df above 1e8, which put mean_ci_width
above the fixed width and sd_ci_width at 0. sd_ci_width lost more,
because one minus the square of a number near 1 keeps only the digits
the two do not share: it was off by 1e-12 at the 118 df of 60 per
group, by 8e-6 at df = 1e5, by 0.1 percent at df = 1e6, and was 57
times its value at df = 1e9. No other row involves them, and at the
df of an ordinary design the mean rows move only in the last digits
(by 4e-15 at 118 df). sd_unbiased() divides by the same factor and
returned s unchanged at N = 1e8 and .998 s at N = 1e12, where
the correction is upward; var_ete() with covariate_value = "sd" uses
the variance and moves by 2e-13 or less at a few hundred cases and by
3e-6 at N = 1e6. Both are now formed from the logarithm of
E(S)/sigma, the mean as its exponential and the variance through
expm1() of twice it. From df = 100 on the logarithm is its
asymptotic expansion, and below that the expansion carried down two
df at a time by an exact recurrence whose terms all have one sign, so
nothing cancels at any df. Against a 320-bit evaluation, both are
right to a few units in the last place: over some 490,000 df from
0.05 to 1e15, the largest relative error found is 5.5e-16, about 2.5
times the machine epsilon. E(S)/sigma is never above 1, and it never
decreases as df grows on grids as fine as 1e-13 apart, including
across every even df from 2 to 100, where the computation changes
form; from one double to the next it can step back by a unit in the
last place, as any value computed with a rounding error can.
ss_aipe_mean() and ss_aipe_two_phase(), both new in this release,
use the same factor.
expected_r() and expected_partial_r() are right at a very large
N. E[r] = rho 2F1(1/2, 1/2; (N + 1)/2; rho^2) Gamma(N/2)^2 /
{Gamma((N - 1)/2) Gamma((N + 1)/2)}, and the gamma ratio is the square
of E(S)/sigma on N - 1 df (the entry above). expected_r() formed it
from three lgamma() values that nearly cancel at a large N, and it
was off by 1e-11 at N = 1e5 and 1e-7 at N = 1e8. The bias rho - E[r]
is only about rho (1 - rho^2) / (2N), so its error was larger in
proportion: at rho = .5 the bias column was 0.2 percent too small at
N = 1e6, 26 percent too large at N = 1e7, 31 times its value at
N = 1e8, and of the wrong sign at N = 3e8, where E[r] came out above
rho. The ratio now comes from the logarithm in the entry above, and the
bias and relative_bias columns are formed directly rather than from
rho - E[r]: relative_bias is one minus the product of the series and
the ratio, formed from their logarithms, and bias is rho times it.
The difference rho - E[r] keeps only the digits the two do not share,
and with the ratio right to a few units in the last place it is still
off by 3e-7, relative, at N = 1e8. Against a 320-bit evaluation the
bias is now right to 2e-14, relative, for |rho| of .9 or less at N
from 4 to 1e12. So that the series carries the digits the bias needs,
the option DMAR.expected_r.tol now stops the sum once a term is below
that fraction of the sum of the terms after the leading 1, rather than
of the whole series, and the sum can run to more terms than in 1.0.x;
the non-convergence warning still compares the last term with the whole
sum, so it is given where it was. At N of 1000 or less, E[r] moves by
less than 2e-12, relative. A rho so small that rho^2 is 0 in double
precision, below about 1.6e-162, gave E[r] = 0 and a relative_bias of
1; it now gives E[r] = rho c4(N)^2. expected_partial_r() calls
expected_r() at N - (p - 1) and moves with it.
manova_split_plot() tests the within-subjects main effect averaged
over the groups, as Maxwell, Delaney, and Kelley (2027, Section 14.6)
do. The default Type III B rows test whether the grand means of the
within-subjects contrasts are zero. DMAR 1.0.x fitted the
between-subjects factor under R's default treatment coding, in which
the intercept that Type III tests is the first group's mean of each
contrast, so its B rows were the simple effect of the within-subjects
factor in the first group, tested against the error matrix pooled over
the groups (the book's Equation 14.66), rather than the main effect.
The factor is now coded with sum-to-zero contrasts whatever
options("contrasts") says, so Type III tests the unweighted grand
means, which the book prefers and notes that SAS and SPSS test by
default (Section 14.6.9 and note 6 of Chapter 14), and
ss_type = 2 tests the weighted grand means, in which each group counts
in proportion to its size. On the reaction times of the book's Table
14.9 (10 young and 10 old participants at angles of 0, 4, and 8
degrees), DMAR 1.0.0 gave F(2, 17) = 36.22 for angle, the value the
book reports for the simple effect of angle among the young
participants (Equation 14.66, p. 810); the book's angle main effect,
F(2, 17) = 110.65 (Equation 14.54, p. 798), is what the function now
returns. With only the 0 and 8 degree conditions (Table 14.7), DMAR
1.0.0 gave F(1, 18) = 75.77 where the book gives 233.41 (p. 792). The
A and interaction rows are as before, as is every row under
ss_type = 2, since none of them depends on the coding. The B rows
were already the main effect whenever the contrasts in force for the
between-subjects factor summed to zero: for an ordered factor under
the default ordered coding, contr.poly, and for an unordered factor
when options("contrasts") named contr.sum or contr.helmert. Under
contr.SAS they were the simple effect in the last group. No argument
gives the 1.0.x B rows. They are the Time rows of the call DMAR
1.0.x made under R's default contrasts,
car::Anova(lm(Y ~ g), idata = data.frame(Time = factor(within, levels = within)), idesign = ~ Time, type = 3),
with Y the matrix of the within columns and g the between-subjects
factor. The help no longer says that Types II and III coincide for a
single between-subjects factor: they give the same tests of A and of
the interaction, and the same test of B only when the groups are of
equal size. The model is now fitted to objects of its own, so a
between-subjects column named Time, for which DMAR 1.0.x reported the
test of A in the B rows, is handled like any other, as are a
between-subjects column named Y or with a name that is not syntactic
and a within-subjects column named Y, with which it stopped with an
error. The function no longer passes on car's warning HF eps > 1 treated as 1, which comes from the univariate tests it does not
report.
reliability_omega(), reliability_alpha(), reliability_H(), and
measurement_invariance() take full information maximum likelihood as
the default of missing as well, so that the omega of
reliability_omega() and that of fa_confirmatory() agree on
incomplete data, and reliability(), which passes missing on only
when it is supplied, follows them. missing = "listwise" gives the
listwise deletion that DMAR 1.0.x applied by default. The default
resolves by one rule in the four functions: the analysis is FIML
whenever an item value is missing and the complete data analysis
otherwise. That differs from fa_confirmatory(), which fits complete
data by FIML too. Data in which no value is missing, and a covariance
matrix, are analyzed as they stand, as DMAR 1.0.x analyzed them,
because the complete-data intervals of alpha and omega (the closed
forms, "adf", and "likelihood", none of which FIML gives) need that
analysis, and so no complete data result changes: FIML would give the
same estimates, but its standard errors come from the observed
information and the Huber-White sandwich, alpha's bootstrap would
refit a saturated model on every replication, about 70 times as long,
and an MLR ladder's robust indices would be the corrected ones of
Zhang and Savalei (2023). A row with every item missing (every model
variable, in measurement_invariance()) does not count as a missing
value, since listwise deletion and FIML both drop it, so data that are
complete but for a blank row keep the complete data analysis.
Incomplete items are analyzed by FIML, so the estimates, their standard
errors, and N change; under FIML the analytic alpha's default interval
is "ml", since Bonett's needs complete cases. An option FIML
cannot accompany (the complete-data closed form intervals, "adf", and
"likelihood" in alpha and omega; ordered items, estimators other
than ML and MLR, se = "robust.sem", and the Satorra-Bentler kind of
test in measurement_invariance()) is used as before when no item
value is missing, and with missing values the call stops and names the
options rather than deleting cases unasked; the checks of omega's
denominator and alpha's estimator come first, so that neither option
named meets another refusal. Auxiliary variables imply FIML, as before.
reliability_H() takes data for the first time in this release, and
reliability_omega_categorical() keeps
listwise deletion as its default. In measurement_invariance(), under
FIML by default or by name, a group that never observes a variable of
the model, or never observes two of them together, is refused by name
before any fit (an error of class dmar_group_coverage, a
dmar_refusal carrying the variables and pairs), with what to do
instead; lavaan stopped with "subscript out of bounds", which the new
default would have made the ordinary result for
holzinger_swineford's t25_paper_form_board_r and t26_flags by
school. An explicit missing = "fiml" with se = "robust.sem" or a
test of the Satorra-Bentler kind now stops, as in fa_confirmatory(),
where through 1.0.1 lavaan fit the complete cases with a warning, and
so does a ladder lavaan fitted by another treatment than the FIML
asked for (it fits GLS, WLS, and ULS in two stages).
reliability_omega() forms the default interval of the model
implied omega on the logit scale. The default ci_method is
"mlr_logistic" where it was "mlr": the same robust standard error,
with the limits formed on the logit scale and transformed back to the
scale of the coefficient (Browne, 1982; Kelley, 2007), so that
reliability_omega(denominator = "model_implied") and the omega row of
fa_confirmatory() give limits of the same form. It applies under
listwise deletion, FIML, and two-stage estimation alike. The change
rests on a Monte Carlo study (in the repository's
methodology/reliability_limits) in which the logit limits met every
criterion fixed before the run, with normal data under listwise
deletion and FIML and with nonnormal data fitted by MLR. The study
formed both kinds of limits from the same standard error, the normal
theory maximum likelihood one for normal data and the Huber-White one
of MLR for nonnormal data, so it compared the scale on which the limits
are formed: with normal data the logit limits covered from .939 to .956
in every cell, against .910 to .952 for the limits on the scale of
omega, whose misses fell mostly on one side. No cell used the
robust.sem standard error of "mlr_logistic" with complete data, which
estimates the same asymptotic variance as the Huber-White one.
ci_method = "mlr" gives the limits of DMAR 1.0.x; ci_method = NULL
is the default, as
it was through 1.0.x. Given a covariance matrix with denominator = "model_implied" and no ci_method, reliability_omega() now uses
"ml_logistic", the maximum likelihood standard error with the limits
formed the same way, as reliability_H() does, where through 1.0.x it
stopped because its default needed raw data; reliability(), which
chose "ml" on that path, follows it. Under wald_df = "missing_information", whose calibration covers the intervals on the
scale of omega only, the default remains "mlr" ("ml" with a
covariance matrix). The estimate and the se row do not change, nor do
robust omega (whose default is no interval) and coefficient alpha's
defaults.
FIML and two-stage reliability fits count distinct cases that
observe an item. A row observed only on auxiliary variables informs
their distribution, but not the items given them; a repeated row adds
no dimension. The count must exceed the items plus auxiliary
variables, in the data, every bootstrap replication, and each
jackknife deletion. Fitted coefficients under listwise deletion count
distinct complete rows. Listwise alpha is offered as an alternative
only when it would accept those rows, with Feldt's interval named
when exactly two are complete. A standard error lavaan could not
compute is NA, rather than zero.
FIML and two-stage reliability coefficients refuse a constant composite. When complete rows outnumber the items, their sum is checked directly. A constant sum makes the saturated likelihood unbounded as its variance tends to zero. Otherwise the EM covariance estimate is checked with a tolerance reflecting EM convergence. Bootstrap replications follow the same rule. H's weighted composite is exempt. For all Holzinger and Swineford pupils on visual perception, cubes, lozenges, and paragraph comprehension, center each pupil's scores on that pupil's mean, then delete the first ten visual perception scores. Version 1.0.1's model implied FIML alpha was 1.49e-13 with limits [0, 9.23e-8]. The data are now refused because the composite is constant in the 291 complete rows.
An indefinite supplied S is named as not a covariance matrix when
its composite variance is not positive. The message gives its
negative eigenvalue, and the composite variance when clearly
negative, rather than calling the matrix singular with a zero
composite variance. For S <- matrix(-.9, 3, 3); diag(S) <- 1,
version 1.0.1 returned alpha 3.38 with limits [4.31, 1] at N = 100
with ci_method = "feldt". This matrix is now refused; positive
semidefinite ipsative matrices keep the constant-composite message.
The equal-loading factor analytic plan uses its fitted
coefficient. Let S <- matrix(.3, 4, 4); diag(S) <- c(1, 1, 1, 4)
and S[4, 1:3] <- S[1:3, 4] <- -.5. At width .20,
ss_aipe_reliability(S = S, model = "True Score", type = "Factor Analytic", width = .20) plans 244 cases, against 786 in 1.0.1, whose
plan used alpha of the correlation matrix. The Parallel model gave
1669 and is now refused as far from parallel. Conversely, S <- diag(c(10, 10, 1, 1)); S[1, 2] <- S[2, 1] <- 4; S[3, 4] <- S[4, 3] <- -.3 has positive alpha but a zero fitted loading. At width .30,
1.0.1 planned 202 for Parallel and 386 for True Score; both now
refuse it.
Highly reliable congeneric populations are checked at a feasible
estimation size: four loadings of .99 with error variances 1 - .99^2 at width .20 still plan 2 cases, as in 1.0.1. The closed form
can remain below the minimum estimation size. Every assurance plan
carries no_interval, including zero, and glance() adds fmi,
necessary_N_complete, and no_interval when present. The closed
form's tidy() width is NA, as in 1.0.1.
fa_confirmatory() refuses correlated factors no case observes together.
Their covariance does not enter the observed-data likelihood; a third
factor observed with both does not identify it. Split the four
spatial and three verbal Holzinger and Swineford tests between forms
using set.seed(3); form <- sample(1:2, 301, replace = TRUE), with
the spatial block missing on form 1 and verbal on form 2. Version
1.0.1 reported the correlation at its starting value, zero, and
omega_spatial .650 with a standard error of zero and limits [.650,
.650]. The fit is now refused by name; fit each factor alone or link
the forms with common items. Orthogonal factors are fitted, since
their correlation is fixed, and their coefficients and standard
errors equal the separate fits. More generally a fit whose
information cannot be inverted reports NA for standard errors,
tests, and intervals in every table output, with a warning.
Zero degrees of freedom with a nonzero statistic gives no TLI or
RMSEA. A binding inequality can leave a positive statistic at zero
degrees of freedom, which lavaan does not count as an equality. For a
factor of three unit-variance variables correlated .8, .8, and .5, at
N = 200 with bounds = "pos.var", the statistic is 32.8 and CFI
.925. Version 1.0.1 reported TLI and NNFI 1 and RMSEA 0 [0, 0]; those
rows are now NA, since they divide by the degrees of freedom. CFI
keeps .925. The RMSEA and following delta_rmsea of such an
invariance rung are also NA. A rung's CFI decides the version of
its indices independently of its test. An exactly fitting saturated
model keeps its earlier convention.
CFA summaries carry the method and reference of their results.
glance() adds estimator, missing, test, fit_indices, and
rmsea_ci_level after its earlier columns. tidy() of CFA,
reliability, mlmr(), and mlmr_mv() results carries wald_df per
row, and reliability's glance() carries it too. The default is
Inf. Rows on a t reference can therefore be stacked with rows on
the normal reference without losing which interval each has.
measurement_invariance(group_partial = character(0)), refused in
1.0.1, now frees nothing, as NULL does. Empty group.partial
passed through ... also frees nothing.
reliability_omega() refuses a single-factor solution that leaves
no reliability coefficient. On three items whose correlations had a
negative product, it returned an omega of 471, from a loading of 43.6
and an error variance of -1899, with only a warning about a Heywood
case. A single factor implies that the product of the three
correlations is the square of the product of the loadings, so no real
loadings reproduce such correlations and the maximum likelihood
estimate does not exist. The function now stops with a message that
quotes the correlations, under either denominator, for covariance
input, and under FIML. It also stops when the error variances of an
improper solution sum below zero, so that the model implies a negative
error variance for the composite and an omega above one, the signature
of a solution that has run away (McDonald, 2004), and the model
implied reliability_alpha() does the same for the tau-equivalent
model. An improper solution whose coefficient is still a proportion is
returned with a warning, as before, and the warning now names the
item, its error variance, and its observed variance; the observed
denominator, which gave no warning of its own, now gives it. In a
bootstrap, a replication with no coefficient is set aside under either
improper setting (on 80 cases with one loading of .95, one such
replication in 200 had put the bootstrap standard error at 1.02 rather
than 0.049), and a replication of ss_aipe_reliability_sensitivity()
whose fit is now refused is set aside rather than returning an absurd
interval. This is a deliberate, loud change for those samples: a
number that described where the optimizer stopped is replaced by an
error. lavaan's own per-replication notices of improper solutions no
longer reach the console.
Each confidence limit of a reliability coefficient is clamped into
[0, 1], and a negative estimate is warned about. Through 1.0.1 the
lower limit was raised to 0 and the upper limit lowered to 1, each on
its own. So when a negative coefficient alpha or KR-20 had an interval
lying wholly below zero, the limits came back inverted, with the
estimate outside them. On four binary items, one nearly the reverse of
another (set.seed(3); z <- rbinom(60, 1, .5); Xb <- cbind(i1 = z, i2 = 1 - z, i3 = rbinom(60, 1, .5), i4 = rbinom(60, 1, .5)); Xb[1:6, 2] <- z[1:6]), reliability_kr20(Xb) gave -0.821 with Feldt limits
[0, -0.171], and reliability_alpha(as.data.frame(Xb)) gave Bonett
limits [0, -0.196]. Both limits are now clamped into [0, 1], so they
stay in order and that interval is [0, 0]. The estimate is returned as
computed, with a warning (class dmar_negative_reliability) that it
lies outside the range of a reliability coefficient and that an item
keyed in the opposite direction is the usual cause. The _logistic
intervals, which gave NA limits without a word, now say why. Only
results whose upper limit, before clamping, was below zero change. In
ss_aipe_reliability_sensitivity() such a replication now has a width
of 0 rather than a negative one, and the warning is muffled inside the
loop. At true_reliability = 0 such a replication also covers the
true value, since [0, 0] contains zero, so none can miss high there;
through 1.0.1 its inverted limits counted as a miss high. With four
items, N = 40, and Bonett's interval
(ss_aipe_reliability_sensitivity(true_reliability = 0, n_items = 4, width = .3, specified_N = 40, ci_method = "bonett", G = 400) after
set.seed(1)), the share of intervals missing zero goes from .0400,
.0325 of it high, to .0075, all of it low. With the "ml" interval it
goes from .0425, .0100 of it high, to .0350, all of it low; its low
misses move from .0325 to .0350 because that interval's standard error
now comes from the covariance matrix (see the next entry). The
function evaluates the interval a study would report, and a miss high
at zero would place the reliability below zero; its help says so.
Above zero the counts do not change.
The "ml" interval of reliability_alpha() and
reliability_kr20() takes its standard error from the covariance
matrix. The van Zyl, Neudecker, and Nel (2000) variance was
evaluated on the correlation matrix of the items, which is the
variance of alpha of standardized items, and paired with alpha of the
raw items. The paper defines it on the covariance matrix, as do Terry
and Kelley (2012, Eq. 8) and Kelley and Pornprasertmanit (2016, Eq.
20), and the help already said it was computed from the covariance
matrix. Whenever the item variances differ, the standard error was
wrong in either direction. With equal loadings .7 and error variances
.1, .2, 1, and 2 at N = 206 it was .0256 where alpha-hat varies with a
standard deviation of .0339, and the 95 percent interval covered .863
of 4,000 samples; it now covers .952. With item standard deviations
from .8 to 1.9 it covered .889 (.944 now), and with the large
variances on the high loading items its standard error was twice the
standard deviation and it covered .9998 (.945 now). Items of equal
variance are unaffected. On six binary items of unequal difficulty
KR-20's standard error moves from .0311 to .0279 against a sampling
standard deviation of .0242. The default Bonett interval and the FIML
and two-stage "ml" intervals, which already used the covariance
matrix, are unchanged, and on complete data the listwise and FIML
standard errors now agree up to the divisor. The code came from MBESS,
whose ci.reliability() (through its internal
.interval.type31_32_type1()) still evaluates the variance on the
correlation matrix, so the two packages' "ml" alpha intervals now
differ when the item variances do.
Item names lavaan cannot parse no longer stop the single-factor
coefficients. Through 1.0.1, reliability_omega() (robust omega,
the default, and the model implied omega), the model implied
reliability_alpha(), and reliability() passed the column names to
lavaan. lavaan stopped on a name such as "item 1" or "1a", and the
error was reported as a model that did not converge; under FIML the
message also gave advice about small samples. The items now enter the
fit as y1, ..., yJ, and every result, message, and warning carries
the user's names. lavaan's warnings, such as "some observed variables
are perfectly correlated; ... variables involved are", are raised
again with them. So are those of the saturated model behind FIML,
which named the variables v1, v2, ... or y1, y2, ... even in
1.0.1, for analytic alpha under FIML and for the fits with auxiliary
variables. reliability_H() from data, new in 1.1.0, works the same
way. On the five verbal tests of holzinger_swineford
(t5_general_information to t9_word_meaning) renamed "item 1" to
"item 5", robust omega is 0.900, the same as under the data's names,
where 1.0.1 stopped with "The single-factor CFA model did not
converge." A fit that lavaan stops with an error is reported with
lavaan's own message rather than as a convergence failure, and the
message names the items and auxiliary variables as the data do, in
reliability_omega(), the model implied reliability_alpha(), and
reliability_omega_categorical(); a fit that ran and did not converge
is still reported as one, with the advice for FIML and two-stage
estimation. reliability_omega_categorical() had reported every
failure of its fit as "The single-factor categorical CFA model did not
converge," which sent the user looking for a problem with the sample.
Its bootstrap sets a replication without a fit aside and counts it as
before, so no estimate, interval, or count changes.
reliability_omega()'s advice on requesting the robust omega interval
now comes after the estimate, so a call that stops on its fit gives no
advice first.
A name given to two columns is refused by name wherever the columns
are selected by name. Through 1.0.1, under FIML the first of two
same-named columns was analyzed twice, and the call reported a model
that did not converge; with auxiliary variables the second was dropped
without a warning. On t1_visual_perception, t2_cubes,
t4_lozenges, and t6_paragraph_comprehension, with the t2_cubes
column also named t1_visual_perception and t9_word_meaning as an
auxiliary variable, reliability_alpha() returned 0.555 with J = 3,
the alpha of the other three tests (0.606 with J = 4 when the columns
are named apart). An auxiliary variable's name given to an item column
as well is refused too.
reliability() picks the coefficient from the items alone.
Auto-detection read every column of data, auxiliary variables
included. So ordered items with a continuous auxiliary variable, or an
integer one with more than 10 values, went to continuous omega under
FIML rather than categorical omega under multiple imputation, and a
factor auxiliary stopped the call with a base R error. The rule now
reads only the columns not named in aux.
fa_confirmatory() reports the average variance extracted and coefficient H
as NA for a factor with a negative error variance, and omega as NA
when the factor's error variances sum below zero. An item whose
error variance is negative has a communality above one. The AVE
averages the communalities, and H, the largest reliability a weighted
sum of the items attains, is at least that communality, since the item
weighted alone attains it, so for such a factor neither is a
proportion of variance. Through 1.0.1 both were reported beside the
Heywood warning: on holzinger_swineford, a factor of visual
perception and lozenges beside one of general information, paragraph
comprehension, and sentence completion had H = 1.321 [0.166, 2.476]
and an AVE of .726. Both rows are now NA, the estimate and its
interval alike, with one warning that names the items and their
communalities. Omega depends on the error variances only through their
sum: while the sum is positive it is a proportion and is reported, as
reliability_omega() reports it (.707 for that factor), and when the
sum is negative the model implies a negative error variance for the
sum score, so omega is NA too (visual perception and cubes beside
three speed tests had omega 1.050, H 1.536, and an AVE of .788). For
ordered items the communality is that of the latent response. Through
1.0.1 an improper ordered factor stopped the whole fit with the
mvtnorm error "'corr' is not a correlation matrix"; its categorical
omega, which is not defined for an improper solution, its AVE, and its
H are now NA, named in the same warning. The classed Heywood warning
is unchanged, and output = "fit" keeps lavaan's defined parameters.
average_variance_extracted() reports NA, with a warning, for a
factor whose solution is improper, as fa_confirmatory() does. An indicator
whose error variance is estimated below zero has a communality above
one, so the AVE of its factor, which averages the communalities, is
not a proportion of variance. fa_confirmatory() now reports that factor's
ave row as NA, but average_variance_extracted(fit) averaged the
squared standardized loadings regardless: for visual perception and
lozenges beside a factor of general information, paragraph
comprehension, and sentence completion in holzinger_swineford, where
visual perception's communality is 1.304, it reported .726. The
factor's ave, and its interval when one is requested, are now NA,
with a warning that names each such indicator and its communality in
the words of fa_confirmatory()'s. The check reads the sign of the error
variance and takes the communality as one minus the standardized error
variance, so it gives the same answer whatever the identification of
the fit, for the latent responses of ordered items under either
parameterization, and in every group of a multiple group fit. Proper
factors are unchanged (.705 for the verbal factor there). When every
factor of the fit is improper, ci_method = "percentile" forms no
interval: no bootstrap is run, and no confidence level is recorded or
printed.
average_variance_extracted() counts the improper replications its
bootstrap keeps, and its warning gives the caution for each factor.
A replication whose resample solution is improper for a factor that
is proper in the observed fit (an indicator with a negative error
variance) is kept, as the reliability family's default improper = "keep" keeps one. Through 1.0.1 it was neither counted nor reported,
though its AVE averages a communality above one and can exceed one.
bootstrap attribute
(attempted, returned, failed, improper_kept, refit_failed,
caution, improper_kept_by_factor, and the rest), and prints the
family's bootstrap line under the table, at the console and in a
knitted report. When two or more factors have an interval, a second
line gives the improper replications kept for each.refit_failed, and the line and the warning both say
that they "failed to refit or did not converge" and were set aside,
where 1.0.1's warning said that they "failed to refit (or did not
converge) and were dropped".holzinger_swineford (set.seed(113); d <- holzinger_swineford[sample(301, 80), ]), fit by
lavaan::cfa() with a verbal factor of paragraph comprehension,
sentence completion, and word meaning and a spatial factor of visual
perception, cubes, and lozenges, then
average_variance_extracted(fit, ci_method = "percentile", B = 200, seed = 113). The spatial factor is proper in the sample (AVE .449),
yet 101 of the 198 replications that returned a value were improper
for it, and its upper limit is 1.19. Version 1.0.1's warning said
only that 2 of 200 replications failed to refit (or did not
converge) and were dropped. The warning now says that they were set
aside, that 101 of the 198 used came from an improper (Heywood)
solution and were kept, and that for spatial 103 of 200 replications
(51.5%) came from a solution improper for it or were set aside,
against the 2.5% beyond each limit of a 95% interval. The line under
the table says "for spatial, 51.5% of the replications came from a
solution improper for the factor or were set aside, at least the
2.5% beyond each limit", and the second line "spatial 101, verbal
0".dmar_ave ahead of dmar_tbl on every
path. No interval changes.average_variance_extracted() reads a loading fixed or constrained
to zero as none. Through 1.0.1 it averaged the squared standardized
loading of every =~ row of a fit, so a loading fixed at zero
(0*item) or constrained to zero (a*item with a == 0) entered the
factor's average as a loading of zero. On holzinger_swineford
(lavaan::cfa(model, data = holzinger_swineford, std.lv = TRUE)), a
visual factor of visual perception, cubes, and lozenges written with
0*t7_sentence beside a verbal factor of general information,
paragraph comprehension, and sentence completion is the same model as
the one without that row (chi square 22.876 on 8 degrees of freedom
for both), yet it had an AVE of .279 rather than the .371 of its three
tests. The verbal factor written with 0*t1_visual_perception beside
the same visual factor had .529 (.705 times 3/4) rather than .705.
Such a row is now left out of the average and out of the check for an
improper solution described above, as factor_scores(),
bifactor_indices(), and reliability_H() read it. With a visual
factor of visual perception and lozenges alone, whose visual
perception error variance is negative, 1.0.1 gave the visual factor
.726 and the verbal factor written with 0*t1_visual_perception .529.
The visual factor's AVE is now NA under that check, and the verbal
factor's is the .705 of the model without the row, since visual
perception is not one of its indicators. Fits without such a row are
unchanged.
fa_confirmatory(), cfa_1(), and cfa_2() refuse group passed through
.... fa_confirmatory() labels every loading, error variance, intercept,
and factor correlation, and lavaan holds a label shared by several
groups equal across them. A group argument therefore fit one pooled
model, every measurement parameter equal across the groups and each
factor's variance fixed at 1 in each: a model of complete invariance
rather than a model by group. lavaan warned about the single labels,
and the table repeated each labeled row once per group with the same
estimate. The call is now an error that names the route:
measurement_invariance(), whose configural model frees every
parameter in each group and whose fit gives reliability_H(fit = , factor = , group = ) the H of one group, and reliability_omega() on
one group's rows.
fa_confirmatory() names the version of its fit rows after FIML, and its
SRMR when the model has a mean structure. Under missing = "ml"
(or "fiml"), the cfi, tli, nnfi, rmsea, and RMSEA interval
rows have always been lavaan's robust indices. For a FIML fit these
are the corrected indices of Zhang and Savalei (2023), version 6
(version 3 with estimator = "MLR"), and they are not computed from
the chi_square row beside them. The srmr row of a model with a
mean structure is lavaan's SRMR, which averages in the mean
residuals, so it is not the SRMR sem_fit_measures() reports: .0229
against .0259 on the verbal and deduction tests of the help page
(hs_factors) with 90 sentence completion scores deleted at random
(set.seed(1), then t7_sentence[sample(301, 90)] <- NA) under
missing = "ml". Lines under the table now say both, and the table
carries them in a new fit_rows attribute, so a knitted report shows
them too. When the robust indices of a fit are not available (lavaan
computes none, and DMAR computes them only for the mean and variance
adjusted statistic), every index row holds one other version and a
line says which. That is the standard indices for a maximum
likelihood fit of continuous items, and the scaled ones, or else the
standard ones, after FIML or two-stage estimation, for ordered items,
or by least squares. Another line names robust indices DMAR computed,
for the mean and variance adjusted statistic. Whether the fit used
FIML is read from the fit, so a fit lavaan made to the complete cases
(with missing = "ml" and a test it cannot combine with FIML) is not
labeled with the FIML corrected indices. An exactly fitting model
with no degrees of freedom gets no version line under FIML or
two-stage estimation; its SRMR and two-stage method lines stay. With
a nonzero statistic the version line names CFI alone and another line
explains the undefined TLI and RMSEA rows. A help paragraph says
which version each fit row is. No number changes here; the next entry
gives the rows that do.
fa_confirmatory() reports one version of its fit indices, and the robust
indices of a fit with the mean and variance adjusted statistic.
cfi, tli, nnfi, rmsea, and RMSEA
interval rows took the first of its robust, scaled, and standard
versions that lavaan had computed, so a row whose robust version was
NA could hold another version than the rows beside it.test = "mean.var.adjusted" passed through ...), for which lavaan
computes no robust indices, every row held the scaled version, whose
population value changes with the kurtosis of the data (Savalei,
2018). The rows now hold the robust indices, computed by DMAR as
Savalei (2018) shows they can be. They equal lavaan's under
estimator = "MLM", and a line under the table says that DMAR
computed them.fa_confirmatory(data = holzinger_swineford, factors = hs_factors, test = "mean.var.adjusted")), CFI .961 and RMSEA .046 [0, .089] become
.993 and .047 [0, .091], both intervals at 90 percent, the level of
1.0.x and the default of rmsea_conf_level. On the three-factor model of nine tests
(see the sem_fit_measures() entry on the mean and variance
adjusted statistic), CFI .734 becomes .932 and RMSEA .089 [.068,
.110] becomes .091 [.070, .113].NA for TLI, NNFI, and RMSEA, as
the entry on zero degrees of freedom explains.mlmr() and mlmr_mv() leave the covariances among the predictors
free when auxiliary variables are supplied. The saturated correlates
of auxiliary were written as a covariance of each auxiliary with
each predictor. That made the predictors endogenous to lavaan, whose
automatic covariances among exogenous variables then no longer reached
them, and nothing else freed them, so with two or more predictors
every covariance among them was fixed at zero and the model that
should be saturated (Graham, 2003) was not. The covariance of every
pair of predictors is now written, and the model over the outcomes,
the predictors, and the auxiliaries is saturated, with chi square 0 on
0 df. Its coefficients are the regression implied by the full
information maximum likelihood means and covariances of every
variable, auxiliaries included (what cov_fiml() returns), and on
complete data they and R squared are those of lm(). A fit with one
predictor, or none, is unchanged. When a predictor has missing values,
the coefficients, standard errors, intervals, and p-values change.
On holzinger_swineford, with t6 regressed on t7 and t9, t9 made
missing at random given t7 for 110 of the 301 students
(set.seed(113)), and t13 as the auxiliary, the model had chi square
129.5 on 1 df and an implied covariance of 0 between t7 and t9, whose
correlation is .69. The t7 slope was 0.449 (SE 0.031) and is now 0.362
(SE 0.044), against 0.362 without the auxiliary and 0.317 on the
complete data, and the t9 slope moves from 0.096 to 0.143 (0.167 on
the complete data). The auxiliary example on the mlmr() page, where
t26_flags is missing for 156 students, had chi square 14.1 on 1 df;
its intercept moves from -0.703 to -0.436 and its t7 slope from 0.488
to 0.476. When the predictors are complete, the coefficients and their
standard errors were already right, since their maximum likelihood
estimates do not depend on the predictors' covariance there, but every
quantity that uses the model implied variance of the outcome was not.
For t6 on t7 and t9 with complete data, R squared was .471 and is now
.602, the value lm() gives; the standardized coefficients move from
.541 and .423 to .469 and .367, and the semi-partial R squared from
.141 and .086 to .106 and .065. The log likelihood moves from -4123.6
on 13 parameters to -4013.7 on 14, and AIC and BIC with it. The
likelihood ratio test of anova() for adding t9 in the first example
was chi square 8.62 and is now 13.72. In mlmr_mv() the first example
gives t6 a t7 slope of 0.362 rather than 0.450 and t26_flags a t9
slope of 0.104 rather than -0.050, and with complete data the R
squared of each outcome is now that of lm() (.602 and .474 for t6
and t8, where they were .471 and .379). The auxiliary simulation in
the mlmr vignette, whose predictors are complete, prints the same
values. With the model saturated, the three estimators of
fraction_missing_information() now agree on such fits as they do
without auxiliaries: in the first example the t7 slope's unstructured
and Hessian estimates were .016 and -.559, and all three are now .254.
wald_df = "missing_information" therefore gives that slope 221.5
degrees of freedom rather than 293.1.
mlmr() and mlmr_mv() refuse an auxiliary variable under listwise
or pairwise deletion. lavaan's listwise deletion drops every row
missing any variable of the model, the auxiliaries included, so
through 1.0.1 an incomplete auxiliary silently shrank the sample.
mlmr(t6_paragraph_comprehension ~ t7_sentence + t9_word_meaning, data = holzinger_swineford, missing = "listwise", auxiliary = "t26_flags")
fitted 145 of the 301 rows, with coefficients of 0.971, 0.313, and
0.178 where lm() gives 1.117, 0.317, and 0.167. An auxiliary carries
information only through the full information likelihood, so the
combination now stops and names the remedy (drop the auxiliary or set
missing = "fiml"), as reliability_alpha() and reliability_H()
stop. Under pairwise and available cases deletion (missing = "pairwise" or "available.cases") the moments are computed from each
pair of variables on its own. With the model saturated, the
coefficients then depend on the moments of the outcomes and predictors
alone, so an auxiliary can change nothing, while the result recorded
it as used. In mlmr(t6_paragraph_comprehension ~ t7_sentence + t9_word_meaning + t26_flags, data = holzinger_swineford, missing = "pairwise", auxiliary = "t13_straight_and_curved_capitals"), 1.0.1's
coefficients agreed with those of the fit without the auxiliary to
within 5e-6; its standard errors differed only because the auxiliary
also fixed the covariances among the predictors at zero (see the entry
above), so the t7 slope's was .0245 against .0355. That combination is
now refused the same way.
Under missing = "fiml" with auxiliary variables, a case whose
items are all missing is used when an auxiliary variable is observed
on it. reliability_alpha() and reliability_omega() set aside
every case with no observed item, with auxiliary variables or without.
The saturated correlates model estimates the items and the auxiliaries
jointly, so such a case informs the item moments through its
auxiliaries, and lavaan's own FIML fit of the model keeps it, as
cov_fiml() does. Setting it aside made the analysis conditional on
having an observed item, which depends on the auxiliaries when the
items go missing together at random given them, and the estimates were
then biased. On the nine tests of the three-factor model (visual
perception, cubes, lozenges, paragraph comprehension, sentence
completion, word meaning, addition, counting groups of dots, and
straight and curved capitals), with the whole block set to NA for 77
pupils at random given general information (set.seed(113), the rows
sample(301, 77, prob = plogis(-1.5 + 1.2 * z)) with z the
standardized t5_general_information, the auxiliary), N goes from
224 to 301, alpha from .610 to .613, model implied omega from .350 to
.358, and robust omega from .228 to .234. reliability_H() from data,
new in 1.1.0, uses such cases too. A result changes only when some
case has every item missing and an auxiliary observed, and N now
counts such cases. Two-stage estimation, new in 1.1.0, uses them as
well: its first stage reads their auxiliaries, and the coefficients
are computed on every case it reads, although lavaan's two-stage
estimator leaves such cases out of its N (see Two-Stage Maximum
Likelihood for Incomplete Data). Under multiple imputation, also new,
reliability_omega_categorical() imputes such a case from its
auxiliaries.
The FIML item covariance matrix behind analytic alpha and robust
omega is the EM estimate. Under missing = "fiml", analytic
reliability_alpha() and the observed denominator of
reliability_omega() apply their formulas to the maximum likelihood
covariance matrix of the items under the saturated model. Through
1.0.1 that matrix came from lavaan fitting the saturated model from
its default starting values, which on many items stops short of the
maximum: on all 26 Holzinger and Swineford tests, with the Pasteur
students missing the last two, it took 2484 iterations and left
covariances up to 0.0105 from the maximum likelihood values, the log
likelihood 3e-7 below its maximum. The matrix is now the EM estimate
cov_fiml() returns, the unrestricted moments lavaan computes for
every FIML fit to the same variables (the EM runs on the standardized
items, as in cov_fiml(), so the matrix does not depend on their
units, and agrees with lavaan's to about seven digits), and the
standard error of the "ml" interval is the delta method one from the
saturated model evaluated at that estimate, with no optimization run.
On the Holzinger and Swineford examples checked (6, 9, 23, and 26
tests, with the Pasteur students' missing tests or with values deleted
at random given general information, with and without an auxiliary
variable), alpha changes by at most 1.5e-6, robust omega by at most
5.8e-6, and the standard error by at most 4.4e-8; on the 26 tests
alpha is .8234684 rather than .8234689. The estimate no longer depends
on the interval asked for, the FIML alpha of complete data equals the
listwise one to machine precision (it differed by 2.3e-9 on nine
tests), and the computation is faster: on 23 incomplete tests about
three times for alpha with its interval and five to ten times for
robust omega. Two-stage analytic alpha, which uses the same matrix,
changes the same way, as does the fraction of missing information
ss_aipe_reliability_sensitivity() records for alpha. Model implied
alpha and model implied omega are unchanged.
Under FIML and two-stage estimation, a pair of variables never
observed together is refused by name. reliability_alpha() and
reliability_omega() now stop, as reliability_H() from data and
cov_fiml(), new in 1.1.0, do, naming each pair of items or auxiliary
variables that no case observes together and pointing to
missing_coverage(). Through 1.0.1 lavaan only warned, and the
covariance it returned for such a pair was arbitrary: on
t1_visual_perception, t2_cubes, t4_lozenges, and
t6_paragraph_comprehension, with t1_visual_perception deleted in
rows 1 to 150 and t2_cubes in rows 151 to 301,
reliability_alpha(missing = "fiml") returned 0.536 with a standard
error of 0 and the interval [0.536, 0.536]. In a bootstrap, a resample
left with such a pair, or a jackknife deletion behind the BCa
acceleration, is set aside with the message, under two-stage
estimation as under FIML. A pair observed together only where one of
them takes a single value, or by one row only, is refused the same way
(see the next entry).
Under FIML and two-stage estimation, a pair of variables observed
together only where one of them takes a single value is refused.
Rows in which one of two variables is constant say nothing about how
the two vary together, so the estimate of their covariance rests on
the form of the normal model, or is not determined at all.
reliability_alpha() and reliability_omega() now name such a pair
and stop, as they do for a pair never observed together, and so do
reliability_H() from data and cov_fiml(), new in 1.1.0. A pair
that only one row observes is refused the same way, and in a
bootstrap a resample that leaves such a pair is set aside.
t1_visual_perception, t2_cubes,
t4_lozenges, and t6_paragraph_comprehension, with
t1_visual_perception deleted in rows 91 to 100 and t2_cubes set
to 25 in rows 1 to 90, the rows that observe t1_visual_perception.
Under missing = "fiml", version 1.0.1 returned robust omega 1.06,
the analytic alpha 0.113 with no interval, and the model implied
omega 0.890 [0.281, 1].t2_cubes among the items. t2_cubes is now refused as constant in
the 90 complete rows (see the entry on an item that does not vary),
and the message says why FIML cannot help.reliability_omega_categorical() refuses fewer than three items.
With two ordered items the single-factor model is not identified: the
one polychoric correlation determines only the product of the two
loadings, while categorical omega depends on each. The value returned
before depended on the start values of the optimizer and on the order
of the columns, and its bootstrap interval reflected where the
optimizer landed. The function now stops with the message
reliability_omega() gives for two continuous items, and
reliability(), whose automatic routing sends two integer items to
categorical omega, lets the error through rather than substituting
another coefficient.
Categorical omega no longer drops an undetermined polychoric
correlation from its total variance. The unrestricted polychoric
correlations behind the observed denominator came from a second,
saturated fit (the approach MBESS also takes). When the data do not
determine a polychoric correlation, as when two items are skewed in
opposite directions and their bivariate table has empty cells, that
fit gives the correlation no weight and returns its starting value, 0,
so the pair was removed from the total variance without notice and the
coefficient was too high, by about .035 for each such pair. In samples
of 500 from the mixed-skew design of Green and Yang (2009), the help
page's example with N <- 500 and seeds 1 to 100 in place of 113, 45
of the 99 samples with a proper solution had at least one such pair.
Against the coefficient with the model implied correlation in the
pair, 1.0.1's was .030 to .045 too high with one undetermined pair (38
samples) and .065 to .088 with two whose loadings are identified (3
samples). The correlations now come from the single-factor fit itself,
which agrees with the saturated fit whenever every correlation is
determined, so every other result is unchanged, and the computation
needs one fit instead of two per bootstrap replication; that fit also
no longer computes a test statistic it never used, which changes no
number. Together the two cut the fitting time of a bootstrap
replication by more than half. When a correlation is not determined,
reliability_omega_categorical(denominator = "observed") stops with
an error naming the pair. When the correlations the data do determine
identify the single-factor loadings, it points to denominator = "model_implied", which does not need the pair. fa_confirmatory() reports that
factor's categorical omega as NA with a warning naming the pair. A
bootstrap meets such pairs far more often than the sample does, since
a resample often leaves out the few respondents who determine the
correlation (about half the replications on the help page's example,
whose sample has no such pair). Those replications are set aside,
counted in the new undetermined element of the bootstrap
attribute, and named in the printed line and the warning. The warning
says that the interval then conditions on the replications that redrew
the respondents who determine the correlation (on the help page's
example, 80, 95, and 97 of 200 in percentile bootstraps with seed 1,
2, and 3; with the improper ones, 80, 96, and 98 were set aside, close
to the half beyond which no interval is reported, as the entry on a
bootstrap of categorical omega that returns fewer than half
describes), and that denominator = "model_implied" sets aside only
the replications whose determined correlations leave the loadings
unidentified (see Either Variance in the Denominator of Every Omega).
This is a deliberate, loud change for those samples only: a number
that rested on an arbitrary 0 is replaced by an error.
An improper categorical solution is named. When the single-factor
fit returns a standardized loading above one (a negative residual
variance, a Heywood case), categorical omega is not defined, and the
function failed with the error "'corr' is not a correlation matrix"
from mvtnorm, under the model implied denominator only after a
message advising that denominator. It now stops with an error naming
each improper item and its loading, before anything is said about
undetermined pairs. A bootstrap replication with an improper solution
is set aside as a non-finite value and counted in the new improper
element of the bootstrap attribute (it was counted as an error,
with the message from mvtnorm), and the warning from lavaan about
the negative variance no longer repeats once per replication.
reliability_omega_categorical() says which sum its coefficient
describes. Categorical omega scores each item's categories 0, 1,
..., C - 1 in the order of its codes (Green & Yang, 2009, Eq. 11;
Kelley & Pornprasertmanit, 2016, Eq. 16), as MBESS and semTools do,
and the help page now says so. When an item's codes are not equally
spaced, or items step by different amounts, the sum of the codes as
given is a different composite, and a message names the items. In the
examples behind the help page its reliability was 0.009 to 0.012 lower
for codes -2, -1, 1, 2, and 0.034 lower for items coded 1, 3, 5 beside
items coded 1, 2, 3. The coefficient is unchanged.
Listwise deletion that leaves too little to analyze says so. With
no complete row, only one, or an item that takes a single value in the
complete rows, reliability_omega_categorical() printed lavaan's
table of variables and stopped with "The single-factor categorical CFA
model did not converge." It now stops before any fit, names the cause,
and offers missing = "pairwise" (when every pair of items is
observed together) or missing = "mi". No estimate changes.
The reliability family treats SPSS user-defined missing values as
missing. A column read by haven::read_sav(user_na = TRUE) keeps a
code declared missing in SPSS (999 for a test not given, 9 for a
refusal) as a value of class haven_labelled_spss, which SPSS and
haven's is.na() treat as missing. Through 1.0.1,
reliability_alpha(), reliability_omega(), and icc() analyzed it
as a score or rating. reliability_kr20() refused it as not 0/1, and
reliability_omega_categorical() counted it as a further category.
fleiss_kappa() summed it as a count. cohen_kappa() dropped such a
pair only when haven was loaded, and otherwise counted the code as a
further category. These functions and reliability() now set, as
reliability_H() from data, new in 1.1.0, does, the declared values
(na_values and na_range) to missing in every column before
anything is deleted or fitted, as read_sav() does by default. A
message names the columns and counts, and haven need not be loaded.
icc() and fleiss_kappa(), which do not accept missing values, then
refuse such a column as they refuse NA.
holzinger_swineford, t1_visual_perception,
t2_cubes, t3_paper_form_board, t4_lozenges, and
t25_paper_form_board_r, with the 156 missing scores of the last
recoded to 999 and declared missing (x <- holzinger_swineford$t25_paper_form_board_r; haven::labelled_spss(ifelse(is.na(x), 999, x), na_values = 999)).
Version 1.0.1 gave alpha 0.008 and omega 0.015; the functions now
give 0.687 and 0.730 under listwise deletion (missing = "listwise", the default through 1.0.x), the values for the same
data with NA.set.seed(1); t <- rnorm(60), then
round(3 + t + rnorm(60, sd = .5)) for each rater in turn), with
the first six ratings of the third declared missing as 9. icc()
gave ICC(2,1) 0.318, against 0.668 on the complete rows, and now
refuses the data. cohen_kappa() of the first and third raters gave
0.170 on 60 pairs and seven categories when haven was not loaded. It
now gives 0.196 on 54 pairs and six, as it did with haven loaded.read_sav()'s default, which already makes these
values NA, are unaffected.reliability_omega_categorical() reads a factor's labels as the
numbers they name. Through 1.0.1 a factor item passed only when its
labels were 1, 2, ..., K in order. A Likert factor labeled 0 to 4, or
one whose lowest category no respondent chose (levels "2" to "5"),
was refused with "column 1 contains non-integer values". The labels
are now read as numbers, and the item is analyzed as those numbers
under every treatment of missing. Listwise and pairwise deletion and
multiple imputation therefore rank its categories alike, and the
factor gives the result of its integer codes. For example, the items
of the help page's example recoded as factors labeled 0 to 4
(as.data.frame(lapply(as.data.frame(items - 1), factor))) were
refused by 1.0.1 and now give 0.720, the value of their codes. Two
kinds of factor are refused, with a message that names the column and
says what to do: one whose labels are words, and an ordered factor
whose levels are not in the order of the numbers they name.
A bootstrap of categorical omega reports no interval when fewer than
half of its replications return a value. Through 1.0.1
reliability_omega_categorical() formed its interval from any two
replications that returned a value. When most were set aside, the
rest were a minority of the resamples, selected by whether they
returned a value, and their spread described those resamples rather
than the sampling distribution of the coefficient.
NA in that case, under
every ci_method and every treatment of missing. A warning and
the printed line say why.set.seed(113); eta <- rnorm(60); y <- 1 * (outer(eta, rep(.7, 6)) + matrix(rnorm(360), 60) * sqrt(1 - .7^2) > 0); y[, 1] <- 0; y[1, 1] <- 1). A percentile bootstrap (B = 200,
seed = 113) set aside 106 replications in 1.0.1 and gave [0.660,
0.921] from the other 94. It now reports no interval.Listwise deletion that leaves fewer than two complete rows stops alpha, KR-20, and omega before anything is computed.
reliability_omega() (either denominator) and the model implied
reliability_alpha() passed the data to lavaan, which printed its
table of the variables and stopped on a variance it could not
compute, and the call reported a model that did not converge. The
analytic reliability_alpha() returned NA with a warning that NaNs
were produced, and reliability_kr20() returned NA for the estimate
and every limit with two such warnings. Example: the first 40 pupils
of holzinger_swineford on visual perception, cubes, lozenges,
paragraph comprehension, and sentence completion, with one score
deleted from every row (d <- holzinger_swineford[1:40, c("t1_visual_perception", "t2_cubes", "t4_lozenges", "t6_paragraph_comprehension", "t7_sentence")]; for (i in 1:40) d[i, (i - 1) %% 5 + 1] <- NA).missing = "fiml", as
reliability_omega_categorical() offers its own treatments for its
items. FIML is offered only when it can use the rows; otherwise the
message says why it cannot, as when a pair of items is never
observed together. An item that no row observes is named instead.
reliability_H() from data, new in 1.1.0, does the same.reliability_alpha() with missing = "fiml", which computes the same coefficient (alpha of 0/1 items is
KR-20) from the incomplete rows as well, when full information
maximum likelihood can use them (otherwise the message says why it
cannot). Example: the nine tests t1_visual_perception to
t9_word_meaning, each scored 1 above its median (as.integer(x > median(x))), with each of the 301 pupils missing one test in
rotation (pupil i misses test i %% 9 + 1). Version 1.0.1 returned
NA; the call the message offers gives 0.752, 95% CI [0.709,
0.795], against 0.756 for KR-20 of the complete data.An item that is not numeric, or that does not vary, is refused by
name. reliability_alpha(), reliability_kr20(), and
reliability_omega() now stop before computing anything when an item
takes a single value in the rows analyzed, and name it, as does
reliability_H() from data, new in 1.1.0. The rows analyzed are the
complete rows under listwise deletion, and the rows that observe the
item under FIML and two-stage estimation. A covariance matrix with a
zero variance on its diagonal is refused the same way.
holzinger_swineford with
t2_cubes set to 5, alpha of t1_visual_perception, t2_cubes,
t4_lozenges, and t6_paragraph_comprehension was 0.487, against
0.548 for the other three. On the first 20 pupils, with the nine
tests t1_visual_perception to t9_word_meaning each scored 1
above its 90th percentile among all 301 pupils (as.numeric(x > quantile(x, 0.9))), no pupil passes four of the tests, and KR-20
was 0.196, against 0.217 for the other five.missing = "fiml" (for
KR-20, reliability_alpha() with missing = "fiml") only when
FIML could use those rows, and otherwise says why it cannot. A
shared column name is a condition to fix first, rather than an
obstacle in the data; fitted coefficients refuse shared names
before checking the rows. FIML is not offered when the composite is
constant in the complete rows and they outnumber the items.stats::cov()'s "is.numeric(x) || is.logical(x) is not TRUE", and
FIML with the report about the saturated model. A logical item is
still read as 0 and 1.A bootstrap replication in which an item takes a single value is set aside, with the refusal. The rule above now holds in every bootstrap replication of every continuous coefficient, and in the jackknife behind the BCa acceleration. Categorical omega already set such replications aside, but passed them to lavaan first. It now refuses them by item name before fitting, without printing lavaan's variable table; their reason is counted as an error with the refusal rather than as an unnamed non-finite value. The same check applies to completed resamples after imputation.
holzinger_swineford, with the nine
tests t1_visual_perception to t9_word_meaning scored 1 above
their medians among all 301 pupils, and t20_deduction scored 1 at
the highest score among the 60 (one pupil). KR-20 is 0.689. Its
percentile interval (B = 1000, seed = 113) was [0.560, 0.779],
with 370 of the 1000 replications computed with t20_deduction
constant. It is now [0.564, 0.775], from the other 630, with the
caution.A composite that does not vary is refused. When the covariances
of the items cancel their variances, the composite has no variance and
no reliability. Ipsative items, whose scores sum to the same total in
every row, are the usual case. reliability_alpha(),
reliability_kr20(), and reliability_omega() now stop and say so.
t1_visual_perception, t2_cubes, and t4_lozenges,
each centered on the pupil's mean of the three (x - rowMeans(x)
for all 301 pupils). Alpha was returned as 1.42e16, with a standard
error of -1.42e15 and NaN limits.The coefficients fitted by the single-factor model need more
complete rows than items. Under listwise deletion, N complete rows
give a covariance matrix of rank at most N - 1. With no more complete
rows than items it is singular, and the maximum likelihood fit is not
defined. That fit lies behind robust and model implied omega and the
model implied alpha, and behind reliability_H() from data, new in
1.1.0.
t1_visual_perception, t2_cubes,
t4_lozenges, t6_paragraph_comprehension, and t7_sentence.missing = "fiml" when FIML could use the incomplete rows, and otherwise
says why it cannot, or that the model needs more cases.Full information maximum likelihood and two-stage estimation need more cases than variables. Both first estimate the saturated model over the items and any auxiliary variables.
reliability_alpha() and reliability_omega() now stop before any
fit and say so, as does reliability_H() from data. For the
analytic alpha, listwise deletion is offered only when it would
accept the complete rows, naming Feldt's interval when exactly two
are complete. Bootstrap and jackknife resamples obey the same rule.t1_visual_perception missing for pupil 4 and t2_cubes for
pupil 5. Version 1.0.1 reported for alpha that the saturated FIML
model did not converge, and for omega that the model did not
converge.At two complete rows, the intervals of alpha and KR-20 that are not
defined there are refused. Two rows give a covariance matrix of rank
one, in which every pair of items correlates +1 or -1, whatever the
items. Each interval is refused for its own reason, as "fisher"
already is below N = 4:
"ml" and "adf" standard errors, and those of their
_logistic variants, are zero for any data. The asymptotic variance
of van Zyl, Neudecker, and Nel vanishes at a matrix of rank one.
Each case's centered cross products are then proportional to the
covariance matrix, which alpha does not change with.holzinger_swineford (pupil ids 1 and
54) on t1_visual_perception, t2_cubes, t4_lozenges,
t6_paragraph_comprehension, and t7_sentence (alpha -0.475).
Version 1.0.1 returned:
"ml" and "adf" (a variance of zero, made negative by
rounding error);_logistic variants;bifactor_indices() puts residual covariances in its model implied
denominators and refuses correlated factors and multiple-group
fits.
Each model implied denominator is now the sum of the corresponding
block of the item correlations the fitted model reproduces. For every
orthogonal fit without residual covariances that equals the earlier
formula to machine precision, so those results are unchanged; for a
fit that specifies residual covariances the earlier formula left them
out, and a positive one made omega, omega_H, and omega_HS too high
(omega_H .646 against .635 on a test sample with one cross-group
residual covariance). A fit that correlates any two of the general
and group factors, which the help page always excluded, is now
refused with an error naming the pair. A multiple-group fit took its
loadings from the first group and its residual variances from the
last, so its table described neither group: for nine items with
general and group loadings of .5 in two groups of 200 (set.seed(11);
the fixture of the package's tests), the general factor's omega was
.865 and its omega_H .623, while the model fitted to the first group
alone gives .862 and .621, and to the second group alone .874 and
.674. Such a fit is now refused (see Calls That Are Now Refused); fit
the model to each group's data to obtain that group's indices.
bifactor_indices() refuses an item on two group factors. The
help page, the argument, and the error message all required each item
to load on exactly one group factor, but only an item on none was
refused. An item on two took the loading and the membership of the
group factor listed last, so its other loading dropped out of ECV,
omega, omega_HS, H, and PUC, and the result depended on the order of
the model syntax. On the 15-test Holzinger and Swineford bifactor
model with lozenges (t4) also on the verbal factor, the spatial row
described three of its four tests (omega_HS .188, H .254), PUC counted
lozenges as a verbal test, and ECV was .441, where every loading
counted gives .432. Such a fit is now refused, with the items and
factors named. A loading fixed or constrained to zero no longer
counts, so a model written with 0*x3, or with a*x3 and a == 0,
on a second group factor gets the indices of the same model written
without that loading; through 1.0.1 the zero loading took the item
from its real factor. An item listed under one group factor keeps that
loading whatever its value, as before.
bifactor_indices() refuses a fit with one group factor. DMAR
1.0.0 refused a fit with no group factor ("No group factors found; a
bifactor model needs a general factor plus two or more group
factors.") and required every item to load on a group factor ("Every
item must load on exactly one group factor."), so it described a
general factor beside a single group factor when every item was listed
under that factor (some of them with loadings fixed at zero, and with
general named, since both factors then listed every item), giving a
table whose indices divide the common variance between those two
factors alone. Such a fit is not a bifactor model with subscales, and
it is now refused, with a message that names the group factor and any
items on no group factor, as fa_bifactor() and
ci_bifactor_indices(), new in 1.1.0, refuse it. There is no warning
period. The message for an item on two group factors now says that an
item loads on one group factor at most, since an item on none is
accepted when there are two or more group factors.
bifactor_indices() reports H as NA for a factor with a
standardized loading at or beyond one. The index counts 1 - l^2 as
each item's error, so such a loading leaves an item an error variance
that is not positive and the index is not defined, as
reliability_H() treats an improper solution. Through 1.0.1 the
loading was left out, and H described the factor's other items, with
only the general Heywood warning to show it. On nine simulated items
(set.seed(7); n <- 150; lg <- c(.9, rep(.6, 8)); ls <- c(.42, rep(.5, 8)); g <- rnorm(n); f <- list(rnorm(n), rnorm(n), rnorm(n)), item i
equal to lg[i] * g + ls[i] * f[[ceiling(i / 3)]] + sqrt(1 - lg[i]^2 - ls[i]^2) * rnorm(n)), fit as a bifactor model with three orthogonal
group factors of three items each and std.lv = TRUE, the first
general loading is 1.06, and the general factor's H was .841, the H of
the other eight loadings. That value was not a limit of nearby
solutions: as that loading rose to .9999, H rose to .9998, and past
one it fell to .841. It is now NA, and every other cell of the table
is unchanged. In ci_bifactor_indices(), new in 1.1.0, such a
replication yields no coefficient and is set aside under either
improper rule.
bifactor_indices() says which H it reports and what its group rows
mean, and gives the maximal reliability of each factor. The help
page called H "the construct reliability (maximal reliability) of
Hancock and Mueller (2001)". The column is the index Rodriguez, Reise,
and Haviland (2016) report: Hancock and Mueller's closed form on one
factor's standardized loadings, which treats the variance an item
shares with the other factors as uncorrelated error. In a bifactor
model that variance is correlated, so the index is not the maximal
reliability of the factor under the fitted model, and it can lie on
either side of it. In the help page's example the general factor's
index is 0.769 and its maximal reliability 0.678, barely above omega_H
(0.673), so no weighting of the items reaches the index; on the
15-test Holzinger and Swineford model they are .789 and .681; and with
group loadings of mixed sign the maximal reliability is the larger. On
a group factor's row the maximal reliability is the larger when the
items of the other subscales measure a strong general factor well,
since the optimal weights remove it through them: with nine items in
three subscales, general loadings .7, and group loadings .3, a group
factor's index is 0.229 and its maximal reliability 0.279. That
maximal reliability belongs to the group factor's regression score
from all nine items, which weights the other subscales' items
negatively. No score of the subscale's own items reaches it, and their
best weighting reaches only the omega_HS of 0.125. The maximal
reliability, l' R^-1 l with R the item correlations the model
reproduces, is the square of the regression score determinacy that
factor_scores() reports. It is now returned in the new attribute
maximal_reliability (NA for an improper solution), printed under the
table, and given an interval by ci_bifactor_indices(); its sample
value, like the index's, runs upward, as the help page says. The ECV
on a group factor's row, which no page defined, is the general
factor's share of the common variance of that factor's items. One
minus it is the group factor's own share, so .85 beside an omega_HS of
.08 says the subscale is mostly general. The omega there is the
reliability of the subscale's total score. Both are now defined on the
help page and in a line under the printed table, as they are for
ci_bifactor_indices(). No value in the table changes.
reliability_H() returns the reliability family table for every
call; the table of 1.0.0 is gone. In 1.0.0 the function took
loadings, se_loadings, and conf_level and returned the row
reliability_H, with ci_lower, ci_upper, and var_H when
se_loadings was supplied. Such a call now returns the table every
other input returns (rows estimate, se, ci_lower,
ci_upper, conf_level, N, N_complete, n_items), with no
warning period. With se_loadings it has the same estimate and the
same logit interval, var_H being the square of se. Without
se_loadings it has the estimate, with se and the limits NA,
because standardized loadings alone carry nothing an interval can be
formed from. Code that read x$value[x$term == "reliability_H"] reads
x$value[x$term == "estimate"]. factor, group, missing, and
aux do not apply to loadings and are refused. Every interval built
from se_loadings warns, with class dmar_H_marginal_se, that it
ignores the covariances among the loading estimates. The help page now
attributes the logit interval to
Browne (1982) rather than Browne (1968). It replaces the unverified
statement that Hancock and Mueller proved H >= omega >= alpha, as does
the function interactions vignette, which said that H is uniformly at
least omega and omega at least alpha, with the bound that holds by
construction: H is at least the model implied omega of the same
single-factor fit; robust omega need not lie below it. Alpha is at
most omega in the population for congeneric items with uncorrelated
errors.
The bootstrap attribute names two causes. Every reliability
coefficient's bootstrap attribute, and its jackknife element,
gains undetermined and improper: the replications among the
non-finite ones that were set aside because the resample left a
polychoric correlation undetermined or gave an improper solution
(zero outside categorical omega). The reasons vector keeps its
1.0.1 shape.
A bootstrap whose returned replicates are all equal returns that
value at both limits. When every item is an exact linear function of
the same variable, every resample gives the same coefficient. The
percentile and BCa intervals of reliability_alpha() and
reliability_kr20() stopped with "argument is of length zero" (after
boot printed "All values of t are equal"), or with "missing value
where TRUE/FALSE needed" when some replications were set aside. They
now return the common value at both limits, as bootstrap_se already
did (and so does bootstrap_se_logistic, whose limits at a value of
one, identical items, were NA, or one where rounding left the
replicates just below it, since the logit is infinite there), with a
warning that the bootstrap distribution is a single point. A BCa
request falls back to that percentile interval and says why in
ci_method. The engine also warns whenever fewer than two
replications return a value, not only when some were set aside.
B is validated by reliability_alpha(), reliability_omega(),
reliability_kr20(), and reliability_omega_categorical(). When a
bootstrap is requested, B must be a single whole number of at least
2, the fewest replications from which a standard deviation or a pair
of quantiles can be formed, as reliability_H() and
ci_bifactor_indices() require. Through 1.0.1 B = 0 or 1 returned
NA limits without a warning (categorical omega did not check B at
all, and B = 1 returned its estimate with NA limits), and a
fractional, negative, or missing B failed inside boot with an error
that did not name it. Each function now stops before anything is
computed, with one message. No estimate or interval changes.
A DMAR table prints to three decimal places, not to three
significant figures. A value in a DMAR table that is not a whole
number printed to digits significant figures, 3 by default, and
format() dropped its trailing zeros and wrote a small value in
scientific notation: 13.018 printed as 13, 0.5596 as 0.56, 233.4107 as
233, 0.041234 as 0.0412, 1.5 as 1.5, and 0.00001234 as 1.23e-05. It
now prints to digits decimal places with its trailing zeros, never in
scientific notation: 13.018, 0.560, 233.411, 0.041, 1.500, and 0.000.
A value that rounds to zero prints as 0.000 whatever its sign. Whole
numbers, such as degrees of freedom and sample sizes, print without a
decimal part as before. A measured quantity that lands on a whole
number, at the bound of its range or where it is fixed, printed as a
whole number too, and where the table shows that it is measured it now
prints to the decimal places of the values beside it (?dmar_tbl
gives the rule):
_lower or _upper, such as an
equivalence margin: a lower limit of exactly 0, which printed as 0,
prints as 0.000 beside an upper limit of 0.083, and a factor
variance fixed at 1 prints as 1.000 beside its interval;pct_ rows
of a sensitivity analysis among them), a mean, median, standard
deviation, or other moment, a test statistic, a standard error, a
noncentrality parameter, the delta and the weights of a model
comparison, and a heterogeneity index, wherever they appear: a CFI
of 1 prints as 1.000 beside a TLI of 1.012, a power of 1 as 1.000
beside a power of 0.840, and a standard deviation of the interval
widths of 0 as 0.000 beside their mean of 1.176;dmacs(), where an intercept given as 2
prints as 2.000 beside one of 2.300.Elsewhere a whole value prints as a count, as before, since nothing in
a long table says which of its other whole numbers are counts: a
planning value given as 15 prints as 15. p-values (digits_p, four
decimal places, with the "< 0.0001" floor) and the information criteria
a table names in its fixed_terms (digits_fixed, three decimal
places) print as before. print(), format(), knit_print(), and
as_kable() show the same table, and the displays that lay out cells
of their own (those of fa_exploratory(), factor_count(), and
factor_scores()) and the lines beneath a table print measured values
the same way: a CFI of 1 prints as 1.000 in the display of
factor_count() as it does in the table of sem_fit_measures(), and a
congruence coefficient of 1 as 1.000 in the table of
factor_congruence() and in the matrix beneath it.
For code written against 1.0.x:
digits argument of print() and format() for a DMAR table,
and options(dmar.digits = ), now count decimal places, not
significant figures: print(x, digits = 5) showed five significant
figures and now shows five decimal places, in the table and in the
numbers a display writes beneath it. digits must be a whole number
of 0 or more; a fractional or negative value, which signif()
accepted, stops with an error that says so.format(x) returns the new display strings.x$value, the other columns, tidy(), and
glance() return the numbers at full precision, as before; read the
column, or print with more decimal places (print(x, digits = 8)),
to see more of a number than the table shows.results_sentence() keeps its own digits, two decimal places by
default, which options(dmar.digits = ) does not change. It now
writes a value that rounds to zero without a minus sign, as the table
does: a lower limit of -0.004 reads "95% CI [0.00, 0.55]", where it
read "[-0.00, 0.55]".A knitted result table carries every line the console prints.
knit_print() and as_kable() wrote only the sum-of-squares type and
the confidence level, so a knitted omega lost its denominator and
residual lines and a knitted bootstrap interval its replication count
(missing since 1.0.1). The tables of sem_fit_measures(),
equivalence_rmsea(), sem_residual_fit(),
sem_information_criteria(), compare_sem(),
fraction_missing_information(), and factor_scores() print lines of
their own with the table (which version of a fit measure was reported
and why, the sample size convention, the verdict of a test, what a
model comparison checked, how a fraction of missing information was
estimated), and a sample size plan for incomplete data prints the
missing data it assumed. The console print and the knitr methods now
read one footer, which every class extends through an internal
generic; a knitted table sets each footer line as a paragraph beneath
it (two lines no longer run together), and as_kable() joins them
into the caption. A knitted fraction_missing_information() table
leaves out the columns the console leaves out, and a knitted
factor_scores() result is the summary the console prints (the first
rows of each factor, the reasons values are missing or borrowed, the
determinacy of each missing data pattern, how to read the intervals)
rather than every row. The console's confidence level is formatted by
the same helper as the knitted one, which prints every conventional
level as before, and a blank line now separates it from the lines
after it. The lines above a fa_confirmatory() table (the measurement structure
each factor's constraints imply, also for cfa_1() and cfa_2()) and
above a mediation_mbco() table (the effects and N, the groups, the
full model's deviance, AIC, BIC, and R squared, how to read each
p_value), which print methods of their own wrote and knit_print()
never called, are set as paragraphs before a knitted table and first
in the as_kable() caption. No stored value changes.
as_kable() and knit_print() escape the lines they write for
LaTeX and HTML. knitr escapes a table's cells and column names but
writes a caption as it is given, and as_kable() passed the table's
lines to it without escaping them. With format = "latex", or the
default format in an .Rnw document, the "%" of "Confidence level:
95%" started a LaTeX comment that swallowed the caption's closing
brace: a LaTeX document did not compile, and an R Markdown PDF printed
the table's LaTeX source as text. An underscore in an item name (the
residual correlation line of an omega) or an ampersand in a citation
(the pairwise deletion line) stopped the compilation as well. The
caption is now escaped for the format knitr renders, and
knit_print() writes its lines in that format's markup: escaped
paragraphs and a verbatim block for LaTeX, where a "%" had silently
cut the rest of its line, and <p> and <pre> elements for HTML.
Markdown output, the default in R Markdown, is unchanged. A caption
passed through ..., which R refused as an argument matched twice, is
refused with a message that says how to caption the table otherwise.
Two DMAR tables knitted in one chunk stay apart. knit_print()
wrote its output with no blank line before or after it, and knitr
writes a chunk's outputs with nothing between them. In a chunk with
echo = FALSE that printed two tables, the second table's header row
was appended to the first table's last footer line ("Confidence level:
95%|term |value |"). Pandoc then read that line as the second table's
header, in R Markdown's HTML and PDF output alike: the footer became a
header cell, the second table lost its last column name, and the first
table lost its footer paragraph. The output is now set off by a blank
line on each side, as knitr sets off a kable of its own.
results_sentence() reads an interval held as ci_lower and
ci_upper, names the estimate the interval belongs to, and reports
the interval's own confidence level. In DMAR 1.0.x it read an interval
only from rows or columns then named lower_limit and upper_limit,
and told every table that held one as ci_lower and ci_upper (a row of contrast_test(),
adjusted_means(), content_validity_index(), fa_confirmatory(), or the
coefficient table of mlmr() or mlmr_mv(), and the tidy() results
that hold their limits as ci_lower and ci_upper) that it "carries
no confidence interval". Every table now names its limits ci_lower
and ci_upper (see Breaking Changes), so the rows of
common_method_adjusted_r(), new in 1.1.0, are read as well, from the
columns of a one-row table, a row beside a generic estimate column
is labeled by its term (so results_sentence(tidy(ci_smd(...)))
writes what results_sentence(ci_smd(...)) writes), and a table with
neither pair is told what was looked for. Two earlier defects change
sentences it wrote before. The estimate of a wide table was its first
numeric column other than the limits, N, and the p-value, so the
first row of the ?responder_analysis example read "n = 60.00, 95% CI
[0.38, 0.62]" and the second row of simple_effects_AB(aov(len ~ supp * factor(dose), data = ToothGrowth)) "F_value = 13.33, 95% CI
[0.04, 0.35], p = 0.0006"; the estimate is now the estimate or
value column when there is one and otherwise the first numeric
column that is not a limit, standard error, test statistic, degrees of
freedom, p-value, confidence level, or count ("estimate = 0.50" and
"eta_squared_partial = 0.20" there). And the confidence level was read
from the table's attribute alone, which the reliability family does
not set, so a 90% interval for omega was written "95% CI"; it is now
read from the attribute, the conf_level row, or the conf_level
column. A tidy() row that the default method builds from a wide
table was labeled "estimate" through 1.0.1 and is now labeled with the
quantity it holds (see the entry on the default tidy() below). Every
other change to a sentence is one of the entries that follow.
Every table that forms a confidence interval records its level, and
results_sentence() states no level a table does not record.
results_sentence() took 95% when a table recorded no level, and many
recorded none:
ci_eta_squared() and ci_omega_squared() (and
ci_eta_squared_partial(), which this release removes);vargha_delaney_A() and pairwise_within();alpha_lower and alpha_upper
(ci_c(), ci_cv(), ci_mahalanobis(), ci_r(), ci_R(),
ci_rc(), ci_reg_coef(), ci_rmsea(), ci_sc(), ci_sm(),
ci_smd(), ci_smd_c(), ci_src());tidy() and glance() results of every table the default
methods handle.A 90% interval was therefore written "95% CI". With
fit <- aov(len ~ supp * factor(dose), data = ToothGrowth),
results_sentence(ci_eta_squared(fit, conf_level = 0.90)[1, ], digits = 3) gave "eta_squared = 0.224, 95% CI [0.073, 0.350]", where the 95%
interval is [0.053, 0.377]. With
set.seed(1); x <- rnorm(40, 0.5); y <- rnorm(40),
results_sentence(vargha_delaney_A(x, y, conf_level = 0.90), digits = 3) gave "A = 0.674, 95% CI [0.565, 0.767], p = 0.0051", where the 95%
interval is [0.543, 0.782].
Each of these tables now records its level: conf_level, or
1 - alpha_lower - alpha_upper when the tail areas are given, as
ci_R2(), ci_pvaf(), ci_snr(), and ci_srsnr() already did.
vargha_delaney_A(), pairwise_within(), and
the intervals formed from tail areas print that level beneath the
table, which they did not.vargha_delaney_A() and pairwise_within() return DMAR tables and
print as the rest do; their columns are unchanged.
vargha_delaney_A() gains tidy() and glance() methods whose row
is named "A".pairwise_within(bonferroni_ci = TRUE) records conf_level, the
level its Bonferroni intervals hold jointly, as
contrast_test(adjust = "bonferroni") and the post hoc intervals
record theirs. The line beneath the table of both functions names
the adjustment and the level of each interval: "Confidence level:
95%, simultaneous across the 3 comparisons (Bonferroni: each
interval at 98.33%)". With set.seed(1); Y <- matrix(rnorm(60), 20) + rep(c(0, 0.3, 0.6), each = 20), the second
row of pairwise_within(Y, bonferroni_ci = TRUE, conf_level = 0.90), whose interval [0.02, 1.07] is the 96.67% interval of that
comparison, was written "95% CI"; it is written "90% CI".ci_eta_squared_generalized() recorded the default 0.95 beside the
interval that alpha_lower = 0.05 and alpha_upper = 0.05 define;
it records 0.90.tidy() of a wide table with an interval adds a
conf_level column (NA when no interval was formed). The default
tidy() of a long table with an interval adds a conf_level column
that holds the level on the rows of the interval and NA on every
other row. Those rows are its estimate and its two limits, or an
estimate's <estimate>_ci_lower and <estimate>_ci_upper rows at
that interval's own level. The rows therefore carry their level when
tidy() results are stacked with rbind(): the rows of
tidy(ci_proportion(30, 60, conf_level = 0.99)) stacked under
tidy(ci_proportion(12, 40, conf_level = 0.90)) give "proportion =
0.50, 99% CI [0.34, 0.66]".glance() of a long table with an interval adds
estimand and conf_level columns (see the next entry):
results_sentence(glance(ci_proportion(30, 100, conf_level = 0.90))) gives "proportion = 0.30, 90% CI [0.23, 0.38]".results_sentence() reads for the first time in 1.1.0
carry their level as well. The coefficient tables of mlmr() and
mlmr_mv() record it. The tidy() results of the reliability
family, fa_confirmatory(), content_validity_index(), mediation_mbco(),
fraction_missing_information(), mlmr(), and mlmr_mv() (with
conf.int = TRUE) carry a conf_level column. For m <- mlmr(t6_paragraph_comprehension ~ t7_sentence + t9_word_meaning, data = holzinger_swineford, conf_level = 0.90),
results_sentence(m$coef_table[2, ]) reads "t7_sentence = 0.32, 90%
CI [0.26, 0.38], p < 0.0001".results_sentence() now refuses a table that records no level, such
as one assembled by hand, unless its new argument conf_level gives
the level. It also refuses a conf_level that differs from the level
the table records. No interval changes.
results_sentence() reports the estimate a long table's interval
belongs to. In a table that holds its interval in ci_lower and
ci_upper rows, the estimate was the first other row, and two
tables list another quantity first.
correction_for_attenuation(r = .3, reliability_x = .8, reliability_y = .7, N = 200) gave "correlation_observed = 0.30, 95%
CI [0.22, 0.56]": the observed correlation beside the interval of
the corrected one, .401.proportion_of_superiority(smd = .5, n_1 = 50, n_2 = 50) gave "smd
= 0.50, 95% CI [0.54, 0.82]", an interval that does not contain the
estimate it names.The two tables now name the row their interval belongs to in a
ci_estimate attribute, which results_sentence() reads:
"correlation_corrected = 0.40, 95% CI [0.22, 0.56]" and
"proportion_of_superiority = 0.69, 95% CI [0.54, 0.82]". Their rows
are unchanged. The tidy() of a long table names the estimate in its
rows, through its conf_level column, and its glance() names the
column that holds the estimate in an estimand column, so the
sentence of either is the table's own. Through 1.0.1 the glance()
sentence of the attenuation table also named the observed correlation.
results_sentence() reports the TOST p-value of an equivalence
test. It appended the first p-value row of a long table. The tables
of equivalence_r(), equivalence_smd(), and equivalence_c() list
the p-values of the two one-sided tests before the TOST p-value, so a
result that is not equivalent was written with the p-value of one
one-sided test.
equivalence_r(r = .15, N = 300, cor_lower = .2, cor_upper = .2),
whose TOST p-value is .187 (not equivalent), gave "r = 0.15, 90% CI
[0.06, 0.24], p = < 0.0001" (the p-value of the lower test, 5e-10).
It now gives "r = 0.15, 90% CI [0.06, 0.24], p = 0.1870".equivalence_smd(smd = .3, n_1 = 100, n_2 = 100, smd_lower = .5, smd_upper = .5) the sentence gives p = 0.0806 where it gave
the lower test's 3e-08.?equivalence_c it gives p = 0.0393 where it gave
the lower test's 3.9e-07.The three tables name p_tost in a p_sentence attribute, which
tidy() and glance() carry, so their sentences give the same
p-value. The meta-analysis tables, whose first p-value row is the
pooled p-value, are unchanged.
results_sentence() writes "p < 0.0001", not "p = < 0.0001". A
p-value below the floor of format_p() was appended as ", p = " and
then "< 0.0001". The sentence now writes the bound:
summary_t_test(mean_1 = 2, sd_1 = 1, n_1 = 30, mean_2 = 0, sd_2 = 1, n_2 = 30) gives "mean_difference = 2.00, 95% CI [1.48, 2.52], p <
0.0001". Larger p-values are written as before.
The default tidy() of a wide table with an interval takes as its
estimate the column the interval belongs to, and its sentence names
that quantity. It took the first numeric column. A row of
simple_effects_AB() therefore gave its F statistic beside the
limits for partial eta squared: for simple_effects_AB(aov(len ~ supp * factor(dose), data = ToothGrowth)), results_sentence() on
its second tidy() row wrote "estimate = 13.33, 95% CI [0.04, 0.35],
p = 0.0006", where the estimate is 0.20. A table whose estimate
column follows a count (responder_analysis()) lost the count.
In a table that carries an interval, the estimate is now the table's
own estimate (or value) column. Otherwise it is the first numeric
column that is not a limit, standard error, test statistic, degrees of
freedom, p-value, confidence level, or count, the rule
results_sentence() follows. The other numeric columns follow under
their own names (the F statistic as F_value, the count as n).
When the estimate comes from a column with a name of its own, an
estimand column after estimate keeps the name, and
results_sentence() labels the row with it; otherwise it labels the
row with its term. The tidy() row's sentence is then the table's
own. content_validity_index() also gains the estimand column
"i_cvi": content_validity_index(matrix(c(4, 4, 3, 4, 3, 1, 4, 3), nrow = 2), conf_level = .90) gives "i_cvi = 1.00, 90% CI [0.47,
1.00]" on its first raw or tidy row, where 1.0.1 refused both as
carrying no interval. A tidy FMI row labels its estimate fmi rather
than the parameter's name. For the existing examples:
ci_mahalanobis(D2 = .5, n_1 = 40, n_2 = 40, p = 3, conf_level = .9);kappa for fleiss_kappa() and cohen_kappa(), mean_difference
for pairwise_within(), and ave for
average_variance_extracted();term for icc() ("ICC(2,1)"), responder_analysis()
("control"), and adjusted_means() (the group).1.0.1 labeled the others "estimate"; it wrote no sentence for the
tidy() rows of average_variance_extracted() and
adjusted_means(), which hold their limits as ci_lower and
ci_upper, and pairwise_within() had no tidy() method. The term
column is unchanged. A table without an interval is tidied as before.
The intervals of ci_eta_squared() for the effects of a fit with an
Error() term are converted with the size of each effect's stratum.
DMAR 1.0.x (in ci_eta_squared() and in ci_eta_squared_partial(),
which this release removes) converted the noncentrality
limits of every effect of an aovlist fit into limits for partial eta
squared with the total number of rows, lambda / (lambda + N). Within
subjects that put the interval on a smaller quantity than the one the
partial eta squared beside it estimates: for the three IQ occasions of
pygmalion analyzed in vignette("DMAR"), 930 rows against
F(2, 618), the 95% interval was [0.111, 0.194] and lay below the
estimate of 0.212. Each effect is now converted with the size of its
own stratum. Within subjects that size is the stratum's total degrees
of freedom, here 2 + 618 = 620, which gives [0.157, 0.265]; the
within-subjects example of ?ci_eta_squared goes from [0.015, 0.322]
to [0.023, 0.417]. In the between-subjects stratum of a mixed design it
is the number of subjects: with the expectancy treatment added to the
pygmalion analysis as a between-subjects factor, the upper limit for
treatment, converted with the 310 children rather than the 930 rows,
goes from 0.018 to 0.052. Fits without an Error() term convert with
the total sample size and the raw-argument interface with the N it is
given, as before, and their results are unchanged; for an effect tested
between subjects the total sample size makes the interval exact for the
population partial eta squared that Steiger (2004), Kelley (2007), and
Smithson (2001, 2003) define. Smithson's conversion with the effect and
error degrees of freedom plus one, which Fidler and Thompson (2001) also
used, agrees with that definition only in a one-way design. The help of
ci_eta_squared() gives the reasons and the literature, which does not
settle the within-subjects case. The raw-argument interface now accepts
an N as small as df_effect + df_error, the size of a within-subjects
stratum that tests one effect, which DMAR 1.0.x refused. It does not
warn when N is the number of rows of a within-subjects design, since
it cannot tell that number from the total sample size of a
between-subjects factorial design.
The parametric interval of ci_eta_squared_generalized() for the
effects of a fit with an Error() term is converted with the size of
each effect's stratum. DMAR 1.0.x converted the noncentrality limits
of every effect of an aovlist fit with the total number of rows, and
within subjects the interval was for a smaller quantity than the one
the estimate estimates: for the three IQ occasions of pygmalion, the
95% interval was [0.019, 0.037] against an estimate of 0.041. Each
effect is now converted with the size of its stratum, as
ci_eta_squared() converts it (the entry above), which gives
[0.029, 0.054]. With that size the conversion maps each limit to the
value of the population generalized eta squared of Olejnik and Algina
(2003) when the ratio of the other sums of squares in its denominator
to the effect's error sum of squares is at the value it converges to.
In the between-subjects stratum of a mixed design the size is the
number of subjects: with the expectancy treatment added to the
pygmalion analysis between subjects, the upper limit for treatment
goes from 0.015 to 0.044. The within-subjects example of
?ci_eta_squared_generalized goes from [0.005, 0.145] to
[0.008, 0.203]. Fits without an Error() term are converted with the
number of observations, as before, and their intervals are unchanged.
The help now sets out the quantity the conversion is made for and says
that the interval cannot be exact when anything measured, or another
stratum's residual, enters the denominator: generalized eta squared
then depends on a second parameter besides the noncentrality
parameter, which alone determines the distribution of the effect's
F, and the interval takes that parameter's sample value as known.
The F interface now accepts an N as small as
df_effect + df_error, the size of a within-subjects stratum that
tests one effect, which DMAR 1.0.x refused.
eta_squared_generalized() and ci_eta_squared_generalized() stop
when an effect of a fit with an Error() term is tested in more than
one stratum. When observations are missing within subjects, or a
covariate varies within them, aov() divides the effect's sum of
squares among strata and tests the effect in each of them. DMAR 1.0.x
then returned a row for every stratum each time the effect's name came
up, each computed from the effect's share in the first stratum: for 20
subjects in two groups observed three times, with four observations
missing, aov(y ~ group * time + Error(subject / time)) gave nine rows
for its five tests, time and group:time four times each, and the
parametric interval stopped with an error from inside the noncentral
F computation ("the condition has length > 1"). No one stratum's
share is the effect's sum of squares, and Olejnik and Algina (2003)
define the estimate for balanced designs, so both functions now stop
and name the effects and strata involved. A fit in which every subject
is observed equally often in every within-subjects condition, and
every covariate is constant within subjects, tests each effect in one
stratum, whatever the sizes of the between-subjects groups, and its
results are unchanged.
ci_eta_squared_generalized() gives the parametric interval from
sums of squares, and eta_squared_generalized() accepts the degrees
of freedom that come with them. The interval needs df_effect,
df_error, and N beside the sums of squares. Without them DMAR 1.0.x
stopped and asked for them; with them it stopped with "'F_effect' must
be a single non-negative number", because eta_squared_generalized(),
which computes the estimate, then read the call as the F interface
as well; called directly with the same arguments, it stopped the same
way. The sums of squares never gave an interval, and with the degrees
of freedom present they gave no estimate either, whatever the method.
Both functions now take df_effect and df_error with the sums of
squares and read only F_effect, F_observed, and df_observed as
the F interface. The estimate is computed from the sums of squares
alone, which need no degrees of freedom, and the interval from the F
they imply, (SS_effect / df_effect) / (SS_error / df_error),
with N checked as the F interface checks it. That F is the
effect's test when SS_error is the sum of squares of the effect's
error term: the residual in a design without an Error() term, and
otherwise the residual of the stratum in which the effect is tested,
with the residuals of the other strata in SS_observed, beside the
measured sources (Olejnik & Algina, 2003). The help of both
functions now says so, and that each F of the F interface divides
by the effect's error mean square. For the one-way analysis of IQ gain
by grade in pygmalion, the sums of squares of the anova() table,
with N = 310, give 0.118 [0.048, 0.176], the row of the fitted model
and of the F interface. Given no fitted model and both interfaces,
ci_eta_squared_generalized() answers, as DMAR 1.0.x did, when they
describe the same test: the F that the sums of squares imply for the
effect, and the ratio of SS_observed to SS_error, agree with the
F interface's to a relative tolerance of 1e-6, the tolerance
eta_squared_generalized() applies to the two estimates, and the
interval is formed from the F-values. When
they disagree it now stops, whatever the method, and so it does when
only part of the F interface comes with the sums of squares; a
fitted model still sets every raw argument aside with a warning, and
eta_squared_generalized(), which computes no interval, still accepts
the two interfaces together when they give the same estimate. DMAR
1.0.x accepted both interfaces whenever they gave the same estimate
and built the interval from the F-values, but the two can agree on
the estimate and still imply different F statistics for the
effect. Sums of squares of 100 for the effect, 70
for the measured sources, and 200 for the error, on 2 and 50 degrees
of freedom, imply an F of 12.5; an F of 25 for the effect and of
85 for a measured source on 1 degree of freedom give the same
estimate, 0.270. With N = 53 the sums of squares give the interval
[0.089, 0.413] and the F-values [0.127, 0.381], which DMAR 1.0.x
reported for the two together. Both functions now also refuse a sum
of squares, an F-value, or a number of degrees of freedom that is
missing or infinite, and ci_eta_squared_generalized() an infinite
N. DMAR 1.0.x stopped on most missing values with R's "missing value
where TRUE/FALSE needed", returned an estimate of 0 or NaN for an
infinite value, and gave the F interface the interval [0, 0] for an
infinite N.
The partial omega squared of each effect of a fit with an Error()
term is computed with the size of its stratum, and the stratum is
named. DMAR 1.0.x (omega_squared_partial(), which this release
removes; omega_squared(fit, estimand = "partial") is its call) computed
df_effect (F - 1) / (df_effect (F - 1) + N) with the total number of
rows: for the three IQ occasions of pygmalion, 930 rows against
F(2, 618), it gave 0.150, below the partial eta squared of 0.212 and
its interval [0.157, 0.265]. With the stratum's 2 + 618 = 620 the
estimate is 0.209, an estimate of the quantity whose interval
ci_eta_squared(estimand = "partial") gives; for treatment in the mixed
design of
the entry above, computed with the 310 children, it goes from 0.004 to
0.012. The value with the number of rows is the
partial omega squared that Olejnik and Algina (2003) compute for a
repeated measures factor, and the raw-argument interface given that
number as N still returns it. The table of an aovlist fit gains a
stratum column after the estimate, as in eta_squared(), which the
description of the function promised; its N column still
reports the number of observations, and the help says how to find the
size of a stratum from summary() of the fit. Fits without an
Error() term are unchanged. The raw-argument interfaces of
omega_squared() and ci_omega_squared() now accept an N as small
as df_effect + df_error, which DMAR 1.0.x refused, so that
ci_omega_squared() gives a row's estimate and its interval from the
row's F, its degrees of freedom, and the size of its stratum, and
omega_squared() gives the same estimate. ci_omega_squared() now
takes an aovlist fit itself (see Total and Partial Eta Squared and
Omega Squared).
The tidy() of an ANOVA effect size table returns every effect.
tidy() of ci_eta_squared(), ci_eta_squared_partial(),
ci_eta_squared_generalized(), and ci_omega_squared() in DMAR 1.0.x
returned the first row of the table alone, so the tidy() of a
factorial fit
dropped every other effect, and glance() was the same row. For
fit <- aov(len ~ supp * factor(dose), data = ToothGrowth),
tidy(ci_eta_squared_partial(fit, conf_level = 0.90)) gave one row,
the supp effect (.224 [.073, .350]); tidy(ci_eta_squared(fit, estimand = "partial", conf_level = 0.90)) gives three, adding
factor(dose) (.773 [.659, .814]) and the interaction (.132 [.011,
.246]).
The term names the effect size and, for ci_eta_squared() and
ci_omega_squared(), its estimand (eta_squared_partial or
eta_squared_total, where 1.0.1 named eta_squared; see One Function
for Each ANOVA Effect Size), and an effect column after it names the
effect; a stratum column follows it for a fit with an Error() term.
A table of one effect and one estimand, as from summary statistics,
therefore gives the row it gave with the effect column added. Its
glance() is that row when tidy() gives one row. Otherwise it is one
row with the effect size (term, the estimands separated by a comma
when there are two), the number of effects (n_effects), N, and
conf_level.
manova_split_plot(), mixed_anova(), and anova_within_two_way()
return DMAR tables. DMAR 1.0.x returned each result as a plain
data.frame, so class(x) was "data.frame" and the table printed
through print.data.frame(), which formats a column of numbers as a
whole: one small p-value put the whole p_value column in
scientific notation. In the example on the help page of
mixed_anova(), the three p-values printed as 2.148782e-05,
1.817787e-12, and 9.565106e-01. Each function now returns its table
with the class c("dmar_tbl", "data.frame"), as anova_within() and
factorial_anova() do, and it prints through the dmar_tbl display
layer: whole numbers such as degrees of freedom without a decimal
part, other values to 3 decimal places, and p-values to 4
decimal places, with one below 0.0001 printed as "< 0.0001" (in that
example, < 0.0001, < 0.0001, and 0.9565). Adding the class changes
nothing else in the object, and as.data.frame(x) returns the table
as a plain data frame. For mixed_anova() and anova_within_two_way()
that is the 1.0.x data frame itself (identical() to it): the
columns, rows, and values are unchanged, as is the
sum_of_squares_type attribute of mixed_anova(). For
manova_split_plot() the columns and rows are those of 1.0.x, but not
every value: its default Type III B rows now test the
within-subjects main effect, where those of 1.0.x tested a simple
effect (see the entry above on that test), and its other rows, with
the B rows under ss_type = 2, agree with those of 1.0.x to
rounding error (within 1e-12, relative, on the designs compared)
rather than exactly. For code written against 1.0.x:
is.data.frame(x) and inherits(x, "data.frame") are TRUE as
before, and x$p_value and the other columns hold the table's
numbers at full precision. Code that tests the class itself sees the
new one: identical(class(x), "data.frame") is FALSE;
class(x) == "data.frame" has length 2, so if () on it stops with
"the condition has length > 1" in R 4.2.0 and later (earlier
versions warn and use the first element, FALSE); and against a
saved 1.0.x result of mixed_anova() or anova_within_two_way(),
identical() is FALSE and all.equal() reports a difference in
the class alone.x[i, ], subset(), and rbind() with x first keep
the class, so their results print the new way too; merge() still
returns a plain data frame, and summary() and as.matrix() return
what they return for the plain data frame.
print() and format() now dispatch to the dmar_tbl methods, so
format(x) returns the display strings described above rather than
those of format.data.frame(). In an R Markdown document a table
that a chunk prints is set by knit_print.dmar_tbl() as a
knitr::kable() table of those strings where 1.0.x showed console
text; knitr::kable(x) called directly formats the numbers itself,
as before.tidy() and glance() answer through the default methods for a
wide DMAR table (see ?dmar_tidiers). tidy() gives one row per
row of the table, its term joining the label columns (effect
with statistic_name, denominator, or adjustment), its
estimate the first numeric column (statistic_value, ss, or
F_value), and the other numeric columns after it under their own
names. The last row of the tidy() of manova_split_plot() is the
table's record of the sum-of-squares type: its term is
"sum_of_squares_type:NA" (the row's statistic_name is NA), its
estimate the type, 2 or 3, and its other columns NA, so code that
summarizes the estimates of the tests leaves that row out, as it
would from the table. glance() gives one row: the number of rows
(n_terms), with conf_level and B_used NA, since no interval
or bootstrap is formed. For the plain data frame of 1.0.x the
generics package has no method, and where broom was loaded its data
frame tidiers, which broom has deprecated, answered: tidy() stopped
with an error on the character columns, and glance() gave nrow,
ncol, complete.obs, and na.fraction. results_sentence()
refuses these tables as before, since none holds a confidence
interval. as_kable(), which stopped on the plain data frame of
1.0.x with "no applicable method", now returns a knitr::kable()
table of the display strings, and dmar_metadata(), new in 1.1.0
and taking only a DMAR table, accepts these tables.ancova() already returned a dmar_tbl in DMAR 1.0.x, and its result
is unchanged.
ss_aipe_reliability_sensitivity(estimator = "omega") returns
intervals. Since 1.0.0 it asked reliability_omega() for the
"mlr" interval under the default observed denominator, for which no
Wald interval is defined; every replication failed silently, and the
table showed no widths and a non-coverage of zero. Omega is now
computed with denominator = "model_implied", the coefficient the
planner's interval is built on, with the "ml" interval the plan
describes (see the entry on the interval its plan was made for). A
replication that returns no interval is now set aside rather than
counted as covering the true value; the proportions are over the
replications that returned one, one warning states how many were set
aside and why, and a run in which none returned an interval stops with
the reason. The rule changes nothing for alpha with Bonett's interval,
its default through 1.0.1.
ss_aipe_reliability_sensitivity() evaluates the assurance plan at
the planner's defaults. With an assurance, the plan whose N is
simulated was searched at 50 and 200 replications (G_initial and
G, then initial_iter and final_iter)
against the planner's defaults of 500 and 5000, and the noise let a
size below the one the assurance needs pass by chance. For four
parallel items of reliability .80, a width of .15, and an assurance of
.80, the evaluated N ranged from 93 to 101 over seeds 1 to 30
(set.seed(s) before each call; mean 96.9) against 98 to 101 (mean
99.2) at the defaults, so a correctly specified plan seemed to fall
short of its assurance. The plan now runs at the planner's defaults,
as the rest of the sensitivity family does, which adds about a second.
With ci_method = "ml" the replications' alpha intervals change as
reliability_alpha()'s do, slightly for the unit variance items
simulated here (mean width .1531 to .1527 in 1,000 replications at N =
74, after set.seed(113)); Bonett's interval, the default through
1.0.1, is unchanged.
ss_aipe_reliability_sensitivity() evaluates the interval its plan
was made for. The plan is ss_aipe_reliability()'s for the parallel
model with the normal theory interval, a Wald interval with the van
Zyl, Neudecker, and Nel (2000) variance. The replications, however,
computed Bonett's (2002) interval for alpha and the "mlr" interval
for omega, so a correctly specified plan was judged by an interval it
was not made for. The default ci_method is now "ml" for both
estimators, with complete or incomplete data. For alpha that is
reliability_alpha(ci_method = "ml"), whose variance at a population
of parallel items is the compound symmetry form the plan uses. For
omega it is reliability_omega(denominator = "model_implied", ci_method = "ml"), whose delta method variance equals that form there
to machine precision. Under missing data these are the Wald intervals
with the FIML standard error that the fmi adjustment plans for.
set.seed(113) before each call
(true_reliability = .8, supposed_reliability = .8, n_items = 4, width = .15, assurance = .8, G = 2000). The plan of 99 cases gives
intervals no wider than .15 in 81.3 percent of the 2,000
replications, against 76.75 percent with ci_method = "bonett" on
the same draws, and a mean width of .1312 against .1346.estimator = "omega", G = 1000), which returned no
intervals in 1.0.x, the default gives 83.3 percent against 81.9
with ci_method = "mlr".G = 20) goes from a mean
width of .1685 and a proportion of .40 to .1630 and .45.ci_method = "bonett" reproduces 1.0.x without an
assurance, apart from the replications the entry on clamped limits
describes. With an assurance the plan is now searched at the
planner's defaults (see the entry on the assurance plan), which
draws other replications: the call above gave 78.65 percent and a
mean width of .1337 in 1.0.1. Any other interval can still be named.Noncentral chi square intervals stay right for a statistic in the
millions. R's pchisq() with a noncentrality stops its series after
10^6 terms, and above a noncentrality of about 2e6 it returns wrong
probabilities (0 where the value is .19 at 3e6), with a warning that
was suppressed. ci_nc_chisq(), and through it ci_rmsea(),
ecvi(), and the intervals of sem_fit_measures() and
equivalence_rmsea(), then returned limits collapsed onto one value
below the statistic. For a statistic of 3e6 on 3000 df the 90 percent
limits were 2979734 twice and are now 2991307 and 3002700; the RMSEA
interval at N = 1e5, with the multiplier N, was [0.09966, 0.09966]
around an estimate of 0.09995 and is now [0.09986, 0.10004]. Above a
noncentrality of 1e5, the distribution function is now summed as the
Poisson mixture of central chi square distributions it is, accurate to
rounding. At and below 1e5 nothing changes. equivalence_rmsea() and
the close fit tests of sem_fit_measures() use the same function: a
bound of .10 at a statistic of 3e6 had a p-value of 0 and was declared
met, where the p-value is .193. For a fitted model,
sem_fit_measures() computes the limits and tests itself once the
statistic, or the noncentrality of a close fit bound, exceeds 1e5. The
robust limits and tests of a FIML, ordered data, or two-stage fit,
which it cannot rebuild, become NA with a warning past 1e6.
ci_rmsea() gains sample_size_convention = c("N-1", "N"). It
sets the multiplier with which the reported RMSEA was computed;
lavaan's default likelihood and Mplus use N. The default keeps every
earlier result. The help page now says correctly that the lower limit
of the 90 percent interval settles the test of close fit and the
upper limit the test of not close fit (it had described the upper
limit as the test of close fit), and that the function assumes one
group.
ci_rmsea() refuses zero degrees of freedom and a sample size that
leaves no multiplier. A saturated model (lavaan reports an RMSEA of
0 on 0 df) had returned limits of Inf with no warning, the lower one
above the estimate. N = 1 under the default convention had given an
upper limit of Inf, and N between 0 and 1 an NA. df must now be
greater than 0; it need not be whole, since a mean and variance
adjusted statistic has fractional degrees of freedom. N must make
the multiplier positive. An invalid conf_level is refused in its own
name rather than as alpha_lower and alpha_upper.
ci_rmsea() forms a 90 percent interval by default. The default
conf_level is 0.90, where DMAR 1.0.x had 0.95, since a 100(1 - 2
alpha) percent interval decides the one-sided tests of close and not
close fit at level alpha (MacCallum, Browne, & Sugawara, 1996, p. 136),
so the limits of the 90 percent interval are those tests at .05 (see
Every RMSEA Interval at 90 Percent by Default). For an RMSEA of .035 on
40 df with N = 425, the interval of the help page's first example, the
limits were .008 and .055 and are .015 and .052. conf_level = 0.95
restores the 1.0.x interval. An interval given by alpha_lower and
alpha_upper is unchanged.
ss_aipe_rmsea() plans a 90 percent interval by default. The default
conf_level is 0.90, where DMAR 1.0.x had 0.95, so that the plan is for
the interval ci_rmsea() now forms by default. A lower level needs a
smaller sample for the same width: for an RMSEA of .045 on 5 df and a
width of .060, necessary_N was 1068 and is 802. conf_level = 0.95
restores the 1.0.x plan.
ss_aipe_rmsea_sensitivity() simulates 90 percent intervals by
default and forms each as lavaan does, from the replication's statistic
with lavaan's multiplier.
conf_level is 0.90, where DMAR 1.0.x had 0.95, as in
ss_aipe_rmsea() and ci_rmsea(). conf_level = 0.95 restores the
level.ci_rmsea(). The interval matched neither convention and was slightly
too wide, that of a sample one case smaller. It is now formed with
lavaan's own multiplier (N under the default likelihood, N - 1 under
likelihood = "wishart" or a least squares estimator passed through
...). This was a defect, and no argument restores it.ci_rmsea()), so every such
replication was given the interval of that statistic, wider than the
interval of its own. Each interval is now formed from the
replication's chi square statistic, as lavaan and sem_fit_measures()
form it, and equals the interval lavaan reports for the sample
whatever the statistic; its upper limit is 0 when the statistic is at
or below the (1 - conf_level) / 2 quantile of the central chi
square, the convention the two share. A model that fits well, many of
whose replications fall below their degrees of freedom, gets narrower
intervals: for the two-factor population of the help page fit by its
own two-factor model (8 df), with N = 200, G = 40, conf_level = 0.90, and set.seed(113), 21 of the 40 statistics fell below 8, the
mean width, .0920 in DMAR 1.0.x, is .0802, and the proportion of
intervals narrower than a width of .05, 0 before, is .10. This was a
defect too, and no argument restores it. A replication whose
statistic exceeds its degrees of freedom gets the interval it had, so
a markedly misspecified model, such as that of the help page's
example, is not affected.necessary_N, the size a
study would collect. The width depends on the sample size only through
the multiplier, and ss_aipe_rmsea() plans with N - 1 (Kelley & Lai,
2011), so under lavaan's default, whose multiplier is N, the realized
intervals are a little narrower than the planned width, by an amount
of order 1/N.set.seed(113), G = 20), DMAR 1.0.x evaluated a
sample of 695 and reported a mean width of .05002; the default now
evaluates 493 for the 90 percent interval, with a mean width of .04990,
and conf_level = 0.95 evaluates 695, with a mean width of .04998.ecvi(fit) divides by the multiplier lavaan used to form the
statistic. For a fit under likelihood = "wishart" or a least
squares estimator (GLS, WLS, ULS, DWLS) that multiplier is N - G, and
the earlier divisor N disagreed with lavaan::fitMeasures(); the
three-factor Holzinger and Swineford model under the Wishart
likelihood gave 0.4220 and now gives lavaan's 0.4234, and the
interval limits move with it. Results for ML under the default normal
likelihood, and for summary input, are unchanged. The help page now
states the identity N ECVI = AIC + 2 l_s only for the likelihood
ratio statistic.
ecvi() applies the checks of the SEM fit family and states its
boundary cases. A fitted model with an inadmissible solution is now
refused, as sem_fit_measures() and sem_information_criteria()
refuse it; a 60 case two-factor fit of the Holzinger and Swineford
tests with a latent variance of -72.9 had an ECVI of 1.116 [0.799,
1.598]. A well fitting model, whose statistic is below the upper
alpha/2 quantile, no longer warns about a prob_greater row that
ecvi() does not return. When the statistic lies at or below the
lower alpha/2 quantile, the upper limit is still NA, now with a
warning in ecvi()'s own terms. The help page says why that limit is
not set to the lower one, and that the untruncated estimate lies below
the interval when the statistic is below its degrees of freedom. n = 0, which returned Inf and NaN, is refused.
An ML fit is read as ML under lavaan 0.7-3. lavaan 0.7-3 lists
Browne's residual based statistic beside the standard one for an ML
fit by default, and measurement_invariance(), which asked whether
the test option was exactly "standard", labeled an ML ladder as
using the scaled difference test and robust fit indices. It now sets
aside the standard statistic and every Browne residual statistic
(alternative tests, not corrections) when deciding whether a fit
carries a robust test. The profile likelihood interval for model
implied omega and alpha no longer warns at every refit that lavaan
cannot compute Browne's statistic under its equality constraint. No
number changes.
measurement_invariance() reports one version of the test and of
the fit indices at every rung, and its attributes name the versions.
Outside FIML the ladder read the names of its measures from its first
rung alone. In the examples, hs_factors is the verbal and deduction
model of the help page.
test = "mean.var.adjusted"), for which lavaan computes no robust indices,
the cfi and rmsea columns held the standard indices while the
fit_indices attribute said "robust". The columns now hold the
robust indices, which DMAR computes because they do not depend on
which corrected statistic accompanies them (Savalei, 2018); they
equal lavaan's under estimator = "MLM", and a line under the table
says so. On the help page's first ladder
(measurement_invariance(holzinger_swineford, hs_factors, group = "school", test = "mean.var.adjusted")) the configural CFI was .9916
and is .9928, the configural RMSEA was .0520 and is .0478, and the
metric delta_cfi was -.0127 and is -.0134. A ladder whose
configural rung is saturated gets them too: on the three verbal
tests (model = "verbal =~ t6_paragraph_comprehension + t7_sentence + t9_word_meaning", configural, metric, and scalar) the
metric delta_cfi was -.0109 and is -.0100.test = "bollen.stine"), the chi_square
column and the difference test were the standard ones, while
fit_indices said "robust" and test named a scaled difference
test. Those numbers stand, and the attributes now say "standard".NA at every
later rung, and no delta_cfi. lavaan leaves the robust indices
NA at the rungs that constrain the thresholds. Every rung now
reports the scaled indices, and fit_indices says "scaled". On
the help page's ordered example (the verbal and deduction tests cut
at their quartiles, configural and thresholds rungs):
NA and are .996 and
.040;delta_cfi was NA and is -.0023.NA on another) or does not compute
it. Such a rung now reports a CFI of 1, which every version implies
(see the entry on a saturated rung).measurement_invariance(missing = "fiml") reports the FIML
corrected fit indices of Zhang and Savalei (2023), and a line under
the table says so. The corrected indices estimate the values
complete data would give, the values a cutoff for a change in fit
(Cheung & Rensvold, 2002) is a statement about, where lavaan's
standard cfi and rmsea move with the amount and kind of missing
data; fa_confirmatory() reports the corrected indices for the same kind of
fit. Under FIML the cfi and rmsea columns, and delta_cfi and
delta_rmsea with them, are lavaan's robust versions, chosen as
fa_confirmatory() chooses them: Zhang and Savalei's version 6, or version 3
with estimator = "MLR".
Through 1.0.1 an ML ladder under FIML reported the standard indices
with lavaan 0.7-2, the oldest lavaan it accepted. With lavaan 0.7-3
it reported the corrected ones, but only because it read every ML
fit as robust (see the entry on lavaan 0.7-3), and with estimator = "MLR" it reported them with either lavaan; those numbers stand.
On the verbal and deduction tests with 90 sentence completion scores
deleted at random (set.seed(113), then t7_sentence[sample(301, 90)] <- NA), the standard and the corrected indices are:
On the help page's FIML example the metric RMSEA is .137 and .140.
One version holds every rung. When lavaan cannot compute the
corrected indices at some rung with degrees of freedom, every rung
falls back to the scaled indices of a scaled fit or to the standard
ones. A saturated rung does not decide the version: its robust CFI,
formed from a ratio of zeros, can be NA, and that rung reports a
CFI of 1, which every version implies. On visual perception, cubes,
and lozenges by school (model = "visual =~ t1_visual_perception + t2_cubes + t4_lozenges") with 60 cube scores deleted at random
(set.seed(1), then t2_cubes[sample(301, 60)] <- NA), configural,
metric, and scalar, lavaan's robust CFI of the saturated configural
rung is NA. Every rung reports the corrected indices (scalar CFI
.852 and RMSEA .172, where the standard indices are .861 and .159),
and the configural CFI is 1.
Whether a ladder used FIML is read from its fits: lavaan fits the
complete cases instead, with a warning, when FIML meets a test it
cannot combine with it (test = "satorra.bentler" or
"mean.var.adjusted"), and such a ladder is reported and labeled
as the listwise ladder it is.
The fit_indices attribute names the version ("robust", or
"scaled" or "standard" in the fallback). A line under the table,
kept in the new fit_columns attribute, says which indices the
columns hold, at the console and in a knitted report. Complete data
and listwise and pairwise deletion are not affected by this entry.
A saturated rung of measurement_invariance() has a CFI of 1.
NA or 1 by rounding. The
table reported the NA, and the next rung's delta_cfi with it. On
the verbal tests of holzinger_swineford under estimator = "MLR"
(items = c("t6_paragraph_comprehension", "t7_sentence", "t9_word_meaning"), group = "school"; configural, metric, and
scalar), the configural CFI was NA and is 1, and the metric
delta_cfi was NA and is -.0097. The three visual tests with 60
cube scores deleted under FIML (see the entry above) behaved the
same way.measurement_invariance() reports no fit index or difference test
of a ULS ladder of continuous items that is formed from the ULS
statistic and changes with their units. The examples are the help
page's first ladder, the verbal and deduction tests by school.
estimator = "ULSM" or "ULSMV" and
continuous items, the rmsea column held lavaan's robust RMSEA,
which lavaan forms by putting that statistic into a correction
derived for the ML fit function. It was 1.57, 4.92, and 4.50 at the
configural, metric, and scalar rungs of a ULSM ladder, and 0.0157,
0.0492, and 0.0532 with every score divided by 10. The rmsea and
delta_rmsea columns are now NA.delta_cfi was -.00005 against -.0046: a judgment against
the cutoff of Cheung and Rensvold (2002) that turned on the units.
The cfi and delta_cfi of those rungs are now NA. The
configural and metric CFI (.9991 and .9886) do not depend on the
units and are unchanged. So are the chi_square column and the
scaled difference test, whose reference distribution does not
depend on the units, though their values do.estimator = "ULS" and test = "standard", the cfi and
rmsea columns held the standard indices formed from the ULS
statistic: CFI .9956 and .9818 and RMSEA 3.51 and 6.37 at the
configural and metric rungs, and CFI 1 and RMSEA 0 at both with the
scores divided by 10. Both columns are now NA.estimator = "ULS", alone or with test = "standard",
lavaan's difference test refers the difference of the ULS statistics
to the chi square distribution. The metric rung's delta_chi_square
was 91712 (p < .0001), against 9.17 (p = .057) with the scores
divided by 10, and the test attribute named a scaled difference
test (a standard one with test = "standard"). delta_chi_square,
p_value, and test are now NA.estimator = "ULS" alone, Browne's residual statistic remains
a valid test. CFI and RMSEA at configural and metric rungs, which
leave the means free, are unchanged. At scalar and strict rungs,
restrictions link means and covariances through the latent means,
so the ULS estimates change with the units. Browne's statistic at
those estimates and its indices change too: CFI, RMSEA, and their
differences are now NA there. On the help page's first ladder,
measurement_invariance(holzinger_swineford, hs_factors, group = "school", estimator = "ULS"), the scalar CFI was .8874 and its
RMSEA .0680 in 1.0.1; both are now NA.NA and why.
estimator = "MLR" is the estimator for nonnormal continuous items.measurement_invariance() under FIML leaves an auxiliary variable's
mean and variance free at every rung. An auxiliary variable passed
through ... as lavaan's aux enters a FIML fit as a saturated
correlate (Graham, 2003). lavaan makes its mean and variance
parameters of each group, and the ladder's group.equal reached them.
The scalar rung equated the auxiliary's means across the groups and
the strict rung its variances, so each of those rung tests also
tested, with one more degree of freedom, whether the auxiliary is
distributed alike in the groups. They are now added to group.partial
at every rung, beside any group_partial terms. The ladder then has
the degrees of freedom of the ladder without the auxiliary, and its
differences are those of the same ladder with the auxiliary's moments
free.
Example: a spatial factor of visual perception, cubes, and lozenges
and a verbal factor of paragraph comprehension, sentence completion,
and word meaning by school, with 60 sentence completion scores deleted
at random (set.seed(1), then t7_sentence[sample(301, 60)] <- NA)
and general information as the auxiliary (means 44.8 and 36.6 in the
two schools).
Under two-stage estimation the auxiliaries enter the first stage only, so its ladders hold no such parameters.
measurement_invariance() merges lavaan's group.partial passed
through ... into group_partial, and refuses group.equal there.
Each rung passes lavaan one group.partial, so through 1.0.1 a call
that gave both group_partial and group.partial stopped with
lavaan's error that a snake_cased name is duplicated. The terms of
both are now freed at every rung, beside those of an auxiliary
variable under FIML. On the six tests by school above under FIML,
without the auxiliary, group_partial = "t2_cubes ~ 1" with
group.partial = "t4_lozenges ~ 1" gives a scalar test of 1.56 on 2
df, where 1.0.1 stopped.
group.equal in ... constrained the configural rung, which was
still reported as configural, and a ladder that went past it stopped
with the same error. On the six tests by school above under listwise
deletion, group.equal = "loadings" with levels = "configural" gave
a configural rung of 25.62 on 20 df against the configural model's
15.27 on 16. It is now an error that names levels and
group_partial.
htmt() returns the same four columns whatever B. ci_upper
existed only when B > 0, so the shape of the table depended on an
argument. It is now always present, NA when no bootstrap is run. A
bootstrap bound also carries the B_used attribute (the resamples
left after any were dropped), which glance() reports. No value
changes.
htmt() takes the absolute value of every correlation, the
monotrait correlations within each block as well as the heterotrait
correlations between blocks. Through 1.0.1 only the heterotrait
correlations were absolute, a mix that matches neither the signed
ratio nor the absolute one semTools::htmt() computes. A reverse
keyed item that was not recoded then lowered its block's average
within-construct correlation while the numerator kept its size, so the
ratio was inflated. Take one such item in a block of four:
set.seed(113); n <- 300
f1 <- rnorm(n); f2 <- 0.5 * f1 + sqrt(0.75) * rnorm(n)
d <- data.frame(a1 = .8 * f1 + rnorm(n, 0, .6),
a2 = .7 * f1 + rnorm(n, 0, .7),
a3 = .8 * f1 + rnorm(n, 0, .6),
a4 = -.6 * f1 + rnorm(n, 0, .8),
b1 = .8 * f2 + rnorm(n, 0, .6),
b2 = .7 * f2 + rnorm(n, 0, .7),
b3 = .6 * f2 + rnorm(n, 0, .8))
blocks <- list(A = c("a1", "a2", "a3", "a4"),
B = c("b1", "b2", "b3"))
htmt(d, blocks)
fa_confirmatory(d, factors = blocks, output = "measurement")
The two constructs, which correlate .5 in the population, had an HTMT
of 1.346, a discriminant validity failure that is not there, and
fa_confirmatory(output = "measurement") reported it beside phi = .473 without
a word; a block of three with such an item stopped the call. Every
correlation now enters as an absolute value, so the ratio does not
depend on how an item is keyed (1.346 becomes .463, the value of the
recoded data). When no correlation within a block is negative, as with
consistently keyed items, the point estimates are unchanged, the
example of ?htmt and three factors of holzinger_swineford
included. The bootstrap bound can move, though: a resample that draws
a negative correlation within a block now enters it with its size,
where 1.0.1 averaged it with its sign or dropped the resample. Take
two blocks whose within-block correlations for B are .04, .11, and
.20:
set.seed(4); n <- 80; f <- rnorm(n); g <- rnorm(n)
d <- data.frame(a1 = f + rnorm(n, sd = .5),
a2 = f + rnorm(n, sd = .5),
b1 = .45 * g + rnorm(n), b2 = .45 * g + rnorm(n),
b3 = .45 * g + rnorm(n))
htmt(d, list(A = c("a1", "a2"), B = c("b1", "b2", "b3")), B = 200,
seed = 113)
The HTMT is .408, as before, and the bound .728 from 200 resamples,
where it was .869 from 198. The bound of the ?htmt example is
unchanged. On complete data the values agree with
semTools::htmt(htmt2 = FALSE, absolute = TRUE). On incomplete data
they agree only when semTools is given the same pairwise-complete
correlations through sample.cov, since by default it deletes
incomplete rows listwise. The help page no longer credits the absolute
form to a 2021 paper missing from its references, and a pair whose
ratio cannot be computed is reported as the next entry describes.
htmt() reports a pair whose ratio cannot be computed as NA, with
a warning, and fa_confirmatory(output = "measurement") keeps its table. A
correlation that cannot be computed leaves a pair's ratio undefined.
That happens with two items never observed together, as in a planned
missing data design whose forms give two blocks to different
respondents, or with a constant item. Through 1.0.1 such a pair across
two blocks was NA without a word, and a pair undefined within a
block stopped the call with "missing value where TRUE/FALSE needed".
The pair is now NA, as is its bootstrap bound, with a warning that
names the pair and the reason, and the other pairs are reported. The
call stops, with a message that says why, only when no pair is
defined. A call with only two blocks whose one pair is undefined,
which returned NA, therefore stops.
fa_confirmatory(output = "measurement") keeps every phi, omega, AVE, and H
row and reports the undefined htmt row as NA with the warning, as
1.0.1 did without the warning. Take two three-item factors of 300
cases with a1 and b1 on different forms:
set.seed(113); n <- 300
f1 <- rnorm(n); f2 <- 0.4 * f1 + sqrt(.84) * rnorm(n)
d <- data.frame(a1 = .8 * f1 + rnorm(n, 0, .6),
a2 = .7 * f1 + rnorm(n, 0, .7),
a3 = .7 * f1 + rnorm(n, 0, .7),
b1 = .8 * f2 + rnorm(n, 0, .6),
b2 = .7 * f2 + rnorm(n, 0, .7),
b3 = .7 * f2 + rnorm(n, 0, .7))
d$a1[c(TRUE, FALSE)] <- NA; d$b1[c(FALSE, TRUE)] <- NA
fa_confirmatory(d, factors = list(A = c("a1", "a2", "a3"),
B = c("b1", "b2", "b3")),
missing = "fiml", output = "measurement")
This gives phi .380 [.239, .521], omega .793 and .784, AVE .558 and .549, and H .798 and .792.
krippendorff_alpha() returns the same seven rows whatever
boot. The ci_lower, ci_upper, and B_used rows existed
only when boot = TRUE, so the shape of the table, and of its
tidy() and glance(), depended on an argument, as htmt()'s did.
They are now always present, and NA when no bootstrap is run, which
is the default: the nominal ratings of Krippendorff (2011, Table 1)
give seven rows where they gave four, with alpha .743 and every other
value unchanged. As in htmt() and average_variance_extracted(),
the confidence level is recorded, and printed, only beside an interval
that was formed. A script that tested for the rows by name
("ci_lower" %in% x$term) should test for a finite value instead.
results_sentence() now refuses a table whose limits are NA rather
than writing "95% CI [NA, NA]". average_variance_extracted() has
returned its ci_lower and ci_upper columns under every ci_method
since 1.0.0, and its tests now pin the columns of its tidy() and
glance() as well.
krippendorff_alpha() checks B and reports the bootstrap
resamples it drops. B must now be a whole number that meets the
rule for bootstrap replications when boot = TRUE (400 at the 95
percent level; see One Rule for Bootstrap Replications); through 1.0.1
any value was accepted, and B = 10 gave a percentile interval from
ten resamples. A resample on which alpha
cannot be computed (too few pairable values, or values all alike) is
dropped as before, but now with one warning that gives the count, and
the limits are withheld when too few resamples remain. When the data
themselves leave alpha undefined (every value alike), no bootstrap is
run: the interval rows stay NA and no confidence level is recorded,
where the table had said B_used = 0 beside "Confidence level: 95%".
Intervals are unchanged.
The bound for close fit in sem_residual_fit() is credited to the
paper that proposed it. The help page attributed the bound of .05
times the average R squared to Ximénez, Maydeu-Olivares, Shi, and
Revuelta (2022). It is the bound of Shi, Maydeu-Olivares, and
DiStefano (2018). Their criterion also requires the largest absolute
standardized residual covariance to be at most .10 (with .10 times the
average R squared and .15 for acceptable fit), and they apply both as
population values through an interval or a test. The page and the SEM
fit vignette now say so, and say that the function tests the first
part and reports the second as a sample value. Ximénez et al. (2022)
are cited for finding that the bound works reasonably well for the
unbiased SRMR in finite samples, except when the loadings are small.
The sem_fit_measures() page and the vignette now also cite Steiger
(1990, p. 178), which states in print the result of Steiger (1989)
behind gamma hat: gamma hat and adjusted gamma hat estimate the
population values of the GFI and the adjusted GFI.
The Steiger and Fouladi (1997) chapter is cited by its title. The
help pages of smd(), ci_smd(), ci_smd_c(), ci_sm(), cles(),
ci_mahalanobis(), ci_nc_t(), ci_sc_ancova(), ss_aipe_smd(),
ss_aipe_smd_sensitivity(), ss_aipe_sm(),
ss_aipe_sm_sensitivity(), ss_aipe_sc(),
ss_aipe_sc_sensitivity(), ss_aipe_sc_ancova(), and
ss_aipe_sc_ancova_sensitivity() gave its title as "Noncentrality
interval estimation and the evaluation of statistical methods"; the
chapter is "Noncentrality interval estimation and the evaluation of
statistical models" (pp. 221--257). The help page of
sem_fit_measures() and the SEM fit vignette, both new in 1.1.0, cite
it by that title for the intervals of gamma hat.
The journal and subtitle of Kelley, Maxwell, and Rausch (2003) are
given as published. The help pages of ci_smd(),
design_consequences(), ss_aipe_c(), ss_aipe_c_ancova(),
ss_aipe_equivalence_r(), ss_aipe_equivalence_smd(),
ss_aipe_mean(), ss_aipe_smd(), and ss_power_smd() gave the
journal as "Evaluation and the Health Professions" and the subtitle as
"Delineating methods of sample size planning"; the journal is
Evaluation & the Health Professions and the subtitle is "Delineating
methods of sample-size planning", as the DOI record of the article
(10.1177/0163278703255242) has them. The volume, issue, and pages,
26(3), 258--287, already matched the record and are unchanged.
The edition of Rencher and Christensen (2012) is given as
published. The help page of manova_split_plot() gave Methods of
multivariate analysis (Wiley, 2012) as the fourth edition; the 2012
book is the third edition, as the catalog record of its print edition
(ISBN 978-0-470-17896-6) and the Crossref record of its review in
International Statistical Review, 81(2), 328--329, have it. The entry
gains the book's DOI, 10.1002/9781118391686.
The edition of Anderson (2003) is given as published. The help
pages of ci_eigenvalue() and ci_mahalanobis() gave An introduction
to multivariate statistical analysis (Wiley, 2003) as the fourth
edition; the 2003 book is the third edition, as the publisher's page
for it (ISBN 978-0-471-36091-9) and catalog records of the print edition
have it, and as the pages of expected_r() and expected_partial_r()
already cite it. The entry on the ci_mahalanobis() page also sets the
title in sentence case, as the other entries do. Crossref holds no
record of the book, so the entries carry no DOI.
ss_aipe_reliability() returns the AIPE planner class with an
assurance. The closed form plan carried class dmar_ss_aipe and the
assurance plan did not, so tidy() and glance() answered
differently by path: for an assurance plan tidy() gave five term and
estimate rows rather than the planned N beside the width. Both paths
now carry the class, so an assurance plan's tidy() is one row (term = "sample_size", estimate, width) and its glance() adds
assurance, assurance_attained, and G as
columns. The values of the table are unchanged (its rows are renamed;
see Breaking Changes in 1.1.0). Two
arguments are now checked before they are used: a vector true_residual_var
for the Parallel model stopped with R's "the condition has length > 1"
and now stops with the function's own message, and a True Score plan
from true_factor_loading without n_items stopped with "argument is
of length zero"
and now says that n_items is needed. A vector true_factor_loading
for the Parallel
model was recycled across the items, so it planned the congeneric
pattern it describes under the Parallel label without saying so
(loadings .8, .7, .7, and .8 with true_residual_var = .4 returned N = 96).
It is now an error, as it already was for the True Score model, with a
message that names the Congeneric model, and both models refuse any
true_factor_loading that is not one number.
ss_aipe_reliability() plans for the coefficient of the raw
items. The tau equivalent model on every route, and the congeneric
model given by S, turned the population covariance matrix into a
correlation matrix before the closed form and the Monte Carlo draws.
That plans for alpha or omega of standardized items rather than for
the coefficient reliability_alpha() and reliability_omega()
estimate, and the same congeneric population entered by loadings or by
S got different searches. Every route now uses the covariance matrix
itself. With equal loadings .7 and error variances .1, .2, 1, and 2,
the normal theory plan for a width of .10 goes from 206 to 367 cases
(the factor analytic one to 242; see the entry on that interval). At
367 the corrected interval averages .1004 wide and covers .941, while
at 206 the 1.0.1 interval was as narrow as planned (.1008) but covered
.869 (1,000 samples of each size, drawn as in the entry on that
interval and analyzed by reliability_alpha(S = S, N = N, ci_method = "ml")). A correlation matrix still plans for standardized items, and
items of equal variance keep the closed form they had.
Seeded assurance searches with the normal theory interval of the tau
equivalent model move a little for any population. Each sample's
interval now takes its standard error from that sample's covariance
matrix, whose variances differ even when the population's do not, and
a population whose equal variances are not one is now drawn from its
covariance matrix. With true_item_covariance = .3 and error variances of .2 on
five items, a width of .10, an assurance of .85, G_initial = 50,
G = 200, and seed 113, the plan goes from 81 cases
(empirical assurance .905) to 79 (.885). The True Score example of the
1.0.1 help page (true_item_covariance = .3,
true_residual_var = c(.2, .3, .3, .2, .3),
width .10) goes from a closed form of 84 to 86 and, with an assurance
of .85 (G_initial = 50, G = 200, seed 113), from 115
to 118. The page now plans for items near unit variance (true_item_covariance = .5, true_residual_var = c(.4, .5, .6, .4, .6)): 108 cases, or 140 with
that assurance (106 and 138 under 1.0.1). Its plan from a correlation
matrix (pop_mat, width .15) with that assurance goes from 120 cases
(empirical assurance .85) to 121 (.855), because its Monte Carlo
intervals now use the sample covariance matrix. The Parallel examples
give the plans they gave (38 cases, and 57 with an assurance of .85).
ss_aipe_reliability() plans coefficient omega from omega's own
variance. Without an assurance the congeneric model returned Terry
and Kelley's (2012) Eq. 19, the variance of alpha, which they used
only as a starting value for a Monte Carlo search on omega's mean
width (p. 390). The closed form is now the delta method variance of
the model implied omega from the expected information of the
single-factor model at the population (Raykov, 2002), the variance
behind reliability_omega(denominator = "model_implied", ci_method = "ml"). When the population is given by S or data, the model is
first fit to it by maximum likelihood. Eq. 19 was too large when
standardized items have unequal loadings and could be far too small
when the item variances differ. At width .15, loadings .9, .8, .3, and
.2 of standardized items go from 297 to 177 (the interval averages
.114 wide at 297 and .153 at 177); loadings .3, .5, 1, and 2 with
error variances .1, .3, 1, and 3 go from 75 to 151 (.211 wide at 75,
.148 at 151); and the help page's example goes from 53 to 52. At width
.10, loadings 2, 1, .6, and .4 with unit error variances go from 360
to 229, and six standardized items with loadings from .4 to .8 from
197 to 183. The widths come from 1,000 samples of each size, drawn as
in the next entry and analyzed by reliability_omega(S = S, N = N, denominator = "model_implied", ci_method = "ml"); three of those at
177 return no interval.
The first-order approximation can miss the mean width when samples at the planned size approach improper solutions or empirical underidentification. The median can also exceed the target. The loadings, item count, and sample size act together; no lower bound on individual loadings separates these populations. Three-item models are just identified, and a sample covariance near zero can make their estimated loadings run away. Wider targets can create the same problem with more items. The help describes these cases and points to an assurance above one half, whose search counts the realized widths. A congeneric population with two items is refused.
ss_aipe_reliability() plans the factor analytic interval of the
tau equivalent model for that interval. Without an assurance the
True Score model returned the normal theory size whatever type was,
so type = "Factor Analytic" planned for the van Zyl, Neudecker, and
Nel (2000) interval of alpha, which through 1.0.1 was evaluated on the
correlation matrix. It did not plan for the interval the user asked
for: the delta method interval of the coefficient implied by the
single-factor model with equal loadings. That is the interval of
reliability_alpha(estimator = "model_implied", ci_method = "ml") and
the one the assurance search draws. That coefficient is the maximum
likelihood estimator under the model, so its variance is no larger
than the normal theory variance of alpha. The closed form now takes
its delta method variance, with the information and gradient of the
one loading summed over the items. A population given by S or data
is first fit by the equal loadings model. Take equal loadings .7 and
error variances .1, .2, 1, and 2 at a width of .10. 1.0.1 returned 206
cases, too few for this interval, which averages .108 wide at 206
cases; about one sample in eight gives an interval no wider than .10.
The normal theory plan for these items is 367, and the factor analytic
plan is now 242, at which that interval averages .10 wide. These
widths come from 1,000 samples of each size drawn after set.seed(1)
as the assurance search draws them, each S <- var(MASS::mvrnorm(N, rep(0, 4), Sigma)) from the population matrix Sigma, and analyzed
by reliability_alpha(S = S, N = N, estimator = "model_implied", ci_method = "ml"). With equal error variances the two variances
coincide, so the Parallel model keeps its closed form, and no normal
theory plan is affected.
ss_aipe_reliability() refuses a population it cannot plan for.
Through 1.0.1 each of the following was planned without a word:
true_residual_var must now be above zero for the
Congeneric model and for the True Score model with the factor
analytic interval, as it already had to be for the Parallel model.
These plans fit the single-factor model, at whose boundary an error
variance of zero lies. At a width of .15, loadings .7, .6, .5, and
.5 with error variances .51, -.1, .75, and .75 gave 81 cases for the
congeneric model, and one loading of .7 on all four items with those
error variances gave 46 for the tau equivalent one.S must. The plan is the one the same matrix gives as S: 74 cases
for that tau equivalent population at a width of .15, by either
route.S or implied on that normal theory
route. Loadings .7, .6, .5, and .5 with error variances .51, -.3,
.75, and .75 imply a matrix whose smallest eigenvalue is -.144;
given as S at a width of .15, it gave 25 cases for the True Score
model and 115 for the Parallel model.true_cov_matrix or data whose
single-factor
fit is improper or has no solution is refused.true_factor_loading = c(.7, .6, 0) with true_residual_var = c(.51, .64, 1) gave 937
cases at a width of .15.true_cov_matrix or
data. Alpha
at or below zero is equivalent only when the item variances are
equal: at a zero loading, the likelihood weights each covariance by
the inverse of the two item variances, and alpha does not. A population supplied through
true_item_covariance or true_factor_loading is refused when its
loading is zero. A
Parallel plan whose alpha is not positive despite a nonzero fitted
loading is refused as far from parallel, with the True Score model
named as the alternative. The normal theory interval still plans
for alpha of such items.The widths in this entry come from 1,000 samples of each size, drawn
as in the previous entry and analyzed by reliability_omega(S = S, N = N, denominator = "model_implied", ci_method = "ml"). With the factor
analytic interval, the assurance search now starts at no fewer than
one more case than there are items, the fewest a single-factor model
can be fit to. A highly reliable population planned for a wide
interval can have a closed form below that: loadings .95 on four items
need two cases for a width of .2. The search used to stop there.
ss_aipe_reliability() warns when the loadings of a congeneric
population differ in sign. The warning names the items that load in
the direction opposite to the others. Such items lower omega of the
unweighted sum, as an item keyed in the opposite direction does when
it is not reverse scored, and the plan is for omega of the sum of the
items as given. Standardized items with loadings .7, -.7, .5, and .5
at a width of .10 now plan 1419 cases with the warning; 1.0.1, with
alpha's closed form, planned 4884 without one.
The assurance of ss_aipe_reliability() counts a simulated study
that returns no interval as one whose interval is too wide. The
assurance is the probability of an interval no wider than desired
(Terry & Kelley, 2012), and a study that ends without an interval has
not had one. Through 1.0.1 a Monte Carlo sample whose single-factor
fit did not converge was set aside, and the proportion was taken over
the rest. A sample whose solution had run away (error variances
summing below zero, a model implied coefficient above one) was kept,
and its interval, often inverted (an estimate of 104 between 104 and
1), was counted as narrow or wide. reliability_omega() now returns
no coefficient for a runaway solution, and the search treats each
sample as the estimation functions treat it. A sample that returns no
interval, whether its fit did not converge, had no solution, or ran
away, counts as wide. Every assurance plan carries their count in
no_interval, zero when all samples returned an interval, and
glance() carries the count as a column. A nonzero count is printed
under the table. Both confidence limits are clamped into [0, 1], as
the estimation functions clamp them, which changes no count. The rule
changes nothing in a search in which every sample returns an
interval, as every normal theory search does. Samples without an
interval are rare unless several loadings are .3 or below. Take
standardized items with loadings .8, .3, .2, and .2 and a width of
.40 (200 samples of each size, drawn and analyzed as in the entry on
the populations the planner refuses):
G_initial = 50, G = 200,
seed 1), 1.0.1 returned 110 cases at an empirical assurance of .626,
taken over the samples whose fit converged. The plan is now 118, at
.545, and 39 of the 200 samples of the final pass return no
interval.ss_aipe_reliability()'s assurance search can go below its starting
size, says why it stops, and works from data. The search started at
the closed form size and only went up. When that size already met the
assurance, as with an assurance near one half or below, or with the
factor analytic interval of the tau equivalent model, which is
narrower than the normal theory interval 1.0.1's closed form
described, the search returned the closed form size whatever the
assurance needed. For the tau equivalent population of equal loadings
.7 and error variances .1, .2, 1, and 2 with the factor analytic
interval, a width of .10, and an assurance of .80, 1.0.1 planned from
the correlation matrix and returned its start, 207 cases, at an
empirical assurance of .99 for standardized items. That is too few for
alpha of the items as they are: at 207 cases .156 of the samples give
an interval no wider than .10 (1,000 samples, drawn as in the entry on
that interval). The plan is now 263 (G_initial = 100,
G = 400, seed = 1), searched up from the factor analytic
closed form of 242. The search now moves down to where the widths it
drew would meet the target and then up one case at a time; a search
whose start falls short, and in which every sample returns an
interval, draws what it drew before from the same matrix (a tau
equivalent population whose variances are not one is now drawn from
its covariance matrix rather than its correlation matrix, and so draws
differently). A size at which no Monte Carlo sample returns an
interval stops with the first error, where the search stopped with
"missing value where TRUE/FALSE needed". How a sample that returns no
interval counts is in the entry on the assurance of
ss_aipe_reliability(). The congeneric model with type = "Normal Theory", which the help already called an error, is refused at once
(the closed form returned an N without reading type). A population
given by data now sets the number of items, so an assurance plan
from data no longer stops with "invalid 'times' argument".
ss_aipe_reliability() names the common covariance of the items for
what it is. The argument was cor_est and was
called the presumed inter-item correlation, but the function takes its
square root as the common loading beside the error variances, so it is the
common inter-item covariance, and it is the correlation only when
true_residual_var = 1 - true_item_covariance, Terry and Kelley's
standardized design. It is now true_item_covariance, and the help says
what it is.
The Parallel examples planned from a covariance of .3 with error
variances of .2, items that correlate .6; they now give the same populations as
standardized items (true_item_covariance = .6, true_residual_var = .4) and say so, and
their plans are the same. The True Score example now plans another
population (see the entry on the coefficient of the raw items). The
page also says that a correlation matrix plans for standardized items,
and how the closed forms and the assurance search work.
The help says what seed = NULL does. The pages of the
reliability family, krippendorff_alpha(), randomization_test(),
randomization_test_paired(), mlmr(), mlmr_mv(),
R2_mixed_effects(), plot_trajectories(), and
plot_trajectories_fitted() said that a NULL seed leaves the random
number state intact. In fact the draws come from, and advance, the
caller's stream; it is a supplied seed that leaves the state as it
was. The help of reliability_alpha() and reliability_omega() also
now names the delta method behind bootstrap_se_logistic (Kelley &
Pornprasertmanit, 2016, Eq. 32). No number changes.
Suggested packages. lavaan's floor rises from 0.7-2 to 0.7-3, the
version every two-stage and factor score result was checked against
(factor_scores() refuses ordered fits under 0.7-2 in any case);
mice (3.18.0 or newer, which keeps ordered factors ordered when it
augments the data) is added for multiple imputation in categorical
omega; and mvtnorm is no longer suggested.
A confidence level is displayed with the digits it needs, never as
100% when it is below 1, and whatever getOption("digits") is.
Through 1.0.1 the labels of plot_ci(), plot_smd(), plot_R2(),
and plot_mediation_mbco() and the caption of correlations_test()
rounded the level to a whole percent, so a 97.5% interval was labeled
98% and a 99.9% interval 100%, a level only an unbounded interval
has: plot_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.975)
read "98% CI [0.043, 0.954]" and now reads "97.5% CI [0.043, 0.954]",
plot_R2(R2 = 0.25, N = 100, p = 5, conf_level = 0.999) read "100%
CI [0.017, 0.477]" and now reads "99.9% CI [0.017, 0.477]", and a
correlations_test() result at conf_level = 0.975 is captioned
"Correlations (Pearson, 97.5% CI)", where it read "98% CI".
results_sentence() wrote the level with getOption("digits")
significant digits: under options(digits = 2),
results_sentence(ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.975)) gave "smd = 0.50, 98% CI [0.04, 0.95]"; it gives "smd =
0.50, 97.5% CI [0.04, 0.95]".ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.99999995) printed "Confidence level: 100%" and the sentence
"100% CI [-0.61, 1.61]". Digits are now added until a level below 1
no longer reads 100 ("99.999995%").getOption("OutDec"), as the
table's numbers and the sentence already did: under options(OutDec = ",") it reads "Confidence level: 97,5%" beneath a table of
0,043 and 0,954, where it read "97.5%".summary() of an mlmr() fit and confint() of mlmr() and
mlmr_mv() wrote the level and the interval columns with
getOption("digits"): under options(digits = 2), a fit with
conf_level = 0.975 gave its intervals at "level = 0.97" over
columns "1.3 %" and "99 %". It gives them at "level = 0.975" over
"1.25 %" and "98.75 %", the labels stats::confint() writes. At
the default settings nothing changes.pairwise_within() and contrast_test() (each
of 28 intervals at a family level of .999 is at 99.996%, which four
significant digits would have written as 100%), the interval
columns of a printed factor_scores() table, and the interval that
equivalence_rmsea() names.No interval and no stored level changes; only the display does.
An interval formed from unequal tail areas says so, and a one-sided
interval is written as the confidence bound it is. An interval with
tail area alpha_lower below its lower limit and alpha_upper above
its upper limit has confidence coefficient 1 - alpha_lower -
alpha_upper whatever the split (Neyman, 1937), and that is the
level the table records, but "95% CI [L, U]" is read as the
equal-tailed interval. Through 1.0.1 nothing recorded the split:
ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, alpha_lower = 0.01, alpha_upper = 0.04) gave the sentence "smd = 0.50, 95% CI [0.03,
0.85]", read as the equal-tailed 95% interval, which is [0.10, 0.90].
Each constructor that takes alpha_lower and alpha_upper
(ci_c(), ci_cv(), ci_eta_squared(),
ci_eta_squared_generalized(),
ci_mahalanobis(), ci_omega_squared(), ci_pvaf(), ci_r(),
ci_R(), ci_R2(), ci_rc(), ci_reg_coef(), ci_rmsea(),
ci_sc(), ci_sm(), ci_smd(), ci_smd_c(), ci_snr(),
ci_src(), and ci_srsnr()) now records the two areas when they
differ, and:
results_sentence() writes "smd = 0.50, 95% CI [0.03,
0.85] (unequal tail areas: 0.01 below, 0.04 above)";alpha_lower = 0 and alpha_upper = 0.05 the sentence of the
ci_smd() call above was "smd = 0.50, 95% CI [-Inf, 0.83]" and is
"smd = 0.50, one-sided 95% upper confidence bound 0.83";
ci_rmsea(rmsea = 0.06, df = 24, N = 300, alpha_lower = 0.05, alpha_upper = 0) was "rmsea = 0.06, 95% CI [0.04, Inf]" and is
"rmsea = 0.06, one-sided 95% lower confidence bound 0.04"; and
ci_R2(R2 = 0.2, N = 100, p = 3, alpha_lower = 0.05, alpha_upper = 0) was "R2 = 0.20, 95% CI [0.08, 1.00]" and is "R2 = 0.20,
one-sided 95% lower confidence bound 0.08". The line beneath the
table says which limit is the bound ("one-sided: the upper limit is
the 95% upper confidence bound, with tail area 0.05 above it and
none below the lower limit"). An interval is taken to be one-sided
only from a tail area of 0, never from a limit, since an
equal-tailed interval can have a lower limit of 0 because the
statistic fell below its critical value;tidy() and glance() add alpha_lower and alpha_upper columns
after conf_level, holding the two areas on the rows of the
interval (NA on the other rows of a long tidy()), so that rows
stacked from several results keep their own split and their
sentence states it. A stack built with rbind() from an
equal-tailed and an unequal-tailed result stops on their different
columns rather than state either split for the other's rows, and
one built with dplyr::bind_rows() holds NA, read as equal
tails, on the equal-tailed rows;plot_ci() names the split in a subtitle (see the entry on
plot_ci()).An equal-tailed interval, the usual one, is unchanged: it records no
areas, prints and reads as before, and its tidy() and glance()
gain no column. No limit changes.
ci_c(), ci_sc(), ci_c_ancova(), ci_sc_ancova(), and
ci_c_ancova_bp() require the standard deviation of the errors and
refuse one that is not a single positive, finite number. The
standard error of a contrast is that standard deviation times a
function of the weights and the sample sizes (and, in ANCOVA, of the
covariate means), and it cannot be recovered from the means. Through
1.0.1 it was checked only on some paths:
ci_c(means = c(1, 2, 3), c_weights = c(1, -1, 0), n_per_group = c(10, 10, 10)), with no s_anova, returned a zero-width interval at the
estimate, -1, -1, and -1, labeled "Confidence level: 95%", with the
sentence "contrast = -1.00, 95% CI [-1.00, -1.00]";ci_c_ancova(adj_means = c(7.5, 12, 14), c_weights = c(1, -1, 0), n_per_group = c(10, 10, 10), cov_means = c(5, 6, 7), SSwithin_x = 750), with
no s_ancova, returned -4.5, -4.5, and -4.5 labeled 95%, and
s_ancova = 0 returned the same;s_anova = -1 in the ci_c()
call gave a lower limit of -0.08 above an upper limit of -1.92, and
s_ancova = -2 in the ci_c_ancova() call gave -2.66 above -6.34;ci_sc() with means and no s_anova, and ci_sc_ancova() with no
s_ancova, stopped with "argument is of length zero";
ci_sc_ancova() with s_ancova = 0 stopped with "lower < upper is
not fulfilled";ci_c_ancova_bp() stopped on an NA with "missing value where
TRUE/FALSE needed" and returned infinite limits for Inf.Each of these is now refused with a message that names the argument
and says what it is: s_anova for ci_c() and ci_sc() on every
path, s_ancova for ci_c_ancova() (either procedure),
ci_sc_ancova(), and ci_c_ancova_bp(), and s_anova for
ci_sc_ancova() when it is the standardizer. A call with neither the
means nor the contrast, which failed with "arguments imply differing
number of rows: 3, 0" in ci_c(), is refused in words. Every
interval formed from a valid standard deviation is unchanged.
plot_ci() titles a DMAR table with the level the table records.
The title came from the conf_level argument, used only for the
label, so a table formed at another level was titled with its
default: plot_ci(ci_smd(smd = 0.5, n_1 = 50, n_2 = 50, conf_level = 0.90)) was titled "95% Confidence Interval" above the limits of a
90% interval, and a 97.5% table the same. The level the table records
(its conf_level attribute, or a conf_level column holding one
value) is now used when conf_level is not given ("90% Confidence
Interval"), and a conf_level that differs from it is refused, since
the plot draws the limits it is given and cannot change their level;
rows that record several levels are refused as well. A frame that
records no level, and explicit estimate, lower, and upper
values, keep the argument's level, .95 by default. An interval with
unequal tail areas is drawn with a subtitle in the words of
results_sentence() ("Unequal tail areas: 0.01 below, 0.04 above",
or "One-sided: the upper limit is the 95% upper confidence bound").
The limits drawn are unchanged.
equivalence_c() has one more row and one more verdict. An
interval that contains -contrast_lower and whose upper limit is below
contrast_upper (delta_lower and delta_upper in 1.0.x) was labeled
"Inconclusive" through version 1.0.1 and is now "Not relevantly
superior only", in the verdict attribute and in
plot_equivalence()'s colors. The second example of the help page,
equivalence_c(contrast = -5.28, se = 2.49, df_error = 399, contrast_upper = 5), with 90% limits -9.39 and -1.17, reads so now.
"Inconclusive" is left for an interval that contains both bounds. Code
that reads the rows by term is unaffected; code that reads them by
position meets an 18th row, nonsuperior, after inferior, and the
first 17 keep their places. No estimate, limit, or p-value changes.
ss_power_R2() stops when null_R2 is not 0, and the test of an
increment in R^2 has its own function. ss_power_R2() plans the
omnibus test that the population squared multiple correlation coefficient
of all p predictors is zero. A nonzero null_R2, which DMAR 1.0.x
accepted, is now an error of class dmar_null_R2_unsupported (a
dmar_refusal) whose message says what MBESS calculated with a nonzero
null_R2 and writes the call to ss_power_R2_change() with the values
given. No
value is returned for such a call, in place of the following.
With fixed predictors, 1.0.x and MBESS formed
(true_R2 - null_R2) / (1 - true_R2), which is Cohen's
(1988) f^2 for the increment when predictors are added to a model that
already explains null_R2, and referred it to an F distribution with
p and N - p - 1 degrees of freedom and noncentrality N f^2. With p
read as the number of predictors added, that is Cohen's calculation for
a total sample size of N plus the number of predictors in the restricted
model: true_R2 = .30, null_R2 = .25, p = 4, alpha_level = .01, and specified_N = 85 gave a power of .2225 with a noncentrality
of 6.071, which is the value Cohen's formula gives at N = 90 with five predictors in the
restricted model, and the search gave 242 where Cohen's form gives 247.
With p read as all of the predictors, as the function defines it, the
number is not the power of the test of the increment, whose numerator
degrees of freedom are the number of predictors added: for .25 against
.20 with p = 5 and specified_N = 200 it was .8047, where the test of
one predictor added to four has power .953 and the test of two added to
three .911 (fixed predictors; .945 and .901 with random predictors), and
the search for power .80 gave 199, where those two tests need 120 and
148 (127 and 154 with random predictors). With random predictors, the
default, 1.0.x did not form that effect size: given true_R2 it
returned alpha_level when true_R2 was at or below null_R2 and the
power of the omnibus test above it (.0500 at true_R2 = .20 and .9999 at
.200001 with p = 5, specified_N = 200, and null_R2 = .20; a search
for .25 against .20 returned 47, the sample size of the omnibus test of
.25), and given true_f2 it took the population value as
(true_f2 + null_R2) / (1 + true_f2) and returned the power of the
omnibus test at it (.9975 for true_f2 = .1, null_R2 = .2, p = 5,
and specified_N = 100, where the same call without null_R2 gave
.6378). The argument alone cannot say which calculation is meant, because
the test of an increment needs the number of predictors in the
restricted model.
What to call instead. The test of the increment:
ss_power_R2_change(true_R2_full = a, true_R2_restricted = b, p_full = , p_restricted = ). The number 1.0.x returned with fixed predictors, and
MBESS returned: ss_power_R2(true_f2 = (a - b) / (1 - a), p = p, random_predictors = FALSE), which gives .8047 and 199 for the second
design above and .2225 and 242 for the first. A call that gave
true_f2 with a nonzero null_R2 is refused in the same way. With
fixed predictors 1.0.x took that true_f2 as the effect size of the
increment, so the same call without null_R2 returns its number; with
random predictors ss_power_R2(true_R2 = (true_f2 + null_R2) / (1 + true_f2), p = p) returns the number 1.0.x gave (.9975 in the case
above). When
true_R2 is at or below null_R2 there is no increment to test: 1.0.x
returned alpha_level (MBESS returns NaN), and the message says so in
place of the two calls, which would stop on a negative effect size.
Whether the population squared multiple correlation coefficient exceeds
a stated value above zero is a third question, for which DMAR has no
planner in this release; that test rejects when the one-sided lower
confidence limit from ci_R2() exceeds the value. null_R2 stays in
the signature so that an existing call is answered with the explanation,
and null_R2 = 0 is accepted. An invalid null_R2 (not a single number
of at least 0 and below 1) keeps its own message.
A search with true_R2 = 0 stops with "With 'true_R2' of 0
the null hypothesis is true, so power equals 'alpha_level' at every
sample size and no sample size attains 'desired_power'." The sample size
search doubles the size and then bisects, so a small effect returns
promptly (true_R2 = .00005 with p = 5, desired_power = .80, and
fixed predictors gives 256,545); the size it returns is the smallest that attains
desired_power, as before, and the search stops with an error at ten
million. No power is returned from an evaluation at which R's noncentral
F warned of lost accuracy ("full precision may not have been achieved
in 'pnbeta'"); with a null value of zero no call was found that reaches
that warning. Results with null_R2 = 0 are unchanged from DMAR 1.0.0:
on 1,500 combinations of true_R2 (from .000001 to .999), p,
specified_N, alpha_level, and predictor model, and 500 searches,
version 1.0.0 and this version returned the same sample size in every
search and powers that differ by at most 6e-11 (the entry below on a
very small true_R2 describes a true_R2 near 1e-8).
ss_power_R2() checks its arguments. alpha_level must be a
single number strictly between 0 and 1 (0 and 1 were accepted and
returned NaN, 0, or 1); true_R2 and null_R2 must be single
numbers of at least 0 and below 1 (true_R2 = 1 returned NaN
with an infinite effect size, and null_R2 = 1 a power of .05 with an
effect size of -1); p must be a whole number of at least 1 (p = 5.5 was accepted); true_f2 must be nonnegative; and a call with
neither true_R2 nor true_f2 says so, where it stopped with
"object 'f_Squared' not found". random_predictors must be TRUE or
FALSE; 1, 0, and the character strings "TRUE" and "FALSE" were
accepted and are now refused.
The random-predictor power of ss_power_R2() no longer returns
NaN for a very small nonzero true_R2. The parameters of
Lee's approximation need the square root of a difference that cancels
when rho^2 is small, and near rho^2 = 1e-8 the difference came out
negative: ss_power_R2(true_R2 = 1e-8, p = 10, specified_N = 12) returned NaN with the warning "NaNs produced" and now returns
.05. The root has a closed form with no subtraction, which the
function shared by ss_power_R2(), ci_R2(), and ss_aipe_R2() now
uses. The limits of ci_R2() and the sample sizes of ss_aipe_R2()
did not change on the grids compared (1,500 and 65 calls). The power
of ss_power_R2() changes by less than 1e-13 for a true_R2 of
.001 or more and by up to 9e-11 at 1e-6, where the earlier square root
carried the larger rounding error.
ss_aipe_semipartial_r() without true_R2_full returns a larger
sample size when true_r_sp squared is above 1/2. DMAR 1.0.x planned with
(1 - r_sp^2)^2, the variance when the variables partialed out explain
none of the outcome. That is the largest the variance can be when
r_sp^2 is at most 1/2, and there the plan is unchanged. Above 1/2 it is
the smallest, and the 1.0.x plan was too small unless the variables
partialed out explained none of the outcome: for r_sp = .8, one variable
partialed out (p = 2), and a width of .10 it
gave 202, where an interval of that width needs 331 when the full-model
R2 is .94 (at N = 202 the mean width in 4,000 replications was .127). The
plan without true_R2_full now uses the largest variance over every
possible full-model R2, r_sp^2 (1 - r_sp^2) when r_sp^2 is above 1/2:
necessary_N for that call was 202 and is 357; for r_sp = .9, p = 3,
and a width of .10 it was 59 and is 240; and for r_sp = .75, p = 2, a
width of .10, and assurance = .90 it was 329 and is 386 (with
seed = 113; the plan with assurance is now found by simulation, as
the entry on the assurance step describes).
true_R2_full = true_r_sp^2 restores the 1.0.x plan made without
assurance (with assurance it does not: the same call then returns
363, because the assurance step changed too).
ss_aipe_partial_r_sensitivity() and
ss_aipe_semipartial_r_sensitivity() simulate p - 1 variables
partialed out, the count their planners plan for, where DMAR 1.0.x
simulated one fewer for the same model. In 1.0.x the count was the
argument J (p = J + 1). It was documented there as the total number
of predictors and the simulated data had J + 1 columns, while the
planner each function calls, and the standard error of the interval each
one formed, took the same J as the number of variables partialed out.
With J = 1 the "partial" correlation simulated was the simple
correlation of two variables, given the interval of a partial
correlation with one variable removed. Both functions now simulate the
model the rest of the family describes: Y, X, and p - 1 variables
partialed out, so the simulated data have p + 1 columns. The planner
call and total_N are unchanged, and the results at a given seed
change, by little, since the interval already allowed for the variables
partialed out: with set.seed(113), true_cor = .40,
specified_N = 30, three variables partialed out (p = 4; J = 3 in
1.0.x), a width of .40, and G = 2000, the standard deviation of the
partial correlations was .1666 and is .1717, and with fisher_z = TRUE,
the interval 1.0.x formed, the proportion of intervals that missed the
true value was .048 and is .057 (Monte Carlo standard error .005). With
the default interval of this release, on the raw scale, that proportion
is .091, which also reflects the change of interval described in the
entry on fisher_z. This was a defect, and no argument restores the
earlier results; a model with nothing partialed out is
ss_aipe_r_sensitivity().
ss_aipe_semipartial_r_sensitivity() forms the interval the planner
plans for. Each replication's interval was formed on Fisher's Z
scale with standard error 1 / sqrt(N - p - 1), by analogy with the
partial correlation. No function of a semipartial correlation alone
stabilizes its variance, which depends on the full-model R2, and
ss_aipe_semipartial_r() plans the width of a Wald interval on the
correlation's own scale. Each interval is now that Wald interval: the
estimate plus and minus the normal critical value times the square root
of the first-order variance (var_semipartial_r() with
under_null = FALSE) at the sample semipartial correlation and the
sample full-model R2, with the limits not truncated at -1 and 1. In the
population 1.0.x simulated, in which the controls explain none of the
outcome, the two intervals nearly coincide: with set.seed(113),
true_r_sp = supposed_r_sp = .30, p = 4 (J = 3 in 1.0.x), a width
of .20, and
G = 2000, the mean width was .1988 and is .1986 at the same
total_N of 323. With true_r_sp = .80, p = 2 (J = 1 in 1.0.x), and a width of .10 the
planner's change above moves total_N from 202 to 357 and the mean
width from .1007 to .0754. This was a defect, and no argument restores
the earlier interval.
randomization_test() and randomization_test_paired() count tied
reassignments by the rounding of the arithmetic, and the studentized
statistic no longer drops the reassignments with no within-group
spread. A reassignment whose statistic equals the observed one
counts as at least as extreme. Earlier versions compared
with a tolerance relative to the largest reference statistic, formed
each variance as a sum of squares less n times a squared mean, and
set a statistic with a zero standard error to 0. The comparison now
allows the rounding bound of the statistic compared, which is
proportional to the size of the scores; variances come from residuals
about each group's own mean; and a reassignment in the reference set
whose standard error is zero is ordered as an infinite statistic with
the sign of its difference (0 when every score is equal). When the
observed data themselves have a zero standard error and a nonzero
difference, statistic = "t" is refused (see the entry on that
refusal below). The p-values that change:
randomization_test(group_1 = c(1, 1, 1, 1, 0), group_2 = c(0, 0, 0, 0, 1), statistic = "t") gave 0.198413 and gives 52/252 =
0.206349. For g_1 <- c(4.800000002, 4.8, 4.800000003, 4.800000001, 4.800000002) and g_2 <- c(8, 8.000000001, 8.000000003, 8.000000003, 8.000000003) it gave 0.003968, half the
smallest value ten scores can produce, and gives 2/252 = 0.007937
(the statistic reported for that call was -339,547,000 and is
-3,938,927,000, the value of the Welch formula for those scores).
randomization_test_paired(rep(0, 6), c(1, 1, 1, 1, 1, -1), statistic = "t") gave 0.1875 and gives 14/64 = 0.21875 (0.09375
and 7/64 = 0.109375 with alternative = "greater").statistic = "t".
randomization_test_paired(1:5, 1:5 + 2, statistic = "t") gave a
statistic of 0 and p = 1. The paired t has a zero standard
error there and no value, so the call is now refused (class
dmar_studentized_statistic_undefined) with a message that says
why and gives the p-value of the mean difference, 2/32 = 0.0625,
which statistic = "mean" returns.x <- c(1000000.002, 1000000.005, 1000000.000, 1000000.002, 1000000.001, 1000000.001) and y <- c(1000000.003, 1000000.007, 1000000.002, 1000000.001, 999999.999, 1000000.001),
randomization_test_paired(x, y) gave 0.75 and gives 0.8125, the
value for the same scores in thousandths without the offset. The
independent test was too small in the same way:
randomization_test(group_1 = 1000000 + c(2, 2, 2, 0) / 1000, group_2 = 1000000 + c(3, 3, 0, 7, 4) / 1000) gave 0.214286 and
gives 28/126 = 0.222222.randomization_test(group_1 = c(3e9, 4, 7, 2), group_2 = c(3, 5, 6, 9, 1)) gave 0.444444 and gives 35/126 = 0.277778;
randomization_test_paired(c(3e9, 4, 7, 2, 5), c(3, 5, 6, 9, 1))
gave 1 and gives 0.6875.randomization_test_paired(rep(0, 6), c(1, 1, 1, 1, 1 + 1e-12, -1)) gave 0.21875 (and 1 with statistic = "t") and
gives 12/64 = 0.1875 with either statistic.Scores with no ties and no extreme offsets give the p-values they
gave: 3,240 seeded calls on continuous data (exact and Monte Carlo,
both statistics, three alternatives, and procrustes_phi()) returned
the same p-value before and after the change. The limits of the shift interval
move by up to about two millionths of a standard error, because the
earlier tolerance counted near ties at the edge of the interval.
Checked against rational arithmetic in 1,800 comparisons on unbalanced
designs, the studentized p-value is exact for binary, three-valued,
Likert, three-decimal (with and without an offset of a million), and
constant scores; with one score of 3e9 among single digits, or scores
within a millionth of one of two values, statistics closer than the
rounding bound are counted as tied and the p-value was above the
exact one in 96 of 600 comparisons and below it in none. ?randomization_test says
so under "Ties".
The repaired statistic costs time, since each reassignment now keeps
the scores of both groups and forms its own residuals: with the
default 10,000 reassignments and the shift interval, a call with 150
scores per group and statistic = "t" takes about 2.8 s where it
took 0.16 s, and one with 1,500 per group about 26 s where it took
1.7 s (statistic = "mean": 0.30 s against 0.11 s, and 3.0 s against
1.1 s). Most of the time is the inversion for the shift interval:
with shift_ci = FALSE the two studentized calls take 0.27 s and
1.3 s.
randomization_test() with statistic = "t" reports the whole line
as the shift interval where no shift can be rejected. With three
scores per group the smallest two-sided p-value is 2/20, so the 95
percent interval is unbounded, as the help says. Under the studentized
statistic the earlier arithmetic lost the reassignments with no
within-group spread at large shifts and returned limits such as
-77,486,400 and 78,526,226 for group_1 = c(80, 84, 79),
group_2 = c(72, 75, 68); the limits are now -Inf and Inf, as
they were for statistic = "mean". The studentized statistic is now
computed on scores divided by a power of 2 near their largest absolute
value, so scores near 1e-200 or 1e200, whose squared residuals
underflow or overflow, give the statistic and p-value of the same
scores in ordinary units (c(3, 5, 9, 4) * 1e-200 against
c(1, 2, 2, 6, 3) * 1e-200 stopped with "missing value where
TRUE/FALSE needed"). The division is exact, and no other result
changes.
statistic = "t" is refused in randomization_test() and
randomization_test_paired() when the observed standard error is
zero and the observed difference is not. The studentized statistic
is the mean difference divided by its standard error. When both
observed groups are constant at different values, or every paired
difference is the same nonzero value, that standard error is 0 and
the statistic has no value. DMAR 1.0.0 reported a paired t of 0
with p = 1 for such differences (randomization_test_paired(1:5, 1:5 + 2, statistic = "t")) and, for two constant groups
(randomization_test(group_1 = rep(5, 4), group_2 = rep(3, 4), statistic = "t")), stopped with an error raised while the
standardized mean difference was computed. Both calls now stop with an error of class
dmar_studentized_statistic_undefined (a dmar_refusal), made
before any arrangement is enumerated or drawn, so a refused Monte
Carlo call leaves the random number stream where it was. The message
says that the denominator is zero, that the observed data lie at the
boundary of the reference set (of the 32 sign patterns, only the
observed one and its mirror image have a zero standard error; of the
70 reassignments of 4 and 4 scores, only the observed one and the one
that exchanges the groups), that this concerns the statistic and not
the evidence, and what statistic = "mean" gives: the mean
difference 2 with the exact p-value 2/32 = 0.0625 for the first
call and 2/70 = 0.02857 for the second. The condition carries that
value as p_value_mean, with reason ("constant_differences" or
"constant_groups"), alternative, and n or n_1 and n_2. The
value is the smallest the design allows in the direction of the data:
1/2^n one-sided and 2/2^n two-sided for n pairs; for a first
group of n_1 among N scores, 1/choose(N, n_1) one-sided, and
two-sided 2/choose(N, n_1) with equal group sizes and
1/choose(N, n_1) otherwise (1/56 for 3 against 5); it is 1 for
the alternative that points away from the data. When the call would
have been run by Monte Carlo sampling the message adds that such a
run estimates the value and cannot report less than
1/(n_resamples + 1): for 30 against 30 constant scores the exact
value is 2/choose(60, 30), about 1.7e-17, and statistic = "mean"
reports 1/10001 = 0.00009999. statistic = "mean" is unchanged in
every case. What is kept: a reassignment inside the reference set
whose standard error is zero is still ordered as Inf or -Inf,
because such reassignments occur in ordinary two-valued and coarsely
measured data and the limit puts them where the difference in means
does (group_1 = c(1, 1, 1, 0) against group_2 = c(0, 0, 0, 1)
with statistic = "t" returns t = 1.4142 and p = 34/70 =
0.4857, with one -Inf and one Inf among the 70 reference values,
the same p as the difference in means; randomization_test_paired( rep(0, 5), c(2, 2, 2, -2, 2), statistic = "t") returns t = 1.5 and
p = 12/32 = 0.375); and when every score is equal the statistic is
0 and p is 1 under either statistic, as before. The comparison that
decides whether the observed data are constant uses the tolerances
already in place, so c(5, 5, 5, 5 + 1e-12) against rep(3, 4) is
not refused and returns t = 8.0e12. ?randomization_test and
?randomization_test_paired have a paragraph, "When the studentized
statistic is undefined", and an example of the refusal.
The shift interval of randomization_test() rejects a shift whose
p-value equals 1 - conf_level. A shift is in the interval when
its p-value is above alpha, and alpha was computed as 1 -
conf_level, which for 0.90 is 0.09999999999999998 in binary
arithmetic, just below the p-value 1/10 that the test at the 0.10
level rejects. Shifts with exactly that p-value were therefore kept,
and the interval was too wide whenever alpha times the number of
reassignments is a whole number (at 0.90 with 10, 20, 70, or 210
reassignments; at 0.80 likewise). For group_1 = c(-1.5, 3.6, 0) and
group_2 = c(4.2, 3.0, -1.7) the 90 percent limits were -Inf and
Inf and are -5.7 and 5.3, and the 80 percent limits were -5.7 and
5.3 and are -4.5 and 1.7. The comparison now allows a few units of
rounding in alpha. Intervals at 0.95 and 0.99 are unchanged; the
defect was in 1.0.0 as well, and an interval was never too narrow.
randomization_test() returns its table when the standardized mean
difference is undefined. With no within-group variance
(group_1 = c(1, 1, 1), group_2 = c(2, 2, 2), or every score equal)
the call stopped inside ci_smd() with "lower < upper is not
fulfilled" or with "missing value where TRUE/FALSE needed". It now
reports the test, the shift interval, and Cliff's delta, with the rows
smd, smd_ci_lower, smd_ci_upper, cles,
cles_ci_lower, and cles_ci_upper as NA and a warning of
class dmar_smd_undefined. For rep(1.1, 4) against rep(0.2, 4)
with statistic = "mean" the statistic is 0.9, p is 2/70, and both
shift limits are 0.9; statistic = "t" is refused there, since both
groups are constant. A response that is not finite (Inf, -Inf)
is refused in both randomization tests ("must be finite"), where it
gave a base R error or a p-value of NA.
plot_randomization_test() shades the reassignments the p-value
counted. The figure used an absolute tolerance of 1e-12 of its own.
For the scores of the first help example multiplied by 1e-13 it shaded
every reassignment (share 1.000 beside p = 0.0238), and for
1e5 + c(.1, .4, .7, .2) against 1e5 + c(.3, .5, .6, .9) it shaded
0.2571 beside p = 0.3143. randomization_test() now stores the
logical vector it counted as the attribute reference_extreme, and
the plot shades those. A reassignment with an infinite studentized
statistic is not drawn, and the subtitle says how many are not, on a
second line. Infinite values occur in the reference distribution
only: the observed statistic is always finite, since
randomization_test() refuses the one case in which it is not, and
its line is always drawn. When every reassignment the p-value
counted is infinite, which can happen only in a Monte Carlo sample
(under enumeration the observed assignment is among those counted),
the subtitle gives p without calling it a shaded area and says that
none is shaded.
procrustes_phi() counts a permutation tied with the observed
arrangement. A permutation that exchanges two items with the same
loading reproduces the observed coefficient, but was compared with it
exactly and could fall one unit in the last place below. With
l1 <- c(0.6, 0.72, 0.65, 0.72, 0.81), l2 <- c(0.61, 0.76, 0.64, 0.68, 0.85), and set.seed(113), procrustes_phi(l1, l2, n_resamples = 60000) gave 0.00772 and gives 0.01637; 2 of the 120 permutations are
at least as extreme, 0.01667. The denominator is computed once and the
comparison allows 4(n + 2) units of machine precision. A call with
no tied loadings gives the p-value it gave.
n_resamples must be a whole number.
randomization_test_paired(x, y, exact = FALSE, n_resamples = 4.5)
drew four sign patterns and divided by 5.5, and procrustes_phi(l1, l2, n_resamples = 2.5) drew two permutations and divided by 3.5;
randomization_test() truncated 4.5 to 4, and procrustes_phi(n_resamples = -3) returned the coefficient with no message. Each now stops with
an error of class dmar_count_invalid unless the count is a single
whole number of at least 1 (the two randomization tests) or at least 0
(procrustes_phi(),
where 0 skips the test). A whole count gives what it gave. The count
is checked whether or not the call draws: randomization_test_paired(x, y, n_resamples = NULL) with eight pairs, which enumerates, returned
its table and now stops, as randomization_test() already did.
The check of n_resamples applies under exact enumeration as well,
where the count is not used.
Help corrections for the randomization tests and
procrustes_phi(). ?randomization_test_paired said that the
paired t is more robust to pair-to-pair variability than the mean
difference. Over the sign patterns the paired t is an increasing
function of the mean difference, so the two give the same p-value
for every data set and differ only in the statistic reported; the page
now says so, and says that exact enumeration is the default through 20
pairs and available through 25. ?randomization_test said that the
mean difference and the studentized statistic agree in a balanced
design with equal variances "to within the discreteness of the
reference distribution"; they give the same p-value in every
balanced design, whatever the variances, and differ only when the
groups are unequal in size, and the second example's comment no longer
recommends the studentized statistic for two groups of five.
?procrustes_phi called the permutation distribution a sampling
distribution and said the function reports the coefficient "with its
uncertainty"; the test takes the two sets of loadings as given, and
the function reports the coefficient and the permutation p-value,
with no standard error or interval.
mediate(): the bootstrap limits of the indirect effect change in
data where a resample makes the predictor or the mediator a linear
combination of the covariates. lm() returned a finite product
a b in such a resample, by dropping a covariate column in place of
the predictor, and that product entered the interval although the
effect is not estimable there. These resamples are now dropped and
counted (see "mediate() codes its covariates once" above). With a
binary predictor (8 of 40 cases) and a binary covariate that differs
from it in two cases, B = 2000, seed = 113: 248 of the 2000
resamples are of this kind; the percentile limits were
[-0.0401, 0.7202] over all 2000 and are [-0.0369, 0.6800] over the
1752 in which the effect is defined, with the warning of class
dmar_bootstrap_replications_dropped. The estimate (0.1925) and every
other row are the same. The same holds in a small sample with several
covariates, where a resample can hold fewer distinct cases than the
model has coefficients: with 12 cases, three continuous covariates,
and B = 600, 12 resamples are dropped and the limits, which were
[-1.397, 2.023] with 127 warnings from lm() about a perfect fit,
are [-1.177, 1.881]. A call whose resamples all leave a and b
estimable returns the limits it returned before, to rounding error
(relative difference below 1e-10; up to 1e-9 for a date-time
predictor, where the earlier fit lost digits to the size of the
origin). No argument restores the earlier limits.
mediate() refuses a logical outcome. A logical y was read as 0
and 1 and the linear model fit to it; the call now stops and says
that the column must be numeric. as.numeric() of the column gives
the earlier table. (A logical x or m stopped before, with
subscript out of bounds.)
mediate() returns 18 rows, not 17: B_used follows B. Code
that reads rows by term is not affected; code that reads the table
by position past row 17 is.
mediate(ci_method = "boot_bca") stops where it returned an interval
from an incomplete jackknife. With one case carrying all the
variation in the predictor (x = c(3, rep(0, 19)), B = 300,
seed = 113) it returned [-0.1173, 0.4605]; it now stops with
dmar_bca_undefined, naming case 1.
mediate() stops when a column is named more than once. A
covariate named twice returned the answer for the covariate entered
once; y among the covariates returned a table from a model with the
outcome among the predictors of the mediator. Both now stop.
nnt_from_smd() reports the confidence set for the number needed to
treat correctly when the interval for the standardized mean difference
includes zero, where DMAR 1.0.0 returned its two ends in reversed
order as if they bounded an interval. The reciprocal of an interval
for the success rate difference (SRD) that includes zero is two
intervals that meet at infinity, the NNT of no effect (Altman, 1998).
DMAR 1.0.0 put the two finite ends of that set in the rows nnt_lower
and nnt_upper, the lower above the upper and the estimate outside
them, with nothing to mark the case:
nnt_from_smd(smd = 0.3, smd_lower = -0.10, smd_upper = 0.70), the
fourth example of the help page (now nnt_from_smd(smd = 0.3, smd_ci_lower = -0.10, smd_ci_upper = 0.70)), returned nnt 5.953,
nnt_lower 2.636, and nnt_upper -17.739, and
nnt_from_smd(smd = 0.1, n_1 = 30, n_2 = 30) returned 17.739, 3.015,
and -4.417. In that case the NNT limits, now the rows nnt_ci_lower
and nnt_ci_upper, are NA, as
the NNT limits of responder_analysis() are, the set is in the new
nnt_confidence_set attribute (for the first call, the rows
(-Inf, -17.739) and (2.636, Inf)), the new nnt_set attribute is
"two_intervals", and the table prints the set beneath it: "NNT <=
-17.739 (harm) or NNT >= 2.636 (benefit), joined at infinity (Altman,
1998)". Read as an interval, the two rows of 1.0.0 held the true NNT
in none of the samples when the population difference is zero and in
34 percent of them at a population standardized mean difference of
0.5 with 20 per group (95 percent intervals); the set of two
intervals holds it at the stated level. When the upper limit of the
standardized mean difference is exactly zero, the lower NNT limit is
now -Inf, where 1.0.0 gave Inf (nnt_from_smd(smd = -0.3, smd_lower = -0.7, smd_upper = 0) returned nnt_lower Inf and
nnt_upper -2.636, and nnt_from_smd(smd = -0.3, smd_ci_lower = -0.7, smd_ci_upper = 0) returns -Inf and -2.636). A result whose interval
excludes zero, or has a
lower limit of exactly zero, is unchanged in its first seven rows,
with one exception at the edge of the arithmetic: the SRD is computed
without the subtraction that lost digits for a standardized mean
difference near zero, so results for |d| below about 1e-8 differ from
1.0.0 in their later digits (at d = 1e-12 the NNT was 1.772373e12 and
is 1.772454e12, which is sqrt(pi) / d), and a d so small that the SRD
rounded to zero keeps its sign (nnt_from_smd(-1e-200) gave an NNT
of Inf and gives -Inf). Every result has the leading class
dmar_nnt, which prints the set, ahead of dmar_tbl and
data.frame, and dmar_metadata()$result is "dmar_nnt"; code that
tests inherits(x, "dmar_tbl") is unaffected. tidy() and glance()
return the rows without the two attributes; the finite ends of a set
of two intervals are 1 / srd_ci_lower and 1 / srd_ci_upper.
srd_ci_lower and srd_ci_upper, are appended after
nnt_ci_upper: the confidence interval for the SRD, which is an
ordinary interval in every case and is the interval Kraemer and
Kupfer (2006) prefer. The seven rows of 1.0.0 keep their positions;
four are renamed (see Renamed Output Rows and Columns).smd must be a single finite number,
conf_level a single number in (0, 1), the limits and the sample
sizes each given as a pair or not at all, and smd_ci_lower not
above smd_ci_upper; DMAR 1.0.0 returned the point estimate without an
interval and without a message when one limit or one sample size
was given, accepted reversed limits, and printed "Confidence level:
700%" for conf_level = 7. An estimate outside its supplied limits
is accepted with a warning. Supplied limits are still used in place
of n_1 and n_2.conf_level is given in the call; DMAR 1.0.0 printed
"Confidence level: 95%" for limits whose level it did not know.The responder_analysis() help page no longer cites Altman (1998)
for withholding the NNT limits. The page said that an interval on
the NNT scale would mislead when the interval for the difference
includes zero. Altman (1998) argues for reporting it: the confidence
set is then two intervals that meet at infinity, with the reciprocals
of the difference limits as its two finite ends. The page now says
that, and that one lower and one upper limit cannot hold such a set,
which is why the two NNT limits are NA. No result changes.
responder_analysis() carries the confidence set for the number
needed to treat, as nnt_from_smd() does, and the help pages of both
functions are written out in full. When the interval for the
difference of the two responder proportions includes zero, the NNT
limits are NA, as before, and the set they could not hold is now
in the result. With two groups the table has the attributes nnt_set
("interval" or "two_intervals", one value per threshold) and
nnt_confidence_set (columns lower and upper, with threshold
first under sweep), on the signed scale 1 / difference, and
group_labels; for two intervals a line under the table states the
set. For the first example of the help page (30 of 60 control and 35
of 60 treated participants at a threshold of 10; difference 0.083,
interval -0.093 to 0.252; NNT 12 with NA limits), earlier versions
printed the table alone, and the table is now followed by "NNT <=
-10.806 (control responds more often) or NNT >= 3.966 (treatment
responds more often), joined at infinity (Altman, 1998)", the
reciprocals of the two limits of the difference. Every result has
the leading class dmar_responder, which prints that line, ahead of
dmar_tbl and data.frame, and dmar_metadata()$result is
"dmar_responder" where it was "dmar_tbl"; code that tests
inherits(x, "dmar_tbl") is unaffected. No row of any table changes
(152 calls compared with the build before this change: largest
difference 0). ?responder_analysis now states what is estimated
and how it differs from the NNT of nnt_from_smd(); the assumptions,
among them that the threshold is fixed before the data are seen and
that sweep describes dependence on the threshold and is not a
search; the Wilson and Newcombe formulas as computed; the
construction of the confidence set; what is returned when the
proportions are equal, when no one or everyone responds in one group
or in all (in the last case the chi square statistic is 0/0 and is
reported as NaN), for a group of one, for outcomes at the
threshold, for missing values, and for more than two groups; what
each row is; the example read number by number; and what
dichotomizing costs, from MacCallum, Zhang, Preacher, and Rucker
(2002): a split at the mean multiplies a correlation by 0.798 and its
square by 0.637, about the loss of 36 percent of the sample.
?nnt_from_smd now states the estimand in words; which assumptions
serve the point conversion and which the interval; the conversion
with a table of the SRD and the NNT for standardized mean differences
of 0.1 to 1.0, which agrees with Table 1 of Kraemer and Kupfer
(2006); case by case why the reciprocal of an interval that includes
zero is two intervals and why one pair of limits cannot hold it; the
edge cases; the threshold NNT beside this one in numbers (at a
standardized mean difference of 0.5: 3.62 here, and at least 5.07 at
any threshold under the same normal model); and two worked examples.
No result of nnt_from_smd() changes (154 calls compared: largest
difference 0).
vargha_delaney_A() stops on input it used to order by a rule the
user did not choose. It converted whatever it was given with
as.numeric(). An unordered factor was read through its integer codes,
which follow the alphabetical order of its labels:
vargha_delaney_A(factor(c("10", "9", "8")), c(10, 9, 8)) returned
A = 0 with limits [0, 0], and a five-category agreement item supplied
as an unordered factor through outcome ~ group returned the value of
its alphabetical codes. Two time differences in different units were
compared as bare numbers:
vargha_delaney_A(as.difftime(c(1, 2, 3), units = "hours"), as.difftime(c(30, 90, 150), units = "mins"))
returned A = 0 with limits [0, 0], where the value is 0.667. A Date
beside a POSIXct was compared as days against seconds. A matrix outcome
in a formula, cbind(s, t) ~ g, was strung out column after column under
a recycled row index, so that 3 rows per group were reported as
6 observations per group. Each of these calls now stops with an error of class
dmar_scores_not_ordered that says what to supply. Two time
differences in the same units, and a time difference outcome in the
formula interface, are on one scale and are ordered as before. Convert
time differences in different units with as.numeric(x, units = "hours"), and supply an ordinal
response as an ordered factor or as numeric scores.
probability_of_superiority_paired() reports both limits at the
estimate, with a warning, when every pair favors the same member. It
returned NaN for both limits and no warning:
probability_of_superiority_paired(c(1, 2, 3, 4), c(4, 5, 6, 7)) gave
an estimate of 1 with limits NaN, NaN, and now gives 1 with limits
1, 1 and the warning of class dmar_interval_degenerate, which says
that the limits are not a confidence interval with the stated coverage.
With a missing value it returned a table of NA; it now removes the
pair.
vargha_delaney_A() and cliff_delta() agree with their earlier
values to rounding, not to the last bit. The rank-based computation
gives the same A bit for bit. On 3,000 samples with and without ties
(2 to 60 observations per group), the largest differences from the
earlier results were 8e-17 in the standard error of A, 8e-16 in its
limits, 8e-15 in Cliff's delta, 2e-16 in its variance, and 2e-14 in its
limits; the proportions of pairs won, lost, and tied were identical.
These are the largest differences on the samples compared, not bounds.
Cliff's delta is now the difference of two exact proportions where it
was the mean of the signs of all pairs (on one sample of 3,000 per
group it was 0.170807999999525 and is 0.170808, the exact ratio). The
z statistic of vargha_delaney_A() moved by at most 1e-13 and its
p-value by at most 4e-16.
meta_es(), meta_smd(), meta_r(), and plot_forest() warn when
the effect sizes are identical under the Hartung-Knapp adjustment,
and the warning states what the conventional analysis gives. When
every effect size is the same, exactly or up to rounding error, the
weighted residuals are zero and so is the adjusted standard error:
meta_es(rep(.5, 4), rep(.04, 4)) returns a standard error of 0, a
t of Inf, and confidence and prediction limits of 0.5 and 0.5.
Those numbers are unchanged (they are what test = "knha" in metafor
returns). Earlier versions returned them with no message. The
functions now give one warning of class dmar_meta_hk_zero_se. It says that
the adjustment multiplies the conventional standard error by the
square root of q, the weighted mean square of the effect sizes
about the pooled estimate, and that q is 0 when the effect sizes do
not differ; it shows the limits (0.5, 0.5); it says that this is not
evidence that the effect is known exactly, since each effect size
has a sampling variance that agreement among four estimates does not
remove; and it states the conventional standard error and the normal
interval at the level of the call, 0.1 and 0.304 to 0.696 for that
call, which hartung_knapp = FALSE reports. The condition carries
n_studies, estimate, se_conventional, lower_conventional, and
upper_conventional (4, 0.5, 0.1, 0.3040036, 0.6959964). When the
common effect size is 0 the message says that the t statistic is
undefined (0 divided by 0), and when the standard error is only
numerically zero (meta_es(c(0.1 + 0.2, 0.3, 0.3), rep(.04, 3)),
standard error 2e-17) it says that the statistic is the estimate
divided by a rounding error. For meta_r() the limits and the normal
interval in the message and in the condition are correlations, as the
reported rows are, and the conventional standard error is on the
Fisher Z scale and is labeled so (meta_r(r = c(.3, .3, .3), n = c(40, 60, 90)): 0.07433 on the Fisher Z scale, normal interval
0.1624 to 0.4262). Effect sizes that differ, however little (0.5,
0.5, and 0.5 + 1e-9), do not raise the warning, and no warning is
given for a q between 0 and 1, which is the method working as
defined.
Meta-analysis results record q and the conventional standard
error. Whenever the Hartung-Knapp adjustment is applied, the tables
of meta_es(), meta_smd(), and meta_r() carry two new
attributes: hartung_knapp_q, the weighted residual mean square on
k - 1 degrees of freedom, and se_conventional, the standard error
without the adjustment (both on the Fisher Z scale for meta_r()).
The adjusted standard error is the square root of the first times
the second. For effect sizes of .30, .32, .29, .31, and .33 with
variances of .04, .05, .03, .06, and .04 they are 0.006860532 and
0.0912870929, and the reported standard error is 0.007561158. The
tables keep their 20 rows, and no reported number changes (1,104
fits compared with the build before this change: largest difference
0). ?meta_es has two new paragraphs. "When q is small" says that
the adjusted interval is narrower than the normal interval exactly
when q is below the squared ratio of the normal quantile to the t
quantile on k - 1 degrees of freedom (0.02, 0.21, 0.50, and 0.75
at k = 2, 3, 5, and 10 for a 95 percent interval), that a small
q belongs to the reference distribution of the method, and that
both intervals should be reported when the adjusted one is much the
narrower. "Identical effect sizes" describes the case the warning is
for.
The help of meta_es() no longer says that the Hartung-Knapp
adjustment keeps the confidence interval near its nominal coverage.
It now gives the adjusted standard error, the conventional one times
the square root of the weighted residual mean square q (Röver,
Knapp, & Friede, 2015), and says that the function applies the
unmodified method: q is not replaced by max(1, q), so when
q < 1 the interval is narrower than the normal interval. For effect
sizes of .30, .32, .29, .31, and .33 with variances of .04, .05, .03,
.06, and .04, q is 0.0069 and the interval is 0.287 to 0.329,
against 0.129 to 0.487 with hartung_knapp = FALSE. The help says
where the coverage of the adjusted interval is at or near the nominal
level (homogeneous or equally precise studies) and where it falls
below it (studies that differ greatly in precision, with heterogeneous
effects, where the normal interval covers less still), and that the
prediction interval uses the same standard error. No result changes.
The REML estimate of the between-study variance is now carried to
the maximum of the restricted likelihood, in whatever units the effect
sizes are. meta_es(), meta_smd(), meta_r(), and plot_forest()
with method = "reml" (the default) searched over tau2 itself with
the default tolerance of optimize(), the fourth root of the machine
epsilon. That search stops within about 0.00003 of the maximum in the
units of tau2, whatever the size of the effects. The search is now
made on tau2 divided by the scale of the data (the largest sampling
variance plus the variance of the effect sizes), stopping at a distance
relative to the estimate, so the estimate for effect sizes multiplied
by a constant c is c squared times the estimate for the original ones.
The change is small for effect sizes of the order of 1. On 600 random
sets of 2 to 15 studies with sampling variances between .005 and 3,
the largest distance of tau2 from the solution of metafor run to
convergence (control = list(threshold = 1e-12)) fell from 0.00003
to 0.0000004 (598 sets drawn afresh; the remaining distance is the
flatness of the restricted likelihood at its maximum), and no estimate
moved between zero and positive; on
those sets tau2 changed by at most 0.000028, the pooled estimate by
at most 0.00006, and its standard error by at most 0.00003 (0.04% of
its size). These are the largest changes on the sets compared, not
bounds. For the 12 simulated studies of the package's tests, tau2
was 0.0279437 and is 0.0279387, the pooled estimate was 0.416452 and
is 0.416453, and its Hartung-Knapp standard error was 0.080996 and is
0.080995.
The change is large for effect sizes in small units, where 0.00003 is
of the size of tau2: the earlier estimate could be zero where the
maximum is positive, or several times too large. For ten simulated
risk differences near .02 with 4,000 to 40,000 per group (sampling
variances near 1e-5; set.seed(2718) in the test file), tau2 was 0
and is 0.0000090755, which is metafor's converged value; the pooled
estimate was 0.021151 and is 0.021255, I2 was 0 and is 52.2 percent,
and the prediction interval was 0.0181 to 0.0242 and is 0.0137 to
0.0289. On 300 random sets with sampling variances between 1e-6 and
1e-4, the earlier estimate disagreed with metafor on zero against
positive in 76 and was off by more than 1 percent in 184; the
estimate now disagrees in none and is within 0.002 percent in all.
An analysis of risk differences, proportions, or other effect sizes
with sampling variances below about .001 should be rerun.
The estimate is now accurate relative to itself, including when one
study's sampling variance is far above the others' (1.0.0 stopped
within about 0.00003 of the maximum in the units of tau2, about 0.5
percent of a tau2 near .01): for the twelve studies of the package's
tests with one sampling variance of 1e6 (.outlier_yivi() in the test
file), tau2 was 0.012048 and is 0.012058, the root of the score
equation; the pooled estimate, -0.0510, and its Hartung-Knapp standard
error, 0.0328, are unchanged to the digits shown.
The results of the other estimators are unchanged by this entry (the
Paule-Mandel estimate and the Q-profile interval for tau2 are the
subject of an entry under Bug Fixes). metafor at its default
threshold of 0.00001 gives 0.0279436 for the 12 studies above, so a
comparison with its default output can differ in the fifth significant
digit of tau2 where it agreed before.
meta_es(), meta_smd(), and plot_forest() refuse an infinite
effect size or sampling variance with a message that says so, and
meta_r() and meta_smd() an infinite sample size. Under
method = "reml", meta_es(c(.1, Inf, .3), rep(.1, 3)) stopped with
"invalid 'xmin' value" from the search and an infinite variance with
"invalid 'xmax' value"; under "pm" and "dl" both stopped with
"missing value where TRUE/FALSE needed". They now stop with "Supply
finite effect sizes" and "Supply a finite sampling variance for each
effect size". Under method = "fe" these calls returned a result and
now stop as well: meta_es(c(.1, .2, .3), c(.1, Inf, .1), method = "fe") gave an estimate of 0.2 (the study with an infinite variance
had a weight of zero), and an infinite effect size gave an infinite
estimate. To pool without a study, leave it out. An infinite n in
meta_r() stopped with "missing value where TRUE/FALSE needed" or,
under method = "fe", returned an estimate of NaN with a standard
error of 0 and no message; an infinite n_1 or n_2 in meta_smd()
stopped with a message about the effect sizes. Each now stops with
its "must give an integer sample size" message.
moments_nc_t() is right at large degrees of freedom and at a large
noncentrality parameter, where DMAR 1.0.0 lost digits, changed the sign
of the excess kurtosis, or returned a negative variance. DMAR 1.0.0
formed the variance, skewness, and excess kurtosis as differences of
the raw moments, which grow with the noncentrality parameter while the
differences do not, so the digits left fell as either argument grew:
the excess kurtosis was off by 1e-7 of its value at 1000 degrees of
freedom and a noncentrality parameter of 5 and by 4e-4 at 2000 and 30,
and moments_nc_t(df = 1e4, ncp = 100) returned -0.0022675 where the
value is 0.0013340. moments_nc_t(df = 1e16, ncp = 2.5) returned a
mean of 1.97e14 and a variance of -3.9e28 (now 2.5 and 1),
moments_nc_t(df = 20, ncp = 1e80) an excess kurtosis of NaN, and
moments_nc_t(df = Inf, ncp = 2) NaN for the mean, variance,
skewness, and excess kurtosis and NA for the standard deviation. The mean now
comes from the logarithm of the chi mean factor that sd_unbiased()
uses, and the other three from the central moments of the square root
of the degrees of freedom over a chi square variate, so that no raw
moments are subtracted; from 100 degrees of freedom on, the third
central moment and fourth cumulant of that square root are their
expansions in the reciprocal of the degrees of freedom. Against values
computed to several hundred digits, for degrees of freedom from just
above each boundary (1, 2, 3, and 4) to 1e300 and a noncentrality
parameter up to 1e6 in magnitude, the largest relative errors found are
3e-16 for the mean, 9e-16 for the variance, 4e-14 for the skewness, and
1.3e-12 for the excess kurtosis; the same limits hold at a
noncentrality parameter of 1e10, 1e100, and 1e300. df = Inf now
returns the normal distribution with mean ncp and variance 1, and an
infinite ncp is refused. For degrees of freedom up to 300 and a
noncentrality parameter up to 5 in magnitude the values differ from
those of 1.0.0 by at most 1.5e-8 of their size (df = 20, ncp = 2.5
agrees to ten digits).
cv(), ci_cv(), sd_unbiased(), ci_proportion(), and
power_equivalence_c() refuse inputs for which DMAR 1.0.0 returned a
number that meant nothing or an error from inside base R. Each
refusal names the argument. In cv(): a negative sd
(cv(mean = 2, sd = -1) returned -0.5), a mean of 0
(cv(mean = 0, sd = 1) returned Inf), cv given with one of mean
and sd (cv(cv = .2, mean = 999) returned 0.2 and ignored the mean),
a cv that is not one finite number (cv(cv = "a") returned a table
holding "a"), an N of 1 or less, or an infinite N, with
unbiased = TRUE (cv(cv = .2, N = 0, unbiased = TRUE) returned
Inf, and N = Inf returned the uncorrected 0.2), and an
unbiased that is not TRUE or FALSE (unbiased = "yes" returned
the uncorrected value). cv() documented single values and now
requires them: a vector cv, mean, sd, or N, and unbiased = 1,
were accepted and are refused (cv(cv = .2, N = c(10, 20), unbiased = TRUE) returned two rows). A negative mean still gives a negative
coefficient. In ci_cv(): a coefficient of variation that is not
positive, whether given as cv, through sd, or by the data
(ci_cv(cv = -.2, N = 20) and ci_cv(mean = 10, sd = -2, N = 20)
returned the limits -0.298 and Inf, and
ci_cv(data = c(-1, -2, -3, -4)) returned -6.859 and Inf), data with
a missing or infinite score or fewer than two scores, data that are
not numeric (a logical vector was read as 0 and 1), and an N that
is not one finite number above 1; ci_cv(cv = 0, N = 20),
ci_cv(cv = .2, N = Inf), and ci_cv(data = c(1, 2, NA, 4)) stopped
with "missing value where TRUE/FALSE needed". A fractional N above 1
is still accepted. In sd_unbiased(): a negative or infinite s
(sd_unbiased(s = -1, N = 20) returned -1.013, and s = Inf returned
Inf), s without N (which stopped with "arguments imply differing
number of rows"), a single score as X (sd_unbiased(X = 5) returned
NA), scores X that are not numeric (a logical vector was read as 0
and 1), an N of 1 or less, a single s or N that is missing, and
vectors s and N of different lengths other than 1. One
inadmissible element refuses the whole call: sd_unbiased(s = 2, N = c(10, 1)) returned 2.056 and NaN and now stops. A missing element
of a vector still gives a missing row (sd_unbiased(s = c(1, NA), N = 10) returns 1.028 and NA). A fractional N and vectors of equal
length give what they gave in 1.0.0; an infinite N gives s
(1.0.0 returned NaN). In ci_proportion(): an infinite count
(ci_proportion(1, Inf) returned the limits 0 and 0). In
power_equivalence_c(): a missing or infinite true_contrast,
n_per_group, true_sd, bound, or df_error, and a missing
alpha_level (true_psi = NA_real_, an infinite per-group size, and
delta_upper = Inf, in the 1.0.x names, each returned a power of 0,
where n_per_group = 1e12 and contrast_upper = 1e6 return 1; a missing
per-group size, true_sd, df_error, or alpha_level given as
NA_real_ stopped with "missing value where TRUE/FALSE needed"). A
fractional n_per_group or df_error gives what it gave.
ss_aipe_cv_sensitivity() leaves a replication whose sample mean is
not positive out of its width and coverage summaries, where DMAR 1.0.0
entered it with the limits ci_cv() gave for a negative coefficient,
an infinite upper limit among them. Such a sample has no confidence
interval for the coefficient of variation, which ci_cv() now refuses. The replication is
left out of every statistic row, mean_cv, median_cv, and sd_cv
included (see the next entry): all ten are taken over the
replications that have an interval; one warning of class
dmar_replications_without_interval states how many were left out,
and the count is attr(x, "simulation")$failed. The
case needs a large coefficient of variation and a small sample: with
true_cv = 1 and specified_N = 10 the sample mean is not positive in
about 8 of 10000 replications. With set.seed(7) and G = 3000 there,
3 replications are left out: pct_ci_miss_low was 0.0287 and is
0.0277, total_type_I_error was 0.0553 and is 0.0544, and
median_ci_width was 3.743 and is 3.735. mean_ci_width was Inf
there in 1.0.0, since ci_cv() has no finite upper limit for some
samples with a positive mean, and is now taken over the finite widths
(see the next entry). A study in which every sample mean is
positive, which is every study at the coefficients of variation the
help page and the vignette use, differs from DMAR 1.0.0 only where the
limits of ci_cv() changed with the exact noncentral t (see Exact
Noncentral t and Studentized Range Distributions).
ss_aipe_cv_sensitivity() summarizes the same replications in every
row. A replication whose sample mean is not positive has no interval,
and its sample coefficient is negative. All ten statistic rows, mean_cv,
median_cv, and sd_cv among them, describe the replications with a
positive mean (DMAR 1.0.0 entered such a replication in every row; see
the entry above). An interval with an infinite upper
limit made mean_ci_width infinite and sd_ci_width undefined; the
two are now over the finite widths, median_ci_width stays over every
width, an infinite width counts as not meeting the target (as it did),
and the number of infinite widths is
attr(, "simulation")$n_infinite_width. DMAR 1.0.0 repeated the
warning ci_cv() gave for such an interval in every replication that
had one. ci_cv() no longer gives that warning (see the entry on
ci_cv() and an infinite upper limit), and the sensitivity function
gives one warning of its own, of class
dmar_replications_with_infinite_limit, that states how many
replications had an infinite upper limit and how they enter the rows
(the count is also the field n_replications of the condition). With
set.seed(88), true_cv = 3,
specified_N = 5, width = .5, and G = 500 (114 replications
without an interval, 346 infinite widths): mean_cv -3.499 in DMAR
1.0.0 and 4.870 now, median_cv 1.400 and 1.884, sd_cv 118.98 and 22.16,
mean_ci_width Inf and 14.85, sd_ci_width NaN and 37.25. With
true_cv = 1, specified_N = 10 (no replication without an interval,
96 infinite widths): mean_ci_width Inf in DMAR 1.0.0 and 15.55 now,
sd_ci_width NaN and 188.3. A simulation in which every mean is
positive and every width finite differs from DMAR 1.0.0 only through
the exact noncentral t limits of ci_cv(). The table
also carries the simulation attribute (replications attempted, with
and without an interval, and exact binomial limits for the share that
met the width) and prints the counts beneath it. In the first
example the coverage rows differ from those of DMAR 1.0.0 as well, for
the reason of the entry above.
The number of replications G must be a whole number, in every
function that has the argument. The 31 sensitivity functions with a
G argument, ss_aipe_indirect_effect(), ss_aipe_composite_sem(),
and ss_power_composite_sem() check it with the one check of a count
(an error of class dmar_count_invalid), before anything is simulated.
The smallest G is what it was: 10 for the three planners, 2 for
ss_seq_c_sensitivity() and ss_aipe_crossed_sensitivity(), and 1
elsewhere. In DMAR 1.0.0 most of them
did not check G: ss_aipe_smd_sensitivity(G = 0) returned a full
table from one replication and G = 2.7 ran two,
ss_aipe_r_sensitivity(G = 0) returned a table of NA,
ss_aipe_cv_sensitivity(G = 0) stopped with "subscript out of
bounds", and G = -3, Inf, NA, or a vector stopped with "invalid
'times' argument"; ss_seq_c_sensitivity() accepted G = 2.5. The
messages of the functions that did check G now read "'G' must be a
single whole number of at least 1." (or 2, or 10), in place of "'G'
must be a positive integer.", "'G' must be a single positive whole
number.", "'G' must be a single number of at least 2.", and "'G' must
be a positive whole number of attempted studies.".
The six ss_aipe_crd_es_*() planners check G (nrep in 1.0.x) the same way at
entry: a fractional number, 0, a negative number, NA, a character
string, an infinite value, or more than one value is an error of class
dmar_count_invalid, before any simulation starts (earlier such a
value reached the simulation unchecked). G = NULL
remains the default of the two planners that fix a size and a budget,
where it skips the simulation. Results for a valid G are
unchanged.
The Hofmann (1977) entry of simple_structure() gives the issue and
pages of the article. The help page gave The Journal of General
Psychology, 96, 58--66; the DOI record of the article
(10.1080/00221309.1977.9920803) has 96(1), 103--110. The entry now
gives that issue and those pages with the DOI, and the Hofmann (1978)
entry on the same page gains its DOI, 10.1207/s15327906mbr1302_9, which
fa_exploratory() already cites. The title, the journal, and the
1978 volume, issue, and pages, 13(2), 247--250, already matched the
records and are unchanged.
The Owen (1968) entry of moments_nc_t() gives the DOI of the
article. The entry carried the DOI of a letter to the editor in the
preceding issue (10.1080/00401706.1968.10490590); it is now
10.1080/00401706.1968.10490594.
The Phillips (1990) entry gives the first page of the article. The
help pages of power_equivalence_c(), power_equivalence_md(),
power_density_equivalence_md(), and power_equivalence_md_plot()
gave the pages as 139--144; they are 137--144.
The correction factor of Hedges (1981) is right at a very large
sample size, in smd(), smd_c(), meta_smd(), expected_smd(), and
var_smd(). The factor J(df) = Gamma(df/2) / {sqrt(df/2)
Gamma((df - 1)/2)} was computed as a difference of two lgamma() values
that nearly cancel at a large df, the form that
design_consequences(), sd_unbiased(), and var_ete() gave up earlier
in this release (the entry on E(S)/sigma). It is now formed from the
logarithm of E(S)/sigma on df - 1 degrees of freedom, log J(df) =
L(df - 1) + log1p(-1/df)/2, two negative terms with nothing to cancel,
through one internal helper that the five functions share. Against a
60-digit evaluation J is right to a relative 3.4e-16 from 2 df to
1e14; the earlier form was off by a relative 5e-11 at 1e5 df and
8e-9 at 1e7, and at 1e8 it returned a J above 1. At ordinary sample
sizes nothing a user sees moves: over 272 planning sets with at most
10,000 per group the largest change in any reported value is 2.6e-10
(in var_smd()), except in meta_smd() under REML, whose iteration
stops at a tolerance that lets the estimate move by about 1e-9 and I^2
by about 4e-6 of a percentage point (no printed digit moves), and with
20 per group smd(unbiased = TRUE) and
expected_smd() agree with 1.0.x to all printed digits. Two results
that subtract nearly equal numbers were wrong much sooner than J
itself, and are now taken from the logarithm with expm1():
the bias row of expected_smd(), true_smd (1/J - 1), and the exact
variance in var_smd(), which is now computed as
df/(df - 2) + lambda^2 {2/(df - 2) - (1/J^2 - 1)}.
expected_smd(true_smd = 0.5, n_1 = 1e8, n_2 = 1e8) reported a bias
of -1.19e-07 (the wrong sign) and now reports 1.875e-09;
var_smd(true_smd = 3, n_1 = 1e8, n_2 = 2e8) reported an exact
variance of -8.5e-06 and now reports 3.0e-08; var_smd(true_smd = 0.5, n_1 = 1e8) reported 1.42e-07 and now reports 2.06e-08; with 100,000
per group the bias of expected_smd(true_smd = 0.5) moves from
1.875105e-06 to
1.875029e-06. Against the 60-digit values the bias and the exact
variance are now right to a relative 7e-16 over 344 planning sets with
up to 2e8 observations; the earlier relative errors there reached 190
and 285. No argument is added.
moments_nc_F() is right at large degrees of freedom. The
variance, skewness, and excess kurtosis were formed as differences of
raw moments, which agree in their leading digits when the degrees of
freedom are large, so the digits that remained were rounding error.
moments_nc_F(df_1 = 1e6, df_2 = 1e6) returned an excess kurtosis of
-5.285e-05 where the value is 6.60006e-05, and a skewness of
0.0060000263 where the value is 0.0060000230. The function now writes
the statistic as (df_2/df_1) X Y with X the noncentral chi square and Y
the reciprocal of the central one, and forms the central moments of
F/E(F) from those of X/E(X) - 1 and Y/E(Y) - 1, which are independent
and have mean 0: every term of each sum is nonnegative, so nothing
cancels (the expressions are on the help page; they were derived for
DMAR and checked against exact rational arithmetic). In 14 cases with
degrees of freedom up to 1e12 and noncentrality up to 1e6 the four
moments are right to a relative 5e-16, where the earlier form was off
by a relative 1.8 at (1e6, 1e6, 0) and 0.36 at (3, 1e6, 1e6). At
ordinary degrees of freedom nothing a user sees moves: over 918
combinations with both degrees of freedom at most 200 and noncentrality
at most 50 the largest relative change is 8.5e-12, and the moments that
do not exist are NA in the same places. The mean is unchanged.
rbind() of DMAR tables no longer carries the first table's
confidence level over rows computed at another. A dmar_tbl keeps
its confidence level, and the other facts its printed lines are
written from, as attributes, and rbind() kept the attributes and the
class of the first table: rbind(ci_smd(..., conf_level = 0.90), ci_smd(..., conf_level = 0.95)) was a table of class dmar_ci_smd
that printed "Confidence level: 90%" beneath six rows, three of them
95 percent limits, and dmar_metadata() of it gave 0.90. rbind() now
has a method for these tables. When every table stacked is a DMAR
table and all have the same classes and attributes, the stack is a
DMAR table with those attributes, as before. In every other case
(different confidence levels, different functions, or a DMAR table
stacked with a plain data frame, in either order) the result is an
ordinary data.frame with the same rows and columns and none of the
DMAR attributes, and no warning: the numbers are all there, and no
printed line states a level that does not hold for every row. To
compare results across levels, stack tidy() rows, which carry the
level in a column: rbind(tidy(x_90), tidy(x_95)).
dplyr::bind_rows() does not call the method. Code that relied on the
class of a mixed stack (inherits(rbind(a, b), "dmar_tbl") with
tables at two levels) now gets FALSE. ?dmar_tbl has a section on
stacking.
ss_aipe_R2() with assurance returns a sample size for a small
population R2, where it stopped with "invalid 'xmin' value". The
procedure inverts the expected value of R2 by matching a value rounded
to two decimals against a grid of expected values. When the one-sided
lower limit it uses is 0, which happens for a small population R2, no
grid value has an expected R2 that low, the match was missing, and
optimize() was given a missing end point. A missing match is now
replaced by the end of the grid when the target lies beyond it and by
the grid value with the nearest expectation otherwise.
ss_aipe_R2(true_R2 = .03, p = 1, width = .2, assurance = .9)
returns 113; of 30 conditions drawn at random with a true_R2 of
.03 to .90, the 8 that stopped now return (107 to 173) and the 22 that
returned give the same sample sizes, as do 40 further conditions that
returned before and 48 without assurance (largest difference 0).
The default p-value of vargha_delaney_A() is larger. The test
statistic is referred to Student's t with Satterthwaite degrees of
freedom where DMAR 1.0.x referred it to the standard normal
distribution (see "The Order-Based Effect Sizes Share One Contract").
The estimate, its standard error, the statistic, and the confidence
interval are unchanged, and test = "z" gives the 1.0.x p-value: on
240 data sets by three levels and two interval methods the eight
numbers of a test = "z" row equal the earlier ones exactly. For the
first example of the help (set.seed(113), 40 per group) the
p-value is 0.0082 on 77.96 degrees of freedom where it was 0.0067;
for vargha_delaney_A(iq_8 ~ treatment, data = pygmalion) it is
0.0892 on 92.48 degrees of freedom where it was 0.0858; and for
x = c(3.5, 5, 6, 7, 8), y = c(1, 2, 3, 4, 4.2) it is 0.0033 on
6.63 degrees of freedom where it was 0.0000076 (the exact permutation
p-value is 8/252 = 0.032). results_sentence() reports the
p-value of the test that was run. The table has two more
columns, df and test, after the nine it had.
Every sensitivity function summarizes its replications over one
denominator and records the counts. The proportion of intervals that
met the width target (pct_ci_less_width), the proportions that missed the
true value, and the summaries of the widths are taken over the
replications that produced a confidence interval, in every
ss_aipe_*_sensitivity() function; a replication without an interval
is in none of those rows. In ss_aipe_reliability_sensitivity(), where a
simulated study can end without an interval, DMAR 1.0.0 took the width
rows over the studies that returned one, as now, and the three rows
for intervals that missed the true value (pct_ci_miss_low,
pct_ci_miss_high, total_type_I_error) over all G, so that a study
without an interval counted as one whose interval held the value: with
two of four simulated studies returning an interval and one of the two
missing the value, the row is 1/2 where DMAR 1.0.0 gave 1/4, and with
no study returning an interval the three rows are NA where it gave 0.
ss_aipe_cv_sensitivity() and ss_aipe_cliff_delta_sensitivity() have
entries of their own. The rule changes nothing in a run in which every
replication has an interval. All 31 sensitivity functions, the two
ss_power_*_sensitivity() functions and ss_seq_c_sensitivity()
included, now attach the attribute simulation, a list with
attempted (G), returned (replications with an interval, or a
test), failed (those without), denominator (which is returned),
width_target_probability (the pct_ci_less_width row), and
width_target_mc_ci (exact binomial limits for it over returned,
which describe simulation error); the two width entries are NA for
a function with no width target. ss_aipe_sem_path_sensitivity()
adds n_redrawn, the data sets whose fit did not converge and were
drawn again (such a data set is replaced, so it is not among the
replications without an interval; the help says how the count is
read). The record is the one place the counts are kept: the rows
suc_rep of ss_aipe_rmsea_sensitivity() and
ss_aipe_sem_path_sensitivity() and num_probs_with_cis of
ss_aipe_R2_sensitivity(), which held the same counts in DMAR 1.0.x,
are removed (returned and failed hold them). The line "Replications" beneath each printed table gives
G, and the three counts when a replication was without an interval. dmar_metadata() returns the record as simulation.
ss_aipe_cliff_delta_sensitivity() no longer counts a completely
separated replication as an interval of width 0. When the two
simulated groups do not overlap, the estimated variance of Cliff's
delta is 0 and cliff_delta() reports both limits at the estimate,
which are not a confidence interval. DMAR 1.0.x took them as an
interval of width 0, which met every width target, and scored its
coverage from the point estimate. Such a replication now has no
interval: its estimate still enters mean_cliff_delta,
median_cliff_delta, and sd_cliff_delta, and the width,
width-target (pct_ci_less_width), and coverage rows describe the
replications with an interval. One warning of class
dmar_replications_without_interval gives the count (fields
n_without_interval and G), the per-replication warnings are not
passed on, the table carries the simulation attribute with the
counts and exact binomial limits for the width proportion, and the
file named in filename records such a replication with its estimate
and NA elsewhere. With set.seed(113),
ss_aipe_cliff_delta_sensitivity(true_delta = 0.9, specified_N = 12, width = 0.4, G = 400) has 162 separated replications of 400.
pct_ci_less_width is 0 where it was 0.405; mean_ci_width 0.966 where
it was 0.575; median_ci_width 0.909 where it was 0.867;
sd_ci_width 0.112 where it was 0.483; pct_ci_miss_low 0 where it
was 0.405; pct_ci_miss_high 0.025 where it was 0.015; and
total_type_I_error 0.025 where it was 0.420. The three rows of the
estimate are unchanged (0.904, 0.944, 0.119). A run in which no
replication is separated returns what it returned: 36 conditions
under a common seed are identical. No argument restores the earlier
result.
results_sentence() does not write limits that are not a confidence
interval as one. For a table of vargha_delaney_A(),
cliff_delta(), or probability_of_superiority_paired() whose
estimated variance is 0, the sentence gives the estimate and the
reason. results_sentence(vargha_delaney_A(c(4, 5, 6), c(1, 2, 3)))
is "A = 1.00 (no confidence interval: the groups are completely
separated, so the estimated variance is 0)" where it was "A = 1.00,
95% CI [1.00, 1.00]"; for cliff_delta() on those groups
"cliff_delta = 1.00 (no confidence interval: the groups are
completely separated, so the estimated variance is 0)" where it was
"cliff_delta = 1.00, 95% CI [1.00, 1.00]"; for
probability_of_superiority_paired(c(1, 2, 3, 4), c(4, 5, 6, 7))
"probability_of_superiority = 1.00 (no confidence interval: every
pair favors the same member, so the estimated variance is 0)" where
DMAR 1.0.0 wrote "probability_of_superiority = 1.00, 95% CI [NaN,
NaN]"; and
for vargha_delaney_A(c(2, 2, 2), c(2, 2, 2)) "A = 0.50 (no
confidence interval: every score is equal, so the estimated variance
is 0)" where it was "A = 0.50, 95% CI [0.50, 0.50]". A p-value is
appended only when the table has one (the permutation test). The
sentence of a tidy() or glance() row of such a table reads "A =
1.00 (no confidence interval: the estimated variance is 0)". The
sentence of every other table is unchanged.
A bootstrap or Monte Carlo interval asked for with fewer
replications than the rule requires is refused (see One Rule for
Bootstrap Replications): fewer than 200, 400, or 2000 at the 90, 95,
or 99 percent level for a percentile or Monte Carlo interval, fewer
than 1000, 2000, or 10000 for a BCa interval, and fewer than 100 for a
bootstrap standard error interval or a Bollen-Stine p-value. Calls
that ran in 1.0.x and now stop include
reliability_alpha(ci_method = "percentile", B = 40) (0.6083 [0.5231,
0.6846] before; with B = 400 the function returns [0.5203, 0.6709],
as it did), reliability_alpha(ci_method = "bca", B = 500) ([0.5204,
0.6692] before; with B = 2000, [0.5348, 0.6768], as it did),
reliability_alpha(ci_method = "bootstrap_se", B = 50),
htmt(B = 100) to htmt(B = 199), parallel_analysis(B = 20),
sem_exact_fit_bootstrap(B = 20) (p = 0.05 from 20 samples before),
R2_mixed_effects(ci_method = "boot", B = 20), and
mlmr(ci_method = "boot", B = 20). The error has class
dmar_count_invalid and names the number;
options(dmar.bootstrap_exploratory = TRUE) runs the call and marks
the result.
An interval formed from fewer than half of the replications
requested is no longer reported. The estimate is returned with NA
limits and a warning of class dmar_bootstrap_insufficient. In
mediate(), the kappa functions, krippendorff_alpha(), htmt(),
the ratio functions, average_variance_extracted(), and
fraction_missing_information() an interval was formed from any 100
replications that remained (two in the last), and in the reliability
family through 1.0.1 from any two. An interval from at least half of
those requested is reported as before, since the number required is
held to the request and not to the replications that remain, and the
warning now has a class, states the counts, and says when those that
remain are fewer than the number required. With two raters and eight
subjects, six of whom both raters place in one category,
cohen_kappa(ci_method = "percentile", B = 400, seed = 113) returns
0.6 [0.0, 1.0] from the 361 resamples in which kappa is defined, as
1.0.x did (and ci_method = "bca", B = 2000 returns [0.0, 1.0] from
1787); krippendorff_alpha(boot = TRUE, B = 400, seed = 113) on five
units of which four are alike returns [1, 1] from 254 resamples;
htmt(B = 200, seed = 113) with an item that is nonzero in one of 12
cases returns the bound 1.1723 from 130 resamples; and
ci_ratio_of_means(ci_method = "percentile", B = 400, seed = 113)
with a denominator group that is nonzero in one of 12 cases returns
3.0 [0.3333, 6.0] from 269 replications. Each of these now warns that
fewer than the number required remained and says to request more.
R2_mixed_effects(), mlmr(), and mlmr_mv() drop a replication
for every statistic when it has no value for one. Each statistic's
limits were the quantiles of its own finite values, so one missing
value moved the interval of that statistic alone and the intervals of
one call rested on different replications. With 500 replications of
which three lack the marginal and two the conditional R-squared (the
fixture of test-dmar_bootstrap.R), the marginal limits were the
quantiles of 497 values and the conditional limits of 498; both are
now the quantiles of the same 495. When every replication returns
every statistic, which is the usual case, the limits do not change:
R2_mixed_effects(ci_method = "boot", B = 400, seed = 113) on the
sleepstudy random intercept model returns 0.2799 [0.1880, 0.4072]
and 0.7043 [0.5838, 0.7964] in both builds.
No interval that is still reported changes for any other reason.
On 75 calls that the rule accepts and in which every interval is
still reported (1311 reported numbers; mediate(), the kappa
functions, krippendorff_alpha(), htmt(), the two ratio functions,
the reliability family, average_variance_extracted(),
fraction_missing_information(), R2_mixed_effects(),
ci_eta_squared_generalized(), mlmr(), mlmr_mv(),
parallel_analysis(), factor_count(), and
sem_exact_fit_bootstrap(), each with a seed), the largest absolute
difference between this build and the earlier one is 0.
ci_nc_t() computes the noncentral t distribution exactly where
R's pt() is not accurate, so every interval built on it is exact.
DMAR 1.0.x inverted pt() everywhere and warned only when the observed
ncp passed 37.62, so an interval whose upper limit alone passed it was
approximate without a warning. Limits inside the accurate range are
unchanged. The interval of ci_smd() for two groups of 20 at d = 10
with 99 percent confidence was [7.003, 12.999] and is [7.003, 13.099];
ci_nc_t(ncp = 34.5, df = 38) gave [26.511, 42.273] and gives
[26.511, 42.447]; ci_nc_t(ncp = 100, df = 1, conf_level = .99) gave
[-107.157, 257.157] and gives [0.394, 280.717]; ci_cv(cv = .05, N = 15) was [0.0369, 0.0818] and is [0.0366, 0.0790]; and ci_sm(sm = 10, N = 30) was [7.315, 12.512] and is [7.412, 12.580]. ci_smd_c(),
ci_sc(), ci_sc_ancova(), ci_reg_coef(), ci_rc(), ci_src(),
smd_trimmed(), and the sensitivity functions of the AIPE planners
inherit the exact limits. The warning "The observed noncentrality
parameter exceeds 37.62 in magnitude" is gone, and the planners that
passed it on (ss_aipe_sc() 425 times for true_std_contrast = 4,
width = .5, assurance = .9) are silent. With more than 1e7 degrees
of freedom the integral is taken over the standardized chi square: its
density is computed without forming the chi square value, and the
argument of the normal distribution function, q s - ncp, is formed
as (q - ncp) + q(s - 1) with s - 1 computed without forming
s, since a double holds neither the chi square value nor s finely
enough there. Against a 256-bit oracle, at 13 values of the degrees of
freedom from 1e7 to 1e32, with q times the standard deviation of s
from .001 to 10, both tails of the distribution function,
P(|T| <= k), and the density are within 2.0e-14, the probability
the integral leaves out past the 1e-14 and 1 - 1e-14 quantiles of the
chi square. A plan of ss_aipe_sc() or ss_aipe_sc_ancova() up to
2^53 per group with J groups has at most J(2^53 - 1) degrees of
freedom, fewer than 4.1e31 for any vector of contrast weights R can
hold.
The AIPE planners built on ci_nc_t() plan for the exact interval,
and some plans change by one or two. ss_aipe_smd(), ss_aipe_sm(),
ss_aipe_sc(), and ss_aipe_sc_ancova() plan for the interval
ci_nc_t() now reports, with exact noncentral t probabilities in
the published quantile assurance steps (that of ss_aipe_sm(), and of
ss_aipe_sc_ancova() by default; the plans of ss_aipe_smd() and
ss_aipe_sc() with assurance are now the exact ones of the entries
below). Of 116 plans scanned with those steps (standardized effects of
1 to 10, widths of .1 to 2.5, with and without an assurance of .90),
11 changed, among
them ss_aipe_smd(true_smd = 5, width = 1) from 127 to 128,
ss_aipe_smd(true_smd = 3, width = .5, assurance = .9) from 277 to
278, ss_aipe_sm(true_sm = 3, width = 1, assurance = .9) from 102 to
104, ss_aipe_sc(true_std_contrast = 8, c_weights = c(1, -1), width = 1, assurance = .9) from 300 to 299, and
ss_aipe_sc_ancova(true_std_contrast = 2, c_weights = c(1, -1), width = .25) from 738 to 739. The two plans with assurance of ss_aipe_smd()
and ss_aipe_sc() quoted here are also their exact plans.
cv_t() returns exact noncentral critical values. With
alpha_level = .05, df = 3, and ncp = 50 the critical values were
29.0756 and 429.1783 and are 28.2828 and 186.4939; with df = 10 and
ncp = 45 they were 31.7440 and 83.9369 and are 31.3532 and 79.0839;
and with one degree of freedom and ncp = 38 the upper value was
infinite and is 1212.586. The area_less and area_greater columns
are exact as well.
cv_t() and ci_nc_t() with fewer than one degree of freedom are
exact, and the critical values of cv_t() there change, by orders of
magnitude with df of 0.3 or less. DMAR 1.0.x took them from qt(),
which inverts pt(), and below one degree of freedom pt() with a
noncentrality is not accurate even inside 37.62. The critical values
moved by factors up to 2e47 with df = 0.05 and 39 with 0.3, by as much
as 15 percent with 0.5, and by less than 1e-4 of themselves with 0.6 and
above (noncentralities up to 37.5 in absolute value, alpha_level .001
and .05): cv_t(.05, 0.1, ncp = 0.5) gave -33554432 and 305284064 and
gives -1.0428e10 and 5.7095e13, and cv_t(.001, 0.5, ncp = 38) gave
-Inf and Inf and gives 8.478261 and 92490299.0. A critical value beyond
the largest double is -Inf or Inf. The limits of ci_nc_t() with
df < 1 change by at most 1.5e-10 of themselves.
The individual intervals of limits_of_agreement() are exact for
large samples. Their noncentrality, k sqrt(n), passes 37.62 from
n = 369 at 95 percent coverage, and past it each confidence limit was
too far from the mean difference, by as much as 0.0011 standard
deviations of the differences. For set.seed(1000); a <- rnorm(1000, 100, 15); b <- a + rnorm(1000, 0.5, 3), the interval on the upper
limit of agreement was [6.05197, 6.68620] and is [6.05119, 6.68513].
The pair intervals (the default) are unchanged.
Power past a noncentrality of 37.62 is exact. The power planners
whose test is a t test (ss_power_smd(), ss_power_c(),
ss_power_sc(), ss_power_c_ancova(), ss_power_contrast(),
ss_power_mixed_effects(), ss_power_reg_coef(), ss_power_rc(),
ss_power_pcm(), and, for the test of a with X fixed,
ss_power_indirect_effect()), the marginal powers
of ss_power_composite_ancova_2group(), the noninferiority power of
power_equivalence_c() and ss_power_equivalence_c(), and
design_consequences() take the noncentral t probabilities from the
exact routines. No planned sample size changes, since a search stops far
below a noncentrality of 37.62; the power reported for a specified
sample size changes where the noncentrality passes 37.62 and the test
has few degrees of freedom (by as much as .28 with one degree of
freedom and .034 with two; with 10 or more it was and is 1 to within
4e-11). ss_power_smd(true_smd = 40, n_1 = 2, alpha_level = .001)
reported a power of .7823613 and reports .7981440; the marginal power
of the group test of ss_power_composite_ancova_2group(true_smd = 40, true_cor = .3, specified_n_per_group = 2, composite_terms = c("group", "covariate"), include_interaction = FALSE, alpha_level = .001) was
.1666 and is .0525; and design_consequences(true_effect = 40, se = 1, df = 2, alpha_level = .001) reported a power of .7823613, a Type S
error of 1.2e-5, and an exaggeration ratio of 1.0207, and reports
.7981440, a Type S error below 1e-300, and 1.0005. The power of a negative
effect at a specified sample size (ss_power_smd(true_smd = -1, n_1 = 50), and the same in ss_power_c(), ss_power_sc(),
ss_power_mixed_effects(), and an example of
ss_power_composite_ancova_2group()) passed on the warning of pt()
full precision may not have been achieved in 'pnt{final}'; these
calls are now silent, with the same values.
The Bayesian t tests use the exact noncentral t density.
bayes_one_sample_t(), bayes_paired_t(), and bayes_independent_t()
integrate the noncentral t likelihood over the prior, and R's dt()
with a noncentrality rests on the normal approximation past 37.62 and
loses accuracy near t = 0 and with many degrees of freedom. It is now
used only with at most 1,000 degrees of freedom, a noncentrality of at
most 37.62, and |t| of at least .1, where results are unchanged, and
the density is integrated exactly elsewhere, except at t = 0, where
it is exp(-ncp^2 / 2) / {B(df / 2, 1/2) sqrt(df)}. R's dt(), which
DMAR 1.0.x used there, computes it at a nonzero noncentrality by
differencing two log gammas, and is off by as much as 3.4e-8 of it
short of 1e7 degrees of freedom (at 9,832,229.8): dt(0, 5.6e6, 1.5)
is 0.12951758933 where the density is 0.12951758988, and
bayes_one_sample_t(mean = 0, sd = 1, N = 5600001) gave a Bayes factor
bf_01 of 2097.196624 in DMAR 1.0.0 and gives 2097.196615.
bayes_one_sample_t(mean = 5.7, sd = 1, N = 50) (t = 40.3) gave a
posterior median of 5.6038, a 95 percent interval of [4.4709, 6.7759],
and a Bayes factor of 5.3589e35, and gives 5.5967, [4.4709, 6.7901], and
5.3537e35. Such calls take about 0.4 seconds, up from 0.05 to 0.1.
The Tukey and Games-Howell procedures, and the Bryant-Paulson
distribution with no covariates, use the exact studentized range
wherever R's qtukey() and ptukey() are not accurate (see Exact
Noncentral t and Studentized Range Distributions). R documents
qtukey() as accurate to the fourth decimal place. It falls short of
that at small error df, by more as the number of means and the
confidence level grow (with 10 means and 2 df its .999 quantile is 41
percent too small, so intervals built on it have a familywise error
rate of about .0029 rather than .001); at large df; far in either tail;
below the .75 point with more than 50 means; and where its search stops
short of the root of ptukey(). The exact computation is used below 9
error df with up to 6 means, 11 with up to 10, 15 with up to 20, 21
with up to 50, 31 with up to 100, 41 with up to 250, 47 with up to 400,
57 with up to 700, 71 with up to 1000, 81 with up to 1500, 91 with up
to 2000, 101 with up to 3000, and 111 with more (measured to 10000
means and extrapolated beyond); above 10000 and up to 10^8 df; for every
quantile at a probability above .999 or below .001; with more than 50
means, below the .75 point (the .80 point with more than 1000 means);
and wherever qtukey() fails or is more than 2e-6 from the root of
ptukey(). Elsewhere the values are those of qtukey() and ptukey()
(above 10^8 df, their infinite-df values), which were within 9.6e-6 of
the exact quantile and 9.1e-7 of the exact probability at every point
checked where they are kept. The changed values:
| call | DMAR 1.0.x | now |
|------|-----------|-----|
| cv_tukey_hsd(.01, df = 2, n_groups = 20) | 31.485 | 26.830 |
| cv_tukey_hsd(.01, df = 3, n_groups = 20) | 14.105 | 13.976 |
| cv_tukey_hsd(.05, df = 2, n_groups = 20) | 11.867 | 11.857 |
| cv_tukey_hsd(.001, df = 2, n_groups = 10) | 41.612 | 71.063 |
| cv_tukey_hsd(.001, df = 12, n_groups = 50) | 8.0894 | 8.0897 |
| cv_tukey_hsd(.05, df = 30000, n_groups = 10) | 3.16368 | 3.16392 |
| cv_tukey_hsd(1e-4, df = 11, n_groups = 10) | 8.491265 | 8.491348 |
| cv_tukey_hsd(1e-5, df = 1000, n_groups = 2000) | 9.1427 | 6.9804 |
| cv_tukey_hsd(.001, df = 41, n_groups = 1000) | 7.282740 | 7.282842 |
| cv_tukey_hsd(.025, df = 12, n_groups = 10) | 4.245304 | 4.245333 |
| cv_tukey_hsd(.0025, df = 23, n_groups = 5) | 4.255839 | 4.255754 |
| cv_tukey_hsd(.025, df = 15000, n_groups = 100) | 4.479376 | 4.479297 |
| qbryant_paulson(.99, n_covariates = 0, n_groups = 20, df = 2) | 44.526 | 37.943 |
| qbryant_paulson(.00001, n_covariates = 0, n_groups = 3, df = 150) | 0.000092 | 0.006023 |
| ci_tukey_kramer(), 20 groups, 2 error df, conf_level = .99: lower limit of "2 - 1" | -4.718 | -3.784 |
| the same, adjusted p of "2 - 1" | .10649 | .10647 |
| ci_games_howell(), the three small groups of the entry under Bug Fixes: lower limit of c - b (2.55 df) | -6.03754 | -6.03731 |
| qbryant_paulson(.99, n_covariates = 1, n_groups = 20, df = 10) | 8.815518 | 8.815520 |
Every example and vignette result DMAR 1.0.x printed is unchanged, as
are the Bryant and Paulson (1976) Table 1 values the tests check. The
probability integral of the range, which the Bryant-Paulson functions
with covariates used below 7 error df in DMAR 1.0.x, is now taken in
two pieces, since in one it could miss the peak of the integrand with
many means (with 7000 means it returned 2.3e-17 where the probability
is 9.5e-6), and with covariates the quantiles and probabilities beyond
.999 and below .001 integrate the exact studentized range
(qbryant_paulson(.9999, 1, 10, 11) was 12.977072 and is 12.977142).
An exact critical value costs a few milliseconds, after about half a
second the first time a number of means is used in a session:
ci_games_howell() with 10 groups of 3 takes about 0.6 s rather than
0.07 s.
Bryant-Paulson values with covariates where DMAR 1.0.x computed them. The new range of the integral (see Bug Fixes) moves the nodes of the quadrature, which moves the values by about 10^-9 at most below 10^5 error df and by less than 10^-6 above, at most in the seventh significant figure (the most for quantiles beyond .999 above 10^7 df). No printed result changes: the examples, the vignettes, and the published Table 1 values the tests check are unchanged. Where DMAR 1.0.x returned about 0, stopped, or returned a wrong critical value, the values are those of the bug fix. Compared with DMAR 1.0.x, the values above 10000 df also carry the change to the exact studentized range in the entry above.
| call | DMAR 1.0.x | now |
|------|-----------|-----|
| pbryant_paulson(3.7, n_covariates = 1, n_groups = 4, df = 1e8) | 5.6e-25 | 0.9559514 |
| qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = 1e8) | error | 3.6331597 |
| qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = 4.09e7) | 4.5327092 | 3.6331598 |
| qbryant_paulson(.95, n_covariates = 1, n_groups = 4, df = Inf) | error | 3.6331596 |
| pbryant_paulson(3.5, n_covariates = 150, n_groups = 3, df = 1e6) | 0 | 0.9644250 |
| qbryant_paulson(.95, n_covariates = 200, n_groups = 3, df = 1e6) | error | 3.3148295 |
| qbryant_paulson(.9999, n_covariates = 3, n_groups = 4, df = 1e7) | 6.0828639 | 6.0828670 |
ss_aipe_r() with assurance returns the smallest sample size that
attains it, computed from the distribution of the sample correlation.
The Fisher's Z interval is no wider than the target exactly when |r|
is at least a value the function solves for, and the probability of
that event is taken from the exact distribution function of r under
bivariate normality; the new rows assurance and assurance_attained
give the request and that probability. DMAR 1.0.x multiplied the plan
by qchisq(assurance, N - 2) / (N - 2). At true_cor = .90 and
width = .10 it returned 72 for assurance = .80 and 92 for .99,
where the probabilities are .672086 and .887068; the plans are now 83
and 122 (.807291 and .990268). At true_cor = 0 it returned 1585 for
width = .10 and assurance = .80, where the plan without assurance,
1538, already attains a probability of 1 and is now returned.
ss_aipe_r_sensitivity() with supposed_cor and assurance changes
with it.
ss_aipe_smd() with assurance returns the smallest per-group sample
size, not below the plan without assurance, at which the exact
probability that the interval is no wider than width is at least
assurance, for |true_smd|. DMAR 1.0.x and MBESS::ss.aipe.smd()
returned the plan of the procedure of Kelley and Rausch (2006, pp.
372--374), which does not compute that probability (see The Assurance
Method of the Sample Size Planners).
true_smd = 1, width = .5, and assurance = .99 it was
150 per group (.994302) and is 149 (.990260). In 45 planning sets
(|true_smd| of 0 to 2, widths of .25 to 1, assurances of .80 to
.99) the published plan attained the request in every set and was one
case larger in 7, six of them at .99.true_smd = 0, width = .8, and assurance = .95 it was 49 (.949878) and is 50 (.994636).true_smd, so reversing the groups
changed the plan: at width = .5 and assurance = .90 it returned 125 per
group at true_smd = -.5 and 133 at -1, below the plans without
assurance, where the probability is .146146 and .064400. They are
now 130 and 144 (.915627 and .904165), as for .5 and 1.ss_aipe_smd_sensitivity()
changes with it.ss_aipe_equivalence_smd() plans for the interval equivalence_smd()
reports, d -/+ t SE(d) with the Student t quantile on n_1 + n_2 - 2
degrees of freedom, where DMAR 1.0.x planned with the narrower standard
normal quantile, so the analysis at the planned size could report an
interval wider than width. The search starts at 2 per group, the
smallest size equivalence_smd() accepts (1.0.x never returned fewer
than 4). For true_smd = 0 and width = 2.5 the plan was 4 per group,
with an expected_width of 2.3262, where equivalence_smd() at 4 per
group reports 2.7481; it is now 5, with 2.3522. For width = 0.5 the
plan was 87 and is 88. The table records the confidence level of the
interval, 1 - 2 alpha, in its conf_level attribute.
ss_aipe_equivalence_smd() computes the probability for assurance
from the distribution of the sample standardized mean difference, and
reports it in the new rows assurance and assurance_attained, by
numerical integration (the form of Kelley and Rausch, 2006, Equation
20). DMAR 1.0.x simulated 2,000 widths of the narrower normal-quantile
interval at each size, so its plans fell short of the request and
differed from run to run. With true_smd = 0.5, width = 0.5, and
assurance = 0.80 it returned 91 per group, where the probability is
.671; the plan is now 92 (.828). With true_smd = 0 it returned 87 or
88; the plan is now 88 (.839). The function draws no random numbers and
has no seed argument.
ss_aipe_equivalence_smd_sensitivity() simulates the interval
equivalence_smd() reports, at conf_level, where DMAR 1.0.x
simulated the noncentral t interval of ci_smd(), which is usually
narrower; the width, miss, and equivalence rows change with it. With
true_smd = 0.3, specified_n_per_group = 61, conf_level = 0.90, and
set.seed(113), the first simulated sample (d = 0.670) was scored with
the limits 0.363 and 0.975 and is now scored with 0.362 and 0.979; with
true_smd = 0, supposed_smd = 0, width = 0.3, smd_upper = 0.5,
G = 2000, and seed = 113, pct_ci_less_width was .856 at 342 per
group and is .736 at 343. The simulated samples are the ones 1.0.0
drew, so the mean_smd, median_smd, and sd_smd rows are unchanged
under a common seed and size. No argument restores the earlier results
of ss_aipe_equivalence_smd() or of its sensitivity function.
equivalence_smd() returns what it returned
before, except for sizes given as integers or counted from raw data
whose product passes 2^31 - 1 (46,341 or more in each group, or 40,000
and 60,000), for which it returned NA (see Bug Fixes).
The plans of ss_aipe_reg_coef(), ss_aipe_rc(), and
ss_aipe_src() without assurance are the smallest sample size whose
interval meets the width, with the expected squared multiple
correlation of the targeted predictor on the other p - 1 predictors.
DMAR 1.0.x, like MBESS::ss.aipe.reg.coef(), did two things
differently. It took the expected squared multiple correlation of X_j
with the other predictors to be that of a regression on p
predictors, which overstates it and so the planned width. And it
searched upward one case at a time from a start, for the central
interval the larger of 2p + 1 and nine below the normal approximation
rounded up, and for the noncentral one the larger of 2p + 2 and five
below the central plan, so it returned the start when a smaller sample
size already met the width. The expectation now has p - 1 predictors
(0 with one predictor), and the search, by doubling and bisection,
covers every sample size from p + 2, one residual degree of freedom.
Examples, old to new:
true_R2_Y_X = .8, true_R2_j_X_without_j = 0, p = 10,
true_reg_coef_j = .3, width = 2: the central plan was 21 and is
14.true_R2_Y_X = .1, true_R2_j_X_without_j = .2, p = 2,
true_reg_coef_j = 0, width = .4, noncentral = TRUE, tails of
.0002: the noncentral plan was 358 and is 355.width = .3: the central plan was 157 and is 156, and the
noncentral plan was 165 and is 164.In 1,530 central planning sets (p from 1 to 15, squared multiple correlations from 0 to .9, widths from .05 to 2, tails of .025 and .025, .005 and .005, and .01 and .04) the plan is one case smaller in 1,329, 2 to 13 smaller in 104, and unchanged in 97. In 810 noncentral sets it is one smaller in 596, 2 to 10 smaller in 177, and unchanged in 37. Every plan equals the smallest sample size from p + 2 that a scan of every N finds, and a noncentral plan takes about half the time it did.
The published approximations with assurance start from these plans
and change with them. They are the chi square plan of Kelley and
Maxwell (2003, Equation 3), DMAR 1.0.x's plan for the central interval
and now the default of ss_aipe_src(noncentral = FALSE), and the
noncentral approximation, the default of ss_aipe_reg_coef(noncentral = TRUE) and ss_aipe_src():
assurance fell to a few
residual degrees of freedom and Equation 3 takes its t quantile
there. At .95 it is smaller in 1,315, unchanged in 80, and 1 to 26
larger in 135. At .80 it attains the request for the interval in the
units of the data in 752 of the sets, where it did in 797.The plans that compute the probability (the exact central plan and
those of assurance_method = "simulation") change only where their
answer was the earlier plan without assurance. In the 1,530 sets, 83
exact plans at .80 and 49 at .95 are 1 to 12 cases smaller, each still
the smallest N at or above the plan without assurance whose
probability meets the request. The exact and simulated plans in the
table of the next entry are unchanged by it, and in 120 noncentral sets
with G = 2000 no simulated plan changed. The sensitivity functions plan through these
planners and change with them (ss_aipe_src_sensitivity() in an entry
below: 157 to 156). No argument restores the earlier plans.
ss_aipe_reg_coef() and ss_aipe_rc() with assurance return, for
the central interval, the smallest sample size at which the interval
is no wider than width with at least that probability, computed
exactly. DMAR 1.0.x used the chi square factor of Kelley and Maxwell
(2003, Equation 3; 2008, Equation 37) for the central interval and, for
the noncentral one, a lower confidence limit of the squared multiple
correlation in place of its population value; neither computes the
probability it attains, which can be more or less than the request.
Three intervals are planned for in 1.1.0:
ss_aipe_reg_coef() with noncentral = FALSE (the default) and
ss_aipe_rc() plan the central interval for the coefficient in the
units of the data, even at unit standard deviations;
ss_aipe_reg_coef(noncentral = TRUE) and ss_aipe_src() the noncentral
interval for the standardized coefficient; and
ss_aipe_src(noncentral = FALSE) (new) the central interval for the
standardized coefficient, Kelley and Maxwell's (2003) Equation 1. For
the central interval in the units of the data the new argument
assurance_method, "analytic" by default and the only method
offered there, gives the smallest sample size, at or above the plan
without assurance, at which that probability, an exact F
probability for random predictors from a multivariate normal
population, is at least assurance (280 for the example of Kelley and
Maxwell, 2008, p. 186, where DMAR 1.0.x and MBESS::ss.aipe.reg.coef()
gave 278). For the standardized coefficient "analytic" gives the
approximations of 1.0.x, from the plans without assurance of this
release (see the entry on those plans), with assurance_attained
NA, and assurance_method = "simulation" estimates the probability
from G samples (default 20000) of the exact Wishart sampling
distribution, with seed, and gives the smallest sample size at which
the estimate reaches assurance; assurance_attained is the estimate
at the returned size, its Monte Carlo standard error is the attribute
assurance_mc_se, and the plan can be a case or two from the one the
exact probability gives. Given squared multiple correlations, which
leave the correlation of Y with the predictor open, the simulated
step uses the least favorable value.
Old and new sample sizes, with the probability each attains for the
interval in the second column: "raw central" (ss_aipe_reg_coef()
with noncentral = FALSE, ss_aipe_rc()), "std noncentral"
(ss_aipe_reg_coef(noncentral = TRUE), ss_aipe_src()), and "std
central" (ss_aipe_src(noncentral = FALSE), whose 1.0.x plan is the
chi square plan of ss_aipe_reg_coef(noncentral = FALSE)). Each
probability is exact or the proportion of 50,000 or more simulated
samples (Monte Carlo standard error at most .0014). The simulated
plans are those of seed = 1 and the default G; another seed can
give a plan a case away. Variances are 1 unless stated:
| planning set, width, assurance | interval | 1.0.x N (attains) | 1.1.0 N (attains) | 1.1.0 "simulation" N (attains) |
|---|---|---|---|---|
| two predictors correlated .3, correlations .4 and .3 with Y, .3, .90 | raw central | 182 (.862) | 187 (.902) | not offered |
| the same, .3, .80 | raw central | 174 (.777) | 176 (.801) | not offered |
| the same, .3, .99 | raw central | 203 (.974) | 213 (.990) | not offered |
| the same, .3, .90 | std central | 182 (.989) | 181 (.987) | 171 (.914) |
| the same, .3, .90 | std noncentral | 180 (.957) | 179 (.947) | 176 (.909) |
| the same, .3, .80 | std noncentral | 177 (.923) | 176 (.907) | 172 (.818) |
| five exchangeable predictors (.4 among them, .3 with Y; Kelley & Maxwell, 2003, pp. 312--313), .2, .80 | std central, the interval of the article | 487 (.884) | 486 (.877) | 478 (.807) |
| the same, .2, .80 | raw central | 487 (.773) | 491 (.800) | not offered |
| the same, .2, .99 | std noncentral | 498 (.944) | 497 (.940) | 517 (.989) |
| Kelley and Maxwell's (2003, p. 313) three predictors, the first, .3, .90 | raw central | 271 (.859) | 278 (.904) | not offered |
| the same, the first, .3, .99 | std noncentral | 314 (.952) | 313 (.949) | 330 (.990) |
| one predictor correlated .4 with Y, .2, .90 | raw central | 362 (.841) | 373 (.904) | not offered |
| the same, .2, .90 | std noncentral | 377 (1.000) | 376 (1.000) | 364 (.911) |
| Kelley and Maxwell's (2008, p. 186) example (p = 5, R^2 .5 and .2, sigma_Y / sigma_X = 10), 2, .85 | raw central | 278 (.838) | 280 (.853) | not offered |
| its standardized analog (coefficient .3, unit standard deviations), .2, .85 | std noncentral | 296 (.894) | 295 (.886) | 291 (.854) |
The default plans for the standardized coefficient are the 1.0.x
approximations, a case smaller through the plans without
assurance; the probabilities they attain were simulated from
200,000 or more samples of the exact sampling distribution (Monte
Carlo standard error at most .0011). The five exchangeable predictors
are Kelley and Maxwell's (2003) example, planned with the central
interval of their Equation 1 (485 in the article, which uses the
normal quantile); at .99 for them the simulated plan's own estimate
is .990 and the simulation's .989, within the Monte Carlo error. The
plan is the same for a coefficient and its negative. The three
sensitivity functions plan through these planners, each for the
interval it simulates, and change with them.
No argument restores the central plans of DMAR 1.0.x with
assurance; for unequal tails see the next entry. The default plans
for the standardized coefficient change only through the plans
without assurance. DMAR 1.0.x stopped with an error on every plan of
the three sensitivity functions with one predictor.
ss_aipe_src(noncentral = FALSE) is new and has no earlier result.
With unequal tails, the central plans of ss_aipe_reg_coef() and
ss_aipe_rc() are for the interval ci_reg_coef() reports (and so
are those of ss_aipe_src(noncentral = FALSE), new in this release).
That interval runs from b_j + t(alpha_lower) SE to b_j + t(1 -
alpha_upper) SE, so its width is t(1 - alpha_lower) + t(1 - alpha_upper)
times the standard error. DMAR 1.0.x took both quantiles at the average
of the two tails, t(1 - (alpha_lower + alpha_upper)/2), in the plan
without assurance and in its chi square step with assurance
(Kelley and Maxwell, 2003, Equation 3), and so planned an interval
narrower than the one reported. For two predictors correlated .3,
correlated .4 and .3 with Y, and a width of .3:
| tails (lower, upper) | plan | 1.0.x N | 1.1.0 N |
|---|---|---|---|
| .01, .04 (or .04, .01) | without assurance | 157 (the interval .3108 wide) | 169 (.2992) |
| .001, .049 | without assurance | 157 (.3634) | 228 (.2996) |
| .01, .04 | assurance = .90 | 182 (attains .709) | 201 (attains .902) |
| .001, .049 | assurance = .90 | 182 | 266 (attains .905) |
| .025, .025 | without assurance; assurance = .90 | 157; 182 | 156; 187 |
The widths are those of the reported interval at the expected squared
multiple correlations, and the probabilities are exact. Without
assurance equal tails give the plans they gave, but for the change in
the entry on the plans without assurance; with assurance the plan
is the exact one of the entry above. The noncentral plans used the two
tails as given and are unchanged by this correction. The chi square
step with both quantiles, the default of ss_aipe_src(noncentral = FALSE) with assurance, gives 194 (attaining .847 for the interval
in the units of the data) and 258. With unequal tails no argument
restores the 1.0.x central plans of ss_aipe_reg_coef() and
ss_aipe_rc() (or the plan of MBESS::ss.aipe.reg.coef()).
ss_aipe_src() is not affected: its noncentral plans used both tails in
1.0.x, so this correction does not change them, and its central plans
are new. The sensitivity functions plan with equal tails, so it does
not change them either.
The limits of the central ci_reg_coef(), ci_c(), and ci_r()
move, by up to 1.25e-7 of themselves near the floor. With the upper
quantile taken in the upper tail (see Bug Fixes) each limit is the
exact one, and the change grows as the tail shrinks. In 2,160 calls of
these three functions, at levels from .90 to .999999998 and tails from
.05 down to 1e-9, against the development versions of 1.1.0, the
limits moved by at most 6.1e-14 of themselves at levels of .90 to .999
and tails of 0 to .05, 2.2e-11 at a tail of 1e-6, 1.1e-9 at a level of
.9999999, and, at a tail of 1e-9 or a level of .999999998, 1.1e-8
(ci_c() and ci_reg_coef()) and 1.25e-7 (ci_r(), whose lower
quantile changed as well). Near the floor a limit moves in about its
eighth or ninth significant digit. ci_c(means = c(1, 0), s_anova = 1, c_weights = c(1, -1), n_per_group = c(20, 20), conf_level = .999999998) (n in 1.0.0) has an upper limit of 3.4711584085612,
symmetric with its lower limit of -1.4711584085612, where DMAR 1.0.0
and the development versions gave 3.47115840279224, and ci_r(r = -.9, N = 20, conf_level = .999999998) (n in 1.0.0) has an upper limit of
-0.0175358196414772, where they gave -0.0175358218277256. The
assurance_attained of the exact central assurance step of
ss_aipe_reg_coef() moves the same way: ss_aipe_reg_coef(true_R2_Y_X = .3, true_R2_j_X_without_j = .2, p = 4, true_reg_coef_j = .2, width = .3, assurance = .8, conf_level = NULL, alpha_lower = .04, alpha_upper = 1e-9) reports 0.80195028591575, where the development versions
reported 0.801950281617816, for the same plan of 643. The 768
noncentral intervals of ci_reg_coef() among those calls were
unchanged, as were ci_sc(), ci_sc_ancova(), ci_sm(), ci_cv(),
ci_smd(), and every sample size in 1,944 further calls, the plans of
ss_aipe_reg_coef(), ss_aipe_sc(), ss_aipe_sc_ancova(),
ss_aipe_c(), and ss_aipe_r() among them, and the 1,530 central
plans of ss_aipe_reg_coef() over a grid of squared multiple
correlations, predictors, widths, and tails and the 1,080 plans of
ss_aipe_sc_ancova() over a grid of contrasts, correlations,
divisors, weights, widths, and levels. (Other entries change some of
these plans: those on the plans without assurance of
ss_aipe_reg_coef() and of ss_aipe_sc_ancova().)
ss_aipe_src_sensitivity() plans the sample size for the
standardized coefficient, for the interval it simulates. The plan
came from ss_aipe_reg_coef() with the supposed standard deviations, so
width was taken in the units of the data while the simulated
intervals were standardized. For true_var_Y = 4, true_cov_YX = c(.8, .6), unit predictor variances correlated .3 (correlations of .4
and .3 with Y), supposed correctly, width = .3, and noncentral = FALSE, the plan was 610, with a mean width of .150, and is 156 (mean
width .300). With assurance = .90 it was 657 (182 with unit supposed
variances). It is 181 by default, the chi square plan of Kelley and
Maxwell (2003, Equation 3) for the central interval for the
standardized coefficient, which the replications compute (.987 of
those intervals are no wider than .3 at 181), and 171 with
assurance_method = "simulation" and seed = 1, the smallest sample
size at which that interval is no wider than .3 with probability at
least .90 (.914 at 171). No argument restores the 1.0.x plans. With
equal variances of Y
and of the targeted predictor, true and supposed, whatever the
variances of the other predictors, and two or more predictors, the
default plans are made as in 1.0.x, a case smaller (see the entry on
the plans without assurance). The noncentral
plan, the default interval, was refused
whenever the supposed variance of Y or of the targeted predictor was
not 1, and every other plan with one predictor stopped with "argument
is of length zero"; such calls now plan (the noncentral plan is 164
here). The numbers for noncentral = FALSE
are also those of DMAR 1.0.x's
ss_aipe_reg_coef_sensitivity(standardize = TRUE).
The standardized regression coefficient in the sensitivity functions
is judged against the standardized coefficient.
ss_aipe_src_sensitivity() standardizes the data of every replication,
but the true and supposed values the intervals were judged against were
unstandardized; both are now the standardized coefficient (Kelley &
Maxwell, 2003, p. 316). For the structure above with specified_N = 200, width = .3, G = 300, and seed 2026, the true value was .681
beside a mean estimate of .334, and every interval missed
(total_type_I_error 1); it is now true_std_reg_coef_j .341, with
total_type_I_error .017 (.033 with noncentral = FALSE). A call with
specified_N changes as well, as does one with one predictor
(true_var_Y = 4, true_cov_YX = .8, true_cov_XX = matrix(1), and
specified_N = 100: .8 in DMAR 1.0.x, .4 now).
ss_aipe_sc() with assurance returns the smallest sample size per
group, not below the plan without assurance, at which the exact
probability that the ci_sc() interval is no wider than width is at
least assurance. DMAR 1.0.x returned, in one row, the plan of the
step of Kelley and Rausch (2006) as Lai and Kelley (2012, pp. 358--359
and 361) apply it to a standardized contrast, which does not compute
that probability (see The Assurance Method of the Sample Size
Planners).
true_std_contrast = 0, c_weights = c(1, -1), width = .8, and assurance = .95 it was 49 per group
(.949878) and is 50 (.994636).true_std_contrast it planned the
magnitude with the tails as given (see Bug Fixes). At
true_std_contrast = -2.5, width = .4, alpha_lower = .001,
alpha_upper = .049, and assurance = .90 it was 522 per group
(.890) and is 523 (.902); at -3, width .5, and tails .001 and .099 it
was 342 (.842) and is 346 (.902).assurance and assurance_attained. The
plan without assurance is unchanged.ss_aipe_sc_sensitivity()
changes with it.ss_aipe_sc_ancova() without assurance returns the smallest sample
size per group whose interval at the planning value is no wider than
width. DMAR 1.0.0 started its search at the fixed point of the
central t approximation, n = C r^2 (t_l + t_u)^2 / w^2 with t on
J(n - 1) - 1 degrees of freedom, and searched only upward. At a small
contrast the central quantiles are farther apart than the noncentral
limits, so the start could already meet the width, and the plan was
then the start: ss_aipe_sc_ancova(true_std_contrast = 0, c_weights = c(1, -1), width = .5) (psi_standardized in 1.0.0) gave 125 per
group, though the interval is .49985 wide at 123 and .50190 at 122. It
now gives 123. The search now starts at 2 per group, the smallest
sample size with positive error degrees of freedom, and the width falls
as the sample size grows from there (checked on 420 combinations of
contrast, ratio of the two error standard deviations, weights for two
to five groups, and level). Of 1,080 plans over contrasts of 0 to 2,
correlations of 0, .5, and .9, both divisors, weights for two to four
groups, widths of .1 to 3.6, and levels of .95 and .99, 446 are
smaller: 304 by 1 per group, 141 by 2, and 1 by 3. None is larger, and
no plan at a contrast of 2 changed. At widths of 1 or less, 198 of 720
are smaller, by 1 or 2. Each plan meets the width, and one fewer does
not. The published assurance step, the default (assurance_method = "analytic"), plans from this plan and changes with it. On the same
grid, 302 plans at an assurance of .80 are 1 or 2 smaller. At .90, 227
are 1 or 2 smaller and 7 are 1 larger: the call above with assurance = .9 gave 125 and gives 124. The step plans without assurance at a
contrast it adjusts for the assurance, so its plan can move by a case
or two per group, in either direction, even where the plan without
assurance at the planning values did not: at true_std_contrast = 2,
true_cor = .5, c_weights = c(1, -1), width = .3, and assurance = .99 it was 545 per group and is 546. The simulated step
(assurance_method = "simulation", new in 1.1.0) also searches upward
from this plan. With conf_level = .99 the call above gave 215 per
group without assurance and gives 213, and with assurance = .8 it
gave 215 and gives 213 by default and 214 with assurance_method = "simulation" and seed = 1 (assurance_attained .886). The default
plans in the table of the ANCOVA entry under The Assurance Method of
the Sample Size Planners are those of 1.0.x.
ss_aipe_sc_ancova_sensitivity() plans through the planner and
changes with it. Past about 1e11 per group the plans rest on the
precision of doubles, and they move as well (Details of
?ss_aipe_sc_ancova). No argument restores the earlier plans.
ss_aipe_sc_sensitivity() standardizes each simulated contrast by the
pooled standard deviation, the square root of the mean square error,
which is the s_anova that ci_sc() takes. DMAR 1.0.x divided by the
mean of the group standard deviations, which is never larger, so its
contrasts ran about 2 percent large with three groups of 10. After
set.seed(113), ss_aipe_sc_sensitivity(true_std_contrast = .5, specified_n_per_group = 10, c_weights = c(.5, .5, -1), width = .8, G = 2000) gave mean_std_contrast .527 and total_type_I_error .070 and
gives .517 and .064. No argument restores the 1.0.x result.
ss_power_indirect_effect() computes power from the exact sampling
distribution of the two path t statistics, with X random by
default (see Bug Fixes). Every result changes, and no argument
restores the 1.0.x results: random_predictors = FALSE fixes X as 1.0.x
did, and with it power_a is the 1.0.x power of the test of a, but
the mediator is random in every design, so the 1.0.x power_b and
power_ab are the powers of no study. power_ab is no
longer power_a times power_b but the probability that both tests
reject, which is below that product. The Sobel power need not rise with
N, since the test refers t statistics on few degrees of freedom to the
normal: with paths of .9 and .05 it falls from .0729 at N = 10 to .0562
at N = 59, and with paths of .8 and .02 and alpha_level = .5 it is
.5072 at N = 12 and .5035 at N = 59, and does not reach .507 again until
N = 216. The search no longer takes one case at a time; it passes over a
range of sample sizes only where an upper bound of the power shows that
none in it reaches desired_power, so the plan is the smallest N of at
least 10 that attains desired_power, whether or not the power rises
with N. It still stops with an error at ten million. The values (1.0.x,
now with X random, now with X fixed):
| call | 1.0.x | now | now, random_predictors = FALSE |
|------|-------|-----|------------------------------------|
| ss_power_indirect_effect(.39, .39, specified_N = 65): power_ab, power_a, power_b | .8017, .9196, .8719 | .7675, .9046, .8503 | .7804, .9196, .8503 |
| ss_power_indirect_effect(.39, .39, specified_N = 75): power_ab | .8706 | .8436 | .8537 |
| ss_power_indirect_effect(.3, -.25, .1, specified_N = 120): power_ab | .7056 | .6876 | .6936 |
| ss_power_indirect_effect(.6, .5, .2, specified_N = 12): power_ab | .2367 | .1631 | .1826 |
| the same four with method = "sobel": power_ab | .6380, .7001, .5639, .3165 | .6157, .7272, .5067, .1226 | .6305, .7410, .5135, .1367 |
| ss_power_indirect_effect(.39, .39, specified_N = 96, method = "sobel") | .8026 | .8818 | .8907 |
| ss_power_indirect_effect(.2, .1, specified_N = 10, alpha_level = .999, method = "sobel") | .9990 | .9983 | .9983 |
| necessary_N, ss_power_indirect_effect(.39, .39, desired_power = .80) | 65 | 69 | 68 |
| the same with method = "sobel" | 96 | 84 | 82 |
| necessary_N, ss_power_indirect_effect(.14, .39, desired_power = .80), joint significance and Sobel | 395, 438 | 398, 412 | 395, 410 |
| necessary_N, ss_power_indirect_effect(.3, -.25, .1, desired_power = .80), joint significance and Sobel | 144, 210 | 148, 179 | 147, 178 |
| necessary_N, ss_power_indirect_effect(.1, .59, desired_power = .95), joint significance and Sobel | 1289, 1312 | 1293, 1296 | 1289, 1292 |
| necessary_N, ss_power_indirect_effect(-.3, .2, desired_power = .50), joint significance and Sobel | 116, 141 | 119, 157 | 118, 155 |
| necessary_N, ss_power_indirect_effect(.0044, .5, desired_power = .80) | 405411 | 405414 | 405411 |
| necessary_N, ss_power_indirect_effect(.8, .02, desired_power = .507, alpha_level = .5, method = "sobel") | 229 | 12 | 10 |
meta_r() with reliabilities gives each corrected correlation the
sampling variance of a corrected correlation. Each correlation
corrected for attenuation, r / sqrt(rxx ryy), was given on the Fisher
Z scale the variance of an uncorrected correlation, 1 / (n - 3), which
is too small (by a factor of 1.96 for r = .40 with both reliabilities
.80). The variance is now the delta method variance
(1 - r^2)^2 / (rxx ryy (1 - r_c^2)^2 (n - 3)), with r_c the corrected
correlation, at each study's observed correlation; it sets the weights
and every result. Without reliabilities nothing changes. For five
studies with r = .40, n = 100, and both reliabilities .80
(method = "fe", hartung_knapp = FALSE), the standard error on the
Fisher Z scale was 0.0454 and is 0.0636, and the interval for the
construct correlation was .430 to .564 and is .401 to .588. For
meta_r(r = c(.21, .35, .42, .18, .30, .47), n = c(48, 120, 75, 300, 60, 150), reliability_x = c(.72, .85, .78, .90, .65, .80), reliability_y = c(.80, .70, .88, .75, .82, .90)), the pooled correlation was .4095 and
is .4051, Q was 20.14 and is 12.05, and I2 was 69.0 percent and is 54.1
percent. ?meta_r states the variance and that the reliabilities are
treated as known constants. An analysis that gave reliabilities to
meta_r() should be rerun.
var_alpha() returns each variance in the row named for its
source. DMAR 1.0.x returned 2J(1 - alpha)^2/((J - 1)(N - 2)) in
the row var_alpha_van_zyl and 2J(1 - alpha)^2/((J - 1)(N - 1))
in the row var_alpha_bonett. Van Zyl, Neudecker, and Nel (2000,
Equation 13) give the second, and Bonett (2003, Equations 2 and 4)
gives the first, on the scale of the logarithm of 1 - alpha; the van
Zyl variance that reliability_alpha(ci_method = "ml") and
ss_aipe_reliability() use is, at a compound symmetric covariance
matrix, the N - 1 form. The two values now sit in the rows their names
attribute; the pair is unchanged. No argument restores the earlier
rows.
fa_confirmatory(output = "standardized") (and cfa_1(), cfa_2(),
and cfa_k()) computes the standardized loading_sum, error_sum,
omega, ave, and H rows of a factor with equal_loading or
equal_error from each item's own standardized loading and error
variance, with delta method standard errors. lavaan's
standardizedSolution(), the earlier source, evaluates a defined
parameter at the first item's standardized value of a shared label. For
four items with unstandardized loadings of .7 and error variances of
.1, .2, 1, and 2 (a covariance matrix, N = 500), equal loadings gave a
standardized AVE, H, and omega of .6583, .9094, and .8730 and give
.5166, .8899, and .7988; equal error variances gave .3155, .6642, and
.6523 and give .3816, .7710, and .7007. lavaan's ambiguity warning no
longer reaches the caller. Other factors and the "measurement" and
"verbose" tables are unchanged.
fa_bifactor(output = "standardized") computes each index that uses
a label several items share in the same way. On holzinger_swineford
with the three groups of the example, equal loadings on spatial, and
equal error variances on memory, omega_general, omega_HS_spatial,
H_spatial, and omega_memory were .9113, .1222, .1972, and .8995 and
are .8731, .2195, .4025, and .7233; with equal loadings on the general
factor, ECV_general and H_general were .0879 and .3627 and are
.1443 and .5133, the values of the "measurement" table.
fa_second_order(output = "standardized") no longer passes on lavaan's
warning about the index rows it leaves out.
The tables of fa_confirmatory() (and of cfa_1(), cfa_2(), and
cfa_k()) withhold every standard error, test, and interval of a fit
whose expected information is singular, with the warning of class
dmar_result_qualified, as they did for a singular observed
information. A two-test factor uncorrelated in the sample with another
factor, fitted by listwise deletion with equal loadings on that factor,
had an H of .853 with a standard error of .028 and the interval [.790,
.900]; the standard errors and intervals are now NA.
average_variance_extracted() refuses a multiple-group or multilevel
fit, with an error of class dmar_argument_invalid that names the
groups (as groups) or the levels (as levels). DMAR 1.0.0 averaged
the squared standardized loadings of every group, or of both levels,
into one row: a factor whose AVE is .810 in one group and .090 in the
other (three items with equal correlations of .81 and .09, groups of
1000 and 100) had an AVE of .450, which describes neither group. The AVE
of one group or level is average_variance_extracted(loadings = ) given
its standardized loadings. bifactor_indices() refuses a multiple-group
fit the same way (see Calls That Are Now Refused).
bifactor_indices() refuses a multilevel fit with an error of class
dmar_argument_invalid. DMAR 1.0.0 read the loadings of the within
level and the residual variances of the between level: for nine items
in three subscales with a general factor g at both levels (200
clusters of 8, set.seed(113)), the general factor's omega was .942 and
its omega_H .740, where the within level's own solution gives .885 and
.695.
average_variance_extracted(), bifactor_indices(), and dmacs()
refuse a fit whose optimizer did not converge
(lavaan::lavInspect(fit, "converged") is FALSE), with an error of
class dmar_not_converged that carries fn and iterations. DMAR
1.0.0 read the point where the optimizer stopped as a solution: a
one-factor fit to three items whose correlations are all .81 (N = 200),
stopped after one iteration, had an AVE of .412, where the converged
fit has .810. A fit that converged to an improper solution is treated
as before.
The standardized cluster randomized planners (ss_aipe_crd_es_*())
measure the width of each simulated interval from its lower limit to
its upper limit. The width was the estimate minus the lower limit,
about half the interval, so every width these planners report is about
twice the earlier one, and a plan made for a width is larger. With
G = 2, seed = 113, ss_aipe_crd_es_n_clusters_fixed_budget(budget = 1000, cluster_size = 20, cluster_cost = 0, individual_cost = 1, true_es = 0.5, es_type = 1, true_icc = 0.25, pr_treat = 0.5) reports an
expected_width of 0.672 where it reported 0.335; with G = 12, seed = 113, ss_aipe_crd_es_n_clusters_fixed_width(width = 0.40, cluster_size = 20, true_es = 0.5, true_icc = 0.10, pr_treat = 0.5) returns 67
clusters where it returned 14, and
ss_aipe_crd_es_n_individuals_fixed_width(width = 0.60, n_clusters = 60, ...) a cluster size of 5 where it returned 2.
The standardized cluster randomized planners use only the intervals
of converged fits, count a failed data set as an infinite width under
assurance, and bracket the target before they bisect. The interval
of a data set is used only when the optimizer status code of the fit
and of each limit's search is 0 or 1, the diagnostic OpenMx gives each
limit is "success", and the limits are finite and bracket the
estimate; any two limits that were numbers were used before. The
assured width is the assurance quantile of all G widths, with the
data sets without a usable interval at infinity: with two of four data
sets failed and the other two at 0.2, the assured width at 0.8 was 0.2,
and the call is now refused. The searches upward for a size stepped up
one cluster or one individual at a time and now step up 1, 3, 7, 15,
and so on, and then halve the gap. When the simulated width falls as
the size grows the design is the same (the calls above still return 67
clusters and a cluster size of 5); with a small G the simulated width
rises and falls, and the two searches can stop at different sizes near
the target (with G = 6, seed = 99,
ss_aipe_crd_es_n_clusters_fixed_width(width = 0.7, cluster_size = 2, true_es = 3, es_type = 1, true_icc = 0.2, pr_treat = 0.5) returns 233
clusters after simulating 16 designs, where stepping one at a time
stopped at 203 after 107).
ss_aipe_crd_es_n_individuals_fixed_width() with es_type = 1 plans
targets it refused, refuses unreachable ones before simulating, and
starts its search lower. With a fixed number of clusters the width
levels off, at a value with a closed form, as the clusters grow. The
search started from the cluster size the unstandardized difference
needs, which uses the t quantile and with few clusters needs far
more: with 10 clusters and an intraclass correlation of 0.2 it needed
more than 100,000 per cluster for every target from 1.19 to about
1.458, and 1,875 for 1.46, which about 10 reach with true_es = 0.5.
It now starts from the smaller of that size and the size at which the
closed-form width, with the within-cluster standard deviation known,
falls below the target; with equal cluster sizes, no covariate, and no
assurance that start is a lower bound on the size the target needs.
width = 1.30, n_clusters = 10, true_es = 0.5, true_icc = 0.2, pr_treat = 0.5, G = 20, seed = 5 returns a cluster size of 18, and
width = 1.46 (seed = 11) returns 9. Targets whose search
started at 500 or less are now searched upward from the smaller start.
When the simulated width falls as the cluster grows the design is the
same; with a small G it can differ, and far fewer
designs are simulated: with G = 20, seed = 5, width = 1.50 returns
8 after simulating 2 designs, where stepping down one size at a time
from 70, as DMAR 1.0.x searched, stops at 16 after 56. A target at or
below the level (1.187 there) is refused without a simulation (class
dmar_argument_invalid, reason "width_not_reachable", the level
as width_limit), and so is one whose start is above 500 (from 1.187
to about 1.1914 there; reason "cluster_size_beyond_reach").
The cluster randomized searches over the number of clusters keep the
floor of two clusters in each condition and return a number they
tested. The floor, ceiling(2 / min(pr_treat, 1 - pr_treat)) + n_covariates (MBESS: "at least two groups in each condition"), is
where the search starts when every number from 100 to 1000 meets the
target. There the search returned 3 + n_covariates, which it had not
tested: ss_aipe_crd_n_clusters_fixed_width(width = 100, cluster_size = 20, pr_treat = .1, true_var_between = .2, true_var_within = .8)
returned 3 clusters, none treated, with an expected_width of Inf,
in DMAR 1.0.0 and the development versions of 1.1.0, and returns 20
(1.534). Where the floor is above 100 (pr_treat below .02) the
search stopped with "wrong sign in 'by' argument" (pr_treat = .01,
now 200 clusters, 1.373) or returned a number below the floor
(pr_treat = .005: 101 clusters, one treated, now 400, 1.365). With
unequal cluster sizes the same call with diff_size = c(-2, 0, 2)
returned 3 clusters with a width of NaN in both, and returns 20.
Every number returned meets the target and is at least the floor. In a
grid of 6,912 calls (pr_treat from .005 to .9, 18 widths from .12 to
8 that are not the width of a design, n_covariates 0 to 2,
assurance NULL and .8, equal sizes and two diff_size patterns)
the 3,989 results of the development versions that met the target from
the floor are unchanged; the 2,923 others change: 919 that stopped
with an error, 1,693 returns of 3 + n_covariates whose width did not
meet the target, and 311 that met it below the floor. With equal
cluster sizes and no assurance the width falls as clusters are
added, and the number returned is the first from the floor that meets
the target. With diff_size, or with an assurance below .5, the
width need not fall at every step, and the number returned can be
larger than the first that meets the target, as before:
ss_aipe_crd_n_clusters_fixed_width(width = 0.363312492117768, cluster_size = 50, pr_treat = .9, true_var_between = 1, true_var_within = 4, conf_level = .9, diff_size = c(.5, 1, 1.5))
returns 1002, although 999 meets the target (1000 and 1001 do not). In
3,000 calls at targets set from the widths of designs, 152 results
were larger than the first such number, 150 of them with diff_size
and 2 at assurance = .3. The two both planners consider the same
floor: ss_aipe_crd_both_fixed_width(width = 3, cluster_cost = 1000, individual_cost = 1, pr_treat = .005, ...) returned 101 clusters of 3
(budget 101,303, one cluster treated) and returns 400 clusters of 2
(budget 400,800). The standardized planners start their search over
the number of clusters from this plan, so a standardized plan that
started from the untested 3 + n_covariates changes: with G = 10, seed = 113, ss_aipe_crd_es_n_clusters_fixed_width(width = 6, cluster_size = 10, true_es = .5, true_icc = .2, pr_treat = .5)
returned 3 clusters (one of them a control cluster) and returns 4; at
pr_treat = .3 it returned 3 (one treated) and returns 5. DMAR 1.0.0
also returned 3 in both. Eleven standardized calls whose start was not
the untested one, among them the calls the entries on the standardized
planners cite (67 and 233 clusters, and a cluster size of 18), are
unchanged.
A target equal to the width of a cluster randomized design is met by
that design. A width meets the target when it is no larger than the
target, inside a bracket of the search as at its ends (see "Bug
Fixes"). DMAR 1.0.0 and the development versions of 1.1.0 asked for a
width below the target inside a bracket, so the width a plan reported,
given back as width, gave the next size:
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8) plans 18
clusters with an expected width of 0.979142316129524, and that width
as the target gave 19 clusters and gives 18;
ss_aipe_crd_n_individuals_fixed_width(width = 1, n_clusters = 30, ...) plans a cluster size of 4 (0.946118731780349), which gave 5 and
gives 4; and the first call with assurance = .8 plans 22 clusters
(0.975082804893471), which gave 23 and gives 22. Only a target equal
to a width the search computes is affected: in 3,000 calls of
ss_aipe_crd_n_clusters_fixed_width() at targets set to the width of
a design and a part in 1e9 above or below it, 741 of the 2,536 results
of the development versions that met the target from the floor change,
every one at a target equal to a design's width (in 672 the number
they returned was one more), and none at the others; in 500 calls of
ss_aipe_crd_n_individuals_fixed_width() at targets set from the
widths of designs 100 plans change (28 of them with equal cluster
sizes), all at a target equal to a design's width, and 24 that stopped
with "'from' must be a finite number" return a size. A standardized
plan starts from these searches, so its start can move by one where
the target equals the unstandardized width of a design. The budget
searches already counted a cost equal to the budget as within it
inside a bracket.
A between-cluster proportion of variance explained no longer
shrinks a cluster randomized plan without a covariate. With
n_covariates = 0 the six unstandardized planners took
true_R2_between out of the between-cluster variance. The six
standardized planners started their search from that smaller plan,
while the data they simulated left it out. DMAR 1.0.0 (R2_between
with num_predictors = 0) and the development versions of 1.1.0 gave
these plans with true_R2_between = 0 and .9:
ss_aipe_crd_n_clusters_fixed_width(width = 1, cluster_size = 20, pr_treat = .5, true_var_between = .2, true_var_within = .8) 18
clusters (expected width 0.979) and 7 (0.962);
ss_aipe_crd_n_individuals_fixed_width(width = 1, n_clusters = 20, ...) a cluster size of 10 (0.994) and 4 (0.881); and
ss_aipe_crd_both_fixed_width(width = 1, cluster_cost = 10, individual_cost = 1, ...) 22 clusters of 7 (budget 374) and 8
clusters of 13 (budget 184). In the development versions
ss_aipe_crd_es_n_clusters_fixed_width(width = 1, cluster_size = 20, true_es = .5, true_icc = .2, pr_treat = .5, G = 20, seed = 113)
planned 18 clusters (0.982) with either value. Its search started at
22 or at 8 and simulated 6 or 8 designs. The calls with .9 are now
refused (see Calls That Are Now Refused). In 216 calls of the six
unstandardized planners (true_R2_between 0, .3, and .9,
true_R2_within 0 and .5, n_covariates 0 to 2, pr_treat .3 and
.5), the 48 with a nonzero true_R2_between and n_covariates = 0
are refused and the other 168 are unchanged.
mediation_mbco() statistics change where the fits were wrong, and
some calls of it and of ss_power_reg_coef() and ss_power_rc() that
returned a result are refused (see Bug Fixes for the calls and
values). ss_power_reg_coef(true_cor_XX = diag(2), true_cor_YX = c(.3, .2), which_predictor = 1.5) returned necessary_N = 89, the plan for
the first predictor, and is refused.
tidy() and results_sentence() read the intervals of ancova(),
factorial_anova(), variance_components_mls(), mediate(),
R2_mixed_effects(), nnt_from_smd(), cles(), meta_es(),
meta_r(), and meta_smd(), whose limits are now named
<estimate>_ci_lower and <estimate>_ci_upper (see Renamed Output
Rows and Columns). The values are unchanged. tidy() of ancova(),
variance_components_mls(), mediate(), nnt_from_smd(), and
cles() has a third column, conf_level, on the rows of each interval
(DMAR 1.0.x returned term and estimate alone), and in tidy() of
the meta-analysis functions that column also marks tau2 and I2 and
their limits. results_sentence() on these tables stopped ("this table
has none of these"); it now writes the sentence of the estimate term
names or, by default, of the first estimate with limits (for
ancova() of the Pygmalion example, "eta_squared_partial = 0.02, 95%
CI [0.00, 0.06]"), and on one row of factorial_anova() with
term = "eta_squared_partial" it writes that row's sentence. A table
that records no level (nnt_from_smd() with supplied limits and no
conf_level) is refused with the message that asks for conf_level.
cles(), nnt_from_smd(), and proportion_of_superiority() still
report their own interval. Now that the limits on d are named for
d, each table holds two intervals named for their estimates, and a
new attribute, ci_source, names d as the estimate whose interval the
function's own is computed from. results_sentence() reports the
common language effect size, the SRD, and the proportion of
superiority, and tidy() gives NA as the confidence level on the
rows of d, as in DMAR 1.0.x; results_sentence(x, term = "smd") now
gives the interval for d, which DMAR 1.0.x refused.
The renames themselves change no value. A call that gives the new names returns what the DMAR 1.0.x call returned, except where another entry of this release changes the result; renamed rows and columns hold the values they held under their 1.0.x names.
mr_cv() estimates the variance of the sample coefficient of
variation by Equation 32 of Kelley, Darku, and Chattopadhyay (2018), so
its results differ from those of Chattopadhyay and Kelley (2016) and of
MBESS for nearly any data. DMAR 1.0.x used Equation 22 of the 2016
article, as MBESS::mr.cv() does. Written with the skewness gamma_1
and the excess kurtosis gamma_2 of the scores, that equation estimates
kappa^2 / 2 + kappa^4 - kappa^3 gamma_1 + kappa^4 gamma_2 / 4, the
first-order term of Bao (2009, Equation 3) as printed, and the kurtosis
term should be kappa^2 gamma_2 / 4 (Equation 30 of the 2018 article,
and the erratum Bao posted for his Equation 3, which ?mr_cv
describes). The two population values are equal only
when the excess kurtosis is 0 or the coefficient of variation is 1; for
gamma scores with a coefficient of variation of .8 the value of
Equation 22 is 0.304 and the corrected value is 0.525. In a sample the
two estimates differ by (k^2 / 4)(1 - k^2)(mu4_hat / s^4 - 3), so the
stopping point changes with normal scores too: in 4,000 simulated runs
with A = 5,000,000, c = 10, and a coefficient of variation of .3 it
changed in 95 percent of the runs, by 8 observations on average, while
the mean final sample size changed little (172 with Equation 22, 170
now). For scores with positive excess kurtosis and a coefficient of
variation below 1 the population value of Equation 22 is the smaller of
the two, and with strongly skewed scores DMAR 1.0.x could stop well
short of the large-sample value sqrt(A / c) xi. In 20,000 simulated runs
with the same A and c, the mean final sample size for gamma scores with
a coefficient of variation of .8 was 375 with Equation 22 and is 480
now, where the large-sample value is 512; for lognormal scores with a
coefficient of variation of .6 it was 220 and is 440, where that value
is 545. For the article's example (35 observations, A = 400,000, and a
sampling cost of 75) DMAR 1.0.x reported a risk of 4891.284 and a
stopping rule that was met, as the article does. mr_cv() now reports a
risk of 4977.259 and a criterion of 1261.332 against N^2 = 1225: the
rule is not met, and another observation is collected. ?mr_cv has a
part on Equation 22.
mr_cv() does not stop sampling on a zero or negative variance
estimate. The estimate can be zero or negative in a small sample,
and DMAR 1.0.x put it into the stopping rule as it was, which lowered
the criterion. mr_cv(data = c(2.3, 0.61, 1.6, 0.54, 0.51, 0.41, 0.67, 0.37), A = 485, sampling_cost = 1), a pilot sample of the size the
function plans for those costs, gave an estimate of -0.058, a risk of
4.468 (less than the 8 already spent on sampling), and a rule that was
met; 18 equal scores gave an estimate of 0 and a rule that was met.
Such an estimate now means that the rule is not met: is_satisfied is
FALSE, the risk and the criterion are NA, the new attribute
variance_unusable is TRUE, and a line under the table says to
collect another observation. Scores that differ only by rounding error
(a range of at most 1e-10 times the largest absolute score), such as
equal gains computed as posttest minus pretest, and an estimate within
1e-10 of the size of its four terms are taken as an estimate of 0. A
positive estimate enters the rule and
the risk as the larger of itself and n^-3 (Equation 31 of the 2018
article). The estimate is usually below n^-3 in a small sample of
scores with a small coefficient of variation: for normal scores, while
n is below about (2 / kappa^2)^(1/3), which is 27 for a coefficient of
variation of .01 and 9 for .05. From the pilot sample size on, n^-3
adds less than 1 to the criterion, so it changes the decision only at
the pilot sample size and only when N^2 there is within 1 of the
criterion, and the stop then usually comes one observation later. With
verbose = TRUE the row v2_n holds the estimate itself in every
case. The third and fourth central moments are computed from centered
scores. The row mu_4_n, which DMAR 1.0.x computed from raw power
sums, lost its digits to rounding for scores far from zero: for
z + 100000 after set.seed(113); z <- rnorm(30) it was 3060.977 on
the computer used here, where 2.531 is right.
mr_cv() and mr_smd() print their decision. Whether the stopping
rule was met was in the attribute is_satisfied only; the printed
table showed the risk, the sample size, and the estimate. A line under
the table now gives the decision, the two numbers compared, and what
to do next: "Stopping rule: not met (N^2 = 1225 is below the criterion,
1261.332); collect another observation." from mr_cv(), for example,
and "Stopping rule: met (the sample size per group, 75, is at least the
criterion, 71.054); sampling can stop." from
mr_smd(d = .5, n_per_group = 75, A = 10000, sampling_cost = 2.4).
The tables of both functions have the class dmar_mr in front of
dmar_tbl, and no row or attribute of DMAR 1.0.x is removed.
var_cv() returns one row, var_cv_normal. It holds the
first-order variance of the sample coefficient of variation under
normality, kappa^2 (1/2 + kappa^2) / (N - 1), which DMAR 1.0.x returned
in the row var_cv_mckay; that number is unchanged. The second row of
1.0.x, var_cv_vangel, is removed. It was
kappa^2 (1/2 + kappa^2 (N + 1)/(N - 1)) / (N - 1), which Vangel (1996)
does not state and for which no source was found, and it is the less
accurate of the two for a population coefficient of .40 or less with
10 to 500 cases. Against the variance of the sample coefficient under
normality, computed with the sample mean kept away from zero, it is
4.9 percent too large at a coefficient of .30 with 10 cases, where the
kept value is 1.4 percent too large; at .50 with 10 cases it is 1.3
percent too large, where the kept value is 5.7 percent too small. Code
that reads a row by its 1.0.x name gets numeric(0), and x$value[2]
is NA.
ci_pvaf() reports the stated coverage when its lower limit is 0.
The actual_coverage row is 1 - alpha_lower - alpha_upper for every
interval the function reports. In DMAR 1.0.0, when the observed F was
below the alpha_lower critical value (an F test not significant at
alpha_lower), so that the lower limit was 0, the row was 1 minus the
p-value of the F test minus alpha_upper: ci_pvaf(F_value = 1.5, df_1 = 4, df_2 = 50, N = 55) returned 0.759 there for a 95 percent
interval. The interval from 0 to the upper limit has the stated
coverage, because a lower limit of 0 cannot lie above the population
value. In 20,000 simulated analyses of 5 groups of 11 at each of five
population values from 0 to .20, the 95 percent interval contained the
population value in 95.0 to 95.1 percent of them, and its lower limit
was 0 in 27 to 98 percent of them. An analysis whose F was so small
that no upper limit exists has no interval and was counted as not
containing the population value (2.5 percent of the analyses at a
population value of 0). The limits, the point estimate, and the tail
columns are unchanged; the help says what the tail columns hold when
the lower limit is 0.
A half-width plan of ss_aipe_R2() holds at every larger sample
size. With which_width = "Lower" or "Upper" the function plans
for the distance from the expected sample squared multiple correlation
to one confidence limit, and it returns the smallest N such that the
distance is no larger than width at that N and at every larger
one. DMAR 1.0.0 searched upward from N = 2p + 3 and returned the
first sample size at which the distance was no larger than width.
With "Upper", larger sample sizes can miss that target: with several
predictors and a small N the expected sample value is far above the
population value, the upper limit is close to that expected value or
below it, and the distance is small or negative; it rises to a maximum
as N grows and then falls. ss_aipe_R2(true_R2 = .70, width = .05, p = 10, which_width = "Upper") (population_R2 in 1.0.x) returned
23, where the distance is .043, and the distance is larger than .05 at
every N from 27 to 201; the call now returns 202. These calls with
which_width = "Upper" and a 95 percent interval change:
| true_R2 | p | width | random_predictors | DMAR 1.0.0 | now |
|---|---|---|---|---|---|
| .50 | 5 | .15 | TRUE | 13 | 33 |
| .70 | 5 | .10 | TRUE | 13 | 36 |
| .70 | 10 | .05 | TRUE | 23 | 202 |
| .10 | 10 | .05 | FALSE | 23 | 216 |
| .30 | 5 | .10 | FALSE | 13 | 107 |
| .30 | 10 | .05 | FALSE | 23 | 449 |
| .50 | 5 | .10 | FALSE | 13 | 44 |
| .50 | 10 | .05 | FALSE | 23 | 279 |
| .70 | 5 | .05 | FALSE | 13 | 123 |
They are the 9 that change among 72 "Upper" plans (true_R2 of .10,
.30, .50, and .70; 3, 5, and 10 predictors; a width of .05, .10, and
.15; random and fixed predictors), and none of the 72 "Lower" plans
for the same conditions changes. Among 3,456 plans (true_R2 from .05
to .90; 1 to 20 predictors; a width from .03 to .30; 90, 95, and 99
percent) 133 change. For 127 "Upper" plans DMAR 1.0.0 returned
2p + 3 and the plan is now larger by 2 to 2,889; 105 of them have 8
or more predictors. One "Upper" plan with random predictors is 4
larger (true_R2 = .30, p = 15, width = .03, conf_level = .99:
3,691 where DMAR 1.0.0 returned 3,687), because the limits for random
predictors are computed to a tolerance and the distance at 3,690 is
above .03 by less than .000001. Five "Lower" plans with random
predictors are 1 smaller: DMAR 1.0.0 began its search for random
predictors at the plan for fixed predictors, and in these the distance
with random predictors is within width one sample size earlier
(ss_aipe_R2(true_R2 = .10, width = .10, p = 3, conf_level = .90, which_width = "Lower") returned 103 and now returns 102). The other
3,323 plans are unchanged, and so is every plan for the full width.
With a half-width the search first bounds the sample size, for which
it needs a sample size, of those it tries up to 10,000,000, at which
the full width of the interval at the expected sample value and the
distances from that value to its two limits are all within width. A
call for which it finds none is refused (class
dmar_argument_invalid, arg = "width"). The refusal does not mean
that no sample size meets width: the bound is about four times the
sample size a half-width needs, and there the noncentrality parameter
is in the millions, beyond the range in which R's noncentral F and
beta distribution functions are accurate (?pf says its code for a
noncentrality parameter is meant for moderate values). With fixed
predictors, true_R2 = .98, 5 predictors, and 95 percent, the full
width of the interval at the expected sample value is .00068 at a
sample size of 26,624 and .00117 at 53,248, where it should be
smaller, and pf() gives no warning. The message says that the
width is narrower than can be planned with R's distribution
functions. It was found only with a very small width: of 600
calls with a width of .002 to .005 (random and fixed predictors;
true_R2 from .10 to .90; 1 to 20 predictors; 90, 95, and 99
percent), 17 of the 120 with .002 and 4 of the 120 with .0025 are
refused, and none with .003 or more. DMAR 1.0.0, which stepped through
the sample sizes one at a time, returned a plan for two such calls
after a wait: ss_aipe_R2(true_R2 = .30, width = .0015, p = 5, which_width = "Upper", random_predictors = FALSE) returned 848,292
after about ten minutes, and with true_R2 = .98 and width = .0005
it returned 11,351 after about ten seconds. Both calls are now
refused.
ci_R2() and ci_R() with random predictors report an upper limit
that does not exist as NA, with a warning. When the observed R2 is
at or below the alpha_upper quantile of its distribution at a
population value of 0, no population value is consistent with the
sample at that level. DMAR 1.0.0 reported an upper limit of 0 there,
with the requested tail areas and no warning: ci_R2(R2 = .003, N = 50, p = 3) gave an interval from 0 to 0 around an observed .003, which is
below the .025 quantile, .0046. The ci_upper row is now NA in its
value and in both tail columns, with a warning of class
dmar_result_qualified, as it already was with fixed predictors.
ss_aipe_R2_sensitivity() with random predictors counts such a
replication among those without an interval, as it does with fixed
predictors; in DMAR 1.0.0 it entered the width rows as an interval of
width 0.
mr_cv() and mr_smd() give a pilot sample size that is a whole
number in exact arithmetic as that number. A sample size is a whole
number, so the formula of the pilot sample size is rounded up, and R
can compute a whole number a little above itself:
mr_cv(pilot = TRUE, A = 1e5, sampling_cost = 1, gamma = .25) and
mr_smd(pilot = TRUE, A = 1e6, sampling_cost = 5, gamma = .25)
returned 101 in DMAR 1.0.0, where the formula gives exactly 100
(100,000^0.4, which R computes as 100.00000000000003). Both now return
100. Pilot sample sizes whose formula is not a whole number are
unchanged.
The two options of expected_r() are named in lower case,
dmar.expected_r.tol and dmar.expected_r.max_iter, like
dmar.digits and dmar.bootstrap_exploratory. DMAR 1.0.0 read
DMAR.expected_r.tol and DMAR.expected_r.max_iter; those names are
no longer read, so a value set under them is ignored and the default is
used. expected_partial_r() reads the same two options.
Never submitted to CRAN: these changes were released in DMAR 1.1.0. The section is kept as the record of the patch as it was prepared.
A patch release for the bootstrap intervals of the reliability family, prompted by a report from an analysis of the Big Five Inventory constructs (2026-10-14), in which every BCa interval for categorical omega failed with "estimated adjustment 'a' is NA" and no interval said how many replications it rested on.
Every bootstrap interval in the reliability family now says how
many replications it conditions on. reliability_alpha(),
reliability_kr20(), reliability_omega(),
reliability_omega_categorical(), and reliability() through them
record, in a bootstrap attribute, how many replications were
attempted, how many returned a finite value, how many were set aside
and why (an error from the refit, or no finite value, as from a fit
that did not converge), and the distinct error messages with their
counts. The standard error is the standard deviation of the returned
replicates and the percentile and BCa limits are their quantiles,
which the help pages now state; the printed table shows the counts
in one line, and a single warning states them whenever a replication
was set aside. Through 1.0.0 a failed replication was dropped
silently. No existing row or attribute changes.
The BCa interval no longer fails when a jackknife refit fails.
With B below the number of cases, boot estimates the BCa
acceleration from jackknife influence values, and one leave-one-out
refit that returned no value made the whole vector NA and stopped
the call. The influence values are now computed in the package under
boot's own rule (regression on the resampling frequencies when at
least as many replications as cases returned a value, the jackknife
otherwise), the leave-one-out refits that fail are counted and set
aside, and the acceleration is estimated from the rest; the
jackknife counts and the acceleration join the bootstrap
attribute. When the acceleration or the bias correction cannot be
estimated at all, the percentile interval is returned in its place
with a warning, and the ci_method attribute reads
"percentile (BCa acceleration could not be estimated: k jackknife replicates failed)", so the substitution is on the record. On a run
where nothing fails the interval and the acceleration equal boot's
to machine precision, in both regimes, so no interval that 1.0.0
computed changes.
reliability_omega() states which intervals apply to which
denominator. The closed forms, the Wald intervals, and the profile
likelihood are derived under the model implied ratio and apply to
denominator = "model_implied" only; the bootstrap methods and
"none" apply to both. The help page now says so in one place, and
a request for a combination that does not apply, such as
ci_method = "mlr" with the default observed denominator, stops
with a message naming the denominator that does.
First public release of DMAR (pronounced "Dee-Mar," for "Design, Measurement, and Analysis in R"), a greatly expanded reimagining of the MBESS package, published on CRAN on 2026-09-21.
Every line of code in every help page example now runs. Earlier
drafts carried a slow call (a bootstrap interval, a Monte Carlo
planner, a refit) as a comment so that a reader could see the
syntax without the check paying for it; the CRAN review of 2026-09-04
asked that no example line be commented out, and none is. Where a
call was slow, it runs with a small replication count and a comment
naming the count a reported analysis deserves (the defaults are
unchanged); where a page demonstrated the same thing twice, it now
demonstrates it once. The package still ships no \donttest{}
and no \dontrun{}, and a parse-based detector in the test suite
fails on any comment line that would parse as code.
The sensitivity family no longer writes a file unless asked to, and
never to a default location. The save argument is gone from
<<SAVE_LIST>>; the single switch is filename, which defaults to
NULL. Supplying a path writes the per-replication results there
exactly as save = TRUE used to; a throwaway run points it at
tempfile(fileext = ".csv").
A supplied seed is now set through withr::local_seed() (withr,
which has no dependencies of its own, joins Imports), by way of one
internal helper that every seeded function calls. The behavior is
what it was, the user's generator state is restored when the
function returns, but the package no longer touches
.Random.seed or the global environment to do it, and no function
uses <<-: the handlers that count clamped or non-converged
replications keep their counts in a local environment.
The six Monte Carlo functions that silenced every warning for their
duration with options(warn = -1) now muffle only the warnings
their loops are known to raise (lavaan convergence and variance
warnings on borderline replicates, the noncentral clamp); any other
warning reaches the user.
The Bryant-Paulson distribution function integrates over the
covariate shrinkage factor after the change of variables
delta = 1 - u^2, under which the Beta weight of Bryant and Paulson
(1976, Equation 12) is smooth on the unit interval for every number
of covariates instead of carrying an integrable singularity at the
endpoint where the distribution puts most of its mass. The two
forms agree to about 1e-10 (checked on a grid of 288 quantile,
covariate, group, and degrees of freedom combinations), the Table 1
reproduction in the tests is unchanged, and a critical value now
costs about a twentieth of what it did, which is what lets every
qbryant_paulson(), cv_bryant_paulson(), and ci_c_ancova_bp()
example run live.
Two smaller changes from the same pass. mlmr(), mlmr_mv(), and
average_variance_extracted() resample through
lavaan::lavBootstrap(), the entry point that replaced
bootstrapLavaan() in lavaan 0.7-2 (the package already requires
that version); the results are identical. The five sensitivity
functions whose ... was never forwarded anywhere
(ss_aipe_sc_sensitivity(), ss_aipe_smd_sensitivity(),
ss_aipe_sm_sensitivity(), ss_aipe_R2_sensitivity(), and
ss_power_R2_sensitivity()) no longer accept it, so a misspelled
or retired argument such as save = TRUE now stops with an error
instead of being swallowed. ss_aipe_cv() with assurance
supplied reports its noncentral accuracy note once, with the
combined count, where it used to report it twice; and
ss_aipe_cv_sensitivity() and ss_aipe_sc_ancova_sensitivity()
report that note once per run, with the number of replications it
arose in, rather than once per replication.
A package-wide audit (numerical verification against complex-step
derivatives and Monte Carlo, documentation contracts run rather
than read, and mechanical convention gates) closed out with these
fixes. analysis_of_change() now fits an intercept-only
polynomial (order = 0) under both methods; it previously stopped
inside stats::poly(), and its no-fit message for the polynomial
now counts occasions instead of suggesting start, which the
polynomial rejects.
The equivalence sensitivity siblings speak their parents'
language: ss_aipe_equivalence_smd_sensitivity() takes
delta_lower / delta_upper and
ss_aipe_equivalence_r_sensitivity() takes rho_lower /
rho_upper, positive magnitudes with the upper bound required and
the lower defaulting to symmetric, exactly as in
equivalence_smd() and equivalence_r(). The signed
equivalence_lower / equivalence_upper spellings, and their
silent default of plus or minus 0.20, are gone.
Every AIPE planner that reports an expected interval width names
the row ci_width_expected and reports the expected full width:
ss_aipe_equivalence_smd() and ss_aipe_mixed_effects()
previously reported the half-width as ci_half_width_expected
while ss_aipe_equivalence_r() reported the full width, a trap
for anyone extracting the row programmatically.
The equivalence trio attaches the conf_level attribute the ci_*
family always carried, and the equivalence_c() verdict label is
spelled "Noninferior only", matching the solid noninferior the
rest of its page uses. equivalence_r() now validates raw data
(matching lengths, at least 4 complete pairs) the way its
summary-statistics path always did. convert_r_Z() and
convert_Z_r() insist on a single value, as documented; vector
input previously recycled the term column into duplicated rows.
A follow-up ruling extended the same single-value guard to the
whole scalar conversion family (convert_R2_f() and its three
siblings, convert_delta_lambda() and its inverse, and
convert_z_normal()), with domain checks where a map's algebra
ends (an R2 at or past 1, a nonpositive group size), so no
conversion can recycle vector input into duplicated rows.
Documentation corrections from the same audit: the four nonlinear
simulators state that the first-order delta method behind
reliability and reliability_by_occasion can drift from the
realized variance ratio when random variances are large, and
document the schedule attribute; the Richards example comment
places the delta = 3 inflection at about 63% of total change, per
its own formula; the mixed-effects examples on
?analysis_of_change now run live.
The four multiple-comparison interval functions moved into the
ci_* family: ci_dunnett, ci_scheffe, ci_games_howell, and
ci_tukey_kramer replace dunnett_ci, scheffe_ci,
games_howell_ci, and tukey_kramer_ci. The suffix forms were the
only four exports naming a confidence interval outside the family
prefix, and each function now pairs with its critical-value sibling
(cv_dunnett, cv_scheffe, cv_tukey_hsd). The package is
unreleased, so the old names are gone rather than aliased.
The noncentral-distribution toolkit is named by one rule
(2026-08-21): nc_ and then the distribution, so ci_nc_t(),
ci_nc_F(), and ci_nc_chisq() replace conf_limits_nct(),
conf_limits_ncf(), and conf_limits_nc_chisq() (the snake-cased
descendants of MBESS's conf.limits.nct() and kin), and
moments_nc_t() and moments_nc_F() replace moments_nct() and
moments_ncf() to match moments_nc_chisq(). The intervals join the
ci_* family because a confidence interval, on the noncentrality
parameter that the effect size intervals are built from, is what
they compute, and "confidence limits" was vocabulary that existed
only in MBESS. Splitting nc from the distribution token retires
the one inherited exception (nct, ncf against nc_chisq) and
restores the capital \emph{F} the package writes everywhere else
(F_value, convert_F_chisq). The noncentral marker stays because
the noncentrality parameter is the estimand. plot_ci() keeps its
name: it is a plot first. The old names are gone rather than
aliased; each page names its earlier self in the description.
The correlation intervals now carry the names of their estimands,
matching the rest of the correlation family (ss_aipe_r,
ss_power_r, var_r, expected_r, equivalence_r): ci_r() is the
confidence interval for the simple Pearson correlation (renamed from
the MBESS-heritage ci_cc(), with the estimate row renamed from
est_cor to r), and ci_R() is the interval for the population
multiple correlation coefficient (its former lowercase alias
spelling is gone). Cross-references across the package were audited
against this distinction and every one now points at the interval it
meant. Because the two natural file names differ only by case, which
case-insensitive filesystems refuse, both functions live in
R/ci_correlation.R and share the ci_correlation help page.
Kish's design effect function is design_effect(); the abbreviation
deft() is gone as a function name, while the returned table keeps
its design_effect and deft rows, deft being the standard term
for the standard error inflation factor, the square root of the
design effect.
The remaining exported alias pairs were resolved to single names.
Fisher's Z is written with a capital Z throughout the package,
because it is the variance-stabilizing transform of a correlation
and not a z-score, so the converts are convert_r_Z() and
convert_Z_r() only (the lowercase spellings are gone) and the
prose and math were swept to match. The limits of agreement
function is limits_of_agreement(), with loa() kept as its short
alias and bland_altman_loa() gone (the method is named for what
it computes; Bland and Altman are credited in the references).
reliability_omega_categorical() and covmat_from_cfa() are the
only names for those functions; the reliability_omega_c() and
covmat_from_cfm() aliases are gone. The dataset alias bindings
HS_Data and Prime_Time are gone as well: every dataset carries
exactly one name, the documented snake_case one, and unlike the
alias bindings the canonical names work with data().
The vocabulary of the longitudinal simulators names what the rows
are: the identifier column identifies units (persons, animals,
trees, classrooms), n counts units, and the former group column
is population, one level per data generating parameter vector.
Measurement schedules can now be unit-specific, in every simulator:
in place of a shared target_times grid, time_range = c(lower, upper) draws each unit's own measurement times uniformly between
the bounds, with occasions fixing the number of times per unit or
c(min, max) letting it vary, so designs such as age at testing
rather than grade at testing are simulated directly. The polynomial
simulator additionally requires every unit's count of occasions to
reach P + 1, so each simulated trajectory identifies the
polynomial it came from.
analysis_of_change() closes the loop the simulators open: it fits
any of the four nonlinear change models, or a polynomial of any
order, to longitudinal data. The default two-stage method fits one
curve per unit, from that unit's data alone, with data-driven
starting values, and summarizes the unit-level parameters by their
mean, their standard deviation and variance across units (the
individual differences), and the standard error of the mean;
method = "mixed" instead estimates the proper random-coefficients
model simultaneously, lme4::lmer() for the polynomial and
nlme::nlme() for the nonlinear curves started at the two-stage
estimates, so sd_units becomes a variance component purged of
estimation noise. The help page positions the function candidly
against nlme::lmList(), nlme::nlsList(), and the self-starting
SS* curves: what it adds is the landmark parameterizations, the
simulator match, and one tidy interface across linear and nonlinear
change. With a single trajectory (or
id = NULL) it reduces to one nonlinear least squares fit reported
with asymptotic standard errors, so N = 1 is one unit's change.
Non-converging units are dropped with a single counted warning and
the effective count travels as the n_used attribute; the full
matrix of unit-level estimates rides along for plotting or as
starting values for a simultaneous nlme::nlme() fit, and the help
page states plainly that the between-unit spread of estimates
includes estimation noise, which a variance-component model
separates.
simulate_longitudinal_polynomial(), sharing its interface (several
populations of units, between-unit parameter variances and
correlations, level-one error
by variance or by target reliability, error correlation structures,
assessment-time jitter) and its long-format return that feeds
plot_trajectories() and nonlinear mixed-model fitters:
simulate_longitudinal_negative_exponential() (asymptotic
regression), simulate_longitudinal_logistic(),
simulate_longitudinal_gompertz(), and
simulate_longitudinal_richards(), whose shape parameter delta
subsumes the logistic (delta = 1) and the Gompertz (delta -> 0) as
special cases, relations the tests pin exactly. The
parameterizations are those of Kelley (2005, dissertation; 2008,
Methodology), in which every parameter is a landmark of the change
process (floor, ceiling, moment of fastest change, curvature,
inflection height) and the intercept-shifting zeta frees the lower
asymptote from zero; the Richards application follows Guo, Cheng,
and Kelley (2016). A new vignette, "Nonlinear Growth Curves and the
Meaning of Their Parameters", illustrates all four curves, the
Richards unification, and the polynomial comparison: a ninth-order
polynomial needs ten uninterpretable coefficients to track a four
parameter Gompertz inside the data and still collapses the moment it
extrapolates.equivalence_smd(), equivalence_r(), and equivalence_c()
replace tost_smd(), tost_r(), and tost_c(), and the AIPE
planners follow (ss_aipe_equivalence_smd(),
ss_aipe_equivalence_r(), with their sensitivity siblings). The
whole family now speaks the same word as the planning and plotting
surfaces that always did (power_equivalence_md(),
ss_power_equivalence_c(), plot_equivalence()). The procedure is
unchanged, Schuirmann's two one-sided tests, and each help page
says so by that name; the reported TOST \emph{p}-value keeps its
standard p_tost term. The old names are gone rather than aliased.ss_aipe_equivalence_r() plans the sample size for an equivalence question
about a Pearson correlation: the smallest \emph{N} whose
100(1 - 2 alpha)% Fisher's \emph{Z} interval (the interval
equivalence_r() and ci_r() invert) has expected width at or below the
target, with an optional Monte Carlo assurance correction. At the
conservative default planning value of 0 the answer has a closed
form, and the tests anchor the search to it. Its Monte Carlo
sibling ss_aipe_equivalence_r_sensitivity() follows the family API and
reports, alongside the family's width and coverage summaries, the
realized proportion of equivalence verdicts inside the chosen
bounds. This fills the gap noted when the equivalence family was
reviewed: the SMD had an AIPE planner for its equivalence interval
and the correlation did not.depression_bdi with its wait list control, the studentized range
procedures on test_market's six panels, the unequal-variance and
rank methods on drinks_trial's skewed outcome, the factorial and
generalized eta squared examples on pygmalion's manipulated
treatment crossed with measured grade, and the correlation,
regression, reliability, and multivariate examples on the
holzinger_swineford battery, whose second-form tests carry real
missingness that now powers the FIML demonstrations in mlmr() and
mlmr_mv() in place of artificially punched holes. Examples that
fit lme4 models keep lme4::sleepstudy, since those pages require
lme4 regardless. The ecvi() example and tests likewise moved from
lavaan's copy of the 1939 data to the package's own
holzinger_swineford. Numeric claims in example comments were
recomputed against the new output throughout.Language: en-US and inst/WORDLIST carries the package's
technical vocabulary, so devtools::spell_check() runs clean.plot_equivalence() no longer passes the fatten argument of
ggplot2::geom_pointrange(), which ggplot2 4.0.0 deprecates. The
point size is set through the size aesthetic at the value that
draws the same figure on every supported ggplot2 (>= 3.4.0), so
building the plot under ggplot2 4.0 no longer raises a deprecation
warning.reliability_omega() and reliability_omega_categorical() now
validate their inputs before the courtesy message that a default
call reports no confidence interval, so a call that is about to
fail with an informative error no longer receives advice first.Config/testthat/parallel: true, with the slowest files scheduled
first through Config/testthat/start-first; the testthat floor
moves to 3.2.0). The heaviest Monte Carlo checks were recalibrated
to smaller replication counts whose assertions still hold with room
to spare, duplicated planner calls across neighboring tests were
consolidated into single calls, and tests no longer leak messages,
notes, or printed tables into the run's output. A full
devtools::test() on the maintainer's machine dropped from about
20 minutes to about 5, with no warnings and nothing skipped.ss_aipe_*_sensitivity() member now returns the family's
documented schema: mean_ci_width / median_ci_width / sd_ci_width
for the realized interval widths, pct_ci_less_w for the proportion
of intervals at or below the planning width, pct_ci_miss_low /
pct_ci_miss_high / total_type_I_error for the empirical
non-coverage (proportions on the 0 to 1 scale, so the total is the
sum of its tails), mean_X / median_X / sd_X for the member's
own estimand, and input echoes named for their unit (total_N or
n_per_group for the evaluated size, true_X, estimated_X,
width, conf_level, and, when one was supplied, assurance).
The core term vector lives once, as the internal registry constant
.SS_AIPE_SENS_CORE_TERMS in R/dmar_tidiers.R, and a family-wide
contract test asserts every member against it. Every member's
@return now lists exactly the rows the function returns.ss_aipe_R2_sensitivity()
drops the lowercase _r2 suffixes and pct_less_w (mean_r2 is
mean_R2, mean_ci_width_r2 is mean_ci_width, the realized-limit
rows are mean_lower_limit and siblings, the one-sided widths are
mean_ci_width_lower / mean_ci_width_upper); the four term-matching
reads inside ss_aipe_R2()'s assurance search were updated with it.
ss_aipe_sc_sensitivity(), ss_aipe_sc_ancova_sensitivity(), and
ss_aipe_sm_sensitivity() retire mean_full_width and
pct_Width_obs_narrower / pct_width_obs_narrower;
ss_aipe_smd_sensitivity() retires pct_less_desired;
ss_aipe_c_ancova_sensitivity() retires width_narrower and
mean_width_obs (its standard error comparison is now
mean_se_ratio); ss_aipe_rmsea_sensitivity() retires rmsea_pop,
desired_width, mean_width, and, most importantly, an output row
named assurance that actually reported the realized width-attainment
proportion, now pct_ci_less_w so the name no longer collides with
the planning input; ss_aipe_sem_path_sensitivity() retires
width_less_than_desired and the type_I_err* trio.ss_aipe_sm_sensitivity(),
ss_aipe_smd_sensitivity(), ss_aipe_sc_ancova_sensitivity() (both
divisor branches), and ss_aipe_c_ancova_sensitivity() (which also
scaled its width-attainment row). All are now proportions on the
0 to 1 scale, completing the sweep that earlier fixed
ss_aipe_sc_sensitivity(), ss_aipe_cv_sensitivity(), and
ss_aipe_reg_coef_sensitivity().divisor = "s_anova" branch of
ss_aipe_sc_ancova_sensitivity() read the confidence limits out of
ci_sc_ancova() by row position ([2, 2] and [4, 2]) when that
function returns three rows, so the branch treated the point estimate
as the lower limit and NA as the upper: its realized widths were NA
and its "Type I error" rows were nonsense. The limits are now read by
term name, as the s_ancova branch always did, and the branch's
summaries are meaningful for the first time.mean_psi and siblings in the three contrast members,
mean_sm, mean_smd, mean_rmsea, and mean_path, computed from
the same replications the widths come from. The echo rows the schema
calls for were added where missing (ss_aipe_R2_sensitivity(),
ss_aipe_cv_sensitivity(), ss_aipe_reg_coef_sensitivity() and its
rc / src wrappers, the contrast members, and
ss_seq_c_sensitivity(), which now echoes half_width, true_psi,
true_sd, alpha_level, and m0).ss_power_R2_sensitivity() and
ss_power_reg_coef_sensitivity() echo their planning inputs (p,
true_R2 / true_b_j, estimated_R2 / estimated_b_j,
desired_power, NA when a size was specified directly, and
alpha_level), and their realized-R^2 rows follow the
meaningful-capital rule (mean_R2, and in the omnibus member
mean_F / F_crit). tidy() and glance() for the
dmar_ss_power_sensitivity class are unchanged; the echoes ride
along in glance().ss_aipe_icc() accepted and documented the six Shrout-Fleiss type
labels but planned every one of them on the single-rater scale, so
average-of-k plans were materially undersized. The planning value and
target width are now interpreted on the scale of the requested form:
an average-of-k value is mapped to the single-rater scale through the
inverse Spearman-Brown relation and each candidate confidence limit
is mapped back (the convention var_icc() uses), and type is now
validated, so an unrecognized label fails instead of silently
planning ICC(1,1). Average-of-k recommendations move: at
rho = .70, k = 3, and width = .20, planning for ICC(1,k) now
recommends n = 110 (realized mean width .20 across 2,000 Monte Carlo
replications of the F-based interval) where the undersized plan
recommended n = 69 (realized width .26); at width = .10 the
recommendation is n = 421, and at rho = .90, width = .10 it is
n = 52. Single-rater plans are unchanged. The planned form travels
as the icc_type attribute on the returned table.ss_aipe_icc_sensitivity() had the matching defect on the generator
side: it treated true_rho as the single-rater ICC no matter the
type. For the average-of-k forms it now simulates data whose
population ICC at the average-of-k level equals true_rho, so the
realized estimates, widths, and coverage refer to the form being
planned; type is validated the same way.?ss_aipe_icc that the assurance correction
over-recommends sample size by 25 to 40 subjects was backwards. At
the page's own condition (rho = .70, k = 3, width = .20,
assurance = .80) the recommended n = 79 delivers an empirical
assurance of about .77, and n = 81 is the smallest sample size that
reaches .80 (10,000 Monte Carlo replications). The Details section
now says so and points to ss_aipe_icc_sensitivity() for checking a
strict assurance target.alpha_lower critical value of
the central F-distribution, the lower noncentrality limit is 0 by
construction; that is a normal consequence of a small observed
effect, not a failure. ci_nc_F() still warns once in that
case, but the message now states the consequence for the interval
(the lower confidence limit is 0) instead of describing achieved
tail probabilities, and every function that builds its interval by
inverting the noncentral F through it (ci_snr(), ci_srsnr(),
ci_pvaf(), ci_R2() with fixed predictors, ci_eta_squared(),
ci_eta_squared_partial(), ci_omega_squared(),
ci_eta_squared_generalized() with the parametric method, and
ci_mahalanobis()) restates it for its own effect size, for example
"the lower confidence limit on the signal-to-noise ratio is 0",
at most once per call. Previously ci_snr() and ci_srsnr()
surfaced the inner wording, which pointed users to a prob_greater
column those functions do not return. The warning carries the
condition class dmar_nc_F_clamp, which the iterative callers
(ss_aipe_R2(), ss_aipe_omega_squared(), factorial_anova(),
simple_effects_AB()) now match by class when deduplicating.uniroot() message: the error now names the function that was
called, reports the F-statistic, degrees of freedom, and tail
probabilities involved, and says what to try.df, the significance lens of design_consequences()
(power, type_s_error, exaggeration_ratio) is now computed from
the noncentral t distribution of the test statistic, the exact
distribution when the standard error is estimated from the data and
the same sampling model the precision lens already used; the
exaggeration ratio integrates the truncated normal moments over the
chi distribution of the estimated standard error. The previous code
evaluated a location-shifted central t, the known-se approximation
behind Gelman and Carlin's retrodesign(), so the help page promised
the noncentral t while the code delivered the approximation.
Small-df results move: at n = 5 per group with a true effect of
d = 1, power is now 0.2863 (was 0.2469; a two-million replication
Monte Carlo of the design gives 0.2860 with simulation standard
error 0.0003, and base R's power.t.test(strict = TRUE) agrees
exactly), the Type S error is 0.00129 (was 0.00937), and the
exaggeration ratio is 1.658 (was 1.915). The differences fade as
df grows (negligible by about 60 per group), and the
df = Inf normal case is unchanged.smd_trimmed() applied the reciprocal of the Algina-Keselman-Penfield
scaling constant at every trimming proportion other than the 0.20
default: the internal constant computed the Winsorized variance of a
standard normal correctly and then returned 1 / sqrt(win_var) where
the definition calls for sqrt(win_var), and a hard-coded 0.642 at
trim = 0.20 masked the error at the default while making the
estimate discontinuous there (trim = 0.1999 returned 2.43 times the
value trim = 0.20 returned). The constant is now sqrt(win_var) at
every trim and the hard-coded branch is gone. At the default the
constant moves from the rounded 0.642 to its exact value 0.6419398,
so estimates at trim = 0.20 change by less than one part in ten
thousand; at any other trim the correction is substantial (previous
estimates were 1.47 times too large at trim = 0.10, 4.99 times too
large at trim = 0.30, and 17.99 times too large at trim = 0.40).
The help page identity is corrected to match: 0.642 is
SD(X_W) / SD(X), that is sqrt(Var(X_W) / Var(X)), for a standard
normal Winsorized at 0.20, not Var(X) / Var(X_W).smd_trimmed()'s help page promised a confidence interval on the
Yuen-Welch degrees of freedom, but the code inverted the noncentral
t at h_1 + h_2 - 2 degrees of freedom with noncentrality
d_R * sqrt(h_1 * h_2 / (h_1 + h_2)); the Yuen-Welch value was
computed, returned in the df_yuen row, and never used. The
interval now follows the construction in Keselman, Algina, Lix,
Wilcox, and Deering (2008): Yuen's t-statistic on the
trimmed-mean difference (their Equation 8) is referred to a
noncentral t distribution with the Yuen-Welch approximate degrees
of freedom (their Equation 9), and the noncentrality limits are
rescaled to the d_R metric. Feeding the summary statistics of the
paper's worked example (their Tables 1 and 3) through this
construction reproduces the printed robust effect size intervals
[0.31, 3.37] and [0.10, 1.11] (p. 119) to the precision the rounded
published inputs support, while h_1 + h_2 - 2 degrees of freedom
give [0.40, 3.31] for the first: the paper's intervals use the
Yuen-Welch value. The former noncentrality also understated the
observed statistic (by about 16% at the default trim under equal
Winsorized variances), so intervals move even where the two degrees
of freedom nearly agree: on the help page example the interval is
now [-0.88, 0.08] where the old construction gave [-0.97, 0.17].lin_ccc() now builds its confidence interval on Lin's (1989)
z-transformed standard error, with the correction noted in Lin
(2000), and the former default method = "king_chinchilli" is
removed. The removed variance was not a consistent estimator of the
sampling variance of the z-transformed CCC: when the two means and
variances are equal it inflates the correct asymptotic variance by
exactly the square of (1 + rho^2) / (1 - rho^2), already about
2.8-fold at a CCC of .5 and several hundred-fold at .95 (and by a
comparable factor otherwise), so its intervals were nearly vacuous.
The help page's own first example returned CCC = .928 with CI
[-0.977, 1.000]; the same example now returns [0.871, 0.960],
matching the independent DescTools::CCC() z-transform interval
to ten decimals, and the Lin interval's simulated coverage is .952
at nominal .95 (bivariate normal, rho = .5, n = 50, where the
removed default covered .982 with intervals 61% wider). The removed
formula was also not King and Chinchilli's (2001) estimator, so
keeping it under that name would have credited a wrong formula to
real authors; a correct King-Chinchilli variance may return in a
later release. Calls that request method = "king_chinchilli" now
fail loudly.fleiss_kappa() computed its z statistic and p-value from a
null variance that treats the category marginals as known constants,
not from the estimated-marginals null variance of Fleiss, Nee, and
Landis (1979), the correction of the standard errors in Fleiss
(1971), even though the help page credited that paper. The two
expressions coincide at uniform marginals, which is how the error
hid, and diverge as the marginals skew: at N = 1000, m = 5, and
marginals (.85, .10, .05), the old null standard deviation was 3.8
times the empirical one (.0303 versus .0080). Because the
known-marginals expression is never smaller than the corrected one
(they are equal only at uniform marginals), every affected z was
understated and every affected p-value overstated; the test was
conservative, never anticonservative. On the help page's own Fleiss
(1971) Table 1 example the z statistic moves from 15.64 to 17.65,
now matching irr::kappam.fleiss to ten decimals. The point
estimate and the confidence interval do not move: kappa is
unchanged, and the se, lower_limit, and upper_limit columns
come from the Gwet (2008) linearization, which was correct all
along.ci_dunnett(alternative = "less") formed its simultaneous upper
bounds with the signed critical value that cv_dunnett() reports for
that alternative, a negative lower-tail quantile, which placed every
bound at diff - |d| * se: below the point estimate it was supposed
to bound, and in contradiction with the adjusted p-values printed
beside it (on the PlantGrowth example both "less" rows reported an
upper limit below zero, a rejection, beside adjusted p-values of
0.162 and 0.989, no rejection). Simultaneous coverage of the true
differences measured 0.00115 against the nominal .95 in a balanced
null design with four groups and n = 10 per group. The critical
value's magnitude is now applied on the side the alternative
dictates, so the "less" bound is diff + |d| * se; coverage in the
same design measures 0.94685, and the PlantGrowth "less" upper
limits move from -0.928 and -0.063 to 0.186 and 1.051, agreeing with
multcomp::glht()'s one-sided limits to about four decimal places
(the remaining daylight is multcomp's simulated-quantile error) and
with the adjusted p-values on every row. The
alternative = "greater" interval already used the positive
critical value and is unchanged, as are the two-sided intervals and
all adjusted p-values.anova_within_two_way()
previously reported these artifacts as estimates. The
Greenhouse-Geisser and Huynh-Feldt rows for such an effect now carry
NA in epsilon, the adjusted degrees of freedom, and the p-value,
with one warning naming the condition; car::Anova likewise
declines to report the corrections when the effect's error matrix is
singular. The unadjusted and lower-bound rows are unchanged: the
lower bound 1/df is Geisser and Greenhouse's a priori bound, not an
estimate, and remains valid however few the subjects (Maxwell,
Delaney, & Kelley, 2027, Chapters 11 and 13).ci_r() accepted any sample size and went quietly wrong below
n = 4: at n = 3 it returned the vacuous interval [-1, 1] (the
Fisher's Z variance 1/(n - 3) is infinite there), and at n = 2 it
returned NaN limits. The function now stops with an informative
error for n < 4, and the n = 3 message explains why no usable
interval exists at that boundary.ci_method = "fisher" in
reliability_alpha(), reliability_omega(), and
reliability_kr20(); the Fisher interval there now stops for N < 4
with the same explanation. The package's other Fisher's Z consumers
(var_r(), equivalence_r(), meta_r(), ss_power_r(),
correlations_test(), correction_for_attenuation(),
expected_r()) already validated their sample sizes.ss_aipe_*_sensitivity family scaled some or
all of their coverage and width-attainment rows by 100 while the
rest of the family reported proportions. ss_aipe_sc_sensitivity()
reported type_I_error_upper and type_I_error_lower as
percentages; ss_aipe_cv_sensitivity() reported pct_ci_less_w,
pct_ci_miss_low, and pct_ci_miss_high as percentages beside a
total_type_I_error that was already a proportion, so its own rows
did not add up; and ss_aipe_reg_coef_sensitivity() (inherited by
ss_aipe_rc_sensitivity() and ss_aipe_src_sensitivity()) scaled
all four. These rows are now proportions on the 0 to 1 scale
everywhere, so total_type_I_error equals the sum of
pct_ci_miss_low and pct_ci_miss_high, and a realized Type I
error compares directly to 1 - conf_level with no per-function
rescaling. Term names are unchanged; only the scale moved.mediation_mbco()'s null hypothesis for an indirect effect is a
union of branches (the product is zero when any factor is), and the
constrained search could converge to a worse-fitting branch on some
platforms and OpenMx builds, inflating the likelihood ratio
statistic. When every constrained algebra is a pure product of free
parameters, the branches are now also fit directly as ordinary
unconstrained models with one factor fixed to zero, and the reported
statistic is defined by the best-fitting branch on every platform.
Non-product constraints keep the multi-start constrained search.p_value, se, ci_lower, ci_upper, p_adjusted,
conf_level, std_estimate, R2, adj_R2, df_residual, and so
on, replacing broom's dotted p.value / std.error / conf.low
across every tidier in the package. One naming system now covers
every DMAR surface; a pipeline that feeds a tool expecting broom's
dotted schema renames the columns at that boundary.content_validity_index(),
dmacs(), measurement_invariance(), and measurement_alignment()
gained bespoke tidy()/glance() pairs (items, ladder rungs, and
groups as terms; the scale and model level summaries as the one-row
glance), and a generic wide branch in the default methods serves
htmt(), average_variance_extracted(), and any future wide table
(term from the label columns, estimate from the first numeric
column, remaining columns passed through). The output vignette's
promise that the verbs answer everywhere is now literally true.B is the number of bootstrap replications everywhere. mlmr()
and mlmr_mv() rename boot_R, R2_mixed_effects() and
ci_eta_squared_generalized() rename R, and
krippendorff_alpha() renames n_boot; the effective-count row is
now B_used. Old argument names fail loudly.
cohen_kappa() and fleiss_kappa() gain bootstrap intervals
(ci_method = "percentile" or "bca", B = 10000, seed),
redeeming the kappa page's own advice that the Wald interval can
have poor small-sample coverage (Blackman & Koval, 2000; Zapf,
Castell, Morawietz, & Karch, 2016). Subjects are resampled, table
input is expanded to the equivalent pairs, and the se, z_value,
and p_value columns keep their asymptotic definitions; only the
interval changes.
average_variance_extracted() gains a percentile bootstrap
(ci_method = "percentile", B = 1000, refitting the model per
replication) and now always returns ci_lower / ci_upper columns
(NA under ci_method = "none"). No interval is possible from
loadings alone, and the function says so.
krippendorff_alpha() no longer bootstraps by default
(boot = FALSE), matching the package-wide rule that no analysis
runs a bootstrap unless asked; its verbal reliability benchmarks
were removed in favor of reporting the coefficient with its
interval.
Failed bootstrap replications are dropped, never fatal:
mediate() no longer errors when a degenerate resample returns no
indirect effect, and htmt() no longer aborts when a resample
leaves a block's average within-construct correlation nonpositive.
Both drop the replication, warn once with the count, and stop only
when fewer than 100 replications survive.
(Correction, 1.1.0: this was not true of every failure. In 1.0.0 a
resample in which a factor level was absent stopped mediate() with
an error; see the 1.1.0 entry on dropped replications.)
No help page wraps its examples in \donttest{} or \dontrun{}
any more. Under R CMD check --as-cran, one \donttest{} block
anywhere makes R run the whole example corpus twice, so the wrapper
cost time rather than saving it. The examples that were slow are
now fast. Replication counts in the cheap demonstrations were
lowered, with a comment naming what a reported analysis deserves.
No example runs a bootstrap confidence interval: those calls
are carried as commented code, so the syntax is still on the page
for a reader who wants it, and the surrounding prose explains the
interval and when to ask for it. The randomization tests keep their
resampling, since permuting the data is what those functions do and
their examples cost a tenth of a second. Anything else expensive is
commented out the same way, each passage introduced by a sentence
saying what it does and why it is not run. The data behind every
example are unchanged. The slowest help page now takes 0.38
seconds, and all 302 together take 8 seconds.
cfa_1() is now the one factor special case of cfa_k(), a
convenience wrapper rather than an independent implementation, and
the new cfa_2() is the two factor sibling. cfa_1() only
requires the data (and, when the data hold more than the items, a
vector of item names); cfa_2() takes the items of each factor as
factor_1 and factor_2. Both forward everything to cfa_k(), so
there is now a single fitting implementation, one output schema
(with confidence interval columns), and one set of fit index
choices; under a robust estimator every CFA surface now reports the
robust index versions. The former cfa_1() extras moved or
dissolved: composite reliability with the observed denominator is
reliability_omega(denominator = "observed"), the omega output
mode is the omega_f1 row of the standard table, and the term
names follow the cfa_k() convention (lambda_f1_<item>,
psi_f1_<item>, omega_f1).data and S are now separate arguments across the CFA family.
Raw data are passed as data and a covariance matrix as S (with
N); an argument never means both. cfa_k() refuses a square
symmetric matrix passed as data, naming the fix. cfa_1() keeps
its conveniences: unnamed input is auto-named y1, y2, ..., and
items defaults to every column.measurement_invariance() now fits any measurement model, not only
the one-factor case. It accepts model as lavaan syntax or as a
named list mapping factors to their items (the items argument
remains as the one-factor convenience), along with ordered,
missing for full information maximum likelihood, group_partial
for the partial invariance case, and parameterization. With ordered
indicators the ladder itself changes: thresholds carry the location
information, so a thresholds rung is fitted between configural and
metric, following Wu and Estabrook (2016) and Millsap and Yun-Tein
(2004). Constraining loadings before thresholds, as the continuous
ladder does, tests the wrong hypothesis for ordered items. The chi
square difference test is the scaled one whenever the estimator is
robust or the data are ordered, and the fitted lavaan objects are
returned on a "fits" attribute so a caller can run score tests and
partial invariance refits without paying for the ladder twice.
dmacs() reports the dMACS effect size of measurement
noninvariance (Nye & Drasgow, 2011): the expected difference between
two groups' measurement equations for an item, integrated over the
focal group's latent distribution and standardized by the pooled item
standard deviation. A score test says a loading or intercept differs
detectably; dMACS says whether the difference is large enough to
matter. The defining integral has a closed form under a normal latent
variable, so no numerical integration is used, and the two agree to
machine precision in the tests. Accepts a fitted multiple group
lavaan model or the parameters a paper reports. The help page states
plainly that the index is meaningless from a configural fit, where
the groups share no metric.
measurement_alignment() implements the alignment method of
Asparouhov and Muthen (2014). Exact invariance essentially never holds
across many groups, which leaves the ladder stalled at configural and
group comparison blocked. Alignment estimates the group factor means
and variances that make the measurement parameters as nearly invariant
as possible, minimizing a simplicity function whose fourth-root
component loss tolerates a few large differences and punishes many
small ones. Both the fixed and free identifications are available, the
optimizer runs from several starts because the surface has local
minima, and the number of distinct optima found is reported so a
solution is never presented as unique when it is not.
irt_grm() fits Samejima's (1969) graded response model. It does
so the way the rest of the package works, by fitting the categorical
factor analysis model with lavaan and converting the solution:
discrimination a_i = lambda_i / sqrt(1 - lambda_i^2) and location
b_ik = tau_ik / lambda_i. The two models are the
same model in different parameterizations (Takane & de Leeuw, 1987),
so this adds item response theory without a second estimation engine
and without leaving the factor analytic tradition. Both the normal
ogive and the logistic metric are available.
irt_information() and plot_irt_information() give the item
and test information functions and the standard error of the latent
trait, SE(theta) = 1 / sqrt(I(theta)). This is precision as a
function of where the respondent sits on the trait, which is what a
single reliability coefficient cannot express: a scale can have
excellent omega and still measure poorly over the range a study cares
about. Verified against the closed form for the dichotomous case,
which the graded model must reproduce exactly.
content_validity_index() computes the item level content
validity index from a panel of expert relevance ratings, with the
modified kappa that corrects it for chance agreement (Polit, Beck, &
Owen, 2007), Lawshe's (1975) content validity ratio, and the scale
level S-CVI/Ave and S-CVI/UA. Each I-CVI carries an exact binomial
confidence interval, because an index computed from five or six
experts is a proportion with real uncertainty and reporting it as a
bare point estimate overstates what a small panel establishes. This
is evidence about the items, gathered before any data are collected.mediation_mbco() implements the model-based constrained
optimization (MBCO) procedure of Tofighi and Kelley (2020,
Psychological Methods): a likelihood ratio test of any smooth
function of path coefficients (an indirect effect, a total effect, a
contrast of two indirect effects) formed by refitting the mediation
model subject to the nonlinear constraint that the function equals
zero. The model is specified in lavaan syntax (observed or latent
variables; parallel or sequential mediators) and fit in OpenMx,
whose optimizers support the nonlinear equality constraints the
null model requires. The function enumerates the total, direct,
total indirect, and every specific indirect pathway from x to
y, reports each with a delta method standard error and a profile
likelihood, Monte Carlo, or Wald confidence interval, tests each
with the MBCO likelihood ratio statistic and its p-value, and
reports AIC and BIC differences and the change in each endogenous
R-squared under every null model. Because the null set of a product
constraint is a union of surfaces, each null model is refit from
several starting configurations and the best feasible solution is
kept, so the reported statistic reflects the globally best-fitting
null model rather than a local branch. Accepts raw data or the
summary statistics a paper reports (covariance matrix, means, and
sample size), which reproduce the raw-data analysis exactly; the
tests replicate the published empirical example from its Table 1
moments. A group argument turns the model into a multiple-group
SEM in which every effect is estimated per group and its
between-group difference is tested, which is moderated mediation
with a categorical moderator. A moderator argument probes
continuous moderation stated in the syntax with : interaction
terms (or precomputed product columns): each moderated pathway
effect becomes a symbolically derived polynomial in the moderator,
reported as conditional effects at probe values (the mean and one
standard deviation either side by default), the index of moderated
mediation (Hayes, 2015) tested by likelihood ratio rather than
bootstrap, and, when a pathway is moderated in several places, a
joint constancy test whose null model imposes several nonlinear
constraints at once. hypotheses accepts a named list whose
multi-expression elements are likewise tested jointly. Constrained
null models are started on every branch of the constraint's null
set (including exact conditional-coefficient branch starts for
probed effects) and each solution is polished by a warm restart, so
the reported statistic reflects the best-fitting feasible null
model. Structural guardrails refuse pathway enumeration for
nonrecursive (feedback) structures and warn on binary endogenous
variables, on interaction terms no declared moderator accounts for,
and on interactions missing their main effect (the principle of
marginality).plot_mediation_mbco() draws the conditional effects of a
moderated mediation_mbco() analysis as curves over the
moderator's observed range, one per moderated pathway effect, with
a pointwise Monte Carlo confidence band, the probed values marked,
a dashed zero line whose band crossings estimate the Johnson-Neyman
boundaries, and a rug of the observed moderator values so
extrapolation is visible. The band's polynomial is the same stored
quantity the table probes, evaluated over the whole range; the help
page is explicit that the band is pointwise and that boundary
locations read from it are estimates, with the table's constancy
test as the formal companion. Returns a plain ggplot object in the
Okabe-Ito palette.var_ete() computes the variance of the estimated treatment
effect at selected covariate values in a two-group ANCOVA with
heterogeneity of regression and a random covariate (Li, McLouth, &
Delaney, 2020), the reimplementation of MBESS::var.ete(); tested
against the MBESS reference on every branch and dogfooded in the
Pygmalion vignette.
tidy() and glance() answer on every DMAR result. Two default
methods on the dmar_tbl class are the floor under the package: any
result table answers tidy() with the broom-shaped long view (term,
estimate) and glance() with the one-row wide view, repeated terms
disambiguated rather than dropped. The family methods sit ahead of the
defaults and keep their richer views, and two families join them: the
ss_aipe_* planners carry a new dmar_ss_aipe class whose tidy()
reports the planned size beside the desired width it was planned
against (term, estimate, width), through the same size registry
the power planners use and a width registry beside it, and whose
glance() widens the echoed inputs. ss_power_pcm(), the one power
planner missing its family class, joins dmar_ss_power, so its
tidy()/glance() no longer error, and
ci_eta_squared_generalized() joins its siblings in dmar_ci_anova.
One sample size vocabulary across every planner. A planner's answer
row is now named for what it counts: necessary_N (total),
necessary_n_per_group, necessary_n_per_cell, or
necessary_n_clusters; a user-fixed size echoes as specified_*; and
total_N appears only as the implied-total companion beside a per-unit
answer, never as an answer's name. Before this sweep the same word
meant different things in different planners: sample_size was the
total in ss_aipe_R2() and its seven siblings but the per-group size
in ss_aipe_c() and its three, ss_aipe_smd() said
sample_size_per_group, ss_power_pcm() carried the MBESS-era ss_c
and ss_t pair (two rows for one number in a balanced design, now one
branch-named row), ss_power_factorial_ancova() used one bare
n_per_cell for both its planned and its user-supplied branch, and the
cluster planners' answers had no prefix at all. Sensitivity outputs
echo the evaluated size under its bare unit name (total_N,
n_per_group), and the six sensitivity functions whose specified_N
argument actually meant a per-group size now call it n_per_group
(ss_aipe_c/sc/sc_ancova/smd/equivalence_smd/pcm_sensitivity). Two term
names in the cluster family that contained literal spaces are
underscored. The tidy()/glance() size recognizer no longer accepts the
legacy names, so a stray old producer fails a test instead of slipping
through, and a 216-table characterization grid recorded before the
sweep reproduces identically after it, so only names moved, never
values. The package is unreleased; old names fail loudly.
The number of predictors is p everywhere. ci_rc(),
ci_src(), ss_aipe_rc(), ss_aipe_src(), ss_power_rc(), and
plot_R2() renamed J to p, and ci_R() renamed K to p,
matching ci_R2(),
ci_reg_coef(), and the rest of the regression family. J remains
only in the partial and semipartial correlation family, where it
counts the variables partialed out (a different quantity; J = 0 is
the simple correlation). The package is unreleased, so the old
names fail loudly rather than being aliased.
ci_rc() and ci_src() are documented as the thin wrappers they
are around ci_reg_coef(), the general engine, with titles that
distinguish the unstandardized, standardized, and general cases.
Every function help-page title is now AP title case with no trailing period, matching base R convention (previously the package mixed sentence case and title case).
A numerical audit compared every displayed formula and quoted magnitude on these pages against the quantity the function computes. In each case the computation was correct and the page was not, so no returned value changes; the documentation now matches the code, and new tests recompute each corrected expression independently and assert agreement with the function output.
expected_partial_r() displayed the expectation with every index
one lower than the Olkin-Pratt formula under the n to n - J
substitution the code makes; as printed, the formula can exceed 1, an
impossible value for the expectation of a correlation. The page now
prints E[r] = rho 2F1(1/2, 1/2; (n - J + 1)/2; rho^2)
Gamma((n - J)/2)^2 / (Gamma((n - J - 1)/2) Gamma((n - J + 1)/2)),
which matches the function to machine precision. The bias prose also
described E[r] - rho while the returned bias column is rho - E[r];
the sign convention now matches the column, and the quoted magnitudes
are the true +0.00624 (rho = 0.4, n = 30, J = 2) and +0.00888
(J = 10).expected_r() quoted bias magnitudes with the wrong sign and
size (-0.024 and -0.006); the true values are +0.021 (rho = 0.5,
n = 10) and +0.0065 (n = 30), as the page's own example already
reported.icc_lmer() printed the variance of the Bonett (2002)
L-transform as 1 / (2 (n - 2)), omitting the k / (k - 1) factor the
code applies; the page now prints k / (2 (k - 1) (n - 2)). A reader
hand-building the interval from the old page at k = 2 would have an
SE too small by a factor of sqrt(2) and roughly .84 coverage instead
of .95.anova_within_two_way() printed the lower-bound epsilon as
1 / (df - 1); the code computes 1 / df, the attainable infimum, with
df the effect's numerator degrees of freedom.loa() printed a symmetric central t interval for the CIs on the
limits of agreement; the code computes the Carkeet (2015) exact
noncentral t interval, which is asymmetric about the sample LoA. The
page now shows the noncentral form: quantiles of the noncentral t
with noncentrality parameter k sqrt(n) for the upper LoA and its
negative for the lower.variance_components_mls() printed a symmetric interval with
unsquared constants and a -MS_b MS_w / n cross term that appears
nowhere in the method; the code computes the genuine Burdick-Graybill
(1992, equations 2.4.1--2.4.5) form, and the page now prints V_L and
V_U with the squared constants and the G_12 / H_12 cross terms.ss_aipe_mixed_effects() printed Var(beta-hat) with a trailing
design-effect factor the code does not apply; for the
cluster-mean-centered level-1 slope the variance is
sigma2_y (1 - rho_I) / (N sigma2_x), and the page now explains why
the design effect does not enter.ci_smd() pointed paired designs to ci_smd_c(), which is the
interval for Glass's estimator (two independent groups, control group
SD) and takes no correlation between paired measurements, so
following the pointer reproduced essentially the independent groups
interval. The page now states plainly that a paired-design SMD
interval is not currently provided.power_fisher_exact() attributed the alternative distribution to
Wallenius; the code computes Fisher's noncentral hypergeometric, the
conditional distribution of one binomial count given the total of two
independent binomials, which is what conditioning on the margins of
the 2 x 2 table produces. The prose and references now cite Fisher
(1935) and Fog's (2008) sampling-methods paper.ci_scheffe() said the default returns a - 1 pairwise contrasts;
it returns all a (a - 1) / 2 of them, as the sibling pages already
stated.A continuation of the audit above, covering another two dozen help pages. As before, the computations were correct and the pages were not; no returned value changes, and where a corrected number is load-bearing a test now recomputes it independently.
ci_rmsea()'s
90 percent example described its upper limit as landing below 0.05;
the limit is 0.052, just above the Browne and Cudeck close fit
threshold, and the page now draws the conclusion that follows (close
fit is not established, even though the point estimate sits below the
threshold). obrien_test()'s Hunter example quoted p = .2595, which
matches neither the quoted F = 1.29 nor the exact statistic; the page
now quotes p = .260, as computed. combine_p()'s Edgington sum for
the Raudenbush example prints 7, not "near 6.9". ci_c()'s
hypertension example described 24 subjects across group sizes of 4,
6, 5, and 5; it says 20.mediation_mbco() separates the published memory-example
likelihood ratios from what its example computes. Run from the
rounded Table 1 summary statistics, the two null branches give
LRT = 71.31 (the best-fitting branch, the statistic reported) and
179.02; the 72.54 and 175.77 the page previously quoted are the
full-precision-moments values, and 175.77 is now attributed to
Tofighi and Kelley (2020) as the published value, from the
worse-fitting branch on which their optimizer stopped. The page notes
that the difference comes from running from the published (rounded)
summary statistics.ss_power_indirect_effect()'s example no longer equates the
approximation with the simulation benchmark. The joint significance
approximation returns necessary_N = 65 for a = b = .39 at power
.80; raw-data simulation puts the power at 65 nearer .77 and reaches
.80 near N = 70, in line with the somewhat larger requirement in
Fritz and MacKinnon's (2007) simulation-based table, and the example
comment now says exactly that. The raw-data validation test's
relative tolerance is widened from 0.03 to 0.06 with the reason
recorded in place: the approximation sits about 3 percent above the
simulation at those settings on any seed, so the old tolerance failed
on some seeds for the approximation gap alone.prime_time_achievement's corporation summaries are recomputed
from the derived corp_id key. The bare corp column merges the
two corporations that share code 2400, so the page overstated the
largest corporation: corporations have 15 to 756 students (median
117), not 15 to 808 (median 124), and the three-level null model
decomposition is 16.29 / 22.72 / 240.43 with a corporation ICC of
0.058 (previously 16.50 / 22.70 / 240.43 and 0.059). The data set's
tests now anchor the corrected values.bessel_errors' expected counts imply a normal sigma near 0.22,
not 0.2. A least squares fit of the half-normal bin probabilities
to Bessel's expected frequencies gives 0.216; at 0.2 the first bin
alone is off by about eight observations.which_width contract of ss_aipe_omega_squared(),
ss_aipe_icc(), and ss_aipe_partial_r() now says what the code
does. The pages described "Lower" / "Upper" as a one-sided
half-width; both settings interpret width as half the full width
and return the same sample size. The pages now say so, note that an
asymmetric interval's realized half-widths differ from each other
and from half the full width, and state that a genuinely one-sided
width target is not currently offered.ci_mahalanobis() documents the tail arguments its backend
accepts. The page promised that supplying alpha_lower and
alpha_upper recomputes conf_level; that call errors, because the
tails pass straight through to ci_nc_F(), which refuses a
non-NULL conf_level beside them. The page now directs users to set
conf_level = NULL and supply both alphas, and a test pins the
contract.ci_nc_F() and ci_nc_chisq() state the
monotonicity each search actually exploits. The pages claimed each
tail probability is strictly decreasing in the noncentrality
parameter; the lower-limit condition works on the upper tail, which
is strictly increasing. Each page now gives the direction per tail
and notes that both roots are located on the decreasing lower-tail
scale.ss_aipe_equivalence_smd() described an
expected half-width target of omega where the code targets omega / 2
(full width omega), and a 1-row return where the table has 5 rows;
both now match, and the return's stale term names are corrected.
The stale sample_size return term on the ss_aipe_c(),
ss_aipe_c_ancova(), and ss_aipe_sm() pages now reads
necessary_n_per_group, necessary_n_per_group, and necessary_N.
ci_eta_squared()'s N is documented as the total number of
observations (for aovlist fits, one more than the sum of all
effect and residual degrees of freedom), matching its Details.
mlmr()'s adj_R2 is documented as using the complete cases
(N_complete), which the code deliberately uses, rather than the
lavaan reported N. ci_c_ancova_bp()'s contrast_type entry
dropped inline width expressions that omitted the 1/sqrt(2) and
sqrt(1/n) factors and now points to the correct display in Details.
ci_sc()'s note no longer contains a sentence directing users to
pass the error variance where the argument (and the rest of the
page) wants the error standard deviation. The multiplicative
diff_size illustration on the ss_aipe_crd_* pages was a verbatim
copy of the additive one; it now works the multiplicative form's
arithmetic (cluster size 25 with diff_size = c(0.8, 1, 1.2) gives
sizes 20, 25, 30, recycled across clusters).equivalence_c() performs the two one-sided tests procedure and the
companion noninferiority test for a linear contrast of group means
against bounds stated in raw units of the response, with one pooled
error term, a summary-statistic and a direct (estimate, SE, df)
interface, a benchmark argument for comparisons against a known
constant, and a five-way verdict (equivalent, superior, inferior,
non-inferior only, inconclusive). Reproduces
emmeans::test(..., side = "equivalence" | "noninferiority") to
machine precision.power_equivalence_c() computes the exact TOST power for a
contrast by integration over the chi distribution of the estimated
error standard deviation, generalizing power_equivalence_md() to
arbitrary weights and unequal group sizes, and the noninferiority
power in closed noncentral t form.ss_power_equivalence_c() finds the smallest per-group sample
size whose exact TOST (or noninferiority) power reaches a target,
the declaration-probability counterpart of the width-targeting
ss_aipe_c().ss_seq_c() and ss_seq_c_sensitivity() implement the
purely sequential fixed-width confidence interval procedure for a
contrast of Chattopadhyay, Bandyopadhyay, Kelley, and Padalunkal
(2025), with cost-optimal allocation across groups; the sensitivity
sibling verifies first-order efficiency and near-nominal coverage by
Monte Carlo.plot_equivalence() draws contrast estimates and their
intervals against the equivalence region, colored by the equivalence_c()
verdict.R2_mixed_effects() reports the Nakagawa and Schielzeth (2013)
marginal and conditional R-squared for a fitted lme4 or nlme
mixed model. It agrees to machine precision with r2mlm and with the
direct Johnson quadratic form across the model classes the tests pin:
random-intercept, correlated random-slope, split (||) random-slope,
multi-slope, and nested two-level models. On split random-slope models
performance/insight omits the random-slope variance from its
marginal and conditional R-squared, so DMAR matches r2mlm and the
Johnson form there and, by design, not performance::r2_nakagawa; the
two agree on random-intercept and correlated-slope models.R2_mixed_effects_decomposition() implements the Rights and Sterba
(2019) integrative framework of mixed-effects model R-squared measures: the full family of
total, within-cluster, and between-cluster measures from a complete
five-source decomposition of the outcome variance, matching the
authors' r2mlm reference implementation to machine precision across
the same tested model classes.knit_print.dmar_tbl() renders
DMAR results as formatted tables in knitted documents, as_kable()
produces a knitr::kable view, and results_sentence() writes an
APA-style "estimate, CI" sentence from any interval-carrying result.broom support (tidy() / glance()) added for the elementary
tests (welch_t(), summary_t_test(), contrast_test()) and the
simultaneous-comparison intervals (ci_dunnett(), ci_tukey_kramer(),
ci_scheffe()).tidy() and glance() summarize every closed-form ss_power_*
planner that reports one size and one power, from the effect size
planners to the ANOVA, ANCOVA, contrast, cluster, and mediation
designs, reporting the design's planning unit (per group, per cell,
per subject, or per cluster, or the total for the one-way ANOVA) beside
its power. The lookup that resolves the size row learned the
design-specific names (n_per_cell, n_subjects, J_per_arm,
ss_per_group), so the meaningful names are kept rather than
homogenized. The Monte Carlo sensitivity siblings
ss_power_R2_sensitivity() and ss_power_reg_coef_sensitivity() carry
a dmar_ss_power_sensitivity class whose tidy() places the empirical
and analytic power side by side.ss_power_composite_ancova_2group() plans the per-group sample size for
composite power in a two-group ANCOVA: the probability that the group
effect, the covariate effect, and the group by covariate interaction
are all significant in the same study, the quantity a design must be
planned against when its conclusion needs more than one result to hold
at once. Composite power is not the product of the marginal powers,
because every test divides by the same error estimate and the tests are
positively dependent even when the effects are orthogonal. A one
dimensional integral over the chi square distribution of that estimate
evaluates the composite deterministically, with no simulation, and the
plot() method draws the population effects the plan rests on. This
opens the new composite power family.ss_power_composite_factorial_ancova() and
ss_power_composite_factorial_anova() carry composite power to any
balanced factorial design: name any set of main effects and interactions
in effects, each with its Cohen's f or partial eta squared, and the
planner returns the per-cell sample size at which all of them are jointly
significant. The effects are noncentral F tests sharing one error
estimate, so the same shared-error integral applies; a single effect
reproduces ss_power_factorial_anova() (or ss_power_factorial_ancova()
with a covariate) exactly, and the two-effect composite matches a direct
simulation. The ANOVA function is the no-covariate case named directly and
admits no covariate. With no covariate the composite is exact to quadrature
precision. The effects can be stated as sizes (Cohen's f or partial eta
squared) or as a full array of population cell means with a common within-cell
standard deviation, from which each effect's f is read off the analysis of
variance decomposition of the means. The plot() method draws the purported
population values: the cell-mean pattern (with error bars of one within-cell
SD) when means were given, or the effect sizes annotated with their marginal
power otherwise, with the composite power in the subtitle either way.ss_power_composite_factorial_ancova_het() carries composite power to a
factorial ANCOVA whose covariate slope differs across the cells. The average
slope (the covariate main effect) and the factor by covariate slope
heterogeneity are then testable effects that can join the composite alongside
the factorial mean effects. The full model fits the means, the covariate, and
every factor by covariate slope, so the residual has N minus twice the cells
degrees of freedom; a "mean", "covariate", or "slope" effect is named in
effects and sized by a Cohen's f or read from population values (cell
means and a covariate outcome correlation per cell, with a common within-cell
SD). The one-factor two-level case reproduces ss_power_composite_ancova_2group() to
machine precision, and a three-effect composite matches a direct simulation of
the heterogeneous-slope model. Kept separate from the common-slope
ss_power_composite_factorial_ancova() for ease of use. The plot() method
draws the population regression line in each cell, so heterogeneous slopes read
as lines of different angle.ss_power_composite_ancova() and ss_power_composite_anova() are the
general entry points to the composite power family, covering a one-way design
with any number of groups as well as any factorial arrangement. The ANCOVA
planner takes a slopes argument, "homogeneous" for one common covariate
slope or "heterogeneous" to let the slope differ across cells and make the
covariate and slope-heterogeneity effects testable; its heterogeneous one-way
case is the a-group generalization of the two-group
ss_power_composite_ancova_2group(), and its two-level case reproduces it
exactly. The ANOVA planner is the no-covariate design named directly, with a
covariate-free interface that points to the ANCOVA version when a covariate is
present. Both accept effect sizes or population values (cell means with a
common within-cell SD) and forward to the factorial planners, so the same
broom tidy()/glance() summaries and plot() figures apply.ss_power_composite_sem() and ss_aipe_composite_sem() carry
composite sample size planning to structural equation models, for both
design goals. The researcher states the population as a fully fixed lavaan
model (or its cov_sem() covariance matrix), labels the parameters of
interest in the free analysis model (structural paths, loadings,
covariances, or :=-defined quantities such as an indirect effect), and
the planner finds the smallest N at which every labeled parameter is
statistically significant in the same study with the desired probability
(ss_power_composite_sem()), or at which every confidence interval is
sufficiently narrow, in expectation or with a stated assurance for the
joint event (ss_aipe_composite_sem()). No closed form covers a set of
dependent SEM estimates, so both planners run an a priori Monte Carlo
simulation (Muthén & Muthén, 2002; Maxwell, Kelley, & Rausch, 2008):
data are drawn from the population covariance matrix, the analysis model
is fit G times per candidate N, and a search seeded by the analytic
Wald approximation brackets and bisects to the smallest integer meeting
the goals; each reported power or proportion travels with its simulation
standard error, and a seed argument makes a plan reproducible. The
power planner joins the dmar_ss_power broom family; marginal
power_<label> and width_within_desired_<label> rows show which
parameter binds the design. Both planners handle a population mean
structure, so latent growth curve targets such as the slope factor's
mean are planned the same way: cov_sem() now also returns mu_theta,
the model implied means of the observed variables, a mu argument
supplies means beside a hand-built Sigma, and the simulated data carry
those means whenever the analysis model has a mean structure. The
"Composite Sample Size Planning for SEM" vignette
(vignette("composite_sem_planning")) walks through the full workflow
for a mediation model with observed variables and for a linear latent
growth curve, planning both composite power and joint accuracy.cohen_h() returns Cohen's h, the effect size for the difference
between two proportions on the arcsine (variance-stabilizing) scale,
h = 2 asin(sqrt(p1)) minus 2 asin(sqrt(p2)). It is signed and is the
proportion analog of the standardized mean difference (smd()), so a given
h carries the same detectability wherever the proportions sit, which a raw
difference does not.contrast_adjusted() tests an arbitrary contrast among
covariate-adjusted cell means in a factorial ANCOVA, matching
emmeans::contrast().dmar_tbl display layer,
and literature-synonym @concept tags (Cohen's d, Cronbach's alpha,
Cohen's U3, ...) make the marquee estimators searchable by their
eponyms.ss_power_smd() and ss_aipe_smd() echo their user-supplied
planning inputs as rows of the returned table, so the assumptions a
design was planned under travel with the result as tidy data rather
than a printed footer. The supposed effect is labeled supposed_smd
to make clear it is a value the researcher posits (a minimally
important effect or a value believed to be true in the population),
not a sample estimate; power planning also echoes desired_power,
alpha_level, and a numeric tails (2 or 1), and AIPE echoes
width (and assurance when supplied). The value column stays
numeric.reliability_omega_h() now places the observed total-variance
denominator on the same maximum-likelihood (N-divisor) metric as the
fitted loadings, matching MBESS::ci.reliability(type = "hierarchical")
and semTools::compRelSEM(obs.var = TRUE).DMAR is the modern, more general reimplementation of MBESS that reflects how the methods are now used across quantitative psychology, sociology, education, management, marketing, and information systems. MBESS remains stable on CRAN; DMAR is the recommended path forward for new users.
reliability_alpha() and reliability_omega() handle missing
data by full information maximum likelihood, with auxiliary
variables. The measurement family thereby catches up with mlmr(),
whose vignette makes the case against listwise deletion that these
functions previously ignored. Two new arguments, also passed through
by reliability(): missing = c("listwise", "fiml"), defaulting to
listwise deletion so no existing result changes, and aux, a
character vector naming auxiliary columns of data that enter as
saturated correlates (Graham, 2003): correlated freely with each
other and with every item's residual, never loading on the factor.
Supplying aux implies missing = "fiml". The analytic alpha
applies the classical formula to the FIML estimate of the item
covariance matrix; the model based estimators fit with lavaan's
missing = "ml"; robust omega's observed total variance comes from
the FIML covariance matrix. MBESS documents an aux argument on
ci.reliability() that no longer runs (upstream API drift in
semTools), so DMAR implements the saturated correlates model
directly in lavaan syntax rather than depending on it. The returned
table now always carries an N_complete row beside N, making the
cost of listwise deletion visible at a glance, and the treatment is
recorded in missing and aux attributes. Interval methods that
cannot be made correct under FIML (the complete-data closed forms,
ADF, the profile likelihood) are errors, never silent fallbacks;
the bootstrap resamples partially observed rows and refits by FIML.
Under MAR missingness driven by an auxiliary, the test suite shows
FIML-with-auxiliary reducing the bias of listwise deletion roughly
38-fold across 200 replications, and a hand-specified saturated
correlates model in lavaan reproduces the estimates exactly.
reliability_omega() gains a denominator argument selecting
how the total variance in the denominator of coefficient omega is
estimated: "observed" (the default: robust omega, the variance of
the composite estimated directly from the data, the coefficient
Kelley and Pornprasertmanit, 2016, call hierarchical omega and
MBESS::ci.reliability() calls type "hierarchical") or
"model_implied" (the textbook form). The two definitions coincide
in the population when the single-factor model is correctly
specified; only robust omega retains its interpretation as the
proportion of the variance of the composite actually computed when
the model is misspecified, which is why it is the default. The help
page states the properties of each choice and the distinction from
the bifactor omega-hierarchical of Zinbarg, Revelle, Yovel, and Li
(2005). The returned object records the choice in a denominator
attribute.
cfa_k() accepts ordered-categorical items (ordered = TRUE or
a vector of item names; raw data required, each factor all ordered
or all continuous). The model is fit by WLSMV to polychoric
correlations with thresholds in the theta parameterization, which
keeps every defined measurement quantity available, with robust
standard errors. Because a sum score of ordered items lives on the
metric of the observed categories rather than the latent response
metric, each ordered factor's omega is the Green and Yang (2009)
categorical sum score omega computed from the same fit; the
substitution is announced in a message, recorded per factor in an
omega_metric attribute, and the delta method interval columns are
NA for those rows (use reliability_omega_categorical() for a
bootstrap interval). A single-factor ordered cfa_k() reproduces
reliability_omega_categorical() to 1e-6 in the regression tests
despite the different parameterizations, which pins the
parameterization invariance of the computation.
reliability_alpha() gains an estimator argument carrying the
two routes to coefficient alpha, which were briefly two functions.
estimator = "analytic" (the default) is the classical closed-form
equation applied to the observed covariance matrix, the number a hand
calculation produces; estimator = "model_implied" is the reliability
implied by the tau-equivalent (equal loadings) single-factor model fit
by maximum likelihood, which brings a delta method standard error, a
robust (Satorra-Bentler) variant, the profile likelihood interval, and
a testable fit of the equal-loadings claim. Users of MBESS will know
them as ci.reliability(type = "alpha") and type = "alpha-cfa");
the point estimates match those to 1e-6 in the regression tests. The
value "model_implied" is deliberately the word reliability_omega()
already uses for the analogous choice, so one vocabulary covers both
coefficients, and the reliability() wrapper gains a matching
estimator argument beside its denominator.
This replaces the short-lived reliability_alpha_analytic(), whose
name was inherited MBESS jargon that described neither the estimand
nor the contrast. Merging the two also removes a defect their
separation created: reliability_alpha(ci_method = "likelihood") used
to report the classical point estimate beside an interval profiling
the model implied coefficient, two different quantities, so on a
misspecified model the interval could exclude the estimate printed
above it (on the psych bfi A scale, estimate 0.431 against an
interval of [0.591, 0.636]). Each interval method now belongs to the
estimator that can supply it, and asking for one the chosen estimator
cannot give is an error naming the estimator to use instead.
Profile likelihood confidence intervals join the closed forms:
ci_method = "likelihood" in reliability_omega() (model implied
denominator) and reliability_alpha() (profiling the
tau-equivalent, CFA-based alpha). The interval is the set of
population values not rejected by the likelihood ratio test,
computed by refitting the single-factor model under a nonlinear
constraint on the model implied reliability. It respects [0, 1], is
not forced to be symmetric, works from raw data or a covariance
matrix, and matches MBESS::ci.reliability(interval.type = "ll")
to the fourth decimal in the regression tests.
No bootstrap runs unless the user requests one, anywhere in the
reliability family. Robust omega and categorical omega, whose
confidence intervals are bootstrap based, report the point estimate
by default with a message naming the exact call that produces the
recommended interval (percentile or BCa for robust omega; BCa for
categorical omega). The model implied omega keeps its closed-form
robust ML interval as the default, and alpha and KR-20 keep their
closed forms. When a bootstrap is requested, B = 10000
replications is the default.
reliability_omega_h() was removed. Its coefficient is exactly
reliability_omega(denominator = "observed"), and the "h" (for
"hierarchical") described no hierarchy in the single-factor model
the function fits while inviting confusion with the bifactor
omega-hierarchical. The reliability() wrapper drops
type = "omega_h" and instead forwards a denominator argument to
reliability_omega(). The documentation refers to the coefficient
as robust omega, records the original motivation (model misfit as
minor common factors, with the coefficient isolating the general
factor's variance against the observed composite variance) and the
authors' retrospective preference for a name stating the behavior,
states the qualifications the word robust requires (robust to
misspecification of the total variance only; distinct from
outlier-robust estimation and from robust standard errors), and
notes the design principle shared with categorical omega: in both,
the total variance in the denominator is not taken from the fitted
factor model.
reliability_omega_categorical() is the categorical omega
function's full name, and its only name (an earlier
reliability_omega_c() alias was removed). The reliability()
wrapper's canonical type is "omega_categorical" with "omega_c"
accepted as a shorthand. The coefficient attribute is now
"omega_categorical".
The reliability vignette was rewritten around the framing of Kelley and Pornprasertmanit (2016): choosing the coefficient (a claim about the measurement model and the composite being scored) and choosing the interval (an empirical performance question their Monte Carlo studies answered, which is what the family's defaults encode). One running example walks alpha versus omega, the two omega denominators under a minor-factor contamination, categorical omega under same versus differing threshold patterns, and coefficient H as a different composite rather than a different assumption.
cohen_kappa() now accepts a published k x k frequency table
(table =) in place of raw rater vectors, and custom weight
matrices in either scaling: agreement weights (diagonal 1) or
Cohen's (1968) ratio-scaled disagreement weights
(weight_scaling = "disagreement", zero diagonal, invariant to
positive rescaling, converted internally via w = 1 - v / max(v)).
Asymmetric weight matrices are supported for validity designs where
the two directions of a confusion carry different costs. The help
page replicates Cohen's (1968) Table 1 analyses in full: unweighted
kappa .492, weighted kappa .348 under his disagreement weights,
.574 with the 6 and 1 weights interchanged, his asymmetric
computer-diagnosis validity example (.353), and his Formula 10 and
13 standard errors (.0901, .0916, z = 3.80), which the examples
reproduce for the historical record while the function reports the
Fleiss, Cohen, and Everitt (1969) standard error that superseded
them. When both raters are supplied as factors with the same level
set, their level order is now respected instead of alphabetical
sorting, which previously could silently misalign ordinal categories
with linear, quadratic, or custom weights. Every result also carries
a cells attribute holding the per-cell detail in the form of
Cohen's Table 1: observed proportion, chance-expected proportion,
and the weights (both scalings when disagreement weights were
supplied), one row per cell of the confusion matrix.
diagnosis_agreement ships Cohen's (1968) Table 1 as a data
set in its original layout (Judge B in rows, Judge A in columns):
one row per cell with the frequency, Cohen's ratio-scaled
disagreement weight, the observed proportion, and the
chance-expected proportion. The reconstruction is verified against
every quantity computed from the table in the paper.
A weighted kappa vignette works Cohen's (1968) illustration in
full on the diagnosis_agreement data: unweighted and weighted
kappa, both weight scalings, his Formula 10 and 13 standard errors
beside the Fleiss-Cohen-Everitt interval, linear and quadratic
weighting with the weighted-kappa-equals-r identity under equal
marginals, and the asymmetric validity example, including a neutral
working of how the printed weight display's orientation relates to
the published values.
cfa_k() fits a confirmatory factor analysis model with one or
more factors, each specified by naming its indicators. The
measurement structure is specified by describing what is constrained
(equal_loading, equal_intercept, equal_error, each a single
value or per-factor), and the function names the classical structure
the description implies (congeneric, essentially tau-equivalent,
tau-equivalent, essentially parallel, or parallel; Graham, 2006) in
the printed header and the "model" attribute. The table reports
every estimate with a confidence interval, and per factor coefficient
omega, the average variance extracted, and coefficient H as lavaan
defined parameters with delta method standard errors and intervals
(no semTools involvement). output = "measurement" gathers the
measurement properties, the latent correlations, and htmt() per
factor pair; output = "fit" hands back the lavaan object so two
descriptor fits feed lavaan::lavTestLRT() directly. The RMSEA
interval level is reported explicitly as the rmsea_ci_level row.
plot_cfa_k() displays the item-level loadings, error variances,
or intercepts of a cfa_k() fit with confidence intervals, one panel
per factor, with a dashed reference line that shows the equated value
(or, for free estimates, the informal "one common value" anchor), so
the equality questions behind the classical structures can be seen
before they are tested.
bifactor_indices() computes the bifactor dimensionality and
reliability indices (ECV, omega, omega hierarchical and hierarchical
subscale, PUC, and coefficient H) from a fitted bifactor lavaan model,
with a guard that flags improper (Heywood) solutions (Rodriguez, Reise, &
Haviland, 2016).
simple_structure() quantifies Thurstonian simple structure in a
loading matrix: Hofmann item complexity, the hyperplane proportion, and
pure/complex item counts.
ecvi() gives the Browne and Cudeck (1989) expected cross-validation
index for a covariance-structure model, with a confidence interval derived
from the noncentral chi square (ci_nc_chisq()); accepts a
lavaan fit or a published fit table.
common_method_single_factor() and common_method_marker()
implement the single-common-factor (Harman) screen and the Lindell and
Whitney (2001) marker-variable adjustment for common method variance.
The Harman screen is implemented factor analytically: a one-factor
model is fit by maximum likelihood (stats::factanal()) and the
reported proportion of variance is the common factor's, not a
principal component's.
cv_bryant_paulson() gives the simultaneous
critical value (the ANCOVA member of the cv_* family, reducing to
sqrt(2) * cv_tukey_hsd() when there are no covariates), and
ci_c_ancova_bp() places simultaneous (familywise) confidence
intervals on contrasts of adjusted means, the familywise counterpart
of the per-comparison ci_c_ancova(). Both are computed exactly from
the Bryant--Paulson generalized studentized range distribution
(qbryant_paulson() / pbryant_paulson() / dbryant_paulson()),
evaluated as a Beta mixture of ptukey() rather than read from a
table. Three vignettes cover the critical values, an end-to-end
ANCOVA workflow, and a simulation confirming exact familywise error
control. Implements Bryant and Paulson (1976) and Bryant and Bruvold
(1980).regions_of_significance(): the values of the covariate at which
two groups differ significantly when the within-group regression
slopes are not equal (heterogeneity of regression), where the group
difference is a function of the covariate rather than a single
number. The boundaries solve the quadratic that sets the squared
group difference against its sampling variance at the critical value,
the Johnson and Neyman (1936) procedure; with more than two groups
the calculation is carried out for every pair.
plot_regions_of_significance() draws the estimated difference
across the covariate with the confidence band the region is read
from, so the plot is the decision rule.cv_f(), cv_chisq(), and cv_bonferroni_f() complete the cv_*
family's coverage of the Maxwell, Delaney, and Kelley (2027) Appendix.
The family had cv_t() and cv_z() but not the F or chi square
counterparts, and the Bonferroni F table had no function at all.
cv_f() covers Appendix Table A.2, cv_bonferroni_f() Table A.3, and
cv_chisq() Table A.9; the tests assert each against the printed
values.cv_f() and cv_chisq() default to alternative = "greater" rather
than the "not_equal" that cv_t() and cv_z() use. Neither
distribution is symmetric, and both are used one-sided in the upper
tail for the tests they serve: a restricted model fits worse than a
full one, so evidence against a restriction is a large F, never a
small one. Both tails remain available, through
alternative = "not_equal" or through alpha_lower and alpha_upper,
for an interval on a variance or on a ratio of variances. Both accept a
noncentral parameter, as cv_t() does.cv_bonferroni_f() reports the per-comparison rate alpha / C back in
its area_greater column, which is the whole of what the adjustment
does. Its help page separates it from the rank-sum procedure of
dunn_test(): Dunn (1961) is the Bonferroni procedure, Dunn (1964) is
the nonparametric one.ci_games_howell(): simultaneous confidence intervals for all
pairwise comparisons when homogeneity of variance is not assumed. Every
other all-pairs procedure in the package pools the within-group
variances into MS_W, so none of them is robust when that assumption
fails. Games-Howell uses a separate error term and a
Welch-Satterthwaite degrees of freedom for each pair, then takes its
critical value from the studentized range, following Maxwell, Delaney,
and Kelley (2027, Chapter 5, Equations 5.13 and 5.14). It is the
heterogeneity-robust counterpart of ci_tukey_kramer() and handles
unequal n as a matter of course. With two groups it is exactly Welch's
t test, which the tests assert against t.test(var.equal = FALSE).
Implements Games and Howell (1976).dunn_test(): Dunn's rank-sum test of all pairwise differences,
the follow-up to a significant Kruskal-Wallis test. It ranks all
observations together, as the omnibus test does, and uses the variance
of the ranks implied by the Kruskal-Wallis null (with the tie
correction), so it stays coherent with the omnibus result in a way that
running a Mann-Whitney test on each pair does not. The method
argument passes the pairwise p-values to p.adjust(). Implements
Dunn (1964). Note this is the nonparametric Dunn procedure, not the
Bonferroni procedure of Dunn (1961) that Maxwell, Delaney, and Kelley
(2027) call Dunn's procedure; the help page disambiguates the two.randomization_test() and randomization_test_paired():
randomization (permutation) tests for two independent groups and for
paired observations. The two-group test refers the observed statistic
to its distribution over reassignments of the observed scores to the
groups, so the p-value needs no assumption about the population's
shape, and the result travels with the effect sizes the test only
screens for: the mean difference with a randomization-based interval,
the standardized mean difference with a noncentral t interval, the
common language effect size, and Cliff's delta. The paired test
treats the within-pair sign of each difference as the randomization
mechanism, enumerating all 2^n sign patterns exactly for small
samples and sampling them otherwise. Implements the logic of Fisher
(1935) as developed by Edgington and Onghena (2007).
plot_randomization_test() displays the randomization distribution
the p-value is read from, with the observed statistic marked.alpha_level everywhere. Twenty-two
functions took a bare alpha while twenty-eight already took
alpha_level, and the bare name was carrying three incompatible
meanings inside one package: the Type I error rate in the cv_*,
equivalence, TOST, sequential, and Fisher-exact families; coefficient
alpha in var_alpha(); and ggplot2 transparency in
plot_trajectories(). A reader who learned one meaning would misread
the others. The Type I error uses are now alpha_level throughout, so
fifty functions share one unambiguous name, and the argument that
echoes it in a result table is named to match. The other two keep
alpha, where it is unmistakable: var_alpha(alpha = ) is the
coefficient the function is named for and parallels
var_omega(omega = ), and transparency is the universal ggplot2
convention in a plotting call where no error rate appears. Since the
package is unreleased, the old name fails loudly rather than being
aliased. alpha_lower and alpha_upper, which name the two tails of
an asymmetric critical value, are unchanged.cv_smm() and cv_dunnett() are now deterministic. Both formerly
obtained their multivariate-t quantile from mvtnorm::qmvt(), a
Monte Carlo integrator whose sampling error reached a few hundredths
at alpha_level = .01, enough to move the second decimal of a tabled
critical value. They now evaluate the quantile by exact numerical
quadrature and root finding: the studentized maximum modulus
factorizes into a single one-dimensional integral over the shared
scale (the m statistics are independent given it), and the
balanced-design Dunnett statistic, with its constant correlation
1/2, factorizes through a one-factor representation into two nested
one-dimensional integrals. The returned values are reproducible to the
solver tolerance, reproduce the published Dunnett and studentized
maximum modulus tables more closely than the Monte Carlo path did, and
no longer depend on mvtnorm. Both functions now accept df = Inf
(the known-variance normal limit).seed argument of cv_smm() and cv_dunnett() is removed: the
computation is no longer random, so there is nothing to seed. Call
sites that passed seed should drop it.ci_dunnett() adjusted p-values are now exact. They were
computed from mvtnorm::pmvt(), a Monte Carlo integrator (and, when
\pkg{mvtnorm} was absent, from a conservative Sidak-Bonferroni
fallback). They now use the same deterministic one-factor integral as
the critical value, so the reported p_adjusted is reproducible and
no longer depends on \pkg{mvtnorm}. The shared numeric engine lives in
R/dunnett_internals.R.cv_* family reproducing the Appendix critical-value tables
and validates them by simulation.qbryant_paulson() / pbryant_paulson() / cv_bryant_paulson()
are now accurate at small error df. They obtained the studentized-range
part of the distribution from stats::ptukey(), whose algorithm loses
accuracy at small df (at nu = 3, k = 20 by ~3e-4 in probability, enough
to move the critical value by ~0.2, and more at nu = 2). For nu < 7 the
studentized-range distribution is now evaluated directly, without
ptukey, by integrating the probability integral of the range against
the chi squared error density. The range CDF is splined and cached per
group count; the knots are placed at a fixed spacing of about 0.003 in the
range argument, fine enough that the monotone (Fritsch and Carlson)
interpolation error stays below 1e-9 and the returned critical value
matches the direct integral to about seven figures, while the small-df
path still costs a fraction of a second per group count. The functions
reproduce Bryant and Paulson's (1976) Table 1 exactly, to the two decimal
places tabled, over the whole of its range: both tail areas, all three
covariate counts, every tabled group count, and every tabled error degrees
of freedom from nu = 2 to nu = 120, which is 1188 critical values in all.
Two entries, q_.01;2,8,3 = 23.165013 and q_.01;2,20,4 = 19.745008, sit
about 1e-5 above the point where the second decimal turns over, so they
round to 23.17 and 19.75 while the 1976 table rounds them down; both were
confirmed to fourteen significant figures by two independent high-order
quadrature engines that share no code with the package. A large-scale
simulation of the statistic confirms the computed values independently.
Values for nu >= 7 are unchanged.adjusted_means() extracts the full adjusted (least-squares)
means table. From an lm/aov factorial or ANCOVA fit it returns
every cell's covariate-adjusted mean with SE and CI, or marginal
means over chosen factors via by, with weights = "equal" (the
population marginal means of Searle, Speed, and Milliken, 1980) or
"proportional" (observed frequencies). The reference grid follows
the model's own terms, so transformed covariates and interactions
are handled exactly, and nonestimable cells in rank-deficient
designs error plainly by name. Validated against emmeans 2.0.3 at
1e-10 across fourteen fits (unbalanced factorials, multiple and
transformed covariates, character and ordered predictors, weighted
fits, an empty-cell design) for both weightings, with the
comparisons pinned in tests and re-run live in
tools/oracle_checks.R; also pinned to ancova()'s adjusted means
and to contrast_adjusted() at 1e-10.
loa() reports the Carkeet (2015) pair intervals by default.
The exact CIs for the limits of agreement considered as a pair, the
construction Carkeet recommends for most uses, replace the per-limit
intervals as the default; method = "individual" keeps the
one-sided tolerance-factor form. Both constructions are anchored in
tests to the paper's printed coefficients and worked example, and a
misattributed comment (the Bland and Altman 1999 approximate SE,
previously credited to Carkeet) is corrected.
Sixteen packages leave Suggests. DMAR's test oracles (MBESS, emmeans, semTools, metafor, mirt, sirt, multcomp, performance, r2mlm, irr, irrCAC, psych, BayesFactor, gsl) are no longer dependencies: every live comparison was replaced by its pinned value, with provenance comments naming package, version, and date, and the live comparisons themselves moved to tools/oracle_checks.R, which re-runs all of them against the installed oracles at release time. BiasedUrn is replaced by a self-contained Fisher noncentral hypergeometric density (agreement 1.2e-13), kableExtra by knitr-only HTML and LaTeX table construction, and AMCP by shipping the depression_bdi data directly. Suggests drops from thirty packages to fourteen, none of them needing compiled system libraries beyond what the remaining features genuinely use.
The Bayes factor functions take informed priors, stated either
way. The prior on the standardized effect can now be an informed
Cauchy, moving prior_location off zero (Gronau, Ly, & Wagenmakers,
2020), or a normal with prior_mean and prior_sd for beliefs
stated as moments; a Cauchy has no mean and no variance, so moment
beliefs could not previously be expressed at all. The two families
are exclusive, and the help pages explain what the Cauchy scale
fixes (the quartiles: half the prior mass within one scale of the
location) and the exact identity linking the families: a Cauchy is a
normal prior whose variance is itself uncertain, and the test suite
uses that identity to cross-validate the two code paths against each
other with no external oracle. Every result now carries the full
posterior of the effect in a "posterior" attribute (a data frame
of delta and density), so any posterior probability can be
computed, not only the reported ones; the prior is echoed in
prior_location and prior_scale rows with the family recorded as
an attribute. Default calls reproduce the previous JZS results
exactly.
The Bayes factor functions accept summary statistics. The JZS
Bayes factor depends on the data only through the t statistic and
the sample sizes, so bayes_one_sample_t(), bayes_paired_t(), and
bayes_independent_t() now take the summary statistics a paper
reports (mean, sd, and n; mean_diff, sd_diff, and n; or
mean_1, sd_1, n_1, mean_2, sd_2, n_2) as readily as raw
data, following the same exactly-one-path rule as the rest of the
package. The summary form is exact, not an approximation, and the
help pages show how a standardized effect size enters (a d of 0.5
is mean_1 = 0.5, mean_2 = 0 with unit standard deviations). Tests
pin the raw and summary forms to each other at machine precision.
The directional test family speaks snake_case. The canonical
alternative values of ci_dunnett(), power_fisher_exact(),
randomization_test(), randomization_test_paired(),
summary_t_test(), and welch_t() are now "two_sided", "less",
and "greater", with the base-R spelling "two.sided" accepted as
an alias so existing calls keep working; anything stored on a
returned object carries the underscore form. The cv_* critical
value family keeps its deliberately wider synonym vocabulary with
"not_equal" canonical, unchanged.
var_omega_squared() returned a variance that did not shrink with
the sample size. The worker multiplied the wrong combination of
terms, and the consequence was not a small bias: the returned variance
converged to a positive constant instead of to zero, so its error grew
with N. Against a 200,000-replication Monte Carlo of the sampling
distribution it was too large by a factor of 1.4 at the smallest cell
tested and by 115 at the largest, and N times the variance climbed from
0.8 at N = 30 to 51 at N = 5100 where the true value settles near 0.31.
Any standard error or Wald interval built on it was badly inflated, and
worse in larger samples. The variance is now the delta-method transfer
of Fleishman's (1980, Eq. 22) exact variance of the unbiased estimator
of the signal-to-noise ratio, carried to the omega squared scale by his
Eq. 8 with Jacobian (1 - omega^2)^2, which tracks the Monte Carlo to
within a few percent across every cell tested and to 1.00 by N = 300.
Two regression tests now guard it: agreement with a fixed-seed Monte
Carlo, and the consistency check that N times the variance stays
bounded, which the old formula would have failed. The help page also no
longer attributes an omega squared variance to Fleishman, who gives one
only for the signal-to-noise ratio and says explicitly that the
correlation ratio has an interval and a median but not a variance.
Generalized eta squared now classifies interactions the way its
sources do. Olejnik and Algina (2003, Eq. 5) and Bakeman (2005)
define an effect as a measured source of variance when any factor
in its term is measured, so a measured-by-manipulated interaction
belongs in the denominator automatically. eta_squared_generalized()
and ci_eta_squared_generalized() had instead included only the
effects explicitly listed in observed, a rule the help page
misattributed to Bakeman (2005), who recommends the opposite.
Listing a factor in observed now also places every interaction
containing it in the denominator, so observed = "c" reproduces the
papers' worked examples as written; an explicit interaction label is
still honored as given. Values change only for model-interface calls
on designs with measured-by-manipulated interactions, where the
corrected values are smaller. Tests now anchor the Keppel and Kirk
worked values printed in Olejnik and Algina (2003).
signal_to_noise_R2()'s nonlinear estimator now follows Muirhead's
(1985) Equation 10 exactly: the c/Y correction is added to the
untruncated linear estimate before truncation at zero, rather than
after the linear estimate was itself truncated. The two orders agree
whenever the sample R-squared is not tiny; for sample R-squared below
p/(N-1) the reported nonlinear value is now smaller, as the paper
defines it.
In ss_aipe_pcm()'s assurance search, the critical t now tracks
each candidate sample size's degrees of freedom instead of reusing
the value frozen from the expected-width search. The frozen t was
very slightly conservative; the correction moves nothing by as much
as one subject, and every documented example returns the same
sample size as before.
The ci_c_ancova() example carried the wrong sum of squares for its
covariate. The Maxwell, Delaney, and Kelley Chapter 9 depression
example needs the within-groups sum of squares of the pretest, which
the chapter data give as 752.5 (and which the sibling
ci_sc_ancova() example already used); the example instead passed
313.37, the within-groups sum of squares the covariate explains in
the posttest, a different row of the same ANCOVA decomposition. The
interval moves only in the third decimal, but the two pages now agree
with each other and with the chapter data, and a test recomputes
752.5 from the data (via the AMCP package) so the constant cannot
drift again.
ci_sc() now works when the noncentrality parameter is supplied
directly. The ncp branch set the noncentrality but never the
standardized contrast, so the documented call errored on the closing
table construction (three terms against two values). The branch now
inverts the same relationship the means and psi branches use
(the noncentrality is the standardized contrast divided by
sqrt(sum(c_weights^2 / n))), so the three parameterizations of one
effect return identical tables; that equivalence and the ncp path's
coverage are now tested.
ci_c() and ci_sc() now recycle a scalar n by the number of
contrast weights. The scalar was recycled by length(means), which
is zero on the psi-only path (and, for ci_sc(), the ncp-only
path), so those documented calls stopped on the n / c_weights
length check. No result changes on the means path, where the two
lengths agree; a means vector whose length differs from
c_weights is now rejected with a clear error instead of being
silently recycled.
ss_aipe_cv() now accepts the documented mu / sigma
parameterization, deriving c_of_v = sigma / mu; previously neither
formal was read and the call stopped on "argument is of length
zero". Supplying c_of_v together with mu or sigma is rejected
as conflicting. The mu / sigma path returns exactly the sample
size of the equivalent c_of_v call (for mu = 10, sigma = 1,
width = .1, conf_level = .99, both give N = 20). The example
prose on the help page said the population coefficient of variation
was .25 while the calls passed .1 (which plans N = 20, where .25
plans N = 100); the prose now matches the calls, and a mu /
sigma example was added.
cfa_1() with missing = "ml" now estimates the mean structure
with free item intercepts. The bare lavaan::lavaan() interface it
calls turns the mean structure on for FIML but leaves int.ov.free
at FALSE, so every item intercept was fixed to zero and the
loadings absorbed the item means; on any data not centered at zero
the FIML estimates were wrong (the function's own example, simulated
with mean zero, could not show it). The fit now matches
lavaan::cfa(..., missing = "ml") exactly. Listwise fits are
unaffected.
ss_power_split_plot_anova() now uses the correct between-subjects
noncentrality. The between-subjects test used the per-group sample
size n where its noncentrality should use the total sample size
N = n a, so the noncentrality was too small by the factor a (the
number of between-subjects groups), halved for two groups. This
understated between-subjects power and, planning in reverse,
overstated the necessary per-group sample size; the within-subjects
and interaction tests were unaffected. The corrected between-subjects
test now agrees with a split-plot aov() simulation and, for two
groups, reproduces the two-level treatment test of
ss_power_mixed_effects() exactly (the between-subjects
F(1, .) is the two-level treatment t squared), the identity that
ties the two planners together. Both a value anchor and the
cross-planner identity are now tested.
cv_smm() and cv_dunnett() now work at a large error df. Both
integrated the chi squared error variate over (0, Inf), and once df
is large that density is a narrow bump far from the origin, so the
adaptive rule sampled its way past the mass and returned zero: at
df = 200 the integral evaluated to exactly 0 for every candidate
quantile, which left the root finder with no bracket and the functions
stopped with "f() values at end points not of opposite sign." The
limits are now the extreme quantiles of that variate, which puts the
quadrature on the mass at any df. Both functions gained a large-df
test; the values they already returned are unchanged.
cv_f() no longer returns NaN when the numerator df is infinite.
Supplying ncp = 0 explicitly sends qf() and pf() down their
noncentral algorithm, which does not admit an infinite numerator df,
so the infinite-numerator column of Appendix Table A.2 came back
NaN. Both functions now pass ncp only when it is nonzero, so the
central algorithm serves the central case. cv_chisq() takes the same
guard.
ancova() now reports the covariate-adjusted omnibus F for the
treatment effect. It previously entered the treatment before the
covariate and read the sequential (Type I) sum of squares, which is
the unadjusted treatment F; the covariate is now entered first so
the treatment's sequential sum of squares is its adjusted sum of
squares.
ss_aipe_sm() corrects the noncentrality used in the assurance
branch (it was the reciprocal sm / sqrt(n) instead of
sm * sqrt(n)), which previously left the assurance target
unreachable and pinned the search to a bracket endpoint.
ci_c() now honors the documented df_error argument; supplying it
previously left the error degrees of freedom undefined and errored.
tidy() and glance() on ss_power_r() and ss_power_smd() now
return the user-supplied sample size on the realized-power path
(previously NA).
ss_aipe_reliability() now computes confidence intervals when the
default interval = TRUE is used (a string-only equality check had
skipped the interval for the logical default).
ss_aipe_reliability(type = "Factor Analytic") works again. The path
read cfa_1()'s legacy list layout ($factor_loadings,
$parameter_cov), which the reworked cfa_1() no longer returns, so it
errored; coefficient omega and its delta method interval now route through
the maintained reliability internals (.omega_fit_cfa(),
.ci_omega_delta()).
mlmr() and mlmr_mv() are the new lm-like front end to
full information maximum likelihood (FIML) regression. The
univariate mlmr() mirrors the lm() API (formula interface,
coef / vcov / confint / summary / anova / predict /
update S3 methods, profile / Wald / bootstrap CIs); the
multivariate sibling mlmr_mv() takes cbind(y1, y2) ~ ... and
models the joint distribution of correlated outcomes, with the
residual covariance among outcomes estimated as part of the fit.
See vignette("mlmr", package = "DMAR") for when the
FIML route actually buys you something over lm() + listwise
deletion.mlmr() and
mlmr_mv() gained an auxiliary argument that brings variables
related to the missingness or to the incomplete outcome into the
model as saturated correlates (Graham, 2003): each auxiliary is
correlated with the outcome residual, every predictor, and each
other auxiliary, but never enters as a predictor, so the focal
regression coefficients keep their meaning. This is the inclusive
analysis strategy (Collins, Schafer, & Kam, 2001); it leaves
complete-data estimates unchanged and, under MAR, recovers
information that listwise deletion discards. The differential
missingness scenario in vignette("mlmr", package = "DMAR") works
through when it helps and when randomization already protects the
estimand.tidy() and glance() integration via the
generics package: a uniform interface to the broom ecosystem
for mlmr, mlmr_mv, cfa_1, the reliability family, the long-
format CI family, the ANOVA effect size CI family, and the power
planner family. purrr::map_dfr(fits, generics::tidy) now works
across DMAR outputs.convert_R2_*
family cut iterative AIPE / sensitivity planning calls by
roughly 3x; representative ss_aipe_R2() calls go from ~2.7s
to ~0.9s.New data set test_market: a small balanced ANCOVA example (sales
by promotion type, adjusted for a baseline covariate) used to
illustrate the Bryant--Paulson simultaneous intervals in
vignette("bryant_paulson_ancova", package = "DMAR") and
ci_c_ancova_bp().
New data set drinks_trial: nine-month follow-up
drinks-per-week outcomes for the N = 88 homeless
alcohol-dependent participants in Smith, Meyers, and Delaney's
(1998) randomized trial of the Community Reinforcement Approach
at the Salvation Army Adult Rehabilitation Center in
Albuquerque, New Mexico. Two consecutive cohorts: Cohort 1
compared Standard, CRA, and CRA + Disulfiram with cell sizes
17, 15, 19; Cohort 2 dropped the disulfiram cell after Cohort 1
results and compared Standard against CRA with cell sizes 20,
17. The outcome ships in raw form (heavily right-skewed, range
0 to 624.6 drinks per week) and on the log10 scale used in the
published analyses to recover approximate normality. Reproduced
in Maxwell, Delaney, and Kelley (2027, Designing Experiments
and Analyzing Data: A Model Comparison Perspective, 4th ed.,
Routledge), Chapter 3, Section 3.10.4.
New data set bessel_errors: Friedrich Wilhelm Bessel's (1818)
9-bin grouped frequency distribution of the absolute errors of
300 stellar position observations made by British Astronomer
Royal James Bradley at the Greenwich Observatory between 1750
and 1762. Both the observed and expected (normal-model)
frequency columns sum to 300, matching Maxwell, Delaney, and
Kelley (2027, 4th ed.), Table 1.4. Documented as a worked
example for approximating moments from grouped frequency data
(frequency-weighted means and variances using bin midpoints)
and for plotting empirical-versus-theoretical frequency
comparisons. Ships in the original grouped form Bessel
reported; individual error values are not extant.
New data set prime_time_achievement (also accessible via the
short alias Prime_Time): the full Indiana Prime
Time third grade achievement evaluation file (Lapsley, Daytner,
Kelley, and Maxwell, 2002, ERIC ED466679), built from the
original Indiana Department of Education SPSS system file.
10,927 students nested in 586 classrooms in 163 schools in 61
school corporations (district x region combinations) in 9
educational service regions on 113 variables. Includes ISTEP+
NCE composites (the criterion in the published HLM analyses),
Gates-MacGinitie and AANCE test scores, NPA cognitive ability
scores, classroom enrollment, pupil to teacher ratio, Prime Time
aide indicator and status, school and corporation demographics
and finance, and three derived unique cluster identifiers
(corp_id, school_id, class_id) that respect the nesting
irregularities in the source file (one corporation ID spans two
regions). Original Indiana DOE variable names and the original
six-category race coding are preserved verbatim; the SPSS
variable labels are retained as a label attribute on every
column. The SPSS Select Cases artifact FILTER_$ and the 888
"not applicable" codes have been dropped or recoded to NA.
Includes documented examples that show level-1, level-2, and
level-3 lmer fits mapped onto the multilevel framework used
in Lapsley et al. (2002) and in Finch, Bolin, and Kelley
(2019, Multilevel Modeling Using R, 2nd ed., CRC Press,
chapters 3, 4, 6, 9, 10).
New data set holzinger_swineford (also accessible via the
short alias HS_Data): the complete Holzinger and
Swineford (1939) factor analysis data, 301 pupils on 26 ability
tests from the Pasteur (n = 156) and Grant-White (n = 145)
elementary schools in Chicago. Variable names follow the MBESS
convention (e.g., t1_visual_perception through t26_flags) so
scripts written against MBESS::HS port over with a single rename
of the object. The values are the corrected version of the data,
identical to MBESS::HS as of MBESS 4.9.3 and to
psychTools::holzinger.raw. The documentation discusses the bi-
factor study design, the five ability blocks (spatial, verbal,
mental speed, memory, reasoning), the Joreskog (1969) 9 test
subset, and the silent post-4.6.0 MBESS correction that is the
source of values still found in sem::HS.data and
OpenMx::HS.ability.data.
New data set pygmalion: the teacher-expectancy data from
Rosenthal and Jacobson's (1968) Pygmalion in the Classroom,
310 elementary school pupils in grades 1 to 6, of whom 64 were
randomly designated to their teachers as likely intellectual
"bloomers" and 246 served as controls, with pretest and follow-up
IQ. This is the classic benchmark for analysis of covariance with
heterogeneity of regression, and the running example for that
topic in Maxwell, Delaney, and Kelley (Designing Experiments and
Analyzing Data, Chapter 9). The within-group slopes (0.778 for
controls, 0.969 for bloomers), the pooled residual variance
(175.3251), and the covariate variance (348.91) reproduce the
worked example for the variance of the estimated treatment effect
at selected covariate values (cf. MBESS::var.ete). The same
numbers ship with the book's data companion AMCP as
chapter_9_exercise_15; here the experimental condition is a
labeled factor (Control, Bloomer) and the columns use DMAR's
descriptive names, with no measured value altered. Pairs with
ancova() and is documented in
vignette("pygmalion", package = "DMAR").
The largest visible change is a uniform snake_case interface for
both function names and argument names.
Argument names that were dot.case in MBESS are snake_case in
DMAR. The most common renames a user will hit when porting a
script:
| MBESS | DMAR |
|----------------------|-------------------|
| conf.level | conf_level |
| Random.Predictors | random_predictors |
| Specified.N | specified_N |
| alpha.lower | alpha_lower |
| alpha.upper | alpha_upper |
| degrees.of.freedom | Use df or the explicit df_1 / df_2 |
| Group.1, Mean.1 | group_1, mean_1 |
The rule of thumb: change every . between words to _, and
lowercase the non-statistical prefix words. Capitals are preserved
when the capital is statistically meaningful: R2 (squared
multiple correlation), N (sample size), S (a covariance
matrix), Lambda (a factor-loadings matrix), F_value (an
F-statistic).
aipe_smd() is renamed to ss_aipe_smd() to align it with the
rest of the ss_aipe_* family.
Every estimation, inference, and planning function returns a
data.frame with term and value columns. (Plotting functions
return a ggplot object, and a few utilities return their natural
type.)
Return objects in MBESS were sometimes named lists, sometimes
data.frames, and sometimes vectors; the unified tidy return makes the
package compose cleanly with the rest of the modern R ecosystem.
Confidence intervals and sample size annotations travel with every
effect size plot. plot_smd(), plot_ci(), and plot_R2()
default to show_ci = TRUE and show_n = TRUE.
There are no deprecation shims. Old MBESS argument names will throw "unused argument" errors at the call site, which is intentional; the fix is mechanical, and a silent-forwarding shim would hide it.
DMAR adds 40+ functions across families that were absent or under-developed in MBESS. New families include:
Mediation, both halves: mediate() analyzes the simple mediation
model (optional covariates) with percentile bootstrap, BCa, Monte
Carlo, and Sobel intervals for the indirect effect, and
ss_power_indirect_effect() plans the study by joint significance
with exact noncentral t component powers (Sobel power for
comparison), validated against raw-data simulation.
mediation_mbco() extends the inference side to arbitrary
mediation structures through the likelihood ratio model comparison
framework (see its own section above).
Measurement invariance: measurement_invariance() fits the
configural, metric, scalar, and strict multi-group ladder for any
measurement model, with a thresholds rung for ordered indicators,
and reports the full comparison table (fit, the likelihood ratio
test per step, and delta CFI / delta RMSEA), pinned to direct lavaan
fits in the tests. See the entry at the top of this file for the
arguments and the returned "fits" attribute.
Construct validity: htmt() (the Henseler heterotrait-monotrait
ratio with an optional bootstrap upper bound) and
average_variance_extracted() (Fornell-Larcker AVE from a lavaan fit
or standardized loadings). Composite reliability needs no new
function; it is coefficient omega (reliability_omega()).
Clinical and behavioral endpoints: responder_analysis() (per-group
responder proportions with Wilson intervals, the Newcombe risk
difference interval, the number needed to treat, an omnibus chi
square, and a threshold sweep) and ci_proportion(), the package's
Wilson score interval for a single proportion.
A meta-analysis family. meta_es() pools any effect sizes given
sampling variances (REML between-study variance by default, with
Paule-Mandel, DerSimonian-Laird, and fixed effect options, the
Hartung-Knapp adjustment on by default, Q-profile confidence
intervals for tau-squared mapped to I-squared, and a prediction
interval always reported); meta_smd() (exact Hedges correction by
default) and meta_r() (Fisher's Z pooling, with optional per-study
attenuation corrections) are the metric front ends. combine_p()
provides the four classical combined significance tests and
meta_contrast() the Rosenthal and Rubin contrast among effect
sizes; plot_forest() draws the studies, the pool, and the
prediction interval. Validated against metafor, and against
Raudenbush (1984) line by line in the teacher expectancy vignette;
the 19 effect sizes of that synthesis ship as teacher_expectancy.
Factorial ANCOVA power: ss_power_factorial_ancova() extends the
factorial ANOVA planner to baseline covariates (error variance
scaled by 1 - R2, one error df per covariate), with the complete
2 x 4 x 3 worked example, planning, simulation, Type III analysis,
interaction plots, and focused complex comparisons, in the
ancova_2x4x3_power vignette.
A new vignette, "Power and Precision for the One-Way ANOVA: A
Model Comparison Perspective," works the one-way design from the
comparison of a full model and a restricted model: the omnibus
power and sample size through ss_power_one_way_anova(), a planned
contrast through ss_power_contrast(), effect size confidence
intervals through ci_omega_squared(), ci_pvaf(), ci_snr(),
and ci_srsnr(), the model comparison made literal with mlmr(),
the Type S and Type M consequences of the design through
design_consequences(), and accuracy in parameter estimation for a
contrast through ss_aipe_c(), following Maxwell, Delaney, and
Kelley (2027).
A Bayesian t family with the probability statement front and
center: bayes_one_sample_t(), bayes_paired_t(), and
bayes_independent_t() report the JZS posterior of the
standardized effect (median, mean, credible interval, and
P(delta > 0 | data)) with the default Bayes factor as a secondary
row, computed by exact quadrature and validated against the
BayesFactor package.
correction_for_attenuation() (renamed from its working name
correct_attenuation()) documents and demonstrates the latent
variable route it approximates: when item-level data exist, prefer
the two-factor model's latent correlation to the plug-in formula.
Moments of the noncentral distributions: moments_nc_t(),
moments_nc_F(), and moments_nc_chisq() return the mean, variance,
standard deviation, skewness, and excess kurtosis of the noncentral
t, F, and chi square distributions, with NA for moments whose
degrees of freedom conditions fail. The noncentral t mean is the
quantity behind the upward bias of the standardized mean difference.
A random-coefficients polynomial growth simulator:
simulate_longitudinal_polynomial() generates longitudinal data of
any polynomial order (order 0 is a flat line) for one or several
groups, ties the level-one error to a target measurement reliability
(reported per occasion), allows assessment-time jitter around the
nominal schedule, and supports structured level-one error covariance
(AR(1), compound symmetry, Toeplitz, heteroscedastic, or a full
matrix). It is the Monte Carlo companion to ss_power_pcm().
ss_power_pcm() now plans power for any polynomial change
coefficient (intercept, linear, quadratic, cubic, ...) through a
trend argument, carrying the general Raudenbush and Liu (2001)
sampling variance; the linear default reproduces the National Youth
Survey benchmark.
orthogonal_polynomial() returns orthogonal polynomial trend
contrast weights stored levels-by-trends (ready for contrasts() and
lm()) and printed in the Maxwell, Delaney, and Kelley Table A.10
layout with a trailing sum-of-squared-weights column.
unbiased_R2() gives the Olkin and Pratt (1958) exactly unbiased
estimator of the population squared multiple correlation alongside
the Ezekiel (1930) adjusted estimator (the adj.r.squared of
summary.lm).
Design consequences: design_consequences() reports what a chosen
design delivers under both of the package's lenses, the significance
lens (power, the Type S wrong-sign error rate, and the Type M
exaggeration ratio, after Gelman and Carlin, 2014, computed exactly
rather than by their simulation) and the precision lens (the expected,
median, and standard deviation of the realized confidence interval
width, and the probability the realized interval beats a target width,
the closed-form versions of the ss_aipe_*_sensitivity() Monte Carlo
terms). Accepts a standard error directly or derives it from sd and
per-group n. Every ss_power_* and ss_aipe_* help page points to
it.
The Spearman correction for attenuation: correction_for_attenuation()
disattenuates a correlation for measurement error in either or both
variables, with a confidence interval when the sample size is
supplied, using reliabilities from the reliability_* family.
New parameterization conversions: convert_d_r() / convert_r_d()
(standardized mean difference and point-biserial correlation, with an
unequal-group factor) and convert_d_or() / convert_or_d() (the
Hasselblad and Hedges logistic link to the odds ratio).
convert_F_chisq() and convert_chisq_F() move a test statistic
between the F and chi square metrics, returning the converted
statistic itself (a value, not a p-value). Two conversions are
offered through one argument, df_denominator. The default,
df_denominator = Inf, treats the F's error variance as known and
returns the standard scaling, df_numerator * F_value (and back,
chi_square / df); an F is a chi square whose error variance is
estimated rather than known, and infinite denominator degrees of
freedom is the case where it is known. A finite df_denominator
instead returns the chi square value with the same upper-tail
probability (the same p-value) as the F, which is exact at any
denominator degrees of freedom; the two conversions agree as
df_denominator grows, so the scaling default is the large-sample
limit of the same family. The finite case is computed from ordinary
upper-tail p-values (no logarithms): the upper tail is used because
the lower-tail probability rounds to 1 in double precision by about
F = 500 at small denominator degrees of freedom, which would send the
result to infinity, whereas the upper-tail computation stays accurate
past F = 1e20. Each help page states the exact computation. The map
preserves the p-value but does not transport a noncentrality
parameter, so noncentral work belongs in ci_nc_F() and
ci_nc_chisq().
Display helpers extending the package's p-value convention to
objects DMAR does not produce: format_p(), print_anova(), and
print_summary().
DMAR no longer requires the GSL system library: the Gauss hypergeometric function behind the exact squared multiple correlation moments is computed in base R (and remains accurate as the squared multiple correlation approaches 1, where the previous route lost precision). Installation now has no system prerequisites.
Plot functions in the plot_* family: plot_smd(), plot_ci(),
plot_R2(), plot_trajectories(), plot_trajectories_fitted().
All built on ggplot2 and colored by default with a neutral,
colorblind-safe palette; each accepts a palette argument
("okabe_ito" or "tableau") and, where applicable, a colors
override for full manual control.
Plot colors come from base R. The plot_* family colors itself
with base R's Okabe-Ito colorblind-safe palette by default, with base
R's Tableau 10 available through each plot's palette argument. DMAR
defines no palette of its own and adds no color dependency; a user who
wants other colors adds an ordinary ggplot2 scale to the plot.
Ordinal and non-parametric effect sizes: cliff_delta(),
vargha_delaney_A(), probability_of_superiority_paired(),
cles(), proportion_of_superiority() (sometimes called Cohen's
U3).
Agreement and reliability extensions: bland_altman_loa(),
lin_ccc(), cohen_kappa(), fleiss_kappa(), gwet_ac(),
krippendorff_alpha(), icc(), icc_lmer(),
reliability_omega_categorical(), reliability_omega_h(),
reliability_H(), reliability_kr20(), plus the var_*
family of asymptotic variance utilities.
Within-subjects, mixed, and multivariate ANOVA: anova_within(),
anova_within_two_way(), mixed_anova(), manova_split_plot(),
mauchly_test(), epsilon_corrections(), pairwise_within(),
simple_effects_AB().
Sample size planning has been broadened, particularly the
ss_power_* family, which now covers between-subjects,
within-subjects, mixed, and multi-level designs across the
chapters of Maxwell, Delaney, and Kelley (2027).
Parameterization conversions in the convert_* family
(convert_R2_f, convert_f_R2, convert_lambda_R2,
convert_R2_lambda, convert_r_Z, convert_Z_r,
convert_delta_lambda, convert_lambda_delta,
convert_cor_cov). Every conversion is exact-invertible and the
inverse direction is shipped as a sibling function where it makes
sense.
A worked simulation study of the AIPE family reports expected and realized CI widths, realized coverage, and assurance-achievement rates across 10,000 Monte Carlo replications per cell. It establishes how the methods perform rather than how they are used, so it is maintained alongside the package rather than shipped in it.
ss_aipe_reliability() no longer embeds a vendored copy of MBESS
3.2.0's ci.reliability() for the Monte Carlo assurance search. The
interval at each candidate sample size now comes from DMAR's own
reliability internals (the single-factor delta method interval for
the factor analytic type; the van Zyl, Neudecker, and Nel closed
form for the normal theory type). A before-and-after grid covering
every model and interval type combination confirmed the computed
interval widths, and therefore every planned sample size, are
unchanged.
icc_lmer() now reads its grouping factor from lme4's grouping list,
so interaction groupings such as (1 | school:teacher) work; and
mlmr_mv gained the tidy() / glance() methods its documentation
promised (one row per coefficient per outcome, with a response
column).
The test suite runs with warnPartialMatchArgs,
warnPartialMatchDollar, and warnPartialMatchAttr enabled, so any
internal reliance on partial matching fails loudly, and the display
layer (the dmar_tbl print methods, format_p(), print_anova(),
print_summary()) is covered by testthat snapshots.
The seed argument defaults to NULL everywhere it is exposed,
meaning the function uses the caller's current RNG state and does not
seed; a user who wants reproducibility passes an explicit integer, and
a supplied seed restores the caller's RNG state on exit. The value
113 is used only in @examples and tests, never baked into a
function default.
The cluster-randomized design helpers under ss_aipe_crd* were
consolidated into a single set of shared internals
(R/ss_aipe_crd_internals.R); both the difference and
effect size families now call the same .find_*_crd_*()
back end.
ss_aipe_pcm() had a typo in the cubic change coefficient
constant (K_3 was coded as 1/1000800, about one tenth of the
correct 1/100800). Because the constant sits in the denominator of
the slope variance, this inflated the cubic variance, and hence the
resolved sample size, roughly tenfold for trend = "cubic". The
constant now matches the closed form
K_p = (p!)^2 / [(2p)! (2p+1)!] of Raudenbush and Liu (2001,
p. 392), and a regression test recovers it directly from an OLS fit.
The linear (1/12) and quadratic (1/720) constants were already
correct.
ss_aipe_pcm() now honors its documented contract that a user may
supply either error_variance or the converted variance
variance_true_minus_estimated_trend. Previously error_variance
was effectively required, and the consistency check between the two
used round(..., 3), which mishandled the small variances typical
of slope-change designs. The cross-check now uses all.equal(),
and supplying neither argument raises an informative error.
ss_aipe_pcm() now scales the within-subject contribution to the
slope variance by frequency^(2p), matching ss_power_pcm() and
Raudenbush and Liu (2001, p. 392). The change coefficient is a
per-unit-time rate, so its sampling variance is
error_variance * frequency^(2p) / (sum of the squared polynomial weights); the factor was previously omitted. Every Kelley and
Rausch (2011) benchmark uses frequency = 1, where the factor is 1,
so all tabled sample sizes are unchanged; the correction only
affects designs with frequency != 1, where it now agrees with the
direct OLS slope variance on the actual time grid.
ss_aipe_pcm_sensitivity() now simulates the same estimand its
planner targets: the between-group difference in mean slopes (the
group-by-time change parameter), using two independent groups, a
pooled standard error, and 2n - 2 degrees of freedom. The previous
simulator built a one-group mean-slope interval, whose width was
systematically 1/sqrt(2) of the planned target at every frequency.
The realized mean CI width now tracks the planned width. The
schema terms mean_slope / median_slope / sd_slope are renamed
to mean_slope_diff / median_slope_diff / sd_slope_diff to
reflect the difference estimand.
Ken Kelley (Department of Information Technology, Analytics, and Operations; Mendoza College of Business; University of Notre Dame) is the package author and maintainer. Bug reports and feature requests are welcomed by email to [email protected]; please put "DMAR" in the subject line.