Guided statistical analysis for research data - test selection, assumption checking, effect sizes, power analysis, Bayesian alternatives, and APA-formatted reporting. Use whenever a user wants to compare groups, test a hypothesis, analyze experimental or survey data, check statistical assumptions, c

Statistical Analysis Overview Conduct hypothesis tests (t tests, ANOVA, chi square), regression, correlation, and Bayesian analyses with systematic assumption checking, effect sizes, and APA style reporting. Define the estimand and sampling units, diagnose model limitations, and report effect estimates with uncertainty. Numerical screens cannot certify assumptions or scientific validity. When to Use This Skill Use this skill when: Conducting statistical hypothesis tests (t tests, ANOVA, chi square, non parametric) Performing regression or correlation analyses Running Bayesian statistical analyses Checking statistical assumptions and diagnostics Calculating effect sizes and conducting power analyses Reporting statistical results in APA format Analyzing experimental or observational data for research Installation Targets Pingouin 0.7.0, SciPy 1.18.1, statsmodels 0.15.0, PyMC 6.3.2 and ArviZ 1.3.0 (reviewed 2026 10 01). Create a separate environment; do not add mutually incompatible scientific stacks to a shared project environment. On Windows use .venv statistics/Scripts/python.exe . Lock the resolved environment for reproduction. Pingouin 0.7 fixes erroneous mixed/repeated measures corrections, categorical ANOVA, and other results; do not reproduce an old analysis by merely changing its reported version. See the [official changelog](https://pingouin stats.org/changelog.html). Pingouin 0.6+ names are p val , cohen d , CI95 , p unc . ttest reports absolute Cohen's d; calculate a signed estimate explicitly. Its power column is observed power and should not be used to interpret a null result. correction='auto' chooses Welch by unequal sample sizes , not by variance evidence. Set correction=True for prespecified Welch inference. PyMC 6 returns an xarray DataTree . ArviZ 1 uses ci prob in summary / plot dist and prob in hdi ; specify 95% intervals explicitly. See [Bayesian guidance](references/bayesian statistics.md). Pingouin omits BF10 for one sided t tests; posterior directional probabilities are not Bayes factors. A two sided BF10 uses its own model/prior and is not the Bayesian counterpart of a Welch unequal variance model. For model specific APIs (OLS, GLM, ARIMA), see the statsmodels skill. For PyMC workflows, see the pymc skill. Analysis Workflow Use the following workflow, documenting prespecified decisions and exploratory departures. 1. Frame the question before touching the data. State the hypothesis, the outcome and predictor variables, and the design (independent vs. paired, number of groups). Specify the target effect, sampling unit, dependence, contrasts and multiplicity family before outcome driven selection. Label adaptive exploration explicitly. 2. Inspect the data. Per group: n, mean, SD, median, missing values. Plot the raw data (histograms or box plots) before any test. Unequal group sizes, missingness, floor/ceiling effects, and outliers all change what test is appropriate — surface them to the user rather than silently working around them. 3. Select the test using the quick reference below, or references/test selection guide.md for designs beyond the basics (counts, time to event, reliability, factorial). 4. Check assumptions with scripts/assumption checks.py . Interpret plots and screens alongside the design and estimand; do not switch tests automatically at a diagnostic p value threshold. Report justified sensitivity analyses and any departures from the plan. 5. Run the test and always compute the effect size alongside it — a p value assesses incompatibility with the null model; the effect estimate and its uncertainty support judgments about practical importance. 6. Report using the APA templates below, including descriptives, exact statistics, effect sizes with CIs, and the assumption checks performed. If the user only needs one step (e.g., "how many participants do I need?"), jump straight to that section — but still confirm the design assumptions the calculation rests on. Test Selection Guide Quick Reference: Choosing the Right Test Use references/test selection guide.md for comprehensive guidance (counts, survival, reliability, factorial designs). Quick reference: Group comparisons: Independent continuous means usually use prespecified Welch's t test or Welch's ANOVA. Pooled Student/ANOVA inference needs justified equal variance. Paired means use a paired t test on aligned within person differences. Repeated or clustered observations require a model of that dependence. Rank questions: Mann Whitney or Kruskal Wallis can address distribution/rank contrasts; they are not automatic tests of medians. Wilcoxon signed rank requires meaningful, symmetrically distributed paired differences. Friedman addresses blocked/repeated rank contrasts. See the reference for ties, small samples, and exact/permutation choices. Relationships: Pearson measures linear association; Spearman measures monotone rank association. Choose by the scientific question, not a normality pretest. Use a suitable regression likelihood for continuous, binary, count or time to event outcomes. Bayesian alternatives: Specify the likelihood, prior and comparison explicitly. Posterior estimation does not automatically yield a Bayes factor for a point null. Assumption Checking Always check assumptions before interpreting test results , and report the checks — reviewers look for them. Use the bundled scripts/assumption checks.py module. Add the skill’s scripts/ directory to PYTHONPATH (the skill root alone is insufficient): For targeted checks, import individual functions: What to Do When Assumptions Are Violated A non significant Shapiro/Levene result does not verify normality/equal variance. There is no universal n=30 robustness rule; tails, skewness, imbalance, dependence and the target estimator matter. For paired t tests diagnose differences; for regression inspect residuals. Rank tests change the question and retain assumptions. Prespecify Welch for independent mean comparisons; use Games Howell pairwise comparisons when appropriate to a heteroscedastic one way design. Regression HC3 addresses heteroscedasticity, not clustering, confounding or a wrong mean model. The helper rejects infinities, undersized or degenerate samples, reports omitted NaNs, retains original outlier positions, and rejects missing group labels. No binary Shapiro verdict is returned above 5000 observations, where its p value may be inaccurate. Grouped output uses outliers per group ; legacy is normal / is homogeneous and the Normal table label mean only non rejection. Independence is established through design review, never this helper. Pass ordered=True to regression diagnostics only for meaningful row order; Durbin Watson is descriptive without a universal cutoff. See [assumptions and diagnostics](references/assumptions and diagnostics.md). Running Statistical Tests Primary libraries: pingouin : user friendly tests that return effect sizes by default — prefer it for standard tests scipy.stats : core statistical tests statsmodels : regression, diagnostics, power analysis pymc + arviz : Bayesian modeling and diagnostics T Test with Complete Reporting ANOVA with Post Hoc Tests Linear Regression with Diagnostics Bayesian T Test This is a shared variance normal model, not Welch's model or a Bayes factor calculation. Choose priors using meaningful units and external knowledge; inspect prior predictions and prior sensitivity. Short synthetic API smoke tests do not establish convergence; inspect finite R hat/ESS/MCSE, divergences and posterior predictions before reporting. Effect Sizes Effect sizes quantify magnitude; p values measure incompatibility with a specified null model under its assumptions. A p value does not prove that an effect exists or measure its practical importance. Report an effect estimate and uncertainty for every test. See references/effect sizes and power.md for the full guide. Quick Reference: Common Effect Sizes Test Effect Size Small Medium Large T test Cohen's d 0.20 0.50 0.80 ANOVA η² p 0.01 0.06 0.14 Correlation r 0.10 0.30 0.50 Regression R² 0.02 0.13 0.26 Chi square (2×2) Cramér's V 0.10 0.30 0.50 Benchmarks are conventions, not laws — a "small" effect can matter enormously (drug side effects) and a "large" one can be trivial. Interpret in context. Calculating Effect Sizes Pingouin returns effect sizes with its tests ( cohen d magnitude from pg.ttest , np2 from pg.anova , hedges from pg.pairwise tukey ; r from pg.corr is already an effect size). Confidence Intervals for Effect Sizes Report a CI for the effect size to show its precision; use analyzed sample sizes after the declared missing data policy. The following is an approximate pooled d interval, not a heteroscedastic standardized effect guarantee. Use pg.compute esci (note: pg.compute effsize from t returns only the point estimate — it does not return a CI): Power Analysis A Priori Power Analysis (Study Planning) Determine required sample size before data collection. These are independent, equal variance planning models; unequal variances, clustering, attrition or a different estimand require appropriate design specific calculations or simulation: Sensitivity Analysis (Post Study) Determine what effect size the study could detect: Note : Post hoc "observed power" (computing power from the observed effect) is circular and misleading — it is a deterministic function of the p value. If a study is done and someone asks about power, run a sensitivity analysis instead. See references/effect sizes and power.md for detailed guidance. Reporting Results Follow references/reporting standards.md for APA style. Every report needs: 1. Descriptive statistics : M, SD, n for all groups/variables 2. Test statistics : Test name, statistic, df, exact p value ( p = .034 , not p < .05 ; use p < .001 only below .001) 3. Effect sizes : With confidence intervals 4. Assumption checks : Which tests were run, results, and actions taken 5. All planned analyses : Including non significant findings — omitting them is cherry picking Example Report Templates The numbers below are illustrative formatting examples, not computed results or a mutually consistent dataset. Replace every number, model and diagnostic statement with the actual analysis. Independent T Test One Way ANOVA Multiple Regression Bayesian Analysis For rank methods, report distribution summaries (often medians/IQRs), the U/W/H statistic, and a rank based effect size (e.g., rank biserial correlation, returned by pg.mwu as RBC ). Bayesian Statistics Consider Bayesian approaches when: You have prior information to incorporate You want direct probability statements about hypotheses ("there is a 95% probability the effect lies in this interval") Small samples or sequential collection require prior sensitivity and an explicit stopping/decision rule; Bayesian methods do not universally remove optional stopping concerns You need to quantify evidence for the null hypothesis The model is complex (hierarchical structure, missing data) See references/bayesian statistics.md for prior specification, Bayes Factors, credible intervals, hierarchical models, and convergence checking (R hat < 1.01, sufficient ESS, posterior predictive checks). Bundled Resources References ( references/ ) test selection guide.md : Decision tree covering group comparisons, relationships, counts, time to event, agreement/reliability, and categorical analysis assumptions and diagnostics.md : Detailed guidance on checking and handling assumption violations effect sizes and power.md : Calculating, inte

statistical-analysis