Guided statistical analysis for research data - test selection, assumption checking, effect sizes, power analysis, Bayesian alternatives, and APA-formatted reporting. Use whenever a user wants to compare groups, test a hypothesis, analyze experimental or survey data, check statistical assumptions, c

Statistical Analysis Overview Conduct hypothesis tests (t tests, ANOVA, chi square), regression, correlation, and Bayesian analyses with systematic assumption checking, effect sizes, and APA style reporting. The goal is an analysis a reviewer could not tear apart: the right test, verified assumptions, honest effect sizes, and a complete write up. When to Use This Skill Use this skill when: Conducting statistical hypothesis tests (t tests, ANOVA, chi square, non parametric) Performing regression or correlation analyses Running Bayesian statistical analyses Checking statistical assumptions and diagnostics Calculating effect sizes and conducting power analyses Reporting statistical results in APA format Analyzing experimental or observational data for research Installation Use uv to install the libraries used in this skill. Pin versions in production; unpinned installs are fine for exploration. Compatibility notes (verified against pingouin 0.6.1, statsmodels 0.14.6, arviz 1.2, 2026): Pingouin 0.6.0 renamed output columns to remove special characters: p val , cohen d , CI95 , p unc (previously p val , cohen d , CI95% , p unc in 0.5.x). Examples below use the current names; if stuck on 0.5.x, use the hyphenated forms. statsmodels + SciPy : use statsmodels =0.14.6 with scipy =1.11 to avoid lazywhere import errors on SciPy 1.16+. ArviZ 1.x : az.summary() now defaults to 89% intervals ( eti89 columns) and the width parameter is ci prob (not hdi prob ). To report a conventional 95% credible interval, pass az.summary(trace, ci prob=0.95) . One sided Bayes Factors are gone from Pingouin : pg.ttest(..., alternative='greater') silently drops the BF10 column, and pg.bayesfactor ttest raises on one sided alternatives. For one sided Bayesian tests, use PyMC directly (compute the posterior probability of the directional hypothesis) or JASP/R's BayesFactor. For model specific APIs (OLS, GLM, ARIMA), see the statsmodels skill. For PyMC workflows, see the pymc skill. Analysis Workflow Every sound analysis follows the same arc. Skipping steps is how analyses end up retracted, so work through them in order and say what you did at each one. 1. Frame the question before touching the data. State the hypothesis, the outcome and predictor variables, and the design (independent vs. paired, number of groups). Commit to a planned test now — choosing the test after peeking at results is p hacking, even when done innocently. 2. Inspect the data. Per group: n, mean, SD, median, missing values. Plot the raw data (histograms or box plots) before any test. Unequal group sizes, missingness, floor/ceiling effects, and outliers all change what test is appropriate — surface them to the user rather than silently working around them. 3. Select the test using the quick reference below, or references/test selection guide.md for designs beyond the basics (counts, time to event, reliability, factorial). 4. Check assumptions with scripts/assumption checks.py . If an assumption fails, switch to the remedial test (table below) and report both the plan and the change. 5. Run the test and always compute the effect size alongside it — a p value says an effect exists; the effect size says whether anyone should care. 6. Report using the APA templates below, including descriptives, exact statistics, effect sizes with CIs, and the assumption checks performed. If the user only needs one step (e.g., "how many participants do I need?"), jump straight to that section — but still confirm the design assumptions the calculation rests on. Test Selection Guide Quick Reference: Choosing the Right Test Use references/test selection guide.md for comprehensive guidance (counts, survival, reliability, factorial designs). Quick reference: Comparing Two Groups: Independent, continuous, normal → Independent t test Independent, continuous, non normal → Mann Whitney U test Paired, continuous, normal → Paired t test Paired, continuous, non normal → Wilcoxon signed rank test Binary outcome → Chi square or Fisher's exact test Comparing 3+ Groups: Independent, continuous, normal → One way ANOVA Independent, continuous, non normal → Kruskal Wallis test Paired, continuous, normal → Repeated measures ANOVA Paired, continuous, non normal → Friedman test Relationships: Two continuous variables → Pearson (normal) or Spearman correlation (non normal) Continuous outcome with predictor(s) → Linear regression Binary outcome with predictor(s) → Logistic regression Bayesian Alternatives: All tests have Bayesian versions providing direct probability statements about hypotheses, Bayes Factors quantifying evidence, and the ability to support the null. See references/bayesian statistics.md . Assumption Checking Always check assumptions before interpreting test results , and report the checks — reviewers look for them. Use the bundled scripts/assumption checks.py module. Run Python from the skill directory ( skills/statistical analysis/ ) or add scripts/ to sys.path : For targeted checks, import individual functions: What to Do When Assumptions Are Violated Normality violated: Mild violation + n 30 per group → Proceed with parametric test (robust) Moderate violation → Use non parametric alternative Severe violation → Transform data or use non parametric test Homogeneity of variance violated: For t test → Use Welch's t test ( pg.ttest applies it automatically with correction='auto' ) For ANOVA → Use Welch's ANOVA ( pg.welch anova ) or Brown Forsythe For regression → Use robust standard errors or weighted least squares Linearity violated (regression): Add polynomial terms, transform variables, or use non linear models / GAM Formal tests get oversensitive as n grows: for n ≥ 100, weigh the Q Q plot more heavily than the Shapiro Wilk p value. See references/assumptions and diagnostics.md for comprehensive guidance. Running Statistical Tests Primary libraries: pingouin : user friendly tests that return effect sizes by default — prefer it for standard tests scipy.stats : core statistical tests statsmodels : regression, diagnostics, power analysis pymc + arviz : Bayesian modeling and diagnostics T Test with Complete Reporting ANOVA with Post Hoc Tests Linear Regression with Diagnostics Bayesian T Test Scale priors to the data (e.g., sigma=10 suits outcomes with SD near 10; use the observed SD as a guide) and state the priors in the report. Effect Sizes Effect sizes quantify magnitude; p values only indicate existence. Report one for every test. See references/effect sizes and power.md for the full guide. Quick Reference: Common Effect Sizes Test Effect Size Small Medium Large T test Cohen's d 0.20 0.50 0.80 ANOVA η² p 0.01 0.06 0.14 Correlation r 0.10 0.30 0.50 Regression R² 0.02 0.13 0.26 Chi square Cramér's V 0.07 0.21 0.35 Benchmarks are conventions, not laws — a "small" effect can matter enormously (drug side effects) and a "large" one can be trivial. Interpret in context. Calculating Effect Sizes Pingouin returns effect sizes with its tests ( cohen d from pg.ttest , np2 from pg.anova , hedges from pg.pairwise tukey ; r from pg.corr is already an effect size). Confidence Intervals for Effect Sizes Report a CI for the effect size to show its precision. Use pg.compute esci (note: pg.compute effsize from t returns only the point estimate — it does not return a CI): Power Analysis A Priori Power Analysis (Study Planning) Determine required sample size before data collection: Sensitivity Analysis (Post Study) Determine what effect size the study could detect: Note : Post hoc "observed power" (computing power from the observed effect) is circular and misleading — it is a deterministic function of the p value. If a study is done and someone asks about power, run a sensitivity analysis instead. See references/effect sizes and power.md for detailed guidance. Reporting Results Follow references/reporting standards.md for APA style. Every report needs: 1. Descriptive statistics : M, SD, n for all groups/variables 2. Test statistics : Test name, statistic, df, exact p value ( p = .034 , not p < .05 ; use p < .001 only below .001) 3. Effect sizes : With confidence intervals 4. Assumption checks : Which tests were run, results, and actions taken 5. All planned analyses : Including non significant findings — omitting them is cherry picking Example Report Templates Independent T Test One Way ANOVA Multiple Regression Bayesian Analysis If a non parametric test was used, report medians rather than means, the U/W/H statistic, and a rank based effect size (e.g., rank biserial correlation, returned by pg.mwu as RBC ). Bayesian Statistics Consider Bayesian approaches when: You have prior information to incorporate You want direct probability statements about hypotheses ("there is a 95% probability the effect lies in this interval") Sample size is small or data collection is sequential (no correction needed for optional stopping) You need to quantify evidence for the null hypothesis The model is complex (hierarchical structure, missing data) See references/bayesian statistics.md for prior specification, Bayes Factors, credible intervals, hierarchical models, and convergence checking (R hat < 1.01, sufficient ESS, posterior predictive checks). Bundled Resources References ( references/ ) test selection guide.md : Decision tree covering group comparisons, relationships, counts, time to event, agreement/reliability, and categorical analysis assumptions and diagnostics.md : Detailed guidance on checking and handling assumption violations effect sizes and power.md : Calculating, interpreting, and reporting effect sizes; power analysis bayesian statistics.md : Priors, Bayes Factors, credible intervals, hierarchical models, diagnostics reporting standards.md : APA style reporting guidelines with worked examples Scripts ( scripts/ ) assumption checks.py : Automated assumption checking with visualizations comprehensive assumption check() : outliers + normality + variance homogeneity in one call check normality() , check normality per group() : Shapiro Wilk with Q Q plots check homogeneity of variance() : Levene's test with box plots check regression diagnostics() : 4 panel residual plots + Shapiro Wilk, Breusch Pagan, Durbin Watson, VIF for fitted OLS models check linearity() , detect outliers() Statistical Integrity These are the practices that keep an analysis defensible. They matter because the most common statistical failures are not computational errors — they are silent flexibility (testing until something works) and selective reporting. 1. Distinguish confirmatory from exploratory. State the planned analysis before running it; label anything discovered along the way as exploratory. 2. Don't shop for significance. If the planned test is non significant, that is the result. Trying alternative tests, subgroups, or outlier removal schemes until p < .05 invalidates the p value. 3. Correct for multiple comparisons when running families of tests (Tukey HSD for post hoc ANOVA; Holm or Benjamini Hochberg FDR for other families) and say which correction was used. 4. A non significant result is not evidence of no effect. With small n, the study may simply have been underpowered — run a sensitivity analysis, or use a Bayesian analysis / equivalence test to actually quantify support for the null. 5. Statistical significance is not practical importance. With large n, trivial effects reach p < .001. Lead the interpretation with the effect size. 6. Understan

statistical-analysis