A Model Output Is Not Valid Until Its Assumptions Are Checked

Statistical software will usually return an estimate even when the selected model is a poor representation of the data-generating process. A coefficient table is therefore not a certificate of validity. Its standard errors, confidence intervals, p-values, and predictions depend on assumptions about design, measurement, functional form, dependence, distributions, missingness, and model specification.

Assumption checking should begin before fitting and end with a short report attached to the result.

01Separate design assumptions from residual diagnostics

Some assumptions cannot be rescued by inspecting a plot. Causal interpretations may require exchangeability, correct temporal ordering, no inappropriate adjustment, consistency, and positivity. Randomized trials depend on valid allocation and appropriate handling of deviations and intercurrent events. Observational analyses depend heavily on how confounding, selection, and measurement were addressed.

These are scientific and design assumptions. Record them explicitly and connect each to design evidence or a sensitivity analysis. A perfect residual plot does not make an unmeasured-confounding claim true.

02Check the model-specific structure

For linear regression, common concerns include linearity of the mean relationship, independent errors, constant conditional variance, influential observations, and whether distributional assumptions are adequate for the intended inference. Residual-versus-fitted plots can reveal curvature or changing variance; time or sequence plots can reveal dependence; influence measures can show whether a few records dominate the estimate.

Logistic regression requires attention to functional form for continuous predictors, sparse data or separation, influential observations, and fit for the intended purpose. Survival models may require assessment of proportional hazards if that model is used, appropriate censoring assumptions, time origin, competing events, and functional form. Repeated or clustered data require a dependence structure or robust approach suited to the design.

The NIST statistical handbook emphasizes graphical residual analysis because non-random structure can signal inadequate functional form, non-constant variation, or dependence. A single normality test is not a complete diagnostic strategy, and with large samples trivial departures can become “significant.”

03Examine data support

Models can be mathematically estimable while relying on weakly supported regions. Check the number of participants and events relative to model complexity, overlap across exposure or treatment groups, empty covariate combinations, extrapolation beyond observed ranges, collinearity, and influential sites or clusters.

For prediction, evaluate performance in data not used to fit or tune the model using an appropriate internal-validation method, and seek external validation for new settings. For explanatory models, do not select covariates solely from univariable p-values; the adjustment strategy should follow the scientific question and assumed causal structure.

04Connect missingness to the estimand

Complete-case analysis, imputation, weighting, and likelihood methods all make assumptions. Describe which variables enter the missingness model, whether important predictors of missingness and outcomes are available, and how the analysis changes under plausible departures.

ICH E9(R1) treats sensitivity analysis as a structured assessment of robustness to assumptions connected to the estimand. “We used multiple imputation” is not itself evidence that missingness was handled credibly.

05Respond proportionately to violations

An assumption check is not a hunt for a model with perfect plots. Models are approximations, and some procedures are robust to modest departures. The question is whether a departure materially affects the target estimate or decision.

Responses may include a transformation, nonlinear term, different variance estimator, cluster-aware model, alternative effect measure, stratification, robust or semiparametric method, revised estimand, or sensitivity analysis. Any change made after seeing outcomes should be documented as an amendment or exploration, not retroactively presented as prespecified.

06Attach an assumptions report

For every primary model, include:

1. estimand and model;
2. assumptions that matter for interpretation;
3. diagnostic method and prespecified decision rule;
4. result, preferably with key plots;
5. corrective or sensitivity analysis;
6. impact on estimate and conclusion; and
7. reviewer and software/code version.

Do not reduce the report to “all assumptions were met.” State what was examined and what remains untestable. The final manuscript can summarize the evidence and link to fuller material.

Before a result is approved, ask a second analyst to identify the three assumptions most capable of reversing the conclusion. If the team cannot show how those assumptions were assessed or stress-tested, the model output is provisional—regardless of how polished the table appears.

Archive the diagnostic figures and sensitivity outputs with the analysis release. Reviewers should be able to connect the final claim to the exact evidence used to judge whether each material assumption was acceptable.

This article is educational and intended for research purposes. It does not provide individual medical advice, diagnosis, or treatment. No patient data were used.