How to Choose a Statistical Test for Your Dissertation

You don't choose a statistical test from a list. Three facts about your study choose it for you: what your research question is asking, how your outcome is measured, and how your observations are related to each other. Once you can state those three things for a research question, there are usually only one or two defensible tests left.

That is also what your committee wants to hear. "I used a t-test because the prior study did" is not a reason. "The question compares two separate groups on a continuous outcome" is.

This guide covers the quantitative tests most dissertations use. If your design is not in the table below, the three questions still apply, but the answer is a model this page does not cover, and that is a good point to get advice.

Three questions that pick the test

Answer these for each research question, in this order.

1. What is the research question asking?

Most quantitative research questions are one of four kinds. The verb usually tells you which.

The question asks aboutTypical wordingFamily of tests
A difference between groups"Does X differ between..."t-tests, ANOVA, chi-square
A change in the same people"Did X change from before to after..."Paired and repeated-measures tests
A relationship between two variables"Is X related to Y..."Correlation
Prediction of an outcome from several variables"Do X, Y, and Z predict..."Regression

If the wording of your research question and the test you plan don't match (the question says "predict" and the plan says t-test), fix one of them before the proposal. Committees read that mismatch as a sign the method was chosen first.

2. How is the outcome measured?

The outcome is the variable you are trying to explain. Its level of measurement narrows the choice more than anything else.

  • Continuous. A score or measurement where the distances between values mean something: a scale total, a test score, blood pressure, hours.
  • Ordinal. Ordered categories where the distances are not equal: a single item answered from strongly disagree to strongly agree, or a rank.
  • Binary. Two categories: passed or failed, stayed or left.
  • Categorical. Three or more categories with no order: unit, specialty, program type.
  • Count. A number of events: falls, absences, visits.

3. How are the observations related?

  • Independent. Each person appears once, and one person's score tells you nothing about another's.
  • Paired or repeated. The same people are measured more than once, or people are matched in pairs.
  • Nested. People are grouped inside units that make them more alike: nurses within hospitals, students within classrooms.

This question is easy to skip, and getting it wrong makes the test wrong. A test built for independent groups gives the wrong answer on paired data, even when everything else is right.

Then count two things: how many groups or time points you are comparing, and how many predictors or control variables you have. Those decide between a t-test and an ANOVA, and between a simple test and a regression.

The lookup table

Find the row that matches your answers. The last column is what to plan for if the usual test's assumptions don't hold.

Comparing groups or time points

Your questionOutcomeUsual testCommon alternative
Do two separate groups differ?ContinuousIndependent-samples t-testWelch's t-test or Mann-Whitney U
Do the same people differ at two time points?ContinuousPaired-samples t-testWilcoxon signed-rank
Do three or more separate groups differ?ContinuousOne-way ANOVA with post hoc testsWelch's ANOVA or Kruskal-Wallis
Do the same people differ across three or more time points?ContinuousRepeated-measures ANOVAMixed (multilevel) model or Friedman test
Do groups change differently over time?ContinuousMixed ANOVA (group by time)Mixed (multilevel) model
Do two grouping variables matter, alone and together?ContinuousTwo-way ANOVARegression with an interaction term
Do groups differ after accounting for a control variable?ContinuousANCOVAMultiple regression
Do two separate groups differ in a yes or no outcome?BinaryChi-square test of independenceFisher's exact test
Did a yes or no outcome change in the same people?BinaryMcNemar's test

Relationships and prediction

Your questionOutcomeUsual testCommon alternative
Are two continuous variables related?ContinuousPearson correlationSpearman correlation
Are two ordinal variables related?OrdinalSpearman correlation
Are two categorical variables related?CategoricalChi-square test of independenceFisher's exact test
Do several variables predict an outcome?ContinuousMultiple regressionRobust standard errors or a transformation
Do several variables predict a yes or no outcome?BinaryLogistic regression
Do several variables predict an ordered outcome?OrdinalOrdinal logistic regressionMultinomial logistic regression
Do several variables predict an outcome with three or more unordered categories?CategoricalMultinomial logistic regression
Do several variables predict a count?CountPoisson regressionNegative binomial regression

Two notes on reading the table. First, the rows build on each other. A t-test is a one-way ANOVA with two groups, and both are special cases of regression, so when you add a control variable to a group comparison you have moved to ANCOVA or regression whether you call it that or not. Second, a nonparametric alternative answers a slightly different question (it compares ranks, not means), so name it in your plan before you see the data and not after a result disappoints you.

Worked example

This example is a made-up study of a fall-prevention training for hospital nurses. Each nurse takes a knowledge test before and after the training.

Research questionWhat it asksOutcomeHow observations are relatedTest
1. Did knowledge scores change from before to after the training?Change in the same peopleKnowledge score (continuous)Paired: each nurse is measured twicePaired-samples t-test
2. Do post-training knowledge scores differ across three units (medical-surgical, intensive care, emergency)?Difference between groupsKnowledge score (continuous)Independent: each nurse is in one unitOne-way ANOVA with post hoc tests
3. Do post-training knowledge score and years of experience predict whether a nurse passes the skills check?PredictionPassed or did not pass (binary)IndependentLogistic regression

Now change one fact at a time and watch the test change:

  • The surveys were anonymous and you can't match each nurse's two scores. Question 1 is no longer paired. You are left comparing two sets of scores as if they came from separate groups, with an independent-samples t-test, which is weaker and harder to defend. If you plan a before-and-after design, plan a way to link the two surveys, such as a code each participant creates.
  • The knowledge test is scored pass or fail. Question 1 becomes McNemar's test, and question 2 becomes a chi-square test.
  • There are two units, not three. Question 2 becomes an independent-samples t-test.
  • You want to compare units while accounting for years of experience. Question 2 becomes ANCOVA or multiple regression.
  • The skills check is a score from 0 to 100. Question 3 becomes multiple regression.

None of these changes involved the software or the sample size. The test changed because the question, the outcome, or the structure of the data changed.

Cases the table doesn't settle

Likert items and scale scores

A single item answered from strongly disagree to strongly agree is ordinal. A scale score made by summing or averaging several such items is usually treated as continuous, and most published research in the social and health sciences does this. Say which one you have. If your outcome is a single item, plan an ordinal or nonparametric test. If it is a multi-item scale, report its reliability and say you are treating the score as continuous. Some committees hold a stricter view, so ask your chair early.

Outcomes that aren't normally distributed

The t-test and ANOVA hold up reasonably well when the outcome is somewhat non-normal, as long as the groups are not tiny and are similar in size. With small or very unequal groups, or a heavily skewed outcome, plan the alternative in the table. For regression, the assumption is about the residuals, not the raw outcome.

Small samples

A nonparametric test does not rescue a study that is too small to detect the effect. With a small sample, plan fewer comparisons and fewer predictors, use exact tests for categorical data, and be direct in your limitations about what the study could and could not detect. See the sample size guide.

Repeated measures with missing time points

Repeated-measures ANOVA drops anyone who is missing any time point. If you expect people to miss a wave, a mixed (multilevel) model uses everyone's available data and is usually the better plan.

Nested data

If your participants come in clusters (nurses within units, students within classrooms, clients within clinicians), their scores are not independent, and the standard tests will make your results look more certain than they are. The usual answers are a multilevel model or cluster-robust standard errors. Which one is workable depends on how many clusters you have, so get advice before the proposal.

Moderation and mediation

These are research questions, not tests. Moderation ("does the effect of X depend on Z?") is tested with an interaction term in a regression or a two-way ANOVA. Mediation ("does X affect Y through M?") is tested by estimating the indirect effect, usually with a bootstrapped confidence interval. Neither one establishes cause on its own. That depends on your design.

Several outcomes

If you have several related outcomes, you can run a separate test for each and adjust for multiple comparisons, or use a multivariate test such as MANOVA. Separate tests with an adjustment are easier to explain and are enough for most dissertations. Ask your chair which the committee expects.

Scales you are building or validating, and latent variables

Factor analysis and structural equation modeling answer different kinds of questions and have their own sample size and reporting requirements. If your study needs them, plan for more time and get advice.

How to write the justification

A justification is your three answers written as a sentence or two, followed by the test and the fallback. It belongs in the data analysis section of Chapter 3, once for each research question. The analysis plan guide covers the rest of that section.

These are models to adapt to your own study, not text to copy. The study is made up for illustration.

Weak: "A paired-samples t-test was chosen because it is commonly used to compare pretest and posttest scores."

Stronger: "Research question 1 asks whether knowledge scores changed from before to after the training. The outcome is a continuous score on the [instrument], and each nurse is measured twice, so the two sets of scores are paired. A paired-samples t-test will be used. If the difference scores are markedly non-normal, a Wilcoxon signed-rank test will be reported instead."

The weak version says the test is popular. The stronger version gives the kind of question, the level of measurement, and the structure of the data, which are the three reasons the test fits. It also names the fallback in advance.

A version for a regression question:

"Research question 3 asks whether post-training knowledge score and years of experience predict whether a nurse passes the skills check. The outcome is binary (passed or did not pass) and there are two predictors, so logistic regression will be used. Odds ratios with 95% confidence intervals will be reported for each predictor."

You can cite a methods text for the test. A citation supports the justification, but it does not replace it.

What committees push back on

Most concerns about test choice come down to a test that was picked for a reason other than fit.

  • A test chosen because a prior study used it. Their question, measures, and design were not yours.
  • Paired data analyzed as if the groups were independent, or the reverse.
  • A continuous variable cut into "high" and "low" groups so a t-test or chi-square can be run. This throws away information. Keep the variable continuous and use correlation or regression.
  • A string of separate t-tests where one ANOVA or one regression answers the question.
  • A single Likert item treated as continuous with no comment.
  • Clustering ignored. Participants recruited by unit, site, or classroom and analyzed as if they were unrelated.
  • Stepwise regression used to pick the predictors. Choose predictors from your framework and literature, and say why each is there.
  • A nonparametric test that appears only after the first test was not significant.
  • The word "predict" or "effect" in the research question, with a design that can only show association. The test does not make the study causal.
  • A test chosen because a committee member said to use it, when you don't understand why it fits or how it works. If someone recommends a test, ask why, and make sure you can explain it yourself.

Checklist before it goes to your chair

  • For each research question, I can say what kind of question it is: difference, change, relationship, or prediction.
  • The wording of each research question matches the test planned for it.
  • Each outcome's level of measurement is stated, and the test fits it.
  • I have said whether the observations are independent, paired, or nested, and the test fits that.
  • No continuous variable has been split into groups without a reason I can defend.
  • Each test has a named alternative if its assumptions fail.
  • Control variables are in the model, not handled with separate tests.
  • Each justification gives my reasons, not only a citation.
  • The power analysis is for the test I actually plan to run.

Get the test selection worksheet

A free two-page Word file: the three questions for each of your research questions, a one-page version of the lookup table, a fill-in version of the justification paragraph, and the checklist.

I'll email you once or twice about dissertation help. No newsletter. Reply "stop" any time.

Related guides

Work with Matt

Not sure which test fits, or whether your committee will accept it? The Methods Consult ($195) is a 90-minute working session with a written summary you can bring to your chair. If you want every research question mapped to its variables and tests before your proposal defense, that's the Analysis Plan ($595): power analysis, assumptions, and the reasoning behind each choice, with one revision included.

Next
Next

How to Write a Data Analysis Plan for Your Dissertation