How to Choose a Statistical Test: A Step-by-Step Guide for Dissertation Data

Knowing how to choose a statistical test comes down to five answers: what your research question asks, what kind of outcome you measured, how many groups or predictors you have, whether your observations are independent, and whether your data meets the assumptions behind the test. Settle those before you open SPSS, R or Excel, and the shortlist usually shrinks to one or two tests.

The methods below cover what most undergraduate, Master’s and PhD projects use: t-tests, ANOVA, chi-square, correlation, regression and their rank-based alternatives. Survival analysis, multilevel models and time-series methods need more specialist advice, so they sit outside the scope here.

Put these questions to yourself before you commit to anything:

• Is my research question asking for a difference, an association or a prediction?
• Did I measure my outcome as a score, a count, a rank or a category?
• Are my groups made of different people, or the same people measured more than once?
• Should I check normality on my raw scores, or on something else?
• What do I do if my supervisor, my software and my textbook disagree?

Which Statistical Test Should I Use? The Five-Question Shortcut

Answer five questions in order and write each answer down. Your shortlist appears as you go.

1. What does my question ask: compare groups, measure an association or predict an outcome?
2. What type is my outcome variable: continuous, count, ordinal, nominal or binary?
3. How many groups, conditions or predictors do I have?
4. Are my observations independent, or paired and repeated?
5. Does my data meet the test’s assumptions, or does a rank-based or adjusted version fit better?

These five questions mirror the decision paths that university and NHS teaching materials publish. The flowchart on Stats and R, the library guides at Southampton and Leeds, and the NHS Fife research study guide all turn on the same things: the type of variable, the number of groups, whether data is paired, and whether the distribution suits a parametric test. If you would rather click through the questions than read them, the free statistical test selector walks you through the same logic.

Start With Your Research Question, Not Your Software

Students often reach for a familiar test, usually the t-test, and then bend the question to fit it. A teaching assistant writing for Stats and R makes a useful observation: most students can run a test once someone names it, but struggle to decide which test to run. Reverse the habit by letting the question pick the test.

The BMJ’s Statistics at Square One puts the first step plainly: ask what the main study hypothesis is. Without a hypothesis there is nothing to test, only something to describe. The same chapter adds that the analysis should reflect the design, so a matched design calls for a matched analysis.

The University of Southampton’s library guide sorts quantitative questions into four kinds, and the wording of yours usually gives it away:

• Descriptive: “What was the median age of my respondents?” This needs descriptive statistics, not a hypothesis test.
• Comparative: “Do part-time and full-time students report different stress scores?” This points to a group comparison such as a t-test or ANOVA.
• Correlational: “Is weekly study time related to exam marks?” This points to correlation or a chi-square test of association, depending on the variable types.
• Regressive: “How well do attendance and prior marks explain a final grade?” This points to a regression model.

One dissertation can need several tests, because each research question or hypothesis may call for its own. The University of Lincoln’s maths and stats support makes the same point. If your question still reads loosely, learn how to write a research question that can be tested before you pick any method. Your hypotheses then follow from the literature, and the guide on how to develop hypotheses from your literature review shows how.

Identify Your Variable Types Before Anything Else

Start with the outcome variable, because it drives the shortlist more than any other choice. University of Sheffield statistics tutors, the NHS Fife guide and Scribbr all sort variables in broadly the same way:

• Continuous: measured on a scale and able to take any value in a range, such as reaction time or an exam mark.
• Discrete count: whole numbers, such as GP visits in a year.
• Ordinal: ordered categories where the gaps are not equal, such as pain rated from none to severe.
• Nominal: categories with no natural order, such as degree subject.
• Binary: two categories, such as pass or fail.

Summarise before you test. The Sheffield guide suggests reporting means and standard deviations for roughly normal scale data, and medians with interquartile ranges when data is skewed or ordinal. A large gap between the mean and the median is an early warning of skew or influential outliers.

Likert Scale Data: Treat It as Scale or Ordinal?

A single five-point item is ordinal, so rank-based methods are the cautious choice. Sheffield’s statistics tutors note that some departments routinely analyse ordinal variables with seven or more categories using parametric tests when the data is roughly normal. They also note that summing or averaging several items from one questionnaire scale produces a variable that can reasonably be treated as scale data. Departments differ, so check what yours expects and explain your reasoning in the methodology.

Independent or Paired? Why Study Design Changes the Test

Ask whether each observation comes from a different person. Marks from two separate seminar groups are independent. The same students’ marks before and after a workshop are paired, and so are matched pairs such as a patient and a carefully chosen control.

The distinction matters because tests for independent groups assume that one person’s score says nothing about another’s. Paired data breaks that assumption, so you need a test built for pairs, such as the paired t-test or Wilcoxon signed-rank test. When the same people are measured three or more times, repeated-measures ANOVA or the Friedman test takes over.

Nesting causes a quieter version of the same problem. Sheffield’s guide gives children within classes as an example: pupils taught by the same teacher share an influence. Multilevel modelling handles that, though it sits beyond most student projects, so raise it with your supervisor early if your sampling looks like this.

Statistical Test Selection Tables: Match Your Question to a Test

Once you know your question, variable types and design, the tables below give the usual starting point. They draw on the decision tables in the UCLA, Southampton and Sheffield guidance, which all agree on the main pairings. Treat them as a first draft, not a rule. UCLA’s own table calls these general guidelines, since most data can be analysed in more than one defensible way, and its full table of tests with software instructions covers many more situations.

Comparing Groups or Conditions

Your situation	Usual parametric test	Rank-based alternative
One sample against a known value	One-sample t-test	One-sample Wilcoxon signed-rank test
Two independent groups	Independent-samples t-test (Welch version if variances differ)	Mann-Whitney U (Wilcoxon rank-sum)
Two related measurements	Paired-samples t-test	Wilcoxon signed-rank test (sign test for ordinal data)
Three or more independent groups	One-way ANOVA (Welch ANOVA if variances differ)	Kruskal-Wallis test
Three or more measurements on the same people	Repeated-measures ANOVA	Friedman test
Two factors, independent groups	Two-way (factorial) ANOVA	Specialist options such as the Scheirer-Ray-Hare test
Independent groups plus repeated measures	Mixed ANOVA	No simple equivalent

Associations and Predictions

Your question	Usual test
Two continuous variables with a linear relationship	Pearson’s correlation (Spearman’s rank or Kendall’s tau if skewed, ordinal or full of tied ranks)
Two categorical variables	Chi-square test of association (Fisher’s exact test when expected counts are small)
The same people answering a yes or no item twice	McNemar’s test
Continuous outcome, one or more predictors	Simple or multiple linear regression
Continuous outcome, groups plus a continuous covariate	ANCOVA
Binary outcome	Binary logistic regression
Ordinal outcome	Ordinal regression
Nominal outcome with three or more categories	Multinomial logistic regression
Count outcome	Poisson regression

When two tests could both answer the question, pick the simpler one. The Southampton guide advises keeping analyses as simple as possible and using complex models only when necessary. Stats and R’s author takes a similar line: with two quantitative variables, he tends to suggest a correlation over a regression at introductory level.

Check the Assumptions That Belong to Your Test

Scribbr lists three assumptions that parametric tests commonly share: independence of observations, similar variance across the groups you compare, and normally distributed quantitative data. Not every test needs all three, and each test applies them to a different thing. That last point catches many students out.

Check Normality on the Right Thing

Running a normality check on your whole dataset is rarely what the test needs. The Sheffield tutors’ guide sets out what to check for each common test:

Test	What should look roughly normal
Independent t-test	The outcome within each group
Paired t-test	The paired differences
One-way ANOVA	The residuals
Repeated-measures ANOVA	The residuals at each time point
Pearson’s correlation	Both variables
Simple linear regression	The residuals

The UCLA guidance makes a related point: strictly speaking, the normality assumption concerns the errors of a model, not the raw outcome, and many models are fairly robust to departures from normality in large samples.

Plot first. A histogram or Q-Q plot shows the shape of the distribution and names the problem, whether that is skew, heavy tails or an outlier. Formal tests come second. Shapiro-Wilk is widely preferred for small to moderate samples, and Kolmogorov-Smirnov is generally less sensitive. However, formal tests have a sample-size trap. The Sheffield guide warns that with fewer than about 20 cases they rarely detect non-normality, and with more than about 50 they can flag departures too small to matter. A significant result on a large sample is therefore a prompt to look at the plot, not an automatic switch to a rank-based test. The Southampton library guide also lists skewness and kurtosis as additional checks.

If you cannot plot the data, NHS Fife’s guide offers two quick signals. Pearson’s skewness coefficient (mean minus median, divided by the standard deviation) beyond roughly plus or minus 0.2 suggests skew. For a variable that can only be positive, a mean minus two standard deviations that falls below zero also hints at skew.

Equal Variances and the Welch Question

For independent t-tests and ANOVA, Levene’s test checks whether group variances are similar. If they differ, the Sheffield guide points to the “equal variances not assumed” row in SPSS output for t-tests, and to Welch’s ANOVA with a Games-Howell post hoc test for three or more groups.

Some statisticians go further. Delacre, Lakens and Leys argued in 2017 that psychologists should use Welch’s t-test by default, because it controls false positives better when variances differ and loses little when they do not. R’s t.test() already works this way. More recent simulation work from TU Delft urges caution, however, finding inflated false-positive rates for Welch’s test with skewed data and unequal group sizes. The practical advice is to state which version you ran and why, and to check what your department expects.

Other Checks Worth Knowing

• Sphericity: repeated-measures ANOVA assumes the variances of the differences between conditions are equal. Mauchly’s test checks this, and the Greenhouse-Geisser or Huynh-Feldt correction applies if it fails.
• Expected counts: chi-square works best when at least 80 percent of expected cell counts exceed 5 and none falls below 1. Fisher’s exact test or merged categories help when they do not.
• Linearity and outliers: Pearson’s correlation and linear regression assume a straight-line relationship. Check a scatterplot, and flag points beyond 1.5 times the interquartile range or beyond a z-score of 3. Southampton’s guide stresses that you should decide whether to keep an outlier by asking why it exists.
• Multicollinearity: in multiple regression, predictors that closely track each other blur their separate effects. The variance inflation factor (VIF) and a correlation matrix both reveal it.
• Minimum sizes: some tests carry floors. NHS Fife notes at least seven pairs for Spearman’s correlation and at least ten discordant pairs for McNemar’s test.

Sample size deserves attention before data collection, not after. Scribbr stresses that a test is only valid when the sample is large enough to approximate the population. A dissertation sample size calculator helps you plan the numbers while you can still change your design.

Parametric vs Non-Parametric Tests: How to Decide

Parametric tests, such as t-tests, ANOVA and Pearson’s correlation, are more powerful when their assumptions hold. Rank-based tests, such as Mann-Whitney U, Kruskal-Wallis and Spearman’s correlation, carry fewer assumptions and cope better with outliers. The price is power. Stats and R points out that when data really is normal, a non-parametric test is less likely to detect a real difference, so the parametric version is better whenever its assumptions are met.

The NHS Fife guide shows why the choice matters. In its teaching example, researchers compared blood loss between two surgical techniques across 319 operations. A t-test on the skewed data gave p = 0.108, while a Mann-Whitney U test gave p = 0.002, which reversed the conclusion. Both tests ran without error, so only the assumption check would have exposed the mismatch.

Three points keep the decision honest:

• Transformation is an option. NHS Fife notes that if you want to estimate how much two group means differ, with a confidence interval, you can transform skewed data (for example with a logarithm) and keep the parametric test. Southampton lists log, square root and Box-Cox transformations for positive skew.
• Ranks change the question. Sheffield’s guide explains that Mann-Whitney compares mean ranks by default, and it compares medians only if the two distributions share the same shape. Report it that way.
• Switching is not a cure-all. A rank-based test has its own conditions, so name the ones you checked.

Five Worked Examples: From Research Question to Test

These scenarios are invented for illustration, so use them to practise the reasoning rather than as a model for your own data.

Confidence Before and After a Simulation Session

A nursing student measures one cohort’s confidence scores before and after a clinical simulation. The same people appear twice, so the data is paired. Calculate each student’s change, then check whether those differences look roughly normal. If they do, use a paired t-test, which also gives a confidence interval for the mean change. If they are skewed or the scores are ordinal, use the Wilcoxon signed-rank test.

Satisfaction Across Three Store Formats

A business student compares customer satisfaction, rated on a single 1 to 5 item, across three independent store formats. A single ordinal item suits the Kruskal-Wallis test, followed by pairwise comparisons with a multiple-testing correction such as Bonferroni. If satisfaction were a summed score across several questions, one-way ANOVA becomes a defensible alternative once the residuals look reasonable.

Study Hours and Exam Marks

A psychology student wants to know whether weekly study hours relate to exam marks. Draw the scatterplot first. A straight-line pattern with no extreme points supports Pearson’s correlation, while a curved, skewed or outlier-heavy pattern favours Spearman’s. If the real aim is to predict marks from hours, attendance and prior grades, move to multiple regression and check the residuals.

Study Mode and Library Use

A public health student asks whether full-time and part-time students differ in whether they used the campus library (yes or no). Both variables are categorical, so a chi-square test of association fits. Enter counts, not percentages, then check the expected cell counts. If several fall below 5, switch to Fisher’s exact test for a 2 by 2 table or merge categories sensibly.

Predicting a First-Sit Pass

An education student wants to predict whether students pass a module at the first attempt, using attendance and prior marks. The outcome is binary, so binary logistic regression applies. Its output expresses each predictor as an odds ratio, which is the figure the results chapter should interpret.

Real data rarely fits any scenario this neatly. When your design mixes repeated measures, small groups and ordinal items, a second opinion on the choice saves a lot of rework, and some students look for methodology support for the analysis plan at that stage. Any help you use should serve as reference and learning material, and you should use it in line with your university’s academic integrity policy.

Common Mistakes When Choosing a Statistical Test (and the Fixes)

• Picking the test before the question. Write the research question and variable types first, then look for the test.
• Checking normality on the wrong thing. Test the differences for a paired t-test and the residuals for ANOVA and regression.
• Ignoring pairing. A before-and-after design analysed as two independent groups wastes information and misstates the uncertainty.
• Running many t-tests instead of one ANOVA. Sheffield’s guide explains that each extra test raises the chance of a false positive. Use ANOVA or Kruskal-Wallis first, then adjusted post hoc comparisons such as Tukey or Bonferroni.
• Entering percentages into a chi-square test. The test compares observed and expected counts, so use the counts.
• Trying tests until one is significant. The American Statistical Association’s 2016 statement on p-values warns against cherry-picking results, a practice it calls data dredging or p-hacking. Decide on the test before you see the output.
• Treating p less than 0.05 as proof. The same statement lists six principles. Two matter here: a p-value is not the probability that your hypothesis is true, and it does not measure the size or importance of an effect.

How to Justify Your Test Choice in the Methodology and Results Chapters

Markers want to see your reasoning, not just a test name. In the methodology, tie each test to a specific research question, state the variable types and design, and say how you checked the assumptions. The guide to writing a dissertation methodology shows where this reasoning sits in the chapter. A sentence along these lines works: “An independent-samples t-test compared mean exam scores between the two teaching groups, after the distribution of scores in each group was checked with histograms and a Shapiro-Wilk test.” Adapt the wording to your own data and checks.

Write each hypothesis in testable form too. The hypothesis builder helps you phrase the null and alternative so they match the test you chose.

In the results, report more than significance. Give descriptive statistics, the test statistic, degrees of freedom where they apply, the exact p-value, an effect size and a confidence interval. Sheffield’s tutors cite Cohen’s benchmarks of about 0.2, 0.5 and 0.8 for small, medium and large standardised differences, but add that such labels are not always meaningful, so interpret them against your field. The walk-through on writing the results chapter covers layout, and students who want a second pair of eyes on the finished tables can look at results chapter support.

Software and Free Resources That Help You Choose

Software runs a test but does not pick one. In SPSS, the Sheffield guide places the independent-samples t-test under Analyse, Compare Means, and puts the rank-based tests under Analyse, Nonparametric Tests. R users get Welch’s t-test from t.test() by default and can run shapiro.test() for a quick Shapiro-Wilk check.

Several UK universities publish free decision aids. Leeds and Southampton both point students to the Sage Research Methods “Which Stats Test” tool, Sheffield’s statstutor project hosts tutorials with SPSS instructions, and Lincoln’s MASH team publishes a test-choosing flowchart and table. Many universities also offer one-to-one statistics appointments, so use them before submission rather than after feedback.

Questions Students Ask About Choosing a Test

What if my data is not normal and my sample is small?

Small samples make normality hard to judge, because formal tests lack the power to detect problems. NHS Fife advises rank-based methods when the distribution is unknown or you have reason to suspect non-normality. If you need a mean difference with a confidence interval, consider a transformation instead, and explain the decision in your methodology.

Can I use a t-test on Likert data?

A single five-point item is ordinal, so Mann-Whitney or Kruskal-Wallis is the cautious route. Summed or averaged scores from a multi-item scale are often analysed as scale data. Ask your supervisor which convention your department follows, then justify the choice in writing.

Do I need Levene’s test before every t-test?

Not necessarily. SPSS reports both versions of the t-test, and some statisticians recommend using Welch’s version by default instead of pre-testing variances. Others urge caution with skewed data and unequal group sizes, so follow your department’s guidance and report which row or version you used.

Can one dissertation use several different tests?

Yes, and most do, because each research question may need its own test. Plan the full set before you collect data and write down which test answers which question. That plan also protects you from the p-hacking the ASA statement describes.

Still working out what to test? Turn a rough idea into a clear question with the research question generator, then come back to the five questions at the top and work through them in order.