Skip to main content

CBE NEXUS · UNDERGRADUATE PSYCHOLOGY

Chapter 11: Statistics and Data Analysis for Psychology

A practical university-level guide to organising data, describing distributions, estimating uncertainty, testing hypotheses and interpreting psychological evidence responsibly.

Undergraduate level · Psychology · Chapter textbook notes

← Psychology course guide

Learning outcomes

Chapter contents

  1. The role of statistics in psychological science
  2. Types of data, variables and measurement scales
  3. Data coding, screening and preparation
  4. Frequency distributions, tables and graphical displays
  5. Measures of central tendency
  6. Measures of variability and distribution shape
  7. The normal distribution and standard scores
  8. Probability, sampling distributions and standard error
  9. Hypothesis testing and p-values
  10. Confidence intervals, effect sizes and practical significance
  11. Comparing two groups: t-tests and alternatives
  12. Analysis of variance and factorial designs
  13. Correlation and simple linear regression
  14. Categorical data: chi-square tests and association measures
  15. Non-parametric methods and robust analysis
  16. Missing data, attrition and imputation
  17. Multiple comparisons, model selection and analytical flexibility
  18. Effect estimation, causal models and interpretation
  19. Reporting statistical results and reproducible workflows
  20. Integrated worked examples for psychology students
  21. Choosing an analysis plan and avoiding common errors
  22. Statistical literacy in clinical, educational and organisational psychology
  23. Statistical reasoning diagrams
  24. Examination questions and model answers
Important: Statistical software produces calculations, not automatic scientific conclusions. Always check the research question, measurement, sampling, assumptions and practical meaning of the result.

1. The role of statistics in psychological science

Statistics provides a disciplined way to organise, summarise and interpret observations. Psychological data arise from people, groups, tasks, questionnaires, interviews coded into categories, physiological recordings and administrative records. These observations vary because people differ, conditions differ and measurements contain error. Statistical reasoning helps distinguish patterns that are likely to be meaningful from fluctuations that could arise through sampling variation. It does not replace theory, sound measurement, ethical practice or careful research design. A sophisticated analysis cannot repair a biased sample, an invalid measure or a poorly defined research question.

Descriptive statistics summarise the data actually observed. They include counts, proportions, measures of central tendency, measures of variability, tables and graphs. Inferential statistics use sample information to estimate population quantities, evaluate hypotheses or quantify uncertainty. The distinction is important: a sample mean is a description of the sample, while a confidence interval around that mean expresses uncertainty about a population parameter under specified assumptions. A p-value is not the probability that a theory is true, and statistical significance does not by itself establish practical importance or causation.

The unit of analysis must be clear before analysis begins. It may be an individual participant, a classroom, a family, a clinical session or a repeated observation within a person. If multiple observations from one person are treated as independent participants, standard errors may be too small and conclusions misleading. Researchers should define the population, sample, variables, design and estimand—the quantity they intend to estimate—before selecting a statistical test. The analysis should follow the research question and data structure rather than choosing a test solely because it is familiar.

2. Types of data, variables and measurement scales

A dataset is a structured collection of observations, usually arranged so that rows represent units and columns represent variables. Each variable needs a clear name, definition, unit, allowable values and coding rule. Categorical variables describe membership in groups; numerical variables represent counts or measured quantities. Discrete numerical variables take separate countable values, while continuous variables can in principle take any value within a range. A variable's appearance in a spreadsheet does not determine its statistical type: a numerical code such as 1, 2 or 3 may simply represent categories.

Nominal variables have categories without natural order, such as programme of study or type of treatment. Ordinal variables have a meaningful order, such as levels of symptom severity, but the distance between adjacent categories may not be equal. Interval variables have equal units but no meaningful absolute zero; ratio variables have equal units and a meaningful zero, as with duration or number of errors. The scale influences whether arithmetic operations and particular summaries are interpretable. Researchers should also consider whether a multi-item scale has been validated as a composite score before treating it as approximately continuous.

Independent variables, dependent variables, covariates, grouping variables and outcomes should be labelled according to the research question. A confounder may distort an association; a mediator may lie on a causal pathway; a moderator may alter the size or direction of an association. These conceptual roles should be specified using substantive reasoning or a causal diagram. Automatically adjusting for every available variable can introduce bias, particularly if a variable is affected by the exposure or is a common consequence of two other variables. Analysis plans should explain why each covariate is included.

3. Data coding, screening and preparation

Data preparation begins with a codebook describing variable names, labels, units, categories, scoring rules and missing-value codes. Coding should preserve the distinction between a genuine zero, a response of ‘none’, a skipped item and a value that was not recorded. Special missing codes should never be allowed to enter calculations as if they were real measurements. Dates, units and category labels should be standardised consistently, while retaining the original data in a protected raw copy. Any transformation should be reproducible and documented.

Screening checks include duplicate records, impossible values, inconsistent categories, unexpected ranges, outliers, data-entry errors and patterns of missingness. A score of 450 on a questionnaire whose maximum is 45 may indicate a misplaced decimal or coding error. The researcher should consult source records where permitted and avoid silently changing a value because it looks unusual. Outliers can be genuine observations that reveal an important subgroup or unusual response. Their influence should be investigated through plots, robust summaries and sensitivity analyses rather than automatic deletion.

Data cleaning decisions should be made without reference to whether they produce a preferred result. Researchers should record the number of records excluded and the reason for each exclusion, define duplicate-resolution rules and maintain a clear audit trail. Derived variables should be calculated from documented source fields using reproducible syntax or formulas. A final quality check should confirm that participant identifiers are unique where required, group labels match the protocol, scale scores follow the scoring manual and the analysis dataset contains only the fields needed for the approved purpose.

4. Frequency distributions, tables and graphical displays

A frequency distribution reports how often each category or value occurs. Frequencies can be expressed as counts or percentages, with denominators stated clearly. Percentages may be calculated among all enrolled participants, among those who answered an item or among a subgroup; these denominators are not interchangeable. A cumulative frequency adds counts up to a specified category and is useful for ordered data. For continuous data, values may be grouped into intervals, but the interval width and boundary choices should be stated because they affect the visual impression.

Bar charts display counts or proportions for categorical groups, with spaces between bars. Histograms display the distribution of numerical data using adjoining intervals. A line graph is often suitable for measurements across ordered time points, while a scatterplot displays the relationship between two numerical variables. Box plots summarise median, quartiles and potential extreme observations, but they do not show every feature of a distribution. Pie charts can be difficult to compare when categories are numerous or proportions are similar; a labelled bar chart is often clearer.

Graphs should have informative titles, labelled axes, units, readable scales and a note explaining the sample or denominator where necessary. Truncated axes can exaggerate differences; unequal interval widths can mislead; three-dimensional effects can distort perception. When comparing groups, use consistent scales and distinguish missing values from zero values. A graph should make the pattern visible without hiding uncertainty or important subgroups. Researchers should choose the display that best matches the variable type and question rather than decorating every result with multiple chart styles.

5. Measures of central tendency

The arithmetic mean is the sum of observations divided by the number of observations. It uses every value and is useful for approximately symmetric numerical distributions without extreme influential observations. However, it can be pulled toward unusually high or low values. The median is the middle ordered value, or the average of the two middle values for an even number of observations. It is resistant to extreme values and is often more representative for skewed distributions such as reaction times or income. The mode is the most frequent value or category and is especially useful for nominal data.

For the observations 4, 5, 5, 6 and 20, the mean is 40 divided by 5, which equals 8. The median is 5 and the mode is also 5. The large value 20 raises the mean, demonstrating why a mean alone may not describe a skewed sample well. Researchers should report the mean with a suitable measure of spread when it is informative, and the median with an interquartile range when the distribution is skewed or ordinal. The choice should be guided by the distribution and the meaning of the measurement scale.

A distribution can have more than one mode, particularly when it contains distinct subgroups. A bimodal pattern may indicate two populations or conditions that should be explored. The mean and median can coincide in a symmetric distribution, but their agreement does not prove normality. Central tendency should be interpreted alongside a graph, variability, sample size and context. Reporting a single average can conceal meaningful differences by age, setting, treatment condition or other pre-specified groups.

6. Measures of variability and distribution shape

Range is the difference between the maximum and minimum values. It is easy to calculate but depends only on two observations. The interquartile range is the difference between the third and first quartiles and describes the spread of the middle half of observations. Variance is the average squared deviation from the mean, with sample variance using a denominator of n minus 1. Standard deviation is the square root of variance and is expressed in the original measurement units. It describes typical dispersion around the mean when the mean is an appropriate centre.

A distribution may be approximately symmetric, positively skewed, negatively skewed, uniform, unimodal or multimodal. Positive skew has a longer right tail, often with the mean greater than the median; negative skew has a longer left tail. Kurtosis describes aspects of tail weight and concentration relative to a reference distribution, but it should not be used as a substitute for visual inspection. Psychological variables may have floor or ceiling effects when many participants score near the minimum or maximum possible value. Such effects can limit sensitivity to change and affect model assumptions.

The standard deviation is not the standard error. Standard deviation describes variation among individual observations; standard error describes the sampling variability of an estimator, such as a sample mean. For a simple random sample, the standard error of the mean is the sample standard deviation divided by the square root of the sample size. Increasing sample size generally reduces the standard error even if the population variability remains unchanged. Reports should identify which quantity is shown and avoid using the vague phrase ‘error bars’ without explanation.

7. The normal distribution and standard scores

The normal distribution is a symmetric, bell-shaped probability model defined by its mean and standard deviation. In an ideal normal distribution, approximately 68% of values lie within one standard deviation of the mean, about 95% within two and about 99.7% within three. These are approximate reference proportions, not rules that every psychological variable must obey. Real data may be skewed, bounded, multimodal or affected by selection. Normality assumptions usually concern the model's errors or sampling distribution, not necessarily every raw variable.

A z-score expresses how far an observation lies from the mean in standard deviation units: z equals the observed score minus the mean, divided by the standard deviation. A score of 70 with a mean of 60 and standard deviation of 5 has z = (70 − 60) ÷ 5 = 2. It is two standard deviations above the mean. Standard scores allow comparison across measures with different units, provided the reference distributions and interpretation are appropriate. A z-score does not by itself indicate clinical importance or percentile rank without distributional assumptions.

Normal probability plots, histograms and residual plots can help assess distributional assumptions. Formal normality tests may be sensitive to sample size: with large samples, small deviations can be statistically significant; with small samples, important deviations may go undetected. Researchers should evaluate the practical impact of departures and use methods appropriate to the design, such as robust procedures, transformations or suitable non-parametric tests. Transformations change the scale of interpretation and should be justified and reported transparently.

8. Probability, sampling distributions and standard error

Probability quantifies uncertainty under a specified model. A sampling distribution describes the values an estimator would take across repeated samples drawn under the same conditions. The standard error is the standard deviation of that sampling distribution. It is not the spread of individual scores. The central limit theorem explains why, under broad conditions, the sampling distribution of a mean becomes approximately normal as sample size grows, even when the population distribution is not normal. The conditions and dependence structure still matter; clustered or heavily dependent observations require different treatment.

For a sample mean, a simple standard error estimate is s divided by the square root of n, where s is the sample standard deviation and n is the number of independent observations. If s is 10 and n is 25, the standard error is 10 ÷ 5 = 2. This calculation assumes an appropriate independent-sampling structure. If the 25 observations are repeated measurements from only five people, treating them as 25 independent participants would underestimate uncertainty. The effective information depends on the number and structure of independent units.

Probability statements are conditional on assumptions about the sampling process and model. A confidence interval procedure with 95% coverage would, over repeated samples under its assumptions, produce intervals containing the true parameter in about 95% of repetitions. For a particular calculated interval, the parameter is fixed in the frequentist framework; the interval either contains it or does not. The common interpretation that there is a 95% probability that the fixed parameter lies in this one interval is not the standard frequentist meaning. Bayesian credible intervals have a different probability interpretation and depend on a prior and likelihood.

9. Hypothesis testing and p-values

A statistical hypothesis test evaluates how compatible the observed data are with a specified null model, using a test statistic and reference distribution. The null hypothesis may state that a population difference or association equals zero, while an alternative hypothesis describes a non-zero or directional effect. The significance level alpha is a pre-specified long-run error threshold, often 0.05, not a boundary between truth and falsehood. The p-value is the probability, assuming the null model and its assumptions, of obtaining a result at least as extreme as the observed result under the test's definition.

A small p-value indicates that the observed data would be relatively unusual under the null model. It does not give the probability that the null hypothesis is true, the probability that the result occurred ‘by chance’, or the size of the effect. A large p-value does not prove there is no effect; the study may be imprecise or underpowered. Researchers should report exact p-values where suitable, alongside effect estimates, confidence intervals, sample size, design and assumptions. Avoid statements such as ‘p = 0.000’; report p below the software's justified precision instead.

A Type I error occurs when a true null hypothesis is rejected; a Type II error occurs when a false null hypothesis is not rejected. Statistical power is the probability of rejecting the null under a specified alternative effect and model. Power depends on sample size, effect magnitude, variability, significance threshold and design. Multiple testing increases the chance of at least one false positive if no correction or coherent analysis strategy is used. Pre-specifying a primary outcome, limiting exploratory tests and transparently labelling exploratory findings help preserve interpretability.

10. Confidence intervals, effect sizes and practical significance

An effect size quantifies the magnitude of a difference or association. It may be expressed in the original units, as a standardised mean difference, a correlation, an odds ratio, a risk ratio or another measure suited to the outcome and design. The original-unit estimate is often the most interpretable for practice. Standardised effects can aid comparison across measures but depend on the standard deviation used and may be difficult to translate into real-world consequences. Researchers should select an effect measure that answers the substantive question.

A confidence interval communicates precision around an estimate under the model's assumptions. A narrow interval indicates greater precision than a wide interval, but precision does not guarantee lack of bias. An interval that includes zero for a difference may still include effects that matter in practice; an interval excluding zero may contain effects too small to matter. Interpretation should consider the full range of plausible values, not merely whether the interval crosses a conventional null value. Clinical or educational importance may require a pre-defined meaningful threshold.

Statistical significance and practical significance are distinct. A very large sample can detect a tiny difference that has little consequence, while a small study may fail to detect a meaningful effect because its estimate is imprecise. Researchers should discuss magnitude, uncertainty, cost, burden, possible harms, feasibility and the values of affected people. Where appropriate, compare estimates with a minimum clinically or educationally important difference established independently of the observed data.

11. Comparing two groups: t-tests and alternatives

An independent-samples t-test compares the means of two independent groups on a numerical outcome. It assumes an appropriate sampling structure, independent observations, and a model for the outcome or residuals; the classical pooled version also assumes equal population variances. Welch's t-test does not require equal variances and is often a sensible default when group variances or sizes differ. A paired-samples t-test analyses within-person differences for matched pairs or repeated measurements, such as scores before and after an intervention in the same participants.

The test statistic expresses the observed mean difference relative to its estimated standard error. Suppose an intervention group has a mean score of 74 and a comparison group has a mean of 68; the observed difference is 6 points. Whether this difference is estimated precisely depends on within-group variability and sample size. The means alone are insufficient to determine a p-value or confidence interval. Researchers must report group sizes, means, standard deviations, the estimated difference, interval and test details.

When assumptions are seriously unsuitable or the outcome is ordinal, non-parametric alternatives may be considered. The Mann–Whitney test compares rank distributions and is not simply a universal test of medians; interpretation depends on distributional shapes. The Wilcoxon signed-rank test analyses paired differences under its assumptions. Permutation or bootstrap methods can be useful when appropriately designed. The choice should reflect the estimand and data structure, not a mechanical normality-test decision.

12. Analysis of variance and factorial designs

Analysis of variance (ANOVA) compares means across two or more groups by partitioning variability into components associated with factors and residual variation. A one-way ANOVA evaluates a categorical factor with multiple levels. A significant omnibus result indicates evidence that not all population means are equal under the model, but it does not identify which groups differ. Planned contrasts or suitable post-hoc comparisons address specific differences while controlling the relevant multiplicity.

Factorial ANOVA examines two or more categorical factors and can test main effects and interactions. A main effect describes the average association of one factor across levels of another; an interaction means the effect of one factor differs across levels of the other. For example, a study may test teaching method and feedback type. If the benefit of a teaching method depends on feedback type, reporting only the overall teaching-method mean may conceal the pattern. Interaction plots and simple-effects estimates help explain the result.

Repeated-measures ANOVA analyses outcomes measured under multiple conditions or time points for the same participants, but it requires attention to within-person dependence and assumptions such as sphericity for certain designs. Mixed-effects models provide a flexible alternative for repeated and clustered data. Researchers should report effect estimates, degrees of freedom, assumptions, follow-up comparisons and the handling of missing observations. An omnibus F statistic should not replace clear descriptions of group means and the pattern of differences.

13. Correlation and simple linear regression

Correlation describes the direction and strength of association between two variables. Pearson's correlation measures linear association for numerical variables and is sensitive to outliers and restricted range. Spearman's rank correlation measures monotonic association using ranks and can be suitable for ordinal data or relationships that are monotonic but not linear. A scatterplot should be inspected before reporting a coefficient because a single number can hide clusters, nonlinearity or influential observations.

Simple linear regression models an outcome as a linear function of a predictor: predicted Y equals an intercept plus a slope multiplied by X. The slope estimates the expected change in the outcome for a one-unit increase in the predictor under the model. If a regression slope is 2.5 points per additional hour of practice, the model estimates an average increase of 2.5 points for each extra hour in the observed range. This is an association unless the design and assumptions support causal interpretation.

Regression assumptions include an appropriate functional form, independent errors conditional on predictors, constant residual variance for conventional inference and suitable treatment of influential observations. Residual plots help diagnose patterns. Extrapolating beyond the observed range is risky. The coefficient of determination, R-squared, describes the proportion of sample variation in the outcome accounted for by the fitted model in the specified context; it does not indicate causation or prove that the model is correct. Prediction intervals and uncertainty should be considered when using a model for individual predictions.

14. Categorical data: chi-square tests and association measures

The chi-square test of independence examines whether two categorical variables are associated in a contingency table. It compares observed cell counts with expected counts under the assumption of independence. Expected counts are determined by row and column totals. The test requires independent observations and adequate expected frequencies for the usual approximation. When tables are sparse, exact methods or justified alternatives may be preferable. The test indicates evidence of association but does not identify its direction or practical strength without examining proportions and an effect measure.

A chi-square goodness-of-fit test compares observed category counts with a specified distribution. For example, a researcher may test whether preferences across three counselling formats match a pre-specified equal distribution. The expected counts must reflect the stated null model. Researchers should report the table, sample size, test statistic, degrees of freedom, p-value and an appropriate effect-size measure such as phi or Cramér's V where relevant. Residual analysis may identify cells contributing strongly to an overall association, with multiplicity considered.

For a two-by-two table, risk ratios and odds ratios may quantify associations depending on the sampling design and outcome frequency. An odds ratio compares the odds of an outcome between groups; it should not automatically be described as a risk ratio, particularly when outcomes are common. Cross-sectional and case-control sampling schemes affect which quantities can be estimated directly. Clear labels and denominators are essential because a percentage calculated across rows answers a different question from a percentage calculated down columns.

15. Non-parametric methods and robust analysis

Non-parametric methods make fewer assumptions about a specific population distribution, but they are not assumption-free. Rank-based procedures may be suitable for ordinal outcomes or highly non-normal data, depending on the research question. They can be less sensitive than parametric methods when parametric assumptions are reasonably met, and their estimands may differ. Researchers should explain what the chosen method compares rather than describing every rank test as a test of medians.

Robust statistics reduce sensitivity to extreme observations or departures from model assumptions. Examples include medians, trimmed means, robust regression and bootstrap confidence intervals. Bootstrapping resamples observations from the sample to approximate an estimator's sampling distribution; it requires a resampling scheme that respects dependence, such as resampling clusters when data are clustered. Robust methods do not correct selection bias, invalid measurement or a fundamentally inappropriate design. They address particular statistical vulnerabilities, not every source of uncertainty.

Sensitivity analysis examines whether conclusions change under reasonable alternative decisions or assumptions. Researchers might compare a primary analysis with and without a pre-defined influential observation, alternative missing-data assumptions, a different but defensible scoring rule or a robust estimator. The primary analysis should remain clear, and sensitivity results should be reported without selecting only the version that supports a preferred narrative. Large discrepancies signal that conclusions are fragile and should be described accordingly.

16. Missing data, attrition and imputation

Missing data occur when a variable is not observed for some units. Missing completely at random means missingness is unrelated to observed or unobserved data under the formal model; missing at random means missingness can be explained by observed information; missing not at random means missingness depends on unobserved values even after conditioning on observed data. These mechanisms cannot usually be established from a simple test. Researchers should investigate patterns, reasons and differences between participants with and without observed outcomes.

Complete-case analysis excludes records with missing values on variables required for a model. It can waste information and may be biased unless demanding assumptions hold. Single imputation, such as replacing every missing score with the mean, generally understates uncertainty and can distort relationships. Multiple imputation creates several plausible completed datasets, analyses each and combines estimates to reflect within- and between-imputation uncertainty. Its validity depends on a suitable imputation model, observed predictors and assumptions about missingness.

Attrition in longitudinal or intervention studies can create bias when loss to follow-up is related to condition, outcome or participant characteristics. Report numbers and reasons for missing follow-up by group where appropriate. Prevention is preferable: reduce burden, maintain contact ethically, make assessments accessible and plan realistic follow-up. Sensitivity analyses can examine how conclusions change under plausible departures from the primary missing-data assumptions. No statistical technique can guarantee recovery of information that was never observed without assumptions.

17. Multiple comparisons, model selection and analytical flexibility

When many hypotheses are tested, the probability of at least one false positive can rise. Family-wise error procedures such as Bonferroni adjustment control the probability of any false rejection within a defined family, although they can reduce power. False discovery rate procedures control the expected proportion of false discoveries among rejected hypotheses under specified conditions. The appropriate method depends on whether the goal is confirmatory control, exploratory screening or prioritisation for further study. The family of tests should be defined transparently.

Analytical flexibility includes choices about outcome scoring, exclusion rules, covariates, transformations, subgroup definitions, time points and statistical models. If these decisions are made after viewing results and only favourable analyses are reported, the nominal p-values no longer describe the full search process. A pre-specified primary analysis, preregistration, complete reporting and clear separation of exploratory work reduce this risk. Exploratory analysis is valuable for discovery when labelled honestly and followed by independent confirmation.

Model selection should be guided by theory, design and predictive or explanatory purpose. Stepwise procedures based only on p-values can produce unstable models and biased coefficient estimates. Overfitting occurs when a model captures random features of the development sample and performs poorly on new data. Cross-validation or external validation can assess predictive performance, but the validation process must be separated from model development. Researchers should report all important model-building decisions and avoid implying that a selected model is uniquely true.

18. Effect estimation, causal models and interpretation

An effect estimate answers a defined question, such as the mean difference under two conditions, the association between two measures or the change in an outcome over time. The estimand should specify the population, conditions, outcome, handling of intercurrent events and summary measure. In causal inference, the target may be an average treatment effect or an effect among a defined subgroup. Different estimands can produce different answers even when based on the same dataset, so the question must be explicit.

Causal diagrams can help represent assumptions about exposure, outcome, confounders, mediators and selection. They clarify why a variable is adjusted for or excluded. Statistical adjustment is not automatically beneficial: conditioning on a collider or controlling for a mediator when estimating a total effect can distort the estimate. Causal claims require assumptions about temporal order, consistency, positivity, exchangeability and measurement, among others. These assumptions should be considered substantively and acknowledged, not hidden behind a regression table.

Interpretation should integrate magnitude, uncertainty, design quality, measurement validity, generalisability and prior evidence. A statistically significant association from a convenience sample may be less persuasive than a precise estimate from a well-designed representative study, depending on the question. A null result may narrow the range of plausible effects without proving exact absence. Researchers should explain which conclusions are supported, which are tentative and what future data would most improve confidence.

19. Reporting statistical results and reproducible workflows

A statistical report should identify the analysis population, outcome definitions, group sizes, summary statistics, model or test, effect estimates, uncertainty intervals and relevant assumptions. Tables should include clear labels, units and denominators. Avoid reporting only p-values or software output. For regression, report coefficients and their uncertainty, the model specification and any relevant diagnostics. For categorical comparisons, show counts and proportions. For longitudinal studies, explain the time structure and how within-person dependence was addressed.

Reproducible workflows use documented scripts, version-controlled files, clear data dictionaries and a record of software and package versions. Raw data should remain unchanged; cleaning and analysis should be performed through traceable steps. A code review or independent reproduction can identify errors. Sharing code and de-identified data can support verification when consent, law, ethics and participant privacy permit. Open science does not justify exposing sensitive records or breaching confidentiality.

The final discussion should distinguish planned from exploratory analyses, describe missing data and exclusions, acknowledge limitations and avoid causal language unsupported by the design. Statistical results should be communicated in plain language for students, practitioners, participants and policy audiences. Visualisations should not exaggerate differences. Corrections should be issued if an error materially affects the report. Reproducibility is a property of the whole research process—from question and data collection to code, interpretation and transparent communication—not merely a file containing statistical syntax.

20. Integrated worked examples for psychology students

Example 1: A researcher measures concentration scores in 25 students. The sample mean is 60 and the sample standard deviation is 10. The standard error of the mean is 10 divided by the square root of 25, or 2. This describes estimated sampling variability under an independent-sampling model. It does not mean that individual students typically score within two points of 60; individual variability is described by the standard deviation of 10.

Example 2: A participant scores 72 on a test with a reference mean of 60 and standard deviation of 6. The z-score is (72 − 60) ÷ 6 = 2. The score is two standard deviations above the reference mean. If the reference distribution is approximately normal, this corresponds to a high relative position, but interpretation requires a suitable norm group, reliable measurement and evidence that the test is appropriate for the person and purpose.

Example 3: Two independent groups complete a memory task. The intervention group mean is 18 correct answers and the comparison group mean is 15. The mean difference is 3 answers. To judge uncertainty, the researcher also needs group sizes and standard deviations, and should choose a test consistent with the design. If participants were measured before and after the intervention, the dependence of repeated scores must be handled rather than treating all measurements as independent. The report should interpret the size of three additional correct answers in educational or practical context.

Example 4: A survey finds that 30 of 50 students who report high study support pass an assessment, compared with 18 of 50 students who report low support. The pass proportions are 60% and 36%, respectively. The difference is 24 percentage points. This descriptive contrast does not establish that support caused passing; prior achievement, resources and selection may influence both. A contingency table and an appropriate association estimate can quantify the pattern, while design and confounding considerations shape the interpretation.

21. Choosing an analysis plan and avoiding common errors

An analysis plan begins with the research question and primary outcome. It specifies the unit of analysis, measurement scale, target population, estimand, descriptive summaries, primary model, covariates, assumption checks, missing-data strategy and sensitivity analyses. It should also define subgroup analyses and multiplicity handling. For a qualitative or mixed-methods study, the plan describes coding, theme development, integration and how contradictory evidence will be examined. A plan should be sufficiently specific to constrain selective reporting while allowing justified adaptations.

Common errors include treating ordinal categories as precise continuous measurements without justification, confusing standard deviation with standard error, interpreting a p-value as the probability that a hypothesis is true, using correlation to claim causation, ignoring clustering, deleting outliers without a rule, and reporting percentages without denominators. Another error is choosing a test from a flowchart without checking whether its assumptions and estimand match the design. Software can calculate a result accurately while the analysis remains conceptually wrong.

A responsible analyst asks: What does each row represent? How were participants selected? What does the measure mean? What comparison answers the question? What assumptions are required? How uncertain is the estimate? Could bias explain the pattern? Would a reasonable alternative analysis change the conclusion? The answers should appear in the methods and discussion, not remain private knowledge of the analyst. Statistical literacy is the ability to reason about evidence, not merely operate a package.

22. Statistical literacy in clinical, educational and organisational psychology

Clinical psychologists use statistics to evaluate assessment properties, symptom change, treatment outcomes and service quality. They must distinguish reliable change from measurement noise, interpret norms cautiously and avoid equating a group average with an individual prognosis. Treatment studies should report adverse events, attrition and meaningful outcome differences. Evidence from trials should be integrated with patient preferences, clinical expertise and context rather than treated as an automatic instruction for every individual.

Educational psychologists may analyse test scores, attendance, classroom observations and intervention outcomes. Students are often nested within classes and schools, so clustered sampling and multilevel structures matter. Growth over time may require repeated-measures or mixed-effects models. Comparisons should account for baseline differences and avoid interpreting a difference in scores without considering curriculum, language, disability access and measurement fairness. Aggregate reporting must protect learners from identification or labelling.

Organisational psychologists analyse employee surveys, performance indicators, turnover and team outcomes. Responses may be nested within departments and organisations, and employees may fear that managers can identify them. Analysts should aggregate only where group sizes and confidentiality rules permit, explain response rates and avoid causal claims from cross-sectional correlations. Across all fields, statistical results should support humane, fair and evidence-informed decisions, not replace professional judgement or the voices of affected people.

Statistical reasoning diagrams

Figure 11.1: From raw observations to interpretation

Raw data
→
Screen & code
→
Describe & visualise
→
Model & estimate

↓

Check assumptions
→
Quantify uncertainty
→
Interpret in context

Statistical analysis is iterative. Data checks and model diagnostics may require revisiting earlier decisions.

Figure 11.2: Choosing a summary for the variable

Identify variable type
→
Categorical?

↙         ↘

Counts & proportions
Numerical: mean/SD or median/IQR

The distribution, measurement scale and research question determine which summary is meaningful.

Figure 11.3: Interpreting an estimated effect

Effect estimate
+
Confidence interval
+
Design quality

↓

Practical importance
→
Responsible conclusion

A p-value alone is not a complete interpretation of evidence.

Structured university examination questions with model answers

Use the model answers as marking guides. In an examination, show calculations, state assumptions and interpret results in the context of the research question.

1. Distinguish descriptive and inferential statistics.

Model answer: Descriptive statistics summarise the observed sample using counts, graphs and measures such as means or medians. Inferential statistics use sample data and a model to estimate population quantities or evaluate hypotheses while quantifying uncertainty.

2. A dataset contains participant ID, gender category, age, reaction time and treatment group. Identify likely variable types.

Model answer: ID is an identifier rather than a quantitative measurement; gender category and treatment group are nominal categorical variables; age and reaction time are numerical ratio-scale measures when recorded in meaningful units. Their distributions and roles in the research question still need checking.

3. Calculate the mean and median of 3, 4, 4, 7 and 12.

Model answer: The sum is 30, so the mean is 30 ÷ 5 = 6. The ordered middle value is 4, so the median is 4. The higher value 12 pulls the mean above the median.

4. Explain why a median may be preferable to a mean for highly skewed data.

Model answer: The median is resistant to extreme observations and often better represents the centre of a skewed distribution. The mean uses all values but can be pulled toward a long tail. The distribution and measurement scale should guide the choice.

5. Differentiate standard deviation and standard error.

Model answer: Standard deviation describes spread among individual observations. Standard error describes sampling variability of an estimator, such as the mean. The standard error of a simple sample mean is commonly estimated as s divided by the square root of n.

6. A sample has standard deviation 12 and n=36. Calculate the standard error of the mean.

Model answer: SE = 12 ÷ √36 = 12 ÷ 6 = 2. This assumes the 36 observations provide independent information under the sampling design.

7. A score is 80, the reference mean is 65 and standard deviation is 5. Calculate and interpret its z-score.

Model answer: z = (80 − 65) ÷ 5 = 3. The score is three standard deviations above the reference mean. Interpretation depends on an appropriate reference group and measurement validity.

8. State the approximate proportions of a normal distribution within one, two and three standard deviations of its mean.

Model answer: Approximately 68% lie within one standard deviation, 95% within two and 99.7% within three, assuming an ideal normal distribution.

9. What does a p-value represent?

Model answer: It is the probability, assuming the null model and its assumptions, of obtaining a test statistic at least as extreme as the observed statistic under the specified test. It is not the probability that the null hypothesis is true.

10. Distinguish Type I and Type II errors.

Model answer: A Type I error rejects a true null hypothesis. A Type II error fails to reject a false null hypothesis. The significance threshold controls the nominal Type I error rate under assumptions; power is the probability of rejecting under a specified alternative.

11. Explain why a non-significant result does not prove there is no effect.

Model answer: The study may have low power, imprecise measurement or a wide confidence interval. The data may be compatible with both negligible and meaningful effects. Estimate magnitude and uncertainty rather than treating non-significance as proof of absence.

12. What information does a confidence interval add to a point estimate?

Model answer: It communicates the precision and range of values compatible with the data under the model and procedure. It helps assess whether effects of practical importance remain plausible, although it does not remove bias.

13. A mean intervention gain is 8 points and comparison gain is 3 points. Calculate the difference in gains and interpret it.

Model answer: The difference is 8 − 3 = 5 points. This is the observed contrast in improvement; causal interpretation depends on design, baseline comparability, attrition and uncertainty.

14. When would a paired-samples t-test be appropriate?

Model answer: When the same participants are measured twice or observations are meaningfully matched, and the analysis concerns within-pair differences. The dependence between paired observations must be retained.

15. What does a one-way ANOVA test?

Model answer: It tests whether there is evidence that not all population means are equal across levels of one categorical factor, under the model assumptions. A significant omnibus test does not identify which groups differ.

16. Explain a statistical interaction in a factorial design.

Model answer: An interaction occurs when the effect of one factor differs across levels of another factor. For example, a study method may improve performance only when accompanied by a particular feedback approach.

17. Distinguish Pearson and Spearman correlation.

Model answer: Pearson correlation measures linear association between numerical variables and can be sensitive to outliers. Spearman correlation uses ranks and measures monotonic association, making it useful for ordinal data or monotonic relationships that are not linear.

18. A correlation is r=0.50. What can and cannot be concluded?

Model answer: There is a positive linear association of moderate magnitude in the observed data. It does not establish causation, rule out confounding or mean that one variable causes 50% of the other.

19. What does the slope in a simple linear regression represent?

Model answer: It estimates the expected change in the outcome associated with a one-unit increase in the predictor under the fitted linear model. It is not automatically a causal effect.

20. What is the purpose of a chi-square test of independence?

Model answer: It evaluates whether two categorical variables are associated by comparing observed cell counts with expected counts under independence. Independent observations and suitable expected frequencies are important.

21. Why can mean imputation be problematic?

Model answer: Replacing missing values with the mean understates variability, distorts relationships and fails to reflect uncertainty about missing values. More defensible approaches depend on the missing-data mechanism and may include multiple imputation.

22. Explain the difference between missing at random and missing not at random.

Model answer: Missing at random means missingness can be explained by observed information in the model. Missing not at random means missingness depends on unobserved values even after conditioning on observed data. These assumptions influence appropriate analysis.

23. What is multiple-comparison inflation?

Model answer: Testing many hypotheses can increase the chance of at least one false positive. Researchers can define test families, prioritise primary outcomes and use suitable family-wise or false-discovery-rate procedures.

24. Define effect size and explain why it matters.

Model answer: Effect size quantifies the magnitude of a difference or association. It helps assess substantive importance and compare evidence; it should be reported with uncertainty and interpreted in original units where possible.

25. What is overfitting in regression or prediction?

Model answer: Overfitting occurs when a model captures random features of the development sample and performs poorly on new data. It can be reduced through theory-guided modelling, adequate sample size, regularisation and independent validation.

26. Why should researchers inspect a scatterplot before reporting correlation?

Model answer: A scatterplot can reveal nonlinearity, clusters, outliers, restricted range and influential observations that a single correlation coefficient conceals.

27. Explain why clustering matters when students are sampled from classrooms.

Model answer: Students in the same classroom may share teachers, resources and context, so their outcomes are correlated. Treating them as independent can underestimate standard errors. Analysis should account for the clustered design.

28. What should be reported alongside a p-value?

Model answer: The effect estimate, confidence interval or other uncertainty measure, sample size, group summaries, analysis method, assumptions and relevant design limitations. Practical importance should also be discussed.

29. Describe two ways to improve statistical reproducibility.

Model answer: Use documented scripts and codebooks, preserve raw data, version-control analysis files, record software versions, maintain an audit trail and share code or suitably protected data when ethics and privacy permit.

30. A study reports p<0.05 for a tiny effect in 20,000 participants. Why might the finding have limited practical importance?

Model answer: Large samples can detect very small differences precisely. Statistical significance does not establish meaningful benefit. Interpret the effect magnitude, interval, costs, harms and context.

31. Give a suitable analysis strategy for comparing repeated stress scores in students nested within several schools.

Model answer: Use a repeated-measures approach that accounts for observations within students and students within schools, such as an appropriate multilevel or mixed-effects model. Define the time effect and primary contrast, inspect assumptions, address missing follow-up and report estimates with uncertainty.

Chapter summary

Statistics supports psychological science by describing observations, estimating population quantities and evaluating uncertainty. Correct analysis begins with clear variables, sound data preparation and appropriate visualisation. Central tendency and variability describe distributions; probability and sampling distributions explain uncertainty; hypothesis tests, confidence intervals and effect sizes support inference. Tests such as t-tests, ANOVA, correlation, regression and chi-square answer different questions and require attention to assumptions and design. Missing data, clustering, multiple testing and analytical flexibility can undermine conclusions. Statistical results must always be interpreted in light of measurement quality, sampling, practical importance and ethical responsibility.

End of Chapter 11. Continue with Chapter 12: Clinical Psychology and Evidence-Based Interventions.

Official Education References

Use these official organisations for curriculum guidance, examinations and further learning information.