Chapter 10: Research Methods and Experimental Design
A comprehensive university-level guide to scientific inquiry, quantitative and qualitative methods, sampling, experimental and observational designs, research ethics and transparent reporting.
Study note: Research methods determine what a study can legitimately claim. Always match the strength of the conclusion to the design, sampling process, measurement quality and limitations.
1. The scientific foundations of psychological research
Psychology uses systematic inquiry to describe, explain, predict and, where ethically justified, influence behaviour and mental processes. A scientific approach begins with a question that can be examined through evidence rather than authority, intuition or anecdote. Researchers make their assumptions explicit, define the population and setting, identify what observations would count as evidence, and consider alternative explanations before collecting data. A theory is an organised set of propositions that explains relationships among constructs; it is not simply a guess. A useful theory generates testable predictions, accommodates relevant findings, and can be revised when evidence conflicts with it.
Scientific reasoning combines induction and deduction. Inductive reasoning moves from observations toward patterns and tentative explanations; deductive reasoning derives predictions from an existing theory and tests them. Neither process is infallible. Observations are shaped by measurement choices, sampling and context, while theories may contain assumptions that are difficult to test. Researchers therefore maintain a transparent record of decisions and distinguish observations from interpretations. Replication, criticism and cumulative evidence help correct individual error. In psychology, findings are usually probabilistic: a result may increase the plausibility of an explanation without proving that it is universally true.
A research question should be focused, answerable with available methods, ethically acceptable and important enough to justify the resources required. For example, ‘Does a brief retrieval-practice programme improve delayed recall among first-year university students compared with rereading?’ identifies an intervention, outcome, comparison and population. The question can be refined by specifying the duration of the programme, the recall test and the follow-up interval. Researchers should avoid questions that assume the answer, use vague terms such as ‘better’, or confuse a value judgement with an empirical question. A clear question guides the literature review, design, analysis and interpretation.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
2. Research paradigms, epistemology and methodological choices
A research paradigm is a broad set of assumptions about what can be known, how knowledge is developed and what counts as credible evidence. Positivist and post-positivist traditions commonly emphasise systematic observation, measurement, hypothesis testing and the possibility of estimating patterns while acknowledging uncertainty and measurement error. Interpretivist approaches emphasise meaning, context and the ways people understand their experiences. Critical approaches examine how power, institutions and inequality shape experience and knowledge. These traditions are not interchangeable labels for particular techniques; they influence the purpose of a study, the researcher's role and the standards used to evaluate conclusions.
Quantitative research represents observations numerically and uses descriptive or inferential analysis to estimate patterns, compare groups or examine relationships. It is useful when constructs can be operationalised and when the research question concerns magnitude, frequency, prediction or group differences. Qualitative research collects rich textual, spoken or observational material to understand meanings, processes and context. It is especially valuable when little is known about a phenomenon or when standardised measures fail to capture participants' perspectives. Mixed-methods research intentionally integrates quantitative and qualitative evidence. Integration must occur in the research question, sampling, analysis or interpretation; merely placing a survey and an interview in the same report does not automatically create a coherent mixed-methods study.
Methodological choice should follow the question rather than fashion. A randomised experiment may estimate the effect of an intervention under specified conditions, while interviews may explain how participants experienced that intervention and why implementation differed. A sequential explanatory design may first identify a quantitative pattern and then interview selected participants to interpret it. A convergent design may collect both forms of evidence during the same phase and compare where they agree or diverge. Researchers should state the rationale for integration, identify the priority given to each strand, and explain how conflicting findings will be handled. Pragmatism often supports selecting methods according to the practical problem while retaining explicit standards of rigour.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
3. Developing research questions, aims and literature reviews
A research aim expresses the broad purpose of a study; objectives break that purpose into achievable tasks. Research questions specify what the study will answer, while hypotheses state predictions about relationships or differences. A directional hypothesis predicts the direction of an effect, whereas a non-directional hypothesis predicts a difference or association without specifying its direction. The null hypothesis commonly represents no effect or no association in a defined population or model. Hypotheses should be grounded in theory and previous evidence, not written after inspecting results to make an unexpected finding appear predicted.
A literature review is an analytical synthesis, not a sequence of unrelated summaries. It identifies established findings, disagreements, methodological limitations, theoretical explanations and gaps that justify the new study. A defensible search strategy records databases, keywords, date limits, inclusion criteria and screening decisions. Researchers should evaluate the quality and relevance of sources, distinguish primary studies from commentary, and avoid treating the number of papers as a substitute for their quality. A conceptual framework maps the proposed constructs and relationships. It helps explain why variables are expected to relate and identifies possible mediators, moderators and confounders.
The review should lead logically to the research problem. Suppose several studies report that sleep duration is associated with student concentration, but differ in how sleep is measured and whether workload is considered. The gap may concern the relationship between objectively recorded sleep and sustained attention after accounting for workload. The study should not claim that no research exists merely because the exact local population has not been studied. Instead, it should explain what is known, what remains uncertain, why the uncertainty matters and how the proposed design can reduce it. A focused review also prevents duplication and helps identify validated measures, ethical concerns and realistic sample requirements.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
4. Constructs, variables and operational definitions
A construct is an abstract concept such as anxiety, attention, resilience, motivation or social support. Constructs cannot usually be observed directly; researchers infer them from indicators such as questionnaire responses, task performance, physiological measures or behaviour. A variable is a characteristic that can take different values across people, occasions or conditions. Operationalisation specifies exactly how a construct will be represented in a study. For example, ‘academic engagement’ might be operationalised as attendance, time on task, a validated self-report scale or a combination of indicators. Each operational definition captures only part of the broader construct.
Independent variables are manipulated or used to define comparison conditions; dependent variables are outcomes measured in response to those conditions. Control variables are held constant by design or included in analysis to reduce alternative explanations. A confounding variable is associated with both the exposure or condition and the outcome in a way that can distort an estimated relationship. A mediator may explain a pathway through which an effect occurs, while a moderator changes the strength or direction of a relationship. These roles depend on the causal question and study design; a variable is not inherently a confounder or mediator in every investigation.
Measurement scales affect what can legitimately be concluded. Nominal data classify observations into categories without inherent order, such as study programme. Ordinal data have ordered categories, such as low, moderate and high perceived stress, but intervals between categories are not necessarily equal. Interval scales have meaningful equal intervals but no absolute zero, while ratio scales have equal intervals and a meaningful zero, such as reaction time or number of correct responses. Researchers should not assume that assigning numbers to categories makes them continuous. The measurement plan should specify units, scoring rules, treatment of missing values, timing and evidence that the measure is reliable and valid for the intended population.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
5. Sampling frames, recruitment and sample size
A target population is the group to which researchers hope their conclusions will apply. The accessible population is the portion that can realistically be reached, while the sampling frame is the operational list or procedure from which participants are selected. A sample is the subset that provides data. Probability sampling gives eligible population members a known, non-zero chance of selection and includes simple random, systematic, stratified and cluster methods. Non-probability methods include convenience, purposive, quota, snowball and volunteer sampling. The method should be justified in relation to the question, access, resources and desired generalisability.
Simple random sampling selects individuals through a chance mechanism. Stratified sampling divides the population into relevant strata and samples within each, which can improve representation of important subgroups. Cluster sampling selects naturally occurring groups, such as schools or clinics, and may reduce travel costs, although observations within clusters may be correlated. Purposive sampling selects participants with characteristics or experience relevant to a qualitative question. Snowball sampling can reach hidden or hard-to-access populations but may reproduce network biases. Convenience sampling is often feasible for student projects, yet its limitations should be acknowledged rather than hidden behind broad claims about the population.
Sample size planning depends on the primary outcome, expected effect, variability, design, significance threshold, desired statistical power, attrition and clustering. A larger sample is not automatically better if recruitment is biased or measurements are poor. For quantitative studies, an a priori power analysis or precision-based calculation should be justified using plausible assumptions and, where possible, evidence from prior studies. Qualitative sample adequacy is usually assessed through the depth and relevance of data, diversity of perspectives and analytic purpose rather than a universal numerical formula. Researchers should report how many people were approached, eligible, enrolled, completed follow-up and included in analysis, with reasons for losses where ethically and practically possible.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
6. Experimental designs and causal inference
An experiment deliberately manipulates an independent variable and observes its effect on an outcome while controlling alternative explanations. A basic parallel-group experiment assigns participants to an intervention or comparison condition and measures the outcome after exposure. A pretest-posttest design measures participants before and after the intervention, allowing change to be described, although change alone does not prove causation. A post-test-only design can be appropriate when randomisation is effective and pretesting could influence responses. Factorial designs manipulate two or more independent variables and can estimate both main effects and interactions.
Random assignment allocates participants to conditions by chance. It differs from random sampling: random assignment strengthens causal comparability between experimental groups, whereas random sampling supports population representativeness. In a well-conducted randomised design, assignment helps balance measured and unmeasured baseline characteristics on average, although chance imbalances remain possible in small samples. Allocation concealment prevents recruiters from predicting the next assignment. Blinding reduces the influence of expectations on participants, intervention providers or outcome assessors. Complete blinding is not always possible, especially in behavioural interventions, so researchers should describe who was blinded and what safeguards were used.
A control group provides a comparison for interpreting outcomes. It may receive usual care, no additional intervention, an attention-matched activity or a placebo where appropriate. A placebo should be ethically acceptable and designed so that expectations are similar across groups without withholding necessary care. The comparison condition must be described precisely. Contamination occurs when participants receive elements of another condition; non-adherence occurs when assigned activities are not followed. Intention-to-treat analysis generally compares participants according to their original allocation and protects the benefit of randomisation, while per-protocol analyses answer a different question and can be biased if adherence is related to outcomes.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
7. Quasi-experimental, correlational and observational designs
Quasi-experimental designs evaluate interventions or exposures without full random assignment. Examples include non-equivalent comparison groups, interrupted time series and regression discontinuity designs. These approaches are useful when randomisation is impractical or unethical, such as evaluating a school policy introduced across a district. Their causal strength depends on assumptions about how comparison groups or time trends approximate the counterfactual outcome. Researchers should identify plausible competing explanations, use repeated measurements where appropriate and avoid presenting an association as proof that the intervention caused the observed change.
Correlational designs measure naturally occurring variables and estimate the direction and strength of their association. A positive correlation means higher values of one variable tend to accompany higher values of another; a negative correlation means higher values tend to accompany lower values. A near-zero linear correlation does not rule out a nonlinear relationship. Correlation does not establish causation because of reverse direction, confounding, selection effects and measurement error. For example, a relationship between time spent studying and examination performance may reflect prior achievement, motivation, teaching quality or access to resources as well as study time itself.
Descriptive studies estimate characteristics, experiences or frequencies without necessarily testing a causal explanation. Surveys can reach many participants but depend on question wording, response rates, sampling and recall. Observational studies record behaviour in natural or structured settings. Naturalistic observation can preserve context but may reduce control; structured observation increases comparability but may change behaviour. Researchers should define the observation unit, coding categories, sampling periods and procedures for inter-rater agreement. Reactivity occurs when people alter behaviour because they know they are being observed. Ethical observation requires careful attention to privacy, consent expectations and the sensitivity of the setting.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
8. Validity, reliability and threats to inference
Internal validity concerns whether the observed difference or relationship can credibly be attributed to the proposed cause within the study. Threats include history, maturation, testing effects, instrumentation changes, regression to the mean, selection differences, attrition and contamination. For example, if a stress-reduction programme is delivered during an examination-free period, lower stress may reflect the calendar rather than the programme. A comparison group observed over the same period can help address shared history effects. Repeated baseline measurements, consistent procedures, allocation concealment and careful follow-up can strengthen internal validity.
External validity concerns the extent to which findings may apply beyond the study sample, setting, intervention or time. Generalisation should be argued, not assumed. A highly controlled laboratory study may isolate a process but differ from everyday behaviour; a field study may be realistic but contain more uncontrolled influences. Ecological validity refers to how well tasks, settings and behaviours represent the real-world context relevant to the research question. Replication across populations, settings, researchers and measurement approaches builds confidence that a finding is not unique to one narrow context.
Reliability concerns consistency of measurement. Test-retest reliability examines stability over time when the construct is expected to remain stable. Inter-rater reliability examines agreement among observers, while internal consistency examines how coherently items function as a scale. Reliability is necessary but not sufficient for validity: a measure can consistently measure the wrong construct. Construct validity includes convergent evidence with related measures and discriminant evidence from distinct constructs. Criterion-related validity examines relationships with relevant outcomes or standards. Researchers should select evidence appropriate to the intended interpretation and population rather than rely on a single coefficient as proof that a measure is valid.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
9. Cross-sectional, longitudinal and special designs
Cross-sectional research measures variables at one point or within a short period. It can describe prevalence and compare groups efficiently, but it is limited for establishing temporal order. Longitudinal research follows people, groups or records over time and can examine trajectories, transitions and temporal associations. It may use repeated measures, panels or cohorts. Attrition can threaten validity when people who remain differ from those who leave. Repeated testing may also influence responses. Researchers should plan retention, document reasons for loss and use analytic methods that match the data structure.
Cohort studies follow people who share a defined characteristic or experience, such as birth period or exposure. Prospective cohorts collect information before outcomes occur; retrospective cohorts use existing records to reconstruct exposure and outcome histories. Case-control designs select participants based on outcome status and compare prior exposures, which can be efficient for uncommon outcomes but are vulnerable to recall and selection bias. Case studies investigate a person, group, institution or event in depth using multiple sources. Their value lies in contextual understanding and analytic insight, not statistical representativeness.
Single-case experimental designs collect repeated observations across phases, such as baseline and intervention, to evaluate change within an individual or small number of cases. Designs may withdraw and reintroduce an intervention, stagger its introduction across behaviours or settings, or compare alternating conditions. They require repeated measurement, stable and interpretable phases, and ethical consideration of whether withdrawing support is acceptable. Natural experiments examine exposures or policy changes that occur without researcher assignment. They can provide useful evidence when a credible comparison and defensible assumptions are available, but the researcher does not control the allocation mechanism.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
10. Survey construction and questionnaire design
A questionnaire translates research constructs into questions and response options. Good items use plain language, one idea at a time and a reference period that participants can understand. Double-barrelled questions ask about two issues together; leading questions imply a preferred answer; loaded wording can trigger defensiveness. Ambiguous terms such as ‘often’ should be replaced with defined timeframes or frequency categories. Response options should be mutually exclusive and collectively adequate for the intended population, with ‘not applicable’ or ‘prefer not to answer’ where appropriate.
Closed questions simplify coding and comparison but can constrain participants' responses. Open-ended questions allow unanticipated detail but require additional time and systematic analysis. Likert-type items offer ordered response categories, commonly ranging from disagreement to agreement or from never to always. Researchers should avoid assuming that a single item is a precise interval measure. Reverse-worded items may reduce automatic agreement but can confuse respondents and create method effects. A questionnaire should be piloted with people similar to the intended participants to test comprehension, completion time, accessibility and technical performance.
Survey administration affects data quality. Modes include paper, telephone, interviewer-administered and online forms. Each has different coverage, privacy, accessibility and social-desirability risks. Reminders may improve response rates but should not become coercive. Researchers should distinguish item non-response from unit non-response and document the response denominator. If a survey is translated, the process should consider conceptual equivalence rather than literal word substitution alone. Cognitive interviewing can reveal how respondents interpret questions and select answers. Data cleaning rules, scoring procedures and treatment of incomplete questionnaires should be specified before analysis where possible.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
11. Qualitative interviewing, focus groups and fieldwork
Qualitative interviews explore participants' experiences, interpretations and actions. Structured interviews use a fixed sequence, semi-structured interviews use a guide with flexible probes, and unstructured interviews allow the conversation to develop around a broad topic. Open prompts invite description, while probes ask for examples, clarification or contrasts. Interviewers should listen actively, avoid imposing assumptions and allow silence when participants are reflecting. The interviewer-participant relationship influences what is shared, so reflexive notes should document relevant context and the researcher's role.
Focus groups use facilitated discussion among several participants to examine shared meanings, disagreements and social norms. Interaction is part of the data: participants may build on, challenge or silence one another. Groups can be useful for exploring language and collective views, but they may be unsuitable for sensitive topics where confidentiality cannot be guaranteed. The facilitator should establish ground rules, manage dominance, invite quieter voices and avoid treating group consensus as proof that every participant agrees. Researchers should explain that they can protect stored data but cannot fully control what group members repeat outside the session.
Fieldwork may include participant observation, documents, diaries, digital traces or artefacts. Ethnographic approaches examine practices within their social and cultural settings and often require sustained engagement. Researchers should record descriptive observations separately from analytic reflections, attend to access and gatekeeping, and consider how their presence shapes the setting. Digital research raises questions about whether online spaces are public, participants' reasonable privacy expectations and the risk of quoting searchable text. Data collection should stop or change when safety, distress or consent concerns arise.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
12. Qualitative analysis and mixed-methods integration
Thematic analysis identifies patterned meaning across a dataset. A typical process involves familiarisation, initial coding, constructing candidate themes, reviewing themes against the data, defining and naming themes, and producing an evidence-based account. Coding can be inductive, deductive or a combination. A theme is more than a topic label: it makes an interpretive claim about a pattern relevant to the research question. Researchers should retain examples that complicate the dominant pattern and avoid selecting only quotations that support their preferred explanation.
Phenomenology explores the structure and meaning of lived experience. Grounded theory develops an explanatory account through iterative data collection and analysis, constant comparison and theoretical sampling. Ethnographic analysis interprets practices, relationships and cultural meanings in context. Narrative analysis examines how people organise events and identity through stories. These approaches have different aims and standards; a project should not claim to use a named methodology merely because it conducted interviews. The analytic approach should be aligned with the question, philosophical assumptions, data and intended contribution.
Rigour in qualitative research can be supported through a clear audit trail, reflexivity, thick description, attention to negative cases, triangulation where appropriate and transparent explanation of analytic decisions. Member reflection may help participants comment on interpretations, but participants are not automatically the final arbiters of every analytic claim. In mixed-methods work, integration may occur by connecting samples, building one phase from another, merging datasets or embedding one strand within another. Joint displays can compare quantitative patterns with qualitative themes. Researchers should explain convergence, complementarity and contradiction rather than forcing all evidence into a single narrative.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
13. Ethics, informed consent and participant protection
Ethical research respects autonomy, seeks to maximise benefit and minimise harm, and distributes burdens and benefits fairly. Informed consent is an ongoing process rather than a signature alone. Participants should receive understandable information about purpose, procedures, duration, foreseeable risks, benefits, confidentiality limits, data use, voluntary participation and withdrawal arrangements. Consent must be free from undue influence. Researchers should distinguish participation in a study from permission to use identifiable quotations, photographs, recordings or data for future purposes.
Deception may sometimes be considered when disclosure would invalidate a study, but it requires strong justification, minimal risk, prior ethical review and an appropriate debriefing process. Debriefing explains the true purpose, addresses concerns and provides contact information. Participants should not be exposed to avoidable distress, humiliation or dangerous procedures. Research involving children or adults with impaired decision-making capacity requires additional safeguards, appropriate permission and respect for assent or the person's expressed wishes. Gatekeeper permission does not replace the individual's consent when individual consent is required.
Privacy and confidentiality are related but distinct. Privacy concerns access to a person or their information; confidentiality concerns how information entrusted to researchers is handled. Data minimisation means collecting only what is needed. Identifiers should be separated from research data where feasible, access should be restricted, storage encrypted and retention periods defined. Researchers should explain circumstances in which confidentiality may have legal or safety limits. Reports should avoid combinations of details that could identify participants indirectly, especially in small communities or sensitive case studies.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
14. Data management, transparency and open science
A data-management plan describes what data will be collected, how they will be documented, stored, secured, shared and eventually deleted or archived. File naming conventions, version control, codebooks and metadata make a project understandable beyond the original researcher. Raw data should be preserved separately from cleaned or transformed datasets. Changes should be documented rather than silently overwritten. Access should follow consent, ethics approval, applicable law and institutional policy; open science does not mean publishing identifiable or sensitive participant data.
Preregistration records research questions, hypotheses, outcomes, design and planned analyses before researchers inspect outcome data or begin the relevant analysis. It helps distinguish confirmatory tests from exploratory analyses. A registered report can obtain peer review of the research question and methods before results are known. Neither approach guarantees quality, but both can reduce undisclosed flexibility. Deviations may be necessary; researchers should report them transparently and explain why they occurred rather than conceal them.
Reproducibility concerns whether the same data and analytic procedures produce the same results. Replicability concerns whether a new study produces compatible evidence. P-hacking occurs when researchers repeatedly explore analytical choices until a statistically significant result appears, without transparently accounting for those choices. HARKing means hypothesising after results are known and presenting the hypothesis as if it were specified in advance. Publication bias occurs when positive or striking results are more likely to be published. Larger, preregistered, multi-site and well-reported studies, registered reports, data and code sharing where safe, and publication of null findings can strengthen the evidence base.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
15. Planning a study from proposal to protocol
A research proposal should explain the problem, rationale, literature, aims, questions, hypotheses where appropriate, design, population, sampling, measures, procedures, analysis plan, ethics, timeline and resources. The methods section must contain enough detail for a knowledgeable reader to understand exactly what will happen. The protocol should identify recruitment scripts, consent materials, intervention fidelity procedures, data-security arrangements and criteria for stopping or modifying the study. A realistic proposal anticipates barriers such as low recruitment, equipment failure, participant burden and staff turnover.
A pilot study tests feasibility and procedures rather than serving as a miniature definitive trial. It can reveal whether instructions are understood, recruitment is practical, measures are acceptable, timing is realistic and data systems function. Pilot findings may justify changes to the protocol. Researchers should distinguish feasibility outcomes from effectiveness outcomes and avoid drawing strong efficacy conclusions from an underpowered pilot. If the protocol changes after recruitment begins, the change should be documented, reviewed where required and reported accurately.
A preregistered analysis plan identifies primary and secondary outcomes, scoring rules, exclusions, handling of missing data, planned comparisons and any covariates. Defining a primary outcome helps reduce selective emphasis after results are known. Researchers should also state how adverse events, protocol deviations and participant withdrawals will be recorded. A project timeline should allow for ethics review, piloting, recruitment, data collection, cleaning, analysis, interpretation and dissemination. Quality assurance includes staff training, standard operating procedures, secure backups and periodic checks that data collection matches the approved protocol.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
16. Basic design calculations and interpretation
Design planning often requires simple calculations before advanced statistical analysis. A change score is calculated as follow-up minus baseline. If a participant's attention score rises from 42 to 51, the change is 51 − 42 = 9 points. Percentage change is calculated as (follow-up − baseline) divided by baseline, multiplied by 100. In this example, (9 ÷ 42) × 100 is approximately 21.4%. Percentage change can be misleading when the baseline is close to zero or the scale has no meaningful ratio zero, so the original units should also be reported.
A simple between-group difference compares group means on a defined outcome. If an intervention group has a mean improvement of 8 points and a comparison group improves by 3 points, the unadjusted difference in improvement is 8 − 3 = 5 points. This contrast is more informative than reporting that the intervention group improved by eight points alone, but causal interpretation still depends on design quality, allocation, attrition and measurement. Researchers should report uncertainty and avoid treating a numerical difference as practically important without considering the scale and context.
Correlation coefficients range from −1 to +1 for common linear correlation measures. A positive value indicates a positive linear association, a negative value an inverse linear association, and a value near zero little linear association. The magnitude should be interpreted in context; there is no universal threshold that makes an association important. A correlation of 0.30 does not mean that one variable causes 30% of the other. Scatterplots should be inspected for outliers, nonlinear patterns and restricted range. Statistical association, practical importance and causal evidence are distinct ideas.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
17. Writing, reporting and communicating research
A research report should allow readers to evaluate the question, methods, evidence and limitations. The introduction establishes the problem and rationale. The methods section describes design, participants, recruitment, measures, procedure and analysis. Results present findings without overstating interpretation, while the discussion relates them to theory and previous work, considers alternative explanations, acknowledges limitations and identifies implications. Abstracts should accurately represent the whole study and should not introduce claims absent from the main report.
Transparent reporting includes participant flow, sample characteristics relevant to the question, missing data, exclusions, deviations from the protocol and all outcomes that were planned. Tables and figures should have clear titles, units, labels and notes. Statistical results should be reported with estimates and uncertainty where appropriate, not only a statement that a result was significant. Qualitative reports should explain the analytic process and support interpretations with carefully selected evidence while protecting identities. Mixed-methods reports should show how strands were integrated.
Communication should be proportionate to the evidence. A single small study rarely justifies a universal recommendation or a claim that an intervention works for everyone. Researchers should distinguish statistical significance from effect size, precision and practical importance. Limitations should be specific: for example, volunteer recruitment from one university may limit transferability, while short follow-up may prevent conclusions about durability. Responsible dissemination includes accessible summaries for participants and communities, appropriate acknowledgement of contributors, disclosure of conflicts of interest and correction of material errors when identified.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
18. Research quality, bias and responsible interpretation
Bias is a systematic process that distorts selection, measurement, analysis or interpretation. Selection bias arises when entry into a study or remaining in it is related to variables relevant to the research question. Information bias arises from inaccurate or differential measurement. Recall bias can affect retrospective self-report; observer bias can influence coding; social desirability can shape responses about sensitive behaviours. Researchers can reduce these risks through clear eligibility criteria, standardised procedures, validated instruments, blinded assessment where feasible and independent checks.
Confounding occurs when a third factor distorts the observed relationship between an exposure and outcome. For example, the relationship between exercise and mood may be influenced by physical health, income, social support or baseline wellbeing. Randomisation can balance confounders on average in experiments; observational studies may use restriction, matching, stratification or statistical adjustment, each with assumptions and limitations. Adjusting for a variable that is actually a mediator or collider can introduce bias, so covariate selection should be guided by a causal model and substantive knowledge rather than automated significance testing.
A limitation is not necessarily a fatal flaw, but its consequences must be explained. Researchers should ask whether a weakness could change the direction, size or interpretation of the finding. Sensitivity analyses can examine how conclusions change under alternative assumptions. Triangulation can compare evidence from different measures, observers or data sources, but agreement does not guarantee truth if all sources share the same bias. Good research practice includes intellectual humility: report what the design supports, what remains uncertain and what further evidence would change the conclusion.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
19. Applications of research methods across psychology
In clinical psychology, research methods evaluate assessment tools, interventions, service delivery and patient experiences. Randomised trials can compare interventions when equipoise and ethical safeguards exist; observational cohorts can examine outcomes in routine care; qualitative interviews can reveal barriers to engagement and meanings of recovery. Clinical research must protect confidentiality, manage risk and ensure that participation does not compromise care. Researchers should distinguish research from clinical treatment and avoid implying that an experimental intervention is established care.
In educational psychology, studies may examine teaching strategies, motivation, classroom behaviour, inclusion and assessment. Cluster randomisation may be appropriate when an intervention is delivered to whole classes or schools, but analysis must account for clustering. Quasi-experimental evaluation may be necessary when policies are introduced at system level. Measures should be age-appropriate, culturally and linguistically suitable, and accessible to learners with disabilities. Researchers should consider teacher workload, parental permission where relevant, student assent and the risk of labelling learners based on research scores.
In organisational and social psychology, field experiments, surveys, interviews and longitudinal designs can examine leadership, teamwork, job satisfaction, discrimination and wellbeing. Researchers must protect employees from retaliation and ensure that managers cannot identify individual responses when confidentiality has been promised. In health psychology, studies may examine behaviour change, adherence, stress and prevention. Across domains, the same principles apply: define constructs, choose a design that fits the question, minimise bias, protect participants and communicate conclusions within the limits of the evidence.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
20. An integrated worked example: evaluating a study-skills intervention
Consider a university department that wants to evaluate a four-week retrieval-practice programme for first-year students. The research question is whether students offered structured retrieval practice achieve higher delayed recall scores than students receiving the department's usual study guidance. The primary outcome is a pre-specified delayed recall test administered two weeks after the programme. Secondary outcomes might include attendance, perceived workload and student experience. The team reviews evidence on retrieval practice, specifies a conceptual model in which repeated retrieval strengthens later access, and defines the intervention dose and comparison condition before recruitment.
If feasible, students are randomly assigned individually or by tutorial group. Group assignment may reduce contamination when students share materials, but it creates clustering and requires enough groups. Allocation is generated independently and concealed until enrolment. Outcome assessors are blinded to condition if practical. The protocol records baseline achievement, programme attendance, exposure to other study supports and follow-up completion. Consent explains that participation is voluntary and does not affect grades. The research team keeps course instructors from accessing identifiable responses and follows institutional data-security procedures.
The analysis plan specifies the primary comparison and how missing follow-up scores will be handled. A pilot checks whether the retrieval tasks are understandable and whether the two-week assessment is feasible. Suppose the intervention group's average gain is 7 points and the comparison group's average gain is 2 points; the difference in gain is 5 points. The team reports the estimate, uncertainty, sample size, attrition and fidelity data. Interviews may then explore why some students used the practice consistently and others did not. The final interpretation considers contamination, sample representativeness, duration and whether a five-point difference is educationally meaningful. The result informs further research rather than automatically proving universal effectiveness.
In applying this principle, the researcher should document the decision, explain its relationship to the research question and consider how it affects the strength of the conclusion. The choice should be made before the results are known whenever possible. If circumstances require a change, the reason and timing should be recorded and reported. This discipline makes the study more interpretable to readers and more useful to researchers who may extend or replicate the work.
Research design diagrams
Figure 10.1: The research process
Research problem
→
Literature & theory
→
Question & hypothesis
→
Design & ethics
↓
Sampling & measures
→
Data collection
→
Analysis
→
Interpretation & report
The process is iterative: findings may refine theory and generate new questions. Ethics and quality assurance apply at every stage.
Figure 10.2: Basic parallel-group experiment
Eligible participants
→
Informed consent
→
Random assignment
↙ ↘
Intervention group
Comparison group
↘ ↙
Same outcome assessment
→
Compare outcomes
Random assignment supports baseline comparability on average; consistent outcome measurement and follow-up protect the comparison.
Figure 10.3: Validity and measurement quality
Construct definition
→
Operational measure
→
Observed score
↓
Reliability: consistency
+
Validity: supported interpretation
Consistency alone is insufficient: the score must also support the intended interpretation for the relevant population and use.
Structured university examination questions with model answers
Answer each question using clear definitions, justified comparisons and examples. Model answers indicate the expected direction and key points; students should expand them into coherent examination responses.
1. Distinguish a theory, hypothesis, research question and operational definition.
Model answer: A theory is an organised explanatory framework; a research question states what the study seeks to answer; a hypothesis is a testable prediction derived from theory or evidence; an operational definition specifies how a construct or variable will be observed or measured. They perform different roles in the logic of a study.
2. Explain the difference between random sampling and random assignment.
Model answer: Random sampling selects people from a population and primarily supports representativeness and generalisation. Random assignment allocates enrolled participants to study conditions and supports causal comparability by balancing confounders on average. One does not guarantee the other.
3. A study finds that students who sleep longer report higher grades. Give three reasons why this correlation does not establish causation.
Model answer: Reverse direction may occur if academic success allows more regular sleep; confounding factors such as health, workload or socioeconomic circumstances may influence both; and measurement error or selection may distort the association. Temporal and experimental evidence would be needed for stronger causal claims.
4. Define internal, external and ecological validity.
Model answer: Internal validity concerns whether the study supports the proposed causal inference within its design. External validity concerns transfer to other populations, settings, interventions or times. Ecological validity concerns how well the tasks and setting represent the real-world context relevant to the question.
5. Describe four threats to internal validity in a pretest-posttest study.
Model answer: History is an external event occurring between measurements; maturation is natural change over time; testing effects arise because the pretest influences later responses; instrumentation occurs when measurement procedures or tools change. Selection and attrition can also threaten validity, especially without a credible comparison group.
6. Compare experimental and quasi-experimental designs.
Model answer: An experiment manipulates a condition and uses random assignment where possible. A quasi-experiment evaluates an intervention or exposure without full random allocation, using strategies such as comparison groups, interrupted time series or regression discontinuity. Quasi-experiments can be valuable but require careful examination of alternative explanations.
7. Explain the purpose of a control group and identify two possible control conditions.
Model answer: A control group provides a comparison for interpreting outcomes under the intervention. It may receive usual care, no additional intervention, an attention-matched activity or a suitable placebo. The choice depends on ethics, the research question and what alternative explanations need to be controlled.
8. What is a confounding variable? Give a psychological research example.
Model answer: A confounder is a factor related to both the exposure or condition and outcome that can distort their observed relationship. For example, prior academic ability may influence both participation in optional study coaching and later examination performance.
9. Explain the difference between nominal, ordinal, interval and ratio measurement.
Model answer: Nominal data classify without order; ordinal data have order but unequal or unknown intervals; interval data have equal intervals without an absolute zero; ratio data have equal intervals and a meaningful zero. The scale constrains valid summaries and interpretations.
10. Why is reliability necessary but insufficient for validity?
Model answer: Reliability means measurement consistency. A measure must be sufficiently consistent to support an interpretation, but it may consistently measure the wrong construct or be unsuitable for a population. Validity requires evidence supporting the intended interpretation and use.
11. Describe stratified and cluster sampling and one advantage of each.
Model answer: Stratified sampling divides a population into relevant subgroups and samples within them, helping ensure subgroup representation. Cluster sampling selects naturally occurring groups, such as schools, and can reduce fieldwork cost. Clustered data require attention to within-cluster similarity.
12. List key factors in determining quantitative sample size.
Model answer: The primary outcome, expected effect, variability, design, desired power, significance threshold, precision, clustering, expected attrition and available resources. Assumptions should be justified and documented rather than selected to produce a convenient sample.
13. What is the difference between a cross-sectional and longitudinal design?
Model answer: A cross-sectional design measures variables at one point or short period; a longitudinal design follows participants, groups or records over time. Longitudinal data can examine trajectories and temporal order but face attrition, repeated-testing and resource challenges.
14. Explain how a focus group differs from an individual interview.
Model answer: A focus group collects data through facilitated interaction among participants, revealing shared norms and disagreement. An individual interview explores one person's account in depth and may be safer for sensitive topics. Group confidentiality cannot be fully guaranteed.
15. Outline the main stages of thematic analysis.
Model answer: Familiarisation with the data, initial coding, constructing candidate themes, reviewing themes against the dataset, defining and naming themes, and writing an evidence-based account. Researchers should preserve contradictory cases and explain analytic decisions.
16. What is reflexivity in qualitative research?
Model answer: Reflexivity is the systematic examination of how the researcher's identity, assumptions, relationships and decisions shape access, data generation and interpretation. It is documented through reflective notes and transparent discussion rather than treated as a claim of neutrality.
17. Explain informed consent as an ongoing process.
Model answer: Participants should understand purpose, procedures, risks, benefits, confidentiality limits, data use, voluntariness and withdrawal arrangements. Researchers must respond to questions and changes in circumstances; a signed form alone does not ensure meaningful consent.
18. When might deception be considered and what safeguards are required?
Model answer: Only when scientifically justified, no adequate non-deceptive alternative exists, risk is minimal and ethics review approves it. Participants should receive appropriate debriefing, an opportunity to ask questions and any withdrawal option specified by the approved protocol.
19. Differentiate privacy and confidentiality.
Model answer: Privacy concerns access to a person or their information; confidentiality concerns how information entrusted to researchers is handled. Both require proportionate collection, secure storage, access controls and clear communication of limits.
20. Define preregistration and explain one benefit and one limitation.
Model answer: Preregistration records questions, hypotheses, outcomes, design and planned analyses before relevant results are inspected. It distinguishes confirmatory from exploratory work and reduces undisclosed flexibility. It cannot guarantee good methods and legitimate deviations may still occur.
21. Explain p-hacking and HARKing.
Model answer: P-hacking is trying multiple analytical choices until a desired significant result appears without transparently accounting for the flexibility. HARKing is presenting a hypothesis developed after seeing results as if it had been specified beforehand. Both misrepresent the evidential status of findings.
22. A baseline score is 40 and follow-up is 50. Calculate the change and percentage change.
Model answer: Change = 50 − 40 = 10 points. Percentage change = (10 ÷ 40) × 100 = 25%. The original score units should also be reported, and percentage change is appropriate only if the scale supports ratio interpretation.
23. An intervention group improves by 9 points and a comparison group by 4 points. What is the difference in improvement?
Model answer: The difference is 9 − 4 = 5 points. This is a descriptive contrast; its causal meaning depends on allocation, baseline comparability, attrition, measurement and uncertainty.
24. Interpret a correlation of −0.60 between perceived stress and wellbeing.
Model answer: It indicates a moderately strong negative linear association in the observed data: higher stress tends to accompany lower wellbeing. It does not show that stress causes lower wellbeing, and context, measurement, confounding and outliers should be examined.
25. Why should a pilot study not usually be treated as a definitive effectiveness trial?
Model answer: A pilot primarily tests feasibility, recruitment, procedures, acceptability and data systems. It is often too small and not designed to estimate effectiveness precisely. Its findings guide refinement of the definitive study.
26. Identify three ethical issues in research involving school learners.
Model answer: Age-appropriate information and assent, parental or guardian permission where required, protection from coercion by teachers, privacy of educational records, minimising distress, accessibility and ensuring that participation or refusal does not affect grades or support.
27. What should a transparent research report disclose about participant flow?
Model answer: Numbers approached, screened, eligible, enrolled, allocated where relevant, followed up and analysed; reasons for exclusion or withdrawal; missing outcomes and attrition; and any deviations from the planned protocol.
28. Explain why statistical significance and practical importance are different.
Model answer: Statistical significance reflects compatibility of data with a specified statistical model or null hypothesis, not the size or value of an effect. Practical importance depends on magnitude, uncertainty, cost, harms, context and whether the difference matters to participants or practice.
29. Describe two ways to strengthen the credibility of a qualitative study.
Model answer: Maintain an audit trail and reflexive record; provide thick contextual description; examine negative cases; use appropriate triangulation; explain sampling and analysis; and support interpretations with evidence while protecting participants.
30. Propose a design to evaluate a new classroom attention programme and justify it.
Model answer: A cluster-randomised trial across classes can reduce contamination when the programme is delivered at class level. Measure attention at baseline and follow-up using a pre-specified age-appropriate instrument, blind outcome assessors where feasible, account for clustering and monitor implementation. If randomisation is not possible, use a matched comparison group with repeated measurements and explicitly address selection and time trends.
Chapter summary
Psychological research is a disciplined process of asking answerable questions, grounding them in theory and evidence, defining constructs, choosing suitable designs, recruiting participants fairly, measuring carefully, analysing transparently and protecting people. Experiments can support causal inference when allocation and implementation are credible; observational and qualitative approaches answer different but equally important questions. Validity, reliability, sampling, ethics and transparency are interconnected. A strong researcher reports both what the evidence suggests and what it cannot establish.
End of Chapter 10. Continue with Chapter 11: Statistics and Data Analysis for Psychology.
Official Education References
Use these official organisations for curriculum guidance, examinations and further learning information.