STAT 1103 Week 6 Notes: Non-Experimental Designs

Summary

Difficulty: ★★☆☆☆

Covers: Non-experimental research, association not causation, group comparison vs correlational questions, categorical vs numeric variables, independent samples t-test, Pearson correlation, effect sizes (Cohen’s d, r), statistical power, sample size and interpretation

Purposes of Psychological Research

Psychological research aims to:

  1. Describe behaviour (what is happening?)
  2. Predict behaviour (what will happen based on other information?)
  3. Explain behaviour (why does it happen?)
  4. Control behaviour (how can we change it?)

Non-experimental research is mainly used for description and prediction.
Experimental research is needed for stronger explanation and control (causality).

Cause & Effect and Causality

A relationship between two variables does not automatically mean one causes the other.

Criteria for a causal relationship

RequirementMeaning
CovarianceVariables must be related
Temporal precedenceCause must happen before effect
Internal validityOther possible explanations are ruled out

Spurious Correlation

  • A correlation that appears meaningful but is not causal
  • Often caused by a third variable
  • Classic example: ice cream sales and murder rates (both increase in summer)
Experimental vs Non-Experimental Designs

Key Distinction: Manipulation and Control

Design TypeWhat happens?What you can conclude
ExperimentalResearcher manipulates IV and controls conditions (often random assignment)Can support causal conclusions (if well-designed)
Non-experimentalResearcher measures naturally occurring variables (no random assignment, no controlled manipulation)Can conclude association or difference, not causation

Internal vs External Validity

ConceptMeaning
Internal validityConfidence that IV caused DV (stronger in experiments)
External validityGeneralisability to real-world populations (often stronger in non-experimental research)
Types of Non-Experimental Methods

1. Single-Variable Research

  • Focuses on one variable only
  • Uses:
    • Descriptive statistics
    • One-sample tests (from Week 5)

Example: “How many psychology students studied science in high school?”

2. Associative / Correlational Research

Focuses on relationships between variables, usually without manipulation.

Two broad forms:

  • Minimal intervention (just measure what already exists)
  • Self-selected groups (may involve a program or choice, but not random assignment)

Example: People choose whether to do a mental health course → groups differ, but it is not experimental.

Two Major Non-Experimental Research Questions

Both involve two variables, but differ in variable type.

RQ TypeVariablesTypical TestCore Question
Group comparisonOne categorical (2 groups) + one numericIndependent samples t-testDo groups differ on the mean of a numeric outcome?
CorrelationalTwo numeric variablesPearson correlation testAre two numeric variables associated (linearly)?
Group Comparison Research Questions

Structure

  • Independent variable (IV): categorical with two groups
  • Dependent variable (DV): numeric

Example RQs:

  • Do junior vs senior staff differ in professionalism?
  • Do Australia vs USA adolescents differ in age of first relationship?
  • Do students who answer tutorial questions differ in confidence compared to those who don’t?

Independent Samples (Two-Sample) t-Test

Statistical Hypotheses

  • H₀: μ₁ = μ₂
  • H₁: μ₁ ≠ μ₂

Assumptions

AssumptionWhat it means
DV is numericOutcome is measured on a number scale
Normality (within groups)DV is approximately normal in each group (check histogram)
Equal variancesVariability similar in groups (check Levene’s test)
IndependenceObservations are independent within and across groups

Interpretation

  • If p ≤ α (usually .05): reject H₀ → evidence groups differ
  • Report effect size (Cohen’s d), not just significance

Limits of Group Comparison in Non-Experimental Research

Even if groups differ, you can’t conclude causality because:

  • groups were not randomly assigned
  • third variables may explain the difference
  • self-selection bias may exist
Correlational Research Questions

Structure

  • Two numeric variables
  • Aim: determine whether higher values on one variable correspond with higher or lower values on the other

Example RQs:

  • Is extraversion related to amount of tutorial talking?
  • Is ATAR correlated with university performance?
  • Is emotion dysregulation related to depressive symptoms?

Pearson Correlation (r)

  • Measures strength and direction of a linear relationship
  • Range: -1 to +1
    • positive r: variables increase together
    • negative r: one increases as the other decreases
    • r ≈ 0: no linear relationship

Interpreting correlation strength

r value rangeInterpretation
0.00–0.10Very weak / none
0.10–0.30Weak
0.30–0.50Moderate
0.50–1.00Strong

Correlation Test (Hypothesis Test)

Hypotheses

  • H₀: ρ = 0
  • H₁: ρ ≠ 0

Assumptions

AssumptionMeaning
Both variables numericMeasured on number scales
Relationship is monotonic/linearScatterplot shows roughly linear trend
IndependenceEach (x, y) pair is independent

Limits of Correlation

Correlation does not establish causation because:

  • direction is unclear (X → Y? Y → X?)
  • third variables may explain the association
Effect Sizes

Statistical significance answers: “Is the effect unlikely due to chance?”
Effect size answers: “How big is the effect?”

Cohen’s d (used for mean comparisons)

Test TypeCohen’s d
One-sample t-testd = (M − μ) / s
Two-sample t-testd = (M₁ − M₂) / sₚ

Effect Size Interpretation

Effect sizeCohen’s dr
Negligible< 0.2< 0.1
Small0.2–0.50.1–0.3
Medium0.5–0.80.3–0.5
Large≥ 0.8≥ 0.5

Note: In correlation, r is already the effect size.

Statistical Power

Definitions

  • Type I error (α): reject H₀ when H₀ is true
  • Type II error (β): fail to reject H₀ when H₀ is false
  • Power (1 − β): probability of detecting an effect if it truly exists

Factors that Increase Power

FactorEffect on power
Larger sample sizeincreases power
Larger effect sizeincreases power
Less variability/noiseincreases power
Higher α (less strict)increases power but increases Type I risk

Why Power Matters

  • Large samples can detect extremely small effects that are statistically significant but practically unimportant
  • Small samples may miss meaningful effects (low power)
Appropriate Conclusions for Non-Experimental Research

Non-experimental designs allow conclusions such as:

  • A difference exists between groups (not why it exists)
  • A relationship/association exists between variables (not causation)

They do not allow strong causal conclusions because:

  • no random assignment
  • limited control over confounds
  • possible self-selection and third-variable explanations

Leave a comment