What Is ‘Best Practice’?

Psychological science aims to understand real human behaviour.
To do this well, research must be:

• Careful
• Transparent
• Replicable
• Representative

Why Is This Important?

Finding a statistically significant result does not mean:

• It applies to everyone
• It applies everywhere
• It justifies extreme conclusions

Culture and WEIRD Psychology

What is WEIRD?

WEIRD = Western, Educated, Industrialised, Rich, Democratic

• Most psychology research uses Western university students
• Especially American college students
• These samples are not representative of the world

Why This Is a Problem

Psychology often assumes findings are universal, but:

• Many effects do not generalise across cultures
• Behaviour can differ by:
  • Culture
  • Gender
  • Socioeconomic background
  • Country
  • Community norms

Note: External validity matters.

Example: Self-Serving Bias

• Western samples show strong self-serving bias
• Some non-Western cultures:
  • Show weaker bias
  • Or even self-effacing bias

• What looks “normal” in one culture may not be universal.

What Can We Do About WEIRD Bias?

• Study broader, more diverse samples
• Replicate studies across:
  • Cultures
  • Genders
  • Contexts
• Be cautious when making universal claims about “human nature”

How Do We Measure Gender Demographics Properly

Old Problems

• Early psychology often:
  • Didn’t measure gender at all
  • Assumed “male” as default
  • Ignored diversity

Modern Understanding

• Gender is not binary
• Many cultures recognise more than two genders
• Measuring gender poorly leads to:
  • Misclassification
  • Invalid conclusions
  • Harmful outcomes

Best Practice for Gender Measurement

• Use inclusive options
• Allow:
  • Open-ended responses
  • Multiple categories
• Be transparent about:
  • What was measured
  • What was not analysed
  • Limitations

The Replication Crisis

What Is the Replication Crisis?

Many famous psychology findings:

• Fail to replicate
• Show much smaller effects when repeated
• Or disappear entirely

Replication = doing the same study again to see if the result holds.

Why Replication Matters

• One study ≠ truth
• Improves reliability
• Results can occur due to:
  • Random chance
  • Sampling variability
  • Methodological flaws

Why Do Studies Fail to Replicate?

Common Reasons

1. Questionable research practices
   • Trying many analyses
   • Only reporting significant results
2. Small sample sizes
   • Effects look bigger than they really are
3. File drawer problem
   • Null results aren’t published
4. P-hacking
   • Searching for any significant p-value

Effect Size vs Statistical Significance

• p-value: “Is there some evidence of an effect?”
• Effect size: “How big is the effect?”

Important notes:

• Large samples → small p-values
• Small effects can be statistically significant
• Good science reports both

Best Practices in Psychological Science

Core Principles

Good research involves:

1. Planning well
2. Conducting studies carefully
3. Reporting everything transparently

Key Best-Practice Tools

1. Preregistration

• Write your hypotheses and analysis plan before collecting data
• Reduces flexibility and bias

2. Open Data & Open Code

• Share:
  • Data
  • Analysis scripts (.do files in Stata)
• Allows:
  • Error checking
  • Replication
  • Meta-analysis

3. Open Access Publishing

• Research available to everyone
• Reduces publication bias
• But quality still needs evaluation

4. Replication

• True effects should appear again and again
• Across samples and contexts

5. Use Effect Sizes

• Don’t rely on p-values alone
• Consider practical importance

Best Practice For Graphs

Good graphs should be:

Clear and Honest

• No unnecessary colours
• No 3D or distortions
• No clutter

Properly Labelled

• Clear title
• Meaningful axis labels
• Units included

Not Misleading

• Axes should start at 0 (or clearly indicate otherwise)
• Show variability (error bars, jitter, trends)

So Overall…

• Psychology is complex because people are complex
• Bias cannot be removed entirely — but it can be reduced
• Science improves through:
  • Transparency
  • Replication
  • Better methods