WEIRD but Also Inconsistent: An Analysis of the Reporting Practices of Participant Samples Across Five Areas of Psychology

Overview

This investigation examined reporting practices for participant demographics and sampling methods across five subdisciplines of psychology. The analysis encompassed 661 articles containing 1293 distinct samples published between January 2021 and December 2023 in journals covering social, health, clinical, developmental, and general psychological science. The study evaluates the extent and consistency with which researchers document participant characteristics including gender, race, socioeconomic status, and education level, alongside patterns in reliance on student populations and crowdsourcing platforms.

Methods and approach

A systematic review of Methods sections was conducted across five core psychology subdisciplines. The sampling frame included articles published over a three-year period (January 2021 through December 2023) by a major professional psychology publisher. Data extraction captured information on participant demographics (gender, race, SES indicators, education), sample composition (student-based versus community samples), and use of crowdsourcing platforms. Quantitative analysis characterized the prevalence and variation of reporting practices across subdisciplines and identified patterns in participant sourcing strategies.

Results

Western-origin participants substantially dominated the sample composition, with participants from Latin America, Eastern Europe, the Middle East, and Africa collectively representing 8.7% of all samples analyzed. Substantial heterogeneity emerged across subdisciplines in reporting practices for demographic variables: the documentation of gender, race, SES indicators, and educational background varied significantly by psychological area. Divergent norms were identified regarding reliance on student participants and crowdsourcing platforms, with differential adoption across the five subdisciplines examined.

Implications

The findings demonstrate inconsistency in demographic reporting standards across psychology subdisciplines, which constrains the ability of researchers and practitioners to evaluate the generalizability of findings across diverse populations. A unified reporting standard applicable across areas of psychology would improve research accessibility and facilitate assessment of external validity. Such standardization would address both the documented geographic underrepresentation and the variability in documentation practices, enabling more systematic evaluation of how empirical results extend beyond the sampled populations.