Workshop "Improving the Informational Value of Studies"

Increased attention to the design of experiments has lead to journals starting to require sample size justifications. As a field, we have realized it is important to combat publication bias by publishing informative null effects. How do you decide upon the sample size in studies? How can you design reliable but efficient studies, that allow you to both show the presence, and the absence, of an effect that is meaningful?

Schedule

  • 09:00-09:15 Introduction
  • 09:15-10:30 Guidance on how to act. Type 1 error control: Why it matters, and how it works in practice. What is ‘p-hacking’? How can you recognize it, and prevent it in your own research?
  • 10:30-10:45 Coffee break
  • 10:45-12:00 What would falsify your hypothesis? How can we specify falsifiable predictions? How do you determine your smallest effect size of interest based on theory, practical relevance, or feasibility, and test for equivalence?
  • 12:00-13:00 Lunch
  • 13:00-14:45 Which sample sizes do you need, and which effects can you study? How do you perform and report a power analysis, and what to do when the sample size is limited by the resources we have?
  • 14:45-15:00 Coffee break
  • 15:00-16:45 Sequential analyses: How can you design studies by repeatedly collecting data without inflating error rates? What are similarities and differences between Frequentist and Bayesian approaches to sequential analyses?
  • 16:45-17:00 Conclusion

Required reading:

Optional but recommended reading:

  • Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141. doi.org/10.1207/s15327965pli0102_1

Date: Friday, 29th October 2021

Time: 9-17 hrs.

Venue: online workshop at your own computer/laptop

Lecturer:

Prof. Dr. Daniël Lakens, Human-Technology interaction group at Eindhoven University of Technology