Measurement matters: the role of measurement in the open science reform

In the last decade, the open science reform has helped us identify and prevent questionable research practices. Challenges and proposed solutions have focused on increasing the reliability and replicability of empirical findings by improving methodological and statistical practices. This talk highlights the often under-appreciated role of measurement and introduces questionable measurement practices (QMPs) as decisions researchers make that raise doubts about the validity of the measures. This threatens the validity of the study conclusions. Many research fields are plagued by a measurement schmeasurement attitude: QMPs are common, hide a stunning source of researcher degrees of freedom, and pose a serious threat to valid inferences, but are largely ignored. This talk describes the scope of the problem overall and focuses on how transparency is a part of the solution.