Asterisk Magazine: Can We Trust Social Science Yet? Everyone likes the idea of evidence-based policy, but it’s hard to realize it when our most reputable social science journals are still publishing poor quality research

>Of course, academia has been aware of the replication crisis since at least the early 2010s, and practices and processes seem to have improved since 2018. A 2024 study led by Abel Brodeur found that 85% of papers in top economics and political science contained code that ran properly (with minor modifications) and produced the results stated in the paper. Much of this improvement is a result of top journals implementing policies to check that the code runs. However, while these policies have become more common in top journals (77% of the papers in this study were published in journals with such policies), they remain rare most other places. And of course, merely running and producing the results in the paper is the lowest possible bar to clear — and 15% of papers in our best journals still can’t clear it.

Holy shit 15% of the papers from the top economics and political science fields still can’t even manage to have working code in them. Not errorless codes, but *code that even manages to run* in the first place.