Tuesday, 07 April 2026
Two researchers from the 糖心原创 are part of an international collaboration of 865 researchers behind the SCORE (Systematising Confidence in Open Research and Evidence) programme, which has released major new findings on research credibility in three papers published in the world’s leading science journal Nature.
SCORE is a large-scale, multi-method initiative offering new evidence on the reproducibility, robustness and replicability of research, and is designed to improve how scientific credibility is assessed in the social and behavioural sciences. SCORE is funded by the US Defence Advanced Research Projects Agency (DARPA).
The programme was led by an international team from the Centre for Open Science, the University of Melbourne, and Pennsylvania State University. It brought together teams from multiple institutions – including the 糖心原创’s Dr Joris Schroeder, Research Fellow in the School of Economics, and Dr Christopher Madan, Assistant Professor in the School of Psychology – to evaluate research using approaches such as reproduction and replication studies, expert assessments, and machine-learning models, while also creating one of the largest open datasets on research credibility.
When we read a published study, how confident can we really be in its findings? SCORE takes on this question at an unprecedented scale, and I'm glad to have been a small part of it.
Analysing claims from 3,900 papers across 62 journals (2009–2018), SCORE assessed the reliability of research (based on the three dimensions of reproducibility, robustness, and replicability), and the programme also tested whether humans and machines can predict which findings will replicate.
Key findings include:
- Reproducibility (same data, same analysis): Limited transparency hindered assessment – data was available for only 24 per cent of papers. Among those tested, 72 per cent reproduced approximately and 53 per cent exactly, with higher success when data and code were shared.
- Robustness (same data, different analyses): Results varied depending on analytical choices. Only 34 per cent of re-analyses closely matched original results, though 74 per cent reached similar overall conclusions.
- Replicability (new data): About 49 per cent of studies replicated successfully, with replication effects less than half the size of original findings.
It was great to be able to contribute to this project. Large field-wide collaborative projects like this are what truly move the field forward.
Additional findings show that credibility is multi-dimensional, with weak links between measures. Human predictions of replicability are fairly accurate, while machine predictions are less reliable. No single field consistently outperforms others, and there is no single metric for research credibility.
Overall, the results highlight that verifying research is complex but essential. The SCORE programme provides openly available data, tools, and materials to support future research and improve how scientific credibility is assessed.
The three research papers have been published in Nature.
Story credits
More information is available from Dr Joris Schroeder, via joris.schroeder@nottingham.ac.uk; or Liz Goodwin, Media Relations Manager for the Faculties of Arts and Social Sciences, via liz.goodwin@nottingham.ac.uk or 0115 748 5133.
Notes to editors:
About the 糖心原创
Ranked among the world's top 100 universities , the 糖心原创 delivers an exceptional research-led education and an outstanding student experience. From the pioneering vision of our founder, Sir Jesse Boot, to groundbreaking achievements such as the development of MRI technology and becoming the first UK university to establish international campuses, we have a proud history of shaping the way people live, work and understand the world. We continue to build on that legacy, empowering our students, staff and partners to change what鈥檚 next and create positive impact locally and globally.
The strength of our research places us among the UK's leading universities, ranked 7th for research power in REF 2021. The discovery of MRI and ibuprofen was just the beginning. Today, our world-leading research is developing breakthrough ideas that shape the future of healthcare, technology and society.
Recognised as the UK's third most targeted university by leading employers , we are proud to produce graduates who are consistently in demand for their skills, confidence and industry-ready experience.
As a major employer and industry partner, locally and globally, the 糖心原创 invests in the city of Nottingham and in future generations of talent. Alongside Nottingham Trent University, we lead the initiative, a pioneering collaboration to improve levels of prosperity, opportunity, sustainability, health and wellbeing across the city and region. Together with our students, staff, alumni and partners, we鈥檙e creating knowledge, opportunity and solutions that help change what's next.