Evidence & Best Practices:
Testing and Recording What Works

Best Practices

The growing ICOR community innovates, describes and implements new workflows, tools, and services that track and measure open access, data, and other outputs (see our ORB Project Library). We are working to advise and implement open practices in research settings with experimental designs that permit meaningful evaluation. The goal is to assemble a compendium of evidence, standards and best practices that keeps pace with innovations in the field.

Metascience

As metascience becomes a recognized practice for evaluating evidence and identifying biases, it’s important for stakeholders from different regions to collaborate, sharing expertise and resources that address common challenges. A unified approach to metascience can provide mechanisms to help us learn from what we are doing and establish a continuous improvement cycle for funding and conducting reproducible and impactful research. It will also provide trusted information for governments and policy makers to make more informed and evidence-based choices.

Experimental Design

ICOR works with research initiatives to advise, help formulate and implement experimental designs that will facilitate tracking, measuring, and analyzing the efficacy of open collaborative policies, tools and practices. Forward-thinking transdisciplinary and multi-institutional programs include Aligning Science Across Parkinson’s (see the ASAP Blueprint for Collaborative Open Science), the Michael J. Fox Foundation for Parkinson’s Disease, the Templeton World Charity Foundation, the Raynor Cerebellum Project, and the Howard Hughes Medical Research Institute.

Through this work and more underway, the following pillars were devised, along with a series of recommendations for each:

1. The Scaffolding: Clear and precise policies that include mandates
2. The Connective Tissue: Sharing of data, resources, and findings
3. The Signaling: Facilitating collaboration and communication with dedicated staff
4. Posting and Publishing: Building the official record
5. The Feedback Loops: Tracking and learning
6. The Motivators: Incentivizing and crediting all contributions
7. The Memory: Lowering risks and measuring success

Connecting and Curating

As this work continues, we aim to connect projects working on common infrastructures, tools, and practices in collaborative open research; curate a guide of best practices in research initiatives and meta-analysis; create a body of evidence and case studies regarding the impact of these efforts; and promote widespread use of best practices and how they can translate to incentives and rewards for collaborative open science.