Activating Open Research

Highlights of ICOR Public Meeting #3

“Activating Open Research” held Dec 13, 2023

Michael Markie, Guest Host

Please see the streaming video and chat record for rich details from speakers and meeting attendees

Community members: suggest topics or volunteer to host future public meetings on this Google Form

Introduction

Open research aims to make the entire research life cycle accessible to all and addresses policies, technology, standards, practices, and procedures. Considerable resources and energy are required for successful implementation, and often success depends on coordination of multiple organizations and initiatives.

Kristen Ratan first explained how ICOR seeks to make these connections and an announcement of a newly launched Canadian initiative. Then guest host Michael Markie moderated talks by four experts who offered the following differing perspectives on approaches and challenges toward activating open research:

  • Dr. Rebecca Taylor-Grant, F1000 Research, on responses to publishing requirements
  • Dr. Malika Ihle, LMU Open Science Center, on multidisciplinary efforts in academia
  • Dr. Nikola Stikov, Canadian Open Neuroscience Platform/NeuroLibre, on nationally funded infrastructures
  • Dr. Matt Lewis, Aligning Science Across Parkinson’s (ASAP), on a funder’s mandates and incentives

Activating the ICOR community

Kristen Ratan, ICOR Co-Founder/Director

2023 has been a landmark year for our community, with the US government declaring it as the “Year of Open Science”, involving “… a multi-agency initiative across the federal government to spark change and inspire open science engagement through events and activities that will advance adoption of open, equitable, and secure science.” Accordingly, in this past year we’ve seen increased round tables, conferences, and funding from federal and private funding organizations. For instance, at least some funders are promoting experiments on incentives and rewards, which was the topic of our second public meeting held in July. Because ICOR is focused on collaboration in addition to open research, with new incentives for team science now being tested, it is critical in 2024 to align ourselves around best practices, standards and metrics in this and other arenas.  

To further this goal, we are establishing ICOR as a bold community effort, starting with our first guest host/moderator. Michael Markie developed this meeting’s agenda with expert speakers addressing the highly relevant topic, Activating Open Research. We are looking for additional ICOR meeting hosts this year to curate agendas and enlist speakers, and hope you’ll add your ideas to this Google Form. Topic ideas, such as community-led peer review, the roles of AI, and human-centric data usage, have also been recorded in the meeting chat and as feedback on the meeting agenda doc. 

Before turning to the meeting topic, Dylan Wade Roskams-Edris, a founding ICOR strategist, briefly described his involvement in an exciting new initiative. 

Dylan Wade Roskams-Edris, Conscience Network Lead, Education and Events

Conscience is an organization founded recently with the help of a $50 million grant from the Canadian government. The mission is to create a drug discovery ecosystem in Canada as well as globally in areas of market failure. Conscience will bring together organizations working in rare diseases, vaccines, and many neurological and pediatric diseases to determine whether an open path to discovery will succeed where traditional closed models have failed. Dylan promoted the recently announced Critical Assessment of Computational Hit Finding Experiments (CACHE) competitions encouraging others to learn more and sign up for the challenges on the CACHE Challenge website, and join the upcoming symposium on March 6-7 in Toronto.

Activating Open Research

Overview

Michael Markie, Guest Moderator

Coming from a publishing background, Michael has witnessed first-hand the difficulties of encouraging and enforcing open research. For this meeting, the enlisted speakers covered institutional, national, and international successes and challenges in activating open research. All of them have participated in innovative approaches to implementing and cultivating open research initiatives. All the speakers’ slides have been shared  and their full presentations and the Q/A discussion can be accessed through the meeting video recording.

Publisher’s perspective on researchers’ attitudes to sharing data

Rebecca Taylor Grant, F1000, Head of Open Data Initiatives [slides]

Rebecca shared her experience in enforcing F1000’s  comprehensive and strict data sharing policy. Authors are required to deposit their data in accessible repositories with open licensing (CCBY or CC0); as well, they must cite the data in their reference list. Peer reviewers are required to access and assess the datasets, an unusually strong requirement for a publisher. Rebecca provided data, evidence and observation regarding these requirements in comparison with other journals’ policies and discussed the State of Open Data 2023 global survey of 6000+ researchers’ attitudes toward data sharing. A few takeaways from the presentation was:

  • Data repositories are rarely used by authors unless there is a mandate
  • The primary incentive for open data sharing is increased citation of datasets and thus greater impact of authors’ research 
  • Making it easier to share data was ranked fairly low as an incentive in the survey – i.e., authors believe they are well served by existing tools and processes such as finding repositories, writing availability statements, obtaining DOIs, and licensing (reiterated in the 2021 survey as shown in the slides)
  • Researchers are generally not interested in following the nuances of publishers best practices and thus expert support is needed from journal publishers
  • F1000 has a well-developed set of guidelines, tool kit, and a new submission system that emphasizes data sharing – collectively this is helping to improve data sharing on the platform.

The primary incentive for open data sharing is increased citation of datasets and thus greater impact of authors’ research

Governing and implementing open research at an academic institution

Malika Ihle, Coordinator, Ludwig Maximilian University of Munich (LMU) Open Science Center (OSC) [slides-pdf]

The LMU OSC is a research-led, university-wide grassroots initiative established in 2017, with the aim of fostering and coordinating open research across the entire university. Currently there are ~90 members in the support group, and ~20 early-career researchers who are Fellows that co-lead grassroots initiatives to ~500 participants at the university. The OSC’s structure is set up to:

  • Simplify new practices and policies for adoption through peer to peer training 
  • Coordinate symposia featuring international presenters, training sessions, workshops that train students and postdocs 
  • Conduct meta research to assess and evaluate research practices, to inform the design of new incentives and evidence-based policies.
  • Training and lectures cover how to: develop a statistical plan prior to collecting data to prevent bias in analyses; create computationally reproducible workflows and how to spot mistakes in data reporting; adhere to FAIR principles; conduct systematic reviews.
  • Build community and network through institutional membership in The Carpentries, a non-profit that specializes in teaching basic software, data, and library skills to researchers
  • Liaise with stakeholders such as the German and UK Reproducibility Networks, the Coalition for Advancing Research Assessment (CoARA), and NASA -CERN-ORFG evidence-based open science policies working group.

Malika also summarized a recent eLife study discussing eleven actions that can be taken to make reproducible research and open science training the norm at research institutes. 

Enabling open sharing of neuroscience data through nationally supported infrastructure

Professor Nikola Stikov, Canadian Open Neuroscience Platform (CONP)and NeuroLibre preprint server for reproducible data analysis [slides]

Nikola, a neuroscientist and specialist in MRI, shared a high-level overview of the CONP. Built on a multi-million grant from academic and nonprofit partners in Canada, the platform provides an infrastructure for the promotion of open-science workflows and the sharing of neuroscience data. Nikola focused his discussion on the NeuroLibre platform that is focused on data sharing within CONP and beyond. His presentation covered the work of Extracting Knowledge from Academic Publications, by Markus Strasser, which states “Close to nothing of what makes science actually work is published as text on the web”. It was also stated as far back as the 90s by Buchheit and Donahoe that a scientific publication is not the scholarship itself, but is merely an advertisement of the scholarship… the real thing is the software development environment and the complete set of instructions that generates the figures.

“Close to nothing of what makes science actually work is published as text on the web”

 – Markus Strasser

Building upon this, Stikov and colleagues published in PLOS Computational Biology (2021) “Beyond advertising: New infrastructures for publishing integrated research objects” which describes the infrastructure behind NeuroLibre, a reproducible or living preprint server, whereby publications incorporate data, code, interactive figures and text – i.e., a repository of neuroscience notebooks (Jupyter) using  powerful cloud computing infrastructure. 

Nikola discussed in detail the five layers to a NeuroLibre publication: 

  1. PDF compatible document: can be printed and read like any other publication, but exists mainly for registration purposes.
  2. Dynamic figures: you can observe the actual data points.
  3. Interactivity: permits exploration of the phenomenon and with real-world data.
  4. Transparency: you can see the code e.g., in Github that generates the outputs.
  5. Reproducibility: you can run the code that generates the outputs.

Reflections on a funder’s mandates and incentives for open, collaborative research

Matt Lewis, Program Officer, Aligning Science Across Parkinson’s (ASAP) [slides]

ASAP is a relatively new program that was launched five years ago through private philanthropic capital with the aim to accelerate the pace of discovery and inform a path to cure Parkinson’s disease. There is a strong focus on collaborative, open research with mandates, incentives, and resources in place to achieve an open approach.  The Collaborative Research Network (CRN) includes 35 funded teams, each with 3 to 5 PIs across 80 institutions; team science being done at an international scale. The goal is to integrate existing Parkinson’s disease researchers in diverse disciplines: genomics, immunology, and the circuitry of brain-body (or peripheral nervous system) interactions.  

Stringent policies regarding collaboration and openness are communicated to research applicants and reinforced throughout the grant period:

  • All work posted in a preprint repository prior to submission to a journal
  • Immediate free online access upon publication with a CC BY 4.0 license for unrestricted reuse
  • All outputs (data, code, lab materials, protocols) deposited in publicly accessible repositories and cited in the publication with permanent identifiers (PIDs)
  • Use of the Research Output Management System (ROMS) to track all outputs within the ASAP Hub, making them accessible to fellow grantees 

It is recommended that grantees connect with open science communities such as ICOR, for real culture change to occur.  

Matt also covered some of the features of the ASAP CRN that promote collaboration:

  • The ASAP Hub is a communication infrastructure for learning about the team researchers, their expertise and projects, their lab resources (which they are required to share), and their outputs
  • Moderated and recorded interest group and working group meetings that bring together a mix of teams and topics discussing their approaches and progress 
  • Community generated templates and standards for sharing data
  • Compliance reviews whereby grantees receive feedback and training in open science best practices while registering their research outputs
  • Project managers for each team, paid by the grant to assist in inter- and intra-team communication and policy adherence

Matt explained that many of the grantees are not skilled in the technical aspects of sharing data and the resulting inertia is the biggest roadblock to getting outputs into the proper format for the computational pipeline. To address this, ASAP has implemented a pilot project with CatalystNeuro, a data engineering and software firm that specializes in working with neurophysiology labs, specifically on an NIH-supported file format called Neurodata Without Borders.  It is also recommended that grantees connect with open science communities such as ICOR, for real culture change to occur.  Matt then pointed to ASAP’s Blueprint for Collaborative Open Science and their recently published article in PLOS Computational Biology – From policy to practice: lessons learned from an open science funding initiative.

See the streaming video and chat record for more rich details from these speakers and meeting attendees

Community members: suggest topics or volunteer to host future public meetings on this Google Form

Featured image by Martin Martz on Unsplash