Epiverse TRACE: A values-based approach to open-source ecosystems

By Anna Carnegie

What is meant by a ‘healthy’ open-source community, and how do we capture it? These are the questions we’ve been asking within the Epiverse TRACE community. Built by an inclusive, global community of contributors, Epiverse TRACE comprises a suite of trustworthy, open-source, epidemiological software tools accessible to users across sectors. The intent is to enable the development of rapid, robust, and reproducible policy-relevant modeling.

In June 2022, the Epiverse TRACE community established blueprints for software development, considering not only technical elements (such as coding conventions), but also crucially, human interactions and the experience of participating in an open-source community. ICOR is working with us on tracking and measuring our goals towards this vision.

Metrics will become targets: make sure they mean something

An inspiration for our discussion were the metrics and analytics being developed for open source community health by our colleagues at CHAOSS. They believe that metrics selected for one’s community will inevitably become targets, which means that the values underpinning the chosen metrics are critical. This was the impetus to begin formally defining the values we wished to inform the metrics for gauging the initiative’s collective success.

The values underpinning a community’s chosen metrics are critical.

Securing collective buy-in of core values

The selection of Epiverse TRACE core values involved the following stepwise actions, which took place over four months:

  • Team audit of project documentation and discussions to date, plus a review of CHAOSS focus areas as a potential library of metrics to employ
  • Draft documentation of values derived from the audit, indicating for each the purpose and potential metrics and data collection measures
  • Circulation of draft document to the full Epiverse TRACE staff: 40 individuals with diverse backgrounds across four academic institutions and our funding partner, data.org
  • In-depth interrogation of potential values, taking place over several weeks, with additional values proposed
    • For example, an exploration of  the concepts of usability versus utility led to the determination that both were distinctive and crucial
  • Clarification and selection session, attended virtually by Epiverse TRACE staff across all partner sites, narrowed core values to 13 possibilities with the aim of selecting 7 to 8
  • Final vote by team members resulting in a list of 8 core values selected and adopted

Epiverse-TRACE community core values

The following list of values will represent the Epiverse-TRACE community, with metrics built out accordingly:

  1. Inclusion – ensuring that our community spaces are inclusive and welcoming to all.
  2. Sustainability of the ecosystem beyond the project’s grand-funded lifetime.
  3. Reciprocity – operating from a stance of give and take.
  4. Practicality of Epiverse TRACE tools and materials for end users – both in terms of usefulness and usability.
  5. Quality within the code development process
  6. Timeliness of the tools developed (e.g., on a regular basis)
  7. Accountability to our community, our funders and each other.
  8. Environmentally conscious – taking steps as a project to minimise the negative consequences of our actions on the climate.

We must now define appropriate metrics for capturing these values, incorporating ‘hard’ and ‘soft’ measures

Next steps: metrics, communication, and ongoing assessment

We must now define appropriate metrics for capturing these values, taking care to incorporate both ‘hard’ and ‘soft’ measures. For example, when considering inclusion, in addition to recording the identity of those in attendance at virtual events and seminars, taking into account those who do or don’t feel empowered to speak.

Each of the core values will be embedded in our discussions, communications, and documentation. Embodying them will require accountability – i.e., admitting errors and making unpopular or difficult decisions. However, we anticipate that our systematic community approach will ameliorate these difficulties.

And as our community grows, there will be periodic reviews to capture new insights and wider perspectives, as well as to question the continuing relevance of each value. We see this as an ongoing conversation, one which we hope will be as engaging as the discussions to date.

To get involved with the community, please complete this short form.

Anna CarnegieAnd in the spirit of accountability, please contact me if you have thoughts on how we can improve our performance as it relates to our values framework.