Is your article EV-TRACKed?

The EV-TRACK knowledgebase is developed to cope with the need for transparency and rigour to increase reproducibility and facilitate standardization of extracellular vesicle (EV) research. The knowledgebase includes a checklist for authors and editors intended to improve the transparency of methodological aspects of EV experiments, allows queries and meta-analysis of EV experiments and keeps track of the current state of the art. Widespread implementation by the EV research community is key to its success.

Introduction

The connection of extracellular vesicles (EVs) to many aspects of human health and disease, as well as to environmental ecosystem dynamics, attracted the attention of a large number of researchers from a wide range of disciplines. While these researchers are generally experts in their own areas, they are often not familiar with the best practices in EV research. The purification of EVs from complex biofluids however represents a considerable challenge and requires a clear understanding of the performance – the strengths but also the limits – of isolation and characterization methods to generate reliable and reproducible data.[1-6] The International Society for Extracellular Vesicles (ISEV) has sought to shed light on these obstacles through release of position papers in the Journal of Extracellular Vesicles and by issuing the Minimal Information for Studies on EVs (MISEV).[7] Of course, guidelines must evolve with the field, and a standardized method for evaluating experiments and manuscripts has been absent. In light of these needs, we launched the EV-TRACK database, EV-METRIC and online toolset, which were recently described in Nature Methods.[8]

How EV-TRACK and EV-METRIC work

The EV-TRACK online toolset is freely accessible at www.evtrack.org and consists of several features to coach researchers through the use of the EV-METRIC, to centralize data on EV characteristics and methods, to query research articles and to involve researchers in decision-making on future improvements to EV-TRACK and its EV-METRIC.

The EV-METRIC is a key feature, designed to reflect the level of scrutiny in validation experiments and reporting of experimental parameters. It is presented as a percentage of fulfilled components from a list of nine, which were argued by the EV-TRACK consortium to be indispensable for unambiguous interpretation and independent replication of EV experiments. Researchers are encouraged to obtain this EV-METRIC before submitting their manuscript for peer review (Box 1). After uploading requested experimental parameters, an EV-TRACK ID is assigned and a preliminary EV-METRIC is calculated. The authors can implement this metric to improve their manuscript. When including the EV-TRACK ID in the material and methods section of a manuscript, journal editors and reviewers will also be able to access the corresponding EV-TRACK data entry, which provides them with a comprehensive overview of the presented data. Upon publication, the data submitted in EV-TRACK are curated by the EV-TRACK administrators, the final EV-METRIC is calculated, and the experiment(s) is(are) included and searchable in the public knowledgebase.

In addition, EV-TRACK allows uploading of methodological parameters of already published experiments. A unique feature of EV-TRACK is the possibility to add unpublished methodological information of an already published EV experiment. For example, since the publication of EV-TRACK, researchers might realize that they forgot to include important experimental information. Since this information is generally available in the lab, it can be added post-publication to increase the reporting transparency.

To date, the knowledgebase includes 1240 articles that were published between 2010 and 2017. These publications can be queried for specific methodological parameters that are not easily searchable in any current biomedical literature database. In addition, by centralizing this information, the EV-TRACK knowledgebase creates a better understanding of EV biology and methodology, which is needed to develop the next generation of experimental guidelines, if and when they are required.

Why EV-TRACK and EV-METRIC are needed

The implementation of different methodologies requires validated controls and adequate reporting of experimental parameters. Failure to follow these principles can result in data that are difficult to interpret and reproduce, as has been reported for other fields in life science.[9] The MISEV guidelines as established by the ISEV board were an important first step in establishing standards for EV research. The average EV-METRIC mildly increased after publication of MISEV (from 19.8% pre-MISEV up to 24.7% post-MISEV; p = 0.004, Mann–Whitney U test) (Figure 1(a)). The average EV-METRIC of studies citing MISEV showed a stronger increase to 35.8% compared to 22.4% for those non-citing (p < 0.001, Mann–Whitney U test) (Figure 1(b)). This increase is mainly attributed to the analysis of non-EV enriched proteins and implementation of complementary methods for particle analysis (Figure 1(c)). The EV-METRIC was thus created as extra incentive for authors to take existing guidelines into account. It informs researchers, editors and reviewers unambiguously about whether or not EV experiments are transparently reported and allows for a better interpretation of EV experiments. Increased methodological rigour may translate into greater scientific value and reproducibility.

Figure 1.

Impact of MISEV guidelines on EV research practice. Scatter plots of EV-METRICS from studies (a) published before versus after publication of MISEV, and (b) citing versus not citing MISEV. The horizontal bars indicate mean EV-METRICs. (c) Spider chart of adherence to individual components of the EV-METRIC for EV studies, stratified for citing or not citing MISEV guidelines.

What EV-TRACK can do for EV researchers

The EV-TRACK online toolset is developed to help authors improve their manuscripts pre- or even post-publication. In many cases, the transparency of articles can be improved substantially by simple disclosure of methodologic parameters, without additional experimentation. EV-TRACK helps researchers to browse relevant EV research articles while familiarizing them with experimental guidelines. By centralizing methodological parameters, EV-TRACK can highlight reporting or characterization deficits that could warrant correction. The EV-METRIC will aid data miners of public databases by facilitating a focus on higher-compliance studies (for example, by linking EV-TRACK to Vesiclepedia). Furthermore, it is currently the only database that links sample types (including species of origin, cell lines, healthy vs. diseased), isolation methods and EV characteristics (size, density, protein markers). In time, it could serve as a valuable resource for meta-analyses of EV research data.

What EV researchers can do for EV-TRACK

It is of critical importance that EV-TRACK and EV-METRIC do not represent the opinion of a select group of researchers, but rather a consensus opinion regarding the best parameters to increase transparency, rigour and reproducibility in EV research. EV-TRACK and EV-METRIC are much more likely to be embraced by the field if a consensus is reached on the experimental parameters that are indispensable for unambiguous interpretation and independent replication of EV experiments by the majority of researchers performing these experiments. Therefore, we stimulate EV researchers to make active contributions to the EV-TRACK online toolset by (1) submitting methodological parameters of pre-submission EV experiments and include EV-TRACK IDs and EV-METRICs in accepted manuscripts for publication; and/or (2) providing recommendations to further improve EV-TRACK and EV-METRIC. This will ensure that at any time the EV-TRACK knowledgebase, including experimental guidelines as represented by the EV-METRIC, will reflect the current state of the art in the research field and evolve according to the field’s needs, to the benefit of all researchers. Active contributions by researchers are recorded and will result in a request to become EV-TRACK consortium member.

Where to go from here

Future updates that we envision for the EV-TRACK online toolset are summarized in Table 1. The EV-TRACK consortium will discuss the inclusion of additional experimental parameters which will most probably be driven by novel EV isolation and quantitation technologies, the use of EVs in functional assays and the widespread use of omics-approaches. Potential examples are the type of RNA isolation kit, RNase treatment, and protein assay kit.[3,11] The use of specialized quantitative techniques (e.g. nanoparticle tracking analysis, tunable resistive pulse sensing, high-resolution flow cytometry) will warrant the inclusion of specific parameters related to these technologies. The increased use of EVs in functional experiments urges the creation of guidelines about EV treatments in cell culture/animal models. Parameters to consider are treatment dose, the number of recipient cells, kinetics of EV treatments and serum depletion of cells, among others, some of which are already searchable through EV-TRACK.[12]

Table 1.

Future updates.

EV-TRACK feature	Vision
Upload	Continuous evaluation of the upload system via user feedback Add data of “missing” articles published in 2015–2016
EV-METRIC	Improved metric dynamics: scarce samples, new sources Conditional guidelines depending on: Study aim (e.g. data upload to Vesiclepedia [10] for omics studies) Sample type (e.g. use of EV-depleted serum for cell culture supernatant) Mode of EV detection (e.g. guidelines for isolation-independent super-resolution microscopy or high-resolution flow cytometry) Include a priori restrictions on (non) EV-enriched proteins that should be assessed
Coaching	Implementation of EV-METRIC system by journals and funding agencies Disseminate EV-TRACK via endorsement by ISEV
EV biology	Include functional assay details Link EV-TRACK to existing EV-related databases (e.g. Vesiclepedia [10])
Methods	When necessary, include relevant parameters to accommodate new and upcoming methods
Query	Continuous expansion of search options and representations of search results according to user feedback
Community	Implement online forum for feedback on the EV-TRACK platform Tailor new features to EV-TRACK users’ recommendations

Journals have the greatest potential for immediate and significant impact on methodological rigour and reporting and, by extension, reproducibility [13]. The EV-TRACK consortium will contact journals to include the EV-METRIC in author’s guidelines to stimulate transparent reporting of EV experiments.

With a member survey in autumn 2016, ISEV has started the process of enhancing the MISEV guidelines. EV-TRACK and EV-METRIC will be an important part of this process as the ISEV membership is involved more closely in decisions.

Conclusion

EV-TRACK and the EV-METRIC are new tools to enhance transparency and interpretation of EV experiments. An increase in reporting rigour will benefit overall reproducibility in EV studies. Most importantly, EV-TRACK is driven by both the input and consensus of researchers that perform the experiments.

Comment on:

EV-TRACK: Transparent reporting and centralizing knowledge in extracellular vesicle research

EV-TRACK Consortium, Van Deun J, Mestdagh P, Agostinis P, Akay Ö, Anand S, Anckaert J, Martinez ZA, Baetens T, Beghein E, Bertier L, Berx G, Boere J, Boukouris S, Bremer M, Buschmann D, Byrd JB, Casert C, Cheng L, Cmoch A, Daveloose D, De Smedt E, Demirsoy S, Depoorter V, Dhondt B, Driedonks TA, Dudek A, Elsharawy A, Floris I, Foers AD, Gärtner K, Garg AD, Geeurickx E, Gettemans J, Ghazavi F, Giebel B, Kormelink TG, Hancock G, Helsmoortel H, Hill AF, Hyenne V, Kalra H, Kim D, Kowal J, Kraemer S, Leidinger P, Leonelli C, Liang Y, Lippens L, Liu S, Lo Cicero A, Martin S, Mathivanan S, Mathiyalagan P, Matusek T, Milani G, Monguió-Tortajada M, Mus LM, Muth DC, Németh A, Nolte-’t Hoen EN, O’Driscoll L, Palmulli R, Pfaffl MW, Primdal-Bengtson B, Romano E, Rousseau Q, Sahoo S, Sampaio N, Samuel M, Scicluna B, Soen B, Steels A, Swinnen JV, Takatalo M, Thaminy S, Théry C, Tulkens J, Van Audenhove I, van der Grein S, Van Goethem A, van Herwijnen MJ, Van Niel G, Van Roy N, Van Vliet AR, Vandamme N, Vanhauwaert S, Vergauwen G, Verweij F, Wallaert A, Wauben M, Witwer KW, Zonneveld MI, De Wever O, Vandesompele J, Hendrix A.

Nat Methods. 2017 Feb 28;14(3):228–232.

Box 1. EV-TRACK: how to

Create a login at www.evtrack.org Upload your manuscript data (sample, isolation, characterization) using the online submission tool Include the assigned EV-TRACK ID and EV-METRIC in your manuscript to facilitate peer review If necessary, modify your entry after the peer review process to make the data reflect the latest version of the manuscript Provide feedback on different features of online toolset Become EV-TRACK consortium member