Resource track
Scope
This MELBA Resource track is dedicated to resources that can facilitate i) reproducible research within the field of biomedical imaging, and ii) fair benchmarking of machine learning algorithms.
MELBA Resource publications are manuscripts that systematically describe publicly available resources, such as i) research datasets (i.e., Data Resource), and ii) open-source software tools (i.e., Tool Resource), that reside at the intersection of biomedical imaging and machine learning.
The aim of MELBA Resource manuscripts is to enhance the sharing and reuse of resources, encourage their broader dissemination and repurposing, as well as acknowledge those who contribute by sharing.
Format of MELBA Resource manuscripts
Manuscripts submitted for consideration to the MELBA Resource track should adhere to the following structure:
- Title
- Abstract
- Background
- This section should provide information about:
- The background and motivation for the specific resource.
- The gap/need in knowledge that makes the presented resource valuable for ML/AI projects.
- Similar resources, further highlighting the unaddressed gap or complementing current existing resources.
- This section should provide information about:
- Summary
- This section should summarize:
- The vision, mission, scope, and objectives of the resource.
- The importance of the resource, e.g., new data of underserved populations.
- The target audience whom the resource serves, e.g., clinicians.
- The resources/parameters needed to use this resource.
- The readiness of the resource for use in a medical imaging ML/AI project. For example, is the presented dataset AI-ready? Does the use of the presented tool require programming knowledge? Is the presented tool executable in a low-code or zero-code way?
- This section should summarize:
- Discussion
- Resource Availability
- Data/Code Location
- This section should only include the resource’s webpage address from a permanent public repository. (Note that DOIs are preferred.)
- Examples of non-commercial repositories are given in the Public Repositories section below.
- Potential Use Cases
- This section should indicate the potential ML/imaging use cases that the authors envision their resource facilitating and the specific potential clinical domain(s).
- Licensing
- Clearly describe the licensing terms.
- If there is an existing licence that is followed this could just be a single sentence. Examples of acceptable licenses include but are not limited to CC-BY, CC-BY-NC, Apache v2, MIT.
- Ethical Considerations
- Language referring to associated documentation, e.g., IRB, Informed Consent.
- For human data, this section must describe the subjects’ consent or opt-in/opt-out process.
- (For Data Resource manuscripts only): Resources related to derivatives of existing data should be accompanied by a clear description of how the provided derivatives comply with the original license.
- Data/Code Location
- Methods
- This section should highlight all methods/techniques/pipelines that the authors used to prepare the presented data (for Data Resource manuscripts) or that were included in the presented tool (for Tool Resource manuscripts).
- Data Details
- This section should indicate inclusion criteria (both subject recruitment and subset selection for inclusion in the resource), as well as describe any associated demographic, clinical, molecular, and other relevant related details.
- Methods Used for the Data Creation
- This section is only relevant for Data Resource manuscripts and should describe:
- All methods and equipment used from data acquisition to final processing.
- The AI-ready state of the dataset. Is it ready to be ingested by a DL algorithm? If not, what are the potential steps that one needs to take for this?
- This section is only relevant for Data Resource manuscripts and should describe:
- Software Stack
- This section is only relevant for Tool Resource manuscripts.
- A textual description and a graphical illustration must be included here.
- Flowchart(s) of the Tool’s Process Flow
- This section is only relevant for Tool Resource manuscripts.
- For example, this flowchart could be indicative of the ML training procedure in this tool.
- This flowchart does not necessarily need to be a formal Data Flow Diagram.
- Functionalities (For Tool Resource manuscripts)
- This section is only relevant for Tool Resource manuscripts.
- Validation
- Description of Approaches Ensuring the Quality of Data
- This section is only relevant for Data Resource manuscripts.
- Subsections included here should correspond to the “Methods Used for the Data Creation” subsection above.
- Experimental Design & Results
- This section is only relevant for Tool Resource manuscripts.
- Indications of the reproducibility of the code.
- CI/CD and Unit Test code coverage.
- Example data from published or unpublished studies.
- Description of Approaches Ensuring the Quality of Data
- Conflicts of Interest
- Acknowledgments
- Funding
- References
MELBA Resource vs Other Publications
MELBA Resource publications are intended to supplement conventional research publications. The release of MELBA Resource manuscripts will not be regarded as diminishing the originality of subsequent MELBA manuscript submissions. Authors are encouraged to consult with editors from other journals to ensure that the publication of a MELBA Resource does not compromise the submission of related research manuscripts to those journals.
Authors may also submit a MELBA Resource manuscript detailing data/tools previously made publicly available or analyzed in prior research manuscripts, provided there is no existing publication specifically describing the resource. In case of an existing publication, a new publication might be submitted only if it significantly enhances the resource's reuse potential or presents a substantial extension of the previously published resource. By releasing the described shared resource, authors automatically confirm their full responsibility for sharing it.
Any pre-existing publication, or manuscript under consideration or in press at any journal, relevant to the MELBA Resource manuscript must be mentioned and explicitly cited in the submitted manuscript, as well as discussed in the accompanying cover letter.
Submission Considerations
A MELBA Resource submission consists of the manuscript following the format described above, and a structured cover letter with statements that must be included in every submission.
Authors submitting a MELBA Resource manuscript are required to deposit their data/tools in a suitable external public repository to ensure the discoverability, utility, reproducibility, and reusability of the resource (see example Public Repositories below). Sole use of a personal or institutional website is not considered adequate. Authors should furnish their materials at a level that facilitates broad utility, encompassing both computed, or curated data, and observed (source) data. For clarity, by ‘observed (source) data’ we refer to observations generated at the source, i.e., a health system. By ‘computed data’ we refer to derivatives of observed data, generated following some computation, e.g., texture features. By ‘curated data’ we refer to observed data that have undergone some curation, for example, co-registered MRI of brain scans.
Should the resource not be publicly accessible at the time of submitting a MELBA Resource manuscript, authors are required to furnish secure links and/or passcodes to access the resource. This allows the editorial team and reviewers to confidentially access and assess the data. Authors are advised against providing personal login details and are not allowed to track individual logins that would compromise the anonymous reviewer evaluation. The editorial team and reviewers commit to treating the manuscript and its associated data/tools with strict confidentiality, utilizing them solely for the purpose of manuscript evaluation prior to publication.
In the event that the data/tools are not accessible at the original repository, authors are obligated to upload the data to another repository and issue an update to the original MELBA Resource manuscript. If the authors fail to respond accordingly, the editorial team reserves the right to retract the publication.
MELBA maintains the right to retract a Resource manuscript post-acceptance if significant issues with the scientific content or violations of our publishing policies emerge.
Public Repositories
The MELBA Resource manuscripts (both Data Resource and Tool Resource) should always be accompanied by a reference to a repository of the described data/tools to enable their long-term preservation and public access (even if this requires a data use agreement), without paywalls, and support open licenses (such as CC-BY, CC-BY-NC, Apache v2, MIT). Specifically for Tool Resource manuscripts, support for versioning and the library dependencies should also be considered. Example repositories are:
- Synapse
- The Cancer Imaging Archive (TCIA)
- Neuroimaging Informatics Tools and Resources Collaboratory (NITRC)
- Zenodo
- EBRAINS
- National Addiction & HIV Data Archive Program (NAHDAP)
- Dryad Digital Repository
- Science Data Bank
- NeuroMorpho.org
- G-Node
- Harvard Dataverse
- UK Data Service
- Image Data Resource
- ClinicalTrials.gov
- OpenNeuro (formerly OpenfMRI)
- SICAS Medical Image Repository (formerly Virtual Skeleton Database)
- National Database for Autism Research (NDAR)
- Open Science Framework
- National Database for Clinical Trials related to Mental Illness (NDCT)
- PhysioNet
- figshare
- Research Domain Criteria Database (RDoCdb)
- NCI Imaging Data Commons