Data Submission
Validation and Upload | QA Procedures
NDA Data Validation and Upload Tool: Pre-Submission QA Procedures
Data must be harmonized to the NDA data dictionaries by either adapting your data to existing dictionaries, which typically is easiest done in the planning stages, or by extending the NDA data standard.
Once harmonized, data are validated against the Data Dictionary. This validation is done locally by the PI before uploading the data to NDA. If the data passes validation it may then be submitted by an authorized user (see Collection Setup sidebar for permissions). Three methods for data submission are:
- Validation and Upload Tool HTML Client: The recommended method for most users for depositing data in NDA is to format them according to the Data Dictionary definition in CSV files (templates are available from individual structure pages on the NDA website), and then use the Validation and Upload Tool in its HTML format to validate and upload them. This tool is available to validate data files without an active user account, but in order to upload you'll need to log in and designate a Collection into which you want to upload your data. This tool is the easiest and most appropriate method for most researchers to us for upload. NDA also maintains a tutorial on how to use it.
- Validation and Upload Tool Python Client: The tool is also available as a Python script and a Python package, which you can install on your machine directly (pip install nda-tools) or from our GitHub. This version allows you to validate, package, submit, and download data using command line arguments. It is compatible with both Python versions 2 and 3. Please contact us at NDAHelp@mail.nih.gov for more information about whether this version might be helpful to you.
- Validation and Submission API: This set of tools enables those with a programmatic background to use the NDA's available APIs for direct data submission. These methods allow for direct data submission from a scanner, sequencer, or research system.
Post-Submission QA Procedures
NDA ensures all data are harmonized to a common definition prior to submission through the Validation and Upload Tool. Post-submission, Quality Assurance (QA) checks are run across records ensuring basic information is accurate. Discrepancies are then sent to labs to be resolved. Checks for age variation, changes in sex, Personally Identifiable Information (PII) within imaging headers, and multiple identifiers assigned to the same participant are just a few of the post submission QA Procedures performed. NDA maintains a centralized document on the details of the Post Submission QA Process.