User Download Threshold

NDA has put into effect a reasonable limit on the rate at which individual users can transfer data out of Amazon Web Services (AWS) storage to non-AWS internet addresses.

  • All users have a download limit of 20 Terabytes
  • This limit applies to the volume of data an individual user can transfer within a 30-day window
  • Only downloads to non-AWS internet addresses will be counted against the limit

This limit has been introduced to control infrastructure costs and to more fairly share finite resources for data storage and access across the NDA user base. The allowance of 20 terabytes is large enough for most exploratory data analyses or for prototyping a computational workflow on local infrastructure.

More importantly, data transferred to AWS internet addresses will not count towards this threshold. No download limit is imposed for data transferred this way. NDA provides direct support for accessing and analyzing data using AWS resources through a Computational Credits program.

Users that exceed this 20 Terabyte threshold will receive an email stating that their access to transfer data has been suspended and the date when data transfer will be available again. The email will also include the total volume transferred to non-AWS internet addresses. Data can be transferred again after the prior 30‑day download volume falls below 20 Terabytes.

Users that are concerned about their download volumes may write the NDA HelpDesk at any time to check their download volume over the past 30 days.

User Download Threshold Examples

A new user that has never accessed data creates a data package whose total volume is 70 Terabytes. The user’s account is created with a 20 Terabyte download threshold and the user has no downloads over the prior 30 days.

Example 1: The user begins downloading the entire data package to an internet address associated with a data storage server in their laboratory (outside of AWS) for data exploration and analysis. The user leaves this download to continue and begins working with the files that are initially downloaded.

If data are transferred at a rate of 1 Terabyte per day, the threshold will be reached after 20 days and the data transfer access will be suspended, at which point the download will be interrupted. After waiting another 11 days, the download volume over the prior 30 days will be 19 Terabytes, which is less than their allotted threshold and data transfer access will be restored.

If data are transferred at a higher rate of 10 Terabytes per day, the threshold will be reached after 2 days and the user will need to wait another 29 days before they will be able to transfer additional data.

Example 2: The user transfers data to compute instances or storage that has an AWS internet address. Their download volume will not count towards the download threshold and access to transfer data will not be suspended. Users can request an AWS account through NDA Computational Credits program.