System Policies

WRDS Policies for working with our computing resources.

Web Queries

  • Downloads from the WRDS website are limited to 2 GB. When output becomes larger than this limit, the file is deleted and the summary page reports 0 kb.
  • Users may run 1 query at a time on the WRDS website.
  • Class accounts may run 15 simultaneous queries on the WRDS website.

If your download exceeds 2GB and is automatically deleted, consider breaking your query up into smaller queries. For example, if you have selected a date range of a year, instead perform two queries selecting 6 months each.

Top of Section

The WRDS Cloud

WRDS maintains a high-performance computing cluster -- the WRDS Cloud -- access to which is included free of charge with every WRDS subscription. The WRDS Cloud is a Linux computing grid shared among all WRDS subscribers. Usage of the WRDS Cloud is subject to the following Terms of Use.

Terms of Use

  • The WRDS Cloud is a shared resource. You agree to not engage in any research programming behavior that negatively affects other users. The WRDS Cloud features a job management engine called Grid Engine (qsub, qsas, qrsh, etc). Working around this system, tampering with it, or attempting to subvert it will result in termination of your WRDS account.
  • The WRDS Cloud, being a computing cluster, is comprised of head nodes and compute nodes. You submit jobs from the head nodes that then run on the compute nodes. The running of CPU- or RAM- intensive jobs on the head nodes -- which are designed only for job submission -- will result in a warning and your jobs being killed. You MUST use qsub, qsas, or qrsh to schedule your job on a compute node.
  • The WRDS Cloud offers plenty of storage space: both dedicated in your home directory, and shared amongst your institution in the scratch directory. ANY DATA YOU UPLOAD TO THE WRDS CLOUD IS YOUR OWN RESPONSIBILITY.
  • The WRDS Cloud is accessible from anywhere in the world, provided you have a valid WRDS account. As such, many automated network protection mechanisms are in place. If you attempt to connect to the WRDS Cloud many times within a very short period of time, your IP address might be automatically banned. If you fail to authenticate many times within a very short period of time, your account might be automatically disabled. Please contact WRDS Support for more information.

Disk Space & Storage

  • Every WRDS user is given 10GB of permanent storage in their home directory located at /home/institution/username
  • Subscribing institutions are given a further 500GB of additional temporary disk space at /scratch/institution that is shared among all users at that school. Files in this directory are deleted after 7 days.
  • Any data you upload to the WRDS Cloud is your own responsibility. Home and scratch directories are not appropriate for HIPAA, FERPA, or other classes of data that require special protection.
  • This space may not be used for sharing copyrighted materials without permission.

SASTemp Directories

The "sastemp" directories are provided for temporary, shared workspace storage, and are not intended for reliable permanent storage of files. Each sastemp directory is 2TB of working space. The output may be saved there at the completion of a program, but only to be IMMEDIATELY removed.
Any file in /sastemp which is either not in use by an active program or being transferred to a user's homeschool can be deleted without prior notice. Files on /sastemp are deleted automatically every 48 hours. Please note: Files kept in your home directory will never be deleted.

  • Files located here are not backed up!
  • Temporary work libraries created by SAS (i.e., SAS_work* subdirectories) are automatically removed when the associated SAS process has ended.
  • If anyone of the /sastemp directories is more than 95% full, the user accounting for the most space on that particular file system will have all of his/her files deleted from it without notice, regardless of file age.
  • Beyond the specific criteria listed above, WRDS may delete /sastemp files without notice if they are being used for long-term storage or a single user's files are preventing others from access to adequate Sastemp space. Fortunately, the latter has been a rare event, and there is often an opportunity to contact the owner before file deletion.

As such, /sastemp is generally to be used for temporary storage of data that your running research program is actively using, and never used for any long-term storage. Remember, this directory is shared with all other WRDS users!

Job Limits

  • Each user may run up to 5 concurrent jobs, and may queue any additional jobs. Additional jobs will wait in the queue until active jobs have finished.
  • Each subscribing institution may run up to 20 concurrent jobs across all users who are members of that institution, and may queue any additional jobs. Additional jobs will wait in the queue until active jobs have finished.
  • Each active job may run for one week (168 hours) of real time (wall clock). Jobs running after 168 hours have elapsed will be automatically terminated.

Resource Limits - Batch Jobs

Batch jobs additionally have the following CPU and RAM usage restrictions:

  • By default, a WRDS Cloud job is assigned to 2 cores (for a total of 4 threads) per job.
  • Users may request up to 8 cores (for a total of 16 threads) per job.
  • By default, a WRDS Cloud job is assigned 8GB per core (for a total of 16GB if left at the default core count).
  • Users may request up to 48GB RAM total per job (calculated as RAM * cores requested).

For more information on requesting additional CPU and RAM resources, please see Batch Jobs on the WRDS Cloud.

Top of Section


Table of Contents