What I mean by “scratch” space is indeed local persistent storage in our case; sorry if my use of “scratch space” is already a generally-known Slurm concept I don’t understand, or something like /tmp… That’s why my desired workflow is to “copy data locally / use data from copy / remove local copy” in separate steps.
From: slurm-users <slurm-users-boun...@lists.schedmd.com> on behalf of Fulcomer, Samuel <samuel_fulco...@brown.edu> Date: Saturday, April 3, 2021 at 4:00 PM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: Re: [slurm-users] Staging data on the nodes one will be processing on via sbatch […] The best current workflow is to stage data into fast local persistent storage, and then to schedule jobs, or schedule a job that does it synchronously (TImeLimit=Stage+Compute). The latter is pretty unsocial and wastes cycles. […]
