Re: [slurm-users] SLURM_JOB_NODELIST not available in prolog / epilog scripts

2018-03-04 Thread John Hearns
Dan, completely off topic here. May I ask what type of simulations are you running? Clearly you probably have a large investment in time in Trick. However as a fan of Julia language let me leave this link here: https://juliaobserver.com/packages/RigidBodyDynamics On 5 March 2018 at 07:31, John He

Re: [slurm-users] SLURM_JOB_NODELIST not available in prolog / epilog scripts

2018-03-04 Thread John Hearns
I completely agree with what Chris says regarding cgroups. Implement them, and you will not regret it. I have worked with other simulation frameworks, which work in a similar fashion to Trick, ie a master process which spawns off independent worker processes on compute nodes. I am thinking on an

Re: [slurm-users] SLURM_JOB_NODELIST not available in prolog / epilog scripts

2018-03-04 Thread Christopher Samuel
On 05/03/18 12:12, Dan Jordan wrote: What is the /correct /way to clean up processes across the nodes given to my program by SLURM_JOB_NODELIST? I'd strongly suggest using cgroups in your Slurm config to ensure that processes are corralled and tracked correctly. You can use pam_slurm_adopt fr

Re: [slurm-users] SLURM_JOB_NODELIST not available in prolog / epilog scripts

2018-03-04 Thread Dan Jordan
Sorry, you are right, the documentation is clear about it being available only in EpilogSlurmctld. I'm quite new to SLURM and I've read some of the documentation, but obviously I haven't grasped it all. I don't quite understand the difference between --epilog vs --task-epilog, EpilogSlurm vs. Epil

Re: [slurm-users] sbatch --immediate

2018-03-04 Thread Christopher Samuel
On 03/03/18 08:32, Andy Riebs wrote: Is this trying to say that the sbatch option isn't as powerful as the option of the same name for srun and salloc, or is it saying that the option might not always work? The former I suspect. That text was added with: commit e063642d555d938b07a2c4ac9fcde9a

Re: [slurm-users] SLURM_JOB_NODELIST not available in prolog / epilog scripts

2018-03-04 Thread Christopher Samuel
On 05/03/18 10:16, Dan Jordan wrote: In my particular case, I need SLURM_JOB_NODELIST, which should be available but it is not. This is only available in PrologSlurmctld, not Prolog, according to those docs. Does that match what you're trying? cheers, Chris

[slurm-users] SLURM_JOB_NODELIST not available in prolog / epilog scripts

2018-03-04 Thread Dan Jordan
According to the documentation , I should be seeing far more environment variables in the context of prolog/epilog scripts. I am currently running version SLURM 17.11.2 and have noticed that only these are available to me: SLURM_JOB_NAME SLURM_PRIO_PR