Hello, thank you for your help.
I had asked on this list because I am a beginner with slurm so I
didn't know how to do this. Also reading at the CC documentation I
believed that installing a venv on each node was the right way to go.
Gareth was right: I needed to sleep forever at the end the worker
script, this works.
Regards,
Yann.
Mark Hahn <h...@mcmaster.ca> a écrit :
Actually, you are not allowed to install things in your home on
computecanada,
this is why you need to install everything in a virtualenv with pip
install. Also, you have to install each virtualenv in
$SLURM_TMDIR which is the
local drive of the node, because everything else is slow, so I
think I cannot
share homes.
How are you prevented from installing things in your home directory?
Just for the record, this is a misapprehension of ComputeCanada policy:
there is no such requirement, no such rule against using home directories.
We have some variation of policy among sites, but nothing more
extreme than home being RO (one site) or requiring jobs to be
submitted from (cwd)
one of the non-home shared filesystems (another site).
For some workflows, it can indeed be convenient to perform a from-scratch
install on a node-local disk. There is an obvious cost to doing this,
and indeed, the install would have to be performed via srun.
regards,
--
Mark Hahn | SHARCnet Sysadmin | h...@sharcnet.ca | http://www.sharcnet.ca
| McMaster RHPCS | h...@mcmaster.ca | 905 525 9140 x24687
| Compute/Calcul Canada |
http://www.computecanada.ca
