You should have the job script itself have the singularity/apptainer
command.
I am guessing you don't want your users to have to deal with that part
for their scripts, so I would suggest using a wrapper script.
You could just have them run something like: cluster_run.sh <path_to_script>
Then cluster_run.sh would call sbatch along with the appropriate commands.
Brian Andrus
On 2/7/2023 9:31 AM, Groner, Rob wrote:
I'm trying to setup the capability where a user can execute:
$: sbatch <parameters> script_to_run.sh
and the end result is that a job is created on a node, and that job
will execute "singularity exec <path to container> script_to_run.sh"
Also, that they could execute:
$: salloc <parameters>
and would end up on a node per their parameters, and instead of a bash
prompt, they have the singularity prompt because they're inside a
running container.
Oddly, I ran: salloc <parameters> /usr/bin/singularity shell <path to
sif> and that allocated and said the node was ready and gave me an
apptainer prompt...cool! But when I asked it what hostname I was on,
I was NOT on the node that it had said was ready, I was still on the
submit node. When I exit out of the apptainer shell, it ends my
allocation. Sooo...it gave me the allocation and started the
apptainer shell, but somehow I was still on the submit node.
As far as the job, I've done some experiments with using
job_submit.lua to replace the script with one that has a singularity
call in it instead, and that might hold some promise. But I'd have to
write the passed-in script to a temp file or something, and then have
singularity exec that. That MIGHT work.
The results for "slurm and singularity" do not describe what I'm
trying to do. The closest thing I can find is what slurm touts on
their website, a leftover from Slurm 2017 talking about a spank plugin
that, as near as I can figure, doesn't exist. I read through the OCI
docs on the slurm website, but it shows that using singularity with
that requires all commands to have sudo. That's not going to work.
I'm running out of ideas here.
Thanks,
Rob
