Could be this quote from the srun man page:
-u, --unbuffered
By default the connection between slurmstepd and the user launched application
is over a pipe. The stdio output written by the application is buffered by the
glibc until it is flushed or the output is set as unbuffered. See setbuf(3). I
Could be this quote from the srun man page:
-u, --unbuffered
By default the connection between slurmstepd and the user launched application
is over a pipe. The stdio output written by the application is buffered by the
glibc until it is flushed or the output is set as unbuffered. See setbuf(3). I
The larger cluster is using NFS. I can see how that could be related to the
difference of behaviours between the clusters.
The buffering behaviour is the same if I tail the file from the node
running the job. The only thing that seems to change the behaviour is
whether I use srun to create a job
Is it being written to NFS? You say on your local dev cluster it's a
single node. Is it also the login node as well as compute? In that case
I guess there is no NFS. Larger cluster will be using some sort of
shared storage, so whichever shared file system you are using likely has
caching.
If you a
Hi Sean,
Thanks for your suggestion!
Adding the -u flag does not seem to have an impact on whether data is
buffered. I also tried adding stdbuf -o0 before the call to srun, to no
avail.
Best,
Maria
On Wed, Feb 10, 2021 at 4:30 AM Sean Maxwell wrote:
> Hi Maria,
>
> Have you tried adding the -
Hi Maria,
Have you tried adding the -u flag (specifies unbuffered) to your srun
command?
https://slurm.schedmd.com/srun.html#OPT_unbuffered
Your description sounds like buffering, so this might help.
Thanks,
-Sean
On Tue, Feb 9, 2021 at 6:49 PM Maria Semple wrote:
> Hello all,
>
> I've noti
