[slurm-users] Handling idle sessions

[Nadav Toledo]

Hello forum, I am trying to deal with idle session for some time, and haven't found a solution i am happy with. The scenario is as follow: users using srun for jupyter-lab(which is fine and even encouraged by me) on image processing cluster with gpus. problem is, I am trying to have some kind of solution to email/cancel their job if their session is idle for X amount of hours. the w command or xprintidle cannot be used , since they both work with ssh but not with slurm(checked that) Writing a script is not as easy as one might think, If i run a script in admin user scope, i need later on to figure out which idle gpu belong to which slurm job. running a script in the user scope is probably better idea, but in which way? crontab is running even user is not logged, how can i force users to run something only when the job start? perhaps some combination of sreport and tres? Thanks for the tips, Nadav