Hi Ahmad,you could use squeue -h -t r --format="%i %e" | sort -k2 to get a list of all running jobs sorted by their endtime.
We use normaly a maintenance reservation with starttime of the mainenance (or with some leading time before) to get the system free of jobs. That make things easier, because if you drain your cluster no new jobs could start. With the reservation jobs with a shorter wallclock time could be backfilled till the reservation/maintenance starts. You can put the reservation anytime in the system but at least or before "<starttime maintenance> minus <longest MaxTime of partition>", e.g.
scontrol create reservation=<name> starttime=<starttime> duration=<duration> user=root flags=maint nodes=ALL
Hope, that helps a little bit, Carsten -- Carsten Beyer Abteilung Systeme Deutsches Klimarechenzentrum GmbH (DKRZ) Bundesstraße 45a * D-20146 Hamburg * Germany Phone: +49 40 460094-221 Fax: +49 40 460094-270 Email: be...@dkrz.de URL: http://www.dkrz.de Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784 Am 05.11.2021 um 23:16 schrieb Ahmad Khalifa:
If I plan maintenance on a certain day, how long before that day should I set the queue to drain mode?! Is there a way to estimate the completion date / time of current running jobs?!Regards.
smime.p7s
Description: S/MIME Cryptographic Signature
