Re: [slurm-users] Slurm Jobscript Archiver

2019-06-20 Thread Christopher Benjamin Coffey
Hi Kevin, We fixed the issue on github. Thanks! Best, Chris — Christopher Coffey High-Performance Computing Northern Arizona University 928-523-1167 On 6/17/19, 8:56 AM, "slurm-users on behalf of Christopher Benjamin Coffey" wrote: Thanks Kevin, we'll put a fix in for that. B

Re: [slurm-users] Slurm Jobscript Archiver

2019-06-17 Thread Christopher Benjamin Coffey
Thanks Kevin, we'll put a fix in for that. Best, Chris — Christopher Coffey High-Performance Computing Northern Arizona University 928-523-1167 On 6/17/19, 12:04 AM, "Kevin Buckley" wrote: On 2019/05/09 23:37, Christopher Benjamin Coffey wrote: > Feel free to try it out and let

Re: [slurm-users] Slurm Jobscript Archiver

2019-06-17 Thread Christopher Benjamin Coffey
Hi Lech, I'm glad that it is working out well with the modifications you've put in place! Yes, there can be a huge volume of jobscripts out there. That’s a pretty good way of keeping it organized! . We've backed up 1.1M jobscripts since its inception 1.5 months ago and aren't too worried yet ab

Re: [slurm-users] Slurm Jobscript Archiver

2019-06-17 Thread Lech Nieroda
Hi Chris, you’ll find the patch for our version attached. Integrate it as you see fit, personally I’d recommend a branch since the two log files approach isn’t really reconcilable with the idea of having separate job files accessible to the respective owner. All filenames and directories are de

Re: [slurm-users] Slurm Jobscript Archiver

2019-06-17 Thread Kevin Buckley
On 2019/05/09 23:37, Christopher Benjamin Coffey wrote: Feel free to try it out and let us know how it works for you! https://github.com/nauhpc/job_archive So Chris, testing it out quickly, and dirtily, using an sbatch with a here document, vis: $ sbatch -p testq <

Re: [slurm-users] Slurm Jobscript Archiver

2019-06-14 Thread Christopher Benjamin Coffey
Hi Lech, I'm glad that it is working out well with the modifications you've put in place! Yes, there can be a huge volume of jobscripts out there. That’s a pretty good way of dealing with it! . We've backed up 1.1M jobscripts since its inception 1.5 months ago and aren't too worried yet about t

Re: [slurm-users] Slurm Jobscript Archiver

2019-06-14 Thread Lech Nieroda
Hello Chris, we’ve tried out your archiver and adapted it to our needs, it works quite well. The changes: - we get lots of jobs per day, ca. 3k-5k, so storing them as individual files would waste too much inodes and 4k-blocks. Instead everything is written into two log files (job_script.log and

[slurm-users] Slurm Jobscript Archiver

2019-05-09 Thread Christopher Benjamin Coffey
Hi All, We created a slurm job script archiver which you may find handy. We initially attempted to do this through slurm with a slurmctld prolog but it really bogged the scheduler down. This new solution is a custom c++ program that uses inotify to watch for job scripts and environment files to