Re: [Beowulf] job scheduler and accounting question

2015-07-14 Thread Christopher Samuel
On 15/07/15 08:25, Michael Gutteridge wrote: > I'll put in a plug for Slurm. I'd say it's got the features you need > and it scales up really well. > > Accounting: it can write to file, but you're best off configuring the > database backend (MySQL, Mariadb). Then there's both the built-in sacc

Re: [Beowulf] job scheduler and accounting question

2015-07-14 Thread Michael Gutteridge
I'll put in a plug for Slurm. I'd say it's got the features you need and it scales up really well. Accounting: it can write to file, but you're best off configuring the database backend (MySQL, Mariadb). Then there's both the built-in sacct tool and (if you're so inclined) queries directly agai

Re: [Beowulf] job scheduler and accounting question

2015-07-14 Thread John Hearns
> We are investigating options for a cluster/cloud scenario where I need to > keep track of CPU, > > >memory, disk used during the runs. This "keeping > track" should be available via command line tools >(preferably in > JSON/XML/CSV output that I can easily parse). Does Apache Mesos suit y

Re: [Beowulf] job scheduler and accounting question

2015-07-14 Thread Skylar Thompson
On 07/14/2015 04:17 PM, Stuart Barkley wrote: > On Tue, 14 Jul 2015 at 15:06 -, Joe Landman wrote: > >> Has the gridengine mess ever been sorted out? > > We are using Son of Grid Engine which > seems to be not-dead-yet. It does seem to rely heavily on just one or

Re: [Beowulf] job scheduler and accounting question

2015-07-14 Thread Stuart Barkley
On Tue, 14 Jul 2015 at 15:06 -, Joe Landman wrote: > Has the gridengine mess ever been sorted out? We are using Son of Grid Engine which seems to be not-dead-yet. It does seem to rely heavily on just one or two developers at this point. Stuart -- I've never been

Re: [Beowulf] job scheduler and accounting question

2015-07-14 Thread Andrew Latham
>From the cloud side of things context wise... So I am seeing and starting to use things like http://www.freedesktop.org/software/systemd/man/systemd.resource-control.html to manage the resources. There is a lot of cool opportunities around the .socket for automation with fewer moving parts. Core

[Beowulf] job scheduler and accounting question

2015-07-14 Thread Joe Landman
Hi folks: Its been a few years since we've had a good use case for a job scheduler, and I'll freely admit I've not paid nearly enough attention to what is currently out there. We are investigating options for a cluster/cloud scenario where I need to keep track of CPU, memory, disk used d