Hi GNU Parallel mailing list.
I'm looking for a way to run a large number of jobs (in parallel) on
several computers. The job consists of running an analysis on several
thousand input files. They way I have this set up right now is to split
the number of input files into chunks and move them t
On 14/03/2017 at 10:54,
Anders Lind wrote:
> I could perhaps set this up using the ssh functionality of parallel, but I
> would need to be able to on the fly stop some machines from running jobs,
> since the computers belong to co-workers who sometimes need their computers
> for their own work.
Anders,
Take a look at the --sqlmaster and --sqlworker options.
I use them to effectively create a jobqueue that any node can pull tasks
from. I do this for long running backups on a parallel filesystem (all
nodes have read/write access to the data and the sql joblog file).
1. Create a list of "t
