[bug #38092] Optionally support distributed memory machines using MPI

Update of bug #38092 (project findutils): Status:None => Postponed Assigned to:None => jay Open/Closed:Open => Closed

[bug #38092] Optionally support distributed memory machines using MPI

Update of bug #38092 (project findutils): Severity: 3 - Normal => 1 - Wish ___ Reply to this item at: ___ M

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #15, bug #38092 (project findutils): So any chance you could show an example of how to make xargs only process every Nth item where N is a preprocessor define? ___ Reply to this item at:

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #14, bug #38092 (project findutils): > Even if there was, you would have to pass this "N" to xargs, > and that would have to be done by the caller - mpirun. Correct although technically N isn't given by mpirun but by the MPI library environment in which a program is executed (ty

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #13, bug #38092 (project findutils): > Is there an easy way to make xargs process only every Nth argument [...]? Even if there was, you would have to pass this "N" to xargs, and that would have to be done by the caller - mpirun. The problem is that you would like to have xargs t

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #12, bug #38092 (project findutils): Is there an easy way to make xargs process only every Nth argument (in addition to all options given to it)? In the end there didn't seem to be after hacking on xargs.c for a while. ___

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #11, bug #38092 (project findutils): If you could provide a patch that makes xargs process every third file I could probably continue from there. ___ Reply to this item at: ___

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #10, bug #38092 (project findutils): > The other obvious - and more readable - way to do this is to > split the argument file into however many chunks you need. > Then just run ... Couldn't I skip xargs completely with this type of a for loop? for f in $(cat files); do process $

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #9, bug #38092 (project findutils): You can also easily test your suggestions with a stock xargs by installing open-mpi from http://www.open-mpi.org/software/ompi/v1.6/downloads/openmpi-1.6.3.tar.bz2 ./configure --prefix=$HOME && make && make install works for me and most distrib

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #8, bug #38092 (project findutils): > TOT_RANKS=# some decimal value > ARGS_PER_RANK=# another decimal value > xargs --process_slot_var=RANK -P $TOT_RANKS -n $ARGS_PER_RANK > sh -c 'mpirun -n $RANK yourcommand "$@"' dollarzero The above doesn't work in practice because an MPI pa

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #7, bug #38092 (project findutils): There are several reasons why the other way around doesn't work: 1) If the final program is not MPI aware mpirun will still have to call xargs but the second xargs would have to take its input from command line arguments, e.g. xargs -t --arg-

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #6, bug #38092 (project findutils): The --process_slot_var is documented in later findutils-4.5.x releases (you can look at the latest version of the manpage here: http://git.savannah.gnu.org/cgit/findutils.git/tree/xargs/xargs.1). It looks to me like your requirements would be

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #5, bug #38092 (project findutils): I don't know much about mpirun, but if mpirun -n 1 xargs --args-file files -P 12 -n 5 ... doesn't work because xargs wouldn't split the files, what about the other way round? xargs --arg-file files -P 12 -n 5 mpirun ... By this, xargs s

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #4, bug #38092 (project findutils): After googling --process-slot-var I don't see how it could be useful, although I haven't found its exact description. Mainly because I don't think it is possible to distunguish between different invocations of xargs started with mpirun (or equi

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #3, bug #38092 (project findutils): For reference: https://en.wikipedia.org/wiki/Message_Passing_Interface One implementation of the above (3-clause BSD): http://www.open-mpi.org ___ Reply to this item at:

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #2, bug #38092 (project findutils): I didn't find an option called --process-slot-var in the man or info page of my xargs, is it the same as --max-procs or -P? If yes then those are not sufficient because in MPI multiple copies of the same program are launched, one per shared mem

[bug #38092] Optionally support distributed memory machines using MPI

Follow-up Comment #1, bug #38092 (project findutils): Your feature request is quite unclear. Could you explain what problem you're trying to solve? Why is the --process-slot-var option of xargs not sufficient for your purpose? ___ Repl

[bug #38092] Optionally support distributed memory machines using MPI

URL: Summary: Optionally support distributed memory machines using MPI Project: findutils Submitted by: None Submitted on: Tue 15 Jan 2013 09:20:38 AM UTC Category: xargs