On Tue, 1 Aug 2006, Xu, Jerry wrote:
Hi, I am maintaining a cluster while lots user uses perl to submit tons of jobs which seems to me like abusing the system. Does everybody meet the same situation? Many user us system call in the perl to do "qsub", shall I ban this? I don't know exactly why it is bad, but it looks to me really bad. Anybody can give me a good reason to ban it with better explanation?
What about using perl scripts to submit jobs to a queuing system constitutes abuse? The whole point of a queuing system is that you can control/monitor how much of a resource any group receives. That is, using it you can make it so that the resource isn't allocated first come first serve, so somebody with an automated job submission tool "wins", but in a prioritized queue where you can balance access no matter how jobs are submitted. I'd say that writing a script to manage personal job submission is intelligence, not abuse -- it saves human time, and human time is the most expensive resource in nearly any computing situation. Also, for many tasks, one sweeps some parameter(s) across a range and submit a systematically related series of jobs. This is clearly something that a computer script can do FAR better than a human pecking at a keyboard. I actually use(d) an adaptive script that monitored the returns from simulational jobs generating data across such a range, recomputed mean/sigma for the data points of interest, and resubmitted jobs on a squeaky wheel basis to uniformly drop sigma. In that way I ended up with a very uniform set of error bars on the data points I ultimately fit, which in turn makes it a lot easier to get a believable fit or to resolve problems with a fit form that might be missed by chance otherwise. So don't stop users from submitting tons of jobs using scripts, just institute a policy such that those jobs have a "lowest priority" and will be run in a fair way as the resource is available. rgb
Sincerely Protein _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
-- Robert G. Brown http://www.phy.duke.edu/~rgb/ Duke University Dept. of Physics, Box 90305 Durham, N.C. 27708-0305 Phone: 1-919-660-2567 Fax: 919-660-2525 email:[EMAIL PROTECTED] _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
