RE: [Beowulf] scheduler and perl

2006-08-03 Thread Clements, Brent M \(SAIC\)
Policies. BC -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Clements, Brent M (SAIC) Sent: Thursday, August 03, 2006 9:03 AM To: Xu, Jerry; Chris Dagdigian; beowulf@beowulf.org Subject: RE: [Beowulf] scheduler and perl If I recall from my LSF days

RE: [Beowulf] scheduler and perl

2006-08-03 Thread Clements, Brent M \(SAIC\)
Xu, Jerry Sent: Wednesday, August 02, 2006 8:38 AM To: Chris Dagdigian; beowulf@beowulf.org Subject: RE: [Beowulf] scheduler and perl Hi, Dear Joe, Chris: Thanks so much for your warm-hearted discussion. I used to manage cluster which is used by much "nicer" MPI application developers

RE: [Beowulf] scheduler and perl

2006-08-02 Thread Xu, Jerry
Original Message- From: Chris Dagdigian [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 01, 2006 9:23 PM To: Xu, Jerry; beowulf@beowulf.org Subject: Re: [Beowulf] scheduler and perl As Joe mention, the way we handle this is by using cluster schedulers sitting on robust hardware platforms

Re: [Beowulf] scheduler and perl

2006-08-02 Thread Robert G. Brown
On Tue, 1 Aug 2006, Xu, Jerry wrote: Hi, I am maintaining a cluster while lots user uses perl to submit tons of jobs which seems to me like abusing the system. Does everybody meet the same situation? Many user us system call in the perl to do "qsub", shall I ban this? I don't know exactly why

Re: [Beowulf] scheduler and perl

2006-08-01 Thread Chris Dagdigian
As Joe mention, the way we handle this is by using cluster schedulers sitting on robust hardware platforms that are capable of handling large numbers of job submissions without problems. Grid Engine and Platform LSF are two capable products that come to mind and scale well. The fact that

Re: [Beowulf] scheduler and perl

2006-08-01 Thread Stuart Midgley
Rather than banning them or publishing behaviour, we simply disable their access to the queue and won't re-instate it until they have contacted us to indicate they have rectified their ways. eg. rather than enforce hard disk quotas, which may result in a job loosing data, we simply detect w

Re: [Beowulf] scheduler and perl

2006-08-01 Thread Joe Landman
Diego M. Vadell wrote: Maybe you can collect some logs and make a list of misbehaving users. Then you can warn the users that you are collecting that information, that bypassing the system will degrade the performance for everybody, and that you may post that information. Sometimes the perspec

Re: [Beowulf] scheduler and perl

2006-08-01 Thread Diego M. Vadell
> Hi Jerry: >> the other example is that use system call and ssh >> to each node and run stuff and bypass the scheduler... Torque 2.1.2 has just been released. It comes with a pam module that, if I understood it right, makes harder (though not impossible) for users to bypass the batch system. The

RE: [Beowulf] scheduler and perl

2006-08-01 Thread Xu, Jerry
--Original Message- From: Joe Landman [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 01, 2006 5:24 PM To: Xu, Jerry Cc: beowulf@beowulf.org Subject: Re: [Beowulf] scheduler and perl Hi Jerry: Its generally a good idea to talk to your users, understand what it is they are doing, and see

Re: [Beowulf] scheduler and perl

2006-08-01 Thread Joe Landman
Hi Jerry: Xu, Jerry wrote: Hi, Thanks, Joe. I am not meaning to "ban" anything immediately, I am just curious how often this happen to the HPC community. Perl/shell is really strong tool, one example is to use loop to submit huge mount of jobs and puts burden on scheduler server, Thats what t

Re: [Beowulf] scheduler and perl

2006-08-01 Thread Joe Landman
Hi Jerry: Its generally a good idea to talk to your users, understand what it is they are doing, and see if you can help them, rather than simply "banning" things. The result of bans of deeply embedded practices usually results in some ... exciting ... meetings, emails, and telephone calls

[Beowulf] scheduler and perl

2006-08-01 Thread Xu, Jerry
Hi, I am maintaining a cluster while lots user uses perl to submit tons of jobs which seems to me like abusing the system. Does everybody meet the same situation? Many user us system call in the perl to do "qsub", shall I ban this? I don't know exactly why it is bad, but it looks to me really bad