On Apr 14, 2007, at 11:50 AM, Kern Sibbald wrote:
> would you re-write the Feature Request using my terminology and
> resubmit it?
Uhm, okay. I'll try, although I'm not sure what terminology I'm missing.
> As I see it, you are asking to implement a "Client Priority = nn"
> directive in
> addition to the current Priority.
>
> The current Priority will be applied to jobs as it currently is,
> then in
> addition if two Jobs from the same Client have the same Priority but
> different Client Priority, the job with the higher Client priority
> (lower
> number) will run first.
Exactly right. And from that it follows that (unlike current
priority), the ability of a higher-priority job to block the
execution of other jobs is limited to jobs on the same client,
increasing efficiency when using concurrent jobs.
> Also, please clarify the the first part of your example (current
> system with one
> priority) by showing the priorities and fictive start and stop wall
> clock
> times for each job, and add to the second part of your example by
> showing the
> two priorities for each job and fictive wall clock times when each
> job starts
> and stops under the proposed two priority system.
=================================
=================================
Item 1: Allow per-client job priority ordering
Origin: Darien Hager <[EMAIL PROTECTED]>
Date: April 18 2007
Status: Revised Request
What:
Provide a "Client Priority = nn" directive for Jobs, similar to the
current "Priority" directive. Default value is 10. This directive is
applied only upon jobs which are slated to run after applying the
currently-implemented "Priority" directive. Additionally, this
directive only serves to order jobs which have the same "Client"
directive.
Why: Some jobs may require a specific ordering, but
specifically in the context of a particular client rather than more
generally across the whole system of clients. The current system can
satisfy this, but is susceptible to significant possible bottlenecks
where a single lengthy job may result in many jobs being delayed and
concurrent job capability being wasted.
Notes: (See example situations at bottom of message for timing examples)
=================================
=================================
--Darien A. Hager
[EMAIL PROTECTED]
=================================
=================================
=======Default Example set-up:
* One director, one Storage Daemon, and 2 client machines.
* In the example, we mainly consider two client machines called
"alpha" and "beta".
* Concurrent jobs are enabled, 2 jobs at a time.
* Assumption: We cannot ensure any ordering using solely scheduled
start times, and must rely on the "Priority" directive. Perhaps the
amount of data on each client is highly variable, or there are
actually many clients (too many to write examples for), etc.
* There are two jobs defined for each client, named with the pattern:
$CLIENTNAME-Database, Scheduled 01:00
$CLIENTNAME-Files, Scheduled 01:30
* Sysadmin requirement: All *-Database jobs must run before *-Files
jobs (for any particular machine). Perhaps because the *-Files jobs
back up log-files generated from the *-Database jobs, so executing
them in the opposite order is undesirable. Alternative example set-
ups could use Backups and Verify jobs, run-script behaviors on a
client machine, etc.
* The client are designated Alpha and Beta.
=======Example of bottleneck:
* Jobs Alpha-Database and Beta-Database have "Priority=20" set.
* Jobs Alpha-Files and Beta-Files have "Priority=21" set.
* These priority settings satisfy the sysadmin requirement.
* In this case, all *-Database jobs must complete before any *-Files
jobs start.
* Alpha and Beta have different usage patterns, so their jobs take
different times.
Concurrent jobs can be run, 2 at a time.
Time needed for each job, assuming no SD bottleneck:
Alpha-Database: 30 minutes
Alpha-Files: 10 minutes
Beta-Database: 10 minutes
Beta-Files: 30 minutes
01:37: Beginning of example:
01:37: The system begins to process Priority=20 jobs
01:37: The system looks for jobs to run.
01:37: Alpha-Database is started
01:37: Beta-Database is started
01:47: Beta-Database completes (10m)
01:47: No other jobs to start in same priority
02:07: Alpha-Database completes (30m)
02:07: The system begins to process Priority=21 jobs
02:07: The system looks for jobs to run.
02:07: Alpha-Files is started
02:07: Beta-Files is started
02:17: Alpha-Files completes (10m)
02:17: No other jobs to start in same priority
02:37: Beta-Files completes (30m)
02:37: End of example period: 60 minutes elapsed
=======Example using new functionality:
* All jobs run at the same default "Priority=20"
* Alpha-Database and Beta-Database have "Client Priority=10"
* Alpha-Files and Beta-Files have "Client Priority=11"
* All jobs' elapsed times match those given in the previous example,
assuming no bottleneck in the Storage Daemon, etc.
01:37: Beginning of example:
01:37: The system begins to process Priority=20 jobs
01:37: The system looks for jobs to run.
01:37: The system skips over Alpha-Files and over Beta-Files because:
A) There are other jobs not yet completed for the same client
B) AND those other jobs have a lower priority number
01:37: Alpha-Database is started
01:37: Beta-Database is started
01:47: Beta-Database completes (10m)
01:47: The system looks for another job to run.
01:47: The system skips over Alpha-Files because:
A) There are other jobs not yet completed for the same client
B) AND those other jobs have a lower priority number
01:47: Beta-Files is started
02:07: Alpha-Database completes (30m)
02:07: The system looks for jobs to run.
02:07: No jobs are skipped over in this case
02:07: Alpha-Files is started
02:17: Alpha-Files completes (10m)
02:17: Beta-Files completes (30m)
02:17: End of example period: 40 minutes elapsed
02:17: The system begins to process Priority=21 jobs [...]
Total time taken for backups: 40 minutes, or 20 minutes less than
other example.
=================================
=================================
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
Bacula-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/bacula-users
