Author: rjung
Date: Tue May 16 15:52:39 2006
New Revision: 407096
URL: http://svn.apache.org/viewcvs?rev=407096&view=rev
Log:
Adding attribute "distance" to balanced workers.
Express Preferences between the balanced workers of an lb worker.
A load balancer will never choose some balanced worker
in case there is another usable worker with lower distance.
Only in case all workers below a given distance are in error, disabled or
stopped,
workers of a larger distance are eligible for balancing.
Modified:
tomcat/connectors/trunk/jk/native/TODO
tomcat/connectors/trunk/jk/native/common/jk_lb_worker.c
tomcat/connectors/trunk/jk/native/common/jk_lb_worker.h
tomcat/connectors/trunk/jk/native/common/jk_shm.h
tomcat/connectors/trunk/jk/native/common/jk_status.c
tomcat/connectors/trunk/jk/native/common/jk_util.c
tomcat/connectors/trunk/jk/xdocs/config/workers.xml
Modified: tomcat/connectors/trunk/jk/native/TODO
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/native/TODO?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/native/TODO (original)
+++ tomcat/connectors/trunk/jk/native/TODO Tue May 16 15:52:39 2006
@@ -2,26 +2,14 @@
$Id$
-1) Counter types in shm
-=======================
+1) Optimize "distance"
+======================
-Why are elected and errors of type size_t?
+Sorting the list of balanced workers by distance would be nice, but:
+How to combine the sorting with the offset implementation (especially
+useful for strategy BUSYNESS under low load).
- /* Number of times the worker was elected */
- volatile size_t elected;
- /* Number of non 200 responses */
- volatile size_t errors;
-
-Shouldn't they be jk_uint64_t?
-
-2) Implement "distance"
-=======================
-
-Add a new attribut "distance" to balanced workers.
-Without session Id or stickyness the lb worker will
-choose between al usable workers of the smallest distance.
-
-3) Reduce number of string comparisons in most_suitable
+2) Reduce number of string comparisons in most_suitable
========================================================
a) redirect/domains
@@ -50,7 +38,7 @@
Could we use hashes instead of string comparisons all the time?
I'm not sure, if a good enough hash takes longer than a string comparison
though.
-4) Optimization of JK_WORKER_USABLE
+3) Optimization of JK_WORKER_USABLE
====================================
We use that one quite a lot. Since it is now a combination of four
@@ -63,7 +51,7 @@
every time we change one of the four other flags). But in terms of
performance that happens rarely.
-5) Code separation between factory, validate and init in lb
+4) Code separation between factory, validate and init in lb
============================================================
The factory contains:
@@ -88,18 +76,23 @@
p->lb_workers[i].s->maintain_time = time(NULL);
-6) Refactor Logging
+5) Refactor Logging
====================
a) Use the same code files for the request logging functions in Apache 1.3 and
2.0.
b) Use the same code files for piped logging in Apache 1.3 and 2.0.
-7) ajpget
+6) ajpget
==========
Combine ajplib and Apache ab to an ajp13 commandline client ajpget.
+7) Manage lb method and locking via jk_status
+=============================================
+
+It's not yet contained in the shm.
+
8) Parsing workers.properties
=============================
@@ -108,7 +101,7 @@
no information will be logged, e.g. when attributes contain typos.
9) Persisting workers.properties
-=================================
+================================
Make workers.properties persist from inside status worker.
@@ -154,7 +147,7 @@
to be generated at the end or directly after service.
11) Access/Modification Time in shm
-==================================
+===================================
a) [Discussion] What will this generally be used for? At the moment,
only jk_status "uses" it, but it only sets the values, it never asks for them.
@@ -173,7 +166,7 @@
or the second call in jk_status coulkd be removed.
12) "Destroy" functionality
-==========================
+===========================
[Hint] Destroy on a worker never seems to free shm,
but I think that was already a flaw without shm.
Modified: tomcat/connectors/trunk/jk/native/common/jk_lb_worker.c
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/native/common/jk_lb_worker.c?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/native/common/jk_lb_worker.c (original)
+++ tomcat/connectors/trunk/jk/native/common/jk_lb_worker.c Tue May 16 15:52:39
2006
@@ -95,28 +95,17 @@
JK_TRACE_EXIT(l);
}
-/* Get the correct lb_value when recovering/starting/enabling a worker.
- * This function needs to be externally synchronized!
+/* Reset all lb values.
*/
-jk_uint64_t restart_value(lb_worker_t *p, jk_logger_t *l)
+void reset_lb_values(lb_worker_t *p, jk_logger_t *l)
{
int i = 0;
- jk_uint64_t curmax = 0;
JK_TRACE_ENTER(l);
if (p->lbmethod != JK_LB_BYBUSYNESS) {
for (i = 0; i < p->num_of_workers; i++) {
- if (p->lb_workers[i].s->lb_value > curmax) {
- curmax = p->lb_workers[i].s->lb_value;
- }
+ p->lb_workers[i].s->lb_value = 0;
}
}
- if (JK_IS_DEBUG_LEVEL(l))
- jk_log(l, JK_LOG_DEBUG,
- "restarting worker with lb_value %"
- JK_UINT64_T_FMT,
- curmax);
- JK_TRACE_EXIT(l);
- return curmax;
}
/* Retrieve the parameter with the given name */
@@ -364,6 +353,7 @@
jk_logger_t *l)
{
unsigned int i;
+ int d = 0;
jk_uint64_t curmin = 0;
worker_record_t *candidate = NULL;
@@ -378,9 +368,12 @@
* not in error state, stopped, disabled or busy.
*/
if (JK_WORKER_USABLE(p->lb_workers[i].s)) {
- if (!candidate || p->lb_workers[i].s->lb_value < curmin) {
+ if (!candidate || p->lb_workers[i].s->distance < d ||
+ (p->lb_workers[i].s->lb_value < curmin &&
+ p->lb_workers[i].s->distance == d)) {
candidate = &p->lb_workers[i];
curmin = p->lb_workers[i].s->lb_value;
+ d = p->lb_workers[i].s->distance;
}
}
}
@@ -400,6 +393,7 @@
unsigned int i;
unsigned int j;
unsigned int offset;
+ int d = 0;
jk_uint64_t curmin = 0;
/* find the least busy worker */
@@ -415,9 +409,12 @@
* not in error state, stopped, disabled or busy.
*/
if (JK_WORKER_USABLE(p->lb_workers[i].s)) {
- if (!candidate || (p->lb_workers[i].s->lb_value < curmin)) {
+ if (!candidate || p->lb_workers[i].s->distance < d ||
+ (p->lb_workers[i].s->lb_value < curmin &&
+ p->lb_workers[i].s->distance == d)) {
candidate = &p->lb_workers[i];
curmin = p->lb_workers[i].s->lb_value;
+ d = p->lb_workers[i].s->distance;
next_offset = i + 1;
}
}
@@ -914,6 +911,8 @@
if (p->lb_workers[i].s->lb_factor < 1) {
p->lb_workers[i].s->lb_factor = 1;
}
+ p->lb_workers[i].s->distance =
+ jk_get_distance(props, worker_names[i]);
if ((s = jk_get_worker_jvm_route(props, worker_names[i],
NULL)))
strncpy(p->lb_workers[i].s->jvm_route, s, JK_SHM_STR_SIZ);
else
Modified: tomcat/connectors/trunk/jk/native/common/jk_lb_worker.h
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/native/common/jk_lb_worker.h?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/native/common/jk_lb_worker.h (original)
+++ tomcat/connectors/trunk/jk/native/common/jk_lb_worker.h Tue May 16 15:52:39
2006
@@ -57,6 +57,13 @@
/* The exponent x is JK_LB_DECAY_MULT*#MAINT_INTV_SINCE_LAST_MAINT */
#define JK_LB_DECAY_MULT (1)
+static const char *lb_locking_type[] = {
+ JK_LB_LM_DEFAULT,
+ JK_LB_LM_PESSIMISTIC,
+ "unknown",
+ NULL
+};
+
static const char *lb_method_type[] = {
JK_LB_METHOD_REQUESTS,
JK_LB_METHOD_TRAFFIC,
@@ -97,7 +104,7 @@
int JK_METHOD lb_worker_factory(jk_worker_t **w,
const char *name, jk_logger_t *l);
-jk_uint64_t restart_value(lb_worker_t *p, jk_logger_t *l);
+void reset_lb_values(lb_worker_t *p, jk_logger_t *l);
void update_mult(lb_worker_t * p, jk_logger_t *l);
#ifdef __cplusplus
Modified: tomcat/connectors/trunk/jk/native/common/jk_shm.h
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/native/common/jk_shm.h?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/native/common/jk_shm.h (original)
+++ tomcat/connectors/trunk/jk/native/common/jk_shm.h Tue May 16 15:52:39 2006
@@ -71,6 +71,8 @@
char domain[JK_SHM_STR_SIZ+1];
/* worker redirect route */
char redirect[JK_SHM_STR_SIZ+1];
+ /* worker distance */
+ volatile int distance;
/* current status of the worker */
volatile int is_disabled;
volatile int is_stopped;
@@ -96,13 +98,13 @@
/* Number of bytes transferred to remote */
volatile jk_uint64_t transferred;
/* Number of times the worker was elected */
- volatile size_t elected;
+ volatile jk_uint64_t elected;
/* Number of non 200 responses */
- volatile size_t errors;
+ volatile jk_uint32_t errors;
/* Number of recovery attempts */
- volatile size_t recoveries;
+ volatile jk_uint32_t recoveries;
/* Number of recovery failures */
- volatile size_t recovery_errors;
+ volatile jk_uint32_t recovery_errors;
};
typedef struct jk_shm_worker jk_shm_worker_t;
Modified: tomcat/connectors/trunk/jk/native/common/jk_status.c
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/native/common/jk_status.c?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/native/common/jk_status.c (original)
+++ tomcat/connectors/trunk/jk/native/common/jk_status.c Tue May 16 15:52:39
2006
@@ -448,7 +448,7 @@
jk_puts(s, "</tr>\n</table>\n<br/>\n");
jk_puts(s, "<table><tr>"
"<th>Name</th><th>Type</th><th>jvmRoute</th><th>Host</th><th>Addr</th>"
-
"<th>Stat</th><th>F</th><th>M</th><th>V</th><th>Acc</th><th>Err</th>"
+
"<th>Stat</th><th>D</th><th>F</th><th>M</th><th>V</th><th>Acc</th><th>Err</th>"
"<th>Wr</th><th>Rd</th><th>Busy</th><th>Max</th><th>RR</th><th>Cd</th></tr>\n");
for (j = 0; j < lb->num_of_workers; j++) {
worker_record_t *wr = &(lb->lb_workers[j]);
@@ -472,11 +472,12 @@
wr->s->in_recovering,
wr->s->is_busy),
"</td>", NULL);
+ jk_printf(s, "<td>%d</td>", wr->s->distance);
jk_printf(s, "<td>%d</td>", wr->s->lb_factor);
jk_printf(s, "<td>%" JK_UINT64_T_FMT "</td>", wr->s->lb_mult);
jk_printf(s, "<td>%" JK_UINT64_T_FMT "</td>", wr->s->lb_value);
- jk_printf(s, "<td>%u</td>", wr->s->elected);
- jk_printf(s, "<td>%u</td>", wr->s->errors);
+ jk_printf(s, "<td>%" JK_UINT64_T_FMT "</td>", wr->s->elected);
+ jk_printf(s, "<td>%" JK_UINT32_T_FMT "</td>", wr->s->errors);
jk_putv(s, "<td>", status_strfsize(wr->s->transferred, buf),
"</td>", NULL);
jk_putv(s, "<td>", status_strfsize(wr->s->readed, buf),
@@ -511,7 +512,9 @@
jk_puts(s, "<input type=\"hidden\" name=\"lb\" ");
jk_printf(s, "value=\"%u\">\n</table>\n", i);
- jk_puts(s, "<table>\n<tr><td>Load factor:</td><td><input
name=\"wf\" type=\"text\" ");
+ jk_puts(s, "<table>\n<tr><td>Distance:</td><td><input
name=\"wx\" type=\"text\" ");
+ jk_printf(s, "value=\"%d\"/></td><tr>\n", wr->s->distance);
+ jk_puts(s, "<tr><td>Load factor:</td><td><input name=\"wf\"
type=\"text\" ");
jk_printf(s, "value=\"%d\"/></td><tr>\n", wr->s->lb_factor);
jk_puts(s, "<tr><td>Route Redirect:</td><td><input name=\"wr\"
type=\"text\" ");
jk_putv(s, "value=\"", wr->s->redirect, NULL);
@@ -577,14 +580,15 @@
}
/* Display legend */
jk_puts(s, "<hr/><table>\n"
- "<tr><th>Name</th><td>Worker route name</td></tr>\n"
+ "<tr><th>Name</th><td>Worker name</td></tr>\n"
"<tr><th>Type</th><td>Worker type</td></tr>\n"
- "<tr><th>jvmRoute</th><td>Worker JVM Route</td></tr>\n"
+ "<tr><th>jvmRoute</th><td>Worker JVM route</td></tr>\n"
"<tr><th>Addr</th><td>Backend Address info</td></tr>\n"
"<tr><th>Stat</th><td>Worker status</td></tr>\n"
- "<tr><th>F</th><td>Load Balancer Factor</td></tr>\n"
- "<tr><th>M</th><td>Load Balancer Multiplicity</td></tr>\n"
- "<tr><th>V</th><td>Load Balancer Value</td></tr>\n"
+ "<tr><th>D</th><td>Worker distance</td></tr>\n"
+ "<tr><th>F</th><td>Load Balancer factor</td></tr>\n"
+ "<tr><th>M</th><td>Load Balancer multiplicity</td></tr>\n"
+ "<tr><th>V</th><td>Load Balancer value</td></tr>\n"
"<tr><th>Acc</th><td>Number of requests</td></tr>\n"
"<tr><th>Err</th><td>Number of failed requests</td></tr>\n"
"<tr><th>Wr</th><td>Number of bytes transferred/min</td></tr>\n"
@@ -656,11 +660,12 @@
wr->s->in_recovering,
wr->s->is_busy) );
+ jk_printf(s, " distance=\"%d\"", wr->s->distance);
jk_printf(s, " lbfactor=\"%d\"", wr->s->lb_factor);
jk_printf(s, " lbmult=\"%" JK_UINT64_T_FMT "\"", wr->s->lb_mult);
jk_printf(s, " lbvalue=\"%" JK_UINT64_T_FMT "\"", wr->s->lb_value);
- jk_printf(s, " elected=\"%u\"", wr->s->elected);
- jk_printf(s, " errors=\"%u\"", wr->s->errors);
+ jk_printf(s, " elected=\"%" JK_UINT64_T_FMT "\"", wr->s->elected);
+ jk_printf(s, " errors=\"%" JK_UINT32_T_FMT "\"", wr->s->errors);
jk_printf(s, " transferred=\"%" JK_UINT64_T_FMT "\"",
wr->s->transferred);
jk_printf(s, " readed=\"%" JK_UINT64_T_FMT "\"", wr->s->readed);
jk_printf(s, " busy=\"%u\"", wr->s->busy);
@@ -747,14 +752,22 @@
jk_shm_lock();
wr->s->is_disabled = i;
wr->s->is_stopped = j;
- wr->s->lb_value = restart_value(lb, l);
+ reset_lb_values(lb, l);
+ /* unlock the shared memory */
+ jk_shm_unlock();
if (i+j==0) {
jk_log(l, JK_LOG_INFO,
- "worker %s restarted in status worker with lb_value %"
+ "worker %s restarted in status worker"
JK_UINT64_T_FMT,
- wr->s->name,
- wr->s->lb_value);
+ wr->s->name);
}
+ }
+ i = status_int("wx", s->query_string, wr->s->distance);
+ if (wr->s->distance!=i) {
+ /* lock shared memory */
+ jk_shm_lock();
+ wr->s->distance = i;
+ reset_lb_values(lb, l);
/* unlock the shared memory */
jk_shm_unlock();
}
Modified: tomcat/connectors/trunk/jk/native/common/jk_util.c
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/native/common/jk_util.c?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/native/common/jk_util.c (original)
+++ tomcat/connectors/trunk/jk/native/common/jk_util.c Tue May 16 15:52:39 2006
@@ -57,6 +57,7 @@
#define SOCKET_KEEPALIVE_OF_WORKER ("socket_keepalive")
#define RECYCLE_TIMEOUT_OF_WORKER ("recycle_timeout")
#define LOAD_FACTOR_OF_WORKER ("lbfactor")
+#define DISTANCE_OF_WORKER ("distance")
/* deprecated directive. Use balance_workers instead */
#define BALANCED_WORKERS ("balanced_workers")
#define BALANCE_WORKERS ("balance_workers")
@@ -82,6 +83,7 @@
#define WORKER_MAINTAIN_PROPERTY_NAME ("worker.maintain")
#define DEFAULT_MAINTAIN_TIME (60)
#define DEFAULT_LB_FACTOR (1)
+#define DEFAULT_DISTANCE (0)
#define LOG_FORMAT ("log_format")
#define TOMCAT32_BRIDGE_NAME ("tomcat32")
@@ -748,6 +750,19 @@
return jk_map_get_int(m, buf, DEFAULT_LB_FACTOR);
}
+int jk_get_distance(jk_map_t *m, const char *wname)
+{
+ char buf[1024];
+
+ if (!m || !wname) {
+ return DEFAULT_DISTANCE;
+ }
+
+ MAKE_WORKER_PARAM(DISTANCE_OF_WORKER);
+
+ return jk_map_get_int(m, buf, DEFAULT_DISTANCE);
+}
+
int jk_get_is_sticky_session(jk_map_t *m, const char *wname)
{
int rc = JK_TRUE;
@@ -781,7 +796,7 @@
char buf[1024];
const char *v;
if (!m || !wname) {
- return DEFAULT_LB_FACTOR;
+ return JK_LB_BYREQUESTS;
}
MAKE_WORKER_PARAM(METHOD_OF_WORKER);
Modified: tomcat/connectors/trunk/jk/xdocs/config/workers.xml
URL:
http://svn.apache.org/viewcvs/tomcat/connectors/trunk/jk/xdocs/config/workers.xml?rev=407096&r1=407095&r2=407096&view=diff
==============================================================================
--- tomcat/connectors/trunk/jk/xdocs/config/workers.xml (original)
+++ tomcat/connectors/trunk/jk/xdocs/config/workers.xml Tue May 16 15:52:39 2006
@@ -373,6 +373,16 @@
</p>
</directive>
+<directive name="distance" default="0" required="false">
+Express Preferences between the balanced workers of an lb worker.
+A load balancer will never choose some balanced worker
+in case there is another usable worker with lower distance.
+<p>
+Only in case all workers below a given distance are in error, disabled or
stopped,
+workers of a larger distance are eligible for balancing.
+</p>
+</directive>
+
<directive name="domain" default="" required="false">
Domain directive can be used only when the worker is a member of the load
balancer.
Workers that share the same domain name are treated as single worker. If
sticky_session
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
