Re: Tomcat 6 - Cluster error.

Wed, 19 Dec 2007 04:15:41 -0800

I think your Membership DropTime 4sec is very small. Some more Load or a FULL GC can drop your member. 
Nomally I use 30 sec.

Peter

Am 19.12.2007 um 11:40 schrieb Raúl García:


Hi,

We are using tomcat 6.0.14.

We use a cluster working at the same machine,
we start it and it seems to be ok, replication is working fine.
But within the next 12h from startup, we see this error at the catalina.out log file, of the first instance, the error repeats forever until we stop 
both

instances and restart them again.

The server is supporting a 0.5 hit per seccond aprox.

CATALINA.OUT
===============
Dec 19, 2007 10:07:30 AM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
performBasicCheck
WARNING: Member added, even though we werent
notified:org.apache.catalina.tribes.membership.MemberImpl[tcp:// localhost:40 
02,localhost,4002,
alive=165023279,id={-42 -48 112 17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 
102 }, payload={}, command={}, domain={}, ]
Dec 19, 2007 10:07:30 AM org.apache.catalina.ha.tcp.SimpleTcpCluster
memberAdded
INFO: Replication member
added:org.apache.catalina.tribes.membership.MemberImpl[tcp:// localhost:4002, 
localhost,4002, alive=165023279,id={-42 -48 112 17 -57

-2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
domain={}, ]
Dec 19, 2007 10:07:34 AM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
memberDisappeared
INFO: Received
memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl [tcp://loc 
alhost:4002,localhost,4002, alive=165028289,id={-42 -48 112 17 -57

-2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
domain={}, ]] message. Will verify.
Dec 19, 2007 10:07:34 AM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
memberDisappeared
INFO: Verification complete. Member still
alive[org.apache.catalina.tribes.membership.MemberImpl[tcp:// localhost:4002, 
localhost,4002, alive=165028289,id={-42

-48 112 17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={},
command={}, domain={}, ]]
Dec 19, 2007 10:07:34 AM org.apache.catalina.ha.tcp.SimpleTcpCluster send 
SEVERE: Unable to send message through cluster sender.
org.apache.catalina.tribes.ChannelException: Operation has timed out (3000 
ms.).; Faulty members:tcp://localhost:4002;
        at
org.apache.catalina.tribes.transport.nio.ParallelNioSender.sendMessage (Paral 
lelNioSender.java:97)
        at
org.apache.catalina.tribes.transport.nio.PooledParallelSender.sendMess age(Po 
oledParallelSender.java:53)
        at
org.apache.catalina.tribes.transport.ReplicationTransmitter.sendMessag e(Repl 
icationTransmitter.java:80)
        at
org.apache.catalina.tribes.group.ChannelCoordinator.sendMessage (ChannelCoord 
inator.java:78)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage (ChannelI 
nterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.interceptors.ThroughputInterceptor.se ndMess 
age(ThroughputInterceptor.java:61)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage (ChannelI 
nterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.interceptors.MessageDispatchIntercept or.sen 
dMessage(MessageDispatchInterceptor.java:73)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage (ChannelI 
nterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector.sendM essage 
(TcpFailureDetector.java:87)
        at
org.apache.catalina.tribes.group.ChannelInterceptorBase.sendMessage (ChannelI 
nterceptorBase.java:75)
        at
org.apache.catalina.tribes.group.GroupChannel.send (GroupChannel.java:216) 
        at
org.apache.catalina.tribes.group.GroupChannel.send (GroupChannel.java:175) 
        at
org.apache.catalina.ha.tcp.SimpleTcpCluster.send (SimpleTcpCluster.java:835) 
        at
org.apache.catalina.ha.tcp.SimpleTcpCluster.sendClusterDomain (SimpleTcpClust 
er.java:814)
        at
org.apache.catalina.ha.session.DeltaManager.send(DeltaManager.java: 586) 
        at
org.apache.catalina.ha.session.DeltaManager.sendCreateSession (DeltaManager.j 
ava:575)
        at
org.apache.catalina.ha.session.DeltaManager.createSession (DeltaManager.java: 
551)
        at
org.apache.catalina.ha.session.DeltaManager.createSession (DeltaManager.java: 
534)
        at
org.apache.catalina.connector.Request.doGetSession(Request.java:2312)
        at
org.apache.catalina.connector.Request.getSession(Request.java:2075)
        at
org.apache.catalina.connector.RequestFacade.getSession (RequestFacade.java:83 
3)
        at pad.kernel.Resolver.service(Resolver.java:266)
at javax.servlet.http.HttpServlet.service(HttpServlet.java: 803) 
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter (Application 
FilterChain.java:290)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter (ApplicationFilterCh 
ain.java:206)
at pad.kernel.EntryPointFilter.doFilter (EntryPointFilter.java:365) 
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter (Application 
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter (ApplicationFilterCh 
ain.java:206)
        at
org.apache.catalina.core.StandardWrapperValve.invoke (StandardWrapperValve.ja 
va:219)
        at
org.apache.catalina.core.StandardContextValve.invoke (StandardContextValve.ja 
va:175)
        at
org.apache.catalina.core.StandardHostValve.invoke (StandardHostValve.java:128 
)
        at
org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java:102 
)
        at
org.apache.catalina.valves.RequestFilterValve.process (RequestFilterValve.jav 
a:269)
        at
org.apache.catalina.valves.RemoteAddrValve.invoke (RemoteAddrValve.java:81) 
        at
org.apache.catalina.core.StandardEngineValve.invoke (StandardEngineValve.java 
:109)
        at
org.apache.catalina.ha.tcp.ReplicationValve.invoke (ReplicationValve.java:347 
)
        at
org.apache.catalina.connector.CoyoteAdapter.service (CoyoteAdapter.java:263) 
        at
org.apache.coyote.http11.Http11Processor.process (Http11Processor.java:844) 
        at
org.apache.coyote.http11.Http11Protocol $Http11ConnectionHandler.process(Http 
11Protocol.java:584)
        at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java: 447) 
        at java.lang.Thread.run(Thread.java:619)
Dec 19, 2007 10:07:36 AM org.apache.catalina.ha.tcp.SimpleTcpCluster
memberDisappeared
INFO: Received member
disappeared:org.apache.catalina.tribes.membership.MemberImpl[tcp:// localhost 
:4002,localhost,4002, alive=165029291,id={-42 -48 112 17

-57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
domain={}, ]
Dec 19, 2007 10:07:36 AM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
performBasicCheck
INFO: Suspect member, confirmed
dead.[org.apache.catalina.tribes.membership.MemberImpl[tcp:// localhost:4002, 
localhost,4002, alive=165029291,id={-42 -48 112
17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={}, 
domain={}, ]]
Dec 19, 2007 10:07:39 AM org.apache.catalina.ha.tcp.SimpleTcpCluster send 
SEVERE: Unable to send message through cluster sender.
org.apache.catalina.tribes.ChannelException: No destination given; No faulty 
members identified.
        at
org.apache.catalina.tribes.group.GroupChannel.send (GroupChannel.java:194) 
        at
org.apache.catalina.tribes.group.GroupChannel.send (GroupChannel.java:175) 
        at
org.apache.catalina.ha.tcp.SimpleTcpCluster.send (SimpleTcpCluster.java:835) 
        at
org.apache.catalina.ha.tcp.SimpleTcpCluster.sendClusterDomain (SimpleTcpClust 
er.java:814)
        at
org.apache.catalina.ha.session.DeltaManager.send(DeltaManager.java: 586) 
        at
org.apache.catalina.ha.session.DeltaManager.sessionExpired (DeltaManager.java 
:1211)
        at
org.apache.catalina.ha.session.DeltaSession.expire (DeltaSession.java:366) 
        at
org.apache.catalina.ha.session.DeltaSession.expire (DeltaSession.java:352) 
        at
org.apache.catalina.session.StandardSession.expire (StandardSession.java:660) 
        at
org.apache.catalina.session.StandardSession.invalidate (StandardSession.java: 
1111)
        at
org.apache.catalina.session.StandardSessionFacade.invalidate (StandardSession 
Facade.java:150)
        at pad.kernel.Resolver.service(Resolver.java:623)
at javax.servlet.http.HttpServlet.service(HttpServlet.java: 803) 
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter (Application 
FilterChain.java:290)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter (ApplicationFilterCh 
ain.java:206)
at pad.kernel.EntryPointFilter.doFilter (EntryPointFilter.java:365) 
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter (Application 
FilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter (ApplicationFilterCh 
ain.java:206)
        at
org.apache.catalina.core.StandardWrapperValve.invoke (StandardWrapperValve.ja 
va:219)
        at
org.apache.catalina.core.StandardContextValve.invoke (StandardContextValve.ja 
va:175)
        at
org.apache.catalina.core.StandardHostValve.invoke (StandardHostValve.java:128 
)
        at
org.apache.catalina.valves.ErrorReportValve.invoke (ErrorReportValve.java:102 
)
        at
org.apache.catalina.valves.RequestFilterValve.process (RequestFilterValve.jav 
a:269)
        at
org.apache.catalina.valves.RemoteAddrValve.invoke (RemoteAddrValve.java:81) 
        at
org.apache.catalina.core.StandardEngineValve.invoke (StandardEngineValve.java 
:109)
        at
org.apache.catalina.ha.tcp.ReplicationValve.invoke (ReplicationValve.java:347 
)
        at
org.apache.catalina.connector.CoyoteAdapter.service (CoyoteAdapter.java:263) 
        at
org.apache.coyote.http11.Http11Processor.process (Http11Processor.java:844) 
        at
org.apache.coyote.http11.Http11Protocol $Http11ConnectionHandler.process(Http 
11Protocol.java:584)
        at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java: 447) 
        at java.lang.Thread.run(Thread.java:619)
Dec 19, 2007 10:07:39 AM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
memberDisappeared
INFO: Received
memberDisappeared[org.apache.catalina.tribes.membership.MemberImpl [tcp://loc 
alhost:4002,localhost,4002, alive=165033299,id={-42 -48 112 17 -57

-2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={}, command={},
domain={}, ]] message. Will verify.
Dec 19, 2007 10:07:39 AM
org.apache.catalina.tribes.group.interceptors.TcpFailureDetector
memberDisappeared
INFO: Verification complete. Member still
alive[org.apache.catalina.tribes.membership.MemberImpl[tcp:// localhost:4002, 
localhost,4002, alive=165033299,id={-42

-48 112 17 -57 -2 73 -111 -109 113 -93 84 6 91 -72 102 }, payload={},
command={}, domain={}, ]]

[...]

And repeats again, and again...

========================


SERVER.XML (Instance_1)
=======================
[...] Connector
<Connector port="8081" protocol="HTTP/1.1" maxHttpHeaderSize="8192" 
emptySessionPath="true"
maxThreads="150" minSpareThreads="100" maxSpareThreads="300" enableLookups="false" redirectPort="81443" acceptCount="1000" 
               debug="0" connectionTimeout="20000"
disableUploadTimeout="true"
               compression="on"
                           compressionMinSize="2048"
                           noCompressionUserAgents="gozilla, traviata"
                           compressableMimeType="text/html,text/xml" />


[...] Cluster Tag

        <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="6">
<Manager className="org.apache.catalina.ha.session.DeltaManager" 
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel
className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership
className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.8"
                        port="45578"
                        frequency="1000"
                        dropTime="4000"/>
            <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="localhost"
                      port="4001"
                      autoBind="100"
                      selectorTimeout="100"
                      maxThreads="12"/>

            <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter "> 
              <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSend er"/> 
            </Sender>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDet ector" 
/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispat ch15In 
terceptor"/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.ThroughputInt ercept 
or"/>
          </Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.* \.txt;"/> 

          <Deployer
className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>
          <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
[...]
================

WEB.XML Instance 1 and 2
========================
We added the    <Context distributable="true" /> tag


SERVER.XML (Instance_2)
=======================
[...] Connector Tag
<Connector port="8082" protocol="HTTP/1.1" maxHttpHeaderSize="8192" 
emptySessionPath="true"
maxThreads="150" minSpareThreads="100" maxSpareThreads="300" enableLookups="false" redirectPort="82443" acceptCount="1000" 
               debug="0" connectionTimeout="20000"
disableUploadTimeout="true"
               compression="on"
                           compressionMinSize="2048"
                           noCompressionUserAgents="gozilla, traviata"
                           compressableMimeType="text/html,text/xml" />

[...] Cluster Tag

        <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="6">
<Manager className="org.apache.catalina.ha.session.DeltaManager" 
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel
className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership
className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.8"
                        port="45578"
                        frequency="1000"
                        dropTime="4000"/>
            <Receiver
className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="localhost"
                      port="4002"
                      autoBind="100"
                      selectorTimeout="100"
                      maxThreads="12"/>

            <Sender
className="org.apache.catalina.tribes.transport.ReplicationTransmitter "> 
              <Transport
className="org.apache.catalina.tribes.transport.nio.PooledParallelSend er"/> 
            </Sender>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.TcpFailureDet ector" 
/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.MessageDispat ch15In 
terceptor"/>
            <Interceptor
className="org.apache.catalina.tribes.group.interceptors.ThroughputInt ercept 
or"/>
          </Channel>
<Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;.*\.css;.* \.txt;"/> 
          <Deployer
className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>
          <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
[...]
================


Can someone help us with this problem?
Any suggestion, idea?

I can send more config files or log text if needed.

Regards
Raúl.



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to