[ANN] Clustering broken 5.5.10-5.5.14
I've been getting a good amount of emails regarding the clustering system being broken in Tomcat 5.5, so I thought I make the announcement here so that we can clear the air once and for all. In tomcat 5.5.10 the underlying communication protocol for the clustering system changed, and not only made the release incompatible with any previous release, but also broke the system completely. This change should have never seen the codebase, but it did. We apologize for this error, it has been fixed in 5.5.15, in the future, we advise developers to not change protocol and low level code in dot-dot releases without complete regression testing. Filip - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Cluster - questions
> It implements aTCP/IP Socket receiver without NIO. This classes very > useful for async mode at some linux platforms. Please, don't remove > them, they are very important for some user. ok, no problem. I will leave them in. > > I think also we must discuss the next steps to change the cluster code. > > Regards > Peter > > > Am 09.02.2006 um 00:40 schrieb Filip Hanik - Dev Lists: > >> What are the SocketReplicationListener and the >> SocketReplicationThread classes? >> >> seems like we are duplicating functionality. I'm planning on >> breaking out the membership and replication logic into a component >> so that we don't mix group messaging logic with session replication >> logic as they are two different classes. >> >> Looks like these classes are only referenced from the >> SimpleTcpCluster and it looks like they are an attempt to do what >> we already do. >> >> Question: Do we need them? Can I remove them? >> >> Filip >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: NIO Connector, please review
I would love to backport it to Tc5.5 and that way also have Comet support in tc5.5.x Filip > +1 - I'm also working on a nio connector ( it is in sandbox ), but the > goals > are quite different, > I want to have it simpler and smaller than the apr model. I think > old-style > connector > should be deprecated, since JDK1.5 is now required there is no point in > having it. > > ( I assume this is intended for tomcat6 ? It could be backported to 5.x, > but > I'm not sure it's a good idea) > > Costin > > On 6/19/06, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote: >> >> gents, >> I've worked on an NIO connector, since the NIO and APR model can be/are >> very similar, this is a fairly straightforward port. >> I'd like for you to review and then continue this as work, as this would >> allow me to have Comet support without the need for native libraries. >> After review, please suggest where I can continue the work, trunk or >> sandbox? >> >> If the attachments don't make it, here is a link >> http://www.halosg.com/connectors.zip >> >> This is not worked completed, but working, I wanna see if I can make >> optimizations, and of course, SSL support >> >> thanks >> Filip >> >> -- >> >> >> Filip Hanik >> >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> >> > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: NIO Connector, please review
> > >> -Original Message- >> From: Filip Hanik - Dev Lists [mailto:[EMAIL PROTECTED] >> Sent: Monday, June 19, 2006 6:31 AM >> To: Tomcat Developers List >> Subject: NIO Connector, please review >> >> gents, >> I've worked on an NIO connector, since the NIO and APR model >> can be/are >> very similar, this is a fairly straightforward port. >> I'd like for you to review and then continue this as work, as >> this would >> allow me to have Comet support without the need for native libraries. >> After review, please suggest where I can continue the work, trunk or >> sandbox? >> >> If the attachments don't make it, here is a link >> http://www.halosg.com/connectors.zip >> >> This is not worked completed, but working, I wanna see if I can make >> optimizations, and of course, SSL support > > I don't see how this could possibly work. You are using non-blocking > sockets, but you consider it to be an error if readSocket returns false > (when all it actually means is that the data hasn't arrived yet :). > > The APR Connector doesn't have this problem since it uses blocking sockets > (for much this reason). there is a discrepency, I need to fix the fill() method, for the other methods, false means, return the socket to the poller, free the thread and come back when you have more data. ala non blocking mode so there is some massaging to do, but they are not that different. And on writing, it does busy loop until it flushed it all out to the os buffer. Filip > >> >> thanks >> Filip >> >> -- >> >> >> Filip Hanik >> > > > > This message is intended only for the use of the person(s) listed above as > the intended recipient(s), and may contain information that is PRIVILEGED > and CONFIDENTIAL. If you are not an intended recipient, you may not read, > copy, or distribute this message or any attachment. If you received this > communication in error, please notify us immediately by e-mail and then > delete all copies of this message and any attachments. > > In addition you should be aware that ordinary (unencrypted) e-mail sent > through the Internet is not secure. Do not send confidential or sensitive > information, such as social security numbers, account numbers, personal > identification numbers and passwords, to us via ordinary (unencrypted) > e-mail. > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: NIO Connector, please review
So it uses NIO to increase concurrency, ie, have more connections open than threads, same way as APR, but writing to the stream is not done non blocking, it still follows the request/response/1-thread model still lot of work to do Filip >> >> >>> -Original Message- >>> From: Filip Hanik - Dev Lists [mailto:[EMAIL PROTECTED] >>> Sent: Monday, June 19, 2006 6:31 AM >>> To: Tomcat Developers List >>> Subject: NIO Connector, please review >>> >>> gents, >>> I've worked on an NIO connector, since the NIO and APR model >>> can be/are >>> very similar, this is a fairly straightforward port. >>> I'd like for you to review and then continue this as work, as >>> this would >>> allow me to have Comet support without the need for native libraries. >>> After review, please suggest where I can continue the work, trunk or >>> sandbox? >>> >>> If the attachments don't make it, here is a link >>> http://www.halosg.com/connectors.zip >>> >>> This is not worked completed, but working, I wanna see if I can make >>> optimizations, and of course, SSL support >> >> I don't see how this could possibly work. You are using non-blocking >> sockets, but you consider it to be an error if readSocket returns false >> (when all it actually means is that the data hasn't arrived yet :). >> >> The APR Connector doesn't have this problem since it uses blocking >> sockets >> (for much this reason). > > there is a discrepency, I need to fix the fill() method, > for the other methods, false means, return the socket to the poller, free > the thread and come back when you have more data. ala non blocking mode > > so there is some massaging to do, but they are not that different. And on > writing, it does busy loop until it flushed it all out to the os buffer. > > > Filip > >> >>> >>> thanks >>> Filip >>> >>> -- >>> >>> >>> Filip Hanik >>> >> >> >> >> This message is intended only for the use of the person(s) listed above >> as >> the intended recipient(s), and may contain information that is >> PRIVILEGED >> and CONFIDENTIAL. If you are not an intended recipient, you may not >> read, >> copy, or distribute this message or any attachment. If you received this >> communication in error, please notify us immediately by e-mail and then >> delete all copies of this message and any attachments. >> >> In addition you should be aware that ordinary (unencrypted) e-mail sent >> through the Internet is not secure. Do not send confidential or >> sensitive >> information, such as social security numbers, account numbers, personal >> identification numbers and passwords, to us via ordinary (unencrypted) >> e-mail. >> >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: NIO Connector, please review
let me fill in the other blanks, the connector will be modified so that it knows when it has enough data to proceed, ala, chunked encoding etc. that is still work to be done. but I wanted let you know how it conceptually could look similar to the APR connector, and how much of the code could be shared. Filip >> >> >>> -Original Message- >>> From: Filip Hanik - Dev Lists [mailto:[EMAIL PROTECTED] >>> Sent: Monday, June 19, 2006 6:31 AM >>> To: Tomcat Developers List >>> Subject: NIO Connector, please review >>> >>> gents, >>> I've worked on an NIO connector, since the NIO and APR model >>> can be/are >>> very similar, this is a fairly straightforward port. >>> I'd like for you to review and then continue this as work, as >>> this would >>> allow me to have Comet support without the need for native libraries. >>> After review, please suggest where I can continue the work, trunk or >>> sandbox? >>> >>> If the attachments don't make it, here is a link >>> http://www.halosg.com/connectors.zip >>> >>> This is not worked completed, but working, I wanna see if I can make >>> optimizations, and of course, SSL support >> >> I don't see how this could possibly work. You are using non-blocking >> sockets, but you consider it to be an error if readSocket returns false >> (when all it actually means is that the data hasn't arrived yet :). >> >> The APR Connector doesn't have this problem since it uses blocking >> sockets >> (for much this reason). > > there is a discrepency, I need to fix the fill() method, > for the other methods, false means, return the socket to the poller, free > the thread and come back when you have more data. ala non blocking mode > > so there is some massaging to do, but they are not that different. And on > writing, it does busy loop until it flushed it all out to the os buffer. > > > Filip > >> >>> >>> thanks >>> Filip >>> >>> -- >>> >>> >>> Filip Hanik >>> >> >> >> >> This message is intended only for the use of the person(s) listed above >> as >> the intended recipient(s), and may contain information that is >> PRIVILEGED >> and CONFIDENTIAL. If you are not an intended recipient, you may not >> read, >> copy, or distribute this message or any attachment. If you received this >> communication in error, please notify us immediately by e-mail and then >> delete all copies of this message and any attachments. >> >> In addition you should be aware that ordinary (unencrypted) e-mail sent >> through the Internet is not secure. Do not send confidential or >> sensitive >> information, such as social security numbers, account numbers, personal >> identification numbers and passwords, to us via ordinary (unencrypted) >> e-mail. >> >> >> - >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: NIO vs BIO speed
man that sucks, I can run the same tests, even hundred thousand iterations, I get a little worse performance than the blocking connector and a little better than the APR connector, and it never crashes for me. I'm in dublin at apachecon this week, but will continue as soon I get back, have a good week! > Filip Hanik - Dev Lists wrote: >> yes, I think they are looking pretty good. And I am fairly confident in >> this new code, as most of it, is old tested APR code. > > (Of course, there hasn't been any new changes, so it's not a surprise it > not working any better for me; I did reboot in the meantime, though, and > I am running JRE 1.5.0_06) > > Performance with NIO is still bad for me (3 times slower than the other > connectors) with /usr/sbin/ab.exe -n 5000 -c 20 -k > http://127.0.0.1:8081/tomcat.gif. Throughput seems to vary wildly during > the test. With /usr/sbin/ab.exe -n 5000 -c 20 > http://127.0.0.1:8081/tomcat.gif (no keepalive), performance is > horrible, and kills the poller thread after a few hundred iterations > with the exception I reported earlier: > Exception in thread "http-8081-Poller-0" java.lang.NullPointerException > at sun.nio.ch.WindowsSelectorImpl$FdMap.remove(Unknown Source) > at sun.nio.ch.WindowsSelectorImpl$FdMap.access$3000(Unknown > Source) > at sun.nio.ch.WindowsSelectorImpl.implDereg(Unknown Source) > at sun.nio.ch.SelectorImpl.processDeregisterQueue(Unknown Source) > at sun.nio.ch.WindowsSelectorImpl.doSelect(Unknown Source) > at sun.nio.ch.SelectorImpl.lockAndDoSelect(Unknown Source) > at sun.nio.ch.SelectorImpl.select(Unknown Source) > at > org.apache.tomcat.util.net.NioEndpoint$Poller.run(NioEndpoint.java:1189) > at java.lang.Thread.run(Unknown Source) > > Rémy > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: NIO vs BIO speed
I'm not seeing the windows problems, I used to have many problems on windows using jdk1.4, but since 1.5 I have no problems at all. I do have a brand new machine, so maybe there is some windows patch on it that I didn't have before, other than that I can't think of anything. I'm showing almost identical results between NIO and blocking IO on both windows and FC4. here is an output from my test run on windows, 100,000 requests on each connector I still have one thing to improve, to use an java.util.concurrent Exchanger on the read that registers back with the Selector and handles the hand off between two threads correctly. C:\development\covalent\ers\apache2.0\bin>java -version java version "1.5.0_06" Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05) Java HotSpot(TM) Client VM (build 1.5.0_06-b05, mixed mode, sharing) --from tomcat startup, two connectors starting up,. Jun 26, 2006 3:16:53 PM org.apache.coyote.http11.Http11Protocol start INFO: Starting Coyote HTTP/1.1 on http-8080 Jun 26, 2006 3:16:53 PM org.apache.coyote.http11.Http11NioProtocol start INFO: Starting Coyote HTTP/1.1 on http-8081 Jun 26, 2006 3:16:53 PM org.apache.catalina.startup.Catalina start INFO: Server startup in 1828 ms C:\development\covalent\ers\apache2.0\bin>ab -n 10 -c 20 -k http://127.0.0.1:8081/tomcat.gif This is ApacheBench, Version 2.0.41-dev <$Revision: 1.2 $> apache-2.0 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/ Benchmarking 127.0.0.1 (be patient) Completed 1 requests Completed 2 requests Completed 3 requests Completed 4 requests Completed 5 requests Completed 6 requests Completed 7 requests Completed 8 requests Completed 9 requests Finished 10 requests Server Software:Apache-Coyote/1.1 Server Hostname:127.0.0.1 Server Port:8081 Document Path: /tomcat.gif Document Length:1934 bytes Concurrency Level: 20 Time taken for tests: 16.734375 seconds Complete requests: 10 Failed requests:0 Write errors: 0 Keep-Alive requests:10 Total transferred: 21640 bytes HTML transferred: 19340 bytes Requests per second:5975.72 [#/sec] (mean) Time per request: 3.347 [ms] (mean) Time per request: 0.167 [ms] (mean, across all concurrent requests) Transfer rate: 12628.38 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect:00 0.0 0 0 Processing: 03 6.3 0 46 Waiting:03 6.3 0 46 Total: 03 6.3 0 46 Percentage of the requests served within a certain time (ms) 50% 0 66% 0 75% 0 80% 15 90% 15 95% 15 98% 15 99% 15 100% 46 (longest request) C:\development\covalent\ers\apache2.0\bin>ab -n 10 -c 20 -k http://127.0.0.1:8080/tomcat.gif This is ApacheBench, Version 2.0.41-dev <$Revision: 1.2 $> apache-2.0 Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright (c) 1998-2002 The Apache Software Foundation, http://www.apache.org/ Benchmarking 127.0.0.1 (be patient) Completed 1 requests Completed 2 requests Completed 3 requests Completed 4 requests Completed 5 requests Completed 6 requests Completed 7 requests Completed 8 requests Completed 9 requests Finished 10 requests Server Software:Apache-Coyote/1.1 Server Hostname:127.0.0.1 Server Port:8080 Document Path: /tomcat.gif Document Length:1934 bytes Concurrency Level: 20 Time taken for tests: 16.937500 seconds Complete requests: 10 Failed requests:0 Write errors: 0 Keep-Alive requests:98218 Total transferred: 216397582 bytes HTML transferred: 193405802 bytes Requests per second:5904.06 [#/sec] (mean) Time per request: 3.388 [ms] (mean) Time per request: 0.169 [ms] (mean, across all concurrent requests) Transfer rate: 12476.75 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect:00 0.3 0 15 Processing: 03 16.8 0 343 Waiting:03 16.7 0 343 Total: 03 16.9 0 343 Percentage of the requests served within a certain time (ms) 50% 0 66% 0 75% 0 80% 0 90% 15 95% 15 98% 15 99% 62 100%343 (longest request) C:\development\covalent\ers\apache2.0\bin> > I haven't tested Filip's connector yet, but that's also my experience with > the AJP/NIO connector: NIO is pretty much useless on Windows. > > -Original Message- > > From: Remy Maucherat [mailto:[EMAIL PROTECTED] > > Sent: Monday, June 26, 2006 4:58 AM > > To: Tomcat Developers
Re: NIO vs BIO speed
yes, I've ran 20 instances of ab (each with 50 connections) at the same time on my windows box, so 1000 concurrent connections. (with tomcat's default maxThread setting, which I think was 200) The NIO connector handles this fine, my box is of course 100% cpu between windows,tomcat and all the ab processes. but it never stops, and the tests runs all the tests and performs pretty well, each ab reports around 1000requests/sec, so I think that windows somehow lines up the processes, not making them truly concurrent, never the less, I need more hardware to run more serious tests. Trying to run jmeter with 1000 connections is useless as jmeter takes up 88% of the CPU so Tomcat gets no cpu to handle the requests with. But I dont think it will have a problem scaling to thousands of keep alive connections. I'd like to see some other folks try out the nio connector to see if they are experiencing the same problems on windows as remy does. it would be nice to establish whether its an anomaly or the majority that would experience what he is experiencing. Filip Mladen Turk wrote: > [EMAIL PROTECTED] wrote: >> I'm not seeing the windows problems, I used to have many problems on >> windows using jdk1.4, but since 1.5 I have no problems at all. I do have a >> brand new machine, so maybe there is some windows patch on it that I >> didn't have before, other than that I can't think of anything. >> > > Did you ever tried using 500+ concurrent connections. > With < 200 BIO is just fine on any machine with 512MB RAM. > > Try to play with larger numbers like few thousands, and > then you'll see the real stuff. > > Regards, > Mladen. > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Filip Hanik - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
