[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #7 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #6) > That's kind of par for the course, isn't it? Any IO operation can fail for > any reason. If you don't catch it and handle it, Tomcat will (eventually) > log it. What else did you expect to happen? I was only surprised by number of different IO exceptions. For some (Stream is not writable) it is not very clear what do they mean. > A "client abort" exception happens when the client makes a request and then > hangs-up the phone before you complete the response. It's fairly common and > I wouldn't really expect tcnative to crash under that circumstance, but it's > certainly possible. > > In general, ClientAbortExceptions can safely be completely ignored, so don't > worry too much about the fact that they are happening. On a busy site with > large responses (e.g. images) I'd expect lots of them. I undestand that. As I stated above it was only mix of different IOException types and quantity what confused me. > Can you try one more thing for me? Downgrade to Java 1.8.0_whatever and > revert your configuration to use APR+tcnative again and let us know if the > crashes (actual segfaults that kill the JVM) continue. I managed the downgrade. Alas lots of traffic ceased at the moment and server seems stable. I removed HTTP/2 upgrade as I understand it needs Java 9. Or am I wrong? > But let's take your ClientAbortException stuff onto the user list to see if > we can't find maybe some more efficient ways to get your images generated. > That synchronized block looks suspicious to me, and creating new threads all > the time is likely to lead to instability. In first version i wasnt using threads at all. It seemed unlikely to get several concurrent request for one image. If it would happened the later one could simply overwrite image or respond with error. But since the crashes started i suspected thread concurency to be a problem. New threads are only spawned when images is requested for the first time. Images is downloaded from different machine, resized and webp version is created. WebP is now created with an external program. I was using native lib and imageio before. I use threads and ConcurentHashMap only to ensure that incoming requests will wait until image processing is finished. This only applies for requests for the same image. Others are free to get cached image or to start new resizing thread. Other option would be to allow only one image process at time, but it would have negative performance impact i guess. Another strange thing i got into was out of memory error (4GB) yesterday evening. No obvious reason of the problem. I have deleted image cache, so they are recreated now. Everything runs well. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #8 from Mark Thomas --- You mentioned imageio. I'm not sure at what point you stooped using it but there have been consistent reports over the years of imageio holding on to references to I/O streams and using them long after they should have been finished with. This has been the cause of APR crashes in the past. e.g.: https://markmail.org/message/xr3c5dixxzetpbez -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1838243 - in /tomcat/trunk/java/org/apache/coyote: AbstractProcessorLight.java http2/Http2UpgradeHandler.java http2/LocalStrings.properties http2/StreamProcessor.java
Author: markt Date: Fri Aug 17 08:56:50 2018 New Revision: 1838243 URL: http://svn.apache.org/viewvc?rev=1838243&view=rev Log: Additional debug logging Modified: tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties tomcat/trunk/java/org/apache/coyote/http2/StreamProcessor.java Modified: tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java?rev=1838243&r1=1838242&r2=1838243&view=diff == --- tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java (original) +++ tomcat/trunk/java/org/apache/coyote/AbstractProcessorLight.java Fri Aug 17 08:56:50 2018 @@ -70,16 +70,20 @@ public abstract class AbstractProcessorL state = SocketState.CLOSED; } -if (state != SocketState.CLOSED && isAsync()) { -state = asyncPostProcess(); -} - if (getLog().isDebugEnabled()) { getLog().debug("Socket: [" + socketWrapper + "], Status in: [" + status + "], State out: [" + state + "]"); } +if (state != SocketState.CLOSED && isAsync()) { +state = asyncPostProcess(); +if (getLog().isDebugEnabled()) { +getLog().debug("Socket: [" + socketWrapper + +"], State after async post processing: [" + state + "]"); +} +} + if (dispatches == null || !dispatches.hasNext()) { // Only returns non-null iterator if there are // dispatches to process. Modified: tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java?rev=1838243&r1=1838242&r2=1838243&view=diff == --- tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java (original) +++ tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java Fri Aug 17 08:56:50 2018 @@ -818,11 +818,19 @@ class Http2UpgradeHandler extends Abstra } Response coyoteResponse = ((Stream) stream).getCoyoteResponse(); if (coyoteResponse.getWriteListener() == null) { +if (log.isDebugEnabled()) { +log.debug(sm.getString("upgradeHandler.notifyAll", +connectionId, stream.getIdentifier())); +} // Blocking, so use notify to release StreamOutputBuffer synchronized (stream) { stream.notifyAll(); } } else { +if (log.isDebugEnabled()) { +log.debug(sm.getString("upgradeHandler.dispatchWrite", +connectionId, stream.getIdentifier())); +} // Non-blocking so dispatch coyoteResponse.action(ActionCode.DISPATCH_WRITE, null); // Need to explicitly execute dispatches on the Modified: tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties?rev=1838243&r1=1838242&r2=1838243&view=diff == --- tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties [UTF-8] (original) +++ tomcat/trunk/java/org/apache/coyote/http2/LocalStrings.properties [UTF-8] Fri Aug 17 08:56:50 2018 @@ -104,6 +104,7 @@ stream.outputBuffer.flush.debug=Connecti streamProcessor.error.connection=Connection [{0}], Stream [{1}], An error occurred during processing that was fatal to the connection streamProcessor.error.stream=Connection [{0}], Stream [{1}], An error occurred during processing that was fatal to the stream +streamProcessor.flushBufferedWrite.entry=Connection [{0}], Stream [{1}], Flushing buffered writes streamProcessor.service.error=Error during request processing streamStateMachine.debug.change=Connection [{0}], Stream [{1}], State changed from [{2}] to [{3}] @@ -115,12 +116,14 @@ upgradeHandler.allocate.left=Connection upgradeHandler.allocate.recipient=Connection [{0}], Stream [{1}], potential recipient [{2}] with weight [{3}] upgradeHandler.connectionError=Connection error upgradeHandler.dependency.invalid=Connection [{0}], Stream [{1}], Streams may not depend on themselves +upgradeHandler.dispatchWrite=Connection [{0}], Stream [{1}], Dispatching to container thread for async writ
[Bug 62633] New: WsSession leaks if cache Session are used for group sending
https://bz.apache.org/bugzilla/show_bug.cgi?id=62633 Bug ID: 62633 Summary: WsSession leaks if cache Session are used for group sending Product: Tomcat 8 Version: 8.5.31 Hardware: PC Status: NEW Severity: normal Priority: P2 Component: WebSocket Assignee: dev@tomcat.apache.org Reporter: 1005136...@qq.com Target Milestone: Created attachment 36098 --> https://bz.apache.org/bugzilla/attachment.cgi?id=36098&action=edit code and war public class MySessionContainer { private final static Map WS_SESSION_USER_MAP = new ConcurrentHashMap<>(); public static void addClient(String id, WsSessionUser user) { WS_SESSION_USER_MAP.put(id, user); } public static WsSessionUser getClient(String id) { return WS_SESSION_USER_MAP.get(id); } public static WsSessionUser removeClient(String id) { return WS_SESSION_USER_MAP.remove(id); } } public class WsSessionUser { private Session session; private ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(1, 1, 1, TimeUnit.MINUTES, new LinkedBlockingQueue<>(2)); public WsSessionUser(Session session) { this.session = session; } public void sendMessageByPool(String message) { try { if (!threadPoolExecutor.isShutdown()) { threadPoolExecutor.execute(() -> sendMessage(message)); } } catch (RejectedExecutionException e) { threadPoolExecutor.shutdownNow(); try { if (session.isOpen()) { session.close(); } } catch (IOException ignored) { } } } public void sendMessage(String message) { try { if (session.isOpen()) { session.getBasicRemote().sendText(message); } } catch (IOException e) { e.printStackTrace(); } } public ThreadPoolExecutor getThreadPoolExecutor() { return threadPoolExecutor; } public Session getSession() { return session; } } @ServerEndpoint("/ws/server") public class MyWebSocketServer { @OnOpen public void onOpen(Session session) { System.out.println("server OnOpen"); MySessionContainer.addClient(session.getId(), new WsSessionUser(session)); } @OnClose public void onClose(Session session) { System.out.println("server OnClose"); WsSessionUser wsSessionUser = MySessionContainer.removeClient(session.getId()); if (wsSessionUser != null) { ThreadPoolExecutor threadPoolExecutor = wsSessionUser.getThreadPoolExecutor(); if (!threadPoolExecutor.isShutdown()) { threadPoolExecutor.shutdownNow(); } try { wsSessionUser.getSession().close(); } catch (IOException e) { e.printStackTrace(); } } } @OnMessage public void onMessage(String message, Session session) { System.out.println("server OnMessage"); } @OnError public void onError(Session session, Throwable error) { System.out.println("server OnError"); } } -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot failure in on tomcat-trunk
The Buildbot has detected a new failure on builder tomcat-trunk while building . Full details are available at: https://ci.apache.org/builders/tomcat-trunk/builds/3525 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: silvanus_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' triggered this build Build Source Stamp: [branch tomcat/trunk] 1838243 Blamelist: markt BUILD FAILED: failed compile_1 Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #9 from jan.pfei...@centrum.cz --- (In reply to Mark Thomas from comment #8) > You mentioned imageio. I'm not sure at what point you stooped using it but > there have been consistent reports over the years of imageio holding on to > references to I/O streams and using them long after they should have been > finished with. This has been the cause of APR crashes in the past. e.g.: > https://markmail.org/message/xr3c5dixxzetpbez Yes, you are right, that was the reason. But there were several chrashes after the change. First was from https://bitbucket.org/luciad/webp-imageio, the second is from https://developers.google.com/speed/webp/download (img2webp.exe). -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[VOTE][RESULT] Release Apache Tomcat 9.0.11
The following votes were cast: Binding: +1: markt, fschumacher, violetagg No other votes were cast. Thanks to everyone who contributed to this release. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1838258 - /tomcat/trunk/webapps/docs/changelog.xml
Author: markt Date: Fri Aug 17 13:00:29 2018 New Revision: 1838258 URL: http://svn.apache.org/viewvc?rev=1838258&view=rev Log: Add the release date for 9.0.11 Modified: tomcat/trunk/webapps/docs/changelog.xml Modified: tomcat/trunk/webapps/docs/changelog.xml URL: http://svn.apache.org/viewvc/tomcat/trunk/webapps/docs/changelog.xml?rev=1838258&r1=1838257&r2=1838258&view=diff == --- tomcat/trunk/webapps/docs/changelog.xml (original) +++ tomcat/trunk/webapps/docs/changelog.xml Fri Aug 17 13:00:29 2018 @@ -67,7 +67,7 @@ - + - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r28803 - /dev/tomcat/tomcat-9/v9.0.11/ /release/tomcat/tomcat-9/v9.0.11/
Author: markt Date: Fri Aug 17 13:13:35 2018 New Revision: 28803 Log: Release Apache Tomcat 9.0.11 Added: release/tomcat/tomcat-9/v9.0.11/ - copied from r28802, dev/tomcat/tomcat-9/v9.0.11/ Removed: dev/tomcat/tomcat-9/v9.0.11/ - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot success in on tomcat-trunk
The Buildbot has detected a restored build on builder tomcat-trunk while building . Full details are available at: https://ci.apache.org/builders/tomcat-trunk/builds/3526 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: silvanus_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' triggered this build Build Source Stamp: [branch tomcat/trunk] 1838258 Blamelist: markt Build succeeded! Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 8.5.33
Hi, На нд, 12.08.2018 г. в 11:43 ч. Mark Thomas написа: > > The proposed Apache Tomcat 8.5.33 release is now available for voting. > > The major changes compared to the 8.5.32 release are: > > - Fix issues with Servlet asynchronous listeners when using the > asynchronous Servlet API in conjunction with HTTP/2. > > - When pre-compiling with JspC, report all compilation errors rather > than stopping after the first error. A new option -failFast can be > used to restore the previous behaviour of stopping after the first > error. > Based on a patch provided by Marc Pompl. > > - Make the Jasper (JSP Engine) Java file generation process > multi-threaded. By default, one thread will be used per core. > Based on a patch by Dan Fabulich. > > Along with lots of other bug fixes and improvements. > > It can be obtained from: > https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.33/ > The Maven staging repo is: > https://repository.apache.org/content/repositories/orgapachetomcat-1192/ > The svn tag is: > http://svn.apache.org/repos/asf/tomcat/tc8.5.x/tags/TOMCAT_8_5_33/ > > The proposed 8.5.33 release is: > [ ] Broken - do not release > [X] Stable - go ahead and release as 8.5.33 +1 Regards, Violeta
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #10 from Christopher Schultz --- Have a look at this page: https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the class shown in that wiki page and re-running. If the crashes stop, we can go ahead and blame them on ImageIO, but really tcnative should not crash in this scenario... I'd expect that you'd just get lots of IOExceptions instead. So tcnative isn't doing its job under this circumstance. (n.b.: http/2 does not require Java 9.) Unfortunately, I don't have win32 debugging environment available to me, so I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to investigate. It's obviously somewhere in the tcnative implementation of SSLSocket.handshake (or something it calls). That function has ~100 lines of code in it, but I'm guessing it's crashing towards the beginning, probably on a read. Can you use a DEBUG build of tcnative to get a better crash report? I'm used to seeing more of a stack trace in native crash dumps. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Securing temporary file-uploads
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, A presentation at DEFCON[1] last week showed how many Android applications were improperly using shared external storage on Android devices and could be vulnerable due to improper handling of temporary files. This was your basic: 1. Download a file from a trusted source (e.g. properly-authenticated, encrypted web connection) 2. Buffer the file in temporary storage 3. Read the file back from temporary storage type setup. The problem is that step 2b. might be "an attacker modifies the file". If step 4 is "use the file to update your own application", and the storage system is untrustworthy (which is the case on Android, since it's shared among all apps running on the device), then there is an opportunity to exploit that application. This got me to thinking about how Tomcat handles file uploads. Since Servlet 3.0, containers must provide the facility and Tomcat uses commons-fileupload to handle most of the heavy-lifting. One feature of that facility is the ability to buffer the file on a local disk if it exceeds some defined size -- to avoid creating large in-memory buffers for file uploads. So Tomcat does some form of the steps 1-3 above, except that the file isn't being downloaded... it's being uploaded. Now, it's really up to the application developer to be sure that the user should be allowed to upload a file, but assuming that user has been authenticated, then the application should be able to trust that Tomcat hasn't been subverted during steps 2-3. If the filesystem is hostile (yes, I know you have lots of problems if that's the case), then Tomcat might end up not delivering the same bytes to the application that were originally uploaded by the user. I'm curious whether anyone cares to look at this scenario in order to mitigate it. I can think of a few ways to mitigate such a potential vulnerability: 1. Tomcat computes a signature of the file as it's being written-out, computes the signature of the file as it's being read back-in for the application, and throws an exception if the file appears to be corrupted. Problems with this solution include not being able to detect the problem until most of the bytes have already been sent to the application. 2. Tomcat encrypts the file as its being written to the disk with a temporary symmetric key. Problems with this solution are additional resources (CPU) required for encryption. The LOE for either of the above doesn't seem very high. Is there any appetite for hardening of this type in Tomcat? Situations where the filesystem (or other programs running on the server) is untrustworthy are probably few and far-between, but there are indeed environments where applications are running under a security manager because those applications are untrusted. I didn't check to see whether Tomcat segregates temporary files for uploads between different applications, but I can see a scenario or two where Tomcat might want to protect an application from having its uploads tampered with. Thanks, - -chris [1] https://blog.checkpoint.com/2018/08/12/man-in-the-disk-a-new-attack-surf ace-for-android-apps/ -BEGIN PGP SIGNATURE- Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlt23PwACgkQHPApP6U8 pFhyWQ/9E19l0tN1T37T/GPo3HsLZ+81MlJ/XKY+LrCQsKLG7Q+q1RnIOIjf3PnL uCy12JdH9eIZK96w35RXp6CP8H4t3FwaDEAoajUr1WYiqxi2L44cLxXd84gw0KGE IwF+yaY91YDIqMLnY7vf8Qq4rFygjML4o9Lu2PqffIdzgYbarWIYyhlx9yKX5FMe 1w+QD5qLvjVRWYig3WN5e+np0bDFZH6YAqQumQWDTLIFZrPq7BE6AlXX8cSXOhN8 LLvbULsusJYrDxiS2yximiyDema69vUfKutFKlRCgW0F+feyRSqy5RjjlUgY4KJ6 0gdwXKaHC4L7SaifKwjABQgIt7SIZPOTIaVIDEf+A2Ge+TcH3jiHJ/djcrY+MvoO 73ZOVr4sf0H3hKjDzDB3METpomP40CdjMhhgE/8WinORl/ZBJ6gPcP+wwsgEMo+I wbRnF/mraBaXHNuBZtx/gFciuUwUmyNBw7s2glYuDpC4cPa5r2naKdDz0an82ESu 9g1j2HYOW7htCCEnNYRTdeQPso3i5hkdr7rIW1pyRWspeoI5I5HxjQeIIrEYFRYa CZ1zmJXqezITvqS/yxPmkijnmT45kvtiVld4RROFwxNEmXghY+iDZ2pxvnZozhoI y2Uzu//8NFEeFNCyIV3iP9b3io/uX81iF51K5MqW/wwPyLlYLAg= =U/5d -END PGP SIGNATURE- - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #11 from jan.pfei...@centrum.cz --- (In reply to Christopher Schultz from comment #10) > Have a look at this page: > https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues If I undestand correctly there is only problem when using ImageIO.write with response output stream. I do not use it that way. I write with ImageIO to filesystem. Then, when image is ready, I read it again and write it to response stream via Buffered IN/Out stream. > Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the > class shown in that wiki page and re-running. If the crashes stop, we can go > ahead and blame them on ImageIO, but really tcnative should not crash in > this scenario... I'd expect that you'd just get lots of IOExceptions > instead. So tcnative isn't doing its job under this circumstance. I left if for weekend with Java 8 to see what happens. Yes there are many IOExceptions already. No crash so far. > Unfortunately, I don't have win32 debugging environment available to me, so > I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to > investigate. It's obviously somewhere in the tcnative implementation of > SSLSocket.handshake (or something it calls). That function has ~100 lines of > code in it, but I'm guessing it's crashing towards the beginning, probably > on a read. > > Can you use a DEBUG build of tcnative to get a better crash report? I'm used > to seeing more of a stack trace in native crash dumps. I will try to get DEBUG build on monday. But there is a lot of other debug info already. As I posted earlier i can attach it. Tomcat generated very long bug report from which i posted the top part and sysinfo part, and several gigs big mdmp file. Do you want to see them or we stick to DEBUG build of tcnative plan? -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62626] Tomcat 9.0.10 APR/Native crashes
https://bz.apache.org/bugzilla/show_bug.cgi?id=62626 --- Comment #12 from Christopher Schultz --- (In reply to jan.pfeifer from comment #11) > (In reply to Christopher Schultz from comment #10) > > Have a look at this page: > > https://wiki.apache.org/tomcat/FAQ/KnownIssues#ImageIOIssues > > If I undestand correctly there is only problem when using ImageIO.write with > response output stream. I do not use it that way. I write with ImageIO to > filesystem. Then, when image is ready, I read it again and write it to > response stream via Buffered IN/Out stream. Hmm, yes, this was in cases where ImageIO was being used to write to the servlet output stream. So it looks like that isn't the problem. > > Try going back up to Java 10 and wrapping your streaming-ImageIO uses in the > > class shown in that wiki page and re-running. If the crashes stop, we can go > > ahead and blame them on ImageIO, but really tcnative should not crash in > > this scenario... I'd expect that you'd just get lots of IOExceptions > > instead. So tcnative isn't doing its job under this circumstance. > > I left if for weekend with Java 8 to see what happens. Yes there are many > IOExceptions already. No crash so far. Just to confirm: this is Java 8 with APR+OpenSSL, not NIO+OpenSSL, correct? If that's the case, there might be some kind of incompatibility between tcnative and Java 9. That would be very surprising to me, but that seems to be where the data are leading us. > > Unfortunately, I don't have win32 debugging environment available to me, so > > I'm not able to turn the 0xe6fc9 offset into a meaningful line of code to > > investigate. It's obviously somewhere in the tcnative implementation of > > SSLSocket.handshake (or something it calls). That function has ~100 lines of > > code in it, but I'm guessing it's crashing towards the beginning, probably > > on a read. > > > > Can you use a DEBUG build of tcnative to get a better crash report? I'm used > > to seeing more of a stack trace in native crash dumps. > > I will try to get DEBUG build on monday. But there is a lot of other debug > info already. As I posted earlier i can attach it. Tomcat generated very > long bug report from which i posted the top part and sysinfo part, and > several gigs big mdmp file. Do you want to see them or we stick to DEBUG > build of tcnative plan? The Java stack traces are less important than the native stack trace. There was only a single item in the native stack trace you already posted. Can you post the part of the native report that is labelled "-- T H R E A D ---"? That should include a lot more relevant information. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62582] Please add OWASP Dependency Check to the build for all Tomcat componets
https://bz.apache.org/bugzilla/show_bug.cgi?id=62582 --- Comment #8 from Christopher Schultz --- (In reply to mgrigorov from comment #4) > As Mark explained there is not much to check anyway because Tomcat doesn't > have dependencies. Well... that's not entirely true. Tomcat does rely on: 1. commons-dbcp[2] 2. commons-fileupload 3. ecj 4. commons-daemon 5. tcnative (technically) But my guess is that 1, 2 above wouldn't be detected because they have been modified from their original and re-packaged with "Tomcat" as the product name (instead of e.g. commons-dbcp). OWASP isn't very good at detecting binary dependencies, so it wouldn't detect 5. I'm not saying this is a great idea, but I'm not saying it's totally worthless, either. I think having an ant target that is runnable by anyone who chooses to run it would be a good thing. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1838274 - in /tomcat/tc8.5.x/trunk: ./ java/org/apache/coyote/ java/org/apache/coyote/http2/
Author: markt Date: Fri Aug 17 19:16:36 2018 New Revision: 1838274 URL: http://svn.apache.org/viewvc?rev=1838274&view=rev Log: Additional debug logging Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/java/org/apache/coyote/AbstractProcessorLight.java tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/LocalStrings.properties tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/StreamProcessor.java Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Aug 17 19:16:36 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,1762168,1762172,1762182,1762201-1762202,1762204,1762208,1762288,1762296,1762324,1762348,1762353,1762362,1762374,1762492,1762503,1762505,1762541,1762608,1762710,1762753,1762766,1762769,1762944,1762947,1762953,1763167,1763179,1763232,1763259
svn commit: r1838275 - in /tomcat/trunk: java/org/apache/coyote/http2/ test/org/apache/coyote/http2/ webapps/docs/
Author: markt Date: Fri Aug 17 19:16:56 2018 New Revision: 1838275 URL: http://svn.apache.org/viewvc?rev=1838275&view=rev Log: Additional fixes for output corruption of response bodies when writing large bodies using asynchronous processing over HTTP/2. Added: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (with props) Modified: tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java tomcat/trunk/java/org/apache/coyote/http2/Stream.java tomcat/trunk/test/org/apache/coyote/http2/Http2TestBase.java tomcat/trunk/webapps/docs/changelog.xml Modified: tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java?rev=1838275&r1=1838274&r2=1838275&view=diff == --- tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java (original) +++ tomcat/trunk/java/org/apache/coyote/http2/Http2AsyncUpgradeHandler.java Fri Aug 17 19:16:56 2018 @@ -280,7 +280,7 @@ public class Http2AsyncUpgradeHandler ex // Reserve as much as possible right away int reservation = (sendfile.end - sendfile.pos > Integer.MAX_VALUE) ? Integer.MAX_VALUE : (int) (sendfile.end - sendfile.pos); sendfile.streamReservation = sendfile.stream.reserveWindowSize(reservation, true); -sendfile.connectionReservation = reserveWindowSize(sendfile.stream, sendfile.streamReservation); +sendfile.connectionReservation = reserveWindowSize(sendfile.stream, sendfile.streamReservation, true); } catch (IOException e) { return SendfileState.ERROR; } @@ -340,7 +340,7 @@ public class Http2AsyncUpgradeHandler ex int reservation = (sendfile.end - sendfile.pos > Integer.MAX_VALUE) ? Integer.MAX_VALUE : (int) (sendfile.end - sendfile.pos); sendfile.streamReservation = sendfile.stream.reserveWindowSize(reservation, true); } -sendfile.connectionReservation = reserveWindowSize(sendfile.stream, sendfile.streamReservation); +sendfile.connectionReservation = reserveWindowSize(sendfile.stream, sendfile.streamReservation, true); } } catch (IOException e) { failed (e, sendfile); Modified: tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java?rev=1838275&r1=1838274&r2=1838275&view=diff == --- tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java (original) +++ tomcat/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java Fri Aug 17 19:16:56 2018 @@ -723,7 +723,7 @@ class Http2UpgradeHandler extends Abstra } -int reserveWindowSize(Stream stream, int reservation) throws IOException { +int reserveWindowSize(Stream stream, int reservation, boolean block) throws IOException { // Need to be holding the stream lock so releaseBacklog() can't notify // this thread until after this thread enters wait() int allocation = 0; @@ -775,12 +775,16 @@ class Http2UpgradeHandler extends Abstra } } if (allocation == 0) { -try { -stream.wait(); -} catch (InterruptedException e) { -throw new IOException(sm.getString( - "upgradeHandler.windowSizeReservationInterrupted", connectionId, -stream.getIdentifier(), Integer.toString(reservation)), e); +if (block) { +try { +stream.wait(); +} catch (InterruptedException e) { +throw new IOException(sm.getString( + "upgradeHandler.windowSizeReservationInterrupted", connectionId, +stream.getIdentifier(), Integer.toString(reservation)), e); +} +} else { +return 0; } } } while (allocation == 0); Modified: tomcat/trunk/java/org/apache/coyote/http2/Stream.java URL: http://svn.apache.org/viewvc/tomcat/trunk/java/org/apache/coyote/http2/Stream.java?rev=1838275&r1=1838274&r2=1838275&view=diff == --- tomcat/trunk/java/org/apache/coyote/http2/Stream.java (original) +++ tomcat/trunk/java/org/apache/coyote/http2/Stream.java Fri A
svn commit: r1838276 - in /tomcat/tc8.5.x/trunk: ./ java/org/apache/coyote/http2/ test/org/apache/coyote/http2/ webapps/docs/
Author: markt Date: Fri Aug 17 19:26:24 2018 New Revision: 1838276 URL: http://svn.apache.org/viewvc?rev=1838276&view=rev Log: Additional fixes for output corruption of response bodies when writing large bodies using asynchronous processing over HTTP/2. Added: tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java - copied unchanged from r1838275, tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/Http2UpgradeHandler.java tomcat/tc8.5.x/trunk/java/org/apache/coyote/http2/Stream.java tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/Http2TestBase.java tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Aug 17 19:26:24 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,
svn commit: r1838277 - in /tomcat/trunk: conf/logging.properties test/org/apache/coyote/http2/TestAsync.java
Author: markt Date: Fri Aug 17 19:51:44 2018 New Revision: 1838277 URL: http://svn.apache.org/viewvc?rev=1838277&view=rev Log: Refactor to a parameterized test Modified: tomcat/trunk/conf/logging.properties tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Modified: tomcat/trunk/conf/logging.properties URL: http://svn.apache.org/viewvc/tomcat/trunk/conf/logging.properties?rev=1838277&r1=1838276&r2=1838277&view=diff == --- tomcat/trunk/conf/logging.properties (original) +++ tomcat/trunk/conf/logging.properties Fri Aug 17 19:51:44 2018 @@ -68,7 +68,7 @@ org.apache.catalina.core.ContainerBase.[ #org.apache.jasper.compiler.TldLocationsCache.level = FINE # To see debug messages for HTTP/2 handling, uncomment the following line: -#org.apache.coyote.http2.level = FINE +org.apache.coyote.http2.level = FINE # To see debug messages for WebSocket handling, uncomment the following line: #org.apache.tomcat.websocket.level = FINE Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java URL: http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java?rev=1838277&r1=1838276&r2=1838277&view=diff == --- tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (original) +++ tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Fri Aug 17 19:51:44 2018 @@ -18,6 +18,9 @@ package org.apache.coyote.http2; import java.io.IOException; import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; import javax.servlet.AsyncContext; import javax.servlet.ServletOutputStream; @@ -28,57 +31,57 @@ import javax.servlet.http.HttpServletRes import org.junit.Assert; import org.junit.Test; +import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; import org.apache.catalina.Context; import org.apache.catalina.Wrapper; import org.apache.catalina.startup.Tomcat; +/* + * Based on + * https://bz.apache.org/bugzilla/show_bug.cgi?id=62614 + * https://bz.apache.org/bugzilla/show_bug.cgi?id=62620 + */ +@RunWith(Parameterized.class) public class TestAsync extends Http2TestBase { private static final int BLOCK_SIZE = 0x8000; -// https://bz.apache.org/bugzilla/show_bug.cgi?id=62614 -@Test -public void testEmptyBothWindowsUpdateConnectionFirst() throws Exception { -doEmptyWindowTest(true, false, false); -} - - -@Test -public void testEmptyBothWindowsUpdateStreamFirst() throws Exception { -doEmptyWindowTest(false, false, false); -} - +@Parameterized.Parameters(name = "{index}: expandConnectionFirst[{0}], " + +"connectionUnlimited[{1}], streamUnlimited[{2}]") +public static Collection parameters() { +Boolean[] booleans = new Boolean[] { Boolean.FALSE, Boolean.TRUE }; +List parameterSets = new ArrayList<>(); -@Test -public void testEmptyConnectionWindowUpdateConnectionFirst() throws Exception { -doEmptyWindowTest(true, false, true); +for (Boolean expandConnectionFirst : booleans) { +for (Boolean connectionUnlimited : booleans) { +for (Boolean streamUnlimited : booleans) { +parameterSets.add(new Object[] { +expandConnectionFirst, connectionUnlimited, streamUnlimited +}); +} +} +} +return parameterSets; } -@Test -public void testEmptyConnectionWindowUpdateStreamFirst() throws Exception { -doEmptyWindowTest(false, false, true); -} +private final boolean expandConnectionFirst; +private final boolean connectionUnlimited; +private final boolean streamUnlimited; -@Test -public void testEmptyStreamWindowUpdateConnectionFirst() throws Exception { -doEmptyWindowTest(true, true, false); +public TestAsync(boolean expandConnectionFirst, boolean connectionUnlimited, +boolean streamUnlimited) { +this.expandConnectionFirst = expandConnectionFirst; +this.connectionUnlimited = connectionUnlimited; +this.streamUnlimited = streamUnlimited; } @Test -public void testEmptyStreamWindowUpdateStreamFirst() throws Exception { -doEmptyWindowTest(false, true, false); -} - - -// No point testing when both Stream and Connection are unlimited - - -private void doEmptyWindowTest(boolean expandConnectionFirst, boolean connectionUnlimited, -boolean streamUnlimited) throws Exception { +public void testEmptyWindow() throws Exception { int blockCount = 4; enableHttp2(); @@ -118,39 +121,42 @@ public class TestAsync extends Http2Test // Headers parser.readFrame(true); // Body -int startingWindowSize = ConnectionSettingsBase.DEFAULT_IN
svn commit: r1838278 - in /tomcat/tc8.5.x/trunk: ./ conf/logging.properties test/org/apache/coyote/http2/TestAsync.java
Author: markt Date: Fri Aug 17 19:52:29 2018 New Revision: 1838278 URL: http://svn.apache.org/viewvc?rev=1838278&view=rev Log: Refactor to a parameterized test Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/conf/logging.properties tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Aug 17 19:52:29 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,1762168,1762172,1762182,1762201-1762202,1762204,1762208,1762288,1762296,1762324,1762348,1762353,1762362,1762374,1762492,1762503,1762505,1762541,1762608,1762710,1762753,1762766,1762769,1762944,1762947,1762953,1763167,1763179,1763232,1763259,1763271-1763272,1763276-1763277,1763319-1763320,1763370,1763372,1763375,1763377,1763393,1763412,1763430,1763450,1763462,1763505,1763511-1763512,1763516,1763518,1763520,1763529,17635
svn commit: r1838279 - /tomcat/trunk/conf/logging.properties
Author: markt Date: Fri Aug 17 19:53:01 2018 New Revision: 1838279 URL: http://svn.apache.org/viewvc?rev=1838279&view=rev Log: Undo debugging change Modified: tomcat/trunk/conf/logging.properties Modified: tomcat/trunk/conf/logging.properties URL: http://svn.apache.org/viewvc/tomcat/trunk/conf/logging.properties?rev=1838279&r1=1838278&r2=1838279&view=diff == --- tomcat/trunk/conf/logging.properties (original) +++ tomcat/trunk/conf/logging.properties Fri Aug 17 19:53:01 2018 @@ -68,7 +68,7 @@ org.apache.catalina.core.ContainerBase.[ #org.apache.jasper.compiler.TldLocationsCache.level = FINE # To see debug messages for HTTP/2 handling, uncomment the following line: -org.apache.coyote.http2.level = FINE +#org.apache.coyote.http2.level = FINE # To see debug messages for WebSocket handling, uncomment the following line: #org.apache.tomcat.websocket.level = FINE - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1838280 - in /tomcat/tc8.5.x/trunk: ./ conf/logging.properties
Author: markt Date: Fri Aug 17 19:53:19 2018 New Revision: 1838280 URL: http://svn.apache.org/viewvc?rev=1838280&view=rev Log: Undo debugging change Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/conf/logging.properties Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Aug 17 19:53:19 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,1762168,1762172,1762182,1762201-1762202,1762204,1762208,1762288,1762296,1762324,1762348,1762353,1762362,1762374,1762492,1762503,1762505,1762541,1762608,1762710,1762753,1762766,1762769,1762944,1762947,1762953,1763167,1763179,1763232,1763259,1763271-1763272,1763276-1763277,1763319-1763320,1763370,1763372,1763375,1763377,1763393,1763412,1763430,1763450,1763462,1763505,1763511-1763512,1763516,1763518,1763520,1763529,1763559,1763565,1763568,1763574,1763619,1763634-1763635,1763718,1763748,1763786,17637
svn commit: r1838281 - /tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java
Author: markt Date: Fri Aug 17 20:07:19 2018 New Revision: 1838281 URL: http://svn.apache.org/viewvc?rev=1838281&view=rev Log: Expand async HTTP/2 tests to cover writes on non-container threads Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java URL: http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java?rev=1838281&r1=1838280&r2=1838281&view=diff == --- tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (original) +++ tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Fri Aug 17 20:07:19 2018 @@ -21,6 +21,10 @@ import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.Collection; import java.util.List; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; +import java.util.concurrent.ScheduledExecutorService; +import java.util.concurrent.TimeUnit; import javax.servlet.AsyncContext; import javax.servlet.ServletOutputStream; @@ -49,7 +53,7 @@ public class TestAsync extends Http2Test private static final int BLOCK_SIZE = 0x8000; @Parameterized.Parameters(name = "{index}: expandConnectionFirst[{0}], " + -"connectionUnlimited[{1}], streamUnlimited[{2}]") +"connectionUnlimited[{1}], streamUnlimited[{2}], useNonContainerThreadForWrite[{3}]") public static Collection parameters() { Boolean[] booleans = new Boolean[] { Boolean.FALSE, Boolean.TRUE }; List parameterSets = new ArrayList<>(); @@ -57,9 +61,12 @@ public class TestAsync extends Http2Test for (Boolean expandConnectionFirst : booleans) { for (Boolean connectionUnlimited : booleans) { for (Boolean streamUnlimited : booleans) { -parameterSets.add(new Object[] { -expandConnectionFirst, connectionUnlimited, streamUnlimited -}); +for (Boolean useNonContainerThreadForWrite : booleans) { +parameterSets.add(new Object[] { +expandConnectionFirst, connectionUnlimited, streamUnlimited, +useNonContainerThreadForWrite +}); +} } } } @@ -70,13 +77,15 @@ public class TestAsync extends Http2Test private final boolean expandConnectionFirst; private final boolean connectionUnlimited; private final boolean streamUnlimited; +private final boolean useNonContainerThreadForWrite; public TestAsync(boolean expandConnectionFirst, boolean connectionUnlimited, -boolean streamUnlimited) { +boolean streamUnlimited, boolean useNonContainerThreadForWrite) { this.expandConnectionFirst = expandConnectionFirst; this.connectionUnlimited = connectionUnlimited; this.streamUnlimited = streamUnlimited; +this.useNonContainerThreadForWrite = useNonContainerThreadForWrite; } @@ -91,7 +100,8 @@ public class TestAsync extends Http2Test Context ctxt = tomcat.addContext("", null); Tomcat.addServlet(ctxt, "simple", new SimpleServlet()); ctxt.addServletMappingDecoded("/simple", "simple"); -Wrapper w = Tomcat.addServlet(ctxt, "async", new AsyncServlet(blockCount)); +Wrapper w = Tomcat.addServlet(ctxt, "async", +new AsyncServlet(blockCount, useNonContainerThreadForWrite)); w.setAsyncSupported(true); ctxt.addServletMappingDecoded("/async", "async"); tomcat.start(); @@ -173,11 +183,18 @@ public class TestAsync extends Http2Test private static final long serialVersionUID = 1L; private final int blockLimit; +private final boolean useNonContainerThreadForWrite; +private final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1); +private volatile Future future; -public AsyncServlet(int blockLimit) { +public AsyncServlet(int blockLimit, boolean useNonContainerThreadForWrite) { this.blockLimit = blockLimit; +this.useNonContainerThreadForWrite = useNonContainerThreadForWrite; } +/* + * Not thread-safe. OK for this test. NOt OK for use in the real world. + */ @Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException { @@ -196,19 +213,41 @@ public class TestAsync extends Http2Test @Override public void onWritePossible() throws IOException { +if (useNonContainerThreadForWrite) { +future = scheduler.schedule(new Runnable() { + +@Override +public void run() { +
svn commit: r1838282 - in /tomcat/tc8.5.x/trunk: ./ test/org/apache/coyote/http2/TestAsync.java
Author: markt Date: Fri Aug 17 20:07:38 2018 New Revision: 1838282 URL: http://svn.apache.org/viewvc?rev=1838282&view=rev Log: Expand async HTTP/2 tests to cover writes on non-container threads Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Aug 17 20:07:38 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,1762168,1762172,1762182,1762201-1762202,1762204,1762208,1762288,1762296,1762324,1762348,1762353,1762362,1762374,1762492,1762503,1762505,1762541,1762608,1762710,1762753,1762766,1762769,1762944,1762947,1762953,1763167,1763179,1763232,1763259,1763271-1763272,1763276-1763277,1763319-1763320,1763370,1763372,1763375,1763377,1763393,1763412,1763430,1763450,1763462,1763505,1763511-1763512,1763516,1763518,1763520,1763529,1763559,1763565,1763
svn commit: r1838286 - in /tomcat/trunk: test/org/apache/coyote/http2/TestAsync.java webapps/docs/changelog.xml
Author: markt Date: Fri Aug 17 20:45:42 2018 New Revision: 1838286 URL: http://svn.apache.org/viewvc?rev=1838286&view=rev Log: Expand Async + HTTP/2 tests to cover larger window sizes. Confirm previous fix fixes BZ 62628. Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java tomcat/trunk/webapps/docs/changelog.xml Modified: tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java URL: http://svn.apache.org/viewvc/tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java?rev=1838286&r1=1838285&r2=1838286&view=diff == --- tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java (original) +++ tomcat/trunk/test/org/apache/coyote/http2/TestAsync.java Fri Aug 17 20:45:42 2018 @@ -46,6 +46,7 @@ import org.apache.catalina.startup.Tomca * Based on * https://bz.apache.org/bugzilla/show_bug.cgi?id=62614 * https://bz.apache.org/bugzilla/show_bug.cgi?id=62620 + * https://bz.apache.org/bugzilla/show_bug.cgi?id=62628 */ @RunWith(Parameterized.class) public class TestAsync extends Http2TestBase { @@ -53,7 +54,8 @@ public class TestAsync extends Http2Test private static final int BLOCK_SIZE = 0x8000; @Parameterized.Parameters(name = "{index}: expandConnectionFirst[{0}], " + -"connectionUnlimited[{1}], streamUnlimited[{2}], useNonContainerThreadForWrite[{3}]") +"connectionUnlimited[{1}], streamUnlimited[{2}], useNonContainerThreadForWrite[{3}]," + +"largeInitialWindow[{4}]") public static Collection parameters() { Boolean[] booleans = new Boolean[] { Boolean.FALSE, Boolean.TRUE }; List parameterSets = new ArrayList<>(); @@ -62,10 +64,12 @@ public class TestAsync extends Http2Test for (Boolean connectionUnlimited : booleans) { for (Boolean streamUnlimited : booleans) { for (Boolean useNonContainerThreadForWrite : booleans) { -parameterSets.add(new Object[] { -expandConnectionFirst, connectionUnlimited, streamUnlimited, -useNonContainerThreadForWrite -}); +for (Boolean largeInitialWindow : booleans) { +parameterSets.add(new Object[] { +expandConnectionFirst, connectionUnlimited, streamUnlimited, +useNonContainerThreadForWrite, largeInitialWindow +}); +} } } } @@ -78,20 +82,23 @@ public class TestAsync extends Http2Test private final boolean connectionUnlimited; private final boolean streamUnlimited; private final boolean useNonContainerThreadForWrite; +private final boolean largeInitialWindow; public TestAsync(boolean expandConnectionFirst, boolean connectionUnlimited, -boolean streamUnlimited, boolean useNonContainerThreadForWrite) { +boolean streamUnlimited, boolean useNonContainerThreadForWrite, +boolean largeInitialWindow) { this.expandConnectionFirst = expandConnectionFirst; this.connectionUnlimited = connectionUnlimited; this.streamUnlimited = streamUnlimited; this.useNonContainerThreadForWrite = useNonContainerThreadForWrite; +this.largeInitialWindow = largeInitialWindow; } @Test public void testEmptyWindow() throws Exception { -int blockCount = 4; +int blockCount = 8; enableHttp2(); @@ -106,19 +113,32 @@ public class TestAsync extends Http2Test ctxt.addServletMappingDecoded("/async", "async"); tomcat.start(); +int startingWindowSize; + openClientConnection(); doHttpUpgrade(); sendClientPreface(); validateHttp2InitialResponse(); +// Reset connection window size after intial response +sendWindowUpdate(0, SimpleServlet.CONTENT_LENGTH); + +if (largeInitialWindow) { +startingWindowSize = ((1 << 17) - 1); +SettingValue sv = +new SettingValue(Setting.INITIAL_WINDOW_SIZE.getId(), startingWindowSize); +sendSettings(0, false, sv); +// Test code assumes connection window and stream window size are the same at the start +sendWindowUpdate(0, startingWindowSize - ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE); +} else { +startingWindowSize = ConnectionSettingsBase.DEFAULT_INITIAL_WINDOW_SIZE; +} + byte[] frameHeader = new byte[9]; ByteBuffer headersPayload = ByteBuffer.allocate(128); buildGetRequest(frameHeader, headersPayload, null, 3, "/async"); writeFrame(frameHeader, headersPayload); -// Reset connection window size after intial response -sendWindowU
svn commit: r1838287 - in /tomcat/tc8.5.x/trunk: ./ test/org/apache/coyote/http2/TestAsync.java webapps/docs/changelog.xml
Author: markt Date: Fri Aug 17 20:46:21 2018 New Revision: 1838287 URL: http://svn.apache.org/viewvc?rev=1838287&view=rev Log: Expand Async + HTTP/2 tests to cover larger window sizes. Confirm previous fix fixes BZ 62628. Modified: tomcat/tc8.5.x/trunk/ (props changed) tomcat/tc8.5.x/trunk/test/org/apache/coyote/http2/TestAsync.java tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml Propchange: tomcat/tc8.5.x/trunk/ -- --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Aug 17 20:46:21 2018 @@ -1,2 +1,2 @@ /tomcat/tc8.0.x/trunk:1809644 -/tomcat/trunk:1734785,1734799,1734845,1734928,1735041,1735044,1735480,1735577,1735597,1735599-1735600,1735615,1736145,1736162,1736209,1736280,1736297,1736299,1736489,1736646,1736703,1736836,1736849,1737104-1737105,1737112,1737117,1737119-1737120,1737155,1737157,1737192,1737280,1737339,1737632,1737664,1737715,1737748,1737785,1737834,1737860,1737903,1737959,1738005,1738007,1738014-1738015,1738018,1738022,1738039,1738043,1738059-1738060,1738147,1738149,1738174-1738175,1738261,1738589,1738623-1738625,1738643,1738816,1738850,1738855,1738946-1738948,1738953-1738954,1738979,1738982,1739079-1739081,1739087,1739113,1739153,1739172,1739176,1739191,1739474,1739492,1739726,1739762,1739775,1739814,1739817-1739818,1739975,1740131,1740324,1740465,1740495,1740508-1740509,1740520,1740535,1740707,1740803,1740810,1740969,1740980,1740991,1740997,1741015,1741033,1741036,1741058,1741060,1741080,1741147,1741159,1741164,1741173,1741181,1741190,1741197,1741202,1741208,1741213,1741221,1741225,1741232,1741409 ,1741501,1741677,1741892,1741896,1741984,1742023,1742042,1742071,1742090,1742093,1742101,1742105,1742111,1742139,1742146,1742148,1742166,1742181,1742184,1742187,1742246,1742248-1742251,1742263-1742264,1742268,1742276,1742369,1742387,1742448,1742509-1742512,1742917,1742919,1742933,1742975-1742976,1742984,1742986,1743019,1743115,1743117,1743124-1743125,1743134,1743425,1743554,1743679,1743696-1743698,1743700-1743701,1744058,1744064-1744065,1744125,1744149,1744194,1744229,1744270,1744323,1744432,1744684,1744697,1744705,1744713,1744760,1744786,1745083,1745142-1745143,1745145,1745177,1745179-1745180,1745227,1745248,1745254,1745337,1745467,1745473,1745535,1745576,1745735,1745744,1746304,1746306-1746307,1746319,1746327,1746338,1746340-1746341,1746344,1746427,1746441,1746473,1746490,1746492,1746495-1746496,1746499-1746501,1746503-1746507,1746509,1746549,1746551,1746554,1746556,1746558,1746584,1746620,1746649,1746724,1746939,1746989,1747014,1747028,1747035,1747210,1747225,1747234,1747253,1747 404,1747506,1747536,1747924,1747980,1747993,1748001,1748253,1748452,1748547,1748629,1748676,1748715,1749287,1749296,1749328,1749373,1749465,1749506,1749508,1749665-1749666,1749763,1749865-1749866,1749898,1749978,1749980,1750011,1750015,1750056,1750480,1750617,1750634,1750692,1750697,1750700,1750703,1750707,1750714,1750718,1750723,1750774,1750899,1750975,1750995,1751061,1751097,1751173,1751438,1751447,1751463,1751702,1752212,1752737,1752745,1753078,1753080,1753358,1753363,1754111,1754140-1754141,1754281,1754310,1754445,1754467,1754494,1754496,1754528,1754532-1754533,1754613,1754714,1754874,1754941,1754944,1754950-1754951,1755005,1755007,1755009,1755132,1755180-1755181,1755185,1755190,1755204-1755206,1755208,1755214,1755224,1755227,1755230,1755629,1755646-1755647,1755650,1755653,1755675,1755680,1755683,1755693,1755717,1755731-1755737,1755812,1755828,1755884,1755890,1755918-1755919,1755942,1755958,1755960,1755970,1755993,1756013,1756019,1756039,1756056,1756083-1756114,1756175,1756288-1 756289,1756408-1756410,1756778,1756798,1756878,1756898,1756939,1757123-1757124,1757126,1757128,1757132-1757133,1757136,1757145,1757167-1757168,1757175,1757180,1757182,1757195,1757271,1757278,1757347,1757353-1757354,1757363,1757374,1757399,1757406,1757408,1757485,1757495,1757499,1757527,1757578,1757684,1757722,1757727,1757790,1757799,1757813,1757853,1757883,1757903,1757976,1757997,1758000,1758058,1758072-1758075,1758078-1758079,1758223,1758257,1758261,1758276,1758292,1758369,1758378-1758383,1758421,1758423,1758425-1758427,1758430,1758443,1758448,1758459,1758483,1758486-1758487,1758499,1758525,1758556,1758580,1758582,1758584,1758588,1758842,1759019,1759212,1759224,1759227,1759252,1759274,1759513-1759516,1759611,1759757,1759785-1759790,1760005,1760022,1760109-1760110,1760135,1760200-1760201,1760227,1760300,1760397,1760446,1760454,1760640,1760648,1761057,1761422,1761491,1761498,1761500-1761501,1761550,1761553,1761572,1761574,1761625-1761626,1761628,1761682,1761740,1761752,1762051-176205 3,1762123,1762168,1762172,1762182,1762201-1762202,1762204,1762208,1762288,1762296,1762324,1762348,1762353,1762362,1762374,1762492,1762503,1762505,1762541,1762608,1762710,1762753,1762766,1762769,1762944,1762947,1762953,1763167,1763179,1763232,1763259,1763271-1763272,1763276-1763277,1763319-1763320,1763370,1763372,1763375,1763377,1763393,1763412,1763430,1763450,176
[Bug 62628] Async servlet over HTTP/2 non-blocking write does not work if client sets custom SETTINGS_INITIAL_WINDOW_SIZE
https://bz.apache.org/bugzilla/show_bug.cgi?id=62628 Mark Thomas changed: What|Removed |Added Status|NEW |RESOLVED Resolution|--- |FIXED --- Comment #4 from Mark Thomas --- Fixed in: - trunk for 9.0.12 onwards - 8.5.x for 8.5.34 onwards Thanks again for the bug report. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[VOTE][RESULT] Release Apache Tomcat 8.5.33
The following votes were cast: Binding: +1: markt, fschumacher, violetagg No other votes were cast. The vote therefore passes. Thanks to everyone who contributed towards this release. Mark - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r1838288 - /tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml
Author: markt Date: Fri Aug 17 20:49:48 2018 New Revision: 1838288 URL: http://svn.apache.org/viewvc?rev=1838288&view=rev Log: Update release date Modified: tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml Modified: tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml URL: http://svn.apache.org/viewvc/tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml?rev=1838288&r1=1838287&r2=1838288&view=diff == --- tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml (original) +++ tomcat/tc8.5.x/trunk/webapps/docs/changelog.xml Fri Aug 17 20:49:48 2018 @@ -80,7 +80,7 @@ - + - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
svn commit: r28814 - /dev/tomcat/tomcat-8/v8.5.33/ /release/tomcat/tomcat-8/v8.5.33/
Author: markt Date: Fri Aug 17 20:51:16 2018 New Revision: 28814 Log: Release Apache Tomcat 8.5.33 Added: release/tomcat/tomcat-8/v8.5.33/ - copied from r28813, dev/tomcat/tomcat-8/v8.5.33/ Removed: dev/tomcat/tomcat-8/v8.5.33/ - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot failure in on tomcat-trunk
The Buildbot has detected a new failure on builder tomcat-trunk while building . Full details are available at: https://ci.apache.org/builders/tomcat-trunk/builds/3529 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: silvanus_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' triggered this build Build Source Stamp: [branch tomcat/trunk] 1838281 Blamelist: markt BUILD FAILED: failed compile_1 Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
buildbot success in on tomcat-trunk
The Buildbot has detected a restored build on builder tomcat-trunk while building . Full details are available at: https://ci.apache.org/builders/tomcat-trunk/builds/3530 Buildbot URL: https://ci.apache.org/ Buildslave for this Build: silvanus_ubuntu Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' triggered this build Build Source Stamp: [branch tomcat/trunk] 1838286 Blamelist: markt Build succeeded! Sincerely, -The Buildbot - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62628] Async servlet over HTTP/2 non-blocking write does not work if client sets custom SETTINGS_INITIAL_WINDOW_SIZE
https://bz.apache.org/bugzilla/show_bug.cgi?id=62628 --- Comment #5 from Dapeng Zhang --- Thanks a lot for the fix! Now almost all my integrated tests passed. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[Bug 62635] New: Async servlet over HTTP/2 response.flushBuffer() intermittently fails
https://bz.apache.org/bugzilla/show_bug.cgi?id=62635 Bug ID: 62635 Summary: Async servlet over HTTP/2 response.flushBuffer() intermittently fails Product: Tomcat 9 Version: 9.0.x Hardware: PC OS: Linux Status: NEW Severity: normal Priority: P2 Component: Servlet Assignee: dev@tomcat.apache.org Reporter: zdap...@google.com Target Milestone: - Sorry to bug you again. Now we were almost there, just had some failures occasionally. Test case: import java.io.IOException; import javax.servlet.AsyncContext; import javax.servlet.ServletOutputStream; import javax.servlet.WriteListener; import javax.servlet.annotation.WebServlet; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; @WebServlet(urlPatterns = {"/asyncflush"}, asyncSupported = true) public class AsyncFlush extends HttpServlet { @Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException { final AsyncContext asyncContext = request.startAsync(); response.setStatus(HttpServletResponse.SC_OK); response.setContentType("application/binary"); final ServletOutputStream output = response.getOutputStream(); output.setWriteListener(new WriteListener() { int j; byte[] bytes = new byte[1000]; int expectedBytesSentOut; @Override public void onWritePossible() throws IOException { System.out.println("onWritePossible called"); if (j > 2000) { System.out.println("complete"); System.out.println("expected bytes sent out: " + expectedBytesSentOut); asyncContext.complete(); return; } while(output.isReady()) { if (j % 5 == 4) { System.out.println("start flush, j = " + j); response.flushBuffer(); System.out.println("end flush, j = " + j); } else { System.out.println("start write, j = " + j); output.write(bytes); expectedBytesSentOut += 1000; System.out.println("end write, j = " + j); } j++; } System.out.println("output.isReady() = " + false); } @Override public void onError(Throwable t) { t.printStackTrace(); } }); } } Run the client command: $ nghttp "http://0.0.0.0:8080/asyncflush"; -v The flush operation hangs up. -- You are receiving this mail because: You are the assignee for the bug. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[GUMP@vmgump-vm3]: Project tomcat-trunk-test-nio2 (in module tomcat-trunk) failed
To whom it may engage... This is an automated request, but not an unsolicited one. For more information please visit http://gump.apache.org/nagged.html, and/or contact the folk at gene...@gump.apache.org. Project tomcat-trunk-test-nio2 has an issue affecting its community integration. This issue affects 1 projects, and has been outstanding for 4 runs. The current state of this project is 'Failed', with reason 'Build Failed'. For reference only, the following projects are affected by this: - tomcat-trunk-test-nio2 : Tomcat 9.x, a web server implementing the Java Servlet 4.0, ... Full details are available at: http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/index.html That said, some information snippets are provided here. The following annotations (debug/informational/warning/error messages) were provided: -DEBUG- Dependency on bnd exists, no need to add for property bndlib.jar. -INFO- Failed with reason build failed -INFO- Project Reports in: /srv/gump/public/workspace/tomcat-trunk/output/logs-NIO2 -INFO- Project Reports in: /srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs -WARNING- No directory [/srv/gump/public/workspace/tomcat-trunk/output/test-tmp-NIO2/logs] The following work was performed: http://vmgump-vm3.apache.org/tomcat-trunk/tomcat-trunk-test-nio2/gump_work/build_tomcat-trunk_tomcat-trunk-test-nio2.html Work Name: build_tomcat-trunk_tomcat-trunk-test-nio2 (Type: Build) Work ended in a state of : Failed Elapsed: 27 mins 11 secs Command Line: /usr/lib/jvm/java-8-oracle/bin/java -Djava.awt.headless=true -Dbuild.sysclasspath=only -Dsun.zip.disableMemoryMapping=true org.apache.tools.ant.Main -Dgump.merge=/srv/gump/public/gump/work/merge.xml -Djunit.jar=/srv/gump/public/workspace/junit/target/junit-4.13-SNAPSHOT.jar -Djava.net.preferIPv4Stack=/srv/gump/public/workspace/tomcat-trunk/true -Dobjenesis.jar=/srv/gump/public/workspace/objenesis/main/target/objenesis-2.7-SNAPSHOT.jar -Dtest.reports=output/logs-NIO2 -Dexecute.test.nio2=true -Dexamples.sources.skip=true -Dbase.path=/srv/gump/public/workspace/tomcat-trunk/tomcat-build-libs -Djdt.jar=/srv/gump/packages/eclipse/plugins/R-4.7.3a-201803300640/ecj-4.7.3a.jar -Dbndlib.jar=/srv/gump/packages/bnd/bndlib-4.0.0/biz.aQute.bndlib-4.0.0.jar -Dcommons-daemon.jar=/srv/gump/public/workspace/apache-commons/daemon/target/commons-daemon-1.1.1-SNAPSHOT.jar -Dtest.openssl.path=/srv/gump/public/workspace/openssl-master/dest-20180818/bin/openssl -Dtest.temp=output/test-tmp-NIO2 -Dtest.accesslog=true -Dexecute.test.nio=false -Dbnd.jar=/srv/gump/packages/bnd/bnd-4.0.0/biz.aQute.bnd-4.0.0.jar -Dexecute.test.apr=false -Dtest.excludePerformance=true -Dtest.relaxTiming=true -Deasymock.jar=/srv/gump/public/workspace/easymock/core/target/easymock-3.7-SNAPSHOT.jar -Dhamcrest.jar=/srv/gump/packages/hamcrest/hamcrest-core-1.3.jar -Dcglib.jar=/srv/gump/packages/cglib/cglib-nodep-2.2.jar test [Working Directory: /srv/gump/public/workspace/tomcat-trunk] CLASSPATH: /usr/lib/jvm/java-8-oracle/lib/tools.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/webapps/examples/WEB-INF/classes:/srv/gump/public/workspace/tomcat-trunk/output/testclasses:/srv/gump/public/workspace/ant/dist/lib/ant.jar:/srv/gump/public/workspace/ant/dist/lib/ant-launcher.jar:/srv/gump/public/workspace/ant/dist/lib/ant-jmf.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit.jar:/srv/gump/public/workspace/ant/dist/lib/ant-junit4.jar:/srv/gump/public/workspace/ant/dist/lib/ant-swing.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-resolver.jar:/srv/gump/public/workspace/ant/dist/lib/ant-apache-xalan2.jar:/srv/gump/public/workspace/xml-commons/java/build/resolver.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/bootstrap.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/bin/tomcat-juli.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/annotations-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/servlet-api.ja r:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jsp-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/el-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/websocket-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jaspic-api.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ant.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-storeconfig.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-coyote.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/jasper-el.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-tribes.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/catalina-ha.jar:/srv/gump/public/workspace/tomcat-trunk/output/build/lib/tomcat-api.jar:/srv/gump/public/work