Re: NIO2 connector
Rémy, On 5/14/25 6:31 PM, Rémy Maucherat wrote: On Wed, May 14, 2025 at 8:52 PM Christopher Schultz wrote: Mark, On 5/13/25 11:20 AM, Mark Thomas wrote: All, This was mentioned briefly before (I think on a BZ issue) but needs a wider discussion before taking action - if we do anything. It has been suggested that there isn't much benefit to maintaining the NIO2 connector and that we could simplify maintenance by removing it (deprecating in 11.0.x, removing in 12.0.x). The current code structure would be retained should there ever be an NIO3 or similar that we wanted to add. NIO and NIO2 performance is very similar. The NIO2 code is arguably more complicated. I've always found it harder to get me head around but that could just be me. There is the odd bug with NIO2 we haven't been able to track down. A similar case might be made for dropping NIO and only maintaining NIO2 although, as stated above, I have a preference for NIO over NIO2. What do folks think? +1 for NIO2 removal, though I don't have much of a technical voice on this topic. I know this proposal actually makes the Tomcat code MORE complicated in order to support both NIO and this, but ... I would love for Tomcat 13 (?) to drop NIO and go back to BIO with 100% virtual threads. This does not make sense to me, for almost the same reason that NIO2 stopped being better than NIO in Tomcat, once NIO got the fixes and refactorings. The problems are: - If implementing BIO in Tomcat, you're not actually doing real BIO. Understood. The gain is that Tomcat code becomes simpler while pushing the complexity down into the JVM to handle the non-blockingness. You have to do an async IO style impl on top of BIO. It's not actually that simple. I started writing code and it instantly looked like a big mess. - The BIO API that has to be used is beyond antiquated. You have to do the item above on top of InputStream and OutputStream. Also understood. Doing this for the BIO-oriented servlet APIs is of course trivial. It's the async APIs which get weird if you want to try to implement them on top of BIO semantics. Servlet async was introduced to solve the problem of BIO with limited resources (threads), and developers who saw those benefits likely spent a lot of time re-writing the most performance-sensitive portions of their applications to use it. But it's terribly error-prone from an application-developer standpoint and the code Tomcat has to manage to make it all work is equally error-prone. I suspect that our resident IO experts (you and markt) are better than the average application developer and so I trust the Tomcat code very much. Of course, feel free to try. Maybe I tried the wrong way or something (blocking NIO2 is still on top of async IO, so not good; I think blocking NIO is the same kind of thing too). Even if everything works out just fine, I would not plan on removing NIO as early as the next iteration (if ever). Wouldn't it be great if we could just rewind time to before Servlet async was added and use BIO-on-VT for everything? This is why I said that Tomcat would become /more/ complicated. My hand-wavy proposal goes something like this: For applications that do not use servlet async APIs, use BIO-on-VT for everything and move on with your life. For applications relying on servlet async APIs, continue to use the NIO implementation. It remains to be seen whether BIO-on-VT has the same performance characteristics as (application) hand-coded async code using Tomcat's NIO connector, but I suspect that most application developers would welcome the trade-off for *significantly* more straightforward code. The speed of progress on VT coming from OpenJDK has been fairly fast. Evidently, they have already (provisionally) resolved the issue of synchronized-code pinning VTs to platform threads (they don't in Java 24, I believe), and the remaining issue is that of native code. My sense is that dipping into native code and getting a platform thread pinned is okay, since the remaining use-case for native code in Tomcat (minus applications) is cryptography, which runs pretty quickly and "comes up for air" all the time. Also there's QUIC, if done it would be really all FFM at its code so not VT compatible. My argument is that the points at which control is handed over to native code are "fast" and should return quickly, back into the Java realm where VT benefits are back on the table. -chris - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
(tomcat-tck) branch main updated: Update sigtest-maven-plugin to 2.6
This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat-tck.git The following commit(s) were added to refs/heads/main by this push: new a2908ce Update sigtest-maven-plugin to 2.6 a2908ce is described below commit a2908ce15dcfd065476c672b2e67266120094907 Author: Mark Thomas AuthorDate: Thu May 15 15:15:26 2025 +0100 Update sigtest-maven-plugin to 2.6 --- annotations-tck/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/annotations-tck/pom.xml b/annotations-tck/pom.xml index 50f7db1..b8bd3df 100644 --- a/annotations-tck/pom.xml +++ b/annotations-tck/pom.xml @@ -43,7 +43,7 @@ jakarta.tck sigtest-maven-plugin -2.1 +2.6 org.netbeans.tools - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: NIO2 connector
On Thu, May 15, 2025 at 7:34 PM Christopher Schultz wrote: > > Rémy, > > On 5/14/25 6:31 PM, Rémy Maucherat wrote: > > On Wed, May 14, 2025 at 8:52 PM Christopher Schultz > > wrote: > >> > >> Mark, > >> > >> On 5/13/25 11:20 AM, Mark Thomas wrote: > >>> All, > >>> > >>> This was mentioned briefly before (I think on a BZ issue) but needs a > >>> wider discussion before taking action - if we do anything. > >>> > >>> It has been suggested that there isn't much benefit to maintaining the > >>> NIO2 connector and that we could simplify maintenance by removing it > >>> (deprecating in 11.0.x, removing in 12.0.x). The current code structure > >>> would be retained should there ever be an NIO3 or similar that we wanted > >>> to add. > >>> > >>> NIO and NIO2 performance is very similar. > >>> > >>> The NIO2 code is arguably more complicated. I've always found it harder > >>> to get me head around but that could just be me. > >>> > >>> There is the odd bug with NIO2 we haven't been able to track down. > >>> > >>> A similar case might be made for dropping NIO and only maintaining NIO2 > >>> although, as stated above, I have a preference for NIO over NIO2. > >>> > >>> What do folks think? > >> > >> +1 for NIO2 removal, though I don't have much of a technical voice on > >> this topic. > >> > >> I know this proposal actually makes the Tomcat code MORE complicated in > >> order to support both NIO and this, but ... I would love for Tomcat 13 > >> (?) to drop NIO and go back to BIO with 100% virtual threads. > > > > This does not make sense to me, for almost the same reason that NIO2 > > stopped being better than NIO in Tomcat, once NIO got the fixes and > > refactorings. > > > > The problems are: > > - If implementing BIO in Tomcat, you're not actually doing real BIO. > > Understood. The gain is that Tomcat code becomes simpler while pushing > the complexity down into the JVM to handle the non-blockingness. > > > You have to do an async IO style impl on top of BIO. It's not actually > > that simple. I started writing code and it instantly looked like a big > > mess. > > - The BIO API that has to be used is beyond antiquated. You have to do > > the item above on top of InputStream and OutputStream. > > Also understood. > > Doing this for the BIO-oriented servlet APIs is of course trivial. It's > the async APIs which get weird if you want to try to implement them on > top of BIO semantics. > > Servlet async was introduced to solve the problem of BIO with limited > resources (threads), and developers who saw those benefits likely spent > a lot of time re-writing the most performance-sensitive portions of > their applications to use it. But it's terribly error-prone from an > application-developer standpoint and the code Tomcat has to manage to > make it all work is equally error-prone. I suspect that our resident IO > experts (you and markt) are better than the average application > developer and so I trust the Tomcat code very much. Yes, ok async IO is more complex in many cases. Not always though, for example NIO2 can be really simple for some client tasks, and in others the completion handler feels very natural to use. NIO2 could have been a little bit more feature rich. It would have addressed a lot of the complexity. Unfortunately it didn't evolve after its very impressive initial release. In SocketWrapperBase, I tried to add: - Knowing if the IO call completed without being async or if it is going async. - Asking for complete writes or reads to avoid pointless completion handler invocation loops. - Allow options for handling invalid concurrent IO attempts, and allowing mixing blocking and async IO. The funny thing is that I found out eventually that this expanded NIO2 worked better on top of NIO than NIO2 (it is really the reason why NIO2 the connector is being removed). > > Of course, feel free to try. Maybe I tried the wrong way or something > > (blocking NIO2 is still on top of async IO, so not good; I think > > blocking NIO is the same kind of thing too). Even if everything works > > out just fine, I would not plan on removing NIO as early as the next > > iteration (if ever). > > Wouldn't it be great if we could just rewind time to before Servlet > async was added and use BIO-on-VT for everything? This is why I said > that Tomcat would become /more/ complicated. My hand-wavy proposal goes > something like this: > > For applications that do not use servlet async APIs, use BIO-on-VT for > everything and move on with your life. For applications relying on > servlet async APIs, continue to use the NIO implementation. > > It remains to be seen whether BIO-on-VT has the same performance > characteristics as (application) hand-coded async code using Tomcat's > NIO connector, but I suspect that most application developers would > welcome the trade-off for *significantly* more straightforward code. Yes, of course BIO is easy to do if you think you can avoid decoupling the IO in the application from the actual blocki
Re: NIO2 connector
Rémy, On 5/15/25 5:07 PM, Rémy Maucherat wrote: On Thu, May 15, 2025 at 7:34 PM Christopher Schultz wrote: Rémy, On 5/14/25 6:31 PM, Rémy Maucherat wrote: On Wed, May 14, 2025 at 8:52 PM Christopher Schultz wrote: Mark, On 5/13/25 11:20 AM, Mark Thomas wrote: All, This was mentioned briefly before (I think on a BZ issue) but needs a wider discussion before taking action - if we do anything. It has been suggested that there isn't much benefit to maintaining the NIO2 connector and that we could simplify maintenance by removing it (deprecating in 11.0.x, removing in 12.0.x). The current code structure would be retained should there ever be an NIO3 or similar that we wanted to add. NIO and NIO2 performance is very similar. The NIO2 code is arguably more complicated. I've always found it harder to get me head around but that could just be me. There is the odd bug with NIO2 we haven't been able to track down. A similar case might be made for dropping NIO and only maintaining NIO2 although, as stated above, I have a preference for NIO over NIO2. What do folks think? +1 for NIO2 removal, though I don't have much of a technical voice on this topic. I know this proposal actually makes the Tomcat code MORE complicated in order to support both NIO and this, but ... I would love for Tomcat 13 (?) to drop NIO and go back to BIO with 100% virtual threads. This does not make sense to me, for almost the same reason that NIO2 stopped being better than NIO in Tomcat, once NIO got the fixes and refactorings. The problems are: - If implementing BIO in Tomcat, you're not actually doing real BIO. Understood. The gain is that Tomcat code becomes simpler while pushing the complexity down into the JVM to handle the non-blockingness. You have to do an async IO style impl on top of BIO. It's not actually that simple. I started writing code and it instantly looked like a big mess. - The BIO API that has to be used is beyond antiquated. You have to do the item above on top of InputStream and OutputStream. Also understood. Doing this for the BIO-oriented servlet APIs is of course trivial. It's the async APIs which get weird if you want to try to implement them on top of BIO semantics. Servlet async was introduced to solve the problem of BIO with limited resources (threads), and developers who saw those benefits likely spent a lot of time re-writing the most performance-sensitive portions of their applications to use it. But it's terribly error-prone from an application-developer standpoint and the code Tomcat has to manage to make it all work is equally error-prone. I suspect that our resident IO experts (you and markt) are better than the average application developer and so I trust the Tomcat code very much. Yes, ok async IO is more complex in many cases. Not always though, for example NIO2 can be really simple for some client tasks, and in others the completion handler feels very natural to use. NIO2 could have been a little bit more feature rich. It would have addressed a lot of the complexity. Unfortunately it didn't evolve after its very impressive initial release. In SocketWrapperBase, I tried to add: - Knowing if the IO call completed without being async or if it is going async. - Asking for complete writes or reads to avoid pointless completion handler invocation loops. - Allow options for handling invalid concurrent IO attempts, and allowing mixing blocking and async IO. The funny thing is that I found out eventually that this expanded NIO2 worked better on top of NIO than NIO2 (it is really the reason why NIO2 the connector is being removed). Of course, feel free to try. Maybe I tried the wrong way or something (blocking NIO2 is still on top of async IO, so not good; I think blocking NIO is the same kind of thing too). Even if everything works out just fine, I would not plan on removing NIO as early as the next iteration (if ever). Wouldn't it be great if we could just rewind time to before Servlet async was added and use BIO-on-VT for everything? This is why I said that Tomcat would become /more/ complicated. My hand-wavy proposal goes something like this: For applications that do not use servlet async APIs, use BIO-on-VT for everything and move on with your life. For applications relying on servlet async APIs, continue to use the NIO implementation. It remains to be seen whether BIO-on-VT has the same performance characteristics as (application) hand-coded async code using Tomcat's NIO connector, but I suspect that most application developers would welcome the trade-off for *significantly* more straightforward code. Yes, of course BIO is easy to do if you think you can avoid decoupling the IO in the application from the actual blocking IO. Unfortunately, in addition to all the fancier IO APIs that were introduced in the Servlet API, you also have to dump HTTP/2 and be HTTP/1.1 only. And accept some additional limitations (timeouts are set on the socket - but I suppose at this po
