Re: Http11NioProtocol deadlocks with useAsyncIO="true" and HTTPS/2.0

2019-07-16 Thread Rémy Maucherat
On Mon, Jul 15, 2019 at 11:30 PM Manuel Dominguez Sarmiento 
wrote:

> Hi, we had been running Tomcat 9.0.17 for quite some time on our high-load
> production servers, using the attached server.xml configuration.
>
> Upon upgrading to 9.0.21 we started experiencing many random deadlocks. We
> run performance advertising campaigns, and our conversion rates dropped to
> below half of what they usually are, which was an obvious consequence of
> our servers randomly locking up. Plus, it was very easy to reproduce the
> deadlocks, which seemed to "magically unlock" when opening a second
> tab/window and opening the same URL that was locked on the other
> window/tab. Doing this unlocked both windows/tabs at once, immediately.
>
> We found this was only happening on HTTPS, but NOT on HTTP. Furthermore,
> we found this was only happening when the browser negotiated an upgrade to
> HTTPS/2.0
> Once we found this, we temporarily removed the  className="org.apache.coyote.http2.Http2Protocol" /> configuration, and all
> was back to normal.
> However, we need HTTP/2, so we continued to look for a proper solution.
>
> Looking at the Tomcat changelog, we found there have been many changes
> since 9.0.17 related to useAsyncIO and HTTP/2. One particular change for
> 9.0.22 caught our attention:
>
>
> *"Remove a source of potential deadlocks when using HTTP/2 when the
> Connector is configured with useAsyncIO as true. (markt)" *We also found
> the following discussion thread, which describes issues similar to what we
> were experiencing:
>
> http://mail-archives.apache.org/mod_mbox/tomcat-dev/201906.mbox/%3c20190606204631.bab6c8a...@gitbox.apache.org%3e
>
> So we upgraded to 9.0.22 thinking that the deadlock would be gone. But
> alas, it was not. The deadlocks remained.
> We found that 9.0.20 changed the default for useAsyncIO from "false" to
> "true". So we changed useAsyncIO back to what it was when we were running
> 9.0.17 (false) and all is back to normal on 9.0.22
>
> So the conclusion is: there are still deadlock bugs on the NIO connector
> with useAsyncIO="true" and upgrades to HTTP/2.0
> Besides fixing them, we believe that the useAsyncIO default should be
> reverted to "false".
>
> Looking forward to the team's feedback. Thanks,
>

You should investigate on the user list, or if you already have details on
how to reproduce the deadlock and/or stack traces that show where it
occurs, you can create a BZ.

Rémy


[tomcat] branch master updated: Add optional dependencies

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 0b5b2e2  Add optional dependencies
0b5b2e2 is described below

commit 0b5b2e28cfeb63657b28798e8ae54c035aca2951
Author: remm 
AuthorDate: Tue Jul 16 09:12:20 2019 +0200

Add optional dependencies

tomcat-resource.json lists all the bundles so it's better to leave that
in by default so that the native image can build without changes.
---
 res/tomcat-maven/pom.xml | 4 
 1 file changed, 4 deletions(-)

diff --git a/res/tomcat-maven/pom.xml b/res/tomcat-maven/pom.xml
index 54cc499..158642c 100644
--- a/res/tomcat-maven/pom.xml
+++ b/res/tomcat-maven/pom.xml
@@ -57,21 +57,17 @@
 ${tomcat.version}
 
 
-
 
-
 
 
 


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Remove LogManager from the command line since it is useless there

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 07ebf68  Remove LogManager from the command line since it is useless 
there
07ebf68 is described below

commit 07ebf688c038ce28cb37e1b78468dbc45d6e64d4
Author: remm 
AuthorDate: Tue Jul 16 09:29:22 2019 +0200

Remove LogManager from the command line since it is useless there

It seems the log manager has to be set during image compilation (it
would be 
-J-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager),
unfortunately what the JULI log manager does seems to be a problem with
Graal and I haven't got it to work.
---
 res/tomcat-maven/README.md | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/res/tomcat-maven/README.md b/res/tomcat-maven/README.md
index d2592cd..aed08d7 100644
--- a/res/tomcat-maven/README.md
+++ b/res/tomcat-maven/README.md
@@ -91,7 +91,8 @@ As Graal does not support dynamic class loading, all Servlets 
and support classe
 in `/WEB-INF/classes` and `/WEB-INF/lib`, must be included as part of the 
tomcat-maven build process, so they are packaged into the
 `target/tomcat-maven-1.0.jar`.
 
-Run Tomcat with the agent in full trace mode.
+Run Tomcat with the agent in full trace mode. The arguments are identical to 
regular Tomcat with the addition of the trace agent which attempts to
+intercept all relevant reflection calls.
 ```
 cd $TOMCAT_MAVEN
 $JAVA_HOME/bin/java 
-agentlib:native-image-agent=trace-output=$TOMCAT_MAVEN/target/trace-file.json 
-Dcatalina.base=. -Djava.util.logging.config.file=conf/logging.properties 
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -jar 
target/tomcat-maven-1.0.jar
@@ -103,7 +104,7 @@ Generate the final json files using 
native-image-configuration then use native i
 ```
 $JAVA_HOME/bin/native-image-configure generate 
--trace-input=$TOMCAT_MAVEN/target/trace-file.json 
--output-dir=$TOMCAT_MAVEN/target
 $JAVA_HOME/bin/native-image --no-server --allow-incomplete-classpath 
--enable-https 
--initialize-at-build-time=org.eclipse.jdt,org.apache.el.parser.SimpleNode,javax.servlet.jsp.JspFactory,org.apache.jasper.servlet.JasperInitializer,org.apache.jasper.runtime.JspFactoryImpl
 -H:+JNI -H:+ReportUnsupportedElementsAtRuntime -H:+ReportExceptionStackTraces 
-H:EnableURLProtocols=http,https,jar 
-H:ConfigurationFileDirectories=$TOMCAT_MAVEN/target/ 
-H:ReflectionConfigurationFiles=$TOMCAT_MAVEN/tomc [...]
-./tomcat-maven-1.0 -Djava.library.path=$JAVA_HOME/jre/lib/amd64 
-Dcatalina.base=. -Djava.util.logging.config.file=conf/logging.properties 
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
+./tomcat-maven-1.0 -Djava.library.path=$JAVA_HOME/jre/lib/amd64 
-Dcatalina.base=. -Djava.util.logging.config.file=conf/logging.properties
 ```
 
 Running in a container is possible, an example `DockerfileGraal` is given. To 
use a native image in a container that is not identical to the build platform,


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 57129] Regression. Load WEB-INF/lib jarfiles in alphabetical order

2019-07-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=57129

--- Comment #25 from Kuba  ---
Well,
after spending hundreds of man days to migrate our applications (including
legacy) to new java and tomcat 9 I encoutered this... When installing
application on production environment after all tests on test environments.
It's non deterministic!

It's seems that some of you guys have not worked with real, huge and old
application. That's why you are arguing to not implement of lib sorting...

Well, I want to redirect my all negative feelings into something constructive:
if I'll fork tomcat on hithub and implement Philippe Busque proposition - is
there any chance you will accept merge request?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 57129] Regression. Load WEB-INF/lib jarfiles in alphabetical order

2019-07-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=57129

Kuba  changed:

   What|Removed |Added

 CC||jsamc...@gmail.com

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 57129] Regression. Load WEB-INF/lib jarfiles in alphabetical order

2019-07-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=57129

--- Comment #26 from Remy Maucherat  ---
It doesn't look like this will be integrated based on previous comments, but
the move to git is supposed to make fork and customization like this easier.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: CI is back

2019-07-16 Thread Mark Thomas
On July 15, 2019 11:08:22 PM UTC, "Rémy Maucherat"  wrote:
>Hi,
>
>It seems CI is back up :) Great news !
>
>However, there's something odd with the build:
>https://ci.apache.org/builders/tomcat-trunk/builds/4485/steps/compile/logs/stdio
>It's downloading NSIS 2.51 while it should be using 3.04. CI might be
>using
>some old properties here, right ?

That is by design. NSIS 3.x wasn't compatible with the version of WINE 
available on the CI server. That might have changed with the OS upgrade.

From memory (I only have my phone and a poor internet connection  at present) 
the properties are overridden in the buildbot config file we can edit.

Mark


>
>This should actually have failed earlier but before the CI wipe the
>existing NSIS download kept being used without checksum.
>https://ci.apache.org/builders/tomcat-trunk/builds/4458/steps/compile/logs/stdio
>testexist:
> [echo] Testing  for
>/home/buildslave/slave/tomcat-trunk/basepath/nsis-2.51/makensis.exe
>
>Rémy


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Http11NioProtocol deadlocks with useAsyncIO="true" and HTTPS/2.0

2019-07-16 Thread Mark Thomas
On July 16, 2019 7:20:28 AM UTC, "Rémy Maucherat"  wrote:
>On Mon, Jul 15, 2019 at 11:30 PM Manuel Dominguez Sarmiento
>
>wrote:
>
>> Hi, we had been running Tomcat 9.0.17 for quite some time on our
>high-load
>> production servers, using the attached server.xml configuration.
>>
>> Upon upgrading to 9.0.21 we started experiencing many random
>deadlocks. We
>> run performance advertising campaigns, and our conversion rates
>dropped to
>> below half of what they usually are, which was an obvious consequence
>of
>> our servers randomly locking up. Plus, it was very easy to reproduce
>the
>> deadlocks, which seemed to "magically unlock" when opening a second
>> tab/window and opening the same URL that was locked on the other
>> window/tab. Doing this unlocked both windows/tabs at once,
>immediately.
>>
>> We found this was only happening on HTTPS, but NOT on HTTP.
>Furthermore,
>> we found this was only happening when the browser negotiated an
>upgrade to
>> HTTPS/2.0
>> Once we found this, we temporarily removed the > className="org.apache.coyote.http2.Http2Protocol" /> configuration,
>and all
>> was back to normal.
>> However, we need HTTP/2, so we continued to look for a proper
>solution.
>>
>> Looking at the Tomcat changelog, we found there have been many
>changes
>> since 9.0.17 related to useAsyncIO and HTTP/2. One particular change
>for
>> 9.0.22 caught our attention:
>>
>>
>> *"Remove a source of potential deadlocks when using HTTP/2 when the
>> Connector is configured with useAsyncIO as true. (markt)" *We also
>found
>> the following discussion thread, which describes issues similar to
>what we
>> were experiencing:
>>
>>
>http://mail-archives.apache.org/mod_mbox/tomcat-dev/201906.mbox/%3c20190606204631.bab6c8a...@gitbox.apache.org%3e
>>
>> So we upgraded to 9.0.22 thinking that the deadlock would be gone.
>But
>> alas, it was not. The deadlocks remained.
>> We found that 9.0.20 changed the default for useAsyncIO from "false"
>to
>> "true". So we changed useAsyncIO back to what it was when we were
>running
>> 9.0.17 (false) and all is back to normal on 9.0.22
>>
>> So the conclusion is: there are still deadlock bugs on the NIO
>connector
>> with useAsyncIO="true" and upgrades to HTTP/2.0
>> Besides fixing them, we believe that the useAsyncIO default should be
>> reverted to "false".
>>
>> Looking forward to the team's feedback. Thanks,
>>
>
>You should investigate on the user list, or if you already have details
>on
>how to reproduce the deadlock and/or stack traces that show where it
>occurs, you can create a BZ.

+1  a thread dump when the deadlock occurs with blocked thread(s) identified 
should be enough to figure out where things are going wrong.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 57129] Regression. Load WEB-INF/lib jarfiles in alphabetical order

2019-07-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=57129

--- Comment #27 from Mark Thomas  ---
Something to keep in mind is the possible impact of the Java module system in
future versions of the Servlet spec. My understanding is that Java modules do
not allow packages to exist in more than one JAR.

On the plus side, this should reduce the chances of an app having multiple JARs
with the same classes as libraries refactor to meet the requirements of the
Java module system. On the negative side, if Jakarta EE adopts the Java module
system (I'd argue against that but I suspect I'd be in the minority) then the
issue described here is only going to be the start of the problems apps are
going to see.

I continue to be of the view that this is an issue best fixed in applications.

An offer to provide a patch is appreciated. I don't have easy access to the
source code but somewhere in org.apache.catalina.webresources is probably the
place to start.

This might end up as an option in Tomcat, it might turn into better hooks in
Tomcat for a custom resources implementation (there are other use cases I am
aware of that would benefit from that) or it might be a patch that could be
dropped into Tomcat's lib dir.  Where it ends up will depend on what changes
are required. The patch would have to be very minimal and the behaviour
optional to be considered for inclusion in Tomcat.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: CI is back

2019-07-16 Thread Rémy Maucherat
On Tue, Jul 16, 2019 at 10:39 AM Mark Thomas  wrote:

> On July 15, 2019 11:08:22 PM UTC, "Rémy Maucherat" 
> wrote:
> >Hi,
> >
> >It seems CI is back up :) Great news !
> >
> >However, there's something odd with the build:
> >
> https://ci.apache.org/builders/tomcat-trunk/builds/4485/steps/compile/logs/stdio
> >It's downloading NSIS 2.51 while it should be using 3.04. CI might be
> >using
> >some old properties here, right ?
>
> That is by design. NSIS 3.x wasn't compatible with the version of WINE
> available on the CI server. That might have changed with the OS upgrade.
>
> From memory (I only have my phone and a poor internet connection  at
> present) the properties are overridden in the buildbot config file we can
> edit.
>

That doesn't sound too hard but I don't know where that is actually located
and how to access it :(

Rémy


>
> Mark
>
>
> >
> >This should actually have failed earlier but before the CI wipe the
> >existing NSIS download kept being used without checksum.
> >
> https://ci.apache.org/builders/tomcat-trunk/builds/4458/steps/compile/logs/stdio
> >testexist:
> > [echo] Testing  for
> >/home/buildslave/slave/tomcat-trunk/basepath/nsis-2.51/makensis.exe
> >
> >Rémy
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>


[tomcat] branch master updated: Document the limitation of --static native image compilation

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new d34e682  Document the limitation of --static native image compilation
d34e682 is described below

commit d34e682de65f0b8d45f2e392231a5704564957b7
Author: remm 
AuthorDate: Tue Jul 16 17:15:55 2019 +0200

Document the limitation of --static native image compilation

No TLS support is possible in that case, as far as I can see (SunEC is
dynamic, and tomcat-native can be static but JNI cannot use that).
---
 res/tomcat-maven/README.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/res/tomcat-maven/README.md b/res/tomcat-maven/README.md
index aed08d7..a3741a5 100644
--- a/res/tomcat-maven/README.md
+++ b/res/tomcat-maven/README.md
@@ -109,4 +109,5 @@ $JAVA_HOME/bin/native-image --no-server 
--allow-incomplete-classpath --enable-ht
 
 Running in a container is possible, an example `DockerfileGraal` is given. To 
use a native image in a container that is not identical to the build platform,
 the `native-image` call will need to use the additional `--static` parameter 
to statically link libraries (this will then require zlib and glibc
-static libraries).
+static libraries). Due to TLS needing dynamic libraries (SunEC or 
tomcat-native), TLS support is not available. If TLS support is needed, the 
native
+image must instead be built on the target platform (see the GraalVM 
documentation for building a native image in a container).


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63567] New: catalina.sh fails to pass LOGGING_MANAGER to jvm during stop

2019-07-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63567

Bug ID: 63567
   Summary: catalina.sh fails to pass LOGGING_MANAGER to jvm
during stop
   Product: Tomcat 8
   Version: 8.5.41
  Hardware: All
OS: Linux
Status: NEW
  Severity: normal
  Priority: P2
 Component: Catalina
  Assignee: dev@tomcat.apache.org
  Reporter: erem...@uvic.ca
  Target Milestone: 

If LOGGING_MANAGER is set (in setenv.sh) startup processing works correctly,
but during shutdown (catalina.sh stop) a number of errors are logged regarding
logging plugins failing to be created.

 main ERROR Could not create plugin of type class
org.apache.logging.log4j.core.appender.FileAppender for element File:
java.lang.ClassCastException:
org.apache.logging.log4j.core.appender.FileManager$FactoryData cannot be cast
to
org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData
java.lang.ClassCastException:
org.apache.logging.log4j.core.appender.FileManager$FactoryData cannot be cast
to
org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData

Adding the LOGGING_MANAGER to the eval of the java runtime in the "stop" case
of catalina.sh resolves this.

Line 519 of catalina.sh

change

  eval "\"$_RUNJAVA\"" $JAVA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" stop


to

  eval "\"$_RUNJAVA\"" $LOGGING_MANAGER $JAVA_OPTS \
-D$ENDORSED_PROP="\"$JAVA_ENDORSED_DIRS\"" \
-classpath "\"$CLASSPATH\"" \
-Dcatalina.base="\"$CATALINA_BASE\"" \
-Dcatalina.home="\"$CATALINA_HOME\"" \
-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
org.apache.catalina.startup.Bootstrap "$@" stop

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: CI is back

2019-07-16 Thread Mark Thomas
On July 16, 2019 1:17:15 PM UTC, "Rémy Maucherat"  wrote:
>On Tue, Jul 16, 2019 at 10:39 AM Mark Thomas  wrote:
>
>> On July 15, 2019 11:08:22 PM UTC, "Rémy Maucherat" 
>> wrote:
>> >Hi,
>> >
>> >It seems CI is back up :) Great news !
>> >
>> >However, there's something odd with the build:
>> >
>>
>https://ci.apache.org/builders/tomcat-trunk/builds/4485/steps/compile/logs/stdio
>> >It's downloading NSIS 2.51 while it should be using 3.04. CI might
>be
>> >using
>> >some old properties here, right ?
>>
>> That is by design. NSIS 3.x wasn't compatible with the version of
>WINE
>> available on the CI server. That might have changed with the OS
>upgrade.
>>
>> From memory (I only have my phone and a poor internet connection  at
>> present) the properties are overridden in the buildbot config file we
>can
>> edit.
>>
>
>That doesn't sound too hard but I don't know where that is actually
>located
>and how to access it :(

https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/

You'll need to authenticate to read or write.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Allow keeping tcpNoDelay untouched (default)

2019-07-16 Thread Mark Thomas
On July 14, 2019 5:10:11 PM UTC, "František Kučera"  
wrote:
>Hello,
>
>I got this error when new HTTP request comes:
>
>14-Jul-2019 18:18:39.296 SEVERE [http-nio-8080-Acceptor] 
>org.apache.tomcat.util.net.NioEndpoint.setSocketOptions Error setting 
>socket options
>     java.net.SocketException: Operace není podporována
>     at sun.nio.ch.Net.setIntOption0(Native Method)
>     at sun.nio.ch.Net.setSocketOption(Net.java:334)
>     at 
>sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:190)
>     at 
>sun.nio.ch.SocketAdaptor.setBooleanOption(SocketAdaptor.java:271)
>     at 
>sun.nio.ch.SocketAdaptor.setTcpNoDelay(SocketAdaptor.java:306)
>     at 
>org.apache.tomcat.util.net.SocketProperties.setProperties(SocketProperties.java:204)
>     at 
>org.apache.tomcat.util.net.NioEndpoint.setSocketOptions(NioEndpoint.java:401)
>     at 
>org.apache.tomcat.util.net.NioEndpoint.setSocketOptions(NioEndpoint.java:73)
>     at 
>org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:115)
>     at java.lang.Thread.run(Thread.java:748)
>
>do not panic, this does not happen in normal conditions - my setup is 
>kind of special. Given socket does not support the tcpNoDelay option,
>so 
>it throws an exception. To get rid of it, I patched the Tomcat's source
>
>code (see attachment, it patches the latest git version).
>
>I am looking for a more clean solution that could be merged in the 
>official sources. IMHO it might keep the state of socket's tcpNoDelay 
>untouched (i.e. do not call setTcpNoDelay() method) if there is no 
>explicit configuration to set this option.
>
>I looked in the git history and current sources and there is the 
>condition: "if (tcpNoDelay != null)" and:
>
>/**
>  * TCP_NO_DELAY option. JVM default used if not set.
>  */
>protected Boolean tcpNoDelay = Boolean.TRUE;
>
>and it was "tcpNoDelay = null;" in 2008. So it seems that it worked
>this 
>way before... But now it always calls setTcpNoDelay(true) if not 
>configured to false (and then it calls setTcpNoDelay(false)). I have
>not 
>found any way to configure Tomcat to not set this option and keep the 
>socket in its original state.
>
>Or there might be a third option in the tcpNoDelay configuration 
> like 
>"default" or "null" besides the "true" and "false" (i.e. convert some 
>booleans to Booleans in the source code and allow the null value). This
>
>might be more backward compatible behavior.
>
>I can help with writing the patch and testing but I would appreciate 
>some design advice.
>
>Context: I am interested in making applications listen on unix domain 
>sockets instead of TCP ones (e.g. Jetty can work this way) and 
>especially in listening on sockets inherited from parent process (the 
>"useInheritedChannel" in Tomcat's Connector configuration). And unix 
>domain socket has no tcpNoDelay option obviously.
>
>Franta

I recommend creating a bugzilla issue for this so that it doesn't get lost.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Test CI

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 80beab5  Test CI
80beab5 is described below

commit 80beab5ed595acb730c17a0c278cc9bb19083ab9
Author: remm 
AuthorDate: Tue Jul 16 18:15:29 2019 +0200

Test CI
---
 java/org/apache/tomcat/util/compat/GraalCompat.java | 1 -
 1 file changed, 1 deletion(-)

diff --git a/java/org/apache/tomcat/util/compat/GraalCompat.java 
b/java/org/apache/tomcat/util/compat/GraalCompat.java
index 66263a1..cd2c725 100644
--- a/java/org/apache/tomcat/util/compat/GraalCompat.java
+++ b/java/org/apache/tomcat/util/compat/GraalCompat.java
@@ -46,5 +46,4 @@ class GraalCompat extends JreCompat {
 public void disableCachingForJarUrlConnections() throws IOException {
 }
 
-
 }


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Test CI again.

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 52c14e3  Test CI again.
52c14e3 is described below

commit 52c14e393d3b9348575f1b45ff96b79f68cbbbc1
Author: remm 
AuthorDate: Tue Jul 16 18:23:10 2019 +0200

Test CI again.
---
 java/org/apache/tomcat/util/compat/GraalCompat.java | 2 --
 1 file changed, 2 deletions(-)

diff --git a/java/org/apache/tomcat/util/compat/GraalCompat.java 
b/java/org/apache/tomcat/util/compat/GraalCompat.java
index cd2c725..53b67a1 100644
--- a/java/org/apache/tomcat/util/compat/GraalCompat.java
+++ b/java/org/apache/tomcat/util/compat/GraalCompat.java
@@ -35,13 +35,11 @@ class GraalCompat extends JreCompat {
 GRAAL = result;
 }
 
-
 static boolean isSupported() {
 // This property does not exist for a native image
 return GRAAL;
 }
 
-
 @Override
 public void disableCachingForJarUrlConnections() throws IOException {
 }


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Http11NioProtocol deadlocks with useAsyncIO="true" and HTTPS/2.0

2019-07-16 Thread Manuel Dominguez Sarmiento
We could find no Java deadlocked threads at all by inspecting jconsole 
(not with the automatic "find deadlocks" functionality, nor by 
inspection of a thread dump). We performed several thread dumps WHILE 
the deadlock was clearly visible on screen (this was very easily 
reproduceable).


The deadlock is definitely there though and goes away as soon as we turn 
off "useAsyncIO".
Since we could not find Java-level deadlocks, we believe the problem 
probably lies in the interaction with native code.
We are using org.apache.catalina.core.AprLifecycleListener as well as 
Tomcat Native 1.2.23 on Linux.
We could not find any pointers in the Tomcat Native changelog dealing 
with similar issues.


- Manuel Dominguez Sarmiento

On 16/07/2019 05:42, Mark Thomas wrote:

On July 16, 2019 7:20:28 AM UTC, "Rémy Maucherat"  wrote:

On Mon, Jul 15, 2019 at 11:30 PM Manuel Dominguez Sarmiento

wrote:


Hi, we had been running Tomcat 9.0.17 for quite some time on our

high-load

production servers, using the attached server.xml configuration.

Upon upgrading to 9.0.21 we started experiencing many random

deadlocks. We

run performance advertising campaigns, and our conversion rates

dropped to

below half of what they usually are, which was an obvious consequence

of

our servers randomly locking up. Plus, it was very easy to reproduce

the

deadlocks, which seemed to "magically unlock" when opening a second
tab/window and opening the same URL that was locked on the other
window/tab. Doing this unlocked both windows/tabs at once,

immediately.

We found this was only happening on HTTPS, but NOT on HTTP.

Furthermore,

we found this was only happening when the browser negotiated an

upgrade to

HTTPS/2.0
Once we found this, we temporarily removed the  configuration,

and all

was back to normal.
However, we need HTTP/2, so we continued to look for a proper

solution.

Looking at the Tomcat changelog, we found there have been many

changes

since 9.0.17 related to useAsyncIO and HTTP/2. One particular change

for

9.0.22 caught our attention:


*"Remove a source of potential deadlocks when using HTTP/2 when the
Connector is configured with useAsyncIO as true. (markt)" *We also

found

the following discussion thread, which describes issues similar to

what we

were experiencing:



http://mail-archives.apache.org/mod_mbox/tomcat-dev/201906.mbox/%3c20190606204631.bab6c8a...@gitbox.apache.org%3e

So we upgraded to 9.0.22 thinking that the deadlock would be gone.

But

alas, it was not. The deadlocks remained.
We found that 9.0.20 changed the default for useAsyncIO from "false"

to

"true". So we changed useAsyncIO back to what it was when we were

running

9.0.17 (false) and all is back to normal on 9.0.22

So the conclusion is: there are still deadlock bugs on the NIO

connector

with useAsyncIO="true" and upgrades to HTTP/2.0
Besides fixing them, we believe that the useAsyncIO default should be
reverted to "false".

Looking forward to the team's feedback. Thanks,


You should investigate on the user list, or if you already have details
on
how to reproduce the deadlock and/or stack traces that show where it
occurs, you can create a BZ.

+1  a thread dump when the deadlock occurs with blocked thread(s) identified 
should be enough to figure out where things are going wrong.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org





[tomcat] branch master updated: Test CI again.

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 3ca8f10  Test CI again.
3ca8f10 is described below

commit 3ca8f109a1593968ab633c3e526275fb0870fba4
Author: remm 
AuthorDate: Tue Jul 16 18:38:52 2019 +0200

Test CI again.
---
 java/org/apache/tomcat/util/compat/GraalCompat.java | 1 +
 1 file changed, 1 insertion(+)

diff --git a/java/org/apache/tomcat/util/compat/GraalCompat.java 
b/java/org/apache/tomcat/util/compat/GraalCompat.java
index 53b67a1..55c143d 100644
--- a/java/org/apache/tomcat/util/compat/GraalCompat.java
+++ b/java/org/apache/tomcat/util/compat/GraalCompat.java
@@ -40,6 +40,7 @@ class GraalCompat extends JreCompat {
 return GRAAL;
 }
 
+
 @Override
 public void disableCachingForJarUrlConnections() throws IOException {
 }


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: Http11NioProtocol deadlocks with useAsyncIO="true" and HTTPS/2.0

2019-07-16 Thread Rémy Maucherat
On Tue, Jul 16, 2019 at 6:34 PM Manuel Dominguez Sarmiento 
wrote:

> We could find no Java deadlocked threads at all by inspecting jconsole
> (not with the automatic "find deadlocks" functionality, nor by inspection
> of a thread dump). We performed several thread dumps WHILE the deadlock was
> clearly visible on screen (this was very easily reproduceable).
>
> The deadlock is definitely there though and goes away as soon as we turn
> off "useAsyncIO".
> Since we could not find Java-level deadlocks, we believe the problem
> probably lies in the interaction with native code.
> We are using org.apache.catalina.core.AprLifecycleListener as well as
> Tomcat Native 1.2.23 on Linux.
> We could not find any pointers in the Tomcat Native changelog dealing with
> similar issues.
>

Ok, so if you would like to continue investigating please use the user
list. This is not a "deadlock", but rather a request not completing
properly for some reason.

Rémy


>
> - Manuel Dominguez Sarmiento
>
> On 16/07/2019 05:42, Mark Thomas wrote:
>
> On July 16, 2019 7:20:28 AM UTC, "Rémy Maucherat"  
>  wrote:
>
> On Mon, Jul 15, 2019 at 11:30 PM Manuel Dominguez Sarmiento 
> 
> wrote:
>
>
> Hi, we had been running Tomcat 9.0.17 for quite some time on our
>
> high-load
>
> production servers, using the attached server.xml configuration.
>
> Upon upgrading to 9.0.21 we started experiencing many random
>
> deadlocks. We
>
> run performance advertising campaigns, and our conversion rates
>
> dropped to
>
> below half of what they usually are, which was an obvious consequence
>
> of
>
> our servers randomly locking up. Plus, it was very easy to reproduce
>
> the
>
> deadlocks, which seemed to "magically unlock" when opening a second
> tab/window and opening the same URL that was locked on the other
> window/tab. Doing this unlocked both windows/tabs at once,
>
> immediately.
>
> We found this was only happening on HTTPS, but NOT on HTTP.
>
> Furthermore,
>
> we found this was only happening when the browser negotiated an
>
> upgrade to
>
> HTTPS/2.0
> Once we found this, we temporarily removed the  className="org.apache.coyote.http2.Http2Protocol" /> configuration,
>
> and all
>
> was back to normal.
> However, we need HTTP/2, so we continued to look for a proper
>
> solution.
>
> Looking at the Tomcat changelog, we found there have been many
>
> changes
>
> since 9.0.17 related to useAsyncIO and HTTP/2. One particular change
>
> for
>
> 9.0.22 caught our attention:
>
>
> *"Remove a source of potential deadlocks when using HTTP/2 when the
> Connector is configured with useAsyncIO as true. (markt)" *We also
>
> found
>
> the following discussion thread, which describes issues similar to
>
> what we
>
> were experiencing:
>
>
>
> http://mail-archives.apache.org/mod_mbox/tomcat-dev/201906.mbox/%3c20190606204631.bab6c8a...@gitbox.apache.org%3e
>
> So we upgraded to 9.0.22 thinking that the deadlock would be gone.
>
> But
>
> alas, it was not. The deadlocks remained.
> We found that 9.0.20 changed the default for useAsyncIO from "false"
>
> to
>
> "true". So we changed useAsyncIO back to what it was when we were
>
> running
>
> 9.0.17 (false) and all is back to normal on 9.0.22
>
> So the conclusion is: there are still deadlock bugs on the NIO
>
> connector
>
> with useAsyncIO="true" and upgrades to HTTP/2.0
> Besides fixing them, we believe that the useAsyncIO default should be
> reverted to "false".
>
> Looking forward to the team's feedback. Thanks,
>
>
> You should investigate on the user list, or if you already have details
> on
> how to reproduce the deadlock and/or stack traces that show where it
> occurs, you can create a BZ.
>
> +1  a thread dump when the deadlock occurs with blocked thread(s) identified 
> should be enough to figure out where things are going wrong.
>
> Mark
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>
>


Re: Http11NioProtocol deadlocks with useAsyncIO="true" and HTTPS/2.0

2019-07-16 Thread Mark Thomas
On July 16, 2019 4:38:46 PM UTC, "Rémy Maucherat"  wrote:
>On Tue, Jul 16, 2019 at 6:34 PM Manuel Dominguez Sarmiento
>
>wrote:
>
>> We could find no Java deadlocked threads at all by inspecting
>jconsole
>> (not with the automatic "find deadlocks" functionality, nor by
>inspection
>> of a thread dump). We performed several thread dumps WHILE the
>deadlock was
>> clearly visible on screen (this was very easily reproduceable).
>>
>> The deadlock is definitely there though and goes away as soon as we
>turn
>> off "useAsyncIO".
>> Since we could not find Java-level deadlocks, we believe the problem
>> probably lies in the interaction with native code.
>> We are using org.apache.catalina.core.AprLifecycleListener as well as
>> Tomcat Native 1.2.23 on Linux.
>> We could not find any pointers in the Tomcat Native changelog dealing
>with
>> similar issues.
>>
>
>Ok, so if you would like to continue investigating please use the user
>list. This is not a "deadlock", but rather a request not completing
>properly for some reason.

Indeed. Deadlock is probably not exactly the right term. But if the OP can 
provide a thread dump taken when the issue is occurring that is likely to be 
helpful in pointing us in the right direction. As a minimum, we'll be able to 
tell if this is a similar issue to the one previously fixed.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Test CI again

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 82e936f  Test CI again
82e936f is described below

commit 82e936fcb5107825a7c0e329f39bce9e9b8fb650
Author: remm 
AuthorDate: Tue Jul 16 18:53:38 2019 +0200

Test CI again
---
 java/org/apache/tomcat/util/compat/GraalCompat.java | 1 -
 1 file changed, 1 deletion(-)

diff --git a/java/org/apache/tomcat/util/compat/GraalCompat.java 
b/java/org/apache/tomcat/util/compat/GraalCompat.java
index 55c143d..53b67a1 100644
--- a/java/org/apache/tomcat/util/compat/GraalCompat.java
+++ b/java/org/apache/tomcat/util/compat/GraalCompat.java
@@ -40,7 +40,6 @@ class GraalCompat extends JreCompat {
 return GRAAL;
 }
 
-
 @Override
 public void disableCachingForJarUrlConnections() throws IOException {
 }


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[tomcat] branch master updated: Add end callback for set properties

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 1dbd439  Add end callback for set properties
1dbd439 is described below

commit 1dbd43973c1a2c2c4a6b675741d06fc2ba3cf2d0
Author: remm 
AuthorDate: Tue Jul 16 21:17:24 2019 +0200

Add end callback for set properties

When processing server.xml, there is no notification of the end of
configuration for objects. In some rare cases, it could be useful and
the next opportunity (Catalina lifecycle) comes way later. As a result,
add a trivial callback to address this and allow some processing on the
properties while still in the stage of processing server.xml.
---
 java/org/apache/tomcat/util/digester/SetPropertiesRule.java | 8 
 webapps/docs/changelog.xml  | 3 +++
 2 files changed, 11 insertions(+)

diff --git a/java/org/apache/tomcat/util/digester/SetPropertiesRule.java 
b/java/org/apache/tomcat/util/digester/SetPropertiesRule.java
index a3b38c7..78c976c 100644
--- a/java/org/apache/tomcat/util/digester/SetPropertiesRule.java
+++ b/java/org/apache/tomcat/util/digester/SetPropertiesRule.java
@@ -30,6 +30,10 @@ import org.xml.sax.Attributes;
 
 public class SetPropertiesRule extends Rule {
 
+public interface Listener {
+void endSetPropertiesRule();
+}
+
 /**
  * Process the beginning of this element.
  *
@@ -78,6 +82,10 @@ public class SetPropertiesRule extends Rule {
 }
 }
 
+if (top instanceof Listener) {
+((Listener) top).endSetPropertiesRule();
+}
+
 }
 
 
diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index 7963b3b..d686655 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -72,6 +72,9 @@
 PKCS#1 formatted private keys when configuring the internal, in-memory
 key store. (markt)
   
+  
+Add optional notifications when ending the set properties rule. (remm)
+  
 
   
   


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



buildbot failure in on tomcat-trunk

2019-07-16 Thread buildbot
The Buildbot has detected a new failure on builder tomcat-trunk while building 
tomcat. Full details are available at:
https://ci.apache.org/builders/tomcat-trunk/builds/4492

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: asf946_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' 
triggered this build
Build Source Stamp: [branch master] 1dbd43973c1a2c2c4a6b675741d06fc2ba3cf2d0
Blamelist: remm 

BUILD FAILED: failed compile

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: CI is back

2019-07-16 Thread Rémy Maucherat
On Tue, Jul 16, 2019 at 5:51 PM Mark Thomas  wrote:

> On July 16, 2019 1:17:15 PM UTC, "Rémy Maucherat"  wrote:
> >On Tue, Jul 16, 2019 at 10:39 AM Mark Thomas  wrote:
> >
> >> On July 15, 2019 11:08:22 PM UTC, "Rémy Maucherat" 
> >> wrote:
> >> >Hi,
> >> >
> >> >It seems CI is back up :) Great news !
> >> >
> >> >However, there's something odd with the build:
> >> >
> >>
> >
> https://ci.apache.org/builders/tomcat-trunk/builds/4485/steps/compile/logs/stdio
> >> >It's downloading NSIS 2.51 while it should be using 3.04. CI might
> >be
> >> >using
> >> >some old properties here, right ?
> >>
> >> That is by design. NSIS 3.x wasn't compatible with the version of
> >WINE
> >> available on the CI server. That might have changed with the OS
> >upgrade.
> >>
> >> From memory (I only have my phone and a poor internet connection  at
> >> present) the properties are overridden in the buildbot config file we
> >can
> >> edit.
> >>
> >
> >That doesn't sound too hard but I don't know where that is actually
> >located
> >and how to access it :(
>
>
> https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/
>
> You'll need to authenticate to read or write.
>

Thanks, so after quite a few tries, I fixed it.

Next issue is:
/home/buildslave/slave/tomcat-trunk/build/build.xml:2100: The executable
wine was not found on the current path. Wine is required to build the
Windows installer when running a release build on a non-Windows platform.
To skip building the Windows installer, set the skip.installer property in
build.properties

I'll add temp code to skip the installer for now since I have no idea how
to install wine.

Rémy


[tomcat] branch master updated: Improve changelog

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new ab0fe88  Improve changelog
ab0fe88 is described below

commit ab0fe88811b631b56e2110cf0448078d4f364bf4
Author: remm 
AuthorDate: Tue Jul 16 21:38:26 2019 +0200

Improve changelog
---
 webapps/docs/changelog.xml | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/webapps/docs/changelog.xml b/webapps/docs/changelog.xml
index d686655..ecfa725 100644
--- a/webapps/docs/changelog.xml
+++ b/webapps/docs/changelog.xml
@@ -73,7 +73,8 @@
 key store. (markt)
   
   
-Add optional notifications when ending the set properties rule. (remm)
+Add callback when finishing the set properties rule in the digester.
+(remm)
   
 
   


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



[Bug 63568] New: Allow keeping tcpNoDelay untouched (default)

2019-07-16 Thread bugzilla
https://bz.apache.org/bugzilla/show_bug.cgi?id=63568

Bug ID: 63568
   Summary: Allow keeping tcpNoDelay untouched (default)
   Product: Tomcat 9
   Version: unspecified
  Hardware: PC
OS: Linux
Status: NEW
  Severity: normal
  Priority: P2
 Component: Catalina
  Assignee: dev@tomcat.apache.org
  Reporter: issues.apache.org.db7c0r...@frantovo.cz
  Target Milestone: -

Created attachment 36661
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=36661&action=edit
hotfix to enable Tomcat listening on an inherited unix domain socket

Hello,

I got this error when new HTTP request comes:

14-Jul-2019 18:18:39.296 SEVERE [http-nio-8080-Acceptor]
org.apache.tomcat.util.net.NioEndpoint.setSocketOptions Error setting socket
options
java.net.SocketException: Operace není podporována
at sun.nio.ch.Net.setIntOption0(Native Method)
at sun.nio.ch.Net.setSocketOption(Net.java:334)
at
sun.nio.ch.SocketChannelImpl.setOption(SocketChannelImpl.java:190)
at
sun.nio.ch.SocketAdaptor.setBooleanOption(SocketAdaptor.java:271)
at
sun.nio.ch.SocketAdaptor.setTcpNoDelay(SocketAdaptor.java:306)
at
org.apache.tomcat.util.net.SocketProperties.setProperties(SocketProperties.java:204)
at
org.apache.tomcat.util.net.NioEndpoint.setSocketOptions(NioEndpoint.java:401)
at
org.apache.tomcat.util.net.NioEndpoint.setSocketOptions(NioEndpoint.java:73)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:115)
at java.lang.Thread.run(Thread.java:748)

do not panic, this does not happen in normal conditions - my setup is kind of
special. Given socket does not support the tcpNoDelay option, so it throws an
exception. To get rid of it, I patched the Tomcat's source code (see
attachment, it patches the latest git version).

I am looking for a more clean solution that could be merged in the official
sources. I suggest adding a third option in the tcpNoDelay configuration
 like "default" or
"null" besides the "true" and "false" (i.e. convert some booleans to Booleans
in the source code and allow the null value) which will cause that
setTcpNoDelay() method will not be called on new incomming connections/sockets.

Please confirm whether this is a good solution – I will then write a new patch.

Original mailing list thread:


Context: I am interested in making applications listen on unix domain sockets
instead of TCP ones (e.g. Jetty can work this way) and especially in listening
on sockets inherited from parent process (the "useInheritedChannel" in Tomcat's
Connector configuration). And unix domain socket has no tcpNoDelay option
obviously.

Franta

-- 
You are receiving this mail because:
You are the assignee for the bug.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: CI is back

2019-07-16 Thread Mark Thomas
On July 16, 2019 7:28:45 PM UTC, "Rémy Maucherat"  wrote:
>On Tue, Jul 16, 2019 at 5:51 PM Mark Thomas  wrote:
>
>> On July 16, 2019 1:17:15 PM UTC, "Rémy Maucherat" 
>wrote:
>> >On Tue, Jul 16, 2019 at 10:39 AM Mark Thomas 
>wrote:
>> >
>> >> On July 15, 2019 11:08:22 PM UTC, "Rémy Maucherat"
>
>> >> wrote:
>> >> >Hi,
>> >> >
>> >> >It seems CI is back up :) Great news !
>> >> >
>> >> >However, there's something odd with the build:
>> >> >
>> >>
>> >
>>
>https://ci.apache.org/builders/tomcat-trunk/builds/4485/steps/compile/logs/stdio
>> >> >It's downloading NSIS 2.51 while it should be using 3.04. CI
>might
>> >be
>> >> >using
>> >> >some old properties here, right ?
>> >>
>> >> That is by design. NSIS 3.x wasn't compatible with the version of
>> >WINE
>> >> available on the CI server. That might have changed with the OS
>> >upgrade.
>> >>
>> >> From memory (I only have my phone and a poor internet connection 
>at
>> >> present) the properties are overridden in the buildbot config file
>we
>> >can
>> >> edit.
>> >>
>> >
>> >That doesn't sound too hard but I don't know where that is actually
>> >located
>> >and how to access it :(
>>
>>
>>
>https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/
>>
>> You'll need to authenticate to read or write.
>>
>
>Thanks, so after quite a few tries, I fixed it.
>
>Next issue is:
>/home/buildslave/slave/tomcat-trunk/build/build.xml:2100: The
>executable
>wine was not found on the current path. Wine is required to build the
>Windows installer when running a release build on a non-Windows
>platform.
>To skip building the Windows installer, set the skip.installer property
>in
>build.properties
>
>I'll add temp code to skip the installer for now since I have no idea
>how
>to install wine.

We need to ask infra to add wine to the list of packages installed on that box. 
That will need an INFRA jira ticket.

Mark

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Re: CI is back

2019-07-16 Thread Rémy Maucherat
On Tue, Jul 16, 2019 at 10:56 PM Mark Thomas  wrote:

> We need to ask infra to add wine to the list of packages installed on that
> box. That will need an INFRA jira ticket.
>

Ok.
https://issues.apache.org/jira/browse/INFRA-18759

Rémy


[tomcat] branch master updated: Better wording for --static problems

2019-07-16 Thread remm
This is an automated email from the ASF dual-hosted git repository.

remm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git


The following commit(s) were added to refs/heads/master by this push:
 new 5fe9555  Better wording for --static problems
5fe9555 is described below

commit 5fe95550d36f53ebe341bf20ea5ce50306b76f52
Author: remm 
AuthorDate: Tue Jul 16 23:11:14 2019 +0200

Better wording for --static problems
---
 res/tomcat-maven/README.md | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/res/tomcat-maven/README.md b/res/tomcat-maven/README.md
index a3741a5..4188ed9 100644
--- a/res/tomcat-maven/README.md
+++ b/res/tomcat-maven/README.md
@@ -108,6 +108,6 @@ $JAVA_HOME/bin/native-image --no-server 
--allow-incomplete-classpath --enable-ht
 ```
 
 Running in a container is possible, an example `DockerfileGraal` is given. To 
use a native image in a container that is not identical to the build platform,
-the `native-image` call will need to use the additional `--static` parameter 
to statically link libraries (this will then require zlib and glibc
-static libraries). Due to TLS needing dynamic libraries (SunEC or 
tomcat-native), TLS support is not available. If TLS support is needed, the 
native
-image must instead be built on the target platform (see the GraalVM 
documentation for building a native image in a container).
+the `native-image` call will need to use the additional `--static` parameter 
to statically link base libraries (this will then require zlib and glibc
+static libraries). Due to TLS using dynamic libraries (SunEC for JSSE and 
tomcat-native for OpenSSL), TLS support is not available with static linking.
+If TLS support is needed, the native image must instead be built on a platform 
identical to the target platform.


-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



buildbot exception in on tomcat-trunk

2019-07-16 Thread buildbot
The Buildbot has detected a build exception on builder tomcat-trunk while 
building tomcat. Full details are available at:
https://ci.apache.org/builders/tomcat-trunk/builds/4493

Buildbot URL: https://ci.apache.org/

Buildslave for this Build: asf946_ubuntu

Build Reason: The AnyBranchScheduler scheduler named 'on-tomcat-commit' 
triggered this build
Build Source Stamp: [branch master] 5fe95550d36f53ebe341bf20ea5ce50306b76f52
Blamelist: remm 

BUILD FAILED: exception shell_5 upload_2

Sincerely,
 -The Buildbot




-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org



Illegal reflective access in TC 9

2019-07-16 Thread Rainer Jung
I noticed (no regression) warnings during unit test run of the following 
types for modern JDK:


[junit] WARNING: An illegal reflective access operation has occurred
[junit] WARNING: Illegal reflective access by ...
[junit] WARNING: Please consider reporting this to the maintainers 
of ...
[junit] WARNING: Use --illegal-access=warn to enable warnings of 
further illegal reflective access operations
[junit] WARNING: All illegal access operations will be denied in a 
future release


For the two lines with "..." I have the following statistics:

Count type

348   by org.apache.catalina.loader.WebappClassLoaderBase 
(file:/path/to/my/output/classes/) to field 
java.io.ObjectStreamClass$Caches.localDescs


=> clearReferencesObjectStreamClassCaches() / clearCache()

  2   by org.apache.catalina.loader.WebappClassLoaderBase 
(file:/path/to/my/output/classes/) to field 
java.util.TimerThread.newTasksMayBeScheduled


=> clearReferencesStopTimerThread()

  2   by org.apache.catalina.loader.WebappClassLoaderBase 
(file:/path/to/my/output/classes/) to field java.lang.Thread.target


=> clearReferencesThreads()

 12   by net.sf.cglib.core.ReflectUtils$2 
(file:/path/to/my/deps/cglib-2.2.2/cglib-nodep-2.2.2.jar) to method 
java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)


In the docs under 
http://tomcat.apache.org/tomcat-9.0-doc/config/context.html a Java 9 
hint about reflection is only mentioned for 
clearReferencesObjectStreamClassCaches (and clearReferencesRmiTargets). 
Furthermore in the source file WebappClassLoaderBase.java, there's 
comments about Java 9 and an exception capture plus test 
jreCompat.isInstanceOfInaccessibleObjectException() in 
checkThreadLocalsForLeaks() and clearReferencesRmiTargets(), but not all 
of the above. Do we need to sync docs and code plus both with the above 
list of runtime warnings?


Finally the docs contain a hint to -XaddExports:...=ALL-UNNAMED, but our 
startup scripts contain --add-opens=...=ALL-UNNAMED for the same modules 
and packages. Not sure, whether we should sync that and in which direction.


Should I open a BZ?

Regards,

Rainer

-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org