Assuring Security by testing
Hi devs, I've been investigating Apache Tomcat within my Bachelor's thesis "Application of security test tools in open source" at the Free University of Berlin (FU Berlin) [1]. Basically, I am looking for security measures which have been taken to prevent security leaks/vulnerabilities especially with security test tools Apache Tomcat is a extremely popular servlet engine. The nature of the application offers to compromise the web apps and reveal sensitive data. It does not seem that Tomcat cannot be tested the classic way web apps are, e.g. testing with fuzzer for SQL injection, parameter tampering, path traversal etc. So far, I have search the repository and the ant build.xml, the homepage and the mailing list.The homepage and mailing list revealed no information at all to me. I did find that you refer to security audit conducted against the 5.0 codebase [2]. Unfortunately, no information was given what was found and what measures have been taken afterwards. Security advisories are taken up by a security team [3]. Does this team or any other group/person take any measures to assure security with testing tools, with a special test plan or functional requirements? Thanks in advance, Michael [1] https://www.inf.fu-berlin.de/w/SE/ThesisFOSSSecurityTools [2] http://tomcat.apache.org/tomcat-5.5-doc/security-manager-howto.html [3] http://tomcat.apache.org/security-6.html -- OOXML - Say NO To Microsoft Office broken standard http://www.noooxml.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Assuring Security by testing
Mark Thomas wrote: Michael Osipov wrote: Security advisories are taken up by a security team [3]. Does this team or any other group/person take any measures to assure security with testing tools, with a special test plan or functional requirements? Hello Mark, I did not expect such a quick and long answer. Thanks first of all! We do occasionally receive reports to the security team that provide outputs from various security testing tools. In short, the output is nearly always complete garbage. For example, on one occasion a handful of XSS issues were reported all of which were invalid whilst valid XSS issues (later reported by others) were completely missed. Were you reported the name of the tools with which the garbage out has been produced? I have yet to see an automated security test tool that offers any useful output against the Tomcat code base. I am investigating some tools too but their are still evolving. If you want to test a security audit tool then you can run it against an old 4.1.x, 5.5.x or 6.0.x tag and see if it identifies any of the the issues listed on the security pages. Yes, that's probably what I can do but I am just a developer using tomcat as a servlet engine. I guess, due to tomcats complexity it'd take some time to understand how to run an attack at all. The majority of our security reports come: - from security researches who review, for whatever reason, parts of the code they believe to be vulnerable to attack - users that discover a security issue through normal use We also review every issue to see if there may be other places in the codebase that are affected that the reporter did not mention. For example we had a couple of XSS in the examples and when we looked at the rest of the examples code we found a few more. Every commit is reviewed by three committers before it is applied. Security is one of the considerations when reviewing a patch. Getting off topic a little, where I think automated tools do have something to offer is in the area of finding bugs. Checking for unused variables etc often highlights (usually minor) bugs. Find bugs, PMD, checkstyle, the stuff built in to Eclipse all have something to offer in this area. I am aware of all the tools you cited, but they don't do necessarily security testing (e.g. checkstyle). Did you ever come across LAPSE [1]? I have investigated some tools, maybe they are in your interest to some extent. Check this article [2] on different tools, nikto [3], and Wfuzz [4]. Thanks again. I have to process you answers first before I proceed asking if you don't mind being asked. Mike [1] http://suif.stanford.edu/~livshits/work/lapse/ [2] http://www.tssci-security.com/archives/2007/11/24/2007-security-testing-tools-in-review/ [3] http://www.cirt.net/nikto2 [4] http://www.edge-security.com/wfuzz.php -- OOXML - Say NO To Microsoft Office broken standard http://www.noooxml.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Assuring Security by testing
Mark Thomas wrote: Jim Manico wrote: The Fortify Opensource project automatically scans the Tomcat codebase on a regular basis. This probably only gives you 10% security coverage at best, but it's a free report form a $50k tool. http://opensource.fortifysoftware.com A great example of why I have don't have much faith (hope for the future yes - faith for the current crop no) in these tools. In summary: - they are looking at 4.1.10, 5.5.20 and 6.? - I don't know which TC6 version they analysed (but I suspect it is quite old) since they never responded to my requests to add me to that project and I lost interest - there are so many false positives I got fed up looking at them - the bug reporting is way to clunky compared to just using Eclipse or any other decent IDE - it missed most (all if I recall correctly - I don't have the time or inclination to check) of the XSS issues we know were in 4.1.10 onwards Mark, if I got you and Jim correctly, the free service provided by Coverity is almost worthless because the positive to false positive rate is awefully bad? From your point of view this tool isn't worth 50 k$? I thought the tools are directly given to the projects. If they do not tell you what they have scanned, it's pretty superfluous to me. Thanks -- OOXML - Say NO To Microsoft Office broken standard http://www.noooxml.org - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: svn commit: r1834798 - in /tomcat/trunk/bin: makebase.bat makebase.sh
Am 2018-07-01 um 20:44 schrieb isa...@apache.org:
Author: isapir
Date: Sun Jul 1 18:44:38 2018
New Revision: 1834798
URL: http://svn.apache.org/viewvc?rev=1834798&view=rev
Log:
Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=62500
Added scripts to create CATALINA_BASE directory
Added:
tomcat/trunk/bin/makebase.bat
tomcat/trunk/bin/makebase.sh
Added: tomcat/trunk/bin/makebase.sh
URL:
http://svn.apache.org/viewvc/tomcat/trunk/bin/makebase.sh?rev=1834798&view=auto
==
--- tomcat/trunk/bin/makebase.sh (added)
+++ tomcat/trunk/bin/makebase.sh Sun Jul 1 18:44:38 2018
@@ -0,0 +1,64 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script creates the directory structure required for running Tomcat
+# in a separate directory by pointing $CATALINA_BASE to it. It copies the
+# conf directory from $CATALINA_HOME, and creates empty directories for
+# bin, logs, temp, and work.
+#
+# If the file $CATALINA_HOME/bin/setenv.sh exists then it is copied to
+# the target directory as well.
+
+# first arg is the target directory
+BASE_TGT=$1
Why not call it CATALINA_BASE?
+if [ -z ${BASE_TGT} ]; then
+# target directory not provided; exit
+echo "Usage: makebase "
+exit 1
+fi
+
+HOME_DIR="$(dirname $(dirname $0))"
Why not call it CATALINA_HOME?
+if [ -d ${BASE_TGT} ]; then
+ # target directory exists
+ echo directory exists
Why no double quotes? Why not "target directory exists"?
+# exit if target directory is not empty
+[ "$(ls -A ${BASE_TGT})" ] && \
+echo "target directory is not empty" && \
+exit 1
+else
+# create the target directory
+mkdir -p ${BASE_TGT}
+fi
+
+for dir in bin logs temp work;
Why not webapps? The default Tomcat config (server.xml) refers to
webapps in CATALINA_BASE. lib is missing: catalina.properties
common.loader refers to it.
+do
+# copy directory with permissions and delete contents if any
+cp -a "${HOME_DIR}/${dir}" "${BASE_TGT}/${dir}"
+rm -fr "${BASE_TGT}/${dir}"/*
+done
Why do you copy and then delete? Why not mkdir directory directly?
+# copy conf directory recursively and preserve permissions
+cp -a "${HOME_DIR}/conf" "${BASE_TGT}/"
Preserving permissions will fail if the target is not under root's
control. E.g., a user wants a private Tomcat. He won't be able to work with.
+# copy setenv.sh if exists
+[ -f "${HOME_DIR}/bin/setenv.sh" ] && \
+cp -p "${HOME_DIR}/bin/setenv.sh" "${BASE_TGT}/bin/"
Here you ignore -p.
+echo created CATALINA_BASE directory at $BASE_TGT
Not quoted again.
You should also warn that the ports in the server.xml are likely bound
already.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1834798 - in /tomcat/trunk/bin: makebase.bat makebase.sh
Am 2018-07-01 um 21:33 schrieb Igal Sapir:
On 7/1/2018 12:18 PM, Michael Osipov wrote:
Am 2018-07-01 um 20:44 schrieb isa...@apache.org:
Author: isapir
Date: Sun Jul 1 18:44:38 2018
New Revision: 1834798
URL: http://svn.apache.org/viewvc?rev=1834798&view=rev
Log:
Fix https://bz.apache.org/bugzilla/show_bug.cgi?id=62500
Added scripts to create CATALINA_BASE directory
Added:
tomcat/trunk/bin/makebase.bat
tomcat/trunk/bin/makebase.sh
Added: tomcat/trunk/bin/makebase.sh
URL:
http://svn.apache.org/viewvc/tomcat/trunk/bin/makebase.sh?rev=1834798&view=auto
==
--- tomcat/trunk/bin/makebase.sh (added)
+++ tomcat/trunk/bin/makebase.sh Sun Jul 1 18:44:38 2018
@@ -0,0 +1,64 @@
+#!/bin/sh
+
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License,
Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This script creates the directory structure required for running
Tomcat
+# in a separate directory by pointing $CATALINA_BASE to it. It
copies the
+# conf directory from $CATALINA_HOME, and creates empty directories for
+# bin, logs, temp, and work.
+#
+# If the file $CATALINA_HOME/bin/setenv.sh exists then it is copied to
+# the target directory as well.
+
+# first arg is the target directory
+BASE_TGT=$1
Why not call it CATALINA_BASE?
+if [ -z ${BASE_TGT} ]; then
+ # target directory not provided; exit
+ echo "Usage: makebase "
+ exit 1
+fi
+
+HOME_DIR="$(dirname $(dirname $0))"
Why not call it CATALINA_HOME?
I was thinking that if CATALINA_BASE or CATALINA_HOME are already set
then I shouldn't mess with them.
You could test that and make it idiotproof, but at the end you can
ignore an external CATALINA_HOME because this script is an
implementation detail of the parent CATALINA_HOME and shall not be seen
as portable. CATALINA_BASE could be an alternative to $1:
CATALINA_BASE=${CATALINA_BASE:1} or similar.
+if [ -d ${BASE_TGT} ]; then
+ # target directory exists
+ echo directory exists
Why no double quotes? Why not "target directory exists"?
Will update accordingly, but do the quotes add anything? It seems to
work fine without them.
You should make your strings consistent. Look below, they are quoted.
+ # exit if target directory is not empty
+ [ "$(ls -A ${BASE_TGT})" ] && \
+ echo "target directory is not empty" && \
+ exit 1
+else
+ # create the target directory
+ mkdir -p ${BASE_TGT}
+fi
+
+for dir in bin logs temp work;
Why not webapps? The default Tomcat config (server.xml) refers to
webapps in CATALINA_BASE. lib is missing: catalina.properties
common.loader refers to it.
webapps make sense, though we might want to allow to create an empty
directory rather than copy the contents of the original webapps.
I don't expect you to copy the webapps itself, but have webapps/ dir
created already. The user can decide to link or to copy manager/. I do
link only at work because my manager app is globally configured.
I don't believe that `lib` is required. I have deployed many instances
without `lib` and never noticed an issue. Am I missing something?
It is not required, but if a user wants to have private libs or override
something from Tomcat, this should work out of the box.
catalina.properties allows that.
+do
+ # copy directory with permissions and delete contents if any
+ cp -a "${HOME_DIR}/${dir}" "${BASE_TGT}/${dir}"
+ rm -fr "${BASE_TGT}/${dir}"/*
+done
Why do you copy and then delete? Why not mkdir directory directly?
On Windows I create the directories. On *nix the permissions were
different so I did it that way to copy the original permissions. I was
also thinking of using `chmod` as a different option.
I wouldn't preserve the permissions because you shouldn't make any
assumptions on the target. If you want to go that route with chmod,
you'll need for files *and* directories, plus for consistency, I'd
expect you to have chown support too. Leave both to the user for now.
+# copy conf directory recursively and preserve permissions
+cp -a "${HOME_DIR}/con
Re: svn commit: r1834801 - in /tomcat/trunk/bin: makebase.bat makebase.sh
Am 2018-07-01 um 21:51 schrieb isa...@apache.org: Author: isapir Date: Sun Jul 1 19:51:28 2018 New Revision: 1834801 Modified: tomcat/trunk/bin/makebase.sh URL: http://svn.apache.org/viewvc/tomcat/trunk/bin/makebase.sh?rev=1834801&r1=1834800&r2=1834801&view=diff == --- tomcat/trunk/bin/makebase.sh (original) +++ tomcat/trunk/bin/makebase.sh Sun Jul 1 19:51:28 2018 -echo created CATALINA_BASE directory at $BASE_TGT +echo "Created CATALINA_BASE directory at $BASE_TGT" + +echo "Attention: The ports in server.xml might be bound by a " +echo " different instance. Each instance must bind " +echo " to a unique host:port combination." I am not sure whether "host:port" is enough. Please have look at the default server.xml. It binds at least three ports: shutdown, http, and ajp. Maybe leave the second sentence our and say: "Please revise your configuration.". Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: svn commit: r1834989 - in /tomcat/trunk/bin: makebase.bat makebase.sh
Am 2018-07-03 um 17:31 schrieb ma...@apache.org: Author: markt Date: Tue Jul 3 15:31:37 2018 New Revision: 1834989 URL: http://svn.apache.org/viewvc?rev=1834989&view=rev Log: Fix line-endings Modified: tomcat/trunk/bin/makebase.bat (props changed) tomcat/trunk/bin/makebase.sh (props changed) Propchange: tomcat/trunk/bin/makebase.bat -- svn:eol-style = native That should be CRLF. Propchange: tomcat/trunk/bin/makebase.sh -- svn:eol-style = native Here LF only. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Inconsistencies makebase.sh
Friends, I still see some inconsistencies and portability issues: 1. Some expressions, some are not. It should be consistently quoted to avoid whitespace issues or similar. 2. "cp -a" is not portable. Some platforms don't have it, e.g., HP-UX which I use for approx 10 years with Tomcat. 3. Evaluation uses sometimes backticks sometimes $(). Seems inconsistent. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: makebase script and webapps
Am 2018-07-03 um 18:27 schrieb Mark Thomas: Hi, I've been looking at the makebase script and thinking about potential use cases. Currently, we copy conf recursively but create an empty webapps. This is inconsistent as any directories under conf will (nearly always) be web application specific config. I think we should either copy web applications and config or not copy both. My suggestion is not to copy web applications and config (i.e. sub dirs of /conf) by default and to add a -webapps option (or similar) to the makebase script and only copy them when the option is present. Thoughts? First of all, I don't like the '-a' approach because you make assumptions about the target you don't know while other operations don't do it. I'd simply do "-r". I do agree with your that it is somewhat inconsistent, but we need to consider a very important point which must be true for our script to work properly: CATALINA_HOME has to remain unmodified, i.e., contents of conf, webapps. Though, the new CATALINA_BASE needs to be complete to start a Tomcat instance, and it is complete with an empty webapps/ dir. conf/ is mandatory. Given that you want to add "--webapps", which will you copy? Manager? Host Manager? All of them? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: makebase script and webapps
Am 2018-07-03 um 22:07 schrieb Mark Thomas: On 03/07/18 21:02, Michael Osipov wrote: Am 2018-07-03 um 18:27 schrieb Mark Thomas: Hi, I've been looking at the makebase script and thinking about potential use cases. Currently, we copy conf recursively but create an empty webapps. This is inconsistent as any directories under conf will (nearly always) be web application specific config. I think we should either copy web applications and config or not copy both. My suggestion is not to copy web applications and config (i.e. sub dirs of /conf) by default and to add a -webapps option (or similar) to the makebase script and only copy them when the option is present. Thoughts? First of all, I don't like the '-a' approach because you make assumptions about the target you don't know while other operations don't do it. I'd simply do "-r". I do agree with your that it is somewhat inconsistent, but we need to consider a very important point which must be true for our script to work properly: CATALINA_HOME has to remain unmodified, i.e., contents of conf, webapps. Though, the new CATALINA_BASE needs to be complete to start a Tomcat instance, and it is complete with an empty webapps/ dir. conf/ is mandatory. The start will be unnecessarily messy if there are context.xml files in sub-directories of conf but not corresponding application in webapps. Then why not copy only the necessary files which Tomcat needs to start? Given that you want to add "--webapps", which will you copy? Manager? Host Manager? All of them? All of them. I think it has to be all or nothing. Anything else will get very complicated very quickly for a script. I think most cases won't want to copy (hence the default) but I can think of a few scenarios where it could be useful. So even if it contains some user deployed apps? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: makebase script and webapps
Am 2018-07-03 um 22:12 schrieb Mark Thomas: On 03/07/18 21:10, Michael Osipov wrote: Am 2018-07-03 um 22:07 schrieb Mark Thomas: On 03/07/18 21:02, Michael Osipov wrote: Am 2018-07-03 um 18:27 schrieb Mark Thomas: Hi, I've been looking at the makebase script and thinking about potential use cases. Currently, we copy conf recursively but create an empty webapps. This is inconsistent as any directories under conf will (nearly always) be web application specific config. I think we should either copy web applications and config or not copy both. My suggestion is not to copy web applications and config (i.e. sub dirs of /conf) by default and to add a -webapps option (or similar) to the makebase script and only copy them when the option is present. Thoughts? First of all, I don't like the '-a' approach because you make assumptions about the target you don't know while other operations don't do it. I'd simply do "-r". I do agree with your that it is somewhat inconsistent, but we need to consider a very important point which must be true for our script to work properly: CATALINA_HOME has to remain unmodified, i.e., contents of conf, webapps. Though, the new CATALINA_BASE needs to be complete to start a Tomcat instance, and it is complete with an empty webapps/ dir. conf/ is mandatory. The start will be unnecessarily messy if there are context.xml files in sub-directories of conf but not corresponding application in webapps. Then why not copy only the necessary files which Tomcat needs to start? That is the point of -webapps. Only copy the context.xml files if you are going to copy the contents of webapps. Given that you want to add "--webapps", which will you copy? Manager? Host Manager? All of them? All of them. I think it has to be all or nothing. Anything else will get very complicated very quickly for a script. I think most cases won't want to copy (hence the default) but I can think of a few scenarios where it could be useful. So even if it contains some user deployed apps? Yes. I think that it completely contradicts the spirit of make base: Give me a fully working, but minimal Tomcat instance. At the end your are trying to write a rsync for that. I think that this will add little benefit. It will only workout if people never work with CATALINA_HOME, but only with _BASE like I do at work. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: makebase script and webapps
Am 2018-07-03 um 22:37 schrieb Igal Sapir: On 7/3/2018 1:15 PM, Michael Osipov wrote: Am 2018-07-03 um 22:12 schrieb Mark Thomas: On 03/07/18 21:10, Michael Osipov wrote: Am 2018-07-03 um 22:07 schrieb Mark Thomas: On 03/07/18 21:02, Michael Osipov wrote: Am 2018-07-03 um 18:27 schrieb Mark Thomas: Hi, I've been looking at the makebase script and thinking about potential use cases. Currently, we copy conf recursively but create an empty webapps. This is inconsistent as any directories under conf will (nearly always) be web application specific config. I think we should either copy web applications and config or not copy both. My suggestion is not to copy web applications and config (i.e. sub dirs of /conf) by default and to add a -webapps option (or similar) to the makebase script and only copy them when the option is present. Thoughts? First of all, I don't like the '-a' approach because you make assumptions about the target you don't know while other operations don't do it. I'd simply do "-r". I do agree with your that it is somewhat inconsistent, but we need to consider a very important point which must be true for our script to work properly: CATALINA_HOME has to remain unmodified, i.e., contents of conf, webapps. Though, the new CATALINA_BASE needs to be complete to start a Tomcat instance, and it is complete with an empty webapps/ dir. conf/ is mandatory. The start will be unnecessarily messy if there are context.xml files in sub-directories of conf but not corresponding application in webapps. Then why not copy only the necessary files which Tomcat needs to start? That is the point of -webapps. Only copy the context.xml files if you are going to copy the contents of webapps. Given that you want to add "--webapps", which will you copy? Manager? Host Manager? All of them? All of them. I think it has to be all or nothing. Anything else will get very complicated very quickly for a script. I think most cases won't want to copy (hence the default) but I can think of a few scenarios where it could be useful. So even if it contains some user deployed apps? Yes. I think that it completely contradicts the spirit of make base: Give me a fully working, but minimal Tomcat instance. At the end your are trying to write a rsync for that. I think that this will add little benefit. It will only workout if people never work with CATALINA_HOME, but only with _BASE like I do at work. I agree with Mark. Copying user deployed apps will be very useful if you want to scale out your application and deploy multiple servers for the app for high availability etc. $ rsync -auhh src target done. I don't need a script for that. Someone who scales will *never* operate from CATALINA_HOME because this makes upgrades pain in the ass. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
[INFRA] Tomcat cannot be built due to back mirrors
Folks, we need to inquire this with INFRA: PS D:\Entwicklung\Projekte\tomcat-8.5.x> ant Buildfile: D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml download-compile: testexist: [echo] Testing for C:\Users\mosipov/tomcat-build-libs/commons-daemon-1.1.0/commons-daemon-1.1.0.jar downloadgz-2: setproxy: trydownload.check: trydownload: [get] Getting: https://www.apache.org/dyn/closer.lua?action=download&filename=/commons/daemon/binaries/commons-daemon-1.1.0-bin.tar.gz [get] To: C:\Users\mosipov\tomcat-build-libs\download-1214738004.tar.gz [get] https://www.apache.org/dyn/closer.lua?action=download&filename=/commons/daemon/binaries/commons-daemon-1.1.0-bin.tar.gz moved to http://ftp.fau.de/apache//commons/daemon/binaries/commons-daemon-1.1.0-bin.tar.gz BUILD FAILED D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml:2618: The following error occurred while executing this line: D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml:2924: The following error occurred while executing this line: D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml:3040: Redirection detected from https to http. Protocol switch unsafe, not allowed. I am on PS D:\Entwicklung\Projekte\tomcat-8.5.x> ant -version Apache Ant(TM) version 1.10.5 compiled on July 10 2018 Do we need to have all HTTP mirrors dropped/checked for HTTPS? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [INFRA] Tomcat cannot be built due to back mirrors
Am 2018-07-26 um 00:55 schrieb Konstantin Kolinko: 2018-07-25 22:33 GMT+03:00 Michael Osipov : Folks, we need to inquire this with INFRA: PS D:\Entwicklung\Projekte\tomcat-8.5.x> ant Buildfile: D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml download-compile: testexist: [echo] Testing for C:\Users\mosipov/tomcat-build-libs/commons-daemon-1.1.0/commons-daemon-1.1.0.jar downloadgz-2: setproxy: trydownload.check: trydownload: [get] Getting: https://www.apache.org/dyn/closer.lua?action=download&filename=/commons/daemon/binaries/commons-daemon-1.1.0-bin.tar.gz [get] To: C:\Users\mosipov\tomcat-build-libs\download-1214738004.tar.gz [get] https://www.apache.org/dyn/closer.lua?action=download&filename=/commons/daemon/binaries/commons-daemon-1.1.0-bin.tar.gz moved to http://ftp.fau.de/apache//commons/daemon/binaries/commons-daemon-1.1.0-bin.tar.gz BUILD FAILED D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml:2618: The following error occurred while executing this line: D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml:2924: The following error occurred while executing this line: D:\Entwicklung\Projekte\tomcat-8.5.x\build.xml:3040: Redirection detected from https to http. Protocol switch unsafe, not allowed. See https://bz.apache.org/bugzilla/show_bug.cgi?id=62164 and https://lists.apache.org/thread.html/f8b04ca6e28e9a9e0fde895df266bec2fd95027791286ab6665dc49a@%3Cdev.tomcat.apache.org%3E "Tag Tomcat 7/8.0" thread of 2018-06-28 I think that we have to revert to using HTTP at least for ASF mirror in Tomcat configuration, as Ant bug has been fixed in 1.9.13/1.10.5 I have already noticed this one too, but even Ant 1.10.5 does not solve this issue. As sad as it sounds, can we go back to HTTP for closer.lua for now? I can't test/compile anything here in Germany unless I change the build.properties. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Timestamps in the manifests
Am 2018-08-06 um 20:24 schrieb Emmanuel Bourg: Hi all, We have 4 timestamps in the manifest of our jar files: DSTAMP: 20180806 TSTAMP: 1443 TODAY: August 6 2018 Bnd-LastModified: 1533578685288 Are they really necessary? This doesn't help making the generated files reproducible [1]. Any objection to remove the DSTAMP, TSTAMP and TODAY attributes? Please drop, they are an annoyance w/o any value. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat 8.5.34
Am 2018-09-05 um 00:52 schrieb Mark Thomas: The proposed Apache Tomcat 8.5.34 release is now available for voting. The major changes compared to the 8.5.33 release are: - Fix multiple issues associated with using the asynchronous Servlet API in combination with HTTP/2 - Add recursion to rewrite substitution parsing - Expand the information in the documentation web application regarding the use of CATALINA_HOME and CATALINA_BASE. Patch provided by Marek Czernek. Along with lots of other bug fixes and improvements. It can be obtained from: https://dist.apache.org/repos/dist/dev/tomcat/tomcat-8/v8.5.34/ The Maven staging repo is: https://repository.apache.org/content/repositories/orgapachetomcat-1194/ The svn tag is: http://svn.apache.org/repos/asf/tomcat/tc8.5.x/tags/TOMCAT_8_5_34/ The proposed 8.5.34 release is: [ ] Broken - do not release [ ] Stable - go ahead and release as 8.5.34 +1 Tests all pass with OpenJDK with OpenJ9 on Windows. Looking forward to deploy it on our servers. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE][RESULT] Release Apache Tomcat 8.5.34
Am 2018-09-10 um 22:18 schrieb Mark Thomas: The following voters were cast: Binding: +1: remm, markt, michaelo, csutherl, ebourg My vote is binding? I am solely a committer at moment, not a PMC member. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [ANN] Apache Tomcat 8.5.34 available
Am 2018-09-11 um 14:53 schrieb Mark Thomas: The Apache Tomcat team announces the immediate availability of Apache Tomcat 8.5.34. Apache Tomcat 8 is an open source software implementation of the Java Servlet, JavaServer Pages, Java Unified Expression Language, Java WebSocket and Java Authentication Service Provider Interface for Containers technologies. Apache Tomcat 8.5.x replaces 8.0.x and includes new features pulled forward from the 9.0.x branch. The notable changes since 8.5.33 include: - Fix multiple issues associated with using the asynchronous Servlet API in combination with HTTP/2 - Add recursion to rewrite substitution parsing - Expand the information in the documentation web application regarding the use of CATALINA_HOME and CATALINA_BASE. Patch provided by Marek Czernek. Please refer to the change log for the complete list of changes: http://tomcat.apache.org/tomcat-8.5-doc/changelog.html Downloads: http://tomcat.apache.org/download-80.cgi Migration guides from Apache Tomcat 7.x and 8.0.x: http://tomcat.apache.org/migration.html Perfect, thank you. I have put it already in production three hours ago. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
JDBC drivers in WEB-INF/lib
Hi folks, in bug 62670 [1] -- contrary to my statement -- Mark pointed out that is it pefectly legal to have a JDBC driver in WEB-INF/lib. While I was living for years with the assumption that this is discouraged because the DriverManager is VM wide and not classloader-wide, plus numerous SO questions have been asked about this. While reading [2] it is still not clear why this is a perfectly legal case. How can we improve the docs to make it crystal clear for users under which circumstances a driver is valid in WEB-INF/lib? [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=62670#c2 [2] http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto.html Regards, Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: JDBC drivers in WEB-INF/lib
Am 2018-09-11 um 23:53 schrieb Christopher Schultz: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Michael, On 9/11/18 3:33 PM, Michael Osipov wrote: in bug 62670 [1] -- contrary to my statement -- Mark pointed out that is it pefectly legal to have a JDBC driver in WEB-INF/lib. While I was living for years with the assumption that this is discouraged because the DriverManager is VM wide and not classloader-wide, plus numerous SO questions have been asked about this. Most answers are based upon speculation and age-old wisdom about decades-old versions of Tomcat. Most of those questions haven't been answered by anyone who actually knows the answers. While reading [2] it is still not clear why this is a perfectly legal case. Putting a JDBC library into WEB-INF/lib is a supported use-case in Tomcat, full stop. That's why it's a legal case: simply because we say it is. How can we improve the docs to make it crystal clear for users under which circumstances a driver is valid in WEB-INF/lib? I can't imagine a case where it should *not* be valid. The documentation[2] simply says that the application won't be able to rely on auto-registration of JDBC drivers (which has, in my experience, always been flaky anyway) and you'll have to go back to the practice of calling Class.forName(driverName) and possibly calling newInstance() on the Class object, just like it was still 1995. I haven't checked, but presumably both of the JDBC pools shipped with Tomcat perform this operation for you, so you don't have to have a ServletContextListener that does it, but you will want to take care to de-register your JDBC driver when your application shuts-down, or you will have a ClassLoader-pinning leak (DriverManager[app CL] -> JDBC Driver Class [webapp CL] -> webapp CL -> all classes loaded for the webapp). There are some drivers (notably MySQL's Connector/J) that are notoriously difficult to kill because they are written by people who don't understand why drivers would ever have to be removed from memory, but that's a different problem. At $work, we still use CATALINA_BASE/lib for out JDBC drivers, but that's simply due to historical deployment strategy. - -chris References left in-tact: [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=62670#c2 [2] http://tomcat.apache.org/tomcat-9.0-doc/jndi-datasource-examples-howto .html Thank you very much for your elaboration. I will keep this as a reference. It still makes sense to put the driver into CATALINA_HOME/lib if you multiple apps running with the same database type. Regards, Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Specifying a URL for the docBase
Am 2018-12-03 um 20:30 schrieb Mark Thomas: Hi, I have been looking at [1]. This is a request to be able to specify a URL for a WAR, including a URL that points to a WAR file packaged inside a JAR. This request is in the context of embedded Tomcat. The embedded aspects are just part of this. Fixes would also be required to the core Tomcat code. The Javadoc for Context indicates that a URL is acceptable for the docBase. However, the code does not support this. The code expects a file path (absolute or relative to appBase) that points to either a directory or a WAR. How can a URL with a scheme be relative?! I've been looking at the code and there are multiple places where the assumption is made that the docBase is a file path. They are fixable but... I am wondering whether it would be better to fix this or to change the Javadoc. In favour of fixing this: - the code would match the intention as described in the Javadoc - the requirements of [1] would be addressed - loading a WAR from a URL offers a lot more flexibility In favour of changing the Javadoc - there are multiple places that would need to be fixed - loading a WAR from a URL broadens the attack surface if a malicious user can get to the configuration files - there doesn't seem to be much demand for loading from a URL (one bug report with no activity and no votes in ~5 years) I'm somewhat on the fence on this one. Thoughts? I strive for simplicity, so I'd go with option 2. Moreover, we should clearly document which URL schemes are supported, e.g., file:// only. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Specifying a URL for the docBase
Am 2018-12-04 um 23:22 schrieb Woonsan Ko: On Wed, Dec 5, 2018 at 5:36 AM Michael Osipov wrote: Am 2018-12-03 um 20:30 schrieb Mark Thomas: Hi, I have been looking at [1]. This is a request to be able to specify a URL for a WAR, including a URL that points to a WAR file packaged inside a JAR. This request is in the context of embedded Tomcat. The embedded aspects are just part of this. Fixes would also be required to the core Tomcat code. The Javadoc for Context indicates that a URL is acceptable for the docBase. However, the code does not support this. The code expects a file path (absolute or relative to appBase) that points to either a directory or a WAR. How can a URL with a scheme be relative?! E.g, jar:file:/path/to/application/bits/myapp.jar!/META-INF/hipshoot/embedded-catalina/webapps/site.war, based on JAR URL [1]. That still makes the original JAR absolute, thus the WAR is absolute. You are simply navigating inside the ZIP file. That, of course, is always relative. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration - Timing
> All, > > I'd like to propose that we make the move from svn to git for Tomcat > 7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete. > > The proposed approach is documented here: > https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration > > I anticipate that the repositories will be read only for a couple of hours. > > I also anticipate that the CI systems - particularly Gump - will take up > to a day to switch over and iron out any problems. > > Any objections? > > If not, I'll start a formal VOTE. Can you leave me a few more days? I'd like to review the wiki page and add some propsals. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Re: Git migration - Timing
> On 11/02/2019 19:57, Michael Osipov wrote: > >> All, > >> > >> I'd like to propose that we make the move from svn to git for Tomcat > >> 7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete. > >> > >> The proposed approach is documented here: > >> https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration > >> > >> I anticipate that the repositories will be read only for a couple of hours. > >> > >> I also anticipate that the CI systems - particularly Gump - will take up > >> to a day to switch over and iron out any problems. > >> > >> Any objections? > >> > >> If not, I'll start a formal VOTE. > > > > Can you leave me a few more days? I'd like to review the wiki page and add > > some propsals. > > What sort of proposals? > > Each of the issues has been discussed and the approach agreed within the > community - on this list - over a period of many months. > > Unless there is a show-stopper issue that has been missed in the > discussions to date, I really do think it is time to move forward with > the actual migration. I completely missed that discussion, truly my fault. It maybe nothing special I'd object or maybe nothing at all. At Maven TLP we have migrated tens of repos, so we gained some experience here. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration - Timing
Am 2019-02-11 um 15:51 schrieb Mark Thomas:
All,
I'd like to propose that we make the move from svn to git for Tomcat
7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete.
The proposed approach is documented here:
https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration
I anticipate that the repositories will be read only for a couple of hours.
I also anticipate that the CI systems - particularly Gump - will take up
to a day to switch over and iron out any problems.
Any objections?
Hi Mark,
thanks for bearing with me. I was now finally able to look at the
proposal. As far as I understand there wil be a single repo for the
entire code base and everything will happen in branches.
Here are my comments:
1. You have set up branches: master, tomcat70, tomcat85, etc, but the
document talks about "Branch names. master, tc8.5, tc8.0, tc7.0 etc"
that seems to be inconsistent.
2. What will happens with Tomcat JDBC Pool? Will in remain in the
Subversion repo or will it move to tomcat-jdbc-pool.git? The code does
not change that offen that justifies a tandem release with Tomcat.
3. Why do you bother to import Tomcat 8.0? It is EOL, leave it in
Subversion. Do you plan to perform any futher merge here?
4. Can we *please* have a readible tag name scheme?! I simply don't
understand why people uppercase everything and replace dots with
underscores. That's just ugly. Consistent tag names would be
"{branch-name}-{version}" for the mono repo. E.g, tomcat85-8.5.40, or
tomcat70-7.0.95. This would heavily reduce sed(1) magic for package
maintainers and improve readability.
5. I assume that mod_jk, native and friends will remain in the
Subversion repo?
6. I don't see in the wiki page an agreement on a good/wellknown Git
commit message scheme: "BZ #123: \n\n"
Regards,
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration - Timing
Am 2019-02-16 um 14:46 schrieb Mark Thomas:
On 16/02/2019 13:39, Michael Osipov wrote:
Am 2019-02-11 um 15:51 schrieb Mark Thomas:
All,
I'd like to propose that we make the move from svn to git for Tomcat
7.0.x, 8.5.x and 9.0.x as soon as the next 7.0.x release is complete.
The proposed approach is documented here:
https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration
I anticipate that the repositories will be read only for a couple of
hours.
I also anticipate that the CI systems - particularly Gump - will take up
to a day to switch over and iron out any problems.
Any objections?
Hi Mark,
thanks for bearing with me. I was now finally able to look at the
proposal. As far as I understand there wil be a single repo for the
entire code base and everything will happen in branches.
Here are my comments:
1. You have set up branches: master, tomcat70, tomcat85, etc, but the
document talks about "Branch names. master, tc8.5, tc8.0, tc7.0 etc"
that seems to be inconsistent.
The demo was set up before the branch names were agreed.
2. What will happens with Tomcat JDBC Pool? Will in remain in the
Subversion repo or will it move to tomcat-jdbc-pool.git? The code does
not change that offen that justifies a tandem release with Tomcat.
It will remain part of the repo for each Tomcat version - as it is now.
3. Why do you bother to import Tomcat 8.0? It is EOL, leave it in
Subversion. Do you plan to perform any futher merge here?
The demo was set up before 8.0.x was made EOL. There are no plans to
move 8.0.c to git.
4. Can we *please* have a readible tag name scheme?! I simply don't
understand why people uppercase everything and replace dots with
underscores. That's just ugly. Consistent tag names would be
"{branch-name}-{version}" for the mono repo. E.g, tomcat85-8.5.40, or
tomcat70-7.0.95. This would heavily reduce sed(1) magic for package
maintainers and improve readability.
The community has not expressed a desire to change the naming scheme.
You are, of course, free to start such a discussion on dev@.
5. I assume that mod_jk, native and friends will remain in the
Subversion repo?
Yes.
6. I don't see in the wiki page an agreement on a good/wellknown Git
commit message scheme: "BZ #123: \n\n"
No such convention exists for subversion. The community has not
expressed a desire to introduce one with the move to git. Again, you are
free to start a discussion on dev@
Thanks for the quick response. I will start the approapriate discussions!
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Git migration: new branch/tag naming scheme
Folks, given that we are currently in the process of migrating to Git I'd like to propose a more readible and with the branch names consistent tag naming scheme. The given approach, for whatsoever reason, performs an uppercase and replaces dots with underscores. This reduces readability, but also requires people (esp. package maintainers) to perform sed(1) magic to convert back and forth. There are bascially two approaches I'd like to discuss: Approch 1: It will reuse the branch name of the current major version, excluding master. Thus, we will have the following prefixes: tomcat90-, tomcat85-, and tomcat70-. Since JDBC Pool remains in place and if it is released separately the prefix would be jdbc-pool-. The version itself would remain as-is and simply appended, e.g., 8.5.40, 9.0.25, etc. Finally it would be tomcat85-8.5.40, tomcat90-9.0.25. Another benefit would be autocompletion in Git CLI. I could simply say: "git checkout tomcat85" or "git checkout tomcat85-" and grab the specific version I want. Approach 2: A simpler, less redundant approach would be naming the branches master, 7.0.x, 8.5.x, etc. and get rid of the "tomcat" prefix at all. The tags would simply be the versions as-is: 8.5.40, 9.0.25, etc. The Git autocompletion will work here too. I personally opt for approach 2 because it is consistent, concise and removes redundancy on always used prefixes. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Git migration: consistent commit message
Folks, most of you know that there is a common convention of how to write decent Git commit messages. The scheme is: \n\n". Where as title (proposed) is: BZ #: If there is no corresponding BZ issue (which almost always should be), just an abtract. The abstract shouldn't contain more than 80 chars (sometimes not possible). This will highly improve commit message readablity in Git CLI, GitHub as well as Gitbox. Everything is trimmed on "\n\n" in the Git world. Personally, I sometimes have trouble to follow the commit log in Subversion to diff two versions between deployments. Therefore, I'd appreciate a better approach for the entire user base. Here are three good examples of a readable log: * https://github.com/apache/maven/commits/master * https://github.com/curl/curl/commits/master * https://github.com/apache/httpcomponents-client/commits/master Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration: new branch/tag naming scheme
Am 2019-02-18 um 11:03 schrieb Mark Thomas: On 18/02/2019 09:13, Rémy Maucherat wrote: On Sat, Feb 16, 2019 at 4:09 PM Michael Osipov wrote: Folks, given that we are currently in the process of migrating to Git I'd like to propose a more readible and with the branch names consistent tag naming scheme. The given approach, for whatsoever reason, performs an uppercase and replaces dots with underscores. This reduces readability, but also requires people (esp. package maintainers) to perform sed(1) magic to convert back and forth. There are bascially two approaches I'd like to discuss: Approch 1: It will reuse the branch name of the current major version, excluding master. Thus, we will have the following prefixes: tomcat90-, tomcat85-, and tomcat70-. Since JDBC Pool remains in place and if it is released separately the prefix would be jdbc-pool-. The version itself would remain as-is and simply appended, e.g., 8.5.40, 9.0.25, etc. Finally it would be tomcat85-8.5.40, tomcat90-9.0.25. Another benefit would be autocompletion in Git CLI. I could simply say: "git checkout tomcat85" or "git checkout tomcat85-" and grab the specific version I want. Approach 2: A simpler, less redundant approach would be naming the branches master, 7.0.x, 8.5.x, etc. and get rid of the "tomcat" prefix at all. The tags would simply be the versions as-is: 8.5.40, 9.0.25, etc. The Git autocompletion will work here too. I personally opt for approach 2 because it is consistent, concise and removes redundancy on always used prefixes. I guess it's hard to argue against option 2. The main downside is that it comes late and Mark already did the work and lots of testing for his proposed plan. The current, community agreed proposal for branch naming is "master, tc8.5 and tc7.0" There were strong views on the branch naming but "master, 8.5.x and 7.0.x" would be consistent with those views. I'm not sure I see much difference between either approach. If there is a strong preference for one over the other - or a good reason to choose one over the other - please make those views known in the next few days. The current proposal, community agreed proposal for tags is to continue with the current naming scheme. Switching to using the version as-is (9.0.1, 9.0.2, etc) is doable. It is just a little more work during the migration. If the as-is naming scheme makes it easier for downstream users then that strikes me as a good reason to change it. Are there any objections to doing so? I'd be happy if we could clean that up.. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration: new branch/tag naming scheme
Am 2019-02-18 um 15:19 schrieb Igal Sapir: On Mon, Feb 18, 2019 at 2:03 AM Mark Thomas wrote: On 18/02/2019 09:13, Rémy Maucherat wrote: On Sat, Feb 16, 2019 at 4:09 PM Michael Osipov wrote: Folks, given that we are currently in the process of migrating to Git I'd like to propose a more readible and with the branch names consistent tag naming scheme. The given approach, for whatsoever reason, performs an uppercase and replaces dots with underscores. This reduces readability, but also requires people (esp. package maintainers) to perform sed(1) magic to convert back and forth. There are bascially two approaches I'd like to discuss: Approch 1: It will reuse the branch name of the current major version, excluding master. Thus, we will have the following prefixes: tomcat90-, tomcat85-, and tomcat70-. Since JDBC Pool remains in place and if it is released separately the prefix would be jdbc-pool-. The version itself would remain as-is and simply appended, e.g., 8.5.40, 9.0.25, etc. Finally it would be tomcat85-8.5.40, tomcat90-9.0.25. Another benefit would be autocompletion in Git CLI. I could simply say: "git checkout tomcat85" or "git checkout tomcat85-" and grab the specific version I want. Approach 2: A simpler, less redundant approach would be naming the branches master, 7.0.x, 8.5.x, etc. and get rid of the "tomcat" prefix at all. The tags would simply be the versions as-is: 8.5.40, 9.0.25, etc. The Git autocompletion will work here too. I personally opt for approach 2 because it is consistent, concise and removes redundancy on always used prefixes. I guess it's hard to argue against option 2. The main downside is that it comes late and Mark already did the work and lots of testing for his proposed plan. The current, community agreed proposal for branch naming is "master, tc8.5 and tc7.0" That's fine by me. There were strong views on the branch naming but "master, 8.5.x and 7.0.x" would be consistent with those views. I'm not sure I see much difference between either approach. If there is a strong preference for one over the other - or a good reason to choose one over the other - please make those views known in the next few days. No good reason. Based on the OP I thought that we are comparing to a more verbose scheme like "tomcat-90-xxx" (without the standard "master" branch). I actually prefer "tc8.5" and "tc7.0" for the branches (over "8.5.x" and "7.0.x"). If tags will only use the numeric versions then this will make it easy to differentiate between branches and tags. tc8.5 could also misread as 8.5 release. 8.5.x implies that this is in development. This a common scheme in many many repos. Where is the benefit keeping the prefix? Git autocompletion will stop at "8.5." and you choose either 'x' or a patch release. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration: new branch/tag naming scheme
Am 2019-02-20 um 17:44 schrieb Mark Thomas: On 20/02/2019 16:14, Igal Sapir wrote: Michael, On Mon, Feb 18, 2019 at 11:53 AM Michael Osipov wrote: Am 2019-02-18 um 15:19 schrieb Igal Sapir: I actually prefer "tc8.5" and "tc7.0" for the branches (over "8.5.x" and "7.0.x"). If tags will only use the numeric versions then this will make it easy to differentiate between branches and tags. tc8.5 could also misread as 8.5 release. 8.5.x implies that this is in development. This a common scheme in many many repos. Where is the benefit keeping the prefix? Git autocompletion will stop at "8.5." and you choose either 'x' or a patch release. If I want to switch branches, which is more often than switching to a tag in my case, I would simply start with 't' without having to go through the tags. Having to go through the tag options when I actually want to switch to a branch is inefficient. Don't get me wrong, I don't care much which ones have a 'tc' prefix (branches) and which do not (tags), I just like that there is a difference between branches and tags. This got me thinking about how many key pressed would actually be required for branches Case A: Branches: 7.0.x, 8.5.x, master Tags: 7.0.11, 8.5.20, 9.0.12 etc. git checkout [7|8|m] master then auto-completes 7.0.x and 8.5.x get to "7.0." and "8.5." respectively and need a further "x" to complete. So that makes (ignoring the "git checkout ") 2 key presses for master and 3 for 7.0.x or 8.5.x Case B: Branches: tc7.0.x, tc8.5.x, master Tags: 7.0.11, 8.5.20, 9.0.12 etc. git checkout [t|m] master then auto-completes 7.0.x and 8.5.x both get to "tc" and need a further "7" or "8" and to complete. So that makes (ignoring the "git checkout ") 2 key presses for master and 4 for 7.0.x or 8.5.x Of course there are complications to this: - I suspect most developers will be using worktrees - keypresses are probably less important than how the GUIs of our preferred tools handles this In a GUI it will show how clearly what is a branch or a tag. There shouldn't be any ambiguity. I think this comes down to whether it is worth using a prefix (tc) to (more clearly) differentiate release branches and tags in those places where the two appear together. To put it another way: Is "9.0.5 vs tc9.0.x" clearer than "9.0.5 vs 9.0.x"? And if it is, do we want that additional clarity? Fo me, it looks inconsistent. "9.0.5 vs 9.0.x" clearly says that the former is a fixed version whereas the latter is a moving target in 9.0. I'd clearly opt for simplicity. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Better integration of RemoteIpValve, AuthenticatorBase and reverse proxies
Hi folks, I have some improvement ideas for several components where I think others would benefit from too. We intend to run a set of apps on Tomcat 8.5 behind Apache 2.4.x for a possible future load balacing scenario. While evaluating this task I have stumbled on the issue that it isn't that trivial to tell HTTPd that Tomcat is peforming the authentication and here is remoe user + auth type, use that for your access logs too. I have a partially working idea I'd like to get upstream (partially reported as BZ 62496): 1. RemoteIpValve sets requestAttributesEnabled by default in [1], I'd like to add here: > org.apache.coyote.Constants.FORWARDED_REQUEST_ATTRIBUTE: "org.apache.tomcat.forwardedRequest" > request.setAttribute(Constants.FORWARDED_REQUEST_ATTRIBUTE, "true"); 2. AuthenticatorBase would pick this up in [2] by checking this attribute and doing: > response.setHeader(remoteUserHeaderName, request.getRemoteUser()); > response.setHeader(authtTypeHeaderName, request.getAuthType()); where default header names are: X-Remote-User, X-Auth-Type. I am not yet certain whether it should require just FORWARDED_REQUEST_ATTRIBUTE, but also another attribute (e.g., boolean respondAuthInfoOnForwardedRequests) also. FORWARDED_REQUEST_ATTRIBUTE is nice because any internal component will know that this request is not an original request, but a forwarded one. On HTTPd I have: Header note X-Remote-User REMOTE_USER Header note X-Auth-Type AUTH_TYPE Header unset X-Remote-User Header unset Auth-Type LuaHookLog /usr/local/etc/apache24/register_remote_user.lua register_remote_user Access logs now look fine for me on both Tomcat and Apache HTTPd. WDYT? Michael [1] https://github.com/apache/tomcat85/blob/trunk/java/org/apache/catalina/valves/RemoteIpValve.java#L666-L676 [2] https://github.com/apache/tomcat85/blob/trunk/java/org/apache/catalina/authenticator/AuthenticatorBase.java#L999-L1001 - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Migrate to git
Am 2019-02-21 um 17:13 schrieb Mark Thomas: This is a VOTE to migrate the primary source code repository for Apache Tomcat 9.0.x, 8.5.x and 7.0.x from svn to git. The migration will be performed as per: https://cwiki.apache.org/confluence/display/TOMCAT/Git+migration with the following changes: - 8.0.x will not be migrated - the tag name format will be changed from "TOMCAT_9_0_5" to "9.0.5" - the branches will be named master, 8.5.x and 7.0.x The proposed date (subject to Infra agreement) for the migration is 26 Feb 2018. The migration process will be: - Make svn read only for trunk, 8.5.x and 7.0.x - Turn off the svn->git replication for trunk, 8.5.x and 7.0.x - Make git://git.apache.org/tomcat.git read/write for me only - Perform the migration as set out in the wiki with the modifications described above - Check the migration - Make git://git.apache.org/tomcat.git read/write for all committers (Note: This automatically makes https://github.com/apache/tomcat read/write as well) The critical work is done at this point. The following tasks are more clean-up and may end up being spread over several days. - Confirm there are no open PRs for https://github.com/apache/tomcat85 and then delete it and git://git.apache.org/tomcat85.git - Confirm there are no open PRs for https://github.com/apache/tomcat70 and then delete it and git://git.apache.org/tomcat70.git - Update the CI systems to pull the source from git - Create /source.html and replace /svn.html with a redirect to /source.html - Update migration guide to pull diffs from gitweb - Update Tomcat Native to pull in source from git hash - Fix anything else we have forgotten about. If anything goes wrong and we can't fix is easily, the fallback is to make svn read-write and go back to using svn while we clean up the git side of things, figure out what went wrong and come up with a better migration plan. [ ] +1 Go ahead with the migration [ ] -1 Postpone the migration because... +1 I hope we can accommodate the proposed changes, e.g., branches. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration read for testing
Am 2019-02-26 um 14:54 schrieb Rémy Maucherat: On Tue, Feb 26, 2019 at 1:33 PM Mark Thomas wrote: All, https://github.com/apache/tomcat is now ready for testing. It should contain: branches - master (9.0.x) - 8.5.x - 7.0.x Tags: - one for each 7.0.x, 8.5.x and 9.0.x release Tags have all been renamed to follow a a.b.c-MODIFIERn format for version number where modifier is RC or M. The repository is probably read/write for all committers now but please refrain from making any changes until we confirm that all is well. If you have some time available now, please test this new repository and report and questions or concerns to this thread. Assuming no issues are discovered, I'd like to formally move over to git later today. That looks quite good :) Should we plan to all work with PRs for large changes ? I'd love to have this, giving some time for the community to watch over the change...and assigning reviewers to it. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration read for testing
Am 2019-02-26 um 13:33 schrieb Mark Thomas: All, https://github.com/apache/tomcat is now ready for testing. It should contain: branches - master (9.0.x) - 8.5.x - 7.0.x Tags: - one for each 7.0.x, 8.5.x and 9.0.x release Tags have all been renamed to follow a a.b.c-MODIFIERn format for version number where modifier is RC or M. That looks so sweet and tidy. I cannot clone the repo at the moment, Gitbox seems to be down in EU. Thank you for your effort! Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Git migration read for testing
Am 2019-03-01 um 21:50 schrieb Mark Thomas: On 01/03/2019 19:54, Mark Thomas wrote: On 01/03/2019 19:00, Coty Sutherland wrote: The email notifications work for when we push commits to the repository, but it looks like we're missing emails when PRs are opened. ACK. I'll talk to infra. Fixed. FYI we have the option to route commits to one location and PRs + comments to another. At the moment, everything goes to dev@ Some projects have separate commits@ and notifications@ lists (and possibly issues@ as well). Personally, I like it all in one place and tend to filter everything into a single folder anyway. But I wanted to mention that the option was there. I'd opt for commits to commits@ and PRs to dev@. For those who'd like see commits will need to subscribe to it. Commits on dev@ produce a lot of traffic not necessarily related to dev discusssions. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] 01/01: Fix wrong protocol version usage
Am 2019-03-31 um 14:50 schrieb Konstantin Kolinko:
-1 (veto).
This was discussed several years ago, and the decision was to use "HTTP/2.0"
https://bz.apache.org/bugzilla/show_bug.cgi?id=58605
I tripped over this when comparing access logs between Tomcat and
Apache. Also, as you have noted, according to the RFC, it is HTTP/2, not
HTTP/2.0. Only the PRI request uses HTTP/2.0 for compat reasons.
I don't see a reason to keep an incorrect proto version. Just because
Mozilla makes it wrong, it doens't mean we have to.
M
вс, 31 мар. 2019 г. в 11:04, :
This is an automated email from the ASF dual-hosted git repository.
michaelo pushed a commit to branch wrong-http2-version
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0e1de0d34302cdea6b3c2a47b03dcca4c7e2f9b7
Author: Michael Osipov
AuthorDate: Sun Mar 31 10:03:29 2019 +0200
Fix wrong protocol version usage
When serving a HTTP/2 request the protocol version was set as "HTTP/2.0"
which does not exist.
---
java/org/apache/coyote/http2/Stream.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java
index 3e64329..437279a 100644
--- a/java/org/apache/coyote/http2/Stream.java
+++ b/java/org/apache/coyote/http2/Stream.java
@@ -126,7 +126,7 @@ class Stream extends AbstractStream implements
HeaderEmitter {
this.coyoteRequest.setSendfile(handler.hasAsyncIO() &&
handler.getProtocol().getUseSendfile());
this.coyoteResponse.setOutputBuffer(http2OutputBuffer);
this.coyoteRequest.setResponse(coyoteResponse);
-this.coyoteRequest.protocol().setString("HTTP/2.0");
+this.coyoteRequest.protocol().setString("HTTP/2");
if (this.coyoteRequest.getStartTime() < 0) {
this.coyoteRequest.setStartTime(System.currentTimeMillis());
}
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] 01/01: Fix wrong protocol version usage
Am 2019-04-01 um 09:36 schrieb Rémy Maucherat: On Sun, Mar 31, 2019 at 7:38 PM Michael Osipov wrote: Am 2019-03-31 um 14:50 schrieb Konstantin Kolinko: -1 (veto). This was discussed several years ago, and the decision was to use "HTTP/2.0" https://bz.apache.org/bugzilla/show_bug.cgi?id=58605 I tripped over this when comparing access logs between Tomcat and Apache. Also, as you have noted, according to the RFC, it is HTTP/2, not HTTP/2.0. Only the PRI request uses HTTP/2.0 for compat reasons. I don't see a reason to keep an incorrect proto version. Just because Mozilla makes it wrong, it doens't mean we have to. The commit was vetoed with a reason given, so it should be reverted for now, and then discussion may continue on the merits. The commit is not part of the master branch. I have intentionally created a PR. I don't like the CTR approach because it procudes mess in the first place on the commit log. I am happy if we can have that discussion on the mailing list or on the PR. For now, only Konstantin has objected, I'd like to hear other comitter's opinion on. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats
Am 2019-04-01 um 13:10 schrieb ma...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new 924d15a https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access
log formats
924d15a is described below
commit 924d15aa09c590f6d1b932a90df8297d333d2b2e
Author: Mark Thomas
AuthorDate: Mon Apr 1 12:08:42 2019 +0100
https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats
Document the differences in behaviour between the LogFormat directive in
httpd and the pattern attribute in the AccessLogValve for %D and %T.
---
webapps/docs/changelog.xml| 6 ++
webapps/docs/config/valve.xml | 9 +++--
2 files changed, 13 insertions(+), 2 deletions(-)
+%T - Time taken to process the request, in seconds. Note: This
+value has millisecond resolution whereas in httpd it has
+second resolution. Behaviour will be align to httpd
+in Tomcat 10 onwards.
One more nit, %T in Tomcat isn't just writing seconds, but seconds with
a fraction:
} else {
// second
buf.append(Long.toString(time / 1000));
buf.append('.');
int remains = (int) (time % 1000);
buf.append(Long.toString(remains / 100));
remains = remains % 100;
buf.append(Long.toString(remains / 10));
buf.append(Long.toString(remains % 10));
}
}
That's not really obvious, I guess.
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats
Am 2019-04-01 um 13:42 schrieb Mark Thomas: On 01/04/2019 12:41, Michael Osipov wrote: Am 2019-04-01 um 13:10 schrieb ma...@apache.org: This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/master by this push: new 924d15a https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats 924d15a is described below commit 924d15aa09c590f6d1b932a90df8297d333d2b2e Author: Mark Thomas AuthorDate: Mon Apr 1 12:08:42 2019 +0100 https://bz.apache.org/bugzilla/show_bug.cgi?id=63286 access log formats Document the differences in behaviour between the LogFormat directive in httpd and the pattern attribute in the AccessLogValve for %D and %T. --- webapps/docs/changelog.xml | 6 ++ webapps/docs/config/valve.xml | 9 +++-- 2 files changed, 13 insertions(+), 2 deletions(-) + %T - Time taken to process the request, in seconds. Note: This + value has millisecond resolution whereas in httpd it has + second resolution. Behaviour will be align to httpd + in Tomcat 10 onwards. One more nit, %T in Tomcat isn't just writing seconds, but seconds with a fraction: Which is covered in the text above. The value is in seconds with millisecond resolution. Ok, right, that's doesn't really popup unambiguously for the non-native English reader. Maybe precision and/or integer/double would really clarify. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] 01/01: Fix wrong protocol version usage
Am 2019-04-02 um 04:53 schrieb Christopher Schultz:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
Michael,
On 3/31/19 04:03, micha...@apache.org wrote:
This is an automated email from the ASF dual-hosted git
repository.
michaelo pushed a commit to branch wrong-http2-version in
repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 0e1de0d34302cdea6b3c2a47b03dcca4c7e2f9b7 Author: Michael
Osipov AuthorDate: Sun Mar 31 10:03:29 2019
+0200
Fix wrong protocol version usage
When serving a HTTP/2 request the protocol version was set as
"HTTP/2.0" which does not exist. ---
java/org/apache/coyote/http2/Stream.java | 2 +- 1 file changed, 1
insertion(+), 1 deletion(-)
diff --git a/java/org/apache/coyote/http2/Stream.java
b/java/org/apache/coyote/http2/Stream.java index 3e64329..437279a
100644 --- a/java/org/apache/coyote/http2/Stream.java +++
b/java/org/apache/coyote/http2/Stream.java @@ -126,7 +126,7 @@
class Stream extends AbstractStream implements HeaderEmitter {
this.coyoteRequest.setSendfile(handler.hasAsyncIO() &&
handler.getProtocol().getUseSendfile());
this.coyoteResponse.setOutputBuffer(http2OutputBuffer);
this.coyoteRequest.setResponse(coyoteResponse); -
this.coyoteRequest.protocol().setString("HTTP/2.0"); +
this.coyoteRequest.protocol().setString("HTTP/2"); if
(this.coyoteRequest.getStartTime() < 0) {
this.coyoteRequest.setStartTime(System.currentTimeMillis()); }
Does this all come down to what is shown in the log files and what is
returned by request.getProtocol() for servlets to consume? It isn't an
issue with an HTTP/2 response at all.
Correct,
Looks like it should be "HTTP/2.0", based upon these references:
RFC 7540 (HTTP/2)
Status line contains neither protocol nor status text: only the
numeric code.
https://tools.ietf.org/html/rfc7540#section-3.1
JavaEE javadoc for HttpServletRequest#getProtocol:
"
Returns the name and version of the protocol the request uses in the
form protocol/majorVersion.minorVersion, for example, HTTP/1.1. For
HTTP servlets, the value returned is the same as the value of the CGI
variable SERVER_PROTOCOL.
"
If we want to change what HttpServletRequest.getProtocol returns, I
think we need to get agreement from the Servlet EG about an exception
for HTTP/2 because anyone reading the documentation for that method
should expect a ".0" on the end of "HTTP/2".
This is of course an argumentation I cannot resist to agree, though the
documentatin of %r is wrong:
%r - First line of the request (method and request URI)
It misses the protocol. Also for the sake of consistency I'd like to see:
coyoteRequest.protocol().setString(Constants.HTTP_20);
I'll drop the PR.
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: Update ServerInfo to reflect actual information instead of placeholders when running development builds
Am 2019-04-11 um 21:49 schrieb csuth...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
csutherl pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new c8748aa Update ServerInfo to reflect actual information instead of
placeholders when running development builds
c8748aa is described below
commit c8748aaf9f3f7bc9f38c5805ed80e1a333696216
Author: Coty Sutherland
AuthorDate: Thu Apr 11 15:45:55 2019 -0400
Update ServerInfo to reflect actual information instead of placeholders
when running development builds
---
java/org/apache/catalina/util/ServerInfo.java | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/java/org/apache/catalina/util/ServerInfo.java
b/java/org/apache/catalina/util/ServerInfo.java
index 020d926..a70b5bf 100644
--- a/java/org/apache/catalina/util/ServerInfo.java
+++ b/java/org/apache/catalina/util/ServerInfo.java
@@ -68,11 +68,11 @@ public class ServerInfo {
} catch (Throwable t) {
ExceptionUtils.handleThrowable(t);
}
-if (info == null)
-info = "Apache Tomcat 9.0.x-dev";
-if (built == null)
+if (info == null || info.equals("Apache Tomcat/@VERSION@"))
+info = "Apache Tomcat/9.0.x-dev";
+if (built == null || built.equals("@VERSION_BUILT@"))
built = "unknown";
-if (number == null)
+if (number == null || number.equals("@VERSION_NUMBER@"))
number = "9.0.x";
I don't like such an approach. This isn't Java, this is the
C-define-from-external approach.
The better approach is to read a build.properties at runtime.
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PROPOSAL] Remove the functional specs from docs webapp
Am 2020-08-10 um 17:46 schrieb Mark Thomas: Hi all, I'd like to propose removing all the functional spec pages from the documentation web application. My reasoning for this proposal is, in short, that we aren't using or maintaining these pages. I don't recall any discussion of these docs on the dev list, proposals to change them, proposals for additions etc. There have been changes but going back over the changes from the last 10 years (and there are very few of them) they each appear to be part of a wider global change that is updating something or removing references to a feature that has been removed. Should someone want to revive these pages, or more likely a subset of them, they'll always be in the history. Thoughts? +1 Can you list them specifically? I am a bit lost which you exactly mean. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: Improve entity tag handling
Am 2020-08-11 um 16:52 schrieb ma...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
markt pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/master by this push:
new bef507e Improve entity tag handling
bef507e is described below
commit bef507e1b7ac2eb0ff012d0d40035e218a5839cc
Author: Mark Thomas
AuthorDate: Tue Aug 11 15:27:45 2020 +0100
Improve entity tag handling
---
conf/web.xml | 8 ++
.../apache/catalina/servlets/DefaultServlet.java | 116 +
.../apache/tomcat/util/http/parser/EntityTag.java | 82
.../TestDefaultServletIfMatchRequests.java | 143 +
webapps/docs/changelog.xml | 7 +
webapps/docs/default-servlet.xml | 8 +-
6 files changed, 284 insertions(+), 80 deletions(-)
diff --git a/conf/web.xml b/conf/web.xml
index a685947..b5e0b30 100644
--- a/conf/web.xml
+++ b/conf/web.xml
@@ -114,6 +114,14 @@
+
+
+
+
+
+
+
+
...
@@ -279,6 +280,8 @@ public class DefaultServlet extends HttpServlet {
*/
private boolean allowPartialPut = true;
+protected boolean useWeakComparisonWithIfMatch = true;
I really must object this. It clearly violates RFC 7232, section 3.1:
An origin server MUST use the strong comparison function when
comparing entity-tags for If-Match...
Even an option for this is wrong. I agree that we cannto produce strong
ETags by default, but it is now better decoupled and a subclass can
handle this. Please retain the semantics as described in RFC 7232.
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: Improve entity tag handling
Am 2020-08-11 um 18:53 schrieb Mark Thomas:
On 11/08/2020 17:29, Michael Osipov wrote:
Am 2020-08-11 um 16:52 schrieb ma...@apache.org:
commit bef507e1b7ac2eb0ff012d0d40035e218a5839cc
Author: Mark Thomas
AuthorDate: Tue Aug 11 15:27:45 2020 +0100
Improve entity tag handling
@@ -279,6 +280,8 @@ public class DefaultServlet extends HttpServlet {
*/
private boolean allowPartialPut = true;
+ protected boolean useWeakComparisonWithIfMatch = true;
I really must object this. It clearly violates RFC 7232, section 3.1:
Prior to this commit the code did not use a strong comparison for
If-Match contrary to the requirements of RFC 7232.
This commit does not change this behaviour (by default)
This commit adds an option so that RFC 7232 compliant behaviour can be
enabled by those who want it without breaking backwards compatibility
for any users that are reliant on the current, non-compliant behaviour.
An origin server MUST use the strong comparison function when
comparing entity-tags for If-Match...
Even an option for this is wrong. I agree that we cannto produce strong
ETags by default, but it is now better decoupled and a subclass can
handle this. Please retain the semantics as described in RFC 7232.
It isn't possible to retain the semantics of RFC 7232 because Tomcat
prior to this commit did not implement them.
If you look at the code prior to this commit, any "W/" was stripped from
the resource ETag and the ETag values in the If-Match header before
comparison. The result of doing that is that the comparison is
effectively a weak one rather than a strong one.
The option is required to preserve backwards compatibility.
Granted. This ultimately means that Tomcat 10 should remove this cruft
and be RFC-compliant. For previous versions a comment about behavioral
change/deprecation should be added too.
Can we agree on this?
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PROPOSAL] Remove the functional specs from docs webapp
Am 2020-08-11 um 21:04 schrieb Mark Thomas: On 11/08/2020 17:30, Michael Osipov wrote: Am 2020-08-10 um 17:46 schrieb Mark Thomas: Hi all, I'd like to propose removing all the functional spec pages from the documentation web application. +1 Can you list them specifically? I am a bit lost which you exactly mean. Everything under: https://tomcat.apache.org/tomcat-10.0-doc/funcspecs/ Thanks, I hav never seen these pages before. A lot of information would be gone, but I prefer rather no information than outdated -- even worse wrong information. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: Extracted CSS styles to external file for better code mainenance
Am 2020-08-16 um 20:05 schrieb isa...@apache.org: This is an automated email from the ASF dual-hosted git repository. isapir pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/master by this push: new 9c5d2e3 Extracted CSS styles to external file for better code mainenance 9c5d2e3 is described below commit 9c5d2e3b633fdb651bc9f11db4aac97ad3ad4df2 Author: Igal Sapir AuthorDate: Sun Aug 16 11:05:05 2020 -0700 Extracted CSS styles to external file for better code mainenance Also replaced gif logo with svg --- java/org/apache/catalina/manager/Constants.java| 79 +- There is a slight problem to that: TOMCAT_CSS is shared throughout the codebase. Do you want to break that up in CSS modules or duplicate code? M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: Extracted CSS styles to external file for better code mainenance
Am 2020-08-16 um 21:58 schrieb Igal Sapir: Michael, On Sun, Aug 16, 2020 at 11:37 AM Michael Osipov wrote: Am 2020-08-16 um 20:05 schrieb isa...@apache.org: This is an automated email from the ASF dual-hosted git repository. isapir pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/master by this push: new 9c5d2e3 Extracted CSS styles to external file for better code mainenance 9c5d2e3 is described below commit 9c5d2e3b633fdb651bc9f11db4aac97ad3ad4df2 Author: Igal Sapir AuthorDate: Sun Aug 16 11:05:05 2020 -0700 Extracted CSS styles to external file for better code mainenance Also replaced gif logo with svg --- java/org/apache/catalina/manager/Constants.java| 79 +- There is a slight problem to that: TOMCAT_CSS is shared throughout the codebase. Do you want to break that up in CSS modules or duplicate code? The new file "manager.css" includes the code from TOMCAT_CSS, which is really just a few lines of CSS [1]. We can certainly add another file with just that but IMO it would be an overkill and would result in an additional unnecessary http request. Accepted! Toda raba! - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Time for Tomcat Native 1.2.25
Am 2020-08-20 um 18:30 schrieb Mark Thomas: Hi, It has been a while since 1.2.24 and there are a few fixes in the changelog (mainly for LibreSSL and better support for a range of platforms). With this in mind, I'm currently intending to tag 1.2.25 in ~24 hours Please go ahead. I have started at some point in the parst to go through ifdefs and identify all LibreSSL versions which implement the OpenSSL counterparts. I hope I can pick that up next months. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat Native 1.2.25
Am 2020-08-21 um 20:22 schrieb Mark Thomas: Version 1.2.25 includes the following changes compared to 1.2.24 - Improvements to LibreSSL support - Improvements to HP_UX support Various other fixes and improvements. See the changelog for details. The proposed release artefacts can be found at [1], and the build was done using tag [2]. A bit late, but here are my findings. I am not sure whether they are in Tomcat or libtcnative. I see reliable failures in tests with LibreSSL 2.9.0 and 3.2.1 (master) with libtcnative master and Tomcat master on FreeBSD 12-STABLE and OpenJDK 1.8.0_265. Please see: http://home.apache.org/~michaelo/tomcat-master-tcnative-master-libressl-2.9.0/ as well as http://home.apache.org/~michaelo/tomcat-master-tcnative-master-libressl-master/. Let me know if you need further information for analysis/fixing. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat Native 1.2.25
Am 2020-08-24 um 15:42 schrieb Mark Thomas: On 24/08/2020 12:58, Michael Osipov wrote: Am 2020-08-21 um 20:22 schrieb Mark Thomas: Version 1.2.25 includes the following changes compared to 1.2.24 - Improvements to LibreSSL support - Improvements to HP_UX support Various other fixes and improvements. See the changelog for details. The proposed release artefacts can be found at [1], and the build was done using tag [2]. A bit late, but here are my findings. I am not sure whether they are in Tomcat or libtcnative. I see reliable failures in tests with LibreSSL 2.9.0 and 3.2.1 (master) with libtcnative master and Tomcat master on FreeBSD 12-STABLE and OpenJDK 1.8.0_265. Is that any different with 1.2.24? 1.2.24 does not compile at all with LibreSSL unless you perform these steps: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246373#c0 I did that, compiled against 2.9.2 and 3.2.1 (master): Same behavior! This isn't a blocker (critical) for you now? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat Native 1.2.25
Am 2020-08-24 um 15:42 schrieb Mark Thomas: On 24/08/2020 12:58, Michael Osipov wrote: Am 2020-08-21 um 20:22 schrieb Mark Thomas: Version 1.2.25 includes the following changes compared to 1.2.24 - Improvements to LibreSSL support - Improvements to HP_UX support Various other fixes and improvements. See the changelog for details. The proposed release artefacts can be found at [1], and the build was done using tag [2]. A bit late, but here are my findings. I am not sure whether they are in Tomcat or libtcnative. I see reliable failures in tests with LibreSSL 2.9.0 and 3.2.1 (master) with libtcnative master and Tomcat master on FreeBSD 12-STABLE and OpenJDK 1.8.0_265. Is that any different with 1.2.24? FWIW, I have tried now 2.9.x, 3.0.x, 3.1.x and 3.2.x (master). All of them crash (JVM crash) TEST-org.apache.tomcat.util.net.TestSSLHostConfigCompat.*.txt. It must be a general problem somewhere. frame #10: 0x000a6e3df807 libtcnative-1.so.0.2.24`Java_org_apache_tomcat_jni_SSLContext_setCertificateRaw(e=0x0008054421e0, o=, ctx=44874834080, javaCert=, javaKey=, idx=0) at sslcontext.c:1192:9 frame #11: 0x000805618587 as well as Stack: [0x7fffdfefe000,0x7fffdfffe000], sp=0x7fffdfffb7c0, free space=1013k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [libssl.so.48+0x42b59] SSL_CTX_use_certificate+0x9 C [libtcnative-1.so.0.2.24+0x24807] Java_org_apache_tomcat_jni_SSLContext_setCertificateRaw+0x1d7 j org.apache.tomcat.jni.SSLContext.setCertificateRaw(J[B[BI)Z+0 j org.apache.tomcat.util.net.openssl.OpenSSLContext.addCertificate(Lorg/apache/tomcat/util/net/SSLHostConfigCertificate;)V+214 j org.apache.tomcat.util.net.AprEndpoint.createSSLContext(Lorg/apache/tomcat/util/net/SSLHostConfig;)V+146 j org.apache.tomcat.util.net.AprEndpoint.bind()V+461 j org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup()V+1 M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE][RESULT] Release Apache Tomcat Native 1.2.25
Am 2020-09-03 um 16:34 schrieb Mark Thomas: The following votes were cast: Binding: +1: markt, mgrigorov, fschumacher +0: schultz The vote therefore passes. I think it is worth noting that there were crashes / unit test failures reported with LibreSSL. This isn't unexpected. There is more work to do for LibreSSL support. Back then, when I discovered the first flaws with LibreSSL, I started going thorugh all ifdefs which disable features with LibreSSL and searching in the Git repo for the proper LibreSSL version. I will publish my findings in an ubrella ticket soon. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] 01/04: Add connection pool to JNDI realm
Am 2020-10-07 um 22:34 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 50de36b7874da98591345e40b374a1e2dd52c188
Author: remm
AuthorDate: Thu Jan 30 17:22:51 2020 +0100
Add connection pool to JNDI realm
This implements a TODO from the class javadoc header.
As described in the javadoc, the idea is to use a pool to avoid blocking
on a single connection, which could possibly become a bottleneck in some
cases. The message formats need to be kept along with the connection
since they are not thread safe.
Preserve the default behavior: sync without pooling (using a Lock object
which is more flexible).
I may backport this since this is limited to the JNDI realm, but only
once it is confirmed to be regression free. Tested with ApacheDS but my
LDAP skills are very limited.
---
java/org/apache/catalina/realm/JNDIRealm.java | 442 -
.../apache/catalina/realm/LocalStrings.properties | 1 +
test/org/apache/catalina/realm/TestJNDIRealm.java | 7 +-
webapps/docs/changelog.xml | 3 +
webapps/docs/config/realm.xml | 7 +
5 files changed, 276 insertions(+), 184 deletions(-)
Salut Rémy,
this is a very very nice improvement to the matter and I gave your idea
a spin my public Active Directory realm. Based on my tests with AD I see
room for improvement: (Note that I don't use the JNDIRealm because
because is it too generic and usable for me)
* You might want to consider to introduce a maxIdleTime to avoid stale
connections, e.g., AD has default value of 15 minutes. Yep, I had had
several RSTs resulting in 401s
* Validating connections optionally might be a good option to detect
stale connections or broken ones ny networks issues. This works for me
very fast:
protected boolean validate(DirContextConnection connection) {
if (logger.isDebugEnabled())
logger.debug(sm.getString("activeDirectoryRealm.validate"));
SearchControls controls = new SearchControls();
controls.setSearchScope(SearchControls.OBJECT_SCOPE);
controls.setCountLimit(1);
controls.setReturningAttributes(new String[] { "objectClass" });
controls.setTimeLimit(500);
try {
NamingEnumeration results =
connection.context.search("", "objectclass=*",
controls);
if (results.hasMore()) {
close(results);
return true;
}
} catch (NamingException e) {
logger.error(sm.getString("activeDirectoryRealm.validate.namingException"), e);
return false;
}
return false;
}
I do this in the acquire() method
* The acquire(), authenticate(), fail, close(), authenticate(),
release() is brittle and proved to fail here. I have a curl script which
does 4 requests in parallel and 1200 requests in total. Pool size is 8.
4 connections in the pool. Rest for 16 minutes, connections are broken
now. Start requests. authenticate() will fail twice because top two
acquired connections from pool are broken. Requests end with 401. I'd
prefer to to modify acquire() is such a fashion:
protected DirContextConnection acquire() {
if (logger.isDebugEnabled())
logger.debug(sm.getString("activeDirectoryRealm.acquire"));
DirContextConnection connection = null;
while (connection == null) {
connection = connectionPool.pop();
if (connection != null) {
long idleTime = System.currentTimeMillis() -
connection.lastBorrowTime;
if (idleTime > maxIdleTime) {
if (logger.isDebugEnabled())
logger.debug(sm.getString("activeDirectoryRealm.exceedMaxIdleTime"));
close(connection);
connection = null;
} else {
boolean valid = validate(connection);
if (valid) {
if (logger.isDebugEnabled())
logger.debug(sm.getString("activeDirectoryRealm.reuse"));
} else {
close(connection);
connection = null
[DISCUSS] Deprecate and remove RealmBase#stripRealmForGss
Folks, I'd like to propose to get rid of that config option in 10 and deprecate in previous versions for the following reasons: * It suffers from abstraction: It assumes that the GSS name is always email style w/o checking its OID * The realm part, if any, is an integeral part of the principal. Much like with an email address' domain. You wouldn't stip here too, would you? * It is a surprise for clients having the princippal mutilated by default. I trip over and over again this when I set up UserDatabaseRealms for testing purposes I wonder why michae...@example.com does not work. * In a multi realm environment, it is perfectly fine and valid to have user1@REALMA and user1@REALMB. These are distinct principals, but treated by RealmBase equally, this has implications. * Finally, when doing cert-based auth in an AD envinronment, is it pretty common to extract the msUPN name from the cert's SAN which is almost always email address (enteprise principal) which would end up in michael.osipov, but where is the rest?! Thoughts? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] 01/04: Add connection pool to JNDI realm
Am 2020-10-13 um 13:49 schrieb Rémy Maucherat: On Tue, Oct 13, 2020 at 11:33 AM Michael Osipov wrote: Am 2020-10-07 um 22:34 schrieb r...@apache.org: This is an automated email from the ASF dual-hosted git repository. remm pushed a commit to branch 8.5.x in repository https://gitbox.apache.org/repos/asf/tomcat.git commit 50de36b7874da98591345e40b374a1e2dd52c188 Author: remm AuthorDate: Thu Jan 30 17:22:51 2020 +0100 Add connection pool to JNDI realm This implements a TODO from the class javadoc header. As described in the javadoc, the idea is to use a pool to avoid blocking on a single connection, which could possibly become a bottleneck in some cases. The message formats need to be kept along with the connection since they are not thread safe. Preserve the default behavior: sync without pooling (using a Lock object which is more flexible). I may backport this since this is limited to the JNDI realm, but only once it is confirmed to be regression free. Tested with ApacheDS but my LDAP skills are very limited. --- java/org/apache/catalina/realm/JNDIRealm.java | 442 - .../apache/catalina/realm/LocalStrings.properties | 1 + test/org/apache/catalina/realm/TestJNDIRealm.java | 7 +- webapps/docs/changelog.xml | 3 + webapps/docs/config/realm.xml | 7 + 5 files changed, 276 insertions(+), 184 deletions(-) Salut Rémy, this is a very very nice improvement to the matter and I gave your idea a spin my public Active Directory realm. Based on my tests with AD I see room for improvement: (Note that I don't use the JNDIRealm because because is it too generic and usable for me) Ok, I only tested this with a local ldap, so not much. I verified it was pooling connections. * You might want to consider to introduce a maxIdleTime to avoid stale connections, e.g., AD has default value of 15 minutes. Yep, I had had several RSTs resulting in 401s Oops. So I don't think adding something like this helps since there could be plenty of issues besides a timeout. Basically it's supposed to retry instead when something fails. Can you give me a stack trace to show what's wrong ? I consider it cheaper to test for a timeout and close actively rather than recover from a resetted connection. Here is a strack trace for concurrency of 4 (with a pool of max 8). Compare the timestamp of the first line and the subsequent ones: 2020-10-12T22:55:47 FEIN [https-openssl-apr-8444-exec-8] net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.release Releasing directory server connection to pool 2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5] net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory server connection from pool 2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5] net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Validating directory server connection from pool 2020-10-12T23:13:06 SCHWERWIEGEND [https-openssl-apr-8444-exec-5] net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Exception validating directory server connection javax.naming.CommunicationException: Connection reset [Root exception is java.net.SocketException: Connection reset]; remaining name '' at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2030) at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1872) at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1797) at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358) at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341) at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:296) at net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate(ActiveDirectoryRealm.java:316) at net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire(ActiveDirectoryRealm.java:285) at net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getPrincipal(ActiveDirectoryRealm.java:244) at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501) at net.sf.michaelo.tomcat.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:165) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:571) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690) at org.apache.catalina.c
Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling
Am 2020-10-13 um 16:05 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/8.5.x by this push:
new 883600b Always retry on a new connection, even when pooling
883600b is described below
commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm
AuthorDate: Tue Oct 13 14:19:54 2020 +0200
Always retry on a new connection, even when pooling
This keeps the same very simple design as for the single connection
scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22 +++---
webapps/docs/changelog.xml| 5 +
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
b/java/org/apache/catalina/realm/JNDIRealm.java
index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);
// open a new directory context.
-connection = get();
+connection = get(true);
// Try the authentication again.
principal = authenticate(connection, username, credentials);
@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
* @exception NamingException if a directory server error occurs
*/
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new connection,
+ * for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error occurs
+ */
+protected JNDIConnection get(boolean create) throws NamingException {
JNDIConnection connection = null;
// Use the pool if available, otherwise use the single connection
if (connectionPool != null) {
-connection = connectionPool.pop();
-if (connection == null) {
+if (create) {
connection = create();
+} else {
+connection = connectionPool.pop();
+if (connection == null) {
+connection = create();
+}
}
} else {
singleConnectionLock.lock();
That suitable and simple approach.
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [DISCUSS] Deprecate and remove RealmBase#stripRealmForGss
Am 2020-10-13 um 12:32 schrieb Mark Thomas: On 13/10/2020 10:48, Michael Osipov wrote: Folks, I'd like to propose to get rid of that config option in 10 and deprecate in previous versions for the following reasons: * It suffers from abstraction: It assumes that the GSS name is always email style w/o checking its OID * The realm part, if any, is an integeral part of the principal. Much like with an email address' domain. You wouldn't stip here too, would you? * It is a surprise for clients having the princippal mutilated by default. I trip over and over again this when I set up UserDatabaseRealms for testing purposes I wonder why michae...@example.com does not work. * In a multi realm environment, it is perfectly fine and valid to have user1@REALMA and user1@REALMB. These are distinct principals, but treated by RealmBase equally, this has implications. * Finally, when doing cert-based auth in an AD envinronment, is it pretty common to extract the msUPN name from the cert's SAN which is almost always email address (enteprise principal) which would end up in michael.osipov, but where is the rest?! Thoughts? No objections to changing the default to false in 10.0.x but I think removal/deprecation is a step too far. OK Not everyone uses the full u...@bigco.com format as the "primary key" in their internal user database. I see lots of variation depending on which system is viewed as the source of truth to which other systems are synchronised. So you will never be able to satisfy all, why try? We have X509UsernameRetriever for certificate authentication. That came out of bug 52500 where the aim was to support a wide variety of formats. The root of that requirement was essentially the same - integration with a variety of systems where the user name could be in in a range of formats. The mapping of that format to an X509 cert just added another layer of complexity. I don't think that this compares (well( because a certificate is a complex object explicitly containing SAN which contain wellknown forms for an entity whereas a GSS name is merely a string with an identifier for its format. Your argumentation could also be applied to an LDAP bind. Active Directory, for instance, accepts a various amounts of input formats for the same entity, yet we don't provide a logic to normalize them in the JNDI Realm. That seems to be unfair to me. It lacks a bit of consistency. If there was user demand for it (I haven't seen any) I'd support a similar interface for SPNEGO. With such an interface in place, deprecation and removal of stripRealmForGss would make sense. See above. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling
Am 2020-10-14 um 12:32 schrieb Rémy Maucherat:
On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov wrote:
Am 2020-10-13 um 16:05 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/8.5.x by this push:
new 883600b Always retry on a new connection, even when pooling
883600b is described below
commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm
AuthorDate: Tue Oct 13 14:19:54 2020 +0200
Always retry on a new connection, even when pooling
This keeps the same very simple design as for the single connection
scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22
+++---
webapps/docs/changelog.xml| 5 +
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
b/java/org/apache/catalina/realm/JNDIRealm.java
index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);
// open a new directory context.
-connection = get();
+connection = get(true);
// Try the authentication again.
principal = authenticate(connection, username,
credentials);
@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
* @exception NamingException if a directory server error occurs
*/
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new
connection,
+ * for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error occurs
+ */
+protected JNDIConnection get(boolean create) throws NamingException
{
JNDIConnection connection = null;
// Use the pool if available, otherwise use the single
connection
if (connectionPool != null) {
-connection = connectionPool.pop();
-if (connection == null) {
+if (create) {
connection = create();
+} else {
+connection = connectionPool.pop();
+if (connection == null) {
+connection = create();
+}
}
} else {
singleConnectionLock.lock();
That suitable and simple approach.
If you have the code for adding a max idle check on hand and tested, you
should add it IMO, it will be more efficient.
I will need to give this a couple more weeks of testing. This is what I
have observed today:
2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132
20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory
server connection from pool
20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory server
connection from pool exceeds max idle time, closing it
20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing directory
server connection
20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new directory
server connection
20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for
username 'w99sezx0' in base ...
As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:
2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70
when the connection is healthy.
Michael
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch master updated: Add extended ErrorReportValve that returns response as JSON instead of HTML
Am 2020-10-14 um 16:55 schrieb kfuj...@apache.org: This is an automated email from the ASF dual-hosted git repository. kfujino pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/master by this push: new 42a0841 Add extended ErrorReportValve that returns response as JSON instead of HTML 42a0841 is described below commit 42a0841d289a35117da842201622454a9f387cd7 Author: KeiichiFujino AuthorDate: Wed Oct 14 23:54:22 2020 +0900 Add extended ErrorReportValve that returns response as JSON instead of HTML Why not properly negotiate the content type? It does not seems that you aren't returning the same content the HTML version does. Moreover, why don't you use the print writer from the response? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling
> On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov wrote:
>
> > Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:
> > > On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov
> > wrote:
> > >
> > >> Am 2020-10-13 um 16:05 schrieb r...@apache.org:
> > >>> This is an automated email from the ASF dual-hosted git repository.
> > >>>
> > >>> remm pushed a commit to branch 8.5.x
> > >>> in repository https://gitbox.apache.org/repos/asf/tomcat.git
> > >>>
> > >>>
> > >>> The following commit(s) were added to refs/heads/8.5.x by this push:
> > >>>new 883600b Always retry on a new connection, even when pooli=
> ng
> > >>> 883600b is described below
> > >>>
> > >>> commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
> > >>> Author: remm
> > >>> AuthorDate: Tue Oct 13 14:19:54 2020 +0200
> > >>>
> > >>> Always retry on a new connection, even when pooling
> > >>>
> > >>> This keeps the same very simple design as for the single
> > connection
> > >>> scenario, for now.
> > >>> ---
> > >>>java/org/apache/catalina/realm/JNDIRealm.java | 22
> > >> +++---
> > >>>webapps/docs/changelog.xml| 5 +
> > >>>2 files changed, 24 insertions(+), 3 deletions(-)
> > >>>
> > >>> diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
> > >> b/java/org/apache/catalina/realm/JNDIRealm.java
> > >>> index 72087ab..98007f7 100644
> > >>> --- a/java/org/apache/catalina/realm/JNDIRealm.java
> > >>> +++ b/java/org/apache/catalina/realm/JNDIRealm.java
> > >>> @@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
> > >>>close(connection);
> > >>>
> > >>>// open a new directory context.
> > >>> -connection =3D get();
> > >>> +connection =3D get(true);
> > >>>
> > >>>// Try the authentication again.
> > >>>principal =3D authenticate(connection, username,
> > >> credentials);
> > >>> @@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
> > >>> * @exception NamingException if a directory server error occu=
> rs
> > >>> */
> > >>>protected JNDIConnection get() throws NamingException {
> > >>> +return get(false);
> > >>> +}
> > >>> +
> > >>> +/**
> > >>> + * Open (if necessary) and return a connection to the configured
> > >>> + * directory server for this Realm.
> > >>> + * @param create when pooling, this forces creation of a new
> > >> connection,
> > >>> + * for example after an error
> > >>> + * @return the connection
> > >>> + * @exception NamingException if a directory server error occurs
> > >>> + */
> > >>> +protected JNDIConnection get(boolean create) throws
> > NamingException
> > >> {
> > >>>JNDIConnection connection =3D null;
> > >>>// Use the pool if available, otherwise use the single
> > >> connection
> > >>>if (connectionPool !=3D null) {
> > >>> -connection =3D connectionPool.pop();
> > >>> -if (connection =3D=3D null) {
> > >>> +if (create) {
> > >>>connection =3D create();
> > >>> +} else {
> > >>> +connection =3D connectionPool.pop();
> > >>> +if (connection =3D=3D null) {
> > >>> +connection =3D create();
> > >>> +}
> > >>>}
> > >>>} else {
> > >>>singleConnectionLock.lock();
> > >>
> > >> That suitable and simple approach.
> > >>
> > >
> > > If you have the code for adding a max idle check on hand and tested, yo=
> u
> > > should add it IMO, it will be more efficient.
> >
> > I will need to give this a couple more weeks of testing. This is
Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling
Am 2020-10-15 um 16:48 schrieb Rémy Maucherat:
On Thu, Oct 15, 2020 at 2:31 PM Michael Osipov <1983-01...@gmx.net> wrote:
On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov
wrote:
Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:
On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov
wrote:
Am 2020-10-13 um 16:05 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/8.5.x by this
push:
new 883600b Always retry on a new connection, even when
pooli=
ng
883600b is described below
commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm
AuthorDate: Tue Oct 13 14:19:54 2020 +0200
Always retry on a new connection, even when pooling
This keeps the same very simple design as for the single
connection
scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22
+++---
webapps/docs/changelog.xml| 5 +
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
b/java/org/apache/catalina/realm/JNDIRealm.java
index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);
// open a new directory context.
-connection =3D get();
+connection =3D get(true);
// Try the authentication again.
principal =3D authenticate(connection, username,
credentials);
@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
* @exception NamingException if a directory server error
occu=
rs
*/
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the
configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new
connection,
+ * for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error
occurs
+ */
+protected JNDIConnection get(boolean create) throws
NamingException
{
JNDIConnection connection =3D null;
// Use the pool if available, otherwise use the single
connection
if (connectionPool !=3D null) {
-connection =3D connectionPool.pop();
-if (connection =3D=3D null) {
+if (create) {
connection =3D create();
+} else {
+connection =3D connectionPool.pop();
+if (connection =3D=3D null) {
+connection =3D create();
+}
}
} else {
singleConnectionLock.lock();
That suitable and simple approach.
If you have the code for adding a max idle check on hand and tested,
yo=
u
should add it IMO, it will be more efficient.
I will need to give this a couple more weeks of testing. This is what I
have observed today:
2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132
20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring
directory server connection from pool
20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory
serve=
r
connection from pool exceeds max idle time, closing it
20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing
directory
server connection
20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new
directory server connection
20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for
username 'w99sezx0' in base ...
As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:
2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70
when the connection is healthy.
Ok, so there's some real incentive to avoid reusing a connection that was
idle for too long.
I made further analysis. I was partially wrong about my statement. The
cause
for the 90 s was a faulty KDC which did not respond in time for a service
ticket. Java Kerberos does 3 retries with 30 s timeout. I have set to 1
retry
an
Re: [DISCUSS] Deprecate and remove RealmBase#stripRealmForGss
Am 2020-10-13 um 21:08 schrieb Michael Osipov: Am 2020-10-13 um 12:32 schrieb Mark Thomas: On 13/10/2020 10:48, Michael Osipov wrote: Folks, I'd like to propose to get rid of that config option in 10 and deprecate in previous versions for the following reasons: * It suffers from abstraction: It assumes that the GSS name is always email style w/o checking its OID * The realm part, if any, is an integeral part of the principal. Much like with an email address' domain. You wouldn't stip here too, would you? * It is a surprise for clients having the princippal mutilated by default. I trip over and over again this when I set up UserDatabaseRealms for testing purposes I wonder why michae...@example.com does not work. * In a multi realm environment, it is perfectly fine and valid to have user1@REALMA and user1@REALMB. These are distinct principals, but treated by RealmBase equally, this has implications. * Finally, when doing cert-based auth in an AD envinronment, is it pretty common to extract the msUPN name from the cert's SAN which is almost always email address (enteprise principal) which would end up in michael.osipov, but where is the rest?! Thoughts? No objections to changing the default to false in 10.0.x but I think removal/deprecation is a step too far. OK Not everyone uses the full u...@bigco.com format as the "primary key" in their internal user database. I see lots of variation depending on which system is viewed as the source of truth to which other systems are synchronised. So you will never be able to satisfy all, why try? We have X509UsernameRetriever for certificate authentication. That came out of bug 52500 where the aim was to support a wide variety of formats. The root of that requirement was essentially the same - integration with a variety of systems where the user name could be in in a range of formats. The mapping of that format to an X509 cert just added another layer of complexity. I don't think that this compares (well( because a certificate is a complex object explicitly containing SAN which contain wellknown forms for an entity whereas a GSS name is merely a string with an identifier for its format. Your argumentation could also be applied to an LDAP bind. Active Directory, for instance, accepts a various amounts of input formats for the same entity, yet we don't provide a logic to normalize them in the JNDI Realm. That seems to be unfair to me. It lacks a bit of consistency. If there was user demand for it (I haven't seen any) I'd support a similar interface for SPNEGO. With such an interface in place, deprecation and removal of stripRealmForGss would make sense. See above. What we basically miss is a Java counterpart of GSS-API's gss_localname(). - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling
Am 2020-10-15 um 16:48 schrieb Rémy Maucherat:
On Thu, Oct 15, 2020 at 2:31 PM Michael Osipov <1983-01...@gmx.net> wrote:
On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov
wrote:
Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:
On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov
wrote:
Am 2020-10-13 um 16:05 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/8.5.x by this
push:
new 883600b Always retry on a new connection, even when
pooli=
ng
883600b is described below
commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm
AuthorDate: Tue Oct 13 14:19:54 2020 +0200
Always retry on a new connection, even when pooling
This keeps the same very simple design as for the single
connection
scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22
+++---
webapps/docs/changelog.xml| 5 +
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
b/java/org/apache/catalina/realm/JNDIRealm.java
index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);
// open a new directory context.
-connection =3D get();
+connection =3D get(true);
// Try the authentication again.
principal =3D authenticate(connection, username,
credentials);
@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
* @exception NamingException if a directory server error
occu=
rs
*/
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the
configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new
connection,
+ * for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error
occurs
+ */
+protected JNDIConnection get(boolean create) throws
NamingException
{
JNDIConnection connection =3D null;
// Use the pool if available, otherwise use the single
connection
if (connectionPool !=3D null) {
-connection =3D connectionPool.pop();
-if (connection =3D=3D null) {
+if (create) {
connection =3D create();
+} else {
+connection =3D connectionPool.pop();
+if (connection =3D=3D null) {
+connection =3D create();
+}
}
} else {
singleConnectionLock.lock();
That suitable and simple approach.
If you have the code for adding a max idle check on hand and tested,
yo=
u
should add it IMO, it will be more efficient.
I will need to give this a couple more weeks of testing. This is what I
have observed today:
2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132
20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring
directory server connection from pool
20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory
serve=
r
connection from pool exceeds max idle time, closing it
20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing
directory
server connection
20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new
directory server connection
20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for
username 'w99sezx0' in base ...
As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:
2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70
when the connection is healthy.
Ok, so there's some real incentive to avoid reusing a connection that was
idle for too long.
I made further analysis. I was partially wrong about my statement. The
cause
for the 90 s was a faulty KDC which did not respond in time for a service
ticket. Java Kerberos does 3 retries with 30 s timeout. I have set to 1
retry
an
Re: [tomcat] branch 8.5.x updated: Always retry on a new connection, even when pooling
Am 2020-10-19 um 16:34 schrieb Rémy Maucherat:
On Mon, Oct 19, 2020 at 3:11 PM Michael Osipov wrote:
Am 2020-10-15 um 16:48 schrieb Rémy Maucherat:
On Thu, Oct 15, 2020 at 2:31 PM Michael Osipov <1983-01...@gmx.net>
wrote:
On Wed, Oct 14, 2020 at 6:32 PM Michael Osipov
wrote:
Am 2020-10-14 um 12:32 schrieb R=C3=A9my Maucherat:
On Tue, Oct 13, 2020 at 8:27 PM Michael Osipov
wrote:
Am 2020-10-13 um 16:05 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/8.5.x by this
push:
new 883600b Always retry on a new connection, even when
pooli=
ng
883600b is described below
commit 883600b8a77c0be93b3a8dc2404e8d1110970ee7
Author: remm
AuthorDate: Tue Oct 13 14:19:54 2020 +0200
Always retry on a new connection, even when pooling
This keeps the same very simple design as for the single
connection
scenario, for now.
---
java/org/apache/catalina/realm/JNDIRealm.java | 22
+++---
webapps/docs/changelog.xml| 5 +
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/java/org/apache/catalina/realm/JNDIRealm.java
b/java/org/apache/catalina/realm/JNDIRealm.java
index 72087ab..98007f7 100644
--- a/java/org/apache/catalina/realm/JNDIRealm.java
+++ b/java/org/apache/catalina/realm/JNDIRealm.java
@@ -1311,7 +1311,7 @@ public class JNDIRealm extends RealmBase {
close(connection);
// open a new directory context.
-connection =3D get();
+connection =3D get(true);
// Try the authentication again.
principal =3D authenticate(connection,
username,
credentials);
@@ -2389,12 +2389,28 @@ public class JNDIRealm extends RealmBase {
* @exception NamingException if a directory server error
occu=
rs
*/
protected JNDIConnection get() throws NamingException {
+return get(false);
+}
+
+/**
+ * Open (if necessary) and return a connection to the
configured
+ * directory server for this Realm.
+ * @param create when pooling, this forces creation of a new
connection,
+ * for example after an error
+ * @return the connection
+ * @exception NamingException if a directory server error
occurs
+ */
+protected JNDIConnection get(boolean create) throws
NamingException
{
JNDIConnection connection =3D null;
// Use the pool if available, otherwise use the single
connection
if (connectionPool !=3D null) {
-connection =3D connectionPool.pop();
-if (connection =3D=3D null) {
+if (create) {
connection =3D create();
+} else {
+connection =3D connectionPool.pop();
+if (connection =3D=3D null) {
+connection =3D create();
+}
}
} else {
singleConnectionLock.lock();
That suitable and simple approach.
If you have the code for adding a max idle check on hand and tested,
yo=
u
should add it IMO, it will be more efficient.
I will need to give this a couple more weeks of testing. This is what
I
have observed today:
2020-10-14T16:01:47.039 147.54.155.198 w99sezx0... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 92132
20609 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring
directory server connection from pool
20610 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Directory
serve=
r
connection from pool exceeds max idle time, closing it
20611 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing
directory
server connection
20612 2020-10-14T16:00:14 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new
directory server connection
20613 2020-10-14T16:01:47 FEIN [https-openssl-apr-8444-exec-166]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching
for
username 'w99sezx0' in base ...
As you can see it took 90 seconds to server the request because the
connection has expired and close took way too long. In average the
request takes:
2020-10-14T13:57:06.730 10.81.50.232 osipovmi@... "GET
/x2tc-proxy-bln/rest/info/targetSystem HTTP/1.1" 200 8 70
when the connection is healthy.
Ok, so there's some real incentive to avoid reusing a connection that
was
idle for too long.
I made further analysis. I was partially wrong about my statement. The
cause
for the 90 s was a
Re: Required build version(s)
I have complained about this many times... On 2024/02/12 20:58:14 Christopher Schultz wrote: > All, > > The release managers have bumped-up their tool chains to Java 17 or > later for all supported releases. Tomcat 11 requires Java 21. Tomcat > 8.5.x only requires Java 11. > > But the documented Java versions for each release are actually: > > Tomcat 11 - Java 21 > Tomcat 10 - Java 11 > Tomcat 9 - Java 8 > Tomcat 8 - Java 7 > > Theoretically, anyone ought to be able to build Tomcat with the > minimum-required Java version for that branch. There are some practical > reasons why you can't build Tomcat 8.5.x with Java 7 but those have more > to do with supporting versions of Java *after* 7 than anything else. > > I'd like to be able to allow anyone to build their Tomcat from source > using our tooling (e.g. ant-based build) and the minimum Java version > (with one exception: No support for Java 7 for builds). > > Take Tomcat 9.0.x for example. The ant-based build will complain if you > are using Java 11 to build, because the build tools demand the use of > Java 17. If you simply change the "required version" from 17 to 11, the > build works perfectly fine. No, it won't. I have written exactly this last week. BND is now at version 7.0 which requires Java 17at runtime. I have downgraded manually to 6.4.0 to test on HP-UX with Tomcat Native 1.3.x. For the sake of latest BND I don't see a strong reason to force Java 17, especially for those who want to test... My overall opinion is to impose the oldest required version to build and test it. (not talking about release). Michael PS: Thank for raising, Chris! - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Some remarks on panama libssl loading
Rainer, I do not fully understand the problem here. We use libtool to solve
exactly this problem with versioned SONAMEs. It will create symlinks to the
SONAME.
Do you expect anyone even with dlopen() to load libfoo.o.{SOVERSION} unless it
is strictly needed?
E.g.:
lrwxr-xr-x 1 root wheel26 2024-03-22 10:20 /usr/lib/libcrypto.so@ ->
../../lib/libcrypto.so.111
lrwxr-xr-x 1 root wheel 13 2024-03-22 10:20 /usr/lib/libssl.so@ ->
libssl.so.111
-r--r--r-- 1 root wheel 608008 2024-03-22 10:20 /usr/lib/libssl.so.111
and so on...
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Some remarks on panama libssl loading
On 2024/04/17 14:21:06 Rainer Jung wrote:
> Am 17.04.24 um 15:34 schrieb Michael Osipov:
> > Rainer, I do not fully understand the problem here. We use libtool to solve
> > exactly this problem with versioned SONAMEs. It will create symlinks to the
> > SONAME.
> > Do you expect anyone even with dlopen() to load libfoo.o.{SOVERSION} unless
> > it is strictly needed?
> >
> > E.g.:
> > lrwxr-xr-x 1 root wheel26 2024-03-22 10:20 /usr/lib/libcrypto.so@
> > -> ../../lib/libcrypto.so.111
> > lrwxr-xr-x 1 root wheel 13 2024-03-22 10:20 /usr/lib/libssl.so@ ->
> > libssl.so.111
> > -r--r--r-- 1 root wheel 608008 2024-03-22 10:20 /usr/lib/libssl.so.111
> > and so on...
>
> Yes, I expect that! anyone is the JVM :(
>
> The problem is, that the Java API does not care about these well thought
> native traditions. You can not open libssl.so.3 using
> System.loadlibrary(String name), because whatever you give it as "name"
> parameter it will always try to open libname.so. It always prepends
> "lib" to name and always suffixes it with plain ".so".
>
> Yes, it might exist as the first in your list of symlinks, but on most
> linux distributions this link is not installed by default, because it is
> only needed when doing compilations. So it is only installed when you
> install development packages for libs.
Ah, now I see your problem, but it looks like a downstream problem of your
distro of choice, no? I wonder how you compile then custom software if .so
isn't present and the linker cannot find it with -L? What if you install the
devel package to have .so link?
M
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Our ASN.1 parser
Hi folks, currenly we have a minimalistic ASN.1 parser in the code tree w/o any testing since it assumes that the passed byte array is properly encoded. Now, I do have some X.509 related improvements which I'd like to bring upstream from my OSS project which I think will benefit everyone using X.509 (processing SAN from a client cert) in the enterprise world, but this requires extending the parser. In fact, I have written a minimalistic parser for my use case with error handling and swapped for the Tomcat's one and most tests fail with ArrayIndexOutOfBoundsException because our code does not check anything. I do not want to write yet another full-blown parser, but do not also want to reinvent the wheel. So several questions come to my mind: 1. Since I do also have other OSS components for Tomcat which do require an ASN.1 parser would our position be use our parser at your own risk or solve the problem yourself? I need only SEQUENCEs, tagged types. Nothing fancy. 2. Should we consider ditching it for something public and shade it like we do with other components? Apache Kerby ASN.1 is quite small and very decent. Let me know what you think! M (Maybe this discussion applies to the ASN.1 writer as well) - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Our ASN.1 parser
On 2024/05/03 08:59:17 Mark Thomas wrote: > > > On 03/05/2024 08:17, Michael Osipov wrote: > > Hi folks, > > > > currenly we have a minimalistic ASN.1 parser in the code tree w/o any > > testing > > The ASN.1 parsing is covered by the test suite. My bad, I didn't make myself clear: there is not negative testing explicitly for bad input data. > > since it assumes that the passed byte array is properly encoded. > > Correct. For the Tomcat use case it is sufficient that parsing fails if > the input is invalid. It doesn't need to be particularly elegant at that > point. > > > Now, I do have some X.509 related improvements which I'd like to bring > > upstream from my OSS project which I think will benefit everyone using > > X.509 (processing SAN from a client cert) in the enterprise world, > > I'm not seeing much/any demand for this. Can you expand on the use case? Yes, look at SAN:otherName. this custom field in CHOICE [0] is used in Active Directory environments to embed the user principal name into the certificate. With that UPN you can authenticate with certs from smartcard and find the subject in AD via LDAP. The actual code: [1] with the change to Tomcat's parser [2]. That contribution is a separate mail and PR, of course. > > but > > this requires extending the parser. In fact, I have written a > > minimalistic parser for my use case with error handling and swapped for > > the Tomcat's one and most tests fail with ArrayIndexOutOfBoundsException > > because our code does not check anything. > > How minimal is minimal? Lines of code / JAR size? My minimal, for the use case at least few hundred lines. > > I do not want to write yet another full-blown parser, but do not also > > want to reinvent the wheel. > > So several questions come to my mind: > > 1. Since I do also have other OSS components for Tomcat which do require > > an ASN.1 parser would our position be use our parser at your own risk or > > solve the problem yourself? I need only SEQUENCEs, tagged types. Nothing > > fancy. > > What do you need that the current Tomcat implementation doesn't provide? UTF8String, typed tagged values. Basically they are trivial. Simple types, not constructed. > I think our position is use Tomcat's if it works for you. We'll consider > PRs for enhancements if it doesn't. You are, of course, free to write > you own or use another implementation. I'll do the PRs for our parser. > > 2. Should we consider ditching it for something public and shade it like > > we do with other components? Apache Kerby ASN.1 is quite small and very > > decent. > > Tomcat - 180 lines. Kerby - 100 kB JAR. > > Kerby looks like a good choice if we want to go this route. > > There have been discussions about a new tomcat-shaded JAR that would > provide all the shaded dependencies we use both internally and with the > migration tool. My general concern with that is the volume of code. The > migration tool is already a 1MB JAR - most of it shaded code that is > never going to be used. There are tradeoffs to make there that need a > longer discussion. It is likely to be one of the topics at the Tomcat > Security day in Bratislava. What is the benefit of a single JAR here for the public? [1] https://github.com/michael-o/tomcatspnegoad/blob/ca5096b4f0ae7b90ef479a31c066e3808aa70ef2/tomcat90/src/main/java/net/sf/michaelo/tomcat/realm/ActiveDirectoryRealm.java#L372-L408 [2] https://github.com/michael-o/tomcatspnegoad/commit/7d7616a834aadfc80d3533f7eea93458bb2bf53f - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Passing down arbitrary auth attributes down to Realm#authenticate()
On 2024/05/07 18:20:46 Mark Thomas wrote:
> On 07/05/2024 19:06, Michael Osipov wrote:
> > Folks,
> >
> > I am working on a custom Authenticator and Realm where I need to pass
> > down a custom value to Realm#authenticate(), more specially a value
> > obtained from javax.security.auth.Subject#getPrivateCredentials().
> > Currently, there is no such facility in the interface. Any idea how to
> > pass this down w/o touching the interface and w/o thread-local values?
> > The only thing I can think of is a custom realm interface, but that
> > means every realm needs to implement it...
>
> Cast to your custom Realm in your custom Authenticator and then call any
> method you like? It sounds like the Realm and Authenticator are coupled
> together anyway.
Yes, but not quite. The authenticator does work with any realm Tomcat provides,
but should also provide this custom realm of mine. So I will likely consider:
if (realm instanceof CustomRealm) {
doFoo()
} else{
doBar()
}
Thanks Mark!
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Passing down arbitrary auth attributes down to Realm#authenticate()
On 2024/05/07 21:10:33 Christopher Schultz wrote: > Michael, > > On 5/7/24 14:06, Michael Osipov wrote: > > Folks, > > > > I am working on a custom Authenticator and Realm where I need to pass > > down a custom value to Realm#authenticate(), more specially a value > > obtained from javax.security.auth.Subject#getPrivateCredentials(). > > Currently, there is no such facility in the interface. Any idea how to > > pass this down w/o touching the interface and w/o thread-local values? > > The only thing I can think of is a custom realm interface, but that > > means every realm needs to implement it... > > This is the entire reason that the securityfilter[1] project exists. > It's quite old but gets around this kind of thing with... a custom > interface. We use it at $work because we want to be able to get IP > addresses to log logins and login failures. > > Tomcat's Realm-related interfaces have always been too restrictive for > me, but I'm not entirely sure how to get around them. > > I had a conversation with markt years ago at an ApacheCon event where I > asked about strategies to help out with this sort of thing, and his > relatively quick answer without thinking about it too much was to > suggest that (a) anything new and major should probably go into the > JASPIC/Jakarta Authentication component and (b) JASPIC/Jakarta > Authentication might already be able to do what I wanted. > > I didn't follow-up at the time, so I can't validate whether he was right > about (b) or whether (a) would have been particularly easy/hard. Chris, that SF project seems quite abandoned :-( I took once a brief look at JASPIC. I must say it may be the solution to my problem, but currently I am not capable of rewriting the entire code base for it.I Still prefer CMS over "custom" because it gives me subjective better integration. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tcnative] switch from using ERR_error_string to ERR_error_string_n
On 2024/05/14 21:15:03 Christopher Schultz wrote: > All, > > I'd like to basically globally-search-and-replace ERR_error_string for > ERR_error_string_n and use a #define constant for both the > initialization of all > > char err[256]; > > and similar strings and use that same constant for all calls to > ERR_error_string_n.. > > Any objections? > > There should really be no effective change, except: > > 1. We can raise that error message length constant and have it affect > the whole library if we choose. > > 2. We will be using a length-aware string-manipulation call which is > better than using one that assumes that the buffer is at least 256 bytes > long. Sounds reasonable to have one unified spot. Though I wonder how to better address this with BZ 67609 and if resizing/realloc would be required?! M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tcnative] Should we make DEBUG builds available for Windows?
On 2024/05/15 13:12:44 Christopher Schultz wrote: > All, > > A recent thread was posted with a tcnative crash with not much in the > way of useful information in the error: > > https://lists.apache.org/thread/m1dbj3w1x1oqftqsbj7jbnvkm2073x1o > > The error details were: > > " > # EXCEPTION_ACCESS_VIOLATION (0xc005) at pc=0x0001800ccd10, > pid=1244, tid=0x0ab0 > # > # JRE version: OpenJDK Runtime Environment (Zulu 8.60.0.21-CA-win64) > (8.0_322-b06) (build 1.8.0_322-b06) > # Java VM: OpenJDK 64-Bit Server VM (25.322-b06 mixed mode windows-amd64 > compressed oops) > # Problematic frame: > # C [tcnative-1.dll+0xccd10] > # > # Core dump written. Default location: D:\Program > Files\apache-tomcat\bin\hs_err_pid1244.mdmp > " > > So, not super helpful unless you happen to have a debugger handy. > > If we had a debug build available for users, we should be able to get > better information coming back from that failure, possibly a complete > native back-trace. > > IIRC, building a debug version just involves adding something obvious > like /DEBUG to the compiler and/or linker and/or NOT stripping-out the > debug symbols after the build is complete. > > Would this represent a burden on the release manager to produce both > kinds of builds for an official release? I would be in favor of this. For that I have tried it yesterday. Produced a debug build of OpenSSL and libtcnative. MSVC has added to new DLLs to the runtime path and I have no idea where to get them, it does not load. One of it is msvcrtd.dll. Are we actuall able to supply that to users otherwise it does not make sense to provide a debug build. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: (tomcat-native) branch main updated: Ensure local reference capacity is available for array allocations.
Not for 1.3.x?
On 2024/05/16 13:52:45 schu...@apache.org wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> schultz pushed a commit to branch main
> in repository https://gitbox.apache.org/repos/asf/tomcat-native.git
>
>
> The following commit(s) were added to refs/heads/main by this push:
> new e49f0fe5c Ensure local reference capacity is available for array
> allocations.
> e49f0fe5c is described below
>
> commit e49f0fe5c26612df01c636e7019cd70d78948976
> Author: Christopher Schultz
> AuthorDate: Thu May 16 09:51:45 2024 -0400
>
> Ensure local reference capacity is available for array allocations.
> ---
> native/src/jnilib.c | 14 --
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/native/src/jnilib.c b/native/src/jnilib.c
> index 342df3b9c..836502c52 100644
> --- a/native/src/jnilib.c
> +++ b/native/src/jnilib.c
> @@ -133,6 +133,9 @@ jstring tcn_new_stringn(JNIEnv *env, const char *str,
> size_t l)
>
> jbyteArray tcn_new_arrayb(JNIEnv *env, const unsigned char *data, size_t len)
> {
> +if ((*env)->EnsureLocalCapacity(env, 1) < 0) {
> +return NULL; /* out of memory error */
> +}
> jbyteArray bytes = (*env)->NewByteArray(env, (jsize)len);
> if (bytes != NULL) {
> (*env)->SetByteArrayRegion(env, bytes, 0, (jint)len, (jbyte *)data);
> @@ -142,15 +145,22 @@ jbyteArray tcn_new_arrayb(JNIEnv *env, const unsigned
> char *data, size_t len)
>
> jobjectArray tcn_new_arrays(JNIEnv *env, size_t len)
> {
> +if ((*env)->EnsureLocalCapacity(env, 1) < 0) {
> +return NULL; /* out of memory error */
> +}
> return (*env)->NewObjectArray(env, (jsize)len, jString_class, NULL);
> }
>
> jstring tcn_new_string(JNIEnv *env, const char *str)
> {
> -if (!str)
> +if (!str) {
> return NULL;
> -else
> +} else {
> +if ((*env)->EnsureLocalCapacity(env, 1) < 0) {
> +return NULL; /* out of memory error */
> +}
> return (*env)->NewStringUTF(env, str);
> +}
> }
>
> char *tcn_get_string(JNIEnv *env, jstring jstr)
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: dev-h...@tomcat.apache.org
>
>
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: (tomcat-native) branch main updated: Use ERR_error_string_n instead of ERR_error_string.
Not for 1.3.x?
On 2024/05/15 13:14:56 schu...@apache.org wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> schultz pushed a commit to branch main
> in repository https://gitbox.apache.org/repos/asf/tomcat-native.git
>
>
> The following commit(s) were added to refs/heads/main by this push:
> new 4eaa5c93c Use ERR_error_string_n instead of ERR_error_string.
> 4eaa5c93c is described below
>
> commit 4eaa5c93c632f1ea80e889b5458d5b95f57b59a2
> Author: Christopher Schultz
> AuthorDate: Wed May 15 09:14:14 2024 -0400
>
> Use ERR_error_string_n instead of ERR_error_string.
>
> Use header-defined constant for error message buffer sizes.
> ---
> native/include/ssl_private.h | 5 +++
> native/src/ssl.c | 8 ++---
> native/src/sslconf.c | 16 +-
> native/src/sslcontext.c | 76
> ++--
> 4 files changed, 55 insertions(+), 50 deletions(-)
>
> diff --git a/native/include/ssl_private.h b/native/include/ssl_private.h
> index 6c5c9d297..96e21275c 100644
> --- a/native/include/ssl_private.h
> +++ b/native/include/ssl_private.h
> @@ -67,6 +67,11 @@ extern ENGINE *tcn_ssl_engine;
> #define SSL_AIDX_ECC (3)
> #define SSL_AIDX_MAX (4)
>
> +/*
> + * The length of error message strings. MUST BE AT LEAST 256.
> + */
> +#define TCN_OPENSSL_ERROR_STRING_LENGTH 256
> +
> /*
> * Define the SSL options
> */
> diff --git a/native/src/ssl.c b/native/src/ssl.c
> index 7624a4e67..838300c53 100644
> --- a/native/src/ssl.c
> +++ b/native/src/ssl.c
> @@ -1114,9 +1114,9 @@ TCN_IMPLEMENT_CALL(jbyteArray, SSL,
> getPeerCertificate)(TCN_STDARGS,
>
> TCN_IMPLEMENT_CALL(jstring, SSL, getErrorString)(TCN_STDARGS, jlong number)
> {
> -char buf[256];
> +char buf[TCN_OPENSSL_ERROR_STRING_LENGTH];
> UNREFERENCED(o);
> -ERR_error_string(number, buf);
> +ERR_error_string_n(number, buf, TCN_OPENSSL_ERROR_STRING_LENGTH);
> return tcn_new_string(e, buf);
> }
>
> @@ -1278,8 +1278,8 @@ TCN_IMPLEMENT_CALL(jboolean, SSL,
> setCipherSuites)(TCN_STDARGS, jlong ssl,
> return JNI_FALSE;
> }
> if (!SSL_set_cipher_list(ssl_, J2S(ciphers))) {
> -char err[256];
> -ERR_error_string(SSL_ERR_get(), err);
> +char err[TCN_OPENSSL_ERROR_STRING_LENGTH];
> +ERR_error_string_n(SSL_ERR_get(), err,
> TCN_OPENSSL_ERROR_STRING_LENGTH);
> tcn_Throw(e, "Unable to configure permitted SSL ciphers (%s)", err);
> rv = JNI_FALSE;
> }
> diff --git a/native/src/sslconf.c b/native/src/sslconf.c
> index e5b18a7ce..02c3513b1 100644
> --- a/native/src/sslconf.c
> +++ b/native/src/sslconf.c
> @@ -94,8 +94,8 @@ TCN_IMPLEMENT_CALL(jlong, SSLConf, make)(TCN_STDARGS, jlong
> pool,
> ec = SSL_ERR_get();
> if (!cctx || ec != 0) {
> if (ec != 0) {
> -char err[256];
> -ERR_error_string(ec, err);
> +char err[TCN_OPENSSL_ERROR_STRING_LENGTH];
> +ERR_error_string_n(ec, err, TCN_OPENSSL_ERROR_STRING_LENGTH);
> tcn_Throw(e, "Could not create SSL_CONF context (%s)", err);
> } else {
> tcn_Throw(e, "Could not create SSL_CONF context");
> @@ -167,8 +167,8 @@ TCN_IMPLEMENT_CALL(jint, SSLConf, check)(TCN_STDARGS,
> jlong cctx,
> value_type = SSL_CONF_cmd_value_type(c->cctx, J2S(cmd));
> ec = SSL_ERR_get();
> if (ec != 0) {
> -char err[256];
> -ERR_error_string(ec, err);
> +char err[TCN_OPENSSL_ERROR_STRING_LENGTH];
> +ERR_error_string_n(ec, err, TCN_OPENSSL_ERROR_STRING_LENGTH);
> tcn_Throw(e, "Could not determine SSL_CONF command type for '%s'
> (%s)", J2S(cmd), err);
> return 0;
> }
> @@ -270,8 +270,8 @@ TCN_IMPLEMENT_CALL(jint, SSLConf, apply)(TCN_STDARGS,
> jlong cctx,
> ec = SSL_ERR_get();
> if (rc <= 0 || ec != 0) {
> if (ec != 0) {
> -char err[256];
> -ERR_error_string(ec, err);
> +char err[TCN_OPENSSL_ERROR_STRING_LENGTH];
> +ERR_error_string_n(ec, err, TCN_OPENSSL_ERROR_STRING_LENGTH);
> tcn_Throw(e, "Could not apply SSL_CONF command '%s' with value
> '%s' (%s)", J2S(cmd), buf != NULL ? buf : J2S(value), err);
> } else {
> tcn_Throw(e, "Could not apply SSL_CONF command '%s' with value
> '%s'", J2S(cmd), buf != NULL ? buf : J2S(value));
> @@ -302,8 +302,8 @@ TCN_IMPLEMENT_CALL(jint, SSLConf, finish)(TCN_STDARGS,
> jlong cctx)
> ec = SSL_ERR_get();
> if (rc <= 0 || ec != 0) {
> if (ec != 0) {
> -char err[256];
> -ERR_error_string(ec, err);
> +char err[TCN_OPENSSL_ERROR_STRING_LENGTH];
> +ERR_error_string_n(ec, err, TCN_OPENSSL_ERROR_STRING_LENGTH);
> tcn_Throw(e, "Could not finish SSL_CONF commands (%s)", err);
> } else {
> tcn_Throw(e, "Could not finish SSL_CONF commands");
>
Re: (tomcat-native) branch main updated: Ensure local reference capacity is available for array allocations.
On 2024/05/17 14:37:32 Christopher Schultz wrote: > Michael, > > On 5/16/24 10:39, Michael Osipov wrote: > > Not for 1.3.x? > > Good question. I wasn't sure how much energy we are expecting to put > into tcnative 1.3.x. > > I have no problem back-porting this if its what the team wants. I expect 1.3.x to live as long as Tomcat 9.x will live. So it should be on par sans the APR stuff, of course. Everything else will cause us pain. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: WebDAV and Microsoft clients
On 2024/05/22 17:21:07 Mark Thomas wrote: > All, > > I've been looking at the WebDav Servlet for the last few days and in > particular how it interacts with Microsoft clients. Which clients are we talking about? Windows Explorer? I know that DAV Redirector/Explorer are quite picky about TLS and authentication. You might want also try CarotDAV. It served me quite well testing mod_dav and Tomcat WebDAV servlet M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: WebDAV and Microsoft clients
On 2024/05/23 08:00:24 Mark Thomas wrote: > On 22/05/2024 21:47, Michael Osipov wrote: > > On 2024/05/22 17:21:07 Mark Thomas wrote: > >> All, > >> > >> I've been looking at the WebDav Servlet for the last few days and in > >> particular how it interacts with Microsoft clients. > > > > Which clients are we talking about? Windows Explorer? > > Yes. The client that gets used when you map a network drive using a > WebDAV endpoint. > > > I know that DAV Redirector/Explorer are quite picky about TLS and > > authentication. > > Thanks. That is useful to know. I'd read that BASIC auth was disabled by > default and required a registry tweak to use. I hadn't see anything > about TLS. I'll keep that in mind. The only realiably way which works here for both mod_dav and Tomcat 'simpl is via TLS and SPNEGO authentication. The whole thing is designed to work in an enterprise environment, rather not at home. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PROPOSAL] Tomcat 10: Remove Server-Side Includes (SSI)
On 2024/06/07 08:05:34 Mark Thomas wrote: > On 06/06/2024 16:30, Christopher Schultz wrote: > > All, > > > > Resurrecting this thread from 2019. > > > > I'd like to remove the SSI configuration from conf/web.xml and put it > > into webapps/docs/ssi-howto.html. > > > > Are there any objections? > > None here. > > Do we want to go further and consider removing it entirely for Tomcat 12 > onwards. Maybe a question for the users list? I need to admit that there are situations where SSI might be prefered over JSP. Example: I needed limited flexibility for some Asciidoctor generated documents dependening whether it is QA or prod. I didn't want to generate multiple sets of documents (reduce complexity). Now some lines of SSI display a proper QA banner. Good enough for the job. Getting JSP or PHP output with Asciidoctor is almost impossible. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [PROPOSAL] Tomcat 10: Remove Server-Side Includes (SSI)
On 2024/06/07 12:54:44 Christopher Schultz wrote: > Michael, > > On 6/7/24 08:01, Michael Osipov wrote: > > On 2024/06/07 08:05:34 Mark Thomas wrote: > >> On 06/06/2024 16:30, Christopher Schultz wrote: > >>> All, > >>> > >>> Resurrecting this thread from 2019. > >>> > >>> I'd like to remove the SSI configuration from conf/web.xml and put it > >>> into webapps/docs/ssi-howto.html. > >>> > >>> Are there any objections? > >> > >> None here. > >> > >> Do we want to go further and consider removing it entirely for Tomcat 12 > >> onwards. Maybe a question for the users list? > > > > I need to admit that there are situations where SSI might be prefered over > > JSP. > > Example: I needed limited flexibility for some Asciidoctor generated > > documents dependening whether it is QA or prod. I didn't want to generate > > multiple sets of documents (reduce complexity). Now some lines of SSI > > display a proper QA banner. Good enough for the job. Getting JSP or PHP > > output with Asciidoctor is almost impossible. > > It's entirely possible to separate SSI into a different project. I > didn't do it because it uses helper-classes in Tomcat for certain things. > > But if SSI is desirable, it can be packaged separately at the cost of > some additional support classes/methods being copied outside of Tomcat. > > I don't want to support it anymore, but it should be easy *for someone > else* to extract and bundle separately :) What is the pain having it off by default, but have the necessary classes still provided in the JARs? They do not require any maintenance. They just work, don't they? - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: (tomcat) branch main updated: Add RealmBase.getPrincipal(GSSName, GSSCredential, GSSContext)
On 2024/06/11 10:48:29 Mark Thomas wrote:
> On 11/06/2024 11:14, micha...@apache.org wrote:
> > This is an automated email from the ASF dual-hosted git repository.
> >
> > michaelo pushed a commit to branch main
> > in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >
> >
> > The following commit(s) were added to refs/heads/main by this push:
> > new 956c4577ca Add RealmBase.getPrincipal(GSSName, GSSCredential,
> > GSSContext)
> > 956c4577ca is described below
> >
> > commit 956c4577caedca7d77c5eb996a83ebafdf80b889
> > Author: Michael Osipov
> > AuthorDate: Tue Jun 11 11:11:29 2024 +0200
> >
> > Add RealmBase.getPrincipal(GSSName, GSSCredential, GSSContext)
> >
> > A realm might want to access extended/additional information from an
> > established GSS context to properly build an authenticated princpal.
> > One prominent example is
> > com.sun.security.jgss.ExtendedGSSContext.inquireSecContext(InquireType).
>
> Do we want to deprecate the old method?
A very good question. If removed in 11 then a custom component forn 10 will not
work in 11. Though, those are just two lines to satisfy ABI. From a consistency
perspective I'd deprecate remove. The problem is that if we deprecate the other
method that would imply that GSSContext could be null and that would not be
acceptable for that method because no inquiry would be possible.
We have two use cases:
* RealmBase.authenticate(GSSContext, boolean):
Truly expects to come from a SPNEGO authenticator and will provide the GSS
context, then passed to new method.
* RealmBase.authenticate(GSSName, GSSCredential):
Does not need to come from a SPNEGO authenticator, e.g. local testing (use
current user) or from X.509 authentication the both obtained with protocol
transition (S4U2proxy), then passed to getPrincipal(GSSName, GSSCredential)
I need to think about wether it'd be whise to remove. What should never happen
is:
protected Principal getPrincipal(GSSName gssName, GSSCredential gssCredential) {
return getPrincipal(gssName, gssCredential, null);
}
protected Principal getPrincipal(GSSName gssName, GSSCredential gssCredential,
GSSContext gssContext) {
...do magic
}
from my point of view. For the time being I'd keep it.
Thoughts?
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: BND 7 and multi release JARs
On 2024/06/11 14:15:48 Rémy Maucherat wrote: > Hi, > > To fix the issue with having Java 22 classes in tomcat-coyote (and > embedded), I was looking at multi release JARs. I think it would work > fine *if* we were building the JARs ourselves (jarIt task), but then > the jars are actually rebuilt with bnd. > > Supposedly bnd 7.0.0 (which we just upgraded to) supports multi > release jars. After looking at their testsuite, it seems adding > "Multi-release: true" to the bnd and having the classes in the right > spot (META-INF/versions/22) would be enough [see: > https://github.com/bndtools/bnd/pull/5581/files ]. Unfortunately, this > keeps doing nothing for me. If anyone can get it to work, let me know. > > Anyway, instead of doing something too complex, I would instead be > back to producing a small tomcat-coyote-ffm jar. Then embedded users > can still use that small jar, even though it's a bit annoying to not > have it included in the big embed jar ... The naming of the jar will > be "stable" since even if adding quic/h3 to it somehow, the jar name > remains appropriate. > > Obviously all the mess comes from the combination of these two items: > - FFM missing the Java 21 cruise ship > - EE 11 downgrading to Java 17 > > :( > > So is it ok if I add a new tomcat-coyote-ffm.jar in lib ? I'd prefer to do both. MRJAR with module-info to solve issues like https://bz.apache.org/bugzilla/show_bug.cgi?id=66631 and a separate JAR. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: BND 7 and multi release JARs
On 2024/06/11 14:15:48 Rémy Maucherat wrote: > Hi, > > To fix the issue with having Java 22 classes in tomcat-coyote (and > embedded), I was looking at multi release JARs. I think it would work > fine *if* we were building the JARs ourselves (jarIt task), but then > the jars are actually rebuilt with bnd. > > Supposedly bnd 7.0.0 (which we just upgraded to) supports multi > release jars. After looking at their testsuite, it seems adding > "Multi-release: true" to the bnd and having the classes in the right > spot (META-INF/versions/22) would be enough [see: > https://github.com/bndtools/bnd/pull/5581/files ]. Unfortunately, this > keeps doing nothing for me. If anyone can get it to work, let me know. I looked at this more closely. That example above I have applied some time with Robert Scholte to my project with Maven [1]. It does work flawlessly. The BND tool for Ant is rather limited. What will work for you is the following: * Add the flag: > $ git diff -U0 > diff --git a/res/META-INF/default.manifest b/res/META-INF/default.manifest > index 8937c41931..87919e7dee 100644 > --- a/res/META-INF/default.manifest > +++ b/res/META-INF/default.manifest > @@ -9,0 +10 @@ X-Compile-Target-JDK: @target.jdk@ > +Multi-Release: true * Add a separate compilation step for the FFM stuff with a different --release option to a separate dir. Update the packaging of the JAR with META-INF/versions/22 and fill with those classes. That will work. But beware, BND generates a module-info.class for the root code. What if the versioned one needs another module-info.class? What if the OSGi metadata need to be different for the FFM code? There is no versioned MANIFEST.MF. If this is not a problem it will do for you. For me it does: $ curl -sL https://search.maven.org/remotecontent?filepath=net/sf/michael-o/activedirectory-dns-locator/0.4.0/activedirectory-dns-locator-0.4.0.jar | tar tzf - META-INF/ META-INF/MANIFEST.MF META-INF/LICENSE.txt META-INF/services/ META-INF/services/com.sun.jndi.ldap.spi.LdapDnsProvider META-INF/services/javax.naming.ldap.spi.LdapDnsProvider META-INF/versions/ META-INF/versions/12/ META-INF/versions/12/net/ META-INF/versions/12/net/sf/ META-INF/versions/12/net/sf/michaelo/ META-INF/versions/12/net/sf/michaelo/activedirectory/ META-INF/versions/12/net/sf/michaelo/activedirectory/ActiveDirectoryLdapDnsProvider.class net/ net/sf/ net/sf/michaelo/ net/sf/michaelo/activedirectory/ net/sf/michaelo/activedirectory/ActiveDirectoryDnsLocator$1.class net/sf/michaelo/activedirectory/ActiveDirectoryDnsLocator$Builder.class net/sf/michaelo/activedirectory/ActiveDirectoryDnsLocator$HostPort.class net/sf/michaelo/activedirectory/ActiveDirectoryDnsLocator$SrvRecord.class net/sf/michaelo/activedirectory/ActiveDirectoryDnsLocator.class net/sf/michaelo/activedirectory/ActiveDirectoryLdapDnsProvider.class ... Michael [1] https://github.com/michael-o/activedirectory-dns-locator/blob/main/pom.xml - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: (tomcat) branch main updated: Add RealmBase.getPrincipal(GSSName, GSSCredential, GSSContext)
On 2024/06/11 16:47:37 Mark Thomas wrote:
> On 11/06/2024 12:45, Michael Osipov wrote:
> > On 2024/06/11 10:48:29 Mark Thomas wrote:
> >> On 11/06/2024 11:14, micha...@apache.org wrote:
> >>> This is an automated email from the ASF dual-hosted git repository.
> >>>
> >>> michaelo pushed a commit to branch main
> >>> in repository https://gitbox.apache.org/repos/asf/tomcat.git
> >>>
> >>>
> >>> The following commit(s) were added to refs/heads/main by this push:
> >>>new 956c4577ca Add RealmBase.getPrincipal(GSSName, GSSCredential,
> >>> GSSContext)
> >>> 956c4577ca is described below
> >>>
> >>> commit 956c4577caedca7d77c5eb996a83ebafdf80b889
> >>> Author: Michael Osipov
> >>> AuthorDate: Tue Jun 11 11:11:29 2024 +0200
> >>>
> >>> Add RealmBase.getPrincipal(GSSName, GSSCredential, GSSContext)
> >>>
> >>> A realm might want to access extended/additional information from an
> >>> established GSS context to properly build an authenticated princpal.
> >>> One prominent example is
> >>> com.sun.security.jgss.ExtendedGSSContext.inquireSecContext(InquireType).
> >>
> >> Do we want to deprecate the old method?
> >
> > A very good question. If removed in 11 then a custom component forn 10 will
> > not work in 11. Though, those are just two lines to satisfy ABI. From a
> > consistency perspective I'd deprecate remove. The problem is that if we
> > deprecate the other method that would imply that GSSContext could be null
> > and that would not be acceptable for that method because no inquiry would
> > be possible.
> >
> > We have two use cases:
> > * RealmBase.authenticate(GSSContext, boolean):
> > Truly expects to come from a SPNEGO authenticator and will provide the GSS
> > context, then passed to new method.
> > * RealmBase.authenticate(GSSName, GSSCredential):
> > Does not need to come from a SPNEGO authenticator, e.g. local testing (use
> > current user) or from X.509 authentication the both obtained with protocol
> > transition (S4U2proxy), then passed to getPrincipal(GSSName, GSSCredential)
> >
> > I need to think about wether it'd be whise to remove. What should never
> > happen is:
> > protected Principal getPrincipal(GSSName gssName, GSSCredential
> > gssCredential) {
> >return getPrincipal(gssName, gssCredential, null);
> > }
> > protected Principal getPrincipal(GSSName gssName, GSSCredential
> > gssCredential, GSSContext gssContext) {
> > ...do magic
> > }
> > from my point of view. For the time being I'd keep it.
> >
> > Thoughts?
>
> If there are use cases for both, I'm happy to keep both.
Let's keep both for at least one release. I need some time to assess the
possible impact of a removal.
M
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: (tomcat) branch main updated: Restore warning to use only this listener with Server
On 2024/06/14 09:39:23 ma...@apache.org wrote:
> This is an automated email from the ASF dual-hosted git repository.
>
> markt pushed a commit to branch main
> in repository https://gitbox.apache.org/repos/asf/tomcat.git
>
>
> The following commit(s) were added to refs/heads/main by this push:
> new 65e0baa525 Restore warning to use only this listener with Server
> 65e0baa525 is described below
>
> commit 65e0baa525f81a585bfb249c5db1ed3c812763f4
> Author: Mark Thomas
> AuthorDate: Fri Jun 14 10:38:48 2024 +0100
>
> Restore warning to use only this listener with Server
> ---
> java/org/apache/catalina/core/AprLifecycleListener.java | 6 ++
> 1 file changed, 6 insertions(+)
>
> diff --git a/java/org/apache/catalina/core/AprLifecycleListener.java
> b/java/org/apache/catalina/core/AprLifecycleListener.java
> index 848eacf647..a06d9672c8 100644
> --- a/java/org/apache/catalina/core/AprLifecycleListener.java
> +++ b/java/org/apache/catalina/core/AprLifecycleListener.java
> @@ -23,6 +23,7 @@ import java.util.List;
> import org.apache.catalina.Lifecycle;
> import org.apache.catalina.LifecycleEvent;
> import org.apache.catalina.LifecycleListener;
> +import org.apache.catalina.Server;
> import org.apache.juli.logging.Log;
> import org.apache.juli.logging.LogFactory;
> import org.apache.tomcat.jni.Library;
> @@ -34,6 +35,8 @@ import org.apache.tomcat.util.res.StringManager;
> /**
> * Implementation of LifecycleListener that will init and and
> destroy APR.
> *
> + * This listener must only be nested within {@link Server} elements.
> + *
> * Only one instance of the APR/Native library may be loaded per JVM.
> Loading multiple instances will trigger a JVM
> * crash - typically when the Connectors are destroyed. This listener
> utilises reference counting to ensure that only
> * one instance of the APR/Native library is loaded at any one time.
> @@ -128,6 +131,9 @@ public class AprLifecycleListener implements
> LifecycleListener {
>
> if (Lifecycle.BEFORE_INIT_EVENT.equals(event.getType())) {
> synchronized (lock) {
> +if (!(event.getLifecycle() instanceof Server)) {
> +log.warn(sm.getString("listener.notServer",
> event.getLifecycle().getClass().getSimpleName()));
> +}
> if (referenceCount++ != 0) {
> // Already loaded (note test is performed before
> reference count is incremented)
> return;
Thank you, Mark. Restore appreciated!
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Release Managers wanted
Am 2021-05-13 um 00:10 schrieb Mark Thomas: All, Assuming 7.0.109 is the last Tomcat 7 release I am the current release manager for 10.0.x, 9.0.x, 8.5.x, migration tool, native and mod_jk. I'd like to share the load and the knowledge a little. The step-by-step release process was documented for Tomcat 7: https://cwiki.apache.org/confluence/display/TOMCAT/ReleaseProcess Looking at the cwiki it does not apply to tcnative..Do you have instructions? My long-term goal would be to provide a CMake-based generator, cross-platform. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint
Am 2021-05-24 um 20:55 schrieb Mark Thomas: On 24/05/2021 18:43, Rémy Maucherat wrote: On Mon, May 24, 2021 at 6:34 PM wrote: This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new 620e06c Remove code deprecated in 10.1.x apart from the APR Endpoint 620e06c is described below commit 620e06c468a02ca98dc4beacf556dd12d2440877 Author: Mark Thomas AuthorDate: Mon May 24 17:34:09 2021 +0100 Remove code deprecated in 10.1.x apart from the APR Endpoint Ah, it's *that* APR time again :) Indeed. My plan was to give it a few days now folks have been reminded that this is going to happen and then do the APR removal in a separate commit. I have an important fix for APR on Windows which affects also Tomcat as well. A very subtile one. Can you hold off for a moment? I will prepare a decent decription for dev@tomcat.a.o today and already have a suitable fix for it. Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint
Am 2021-05-25 um 09:44 schrieb Mark Thomas: On 25/05/2021 08:27, Michael Osipov wrote: Am 2021-05-24 um 20:55 schrieb Mark Thomas: On 24/05/2021 18:43, Rémy Maucherat wrote: On Mon, May 24, 2021 at 6:34 PM wrote: This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new 620e06c Remove code deprecated in 10.1.x apart from the APR Endpoint 620e06c is described below commit 620e06c468a02ca98dc4beacf556dd12d2440877 Author: Mark Thomas AuthorDate: Mon May 24 17:34:09 2021 +0100 Remove code deprecated in 10.1.x apart from the APR Endpoint Ah, it's *that* APR time again :) Indeed. My plan was to give it a few days now folks have been reminded that this is going to happen and then do the APR removal in a separate commit. I have an important fix for APR on Windows which affects also Tomcat as well. A very subtile one. Can you hold off for a moment? I will prepare a decent decription for dev@tomcat.a.o today and already have a suitable fix for it. What is the benefit of holding off on the removal of the APR endpoint from 10.1.x? Just for clarification, 10.x on main will move to 10.1.x and APR will only run on 9 and 8.5? M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch main updated: Remove code deprecated in 10.1.x apart from the APR Endpoint
Am 2021-05-25 um 10:06 schrieb Mark Thomas: On 25/05/2021 09:03, Michael Osipov wrote: Am 2021-05-25 um 09:44 schrieb Mark Thomas: On 25/05/2021 08:27, Michael Osipov wrote: Am 2021-05-24 um 20:55 schrieb Mark Thomas: On 24/05/2021 18:43, Rémy Maucherat wrote: On Mon, May 24, 2021 at 6:34 PM wrote: This is an automated email from the ASF dual-hosted git repository. markt pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/tomcat.git The following commit(s) were added to refs/heads/main by this push: new 620e06c Remove code deprecated in 10.1.x apart from the APR Endpoint 620e06c is described below commit 620e06c468a02ca98dc4beacf556dd12d2440877 Author: Mark Thomas AuthorDate: Mon May 24 17:34:09 2021 +0100 Remove code deprecated in 10.1.x apart from the APR Endpoint Ah, it's *that* APR time again :) Indeed. My plan was to give it a few days now folks have been reminded that this is going to happen and then do the APR removal in a separate commit. I have an important fix for APR on Windows which affects also Tomcat as well. A very subtile one. Can you hold off for a moment? I will prepare a decent decription for dev@tomcat.a.o today and already have a suitable fix for it. What is the benefit of holding off on the removal of the APR endpoint from 10.1.x? Just for clarification, 10.x on main will move to 10.1.x and APR will only run on 9 and 8.5? Correct. 10.1.x will continue to support the use of the Tomcat Native Connector (i.e. the native library) to provide OpenSSL support for NIO and NIO2. The plan is that version 2 of that library will be significantly reduced in scope to only provide the functionality necessary to hook into OpenSSL. Alright, please decide yourself whether this should go into 10.x [1] for those who want branch off/retain APR here. Otherwise I will provide a polished patch for < 10. Michael [1] https://github.com/michael-o/tomcat/commit/5393bcb991b924a18169ba34264ec2bc5c009b22 - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
APR connector on Windows 8+ fails to listen on all addresses
Folks, we needed to deploy Tomcat 9.0.x on a Windows server (no jokes, please), but the contractor wasn't able to configure the APR connector to accept on external interfaces even after a day. After my analysis it turned out be a subtile bug in libapr which affects Windows users only. I am also surprised why no one complained before. Setup: * Windows 8+ or Windows Server 2016/2019 * Have at least IPv6 available, no IP addresses necessary, ::1 is sufficient * Any Tomcat with libtcnative 1.2.28 with the DLL compiled by Mark Thomas. * Start Tomcat with the AprLifecycleListener and make sure that no address (hostname) is set. To make a long investigation story short: libapr, thus libtcnative suffer from a very subtile bug only visible on dual-stack systems. Since on INET6 sockets IPV6_V6ONLY is 1 by default on Windows, no IPv4 addresses are bound. In the case above, Tomcat is only accessible on ::1. APR is supposed to set IPV6_V6ONLY to 0 by default, but this fails because APR 1.7.x does not recognise anything above Windows 7 and assumes it to be Windows XP by default. As you might know Vista was the first Windows with true IPv6 an dual-sockets. When setsockopt is invoked APR gives you 70023, not implemented. I was able, according to Mark's instructions, to compile OpenSSL, APR and Tomcat Native on Windows 10 and deploy on Windows Server 2019. I'd like to push * https://github.com/michael-o/tomcat/compare/main...clean-bind * https://github.com/michael-o/tomcat-native/compare/main...clean-bind as well as the real fix in APR 1.7.x: https://github.com/michael-o/apr/compare/1.7.x...1.7.x-windows I ran all unit tests (main) with those modifications on these platforms: * Windows 10, APR 1.7.0, 1.7.1-dev * Windows Server 2019, APR 1.7.0, 1.7.1-dev * FreeBSD 12-STABLE, APR 1.7.0, 1.7.1-dev * RHEL 7, APR 1.4.8 * HP-UX 11i, APR 1.6.6 Some hosts are dual-stack, some IPv4 only. Moreover, I wrote a simple program which binds the socket for tracing only: https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e It runs as expected on the above platforms, even with zone id on link-local addresses. If no one objects, I'll merge soon. Mark, I don't know when the next APR release will happen, but I consider this to be very annoying. Maybe it makes sense to push 1.2.29 with APR 1.7.1-dev to please Windows users? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Renamings with APR removal in 10.1.x
Mark, since you are going to remove all bits soon, will you rename Java items with still carry the APR substring? E.g., AprLifecycleListener will be a misleading name for obvious reasons. Michael - To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org For additional commands, e-mail: dev-h...@maven.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: Renamings with APR removal in 10.1.x
Am 2021-05-26 um 15:29 schrieb Mark Thomas: On 25/05/2021 17:27, Michael Osipov wrote: Mark, since you are going to remove all bits soon, will you rename Java items with still carry the APR substring? E.g., AprLifecycleListener will be a misleading name for obvious reasons. I hadn't considered that but it seems like a good opportunity to remove some of the confusion that that naming has caused. I think a switch to TomcatNativeLifecylcleListener would work. My expectation is that in the early milestone releases for 10.1.x that the listener looks for Tomcat Native 1.2.x but at some point before the first stable release it switches to Tomcat Native 2.0.x. That assumes we find the time to create Tomcat Native 2.0.x. Given that it will mostly be deleting code, that should be doable. Have also a look at build.xml: test.apr.loc should also be renamed to test.tomcat-native.loc. - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: APR connector on Windows 8+ fails to listen on all addresses
Am 2021-05-26 um 15:25 schrieb Mark Thomas: On 25/05/2021 17:23, Michael Osipov wrote: Nice research. Mark, I don't know when the next APR release will happen, but I consider this to be very annoying. Maybe it makes sense to push 1.2.29 with APR 1.7.1-dev to please Windows users? I'd prefer to use a released version of APR but failing that we can provide a patch against 1.7.0. If you can add the appropriate patch here: https://github.com/apache/tomcat-native/tree/main/native/srclib/apr I can include it in the next release. The change has been committed to apr-1.7.x, here [1] is the changeset, compiled for me nicely several times. Would be good if you could also check at your end. I will inquire on dev@apr.a.o if someone is willing to perform 1.7.1 soon. We are close to starting the June releases so if we move quickly on a Tomcat-Native release now, we can included a fixed version of Tomcat Native in the next set of Tomcat releases. I have also applied by changes (simplifcations) to Tomcat Native, but Mladen objected [2] and asked me to move to a PR to discuss [3], but I am still waiting for an explanation of his objection. The purpose of the change was to simplify and have consistency with the Java code where we don't do such things. Hopefully we can setttle this before 1.2.29. M [1] https://github.com/apache/apr/commit/2bcd4b3ddb108d16f1c758c00a45de9aef57aa3a [2] https://github.com/apache/tomcat-native/commit/420cd1c159e4f27bb5d2a873dbd1fb7ea5d3473c [3] https://github.com/apache/tomcat-native/pull/9 - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: APR connector on Windows 8+ fails to listen on all addresses
Am 2021-05-27 um 07:39 schrieb Mladen Turk: APR has to be compiled with -D_WIN32_WINNT=0x0600 -DWINVER=0x0600 Same with Tomcat Native Use apr-1.6.x Both don't work out for the following reasons: * Your setenv.bat sets WINVER=0x0601 which fails for the NMakefile which in turn requires a symbolic name instead of a hex value, I have to pass WINVER=WIN7 anyway, then the rest works. * Even if I do, the check in APR is performed at runtime for the OS version, therefore setting the socket option fails. * How is 1.6.x going me to help here if it does not now anything about Windows 8, 10? This [1] is what you need to make it work. Kindly respond to my PR. M [1] https://github.com/apache/apr/commit/2bcd4b3ddb108d16f1c758c00a45de9aef57aa3a On 25/05/2021 18:23, Michael Osipov wrote: Folks, we needed to deploy Tomcat 9.0.x on a Windows server (no jokes, please), but the contractor wasn't able to configure the APR connector to accept on external interfaces even after a day. After my analysis it turned out be a subtile bug in libapr which affects Windows users only. I am also surprised why no one complained before. Setup: * Windows 8+ or Windows Server 2016/2019 * Have at least IPv6 available, no IP addresses necessary, ::1 is sufficient * Any Tomcat with libtcnative 1.2.28 with the DLL compiled by Mark Thomas. * Start Tomcat with the AprLifecycleListener and make sure that no address (hostname) is set. To make a long investigation story short:http://svn.apache.org/viewvc?rev=1889037&view=rev libapr, thus libtcnative suffer from a very subtile bug only visible on dual-stack systems. Since on INET6 sockets IPV6_V6ONLY is 1 by default on Windows, no IPv4 addresses are bound. In the case above, Tomcat is only accessible on ::1. APR is supposed to set IPV6_V6ONLhttp://svn.apache.org/viewvc?rev=1889037&view=revY to 0 by default, but this fails because APR 1.7.x does not recognise anything above Windows 7 and assumes it to be Windows XP by default. As you might know Vista was the first Windows with true IPv6 an dual-sockets. When setsockopt is invoked APR gives you 70023, not implemented. I was able, according to Mark's instructions, to compile OpenSSL, APR and Tomcat Native on Windows 10 and deploy on Windows Server 2019. I'd like to push * https://github.com/michael-o/tomcat/compare/main...clean-bind * https://github.com/michael-o/tomcat-native/compare/main...clean-bind as well as the real fix in APR 1.7.x: https://github.com/michael-o/apr/compare/1.7.x...1.7.x-windows I ran all unit tests (main) with those modifications on these platforms: * Windows 10, APR 1.7.0, 1.7.1-dev * Windows Server 2019, APR 1.7.0, 1.7.1-dev * FreeBSD 12-STABLE, APR 1.7.0, 1.7.1-dev * RHEL 7, APR 1.4.8 * HP-UX 11i, APR 1.6.6 Some hosts are dual-stack, some IPv4 only. Moreover, I wrote a simple program which binds the socket for tracing only: https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e It runs as expected on the above platforms, even with zone id on link-local addresses. If no one objects, I'll merge soon. Mark, I don't know when the next APR release will happen, but I consider this to be very annoying. Maybe it makes sense to push 1.2.29 with APR 1.7.1-dev to please Windows users? Michael - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: APR connector on Windows 8+ fails to listen on all addresses
Am 2021-05-27 um 11:52 schrieb Mark Thomas: Michael, I think we need to step back a bit. I am unable to recreate the issue you describe. I am using: - Windows Server 2019 - IPv6 enabled (and IPv4 and both have addresses) - Apache Tomcat 9.0.46 - Apache Tomcat Native 1.2.28 - AprLifecycleListener is enabled - No address configured for Connector - Logs confirm http-apr-8080 I can access the default home page via both IPv4 and IPv6 from both the local machine and remotely. On 25/05/2021 18:23, Michael Osipov wrote: >>> * Windows 8+ or Windows Server 2016/2019 * Have at least IPv6 available, no IP addresses necessary, ::1 is sufficient * Any Tomcat with libtcnative 1.2.28 with the DLL compiled by Mark Thomas. * Start Tomcat with the AprLifecycleListener and make sure that no address (hostname) is set. I think I have replicated the above but obviously I have missed something. What do I need to do to recreate your results? Please try my simple program from this gist: https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e Vanilla DLL on Windows 10: PS Z:\> java "-cp" ".;C:\Entwicklung\Projekte\tomcat-native\dist\tomcat-native-1.2.29-dev.jar" "-Djava.library.path=C:\Entwicklung\Programme\apache-tomcat-9.0.46\bin" AprTest libtcnative version: 1.2.28, libapr version 1.7.0 hostname: null, family: 2, next: 522994096 setsockopt IPV6_V6ONLY status: 70023 socket bind status: 0 socket listen status: 0 An IPv6 only socket is bound because default is 1 on the socket option and one cannot set to 0: PS C:\Users\osipovmi> netstat -a | Select-String -Pattern TCP[::]: md2pcvtc:0 ABHÖREN then compiled with patches: PS Z:\> java "-cp" ".;C:\Entwicklung\Projekte\tomcat-native\dist\tomcat-native-1.2.29-dev.jar" "-Djava.library.path=C:\Entwicklung\Projekte\tomcat-native\native\WIN7_X64_DLL_RELEASE" AprTest libtcnative version: 1.2.29-dev, libapr version 1.7.1-dev hostname: null, family: 2, next: 523321776 setsockopt IPV6_V6ONLY status: 0 socket bind status: 0 socket listen status: 0 An IPv6 socket with dual-stack has been bound: PS C:\Users\osipovmi> netstat -a | Select-String -Pattern TCP0.0.0.0: md2pcvtc:0 ABHÖREN TCP[::]: md2pcvtc:0 ABHÖREN Please note that I have IPv6 enabled, but no IP addresses configured. By default only ::1 is present. Addresses: PS C:\Users\osipovmi> Get-NetIPAddress | Format-Table ifIndex IPAddress PrefixLength PrefixOrigin SuffixOrigin AddressState PolicyStore --- - --- 1 ::1 128 WellKnown WellKnownPreferredActiveStore 20 169.254.228.218 16 WellKnown Link TentativeActiveStore 32 192.168.58.19328 Manual Manual PreferredActiveStore 15 169.254.26.74 16 WellKnown Link TentativeActiveStore 18 169.254.76.82 16 WellKnown Link TentativeActiveStore 22 169.254.155.142 16 WellKnown Link TentativeActiveStore 21 192.168.1.23 24 Dhcp Dhcp PreferredActiveStore 1 127.0.0.1 8 WellKnown WellKnownPreferredActiveStore Does this help? M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: APR connector on Windows 8+ fails to listen on all addresses
Prefences in the OS: PS C:\Users\osipovmi> netsh interface ipv6 show prefixpolicies Der aktive Status wird abgefragt... Vorgänger Label Präfix -- - 50 0 :::0:0/96 40 1 ::1/128 30 2 ::/0 20 3 2002::/16 5 5 2001::/32 3 13 fc00::/7 1 11 fec0::/10 1 12 3ffe::/16 1 4 ::/96 Taken from https://superuser.com/a/436944/222550 - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: APR connector on Windows 8+ fails to listen on all addresses
Am 2021-05-27 um 15:41 schrieb Mark Thomas: On 27/05/2021 13:24, Mark Thomas wrote: On 27/05/2021 11:12, Michael Osipov wrote: Am 2021-05-27 um 11:52 schrieb Mark Thomas: Michael, I think we need to step back a bit. I am unable to recreate the issue you describe. I am using: - Windows Server 2019 - IPv6 enabled (and IPv4 and both have addresses) - Apache Tomcat 9.0.46 - Apache Tomcat Native 1.2.28 - AprLifecycleListener is enabled - No address configured for Connector - Logs confirm http-apr-8080 I can access the default home page via both IPv4 and IPv6 from both the local machine and remotely. Please try my simple program from this gist: https://gist.github.com/michael-o/dfb86df472f62d2b2dff6ef12ee3758e Does this help? Thanks. That helped me narrow down where the differences where. If you run Tomcat as a service it binds both 0.0.0.0:8080 and [::]:8080. If you run it from the command line, it only binds [::]:8080. I'll take a look at the proposed APR patch next. I can confirm that the patch to APR fixes the issue for me too. With the original 1.2.28 DLL, starting Tomcat on the command line only binds to IPv6. With a patched 1.2.29-dev DLL, starting Tomcat on the command line binds to IPv6 and IPv4. Pefect! I'll tag and start the release process for Tomcat Native shortly. Before you do, I'd like to settle https://github.com/apache/tomcat-native/pull/9 This aligns with the NIO connectors for obtaining addresses by relying on the OS. Please have a look and share your opinion. M - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [VOTE] Release Apache Tomcat Native 1.2.30
Am 2021-06-01 um 11:53 schrieb Mark Thomas: Resending with correct subject line... Version 1.2.30 includes the following changes compared to 1.2.28 - Fix an issue where some Windows systems in some configurations would only listen on IPv6 addresses on dual stack systems even though configured to listen on both IPv6 and IPv4 addresses. - Complete the fix for BZ 65181 - Correct constants used for Windows versions. Expand versions supported. The proposed release artefacts can be found at [1], and the build was done using tag [2]. The Apache Tomcat Native 1.2.30 release is [ ] Stable, go ahead and release [ ] Broken because of ... +1 - To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
Re: [tomcat] branch main updated: Portable temp path code
Am 2021-06-30 um 17:45 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/tomcat.git
The following commit(s) were added to refs/heads/main by this push:
new d2ef158 Portable temp path code
d2ef158 is described below
commit d2ef158945a4aca00dc06900c65c059cbb66c410
Author: remm
AuthorDate: Wed Jun 30 17:45:20 2021 +0200
Portable temp path code
---
test/org/apache/tomcat/util/net/TestXxxEndpoint.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
b/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
index 653e751..efc4dd1 100644
--- a/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
+++ b/test/org/apache/tomcat/util/net/TestXxxEndpoint.java
@@ -213,7 +213,9 @@ public class TestXxxEndpoint extends TomcatBaseTest {
c.getProtocolHandlerClassName().contains("NioProtocol")
&& JreCompat.isJre16Available());
-final String unixDomainSocketPath = "/tmp/testUnixDomainSocket";
+File tempPath = File.createTempFile("tomcat", ".uds");
The common extension for UDS is actually '.sock'
-
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
