Am 2020-10-13 um 13:49 schrieb Rémy Maucherat:
On Tue, Oct 13, 2020 at 11:33 AM Michael Osipov <micha...@apache.org> wrote:
Am 2020-10-07 um 22:34 schrieb r...@apache.org:
This is an automated email from the ASF dual-hosted git repository.
remm pushed a commit to branch 8.5.x
in repository https://gitbox.apache.org/repos/asf/tomcat.git
commit 50de36b7874da98591345e40b374a1e2dd52c188
Author: remm <r...@apache.org>
AuthorDate: Thu Jan 30 17:22:51 2020 +0100
Add connection pool to JNDI realm
This implements a TODO from the class javadoc header.
As described in the javadoc, the idea is to use a pool to avoid
blocking
on a single connection, which could possibly become a bottleneck in
some
cases. The message formats need to be kept along with the connection
since they are not thread safe.
Preserve the default behavior: sync without pooling (using a Lock
object
which is more flexible).
I may backport this since this is limited to the JNDI realm, but
only
once it is confirmed to be regression free. Tested with ApacheDS
but my
LDAP skills are very limited.
---
java/org/apache/catalina/realm/JNDIRealm.java | 442
++++++++++++---------
.../apache/catalina/realm/LocalStrings.properties | 1 +
test/org/apache/catalina/realm/TestJNDIRealm.java | 7 +-
webapps/docs/changelog.xml | 3 +
webapps/docs/config/realm.xml | 7 +
5 files changed, 276 insertions(+), 184 deletions(-)
Salut Rémy,
this is a very very nice improvement to the matter and I gave your idea
a spin my public Active Directory realm. Based on my tests with AD I see
room for improvement: (Note that I don't use the JNDIRealm because
because is it too generic and usable for me)
Ok, I only tested this with a local ldap, so not much. I verified it was
pooling connections.
* You might want to consider to introduce a maxIdleTime to avoid stale
connections, e.g., AD has default value of 15 minutes. Yep, I had had
several RSTs resulting in 401s
Oops. So I don't think adding something like this helps since there could
be plenty of issues besides a timeout. Basically it's supposed to retry
instead when something fails. Can you give me a stack trace to show what's
wrong ?
I consider it cheaper to test for a timeout and close actively rather
than recover from a resetted connection. Here is a strack trace for
concurrency of 4 (with a pool of max 8). Compare the timestamp of the
first line and the subsequent ones:
2020-10-12T22:55:47 FEIN [https-openssl-apr-8444-exec-8]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.release Releasing directory
server connection to pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory
server connection from pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Validating directory
server connection from pool
2020-10-12T23:13:06 SCHWERWIEGEND [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Exception validating
directory server connection
javax.naming.CommunicationException: Connection reset [Root exception is
java.net.SocketException: Connection reset]; remaining name ''
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2030)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1872)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1797)
at
com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at
javax.naming.directory.InitialDirContext.search(InitialDirContext.java:296)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate(ActiveDirectoryRealm.java:316)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire(ActiveDirectoryRealm.java:285)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getPrincipal(ActiveDirectoryRealm.java:244)
at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501)
at
net.sf.michaelo.tomcat.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:165)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633)
at
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:571)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:615)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2070)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:211)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at
com.sun.jndi.ldap.sasl.SaslInputStream.readFully(SaslInputStream.java:166)
at com.sun.jndi.ldap.sasl.SaslInputStream.fill(SaslInputStream.java:123)
at com.sun.jndi.ldap.sasl.SaslInputStream.read(SaslInputStream.java:90)
at com.sun.jndi.ldap.Connection.run(Connection.java:846)
... 1 more
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Validating directory
server connection from pool
2020-10-12T23:13:06 SCHWERWIEGEND [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Exception validating
directory server connection
javax.naming.CommunicationException: Connection reset [Root exception is
java.net.SocketException: Connection reset]; remaining name ''
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2030)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1872)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1797)
at
com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at
javax.naming.directory.InitialDirContext.search(InitialDirContext.java:296)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate(ActiveDirectoryRealm.java:316)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire(ActiveDirectoryRealm.java:285)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getPrincipal(ActiveDirectoryRealm.java:244)
at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501)
at
net.sf.michaelo.tomcat.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:165)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633)
at
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:571)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:615)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2070)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:211)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at
com.sun.jndi.ldap.sasl.SaslInputStream.readFully(SaslInputStream.java:166)
at com.sun.jndi.ldap.sasl.SaslInputStream.fill(SaslInputStream.java:123)
at com.sun.jndi.ldap.sasl.SaslInputStream.read(SaslInputStream.java:90)
at com.sun.jndi.ldap.Connection.run(Connection.java:846)
... 1 more
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Validating directory
server connection from pool
2020-10-12T23:13:06 SCHWERWIEGEND [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Exception validating
directory server connection
javax.naming.CommunicationException: Connection reset [Root exception is
java.net.SocketException: Connection reset]; remaining name ''
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2030)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1872)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1797)
at
com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at
javax.naming.directory.InitialDirContext.search(InitialDirContext.java:296)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate(ActiveDirectoryRealm.java:316)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire(ActiveDirectoryRealm.java:285)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getPrincipal(ActiveDirectoryRealm.java:244)
at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501)
at
net.sf.michaelo.tomcat.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:165)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633)
at
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:571)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:615)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2070)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:211)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at
com.sun.jndi.ldap.sasl.SaslInputStream.readFully(SaslInputStream.java:166)
at com.sun.jndi.ldap.sasl.SaslInputStream.fill(SaslInputStream.java:123)
at com.sun.jndi.ldap.sasl.SaslInputStream.read(SaslInputStream.java:90)
at com.sun.jndi.ldap.Connection.run(Connection.java:846)
... 1 more
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Validating directory
server connection from pool
2020-10-12T23:13:06 SCHWERWIEGEND [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate Exception validating
directory server connection
javax.naming.CommunicationException: Connection reset [Root exception is
java.net.SocketException: Connection reset]; remaining name ''
at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:2030)
at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1872)
at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1797)
at
com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
at
com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
at
javax.naming.directory.InitialDirContext.search(InitialDirContext.java:296)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.validate(ActiveDirectoryRealm.java:316)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire(ActiveDirectoryRealm.java:285)
at
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getPrincipal(ActiveDirectoryRealm.java:244)
at org.apache.catalina.realm.RealmBase.authenticate(RealmBase.java:501)
at
net.sf.michaelo.tomcat.authenticator.SpnegoAuthenticator.doAuthenticate(SpnegoAuthenticator.java:165)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:633)
at
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:571)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:615)
at
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:818)
at
org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2070)
at
org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:211)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at
com.sun.jndi.ldap.sasl.SaslInputStream.readFully(SaslInputStream.java:166)
at com.sun.jndi.ldap.sasl.SaslInputStream.fill(SaslInputStream.java:123)
at com.sun.jndi.ldap.sasl.SaslInputStream.read(SaslInputStream.java:90)
at com.sun.jndi.ldap.Connection.run(Connection.java:846)
... 1 more
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.close Closing directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-5]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-8]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory
server connection from pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-8]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-10]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory
server connection from pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-10]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new directory
server connection
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-4]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.acquire Acquiring directory
server connection from pool
2020-10-12T23:13:06 FEIN [https-openssl-apr-8444-exec-4]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.open Opening new directory
server connection
2020-10-12T23:13:07 FEIN [https-openssl-apr-8444-exec-10]
net.sf.michaelo.tomcat.realm.ActiveDirectoryRealm.getUser Searching for
username 'osipovmi' in base 'DC=ad001,DC=siemens,DC=net' and attribute
'sAMAccountName' with mapper 'SamAccountNameRfc2247Mapper'
https://github.com/apache/tomcat/blob/master/java/org/apache/catalina/realm/JNDIRealm.java#L1306
Looking at the code, it does retry, but it would simply get a new
connection from the pool again. So if it gets a second bad connection, then
it would most likely fail. Is it what happened for you ?
Correct!
A solution could be to create a new connection for the pooled case, to make
up for that.
That is one, but also consider that other connections in the stack are
broken too in such a case. So the next authenticate request would run in
into the same issue.
* Validating connections optionally might be a good option to detect
stale connections or broken ones ny networks issues. This works for me
very fast:
protected boolean validate(DirContextConnection connection) {
if (logger.isDebugEnabled())
logger.debug(sm.getString("activeDirectoryRealm.validate"));
SearchControls controls = new SearchControls();
controls.setSearchScope(SearchControls.OBJECT_SCOPE);
controls.setCountLimit(1);
controls.setReturningAttributes(new String[] {
"objectClass" });
controls.setTimeLimit(500);
try {
NamingEnumeration<SearchResult> results =
connection.context.search("", "objectclass=*",
controls);
if (results.hasMore()) {
close(results);
return true;
}
} catch (NamingException e) {
logger.error(sm.getString("activeDirectoryRealm.validate.namingException"),
e);
return false;
}
return false;
}
I do this in the acquire() method
Ok, but it has a cost and I fundamentally don't see why it's different from
performing the actual operation.
It has a cost (like "select 1 from dual"), but consider that better than
solving the connection issue in authenticate() and having the user auth
fail at all.
M
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org
For additional commands, e-mail: dev-h...@tomcat.apache.org
