[
https://issues.apache.org/jira/browse/SOLR-12859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017689#comment-17017689
]
Cao Manh Dat edited comment on SOLR-12859 at 1/17/20 9:31 AM:
--------------------------------------------------------------
To be honest, I'm never fully understand the current authentication framework
of Solr. When I did the HTTP/2 things, I basically convert the current
interceptor of Apache HttpClient to an equivalent version. After spend sometime
to look at the current code and the documentation. I'm guessing that
{{isSolrThread()}} is a naive/workaround way to check whether the request is
about to send to another node was actually sent by a Solr node or not?
Let's look into this comment
{quote}
//if this is not running inside a Solr threadpool (as in testcases)
// then no need to add any header
{quote}
above comment will make sense if we notice how the interceptors was added for
Apache HttpClient
{{HttpClientUtil.addRequestInterceptor(interceptor)}} -> interceptor is added
into a static variable. This is ok if a JVM only host one node, but with test,
a JVM will host several nodes, so there are several PKI interceptors will be
added to that static variable. Moreoever every Apache HttpClient created by
HttpClientUtil will share the same list of interceptors even with client
created in test. So a client created by test, before sending a request will go
through n generateToken() interceptors.
In that case how generateToken() can discriminate a request sent from a node
with a request sent from a test method if all client will use a same list of
interceptors? The naive solution was setting a flag called {{isSolrThread}} to
distinguish these two case.
In most of cases, a request sent by a node will be sent from a thread from a
threadPool created by {{ExecutorUtil}}. So to make auth tests pass
{{isServerPool.set(Boolean.TRUE);}} is set before calling any {{Runnable}}.
With all of these context less review the mystery code again
{code}
SolrRequestInfo reqInfo = getRequestInfo();
String usr;
if (reqInfo != null) {
// 1.Author's idea: Ok, the thread is holding a request, if
authentication is enabled, the req must hold a Principal
Principal principal = reqInfo.getUserPrincipal();
if (principal == null) {
// 2. Author's idea: the req did not pass authentication since
Principal is not set, do not need to do anything here!
// my comment: this is not true, SolrRequestInfo is also used as a
garbage to put data into so many place rely on data inside SolrRequestInfo, the
present of SolrRequestInfo does not mean that it comes from outside.
return Optional.empty();
} else {
usr = principal.getName();
}
} else {
if (!isSolrThread()) {
// 3. Author's idea: so the req is not sent inside a thread created by
ExecutorUtil, it must come from test code or outside world
// my comment: it is not true, since in
{{DocExpirationUpdateProcessorFactory}} a {{ScheduledThreadPoolExecutor}} was
used instead of a threadPool created by ExecutorUtil
return Optional.empty();
}
// 4. Author's idea: if the req is sent by ExecutorUtil, it must come out
from this node.
usr = "$"; //special name to denote the user is the node itself
}
{code}
But with new HTTP/2 client, interceptor is added to each client object, so
there are no single static variable here -> no sharing interceptors between
clients of nodes and clients of test -> if the interceptor's code is called it
must be sent from a node. So the mystery block can be changed to for the
interceptor of HTTP/2 client
{code}
SolrRequestInfo reqInfo = getRequestInfo();
String usr = NODE_IS_USER;
if (reqInfo != null && reqInfo.getUserPrincipal() != null)
usr = reqInfo.getUserPrincipal().getName()
{code}
was (Author: caomanhdat):
To be honest, I'm never fully understand the current authentication framework
of Solr. When I did the HTTP/2 things, I basically convert the current
interceptor of Apache HttpClient to an equivalent version. After spend sometime
to look at the current code and the documentation. I'm guessing that
{{isSolrThread()}} is a naive/workaround way to check whether the request is
about to send to another node was actually sent by a Solr node or not?
Let's look into this comment
{quote}
//if this is not running inside a Solr threadpool (as in testcases)
// then no need to add any header
{quote}
above comment will make sense if we notice how the interceptors was added for
Apache HttpClient
{{HttpClientUtil.addRequestInterceptor(interceptor)}} -> interceptor is added
into a static variable. This is ok if a JVM only host one node, but with test,
a JVM will host several nodes, so there are several PKI interceptors will be
added to that static variable. Moreoever every Apache HttpClient created by
HttpClientUtil will share the same list of interceptors even with client
created in test. So how we can discriminate a request sent from a client inside
a node with a request sent from a client inside a test method if all client
will use a same list of interceptors? The naive solution was setting a flag
called {{isSolrThread}} to distinguish these two case.
In most of cases, a request sent by a node will be sent from a thread from a
threadPool created by {{ExecutorUtil}}. So to make auth tests pass
{{isServerPool.set(Boolean.TRUE);}} is set before calling any {{Runnable}}.
With all of these context less review the mystery code again
{code}
SolrRequestInfo reqInfo = getRequestInfo();
String usr;
if (reqInfo != null) {
// 1.Author's idea: Ok, the thread is holding a request, if
authentication is enabled, the req must hold a Principal
Principal principal = reqInfo.getUserPrincipal();
if (principal == null) {
// 2. Author's idea: the req did not pass authentication since
Principal is not set, do not need to do anything here!
// my comment: this is not true, SolrRequestInfo is also used as a
garbage to put data into so many place rely on data inside SolrRequestInfo, the
present of SolrRequestInfo does not mean that it comes from outside.
return Optional.empty();
} else {
usr = principal.getName();
}
} else {
if (!isSolrThread()) {
// 3. Author's idea: so the req is not sent inside a thread created by
ExecutorUtil, it must come from test code or outside world
// my comment: it is not true, since in
{{DocExpirationUpdateProcessorFactory}} a {{ScheduledThreadPoolExecutor}} was
used instead of a threadPool created by ExecutorUtil
return Optional.empty();
}
// 4. Author's idea: if the req is sent by ExecutorUtil, it must come out
from this node.
usr = "$"; //special name to denote the user is the node itself
}
{code}
But with new HTTP/2 client, interceptor is added to each client object, so
there are no single static variable here -> no sharing interceptors between
clients of nodes and clients of test -> if the interceptor's code is called it
must be sent from a node. So the mystery block can be changed to for the
interceptor of HTTP/2 client
{code}
SolrRequestInfo reqInfo = getRequestInfo();
String usr = NODE_IS_USER;
if (reqInfo != null && reqInfo.getUserPrincipal() != null)
usr = reqInfo.getUserPrincipal().getName()
{code}
> DocExpirationUpdateProcessorFactory does not work with BasicAuth
> ----------------------------------------------------------------
>
> Key: SOLR-12859
> URL: https://issues.apache.org/jira/browse/SOLR-12859
> Project: Solr
> Issue Type: Bug
> Affects Versions: 7.5
> Reporter: Varun Thacker
> Priority: Major
> Attachments: SOLR-12859.patch
>
>
> I setup a cluster with basic auth and then wanted to use Solr's TTL feature (
> DocExpirationUpdateProcessorFactory ) to auto-delete documents.
>
> Turns out it doesn't work when Basic Auth is enabled. I get the following
> stacktrace from the logs
> {code:java}
> 2018-10-12 22:06:38.967 ERROR (autoExpireDocs-42-thread-1) [ ]
> o.a.s.u.p.DocExpirationUpdateProcessorFactory Runtime error in periodic
> deletion of expired docs: Async exception during distributed update: Error
> from server at http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6:
> require authentication
> request:
> http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F192.168.0.8%3A8983%2Fsolr%2Fgettingstarted_shard1_replica_n2%2F&wt=javabin&version=2
> org.apache.solr.update.processor.DistributedUpdateProcessor$DistributedUpdatesAsyncException:
> Async exception during distributed update: Error from server at
> http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6: require
> authentication
> request:
> http://192.168.0.8:8983/solr/gettingstarted_shard2_replica_n6/update?update.distrib=TOLEADER&distrib.from=http%3A%2F%2F192.168.0.8%3A8983%2Fsolr%2Fgettingstarted_shard1_replica_n2%2F&wt=javabin&version=2
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.doFinish(DistributedUpdateProcessor.java:964)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.DistributedUpdateProcessor.finish(DistributedUpdateProcessor.java:1976)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.finish(LogUpdateProcessorFactory.java:182)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.UpdateRequestProcessor.finish(UpdateRequestProcessor.java:80)
> ~[solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df -
> jimczi - 2018-09-18 13:07:55]
> at
> org.apache.solr.update.processor.DocExpirationUpdateProcessorFactory$DeleteExpiredDocsRunnable.run(DocExpirationUpdateProcessorFactory.java:419)
> [solr-core-7.5.0.jar:7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi
> - 2018-09-18 13:07:55]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [?:1.8.0_112]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [?:1.8.0_112]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [?:1.8.0_112]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> [?:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [?:1.8.0_112]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [?:1.8.0_112]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_112]{code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
