thelabdude commented on a change in pull request #2132:
URL: https://github.com/apache/lucene-solr/pull/2132#discussion_r554052572
##########
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/DrillStream.java
##########
@@ -252,29 +253,27 @@ public void setStreamContext(StreamContext streamContext)
{
}
protected void constructStreams() throws IOException {
-
try {
-
Object pushStream = ((Expressible)
tupleStream).toExpression(streamFactory);
-
- List<String> shardUrls = getShards(this.zkHost, this.collection,
this.streamContext);
-
- for(int w=0; w<shardUrls.size(); w++) {
- ModifiableSolrParams paramsLoc = new ModifiableSolrParams();
- paramsLoc.set(DISTRIB,"false"); // We are the aggregator.
- paramsLoc.set("expr", pushStream.toString());
- paramsLoc.set("qt","/export");
- paramsLoc.set("fl", fl);
- paramsLoc.set("sort", sort);
- paramsLoc.set("q", q);
- String url = shardUrls.get(w);
- SolrStream solrStream = new SolrStream(url, paramsLoc);
+ final ModifiableSolrParams paramsLoc = new ModifiableSolrParams();
+ paramsLoc.set(DISTRIB,"false"); // We are the aggregator.
+ paramsLoc.set("expr", pushStream.toString());
+ paramsLoc.set("qt","/export");
+ paramsLoc.set("fl", fl);
+ paramsLoc.set("sort", sort);
+ paramsLoc.set("q", q);
+ getReplicas(this.zkHost, this.collection, this.streamContext,
paramsLoc).forEach(r -> {
Review comment:
This refactor makes use of the enhancement provided by SOLR-14987 to
reuse HttpSolrClients per host vs. one per replica
##########
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/FacetStream.java
##########
@@ -20,15 +20,19 @@
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
+import java.util.HashMap;
Review comment:
This source file did not adhere to the community code format so I
reformatted it. It's bad practice in general, but since my PR was getting
dinged for format issues, I chose to fix globally in this file vs. piecemeal.
##########
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/FacetStream.java
##########
@@ -537,14 +555,29 @@ public void setStreamContext(StreamContext context) {
}
public void open() throws IOException {
- if(cache != null) {
+ if (cache != null) {
cloudSolrClient = cache.getCloudSolrClient(zkHost);
} else {
final List<String> hosts = new ArrayList<>();
hosts.add(zkHost);
cloudSolrClient = new Builder(hosts,
Optional.empty()).withSocketTimeout(30000).withConnectionTimeout(15000).build();
}
+ // Parallelize the facet expression across multiple collections for an
alias using plist if possible
+ if (params.getBool("plist", defaultPlistEnabled)) {
Review comment:
this is the main hook to see if the auto-plist approach should apply to
this facet expression
##########
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/FacetStream.java
##########
@@ -842,10 +881,99 @@ public int getCost() {
@Override
public StreamComparator getStreamSort() {
- if(bucketSorts.length > 1) {
- return new MultipleFieldComparator(bucketSorts);
+ return (bucketSorts.length > 1) ? new MultipleFieldComparator(bucketSorts)
: bucketSorts[0];
+ }
+
+ @Override
Review comment:
Implementation of the `ParallelMetricsRollup` interface starts here ...
##########
File path:
solr/solrj/src/java/org/apache/solr/client/solrj/io/stream/metrics/MaxMetric.java
##########
@@ -86,7 +86,7 @@ public void update(Tuple tuple) {
if(l > longMax) {
longMax = l;
}
- } else {
+ } else if(o instanceof Long) {
Review comment:
the current format is consistent with the rest of the source file, best
not to mix formatting even if it's wrong
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
