[
https://issues.apache.org/jira/browse/SOLR-14537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17151231#comment-17151231
]
Ishan Chattopadhyaya commented on SOLR-14537:
---------------------------------------------
I see some gradle precommit failures due to this.
{code:java}
> Task :solr:core:compileTestJava
/home/ishan/code/lucene-solr/solr/core/src/test/org/apache/solr/handler/export/TestExportWriter.java:764:
warning: [unchecked] unchecked conversion
rspMap = mapper.readValue(rsp, HashMap.class);
^
required: Map<String,Object>
found: HashMap
/home/ishan/code/lucene-solr/solr/core/src/test/org/apache/solr/handler/export/TestExportWriter.java:765:
warning: [unchecked] unchecked cast
docs = (List<Map<String, Object>>) Utils.getObjectByPath(rspMap, false,
"/response/docs");
^
required: List<Map<String,Object>>
found: Object
/home/ishan/code/lucene-solr/solr/core/src/test/org/apache/solr/handler/export/TestExportWriter.java:779:
warning: [unchecked] unchecked conversion
rspMap = mapper.readValue(rsp, HashMap.class);
^
required: Map<String,Object>
found: HashMap
/home/ishan/code/lucene-solr/solr/core/src/test/org/apache/solr/handler/export/TestExportWriter.java:781:
warning: [unchecked] unchecked cast
docs = (List<Map<String, Object>>) Utils.getObjectByPath(rspMap, false,
"/response/docs");
^
required: List<Map<String,Object>>
found: Object
error: warnings found and -Werror specified
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
1 error
4 warnings> Task :solr:core:compileTestJava FAILED
WARNING: there were unreferenced files under license folder:
-
/home/ishan/code/lucene-solr/solr/licenses/org.restlet.ext.servlet-2.4.0.jar.sha1FAILURE:
Build failed with an exception.
{code}
I'll take a look. By the way, in the attached PR here, there were gradle
precommit failures too.
> Improve performance of ExportWriter
> -----------------------------------
>
> Key: SOLR-14537
> URL: https://issues.apache.org/jira/browse/SOLR-14537
> Project: Solr
> Issue Type: Improvement
> Security Level: Public(Default Security Level. Issues are Public)
> Components: Export Writer
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Priority: Major
> Fix For: 8.7
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Retrieving, sorting and writing out documents in {{ExportWriter}} are three
> aspects of the /export handler that can be further optimized.
> SOLR-14470 introduced some level of caching in {{StringValue}}. Further
> options for caching and speedups should be explored.
> Currently the sort/retrieve and write operations are done sequentially, but
> they could be parallelized, considering that they block on different channels
> - the first is index reading & CPU bound, the other is bound by the receiving
> end because it uses blocking IO. The sorting and retrieving of values could
> be done in parallel with the operation of writing out the current batch of
> results.
> One possible approach here would be to use "double buffering" where one
> buffered batch that is ready (already sorted and retrieved) is being written
> out, while the other batch is being prepared in a background thread, and when
> both are done the buffers are swapped. This wouldn't complicate the current
> code too much but it should instantly give up to 2x higher throughput.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
