[doris] branch master updated: [fix](DOE) only return first batch data in ES 8.x (#16025)

[yiguolei]

This is an automated email from the ASF dual-hosted git repository.

yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
     new 3bff5ebf9a [fix](DOE) only return first batch data in ES 8.x (#16025)
3bff5ebf9a is described below

commit 3bff5ebf9ab95b9daa9ded9fd3b77b536c2bf6af
Author: qiye <jianliang5...@gmail.com>
AuthorDate: Wed Jan 18 09:28:34 2023 +0800

    [fix](DOE) only return first batch data in ES 8.x (#16025)
    
    Do not use terminate_after and size together in scroll request of ES 8.x.
---
 be/src/exec/es/es_scan_reader.cpp | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/be/src/exec/es/es_scan_reader.cpp 
b/be/src/exec/es/es_scan_reader.cpp
index e262020a51..0002d8f2d6 100644
--- a/be/src/exec/es/es_scan_reader.cpp
+++ b/be/src/exec/es/es_scan_reader.cpp
@@ -76,8 +76,8 @@ ESScanReader::ESScanReader(const std::string& target,
         std::stringstream scratch;
         // just send a normal search  against the elasticsearch with 
additional terminate_after param to achieve terminate early effect when limit 
take effect
         if (_type.empty()) {
+            // `terminate_after` and `size` can not be used together in scroll 
request of ES 8.x
             scratch << _target << REQUEST_SEPARATOR << _index << "/_search?"
-                    << "terminate_after=" << props.at(KEY_TERMINATE_AFTER)
                     << REQUEST_PREFERENCE_PREFIX << _shards << "&" << 
filter_path;
         } else {
             scratch << _target << REQUEST_SEPARATOR << _index << 
REQUEST_SEPARATOR << _type
@@ -92,9 +92,10 @@ ESScanReader::ESScanReader(const std::string& target,
         // scroll request for scanning
         // add terminate_after for the first scroll to avoid decompress all 
postings list
         if (_type.empty()) {
+            // `terminate_after` and `size` can not be used together in scroll 
request of ES 8.x
             scratch << _target << REQUEST_SEPARATOR << _index << "/_search?"
                     << "scroll=" << _scroll_keep_alive << 
REQUEST_PREFERENCE_PREFIX << _shards
-                    << "&" << filter_path << "&terminate_after=" << 
batch_size_str;
+                    << "&" << filter_path;
         } else {
             scratch << _target << REQUEST_SEPARATOR << _index << 
REQUEST_SEPARATOR << _type
                     << "/_search?"


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org