[doris] branch branch-1.2-lts updated: [branch1.2](broker)(orc) catch exception in orc reader and fix filesystem close issue in broker (#24128)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
     new 68b33583f8 [branch1.2](broker)(orc) catch exception in orc reader and 
fix filesystem close issue in broker (#24128)
68b33583f8 is described below

commit 68b33583f83b4aa26d59f1ee488fa222a21148a0
Author: Mingyu Chen <morning...@163.com>
AuthorDate: Sun Sep 10 13:01:30 2023 +0800

    [branch1.2](broker)(orc) catch exception in orc reader and fix filesystem 
close issue in broker (#24128)
    
    1. catch exception in orc reader, to avoid BE crash
    2. Do not close filesystem in Broker, it is unnecessary and will return 
error and causing BE crash
---
 be/src/vec/exec/format/orc/vorc_reader.cpp                   | 12 ++++++++----
 .../java/org/apache/doris/broker/hdfs/BrokerFileSystem.java  |  3 ++-
 2 files changed, 10 insertions(+), 5 deletions(-)

diff --git a/be/src/vec/exec/format/orc/vorc_reader.cpp 
b/be/src/vec/exec/format/orc/vorc_reader.cpp
index 7a87c40731..b3e447e8a5 100644
--- a/be/src/vec/exec/format/orc/vorc_reader.cpp
+++ b/be/src/vec/exec/format/orc/vorc_reader.cpp
@@ -770,10 +770,14 @@ Status OrcReader::get_next_block(Block* block, size_t* 
read_rows, bool* eof) {
         SCOPED_RAW_TIMER(&_statistics.get_batch_time);
         // reset decimal_scale_params_index
         _decimal_scale_params_index = 0;
-        if (!_row_reader->next(*_batch)) {
-            *eof = true;
-            *read_rows = 0;
-            return Status::OK();
+        try {
+            if (!_row_reader->next(*_batch)) {
+                *eof = true;
+                *read_rows = 0;
+                return Status::OK();
+            }
+        } catch (std::exception& e) {
+            return Status::InternalError("orc read fail. reason = {}", 
e.what());
         }
     }
     const auto& batch_vec = 
down_cast<orc::StructVectorBatch*>(_batch.get())->fields;
diff --git 
a/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/BrokerFileSystem.java
 
b/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/BrokerFileSystem.java
index 22f6925d31..e3e27fe1c2 100644
--- 
a/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/BrokerFileSystem.java
+++ 
b/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/BrokerFileSystem.java
@@ -55,7 +55,8 @@ public class BrokerFileSystem {
         try {
             if (this.dfsFileSystem != null) {
                 try {
-                    this.dfsFileSystem.close();
+                    // do not close file system, it will be closed 
automatically.
+                    // this.dfsFileSystem.close();
                 } catch (Exception e) {
                     logger.error("errors while close file system", e);
                 } finally {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org