[doris] 03/15: [bug](bdbje) Add retry for reSetupBdbEnvironment() `restore.execute()` (#18777)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git

commit e3da0c0e910e06706a4dfcb63d7803ec7af25b43
Author: Lei Zhang <27994433+swjtu-zhang...@users.noreply.github.com>
AuthorDate: Fri Apr 21 20:58:42 2023 +0800

    [bug](bdbje) Add retry for reSetupBdbEnvironment() `restore.execute()` 
(#18777)
    
    * In reSetupBdbEnvironment() `restore.execute()` may throw 
NullPointerException,
      add retry for `restore.execute()`
---
 .../apache/doris/journal/bdbje/BDBJEJournal.java   | 23 ++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java 
b/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java
index 5376fd9547..72617c9b43 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBJEJournal.java
@@ -356,10 +356,25 @@ public class BDBJEJournal implements Journal { // 
CHECKSTYLE IGNORE THIS LINE: B
         // ATTN: here we use `getServingEnv()`, because only serving catalog 
has
         // helper nodes.
         Pair<String, Integer> helperNode = Env.getServingEnv().getHelperNode();
-        NetworkRestore restore = new NetworkRestore();
-        NetworkRestoreConfig config = new NetworkRestoreConfig();
-        config.setRetainLogFiles(false);
-        restore.execute(insufficientLogEx, config);
+
+        for (int i = 0; i < RETRY_TIME; i++) {
+            try {
+                NetworkRestore restore = new NetworkRestore();
+                NetworkRestoreConfig config = new NetworkRestoreConfig();
+                config.setRetainLogFiles(false);
+                restore.execute(insufficientLogEx, config);
+                break;
+            } catch (Exception e) {
+                LOG.warn("retry={}, reSetupBdbEnvironment exception:", i, e);
+                try {
+                    Thread.sleep(5 * 1000);
+                    LOG.warn("after sleep insufficientLogEx:", 
insufficientLogEx);
+                } catch (InterruptedException e1) {
+                    LOG.warn("InterruptedException", e1);
+                }
+            }
+        }
+
         bdbEnvironment.close();
         bdbEnvironment.setup(new File(environmentPath), selfNodeName, 
selfNodeHostPort,
                 helperNode.first + ":" + helperNode.second, 
Env.getServingEnv().isElectable());


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org