[doris] 04/15: [fix](s3 outfile) Add the`use_path_style` parameter for s3 outfile (#15288)

[morningman]

This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git

commit 5c4a99dff9ebda02434f45c54db440ff7f9d6ed2
Author: Tiewei Fang <43782773+bepppo...@users.noreply.github.com>
AuthorDate: Fri Dec 23 16:22:06 2022 +0800

    [fix](s3 outfile) Add the`use_path_style` parameter for s3 outfile (#15288)
    
    Currently, `outfile` did not support `use_path_style` parameter and use 
`virtual-host style` by default,
    however some Object-storage may only support `use_path_style` access mode.
    
    This pr add the`use_path_style` parameter for s3 outfile, so that different 
object-storage can use different access mode.
---
 .../sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md  | 1 +
 .../sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md  | 1 +
 fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java | 4 ++++
 3 files changed, 6 insertions(+)

diff --git 
a/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md 
b/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
index b094fcdd3f..d46b2164d7 100644
--- 
a/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
+++ 
b/docs/en/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
@@ -101,6 +101,7 @@ illustrate:
     AWS_ACCESS_KEY
     AWS_SECRET_KEY
     AWS_REGION
+    use_path_stype: (optional) default false . The S3 SDK uses the 
virtual-hosted style by default. However, some object storage systems may not 
be enabled or support virtual-hosted style access. At this time, we can add the 
use_path_style parameter to force the use of path style access method.
     ```
 
 ### example
diff --git 
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
 
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
index 82f8ecec33..25b35f5894 100644
--- 
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
+++ 
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Manipulation-Statements/OUTFILE.md
@@ -104,6 +104,7 @@ INTO OUTFILE "file_path"
     AWS_ACCESS_KEY
     AWS_SECRET_KEY
     AWS_REGION
+    use_path_stype: (选填) 默认为false 。S3 SDK 默认使用 virtual-hosted style 
方式。但某些对象存储系统可能没开启或不支持virtual-hosted style 方式的访问，此时可以添加 use_path_style 参数来强制使用 
path style 访问方式。
     ```
 
 ### example
diff --git 
a/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java 
b/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
index 3d3f82b74b..78f57cff9c 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/OutFileClause.java
@@ -648,6 +648,10 @@ public class OutFileClause {
             }
         }
         if (storageType == StorageBackend.StorageType.S3) {
+            if (properties.containsKey(S3Resource.USE_PATH_STYLE)) {
+                brokerProps.put(S3Resource.USE_PATH_STYLE, 
properties.get(S3Resource.USE_PATH_STYLE));
+                processedPropKeys.add(S3Resource.USE_PATH_STYLE);
+            }
             S3Storage.checkS3(brokerProps);
         } else if (storageType == StorageBackend.StorageType.HDFS) {
             if (!brokerProps.containsKey(HdfsResource.HADOOP_FS_NAME)) {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org