This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new b0960626805 [feature-wip](arrow-flight)(step6) Support regression test
(#27847)
b0960626805 is described below
commit b096062680510bf0417f4f40e3fe17d69443481c
Author: Xinyi Zou <zouxiny...@gmail.com>
AuthorDate: Mon Dec 4 19:23:56 2023 +0800
[feature-wip](arrow-flight)(step6) Support regression test (#27847)
Design Documentation Linked to #25514
Regression test add a new group: arrow_flight_sql,
./run-regression-test.sh -g arrow_flight_sql to run regression-test, can
use jdbc:arrow-flight-sql to run all Suites whose group contains
arrow_flight_sql.
./run-regression-test.sh -g p0,arrow_flight_sql to run regression-test, can
use jdbc:arrow-flight-sql to run all Suites whose group contains
arrow_flight_sql, and use jdbc:mysql to run other Suites whose group contains
p0 but does not contain arrow_flight_sql.
Requires attention, the formats of jdbc:arrow-flight-sql and jdbc:mysql and
mysql client query results are different, for example:
Datatime field type: jdbc:mysql returns 2010-01-02T05:09:06, mysql client
returns 2010-01-02 05:09:06, jdbc:arrow-flight-sql also returns 2010-01-02
05:09 :06.
Array and Map field types: jdbc:mysql returns ["ab", "efg", null], {"f1":
1, "f2": "a"}, jdbc:arrow-flight-sql returns ["ab ","efg",null],
{"f1":1,"f2":"a"}, which is missing spaces.
Float field type: jdbc:mysql and mysql client returns 6.333,
jdbc:arrow-flight-sql returns 6.333000183105469, in
query_p0/subquery/test_subquery.groovy.
If the query result is empty, jdbc:arrow-flight-sql returns empty and
jdbc:mysql returns \N.
use database; and query should be divided into two SQL executions as much
as possible. otherwise the results may not be as expected. For example: USE
information_schema; select cast ("0.0101031417" as datetime) The result is
2000-01-01 03:14:1 (constant fold), select cast ("0.0101031417" as datetime)
The result is null (no constant fold),
In addition, doris jdbc:arrow-flight-sql still has unfinished parts, such
as:
Unsupported data type: Decimal256. INVALID_ARGUMENT: [INTERNAL_ERROR]Fail
to convert block data to arrow data, error: [E3] write_column_to_arrow with
type Decimal256
Unsupported null value of map key. INVALID_ARGUMENT: [INTERNAL_ERROR]Fail
to convert block data to arrow data, error: [E33] Can not write null value of
map key to arrow.
Unsupported data type: ARRAY<MAP<TEXT,TEXT>>
jdbc:arrow-flight-sql not support connecting to specify DB name, such
asjdbc:arrow-flight-sql://127.0.0.1:9090/{db_name}", In order to be compatible
with regression-test, use db_nameis added before all SQLs
whenjdbc:arrow-flight-sql` runs regression test.
select timediff("2010-01-01 01:00:00", "2010-01-02 01:00:00");, error
java.lang.NumberFormatException: For input string: "-24:00:00"
---
be/src/common/status.h | 8 +++
be/src/util/arrow/row_batch.cpp | 10 ++-
be/src/util/arrow/utils.cpp | 5 +-
.../java/org/apache/doris/qe/StmtExecutor.java | 13 ++--
.../arrowflight/DorisFlightSqlProducer.java | 44 ++++++++----
.../service/arrowflight/DorisFlightSqlService.java | 5 +-
.../arrowflight/results/FlightSqlChannel.java | 2 +-
.../sessions/FlightSessionsManager.java | 7 --
.../sessions/FlightSessionsWithTokenManager.java | 17 ++++-
regression-test/conf/regression-conf.groovy | 6 ++
regression-test/framework/pom.xml | 22 ++++++
.../org/apache/doris/regression/Config.groovy | 23 +++++-
.../doris/regression/action/ExplainAction.groovy | 10 ++-
.../org/apache/doris/regression/suite/Suite.groovy | 84 ++++++++++++++++++++--
.../doris/regression/suite/SuiteContext.groovy | 34 ++++++++-
.../apache/doris/regression/util/JdbcUtils.groovy | 6 ++
.../query_p0/aggregate/aggregate_count1.groovy | 2 +-
.../query_p0/aggregate/select_distinct.groovy | 2 +-
.../join_with_column_casesensetive.groovy | 2 +-
.../suites/query_p0/cast/test_cast.groovy | 2 +-
.../query_p0/except/test_query_except.groovy | 2 +-
.../query_p0/group_concat/test_group_concat.groovy | 2 +-
.../grouping_sets/test_grouping_sets1.groovy | 2 +-
.../suites/query_p0/having/having.groovy | 2 +-
.../query_p0/intersect/test_intersect.groovy | 2 +-
.../suites/query_p0/join/test_join2.groovy | 2 +-
.../suites/query_p0/join/test_left_join1.groovy | 2 +-
.../join/test_nestedloop_outer_join.groovy | 2 +-
.../join/test_partitioned_hash_join.groovy | 2 +-
.../query_p0/lateral_view/lateral_view.groovy | 2 +-
.../query_p0/limit/OffsetInSubqueryWithJoin.groovy | 2 +-
.../query_p0/literal_view/lietral_test.groovy | 2 +-
.../query_p0/operator/test_set_operator.groovy | 2 +-
.../session_variable/test_default_limit.groovy | 2 +-
.../query_p0/show/test_show_create_table.groovy | 2 +-
.../test_aggregate_all_functions.groovy | 2 +-
.../case_function/test_case_function_null.groovy | 2 +-
.../hash_functions/test_hash_function.groovy | 2 +-
.../ip_functions/test_ip_functions.groovy | 2 +-
.../json_function/test_query_json_insert.groovy | 2 +-
.../json_functions/test_json_function.groovy | 2 +-
.../sql_functions/math_functions/test_conv.groovy | 2 +-
.../test_multi_string_search.groovy | 2 +-
.../spatial_functions/test_gis_function.groovy | 2 +-
.../string_functions/test_string_function.groovy | 2 +-
.../table_function/explode_split.groovy | 2 +-
.../sql_functions/test_alias_function.groovy | 2 +-
.../query_p0/sql_functions/test_in_expr.groovy | 2 +-
.../query_p0/sql_functions/test_predicate.groovy | 2 +-
.../test_width_bucket_function.groovy | 2 +-
.../window_functions/test_window_fn.groovy | 2 +-
.../suites/query_p0/subquery/test_subquery2.groovy | 2 +-
.../suites/query_p0/test_data_type_marks.groovy | 2 +-
.../suites/query_p0/test_dict_with_null.groovy | 2 +-
.../query_p0/test_orderby_nullliteral.groovy | 2 +-
.../suites/query_p0/test_select_constant.groovy | 2 +-
.../test_select_with_predicate_like.groovy | 2 +-
.../test_select_with_predicate_prune.groovy | 2 +-
.../test_two_phase_read_with_having.groovy | 2 +-
.../query_p0/type_inference/test_largeint.groovy | 2 +-
.../with/test_with_and_two_phase_agg.groovy | 2 +-
61 files changed, 298 insertions(+), 88 deletions(-)
diff --git a/be/src/common/status.h b/be/src/common/status.h
index e7a48c5a055..72f3ffa6dcd 100644
--- a/be/src/common/status.h
+++ b/be/src/common/status.h
@@ -447,6 +447,7 @@ public:
void to_protobuf(PStatus* status) const;
std::string to_string() const;
+ std::string to_string_no_stack() const;
/// @return A json representation of this status.
std::string to_json() const;
@@ -519,6 +520,13 @@ inline std::string Status::to_string() const {
return ss.str();
}
+inline std::string Status::to_string_no_stack() const {
+ std::stringstream ss;
+ ss << '[' << code_as_string() << ']';
+ ss << msg();
+ return ss.str();
+}
+
// some generally useful macros
#define RETURN_IF_ERROR(stmt) \
do { \
diff --git a/be/src/util/arrow/row_batch.cpp b/be/src/util/arrow/row_batch.cpp
index 46c397e3f66..a010b58b722 100644
--- a/be/src/util/arrow/row_batch.cpp
+++ b/be/src/util/arrow/row_batch.cpp
@@ -48,6 +48,9 @@ using strings::Substitute;
Status convert_to_arrow_type(const TypeDescriptor& type,
std::shared_ptr<arrow::DataType>* result) {
switch (type.type) {
+ case TYPE_NULL:
+ *result = arrow::null();
+ break;
case TYPE_TINYINT:
*result = arrow::int8();
break;
@@ -79,6 +82,7 @@ Status convert_to_arrow_type(const TypeDescriptor& type,
std::shared_ptr<arrow::
case TYPE_DATETIMEV2:
case TYPE_STRING:
case TYPE_JSONB:
+ case TYPE_OBJECT:
*result = arrow::utf8();
break;
case TYPE_DECIMALV2:
@@ -95,6 +99,9 @@ Status convert_to_arrow_type(const TypeDescriptor& type,
std::shared_ptr<arrow::
case TYPE_IPV6:
*result = arrow::utf8();
break;
+ case TYPE_DECIMAL256:
+ *result = std::make_shared<arrow::Decimal256Type>(type.precision,
type.scale);
+ break;
case TYPE_BOOLEAN:
*result = arrow::boolean();
break;
@@ -131,7 +138,8 @@ Status convert_to_arrow_type(const TypeDescriptor& type,
std::shared_ptr<arrow::
break;
}
default:
- return Status::InvalidArgument("Unknown primitive type({})",
type.type);
+ return Status::InvalidArgument("Unknown primitive type({}) convert to
Arrow type",
+ type.type);
}
return Status::OK();
}
diff --git a/be/src/util/arrow/utils.cpp b/be/src/util/arrow/utils.cpp
index 85268174c63..5ccff849034 100644
--- a/be/src/util/arrow/utils.cpp
+++ b/be/src/util/arrow/utils.cpp
@@ -36,7 +36,10 @@ arrow::Status to_arrow_status(const Status& status) {
if (status.ok()) {
return arrow::Status::OK();
} else {
- return arrow::Status::Invalid(status.to_string());
+ // The length of exception msg returned to the ADBC Client cannot
larger than 8192,
+ // otherwise ADBC Client will receive:
+ // `INTERNAL: http2 exception Header size exceeded max allowed size
(8192)`.
+ return arrow::Status::Invalid(status.to_string_no_stack());
}
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
index a383c6526d9..99178ce4e96 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java
@@ -1417,11 +1417,14 @@ public class StmtExecutor {
}
// handle selects that fe can do without be, so we can make sql tools
happy, especially the setup step.
- Optional<ResultSet> resultSet = planner.handleQueryInFe(parsedStmt);
- if (resultSet.isPresent()) {
- sendResultSet(resultSet.get());
- LOG.info("Query {} finished", DebugUtil.printId(context.queryId));
- return;
+ // TODO FE not support doris field type conversion to arrow field type.
+ if (!context.getConnectType().equals(ConnectType.ARROW_FLIGHT_SQL)) {
+ Optional<ResultSet> resultSet =
planner.handleQueryInFe(parsedStmt);
+ if (resultSet.isPresent()) {
+ sendResultSet(resultSet.get());
+ LOG.info("Query {} finished",
DebugUtil.printId(context.queryId));
+ return;
+ }
}
MysqlChannel channel = null;
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlProducer.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlProducer.java
index b7eda2c3ff5..20ea5836483 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlProducer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlProducer.java
@@ -88,6 +88,12 @@ import java.util.UUID;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
+/**
+ * Implementation of Arrow Flight SQL service
+ *
+ * All methods must catch all possible Exceptions, print and throw CallStatus,
+ * otherwise error message will be discarded.
+ */
public class DorisFlightSqlProducer implements FlightSqlProducer,
AutoCloseable {
private static final Logger LOG =
LogManager.getLogger(DorisFlightSqlProducer.class);
private final Location location;
@@ -175,9 +181,9 @@ public class DorisFlightSqlProducer implements
FlightSqlProducer, AutoCloseable
private FlightInfo executeQueryStatement(String peerIdentity,
ConnectContext connectContext, String query,
final FlightDescriptor descriptor) {
- Preconditions.checkState(null != connectContext);
- Preconditions.checkState(!query.isEmpty());
try {
+ Preconditions.checkState(null != connectContext);
+ Preconditions.checkState(!query.isEmpty());
// After the previous query was executed, there was no
getStreamStatement to take away the result.
connectContext.getFlightSqlChannel().reset();
final FlightSqlConnectProcessor flightSQLConnectProcessor = new
FlightSqlConnectProcessor(connectContext);
@@ -294,6 +300,9 @@ public class DorisFlightSqlProducer implements
FlightSqlProducer, AutoCloseable
public void createPreparedStatement(final
ActionCreatePreparedStatementRequest request, final CallContext context,
final StreamListener<Result> listener) {
// TODO can only execute complete SQL, not support SQL parameters.
+ // For Python: the Python code will try to create a prepared statement
(this is to fit DBAPI, IIRC) and
+ // if the server raises any error except for NotImplemented it will
fail. (If it gets NotImplemented,
+ // it will ignore and execute without a prepared statement.) see:
https://github.com/apache/arrow/issues/38786
executorService.submit(() -> {
ConnectContext connectContext =
flightSessionsManager.getConnectContext(context.peerIdentity());
try {
@@ -344,20 +353,27 @@ public class DorisFlightSqlProducer implements
FlightSqlProducer, AutoCloseable
public Runnable
acceptPutPreparedStatementUpdate(CommandPreparedStatementUpdate command,
CallContext context,
FlightStream flightStream, StreamListener<PutResult> ackStream) {
return () -> {
- while (flightStream.next()) {
- final VectorSchemaRoot root = flightStream.getRoot();
- final int rowCount = root.getRowCount();
- // TODO support update
- Preconditions.checkState(rowCount == 0);
-
- final int recordCount = -1;
- final DoPutUpdateResult build =
DoPutUpdateResult.newBuilder().setRecordCount(recordCount).build();
- try (final ArrowBuf buffer =
rootAllocator.buffer(build.getSerializedSize())) {
- buffer.writeBytes(build.toByteArray());
- ackStream.onNext(PutResult.metadata(buffer));
+ try {
+ while (flightStream.next()) {
+ final VectorSchemaRoot root = flightStream.getRoot();
+ final int rowCount = root.getRowCount();
+ // TODO support update
+ Preconditions.checkState(rowCount == 0);
+
+ final int recordCount = -1;
+ final DoPutUpdateResult build =
DoPutUpdateResult.newBuilder().setRecordCount(recordCount).build();
+ try (final ArrowBuf buffer =
rootAllocator.buffer(build.getSerializedSize())) {
+ buffer.writeBytes(build.toByteArray());
+ ackStream.onNext(PutResult.metadata(buffer));
+ }
}
+ ackStream.onCompleted();
+ } catch (Exception e) {
+ String errMsg = "acceptPutPreparedStatementUpdate failed, " +
e.getMessage() + ", "
+ + Util.getRootCauseMessage(e);
+ LOG.warn(errMsg, e);
+ throw
CallStatus.INTERNAL.withDescription(errMsg).withCause(e).toRuntimeException();
}
- ackStream.onCompleted();
};
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlService.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlService.java
index 08e91a15ed6..fe5a60f0cc2 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlService.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/DorisFlightSqlService.java
@@ -18,6 +18,7 @@
package org.apache.doris.service.arrowflight;
import org.apache.doris.common.Config;
+import org.apache.doris.service.FrontendOptions;
import
org.apache.doris.service.arrowflight.auth2.FlightBearerTokenAuthenticator;
import org.apache.doris.service.arrowflight.sessions.FlightSessionsManager;
import
org.apache.doris.service.arrowflight.sessions.FlightSessionsWithTokenManager;
@@ -39,13 +40,13 @@ import java.io.IOException;
public class DorisFlightSqlService {
private static final Logger LOG =
LogManager.getLogger(DorisFlightSqlService.class);
private final FlightServer flightServer;
- private volatile boolean running;
private final FlightTokenManager flightTokenManager;
private final FlightSessionsManager flightSessionsManager;
+ private volatile boolean running;
public DorisFlightSqlService(int port) {
BufferAllocator allocator = new RootAllocator();
- Location location = Location.forGrpcInsecure("0.0.0.0", port);
+ Location location =
Location.forGrpcInsecure(FrontendOptions.getLocalHostAddress(), port);
this.flightTokenManager = new
FlightTokenManagerImpl(Config.arrow_flight_token_cache_size,
Config.arrow_flight_token_alive_time);
this.flightSessionsManager = new
FlightSessionsWithTokenManager(flightTokenManager);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/results/FlightSqlChannel.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/results/FlightSqlChannel.java
index 5eeb89ba031..8ce5f72b400 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/results/FlightSqlChannel.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/results/FlightSqlChannel.java
@@ -140,7 +140,7 @@ public class FlightSqlChannel {
public void addOKResult(String queryId, String query) {
final FlightSqlResultCacheEntry flightSqlResultCacheEntry = new
FlightSqlResultCacheEntry(
- createOneOneSchemaRoot("StatusResult", "OK"), query);
+ createOneOneSchemaRoot("StatusResult", "0"), query);
resultCache.put(queryId, flightSqlResultCacheEntry);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsManager.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsManager.java
index 275bc8085dd..2a583978d39 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsManager.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsManager.java
@@ -20,13 +20,10 @@ package org.apache.doris.service.arrowflight.sessions;
import org.apache.doris.analysis.UserIdentity;
import org.apache.doris.catalog.Env;
-import org.apache.doris.common.ErrorCode;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.service.ExecuteEnv;
import org.apache.doris.system.SystemInfoService;
-import org.apache.arrow.flight.CallStatus;
-
/**
* Manages Flight User Session ConnectContext.
*/
@@ -65,10 +62,6 @@ public interface FlightSessionsManager {
connectContext.getEnv().getAuth().getInsertTimeout(connectContext.getQualifiedUser()));
connectContext.setConnectScheduler(ExecuteEnv.getInstance().getScheduler());
- if
(!ExecuteEnv.getInstance().getScheduler().registerConnection(connectContext)) {
-
connectContext.getState().setError(ErrorCode.ERR_TOO_MANY_USER_CONNECTIONS,
"Reach limit of connections");
- throw CallStatus.UNAUTHENTICATED.withDescription("Reach limit of
connections").toRuntimeException();
- }
return connectContext;
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
index fc0e7929037..26a48f0cfd2 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/service/arrowflight/sessions/FlightSessionsWithTokenManager.java
@@ -17,6 +17,7 @@
package org.apache.doris.service.arrowflight.sessions;
+import org.apache.doris.common.ErrorCode;
import org.apache.doris.common.util.Util;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.service.ExecuteEnv;
@@ -27,13 +28,17 @@ import org.apache.arrow.flight.CallStatus;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import java.util.concurrent.atomic.AtomicInteger;
+
public class FlightSessionsWithTokenManager implements FlightSessionsManager {
private static final Logger LOG =
LogManager.getLogger(FlightSessionsWithTokenManager.class);
private final FlightTokenManager flightTokenManager;
+ private final AtomicInteger nextConnectionId;
public FlightSessionsWithTokenManager(FlightTokenManager
flightTokenManager) {
this.flightTokenManager = flightTokenManager;
+ this.nextConnectionId = new AtomicInteger(0);
}
@Override
@@ -65,8 +70,16 @@ public class FlightSessionsWithTokenManager implements
FlightSessionsManager {
return null;
}
flightTokenDetails.setCreatedSession(true);
- return FlightSessionsManager.buildConnectContext(peerIdentity,
flightTokenDetails.getUserIdentity(),
- flightTokenDetails.getRemoteIp());
+ ConnectContext connectContext =
FlightSessionsManager.buildConnectContext(peerIdentity,
+ flightTokenDetails.getUserIdentity(),
flightTokenDetails.getRemoteIp());
+ connectContext.setConnectionId(nextConnectionId.getAndAdd(1));
+ connectContext.resetLoginTime();
+ if
(!ExecuteEnv.getInstance().getScheduler().registerConnection(connectContext)) {
+ connectContext.getState()
+ .setError(ErrorCode.ERR_TOO_MANY_USER_CONNECTIONS,
"Reach limit of connections");
+ throw CallStatus.UNAUTHENTICATED.withDescription("Reach limit
of connections").toRuntimeException();
+ }
+ return connectContext;
} catch (IllegalArgumentException e) {
LOG.error("Bearer token validation failed.", e);
throw CallStatus.UNAUTHENTICATED.toRuntimeException();
diff --git a/regression-test/conf/regression-conf.groovy
b/regression-test/conf/regression-conf.groovy
index e47547615a8..482403e0009 100644
--- a/regression-test/conf/regression-conf.groovy
+++ b/regression-test/conf/regression-conf.groovy
@@ -182,6 +182,12 @@ s3Endpoint = "cos.ap-hongkong.myqcloud.com"
s3BucketName = "doris-build-hk-1308700295"
s3Region = "ap-hongkong"
+//arrow flight sql test config
+extArrowFlightSqlHost = "127.0.0.1"
+extArrowFlightSqlPort = 9090
+extArrowFlightSqlUser = "root"
+extArrowFlightSqlPassword= ""
+
// iceberg rest catalog config
iceberg_rest_uri_port=18181
diff --git a/regression-test/framework/pom.xml
b/regression-test/framework/pom.xml
index 011c97b77b6..e737b1d74e9 100644
--- a/regression-test/framework/pom.xml
+++ b/regression-test/framework/pom.xml
@@ -76,7 +76,24 @@ under the License.
<groovy-eclipse-compiler.version>3.7.0</groovy-eclipse-compiler.version>
<antlr.version>4.9.3</antlr.version>
<hadoop.version>2.8.0</hadoop.version>
+ <arrow.version>15.0.0-SNAPSHOT</arrow.version>
</properties>
+ <profiles>
+ <profile>
+ <id>general-env</id>
+ <activation>
+ <property>
+ <name>!env.CUSTOM_MAVEN_REPO</name>
+ </property>
+ </activation>
+ <repositories>
+ <repository>
+ <id>arrow-apache-nightlies</id>
+ <url>https://nightlies.apache.org/arrow/java</url>
+ </repository>
+ </repositories>
+ </profile>
+ </profiles>
<build>
<plugins>
<plugin>
@@ -296,5 +313,10 @@ under the License.
<artifactId>hive-jdbc</artifactId>
<version>2.3.7</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.arrow</groupId>
+ <artifactId>flight-sql-jdbc-driver</artifactId>
+ <version>${arrow.version}</version>
+ </dependency>
</dependencies>
</project>
diff --git
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy
index a079b9e6eb7..e240264060f 100644
---
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy
+++
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/Config.groovy
@@ -535,6 +535,21 @@ class Config {
return DriverManager.getConnection(dbUrl, jdbcUser, jdbcPassword)
}
+ Connection getConnectionByArrowFlightSql(String dbName) {
+ Class.forName("org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver")
+ String arrowFlightSqlHost = otherConfigs.get("extArrowFlightSqlHost")
+ String arrowFlightSqlPort = otherConfigs.get("extArrowFlightSqlPort")
+ String arrowFlightSqlUrl =
"jdbc:arrow-flight-sql://${arrowFlightSqlHost}:${arrowFlightSqlPort}" +
+ "/?useServerPrepStmts=false&useSSL=false&useEncryption=false"
+ // TODO jdbc:arrow-flight-sql not support connect db
+ String dbUrl = buildUrlWithDbImpl(arrowFlightSqlUrl, dbName)
+ tryCreateDbIfNotExist(dbName)
+ log.info("connect to ${dbUrl}".toString())
+ String arrowFlightSqlJdbcUser =
otherConfigs.get("extArrowFlightSqlUser")
+ String arrowFlightSqlJdbcPassword =
otherConfigs.get("extArrowFlightSqlPassword")
+ return DriverManager.getConnection(dbUrl, arrowFlightSqlJdbcUser,
arrowFlightSqlJdbcPassword)
+ }
+
String getDbNameByFile(File suiteFile) {
String dir = new File(suitePath).relativePath(suiteFile.parentFile)
// We put sql files under sql dir, so dbs and tables used by cases
@@ -588,7 +603,7 @@ class Config {
log.info("Reset jdbcUrl to ${jdbcUrl}".toString())
}
- public static String buildUrlWithDb(String jdbcUrl, String dbName) {
+ public static String buildUrlWithDbImpl(String jdbcUrl, String dbName) {
String urlWithDb = jdbcUrl
String urlWithoutSchema = jdbcUrl.substring(jdbcUrl.indexOf("://") + 3)
if (urlWithoutSchema.indexOf("/") >= 0) {
@@ -605,6 +620,12 @@ class Config {
// e.g: jdbc:mysql://locahost:8080
urlWithDb += ("/" + dbName)
}
+
+ return urlWithDb
+ }
+
+ public static String buildUrlWithDb(String jdbcUrl, String dbName) {
+ String urlWithDb = buildUrlWithDbImpl(jdbcUrl, dbName);
urlWithDb = addSslUrl(urlWithDb);
urlWithDb = addTimeoutUrl(urlWithDb);
diff --git
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/ExplainAction.groovy
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/ExplainAction.groovy
index 58a979b465f..1458c78aec8 100644
---
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/ExplainAction.groovy
+++
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/ExplainAction.groovy
@@ -31,10 +31,12 @@ class ExplainAction implements SuiteAction {
private SuiteContext context
private Set<String> containsStrings = new LinkedHashSet<>()
private Set<String> notContainsStrings = new LinkedHashSet<>()
+ private String coonType
private Closure checkFunction
- ExplainAction(SuiteContext context) {
+ ExplainAction(SuiteContext context, String coonType = "JDBC") {
this.context = context
+ this.coonType = coonType
}
void sql(String sql) {
@@ -115,7 +117,11 @@ class ExplainAction implements SuiteAction {
ResultSetMetaData meta = null
try {
def temp = null
- (temp, meta) = JdbcUtils.executeToList(context.getConnection(),
explainSql)
+ if (coonType == "JDBC") {
+ (temp, meta) =
JdbcUtils.executeToList(context.getConnection(), explainSql)
+ } else if (coonType == "ARROW_FLIGHT_SQL") {
+ (temp, meta) =
JdbcUtils.executeToList(context.getArrowFlightSqlConnection(), (String) ("USE
${context.dbName};" + explainSql))
+ }
explainString = temp.stream().map({row ->
row.get(0).toString()}).collect(Collectors.joining("\n"))
return new ActionResult(explainString, null, startTime,
System.currentTimeMillis(), meta)
} catch (Throwable t) {
diff --git
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/Suite.groovy
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/Suite.groovy
index f860f20e3a3..234c50561b6 100644
---
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/Suite.groovy
+++
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/Suite.groovy
@@ -44,6 +44,7 @@ import org.slf4j.Logger
import org.slf4j.LoggerFactory
import groovy.util.logging.Slf4j
+import java.sql.Connection
import java.util.concurrent.Callable
import java.util.concurrent.Future
import java.util.concurrent.atomic.AtomicBoolean
@@ -257,24 +258,65 @@ class Suite implements GroovyInterceptable {
return context.getSyncer(this)
}
+ List<List<Object>> sql_impl(Connection conn, String sqlStr, boolean
isOrder = false) {
+ logger.info("Execute ${isOrder ? "order_" : ""}sql:
${sqlStr}".toString())
+ def (result, meta) = JdbcUtils.executeToList(conn, sqlStr)
+ if (isOrder) {
+ result = DataUtils.sortByToString(result)
+ }
+ return result
+ }
+
+ List<List<Object>> jdbc_sql(String sqlStr, boolean isOrder = false) {
+ return sql_impl(context.getConnection(), sqlStr, isOrder)
+ }
+
+ List<List<Object>> arrow_flight_sql(String sqlStr, boolean isOrder =
false) {
+ return sql_impl(context.getArrowFlightSqlConnection(), (String) ("USE
${context.dbName};" + sqlStr), isOrder)
+ }
+
List<List<Object>> sql(String sqlStr, boolean isOrder = false) {
+ if (context.useArrowFlightSql()) {
+ return arrow_flight_sql(sqlStr, isOrder)
+ } else {
+ return jdbc_sql(sqlStr, isOrder)
+ }
+ }
+
+ List<List<Object>> arrow_flight_sql_no_prepared (String sqlStr, boolean
isOrder = false){
logger.info("Execute ${isOrder ? "order_" : ""}sql:
${sqlStr}".toString())
- def (result, meta) = JdbcUtils.executeToList(context.getConnection(),
sqlStr)
+ def (result, meta) =
JdbcUtils.executeQueryToList(context.getArrowFlightSqlConnection(), (String)
("USE ${context.dbName};" + sqlStr))
if (isOrder) {
result = DataUtils.sortByToString(result)
}
return result
}
- List<List<Object>> insert_into_sql(String sqlStr, int num) {
+ List<List<Object>> insert_into_sql_impl(Connection conn, String sqlStr,
int num) {
logger.info("insert into " + num + " records")
- def (result, meta) = JdbcUtils.executeToList(context.getConnection(),
sqlStr)
+ def (result, meta) = JdbcUtils.executeToList(conn, sqlStr)
return result
}
- def sql_return_maparray(String sqlStr) {
+ List<List<Object>> jdbc_insert_into_sql(String sqlStr, int num) {
+ return insert_into_sql_impl(context.getConnection(), sqlStr, num)
+ }
+
+ List<List<Object>> arrow_flight_insert_into_sql(String sqlStr, int num) {
+ return insert_into_sql_impl(context.getArrowFlightSqlConnection(),
(String) ("USE ${context.dbName};" + sqlStr), num)
+ }
+
+ List<List<Object>> insert_into_sql(String sqlStr, int num) {
+ if (context.useArrowFlightSql()) {
+ return arrow_flight_insert_into_sql(sqlStr, num)
+ } else {
+ return jdbc_insert_into_sql(sqlStr, num)
+ }
+ }
+
+ def sql_return_maparray_impl(Connection conn, String sqlStr) {
logger.info("Execute sql: ${sqlStr}".toString())
- def (result, meta) = JdbcUtils.executeToList(context.getConnection(),
sqlStr)
+ def (result, meta) = JdbcUtils.executeToList(conn, sqlStr)
// get all column names as list
List<String> columnNames = new ArrayList<>()
@@ -294,6 +336,22 @@ class Suite implements GroovyInterceptable {
return res;
}
+ def jdbc_sql_return_maparray(String sqlStr) {
+ return sql_return_maparray_impl(context.getConnection(), sqlStr)
+ }
+
+ def arrow_flight_sql_return_maparray(String sqlStr) {
+ return sql_return_maparray_impl(context.getArrowFlightSqlConnection(),
(String) ("USE ${context.dbName};" + sqlStr))
+ }
+
+ def sql_return_maparray(String sqlStr) {
+ if (context.useArrowFlightSql()) {
+ return arrow_flight_sql_return_maparray(sqlStr)
+ } else {
+ return jdbc_sql_return_maparray(sqlStr)
+ }
+ }
+
List<List<Object>> target_sql(String sqlStr, boolean isOrder = false) {
logger.info("Execute ${isOrder ? "order_" : ""}target_sql:
${sqlStr}".toString())
def (result, meta) =
JdbcUtils.executeToList(context.getTargetConnection(this), sqlStr)
@@ -388,7 +446,11 @@ class Suite implements GroovyInterceptable {
}
void explain(Closure actionSupplier) {
- runAction(new ExplainAction(context), actionSupplier)
+ if (context.useArrowFlightSql()) {
+ runAction(new ExplainAction(context, "ARROW_FLIGHT_SQL"),
actionSupplier)
+ } else {
+ runAction(new ExplainAction(context), actionSupplier)
+ }
}
void createMV(String sql) {
@@ -608,6 +670,8 @@ class Suite implements GroovyInterceptable {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveDockerConnection(),
(PreparedStatement) arg)
}else if (tag.contains("hive_remote")) {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveRemoteConnection(),
(PreparedStatement) arg)
+ } else if (tag.contains("arrow_flight_sql") ||
context.useArrowFlightSql()) {
+ tupleResult =
JdbcUtils.executeToStringList(context.getArrowFlightSqlConnection(),
(PreparedStatement) arg)
}
else{
tupleResult =
JdbcUtils.executeToStringList(context.getConnection(), (PreparedStatement) arg)
@@ -617,6 +681,9 @@ class Suite implements GroovyInterceptable {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveDockerConnection(), (String) arg)
}else if (tag.contains("hive_remote")) {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveRemoteConnection(), (String) arg)
+ } else if (tag.contains("arrow_flight_sql") ||
context.useArrowFlightSql()) {
+ tupleResult =
JdbcUtils.executeToStringList(context.getArrowFlightSqlConnection(),
+ (String) ("USE ${context.dbName};" + (String) arg))
}
else{
tupleResult =
JdbcUtils.executeToStringList(context.getConnection(), (String) arg)
@@ -646,6 +713,8 @@ class Suite implements GroovyInterceptable {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveDockerConnection(),
(PreparedStatement) arg)
}else if (tag.contains("hive_remote")) {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveRemoteConnection(),
(PreparedStatement) arg)
+ } else if (tag.contains("arrow_flight_sql") ||
context.useArrowFlightSql()) {
+ tupleResult =
JdbcUtils.executeToStringList(context.getArrowFlightSqlConnection(),
(PreparedStatement) arg)
}
else{
tupleResult =
JdbcUtils.executeToStringList(context.getConnection(), (PreparedStatement) arg)
@@ -655,6 +724,9 @@ class Suite implements GroovyInterceptable {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveDockerConnection(), (String) arg)
}else if (tag.contains("hive_remote")) {
tupleResult =
JdbcUtils.executeToStringList(context.getHiveRemoteConnection(), (String) arg)
+ } else if (tag.contains("arrow_flight_sql") ||
context.useArrowFlightSql()) {
+ tupleResult =
JdbcUtils.executeToStringList(context.getArrowFlightSqlConnection(),
+ (String) ("USE ${context.dbName};" + (String) arg))
}
else{
tupleResult =
JdbcUtils.executeToStringList(context.getConnection(), (String) arg)
diff --git
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/SuiteContext.groovy
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/SuiteContext.groovy
index 31eed478399..cdbdf2df2d3 100644
---
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/SuiteContext.groovy
+++
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/suite/SuiteContext.groovy
@@ -43,6 +43,7 @@ class SuiteContext implements Closeable {
public final String group
public final String dbName
public final ThreadLocal<ConnectionInfo> threadLocalConn = new
ThreadLocal<>()
+ public final ThreadLocal<ConnectionInfo> threadArrowFlightSqlConn = new
ThreadLocal<>()
public final ThreadLocal<Connection> threadHiveDockerConn = new
ThreadLocal<>()
public final ThreadLocal<Connection> threadHiveRemoteConn = new
ThreadLocal<>()
private final ThreadLocal<Syncer> syncer = new ThreadLocal<>()
@@ -128,6 +129,14 @@ class SuiteContext implements Closeable {
return getConnection()
}
+ boolean useArrowFlightSql() {
+ if (group.contains("arrow_flight_sql") &&
config.groups.contains("arrow_flight_sql")) {
+ return true
+ }
+ return false
+ }
+
+ // jdbc:mysql
Connection getConnection() {
def threadConnInfo = threadLocalConn.get()
if (threadConnInfo == null) {
@@ -140,6 +149,19 @@ class SuiteContext implements Closeable {
return threadConnInfo.conn
}
+ Connection getArrowFlightSqlConnection(){
+ def threadConnInfo = threadArrowFlightSqlConn.get()
+ if (threadConnInfo == null) {
+ threadConnInfo = new ConnectionInfo()
+ threadConnInfo.conn = config.getConnectionByArrowFlightSql(dbName)
+ threadConnInfo.username = config.jdbcUser
+ threadConnInfo.password = config.jdbcPassword
+ threadArrowFlightSqlConn.set(threadConnInfo)
+ }
+ println("Use arrow flight sql connection")
+ return threadConnInfo.conn
+ }
+
Connection getHiveDockerConnection(){
def threadConn = threadHiveDockerConn.get()
if (threadConn == null) {
@@ -371,7 +393,17 @@ class SuiteContext implements Closeable {
log.warn("Close connection failed", t)
}
}
-
+
+ ConnectionInfo arrow_flight_sql_conn = threadArrowFlightSqlConn.get()
+ if (arrow_flight_sql_conn != null) {
+ threadArrowFlightSqlConn.remove()
+ try {
+ arrow_flight_sql_conn.conn.close()
+ } catch (Throwable t) {
+ log.warn("Close connection failed", t)
+ }
+ }
+
Connection hive_docker_conn = threadHiveDockerConn.get()
if (hive_docker_conn != null) {
threadHiveDockerConn.remove()
diff --git
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/util/JdbcUtils.groovy
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/util/JdbcUtils.groovy
index 8791dd289b3..bc02f08f80c 100644
---
a/regression-test/framework/src/main/groovy/org/apache/doris/regression/util/JdbcUtils.groovy
+++
b/regression-test/framework/src/main/groovy/org/apache/doris/regression/util/JdbcUtils.groovy
@@ -37,6 +37,12 @@ class JdbcUtils {
}
}
+ static Tuple2<List<List<Object>>, ResultSetMetaData>
executeQueryToList(Connection conn, String sql) {
+ conn.createStatement().withCloseable { stmt ->
+ return toList(stmt.executeQuery(sql))
+ }
+ }
+
static PreparedStatement prepareStatement(Connection conn, String sql) {
return conn.prepareStatement(sql);
}
diff --git a/regression-test/suites/query_p0/aggregate/aggregate_count1.groovy
b/regression-test/suites/query_p0/aggregate/aggregate_count1.groovy
index cf657cc8ef3..3971f304e38 100644
--- a/regression-test/suites/query_p0/aggregate/aggregate_count1.groovy
+++ b/regression-test/suites/query_p0/aggregate/aggregate_count1.groovy
@@ -17,7 +17,7 @@
* under the License.
*/
-suite("aggregate_count1", "query") {
+suite("aggregate_count1", "query,arrow_flight_sql") {
sql """ DROP TABLE IF EXISTS aggregate_count1 """
sql """create table if not exists aggregate_count1 (
name varchar(128),
diff --git a/regression-test/suites/query_p0/aggregate/select_distinct.groovy
b/regression-test/suites/query_p0/aggregate/select_distinct.groovy
index 6456158bdad..2d6a8679d87 100644
--- a/regression-test/suites/query_p0/aggregate/select_distinct.groovy
+++ b/regression-test/suites/query_p0/aggregate/select_distinct.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("select_distinct") {
+suite("select_distinct", "arrow_flight_sql") {
sql """DROP TABLE IF EXISTS decimal_a;"""
sql """DROP TABLE IF EXISTS decimal_b;"""
sql """DROP TABLE IF EXISTS decimal_c;"""
diff --git
a/regression-test/suites/query_p0/casesensetive_column/join_with_column_casesensetive.groovy
b/regression-test/suites/query_p0/casesensetive_column/join_with_column_casesensetive.groovy
index a5d378f976f..49fe9c50448 100644
---
a/regression-test/suites/query_p0/casesensetive_column/join_with_column_casesensetive.groovy
+++
b/regression-test/suites/query_p0/casesensetive_column/join_with_column_casesensetive.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("join_with_column_casesensetive") {
+suite("join_with_column_casesensetive", "arrow_flight_sql") {
def tables=["ad_order_data_v1","ad_order_data"]
for (String table in tables) {
diff --git a/regression-test/suites/query_p0/cast/test_cast.groovy
b/regression-test/suites/query_p0/cast/test_cast.groovy
index a7ce1c41fa2..c0df52b39e7 100644
--- a/regression-test/suites/query_p0/cast/test_cast.groovy
+++ b/regression-test/suites/query_p0/cast/test_cast.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite('test_cast') {
+suite('test_cast', "arrow_flight_sql") {
def date = "date '2020-01-01'"
def datev2 = "datev2 '2020-01-01'"
def datetime = "timestamp '2020-01-01 12:34:45'"
diff --git a/regression-test/suites/query_p0/except/test_query_except.groovy
b/regression-test/suites/query_p0/except/test_query_except.groovy
index ff8d3e68008..a13fd76e7a9 100644
--- a/regression-test/suites/query_p0/except/test_query_except.groovy
+++ b/regression-test/suites/query_p0/except/test_query_except.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_query_except") {
+suite("test_query_except", "arrow_flight_sql") {
// test query except, depend on query_test_data_load.groovy
sql "use test_query_db"
qt_select_except1 """
diff --git
a/regression-test/suites/query_p0/group_concat/test_group_concat.groovy
b/regression-test/suites/query_p0/group_concat/test_group_concat.groovy
index 2247cc4e5fc..8bacf756642 100644
--- a/regression-test/suites/query_p0/group_concat/test_group_concat.groovy
+++ b/regression-test/suites/query_p0/group_concat/test_group_concat.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_group_concat") {
+suite("test_group_concat", "query,p0,arrow_flight_sql") {
qt_select """
SELECT group_concat(k6) FROM test_query_db.test where
k6='false'
"""
diff --git
a/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
b/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
index 1f12de6628a..f8180b0ab43 100644
--- a/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
+++ b/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_grouping_sets1") {
+suite("test_grouping_sets1", "arrow_flight_sql") {
qt_select """
select
col1
diff --git a/regression-test/suites/query_p0/having/having.groovy
b/regression-test/suites/query_p0/having/having.groovy
index e54bc3f9a68..d2e4bfc2ead 100644
--- a/regression-test/suites/query_p0/having/having.groovy
+++ b/regression-test/suites/query_p0/having/having.groovy
@@ -19,7 +19,7 @@
//
/testing/trino-product-tests/src/main/resources/sql-tests/testcases/aggregate
// and modified by Doris.
-suite("having") {
+suite("having", "query,p0,arrow_flight_sql") {
sql "set enable_nereids_planner=false;"
sql """DROP TABLE IF EXISTS supplier"""
sql """CREATE TABLE `supplier` (
diff --git a/regression-test/suites/query_p0/intersect/test_intersect.groovy
b/regression-test/suites/query_p0/intersect/test_intersect.groovy
index 1c007b95d7d..7919bec324b 100644
--- a/regression-test/suites/query_p0/intersect/test_intersect.groovy
+++ b/regression-test/suites/query_p0/intersect/test_intersect.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_intersect") {
+suite("test_intersect", "arrow_flight_sql") {
qt_select """
SELECT * FROM (SELECT k1 FROM test_query_db.baseall
INTERSECT SELECT k1 FROM test_query_db.test) a ORDER BY k1
diff --git a/regression-test/suites/query_p0/join/test_join2.groovy
b/regression-test/suites/query_p0/join/test_join2.groovy
index 7c621d15db4..b64d40995a4 100644
--- a/regression-test/suites/query_p0/join/test_join2.groovy
+++ b/regression-test/suites/query_p0/join/test_join2.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_join2", "query,p0") {
+suite("test_join2", "query,p0,arrow_flight_sql") {
def DBname = "regression_test_join2"
def TBname1 = "J1_TBL"
def TBname2 = "J2_TBL"
diff --git a/regression-test/suites/query_p0/join/test_left_join1.groovy
b/regression-test/suites/query_p0/join/test_left_join1.groovy
index d4cbeeee65e..104adab4a85 100644
--- a/regression-test/suites/query_p0/join/test_left_join1.groovy
+++ b/regression-test/suites/query_p0/join/test_left_join1.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_left_join1", "query,p0") {
+suite("test_left_join1", "query,p0,arrow_flight_sql") {
def tableName = "test_left_join1"
sql """drop table if exists ${tableName}"""
diff --git
a/regression-test/suites/query_p0/join/test_nestedloop_outer_join.groovy
b/regression-test/suites/query_p0/join/test_nestedloop_outer_join.groovy
index ad19e554690..f99dfa04244 100644
--- a/regression-test/suites/query_p0/join/test_nestedloop_outer_join.groovy
+++ b/regression-test/suites/query_p0/join/test_nestedloop_outer_join.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_nestedloop_outer_join", "query_p0") {
+suite("test_nestedloop_outer_join", "query_p0,arrow_flight_sql") {
def tbl1 = "test_nestedloop_outer_join1"
def tbl2 = "test_nestedloop_outer_join2"
diff --git
a/regression-test/suites/query_p0/join/test_partitioned_hash_join.groovy
b/regression-test/suites/query_p0/join/test_partitioned_hash_join.groovy
index cbe09ec527f..676cdd06274 100644
--- a/regression-test/suites/query_p0/join/test_partitioned_hash_join.groovy
+++ b/regression-test/suites/query_p0/join/test_partitioned_hash_join.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_partitioned_hash_join", "query,p0") {
+suite("test_partitioned_hash_join", "query,p0,arrow_flight_sql") {
sql "drop table if exists test_partitioned_hash_join_l"
sql "drop table if exists test_partitioned_hash_join_r"
sql """ create table test_partitioned_hash_join_l (
diff --git a/regression-test/suites/query_p0/lateral_view/lateral_view.groovy
b/regression-test/suites/query_p0/lateral_view/lateral_view.groovy
index 6027922cdef..a301213d349 100644
--- a/regression-test/suites/query_p0/lateral_view/lateral_view.groovy
+++ b/regression-test/suites/query_p0/lateral_view/lateral_view.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("lateral_view") {
+suite("lateral_view", "arrow_flight_sql") {
sql """ DROP TABLE IF EXISTS `test_explode_bitmap` """
sql """
CREATE TABLE `test_explode_bitmap` (
diff --git
a/regression-test/suites/query_p0/limit/OffsetInSubqueryWithJoin.groovy
b/regression-test/suites/query_p0/limit/OffsetInSubqueryWithJoin.groovy
index da0c7231f42..caa75ac7be3 100644
--- a/regression-test/suites/query_p0/limit/OffsetInSubqueryWithJoin.groovy
+++ b/regression-test/suites/query_p0/limit/OffsetInSubqueryWithJoin.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_offset_in_subquery_with_join", "query") {
+suite("test_offset_in_subquery_with_join", "query,arrow_flight_sql") {
// define a sql table
def testTable = "test_offset_in_subquery_with_join"
diff --git a/regression-test/suites/query_p0/literal_view/lietral_test.groovy
b/regression-test/suites/query_p0/literal_view/lietral_test.groovy
index b19f33e2939..5e0695fbe55 100644
--- a/regression-test/suites/query_p0/literal_view/lietral_test.groovy
+++ b/regression-test/suites/query_p0/literal_view/lietral_test.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("literal_view_test") {
+suite("literal_view_test", "arrow_flight_sql") {
sql """DROP TABLE IF EXISTS table1"""
diff --git a/regression-test/suites/query_p0/operator/test_set_operator.groovy
b/regression-test/suites/query_p0/operator/test_set_operator.groovy
index 8a9512e9770..ac091a9d8bb 100644
--- a/regression-test/suites/query_p0/operator/test_set_operator.groovy
+++ b/regression-test/suites/query_p0/operator/test_set_operator.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_set_operators", "query,p0") {
+suite("test_set_operators", "query,p0,arrow_flight_sql") {
sql """
DROP TABLE IF EXISTS t1;
diff --git
a/regression-test/suites/query_p0/session_variable/test_default_limit.groovy
b/regression-test/suites/query_p0/session_variable/test_default_limit.groovy
index 162eda19541..ad28f98aa39 100644
--- a/regression-test/suites/query_p0/session_variable/test_default_limit.groovy
+++ b/regression-test/suites/query_p0/session_variable/test_default_limit.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite('test_default_limit') {
+suite('test_default_limit', "arrow_flight_sql") {
sql 'drop table if exists baseall'
sql 'drop table if exists bigtable'
diff --git a/regression-test/suites/query_p0/show/test_show_create_table.groovy
b/regression-test/suites/query_p0/show/test_show_create_table.groovy
index 6325cbe319f..1e3fc7ff5cb 100644
--- a/regression-test/suites/query_p0/show/test_show_create_table.groovy
+++ b/regression-test/suites/query_p0/show/test_show_create_table.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_show_create_table", "query") {
+suite("test_show_create_table", "query,arrow_flight_sql") {
String tb_name = "tb_show_create_table";
try {
sql """drop table if exists ${tb_name} """
diff --git
a/regression-test/suites/query_p0/sql_functions/aggregate_functions/test_aggregate_all_functions.groovy
b/regression-test/suites/query_p0/sql_functions/aggregate_functions/test_aggregate_all_functions.groovy
index dbc6a998a5c..5748fac0894 100644
---
a/regression-test/suites/query_p0/sql_functions/aggregate_functions/test_aggregate_all_functions.groovy
+++
b/regression-test/suites/query_p0/sql_functions/aggregate_functions/test_aggregate_all_functions.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_aggregate_all_functions") {
+suite("test_aggregate_all_functions", "arrow_flight_sql") {
sql "set batch_size = 4096"
diff --git
a/regression-test/suites/query_p0/sql_functions/case_function/test_case_function_null.groovy
b/regression-test/suites/query_p0/sql_functions/case_function/test_case_function_null.groovy
index 269a0bf0db8..5138db6e73b 100644
---
a/regression-test/suites/query_p0/sql_functions/case_function/test_case_function_null.groovy
+++
b/regression-test/suites/query_p0/sql_functions/case_function/test_case_function_null.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_case_function_null", "query,p0") {
+suite("test_case_function_null", "query,p0,arrow_flight_sql") {
sql """ drop table if exists case_null0 """
sql """ create table case_null0 (
`c0` decimalv3(17, 1) NULL,
diff --git
a/regression-test/suites/query_p0/sql_functions/hash_functions/test_hash_function.groovy
b/regression-test/suites/query_p0/sql_functions/hash_functions/test_hash_function.groovy
index 4c7b925a8a0..d44518509da 100644
---
a/regression-test/suites/query_p0/sql_functions/hash_functions/test_hash_function.groovy
+++
b/regression-test/suites/query_p0/sql_functions/hash_functions/test_hash_function.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_hash_function") {
+suite("test_hash_function", "arrow_flight_sql") {
sql "set batch_size = 4096;"
sql "set enable_profile = true;"
diff --git
a/regression-test/suites/query_p0/sql_functions/ip_functions/test_ip_functions.groovy
b/regression-test/suites/query_p0/sql_functions/ip_functions/test_ip_functions.groovy
index 6991da94c26..60ba7d3c076 100644
---
a/regression-test/suites/query_p0/sql_functions/ip_functions/test_ip_functions.groovy
+++
b/regression-test/suites/query_p0/sql_functions/ip_functions/test_ip_functions.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_ip_functions") {
+suite("test_ip_functions", "arrow_flight_sql") {
sql "set batch_size = 4096;"
qt_sql "SELECT ipv4numtostring(-1);"
diff --git
a/regression-test/suites/query_p0/sql_functions/json_function/test_query_json_insert.groovy
b/regression-test/suites/query_p0/sql_functions/json_function/test_query_json_insert.groovy
index c885e3ae343..b5865034538 100644
---
a/regression-test/suites/query_p0/sql_functions/json_function/test_query_json_insert.groovy
+++
b/regression-test/suites/query_p0/sql_functions/json_function/test_query_json_insert.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_query_json_insert", "query") {
+suite("test_query_json_insert", "query,arrow_flight_sql") {
qt_sql "select json_insert('{\"a\": 1, \"b\": [2, 3]}', '\$', null);"
qt_sql "select json_insert('{\"k\": [1, 2]}', '\$.k[0]', null, '\$.[1]',
null);"
def tableName = "test_query_json_insert"
diff --git
a/regression-test/suites/query_p0/sql_functions/json_functions/test_json_function.groovy
b/regression-test/suites/query_p0/sql_functions/json_functions/test_json_function.groovy
index 474097e5047..0bbbdb9c018 100644
---
a/regression-test/suites/query_p0/sql_functions/json_functions/test_json_function.groovy
+++
b/regression-test/suites/query_p0/sql_functions/json_functions/test_json_function.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_json_function") {
+suite("test_json_function", "arrow_flight_sql") {
sql "set batch_size = 4096;"
qt_sql "SELECT get_json_double('{\"k1\":1.3, \"k2\":\"2\"}', \"\$.k1\");"
diff --git
a/regression-test/suites/query_p0/sql_functions/math_functions/test_conv.groovy
b/regression-test/suites/query_p0/sql_functions/math_functions/test_conv.groovy
index 6c4867174d1..3a74abfe9c8 100644
---
a/regression-test/suites/query_p0/sql_functions/math_functions/test_conv.groovy
+++
b/regression-test/suites/query_p0/sql_functions/math_functions/test_conv.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_conv") {
+suite("test_conv", "arrow_flight_sql") {
qt_select "SELECT CONV(15,10,2)"
sql """ drop table if exists test_conv; """
diff --git
a/regression-test/suites/query_p0/sql_functions/search_functions/test_multi_string_search.groovy
b/regression-test/suites/query_p0/sql_functions/search_functions/test_multi_string_search.groovy
index 5a3229ce361..111fcaf7173 100644
---
a/regression-test/suites/query_p0/sql_functions/search_functions/test_multi_string_search.groovy
+++
b/regression-test/suites/query_p0/sql_functions/search_functions/test_multi_string_search.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_multi_string_search") {
+suite("test_multi_string_search", "arrow_flight_sql") {
def table_name = "test_multi_string_search_strings"
sql """ DROP TABLE IF EXISTS ${table_name} """
diff --git
a/regression-test/suites/query_p0/sql_functions/spatial_functions/test_gis_function.groovy
b/regression-test/suites/query_p0/sql_functions/spatial_functions/test_gis_function.groovy
index e98e11ba7e6..f76cb44cb4a 100644
---
a/regression-test/suites/query_p0/sql_functions/spatial_functions/test_gis_function.groovy
+++
b/regression-test/suites/query_p0/sql_functions/spatial_functions/test_gis_function.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_gis_function") {
+suite("test_gis_function", "arrow_flight_sql") {
sql "set batch_size = 4096;"
qt_sql "SELECT ST_AsText(ST_Point(24.7, 56.7));"
diff --git
a/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
b/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
index 6a069923226..4f9faac47f9 100644
---
a/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
+++
b/regression-test/suites/query_p0/sql_functions/string_functions/test_string_function.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_string_function") {
+suite("test_string_function", "arrow_flight_sql") {
sql "set batch_size = 4096;"
qt_sql "select elt(0, \"hello\", \"doris\");"
diff --git
a/regression-test/suites/query_p0/sql_functions/table_function/explode_split.groovy
b/regression-test/suites/query_p0/sql_functions/table_function/explode_split.groovy
index b7dd4d64079..53db931c03b 100644
---
a/regression-test/suites/query_p0/sql_functions/table_function/explode_split.groovy
+++
b/regression-test/suites/query_p0/sql_functions/table_function/explode_split.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("explode_split") {
+suite("explode_split", "arrow_flight_sql") {
def tableName = "test_lv_str"
sql """ DROP TABLE IF EXISTS ${tableName} """
diff --git
a/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy
b/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy
index c53e2e89b55..9cf4003c27c 100644
--- a/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy
+++ b/regression-test/suites/query_p0/sql_functions/test_alias_function.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite('test_alias_function') {
+suite('test_alias_function', "arrow_flight_sql") {
sql '''
CREATE ALIAS FUNCTION IF NOT EXISTS f1(DATETIMEV2(3), INT)
with PARAMETER (datetime1, int1) as date_trunc(days_sub(datetime1,
int1), 'day')'''
diff --git a/regression-test/suites/query_p0/sql_functions/test_in_expr.groovy
b/regression-test/suites/query_p0/sql_functions/test_in_expr.groovy
index d04a9471acd..e3839a65c48 100644
--- a/regression-test/suites/query_p0/sql_functions/test_in_expr.groovy
+++ b/regression-test/suites/query_p0/sql_functions/test_in_expr.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_in_expr", "query") {
+suite("test_in_expr", "query,arrow_flight_sql") {
def nullTableName = "in_expr_test_null"
def notNullTableName = "in_expr_test_not_null"
diff --git
a/regression-test/suites/query_p0/sql_functions/test_predicate.groovy
b/regression-test/suites/query_p0/sql_functions/test_predicate.groovy
index 20b3c179ad5..6cca6b62c99 100644
--- a/regression-test/suites/query_p0/sql_functions/test_predicate.groovy
+++ b/regression-test/suites/query_p0/sql_functions/test_predicate.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_predicate") {
+suite("test_predicate", "arrow_flight_sql") {
sql """drop table if exists t1;"""
sql """
create table t1 (
diff --git
a/regression-test/suites/query_p0/sql_functions/width_bucket_fuctions/test_width_bucket_function.groovy
b/regression-test/suites/query_p0/sql_functions/width_bucket_fuctions/test_width_bucket_function.groovy
index 7ed23d697e3..c417671e4de 100644
---
a/regression-test/suites/query_p0/sql_functions/width_bucket_fuctions/test_width_bucket_function.groovy
+++
b/regression-test/suites/query_p0/sql_functions/width_bucket_fuctions/test_width_bucket_function.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_width_bucket_function") {
+suite("test_width_bucket_function", "arrow_flight_sql") {
qt_sql "select width_bucket(1, 2, 3, 2)"
qt_sql "select width_bucket(null, 2, 3, 2)"
qt_sql "select width_bucket(6, 2, 6, 4)"
diff --git
a/regression-test/suites/query_p0/sql_functions/window_functions/test_window_fn.groovy
b/regression-test/suites/query_p0/sql_functions/window_functions/test_window_fn.groovy
index 22a9e798f0a..f4b62846bca 100644
---
a/regression-test/suites/query_p0/sql_functions/window_functions/test_window_fn.groovy
+++
b/regression-test/suites/query_p0/sql_functions/window_functions/test_window_fn.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_window_fn") {
+suite("test_window_fn", "arrow_flight_sql") {
def tbName1 = "empsalary"
def tbName2 = "tenk1"
sql """ DROP TABLE IF EXISTS ${tbName1} """
diff --git a/regression-test/suites/query_p0/subquery/test_subquery2.groovy
b/regression-test/suites/query_p0/subquery/test_subquery2.groovy
index e572459cc72..a14a44fa152 100644
--- a/regression-test/suites/query_p0/subquery/test_subquery2.groovy
+++ b/regression-test/suites/query_p0/subquery/test_subquery2.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_subquery2") {
+suite("test_subquery2", "arrow_flight_sql") {
sql """DROP TABLE IF EXISTS subquerytest2"""
diff --git a/regression-test/suites/query_p0/test_data_type_marks.groovy
b/regression-test/suites/query_p0/test_data_type_marks.groovy
index ccbe727759c..f2de3d2d5b7 100644
--- a/regression-test/suites/query_p0/test_data_type_marks.groovy
+++ b/regression-test/suites/query_p0/test_data_type_marks.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_data_type_marks") {
+suite("test_data_type_marks", "arrow_flight_sql") {
def tbName = "org"
sql "DROP TABLE IF EXISTS ${tbName}"
sql """
diff --git a/regression-test/suites/query_p0/test_dict_with_null.groovy
b/regression-test/suites/query_p0/test_dict_with_null.groovy
index b3738bb68aa..83d253fa4d1 100644
--- a/regression-test/suites/query_p0/test_dict_with_null.groovy
+++ b/regression-test/suites/query_p0/test_dict_with_null.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("dict_with_null", "query") {
+suite("dict_with_null", "query,arrow_flight_sql") {
def tableName = "test_dict_with_null"
sql "DROP TABLE IF EXISTS ${tableName}"
sql """
diff --git a/regression-test/suites/query_p0/test_orderby_nullliteral.groovy
b/regression-test/suites/query_p0/test_orderby_nullliteral.groovy
index fe11c778af0..e806060c8bc 100644
--- a/regression-test/suites/query_p0/test_orderby_nullliteral.groovy
+++ b/regression-test/suites/query_p0/test_orderby_nullliteral.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("orderby_nullliteral", "query") {
+suite("orderby_nullliteral", "query,arrow_flight_sql") {
def tableName = "test_orderby_nullliteral"
sql "DROP TABLE IF EXISTS ${tableName}"
diff --git a/regression-test/suites/query_p0/test_select_constant.groovy
b/regression-test/suites/query_p0/test_select_constant.groovy
index 6015e19576c..68f0a28a20e 100644
--- a/regression-test/suites/query_p0/test_select_constant.groovy
+++ b/regression-test/suites/query_p0/test_select_constant.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_select_constant") {
+suite("test_select_constant", "arrow_flight_sql") {
qt_select1 'select 100, "test", date("2021-01-02");'
qt_select_geo1 'SELECT
ST_AsText(ST_Point(123.12345678901234567890,89.1234567890));'
}
diff --git
a/regression-test/suites/query_p0/test_select_with_predicate_like.groovy
b/regression-test/suites/query_p0/test_select_with_predicate_like.groovy
index 9491c4271ca..0d01f1b958a 100644
--- a/regression-test/suites/query_p0/test_select_with_predicate_like.groovy
+++ b/regression-test/suites/query_p0/test_select_with_predicate_like.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_select_with_predicate_like") {
+suite("test_select_with_predicate_like", "arrow_flight_sql") {
def tables=["test_basic_agg"]
for (String table in tables) {
diff --git
a/regression-test/suites/query_p0/test_select_with_predicate_prune.groovy
b/regression-test/suites/query_p0/test_select_with_predicate_prune.groovy
index 768e04b4c32..ccd1b9160fb 100644
--- a/regression-test/suites/query_p0/test_select_with_predicate_prune.groovy
+++ b/regression-test/suites/query_p0/test_select_with_predicate_prune.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_select_with_predicate_prune") {
+suite("test_select_with_predicate_prune", "arrow_flight_sql") {
sql """
drop table if exists `test_select_with_predicate_prune`;
"""
diff --git
a/regression-test/suites/query_p0/test_two_phase_read_with_having.groovy
b/regression-test/suites/query_p0/test_two_phase_read_with_having.groovy
index eda2240d4ba..d0207960db8 100644
--- a/regression-test/suites/query_p0/test_two_phase_read_with_having.groovy
+++ b/regression-test/suites/query_p0/test_two_phase_read_with_having.groovy
@@ -14,7 +14,7 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-suite("test_two_phase_read_with_having") {
+suite("test_two_phase_read_with_having", "arrow_flight_sql") {
sql """ set enable_partition_topn = 1; """
sql """ set topn_opt_limit_threshold = 1024; """
diff --git
a/regression-test/suites/query_p0/type_inference/test_largeint.groovy
b/regression-test/suites/query_p0/type_inference/test_largeint.groovy
index d5cbfa4b479..161359cfa97 100644
--- a/regression-test/suites/query_p0/type_inference/test_largeint.groovy
+++ b/regression-test/suites/query_p0/type_inference/test_largeint.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_largeint") {
+suite("test_largeint", "arrow_flight_sql") {
def tbName = "test_largeint"
sql "DROP TABLE IF EXISTS ${tbName}"
sql """
diff --git
a/regression-test/suites/query_p0/with/test_with_and_two_phase_agg.groovy
b/regression-test/suites/query_p0/with/test_with_and_two_phase_agg.groovy
index 99164a999c5..d563ef16305 100644
--- a/regression-test/suites/query_p0/with/test_with_and_two_phase_agg.groovy
+++ b/regression-test/suites/query_p0/with/test_with_and_two_phase_agg.groovy
@@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
-suite("test_with_and_two_phase_agg") {
+suite("test_with_and_two_phase_agg", "arrow_flight_sql") {
def tableName = "test_with_and_two_phase_agg_table"
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
