[doris] branch master updated: [feature](restore) Add new property 'reserve_dynamic_partition_enable' to restore statement (#12498)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new e08ba8d573 [feature](restore) Add new property
'reserve_dynamic_partition_enable' to restore statement (#12498)
e08ba8d573 is described below
commit e08ba8d5732bba73aa821d8c7fa1c44e072ab253
Author: xueweizhang
AuthorDate: Thu Oct 13 11:16:15 2022 +0800
[feature](restore) Add new property 'reserve_dynamic_partition_enable' to
restore statement (#12498)
Add restore new property 'reserve_dynamic_partition_enable', which means
you can
get a table with dynamic_partition_enable property which has the same value
as before the backup. before this commit, you always get a table with
property
'dynamic_partition_enable=false' when restore.
---
.../Backup-and-Restore/RESTORE.md | 1 +
.../Backup-and-Restore/RESTORE.md | 1 +
.../org/apache/doris/analysis/RestoreStmt.java | 19 +++
.../org/apache/doris/analysis/ShowRestoreStmt.java | 6 ++---
.../org/apache/doris/backup/BackupHandler.java | 3 ++-
.../java/org/apache/doris/backup/RestoreJob.java | 27 +-
.../java/org/apache/doris/catalog/OlapTable.java | 4 ++--
.../org/apache/doris/catalog/TableProperty.java| 6 +++--
.../org/apache/doris/backup/RestoreJobTest.java| 2 +-
.../org/apache/doris/catalog/OlapTableTest.java| 4 ++--
10 files changed, 56 insertions(+), 17 deletions(-)
diff --git
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
index 5ed19eb5d7..d176816a3c 100644
---
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
+++
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
@@ -58,6 +58,7 @@ illustrate:
- "backup_timestamp" = "2018-05-04-16-45-08": Specifies which time version
of the corresponding backup to restore, required. This information can be
obtained with the `SHOW SNAPSHOT ON repo;` statement.
- "replication_num" = "3": Specifies the number of replicas for the restored
table or partition. Default is 3. If restoring an existing table or partition,
the number of replicas must be the same as the number of replicas of the
existing table or partition. At the same time, there must be enough hosts to
accommodate multiple replicas.
- "reserve_replica" = "true": Default is false. When this property is true,
the replication_num property is ignored and the restored table or partition
will have the same number of replication as before the backup. Supports
multiple tables or multiple partitions within a table with different
replication number.
+ - "reserve_dynamic_partition_enable" = "true": Default is false. When this
property is true, the restored table will have the same value of
'dynamic_partition_enable' as before the backup. if this property is not true,
the restored table will set 'dynamic_partition_enable=false'.
- "timeout" = "3600": The task timeout period, the default is one day. in
seconds.
- "meta_version" = 40: Use the specified meta_version to read the previously
backed up metadata. Note that this parameter is used as a temporary solution
and is only used to restore the data backed up by the old version of Doris. The
latest version of the backup data already contains the meta version, no need to
specify it.
diff --git
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
index 2f912c6b92..4461c47505 100644
---
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
+++
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
@@ -57,6 +57,7 @@ PROPERTIES ("key"="value", ...);
- "backup_timestamp" = "2018-05-04-16-45-08"：指定了恢复对应备份的哪个时间版本，必填。该信息可以通过
`SHOW SNAPSHOT ON repo;` 语句获得。
- "replication_num" =
"3"：指定恢复的表或分区的副本数。默认为3。若恢复已存在的表或分区，则副本数必须和已存在表或分区的副本数相同。同时，必须有足够的 host 容纳多个副本。
- "reserve_replica" = "true"：默认为 false。当该属性为 true 时，会忽略 replication_num
属性，恢复的表或分区的副本数将与备份之前一样。支持多个表或表内多个分区有不同的副本数。
+ - "reserve_dynamic_partition_enable" = "true"：默认为 false。当该属性为 true
时，恢复的表会保留该表备份之前的'dynamic_partition_enable'属性值。该值不为true时，则恢复出来的表的'dynamic_partition_enable'属性值会设置为false。
- "
[doris-shade] branch 1.0.2-release updated: [fix](spark-load) remove comflict jar for spark_load of Doris v1.2 (#28)
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch 1.0.2-release in repository https://gitbox.apache.org/repos/asf/doris-shade.git The following commit(s) were added to refs/heads/1.0.2-release by this push: new 5d782d0 [fix](spark-load) remove comflict jar for spark_load of Doris v1.2 (#28) 5d782d0 is described below commit 5d782d0c31e7346098ac40ac63660eebb78a7112 Author: Yulei-Yang AuthorDate: Wed Oct 25 10:33:25 2023 +0800 [fix](spark-load) remove comflict jar for spark_load of Doris v1.2 (#28) --- hive-shade-3/pom.xml | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/hive-shade-3/pom.xml b/hive-shade-3/pom.xml index 7783dc1..5d8370b 100644 --- a/hive-shade-3/pom.xml +++ b/hive-shade-3/pom.xml @@ -98,6 +98,10 @@ under the License. org.apache.hadoop hadoop-yarn-server-common + +org.apache.hadoop +hadoop-yarn-api + @@ -337,4 +341,4 @@ under the License. - \ No newline at end of file + - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [branch-1.2](bug) Fix revise tablet causes replica_id is 0 in tablet meta in version 1.2 (#25728)
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/branch-1.2-lts by this push: new 2159a719bfd [branch-1.2](bug) Fix revise tablet causes replica_id is 0 in tablet meta in version 1.2 (#25728) 2159a719bfd is described below commit 2159a719bfdd00ee89c4a706f57980d37cb5428d Author: xy720 <22125576+xy...@users.noreply.github.com> AuthorDate: Wed Oct 25 14:16:01 2023 +0800 [branch-1.2](bug) Fix revise tablet causes replica_id is 0 in tablet meta in version 1.2 (#25728) --- be/src/olap/tablet_meta.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/be/src/olap/tablet_meta.cpp b/be/src/olap/tablet_meta.cpp index a7029913a8c..07448797f85 100644 --- a/be/src/olap/tablet_meta.cpp +++ b/be/src/olap/tablet_meta.cpp @@ -229,6 +229,7 @@ TabletMeta::TabletMeta(const TabletMeta& b) : _table_id(b._table_id), _partition_id(b._partition_id), _tablet_id(b._tablet_id), + _replica_id(b._replica_id), _schema_hash(b._schema_hash), _shard_id(b._shard_id), _creation_time(b._creation_time), - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [branch-1.2](TabletScheduler) Fix need further repair task stuck in running state #25991
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 40927282f67 [branch-1.2](TabletScheduler) Fix need further repair task
stuck in running state #25991
40927282f67 is described below
commit 40927282f671f1095afe9adcf89bd2e39fe0ddf5
Author: xy720 <22125576+xy...@users.noreply.github.com>
AuthorDate: Fri Oct 27 11:58:48 2023 +0800
[branch-1.2](TabletScheduler) Fix need further repair task stuck in running
state #25991
---
fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
index b889a70430a..b904654c74d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
@@ -881,7 +881,8 @@ public class TabletSchedCtx implements
Comparable {
// addReplica() method will add this replica to tablet inverted
index too.
tablet.addReplica(cloneReplica);
-} else if (tabletStatus == TabletStatus.VERSION_INCOMPLETE) {
+} else {
+// tabletStatus is VERSION_INCOMPLETE || NEED_FURTHER_REPAIR
Preconditions.checkState(type == Type.REPAIR, type);
// double check
Replica replica = tablet.getReplicaByBackendId(destBackendId);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [fix](catalog) wrong required slot info of iceberg causing BE crash (#26114)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 48f0ff975d1 [fix](catalog) wrong required slot info of iceberg causing
BE crash (#26114)
48f0ff975d1 is described below
commit 48f0ff975d16ff89b9bf4c74ed453aea4da8eddd
Author: xueweizhang
AuthorDate: Tue Oct 31 12:50:08 2023 +0800
[fix](catalog) wrong required slot info of iceberg causing BE crash (#26114)
---
.../planner/external/ExternalFileScanNode.java | 3 ++
.../planner/external/iceberg/IcebergApiSource.java | 15 ++-
.../planner/external/iceberg/IcebergHMSSource.java | 5 +++
.../external/iceberg/IcebergScanProvider.java | 4 ++
.../planner/external/iceberg/IcebergSource.java| 2 +
.../test_external_catalog_iceberg_common.out | 3 ++
.../test_external_catalog_iceberg_common.groovy| 50 ++
7 files changed, 81 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/ExternalFileScanNode.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/ExternalFileScanNode.java
index c94305233ff..124acdb71ac 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/ExternalFileScanNode.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/ExternalFileScanNode.java
@@ -355,6 +355,9 @@ public class ExternalFileScanNode extends ExternalScanNode {
this.readPartitionNum = ((HiveScanProvider)
scanProvider).getReadPartitionNum();
((HiveScanProvider) scanProvider).updateRequiredSlots(context);
}
+if (scanProvider instanceof IcebergScanProvider) {
+((IcebergScanProvider)
scanProvider).updateRequiredSlots(context);
+}
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergApiSource.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergApiSource.java
index 35b45282c50..19333a5a2b9 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergApiSource.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergApiSource.java
@@ -94,6 +94,20 @@ public class IcebergApiSource implements IcebergSource {
.map(PartitionField::name).collect(Collectors.toList());
List columns = icebergExtTable.getBaseSchema(false);
context.params.setNumOfColumnsFromFile(columns.size() -
partitionKeys.size());
+updateRequiredSlots(context);
+return context;
+}
+
+@Override
+public void updateRequiredSlots(ExternalFileScanNode.ParamCreateContext
context) throws UserException {
+updateRequiredSlots(context, null);
+}
+
+public void updateRequiredSlots(ExternalFileScanNode.ParamCreateContext
context, List partitionKeys) throws UserException {
+context.params.unsetRequiredSlots();
+if (partitionKeys == null) {
+partitionKeys =
originTable.spec().fields().stream().map(PartitionField::name).collect(Collectors.toList());
+}
for (SlotDescriptor slot : desc.getSlots()) {
if (!slot.isMaterialized()) {
continue;
@@ -103,7 +117,6 @@ public class IcebergApiSource implements IcebergSource {
slotInfo.setIsFileSlot(!partitionKeys.contains(slot.getColumn().getName()));
context.params.addToRequiredSlots(slotInfo);
}
-return context;
}
@Override
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergHMSSource.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergHMSSource.java
index 747d7fd6f67..997a037f2ba 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergHMSSource.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergHMSSource.java
@@ -68,6 +68,11 @@ public class IcebergHMSSource implements IcebergSource {
return hiveScanProvider.createContext(null);
}
+@Override
+public void updateRequiredSlots(ExternalFileScanNode.ParamCreateContext
context) throws UserException {
+hiveScanProvider.updateRequiredSlots(context);
+}
+
@Override
public TableIf getTargetTable() {
return hiveScanProvider.getTargetTable();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergScanProvider.java
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergScanProvider.java
index e8295a3fc33..d0ff1c4d1ca 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergScanProvider.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/planner/external/iceberg/IcebergScanProvider.java
@@ -263,6
(doris) branch master updated (89cf828f03e -> a5ef90dacc4)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 89cf828f03e [enhancement](udf) add the switch enable_java_udf to
control the creation and use of java_udf. (#26213)
add a5ef90dacc4 [enhancement](recover) support skipping missing version in
select by session variable (#25654)
No new revisions were added by this update.
Summary of changes:
be/src/olap/base_tablet.h | 3 +-
be/src/olap/schema_change.cpp | 2 +-
be/src/olap/tablet.cpp | 23
be/src/olap/tablet.h | 8 ++--
be/src/runtime/runtime_state.h | 4 ++
be/src/vec/exec/scan/new_olap_scan_node.cpp| 2 +-
be/src/vec/exec/scan/new_olap_scanner.cpp | 3 +-
be/test/olap/tablet_test.cpp | 4 +-
docs/en/docs/admin-manual/config/fe-config.md | 14 ---
docs/en/docs/advanced/variables.md | 4 ++
docs/zh-CN/docs/admin-manual/config/fe-config.md | 18 -
docs/zh-CN/docs/advanced/variables.md | 4 ++
.../main/java/org/apache/doris/common/Config.java | 14 ---
.../java/org/apache/doris/catalog/Replica.java | 18 +
.../main/java/org/apache/doris/catalog/Tablet.java | 4 +-
.../org/apache/doris/planner/OlapScanNode.java | 34 ++---
.../java/org/apache/doris/qe/SessionVariable.java | 17 +
gensrc/thrift/PaloInternalService.thrift | 2 +
.../test_skip_missing_version.out} | 4 +-
.../test_skip_missing_version.groovy | 43 +++---
20 files changed, 124 insertions(+), 101 deletions(-)
copy
regression-test/data/{correctness/test_constant_push_down_through_outer_join.out
=> query_p0/session_variable/test_skip_missing_version.out} (71%)
copy
fe/fe-core/src/main/java/org/apache/doris/catalog/SinglePartitionInfo.java =>
regression-test/suites/query_p0/session_variable/test_skip_missing_version.groovy
(53%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](backup) support aws_token for backup/restore (#26275)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 62723c933f5 [fix](backup) support aws_token for backup/restore (#26275)
62723c933f5 is described below
commit 62723c933f5ae7d4c9bb6017491481f78f1ffbcd
Author: xueweizhang
AuthorDate: Thu Nov 2 20:52:53 2023 +0800
[fix](backup) support aws_token for backup/restore (#26275)
---
be/src/util/s3_util.cpp | 6 ++
be/src/util/s3_util.h | 9 ++---
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/be/src/util/s3_util.cpp b/be/src/util/s3_util.cpp
index 9c938322094..25d8d7f45cc 100644
--- a/be/src/util/s3_util.cpp
+++ b/be/src/util/s3_util.cpp
@@ -139,6 +139,9 @@ std::shared_ptr
S3ClientFactory::create(const S3Conf& s3_conf
Aws::Auth::AWSCredentials aws_cred(s3_conf.ak, s3_conf.sk);
DCHECK(!aws_cred.IsExpiredOrEmpty());
+if (!s3_conf.token.empty()) {
+aws_cred.SetSessionToken(s3_conf.token);
+}
Aws::Client::ClientConfiguration aws_config =
S3ClientFactory::getClientConfiguration();
aws_config.endpointOverride = s3_conf.endpoint;
@@ -176,6 +179,9 @@ Status S3ClientFactory::convert_properties_to_s3_conf(
StringCaseMap properties(prop.begin(), prop.end());
s3_conf->ak = properties.find(S3_AK)->second;
s3_conf->sk = properties.find(S3_SK)->second;
+if (properties.find(S3_TOKEN) != properties.end()) {
+s3_conf->token = properties.find(S3_TOKEN)->second;
+}
s3_conf->endpoint = properties.find(S3_ENDPOINT)->second;
s3_conf->region = properties.find(S3_REGION)->second;
diff --git a/be/src/util/s3_util.h b/be/src/util/s3_util.h
index ec72d4d17a0..393b9d7f118 100644
--- a/be/src/util/s3_util.h
+++ b/be/src/util/s3_util.h
@@ -53,6 +53,7 @@ const static std::string S3_CONN_TIMEOUT_MS =
"AWS_CONNECTION_TIMEOUT_MS";
struct S3Conf {
std::string ak;
std::string sk;
+std::string token;
std::string endpoint;
std::string region;
std::string bucket;
@@ -64,9 +65,10 @@ struct S3Conf {
std::string to_string() const {
return fmt::format(
-"(ak={}, sk=*, endpoint={}, region={}, bucket={}, prefix={},
max_connections={}, "
-"request_timeout_ms={}, connect_timeout_ms={},
use_virtual_addressing={})",
-ak, endpoint, region, bucket, prefix, max_connections,
request_timeout_ms,
+"(ak={}, sk=*, token={}, endpoint={}, region={}, bucket={},
prefix={}, "
+"max_connections={}, request_timeout_ms={},
connect_timeout_ms={}, "
+"use_virtual_addressing={})",
+ak, token, endpoint, region, bucket, prefix, max_connections,
request_timeout_ms,
connect_timeout_ms, use_virtual_addressing);
}
@@ -74,6 +76,7 @@ struct S3Conf {
uint64_t hash_code = 0;
hash_code += Fingerprint(ak);
hash_code += Fingerprint(sk);
+hash_code += Fingerprint(token);
hash_code += Fingerprint(endpoint);
hash_code += Fingerprint(region);
hash_code += Fingerprint(bucket);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (91946dc8b08 -> 03ab822ba63)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 91946dc8b08 [minor](serde) 'null' in the serde should be represented by constants, instead of a string variable (#26301) add 03ab822ba63 [fix](backup) support aws_token for backup/restore (#26242) No new revisions were added by this update. Summary of changes: be/src/util/s3_util.cpp | 6 ++ be/src/util/s3_util.h | 9 ++--- 2 files changed, 12 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [fix](function) the result of timestampdiff() is wrong (#26154)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 11dbdceee92 [fix](function) the result of timestampdiff() is wrong
(#26154)
11dbdceee92 is described below
commit 11dbdceee92d99939ad1812173428f57e3d2851c
Author: lw112 <131352377+felixw...@users.noreply.github.com>
AuthorDate: Fri Nov 3 10:42:16 2023 +0800
[fix](function) the result of timestampdiff() is wrong (#26154)
---
be/src/vec/runtime/vdatetime_value.h | 21 +++--
1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/be/src/vec/runtime/vdatetime_value.h
b/be/src/vec/runtime/vdatetime_value.h
index 0db31dae209..0080533d3c9 100644
--- a/be/src/vec/runtime/vdatetime_value.h
+++ b/be/src/vec/runtime/vdatetime_value.h
@@ -184,6 +184,7 @@ static constexpr uint32_t MIN_YEAR = 0;
static constexpr uint32_t DATEV2_YEAR_WIDTH = 23;
static constexpr uint32_t DATETIMEV2_YEAR_WIDTH = 18;
+static constexpr uint32_t DATETIMEV2_MONTH_WIDTH = 4;
static RE2 time_zone_offset_format_reg("^[+-]{1}\\d{2}\\:\\d{2}$");
@@ -1316,8 +1317,8 @@ int64_t datetime_diff(const DateV2Value& ts_value1,
const DateV2Value& t
int month = (ts_value2.year() - ts_value1.year()) * 12 +
(ts_value2.month() - ts_value1.month());
if constexpr (std::is_same_v) {
-int shift_bits = DateV2Value::is_datetime ?
DATETIMEV2_YEAR_WIDTH + 5
- : DATEV2_YEAR_WIDTH
+ 5;
+int shift_bits = DateV2Value::is_datetime ?
DATETIMEV2_YEAR_WIDTH + DATETIMEV2_MONTH_WIDTH
+ : DATEV2_YEAR_WIDTH
+ DATETIMEV2_MONTH_WIDTH;
decltype(ts_value2.to_date_int_val()) minus_one = -1;
if (month > 0) {
month -= ((ts_value2.to_date_int_val() & (minus_one >>
shift_bits)) <
@@ -1330,23 +1331,23 @@ int64_t datetime_diff(const DateV2Value& ts_value1,
const DateV2Value& t
auto ts1_int_value = ((uint64_t)ts_value1.to_date_int_val()) <<
TIME_PART_LENGTH;
if (month > 0) {
month -= ((ts_value2.to_date_int_val() &
- (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH + 5))) <
- (ts1_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + 5;
+ (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH +
DATETIMEV2_MONTH_WIDTH))) <
+ (ts1_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + DATETIMEV2_MONTH_WIDTH;
} else if (month < 0) {
month += ((ts_value2.to_date_int_val() &
- (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH + 5))) >
- (ts1_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + 5;
+ (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH +
DATETIMEV2_MONTH_WIDTH))) >
+ (ts1_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + DATETIMEV2_MONTH_WIDTH;
}
} else {
auto ts2_int_value = ((uint64_t)ts_value2.to_date_int_val()) <<
TIME_PART_LENGTH;
if (month > 0) {
-month -= ((ts2_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + 5))) <
+month -= ((ts2_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + DATETIMEV2_MONTH_WIDTH))) <
(ts_value1.to_date_int_val() &
- (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH + 5;
+ (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH +
DATETIMEV2_MONTH_WIDTH;
} else if (month < 0) {
-month += ((ts2_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + 5))) >
+month += ((ts2_int_value & (uint64_minus_one >>
(DATETIMEV2_YEAR_WIDTH + DATETIMEV2_MONTH_WIDTH))) >
(ts_value1.to_date_int_val() &
- (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH + 5;
+ (uint64_minus_one >> (DATETIMEV2_YEAR_WIDTH +
DATETIMEV2_MONTH_WIDTH;
}
}
return month;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-2.0 updated: [Improvement](meta) add default_value column & is changed column for result of show variables stmt (#23585)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 3ed1f40080 [Improvement](meta) add default_value column & is changed
column for result of show variables stmt (#23585)
3ed1f40080 is described below
commit 3ed1f4008032e3dc718f7794cd8d994f84fc913e
Author: Yulei-Yang
AuthorDate: Tue Aug 29 10:14:12 2023 +0800
[Improvement](meta) add default_value column & is changed column for result
of show variables stmt (#23585)
---
.../main/java/org/apache/doris/analysis/ShowVariablesStmt.java | 4
.../src/main/java/org/apache/doris/catalog/SchemaTable.java | 8 ++--
fe/fe-core/src/main/java/org/apache/doris/qe/VariableMgr.java| 9 +
.../java/org/apache/doris/analysis/ShowVariablesStmtTest.java| 6 --
4 files changed, 23 insertions(+), 4 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowVariablesStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowVariablesStmt.java
index 2670a59414..dcdfc33d57 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowVariablesStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowVariablesStmt.java
@@ -32,10 +32,14 @@ public class ShowVariablesStmt extends ShowStmt {
private static final Logger LOG =
LogManager.getLogger(ShowVariablesStmt.class);
private static final String NAME_COL = "Variable_name";
private static final String VALUE_COL = "Value";
+private static final String DEFAULT_VALUE_COL = "Default_Value";
+private static final String CHANGED_COL = "Changed";
private static final ShowResultSetMetaData META_DATA =
ShowResultSetMetaData.builder()
.addColumn(new Column(NAME_COL,
ScalarType.createVarchar(20)))
.addColumn(new Column(VALUE_COL,
ScalarType.createVarchar(20)))
+.addColumn(new Column(DEFAULT_VALUE_COL,
ScalarType.createVarchar(20)))
+.addColumn(new Column(CHANGED_COL,
ScalarType.createVarchar(20)))
.build();
private SetType type;
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/SchemaTable.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/SchemaTable.java
index 6de21cf7c6..fc70eb3fb9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/SchemaTable.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/SchemaTable.java
@@ -151,11 +151,15 @@ public class SchemaTable extends Table {
.put("session_variables",
new SchemaTable(SystemIdGenerator.getNextId(),
"session_variables", TableType.SCHEMA,
builder().column("VARIABLE_NAME",
ScalarType.createVarchar(64))
-.column("VARIABLE_VALUE",
ScalarType.createVarchar(1024)).build()))
+.column("VARIABLE_VALUE",
ScalarType.createVarchar(1024))
+.column("DEFAULT_VALUE",
ScalarType.createVarchar(1024))
+.column("CHANGED",
ScalarType.createVarchar(4)).build()))
.put("global_variables",
new SchemaTable(SystemIdGenerator.getNextId(),
"global_variables", TableType.SCHEMA,
builder().column("VARIABLE_NAME",
ScalarType.createVarchar(64))
-.column("VARIABLE_VALUE",
ScalarType.createVarchar(1024)).build()))
+.column("VARIABLE_VALUE",
ScalarType.createVarchar(1024))
+.column("DEFAULT_VALUE",
ScalarType.createVarchar(1024))
+.column("CHANGED",
ScalarType.createVarchar(4)).build()))
.put("columns",
new SchemaTable(SystemIdGenerator.getNextId(), "columns",
TableType.SCHEMA,
builder().column("TABLE_CATALOG",
ScalarType.createVarchar(512))
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/VariableMgr.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/VariableMgr.java
index 853e0679e2..3588692dd2 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/VariableMgr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/VariableMgr.java
@@ -719,6 +719,15 @@ public class VariableMgr {
}
}
+VarContext varContext = ctxByVarName.get(entry.getKey());
+if (varContext != null) {
+row.add(varContext.defaultValue);
+row.add(row.get(
[doris] branch master updated (5b641ebd40 -> 0128dd42d9)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 5b641ebd40 [feature-wip](catalog) support deltalake catalog step1-metadata (#22493) add 0128dd42d9 [fix](regexp_extract_all) fix be OOM when quering with regexp_extrac… (#23284) No new revisions were added by this update. Summary of changes: be/src/vec/functions/function_regexp.cpp| 5 + .../sql_functions/string_functions/test_string_function_regexp.out | 6 ++ .../string_functions/test_string_function_regexp.groovy | 2 ++ 3 files changed, 13 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [branch-1.2](bug) fix fe schedule clone task stuck in running state #26397
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new d1ffdc76d64 [branch-1.2](bug) fix fe schedule clone task stuck in
running state #26397
d1ffdc76d64 is described below
commit d1ffdc76d641c74cb843265e6630564ff1d4be7f
Author: xy720 <22125576+xy...@users.noreply.github.com>
AuthorDate: Tue Nov 7 10:10:28 2023 +0800
[branch-1.2](bug) fix fe schedule clone task stuck in running state #26397
---
.../java/org/apache/doris/clone/TabletSchedCtx.java | 8
.../java/org/apache/doris/clone/TabletScheduler.java | 19 +++
2 files changed, 23 insertions(+), 4 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
index b904654c74d..bad60d8ec43 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletSchedCtx.java
@@ -293,6 +293,10 @@ public class TabletSchedCtx implements
Comparable {
return failedSchedCounter;
}
+public void resetFailedSchedCounter() {
+failedSchedCounter = 0;
+}
+
public void increaseFailedRunningCounter() {
++failedRunningCounter;
}
@@ -301,6 +305,10 @@ public class TabletSchedCtx implements
Comparable {
return failedRunningCounter;
}
+public boolean isExceedFailedRunningLimit() {
+return failedRunningCounter >= RUNNING_FAILED_COUNTER_THRESHOLD;
+}
+
public void setLastSchedTime(long lastSchedTime) {
this.lastSchedTime = lastSchedTime;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletScheduler.java
b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletScheduler.java
index 30d6c23d8a5..0116ac3404b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/clone/TabletScheduler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/clone/TabletScheduler.java
@@ -1530,12 +1530,23 @@ public class TabletScheduler extends MasterDaemon {
try {
tabletCtx.finishCloneTask(cloneTask, request);
} catch (SchedException e) {
-tabletCtx.increaseFailedRunningCounter();
tabletCtx.setErrMsg(e.getMessage());
if (e.getStatus() == Status.RUNNING_FAILED) {
-stat.counterCloneTaskFailed.incrementAndGet();
-addToRunningTablets(tabletCtx);
-return false;
+tabletCtx.increaseFailedRunningCounter();
+if (!tabletCtx.isExceedFailedRunningLimit()) {
+stat.counterCloneTaskFailed.incrementAndGet();
+tabletCtx.releaseResource(this);
+tabletCtx.resetFailedSchedCounter();
+tabletCtx.setState(TabletSchedCtx.State.PENDING);
+dynamicAdjustPrioAndAddBackToPendingTablets(tabletCtx,
e.getMessage());
+return false;
+} else {
+// unrecoverable
+stat.counterTabletScheduledDiscard.incrementAndGet();
+finalizeTabletCtx(tabletCtx,
TabletSchedCtx.State.CANCELLED, Status.UNRECOVERABLE,
+e.getMessage());
+return true;
+}
} else if (e.getStatus() == Status.UNRECOVERABLE) {
// unrecoverable
stat.counterTabletScheduledDiscard.incrementAndGet();
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [fix](thrift)limit be and fe thrift server max pkg size,avoid accepting error or too large package causing OOM (#26179)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new c35d2aed467 [fix](thrift)limit be and fe thrift server max pkg
size,avoid accepting error or too large package causing OOM (#26179)
c35d2aed467 is described below
commit c35d2aed4675c71aa22820ff6e65690f5869a4f8
Author: ryanzryu <143597717+ryanz...@users.noreply.github.com>
AuthorDate: Tue Nov 7 16:41:38 2023 +0800
[fix](thrift)limit be and fe thrift server max pkg size,avoid accepting
error or too large package causing OOM (#26179)
---
be/src/common/config.h | 3 +++
be/src/util/thrift_server.cpp| 5 +
docs/zh-CN/docs/admin-manual/config/be-config.md | 6 ++
docs/zh-CN/docs/admin-manual/config/fe-config.md | 12
.../src/main/java/org/apache/doris/common/Config.java| 6 ++
.../src/main/java/org/apache/doris/common/ThriftServer.java | 6 +++---
6 files changed, 35 insertions(+), 3 deletions(-)
diff --git a/be/src/common/config.h b/be/src/common/config.h
index 2cb34fd6b08..660865c36e1 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -929,6 +929,9 @@ CONF_mBool(allow_invalid_decimalv2_literal, "false");
// data page size for primary key index.
CONF_Int32(primary_key_data_page_size, "32768");
+// the max package size be thrift server can receive,avoid accepting error or
too large package causing OOM,default 20M
+CONF_Int32(be_thrift_max_pkg_bytes, "2000");
+
#ifdef BE_TEST
// test s3
CONF_String(test_s3_resource, "resource");
diff --git a/be/src/util/thrift_server.cpp b/be/src/util/thrift_server.cpp
index c3dc639eb87..835fc7eaf92 100644
--- a/be/src/util/thrift_server.cpp
+++ b/be/src/util/thrift_server.cpp
@@ -285,6 +285,11 @@ Status ThriftServer::start() {
DCHECK(!_started);
std::shared_ptr
protocol_factory(
new apache::thrift::protocol::TBinaryProtocolFactory());
+// binary_protocal_factory for setStringSizeLimit function
+std::shared_ptr
binary_protocal_factory =
+
std::dynamic_pointer_cast(
+protocol_factory);
+
binary_protocal_factory->setStringSizeLimit(config::be_thrift_max_pkg_bytes);
std::shared_ptr thread_mgr;
std::shared_ptr thread_factory
=
std::make_shared();
diff --git a/docs/zh-CN/docs/admin-manual/config/be-config.md
b/docs/zh-CN/docs/admin-manual/config/be-config.md
index 8156d2b2a05..fe147ca47c1 100644
--- a/docs/zh-CN/docs/admin-manual/config/be-config.md
+++ b/docs/zh-CN/docs/admin-manual/config/be-config.md
@@ -1446,4 +1446,10 @@ load tablets from header failed, failed tablets size:
xxx, path=xxx
* 描述: 是否在导入json数据时用simdjson来解析。
* 默认值: false
+
+ `be_thrift_max_pkg_bytes`
+
+* 描述: be节点thrift端口最大接收包大小
+* 默认值: 2000
+
diff --git a/docs/zh-CN/docs/admin-manual/config/fe-config.md
b/docs/zh-CN/docs/admin-manual/config/fe-config.md
index e54ff756229..d78ae3804c1 100644
--- a/docs/zh-CN/docs/admin-manual/config/fe-config.md
+++ b/docs/zh-CN/docs/admin-manual/config/fe-config.md
@@ -2734,3 +2734,15 @@ show data （其他用法：HELP SHOW DATA）
这个参数主要用于避免因 external catalog 无法访问、信息过多等原因导致的查询 `information_schema` 超时的问题。
+
+ `fe_thrift_max_pkg_bytes`
+
+默认值：2000
+
+是否可以动态配置：false
+
+是否为 Master FE 节点独有的配置项：false
+
+用于限制fe节点thrift端口可以接收的最大包长度，避免接收到过大或者错误的包导致OOM
+
+
diff --git a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
index edfd56c522c..2a66a609643 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/common/Config.java
@@ -2053,5 +2053,11 @@ public class Config extends ConfigBase {
@ConfField(mutable = true)
public static boolean use_mysql_bigint_for_largeint = false;
+
+/**
+* the max package size fe thrift server can receive,avoid accepting error
or too large package causing OOM,default 20M
+*/
+@ConfField
+public static int fe_thrift_max_pkg_bytes = 2000;
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
b/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
index d101f5d227b..eda2f2e4d6d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/ThriftServer.java
@@ -91,14 +91,14 @@ public class ThriftServer {
private void createSimpleServer() throws TTransportException {
TServer.Args args = new TServer.Args(new
TServerSocket(port)).protocolFactory(
-new TBinaryProtocol.Factory()).processor(processor);
+new TBinaryProtocol.Factory(Config.f
(doris) branch master updated (3cdbb6e6372 -> daea751a986)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 3cdbb6e6372 [Bug](materialized-view) fix some bugs on create mv with percentile_approx (#26528) add daea751a986 [Improvement](auditlog) add column catalog for audit log and audit log table (#26403) No new revisions were added by this update. Summary of changes: docs/en/docs/ecosystem/audit-plugin.md | 2 ++ docs/zh-CN/docs/ecosystem/audit-plugin.md | 2 ++ fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java | 7 +++ fe/fe-core/src/main/java/org/apache/doris/qe/AuditLogHelper.java | 5 + .../main/java/org/apache/doris/plugin/audit/AuditLoaderPlugin.java | 1 + .../main/java/org/apache/doris/plugin/audit/DorisStreamLoader.java | 4 ++-- pytest/deploy/start.py | 1 + 7 files changed, 20 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [Improvement](auditlog) add column catalog for audit log and audit log table (#26515)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new b1c90ea43da [Improvement](auditlog) add column catalog for audit log
and audit log table (#26515)
b1c90ea43da is described below
commit b1c90ea43daaf663ad95d384ca5ccd17bf216c2f
Author: Yulei-Yang
AuthorDate: Wed Nov 8 14:27:16 2023 +0800
[Improvement](auditlog) add column catalog for audit log and audit log
table (#26515)
---
docs/en/docs/ecosystem/audit-plugin.md | 2 ++
docs/zh-CN/docs/ecosystem/audit-plugin.md | 2 ++
fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java | 7 +++
fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java | 5 +
.../main/java/org/apache/doris/plugin/audit/AuditLoaderPlugin.java | 1 +
.../main/java/org/apache/doris/plugin/audit/DorisStreamLoader.java | 4 ++--
6 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/docs/en/docs/ecosystem/audit-plugin.md
b/docs/en/docs/ecosystem/audit-plugin.md
index d4823aa5047..4af1f2a20b0 100644
--- a/docs/en/docs/ecosystem/audit-plugin.md
+++ b/docs/en/docs/ecosystem/audit-plugin.md
@@ -66,6 +66,7 @@ create table doris_audit_db__.doris_audit_log_tbl__
`time` datetime not null comment "Query start time",
client_ip varchar(32) comment "Client IP",
user varchar(64) comment "User name",
+catalog varchar(128) comment "Catalog of this query",
db varchar(96) comment "Database of this query",
state varchar(8) comment "Query result state. EOF, ERR, OK",
error_code int comment "Error code of failing query.",
@@ -102,6 +103,7 @@ create table doris_audit_db__.doris_slow_log_tbl__
`time` datetime not null comment "Query start time",
client_ip varchar(32) comment "Client IP",
user varchar(64) comment "User name",
+catalog varchar(128) comment "Catalog of this query",
db varchar(96) comment "Database of this query",
state varchar(8) comment "Query result state. EOF, ERR, OK",
error_code int comment "Error code of failing query.",
diff --git a/docs/zh-CN/docs/ecosystem/audit-plugin.md
b/docs/zh-CN/docs/ecosystem/audit-plugin.md
index 73668f1246e..75615402d79 100644
--- a/docs/zh-CN/docs/ecosystem/audit-plugin.md
+++ b/docs/zh-CN/docs/ecosystem/audit-plugin.md
@@ -66,6 +66,7 @@ create table doris_audit_db__.doris_audit_log_tbl__
`time` datetime not null comment "Query start time",
client_ip varchar(32) comment "Client IP",
user varchar(64) comment "User name",
+catalog varchar(128) comment "Catalog of this query",
db varchar(96) comment "Database of this query",
state varchar(8) comment "Query result state. EOF, ERR, OK",
error_code int comment "Error code of failing query.",
@@ -102,6 +103,7 @@ create table doris_audit_db__.doris_slow_log_tbl__
`time` datetime not null comment "Query start time",
client_ip varchar(32) comment "Client IP",
user varchar(64) comment "User name",
+catalog varchar(128) comment "Catalog of this query",
db varchar(96) comment "Database of this query",
state varchar(8) comment "Query result state. EOF, ERR, OK",
error_code int comment "Error code of failing query.",
diff --git a/fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java
b/fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java
index 597d156ea05..0bd5c233cb0 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/plugin/AuditEvent.java
@@ -56,6 +56,8 @@ public class AuditEvent {
public String clientIp = "";
@AuditField(value = "User")
public String user = "";
+@AuditField(value = "Catalog")
+public String catalog = "";
@AuditField(value = "Db")
public String db = "";
@AuditField(value = "State")
@@ -126,6 +128,11 @@ public class AuditEvent {
return this;
}
+public AuditEventBuilder setCatalog(String catalog) {
+auditEvent.catalog = catalog;
+return this;
+}
+
public AuditEventBuilder setDb(String db) {
auditEvent.db = db;
return this;
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
index 112cbc030ac..4b99a21beb9 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
+++
[doris] branch master updated: [Improvement](external table) support hive external table which stores data on tencent chdfs (#15125)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 0fa4c78e84 [Improvement](external table) support hive external table
which stores data on tencent chdfs (#15125)
0fa4c78e84 is described below
commit 0fa4c78e84c6b4f20d2db8c6f0c2787100b1ad7b
Author: Yulei-Yang
AuthorDate: Thu Dec 22 14:32:55 2022 +0800
[Improvement](external table) support hive external table which stores data
on tencent chdfs (#15125)
---
.../src/main/java/org/apache/doris/analysis/BrokerDesc.java | 1 +
.../src/main/java/org/apache/doris/backup/BlobStorage.java| 11 +--
.../src/main/java/org/apache/doris/backup/HdfsStorage.java| 2 +-
.../org/apache/doris/catalog/HiveMetaStoreClientHelper.java | 9 +
.../src/main/java/org/apache/doris/common/FeConstants.java| 1 +
.../main/java/org/apache/doris/planner/BrokerScanNode.java| 3 ++-
.../src/main/java/org/apache/doris/planner/HiveScanNode.java | 3 +++
7 files changed, 22 insertions(+), 8 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/BrokerDesc.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/BrokerDesc.java
index d0437f13c1..bac9e80b08 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/BrokerDesc.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/BrokerDesc.java
@@ -117,6 +117,7 @@ public class BrokerDesc extends StorageDesc implements
Writable {
case STREAM:
return TFileType.FILE_STREAM;
case BROKER:
+case OFS:
default:
return TFileType.FILE_BROKER;
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/BlobStorage.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/BlobStorage.java
index eb8f0402dc..e4a3362490 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/BlobStorage.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/BlobStorage.java
@@ -50,8 +50,15 @@ public abstract class BlobStorage implements Writable {
public static BlobStorage create(String name, StorageBackend.StorageType
type, Map properties) {
if (type == StorageBackend.StorageType.S3) {
return new S3Storage(properties);
-} else if (type == StorageBackend.StorageType.HDFS) {
-return new HdfsStorage(properties);
+} else if (type == StorageBackend.StorageType.HDFS || type ==
StorageBackend.StorageType.OFS) {
+BlobStorage storage = new HdfsStorage(properties);
+// as of ofs files, use hdfs storage, but it's type should be ofs
+if (type == StorageBackend.StorageType.OFS) {
+storage.setType(type);
+storage.setName(type.name());
+}
+
+return storage;
} else if (type == StorageBackend.StorageType.BROKER) {
return new BrokerStorage(name, properties);
} else {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/HdfsStorage.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/HdfsStorage.java
index e245ad6377..be858c6648 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/HdfsStorage.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/HdfsStorage.java
@@ -568,6 +568,6 @@ public class HdfsStorage extends BlobStorage {
@Override
public StorageBackend.StorageType getStorageType() {
-return StorageBackend.StorageType.HDFS;
+return this.getType();
}
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/HiveMetaStoreClientHelper.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/HiveMetaStoreClientHelper.java
index ec097c3593..e43cecb0c7 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/HiveMetaStoreClientHelper.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/HiveMetaStoreClientHelper.java
@@ -32,7 +32,6 @@ import org.apache.doris.analysis.SlotRef;
import org.apache.doris.analysis.StorageBackend;
import org.apache.doris.analysis.StringLiteral;
import org.apache.doris.backup.BlobStorage;
-import org.apache.doris.backup.S3Storage;
import org.apache.doris.common.Config;
import org.apache.doris.common.DdlException;
import org.apache.doris.common.UserException;
@@ -208,7 +207,8 @@ public class HiveMetaStoreClientHelper {
private static String getAllFileStatus(List
fileStatuses,
List> remoteIterators,
BlobStorage storage) throws UserException {
-boolean onS3 = storage instanceof S3Storage;
+boolean needFullPath = storage.getStorageType() ==
StorageBackend.StorageType.S3
+|| storage.getStorageType() == StorageBackend.StorageType.OFS;
String hdfsUrl = "";
Queue> queue =
Queues.newArrayDeque(remoteIterators);
while
[doris] branch branch-1.2-lts updated: [bugfix](compaction) fix quick compaction core (#17378)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 736bd416cf [bugfix](compaction) fix quick compaction core (#17378)
736bd416cf is described below
commit 736bd416cf8aa7f94065b651be5697f5b63fbe71
Author: yixiutt <102007456+yixi...@users.noreply.github.com>
AuthorDate: Fri Mar 3 11:11:14 2023 +0800
[bugfix](compaction) fix quick compaction core (#17378)
---
be/src/olap/tablet.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/be/src/olap/tablet.cpp b/be/src/olap/tablet.cpp
index 2b096d5abb..4f05cacd24 100644
--- a/be/src/olap/tablet.cpp
+++ b/be/src/olap/tablet.cpp
@@ -969,7 +969,7 @@ Status
Tablet::pick_quick_compaction_rowsets(std::vector* input
quick_compaction_rowsets[idx].push_back(sortedRowset[i]);
} else {
idx++;
-if (idx > max_series_num) {
+if (idx >= max_series_num) {
break;
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](restore) fix bug when replay restore with reserve dynamic partition cause OBSERVER down (#17327)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new 02b54aeb7d [fix](restore) fix bug when replay restore with reserve
dynamic partition cause OBSERVER down (#17327)
02b54aeb7d is described below
commit 02b54aeb7d93ae31b07d5ceb01c03b7d793bf4aa
Author: xueweizhang
AuthorDate: Fri Mar 24 16:13:51 2023 +0800
[fix](restore) fix bug when replay restore with reserve dynamic partition
cause OBSERVER down (#17327)
ehen replay restore a table with reserve_dynamic_partition_enable=true,
must registerOrRemoveDynamicPartitionTable with isReplay=true, or maybe
cause
OBSERVER can not replay restore auditlog success.
Signed-off-by: nextdreamblue
---
fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
index fd3194c08f..9fd63d4f67 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
@@ -1452,7 +1452,7 @@ public class RestoreJob extends AbstractJob {
// set all restored partition version and version hash
// set all tables' state to NORMAL
-setTableStateToNormal(db, true);
+setTableStateToNormal(db, true, isReplay);
for (long tblId : restoredVersionInfo.rowKeySet()) {
Table tbl = db.getTableNullable(tblId);
if (tbl == null) {
@@ -1620,7 +1620,7 @@ public class RestoreJob extends AbstractJob {
Database db = catalog.getDbNullable(dbId);
if (db != null) {
// rollback table's state to NORMAL
-setTableStateToNormal(db, false);
+setTableStateToNormal(db, false, isReplay);
// remove restored tbls
for (Table restoreTbl : restoredTbls) {
@@ -1697,7 +1697,7 @@ public class RestoreJob extends AbstractJob {
LOG.info("finished to cancel restore job. is replay: {}. {}",
isReplay, this);
}
-private void setTableStateToNormal(Database db, boolean committed) {
+private void setTableStateToNormal(Database db, boolean committed, boolean
isReplay) {
for (String tableName : jobInfo.backupOlapTableObjects.keySet()) {
Table tbl =
db.getTableNullable(jobInfo.getAliasByOriginNameIfSet(tableName));
if (tbl == null) {
@@ -1731,7 +1731,7 @@ public class RestoreJob extends AbstractJob {
}
if (committed && reserveDynamicPartitionEnable) {
if (DynamicPartitionUtil.isDynamicPartitionTable(tbl)) {
-
DynamicPartitionUtil.registerOrRemoveDynamicPartitionTable(db.getId(), olapTbl,
false);
+
DynamicPartitionUtil.registerOrRemoveDynamicPartitionTable(db.getId(), olapTbl,
isReplay);
catalog.getDynamicPartitionScheduler().createOrUpdateRuntimeInfo(tbl.getId(),
DynamicPartitionScheduler.LAST_UPDATE_TIME,
TimeUtils.getCurrentFormatTime());
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Enhancement](spark load)Support spark time out config (#17108)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new ce79ff947a [Enhancement](spark load)Support spark time out config
(#17108)
ce79ff947a is described below
commit ce79ff947a6d6456f4719076235b0115daa9733a
Author: liujinhui <965147...@qq.com>
AuthorDate: Thu Mar 30 20:12:46 2023 +0800
[Enhancement](spark load)Support spark time out config (#17108)
---
.../doris/load/loadv2/SparkEtlJobHandler.java | 5 -
.../doris/load/loadv2/SparkLauncherMonitor.java| 22 ++
.../load/loadv2/SparkLauncherMonitorTest.java | 7 ++-
3 files changed, 28 insertions(+), 6 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkEtlJobHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkEtlJobHandler.java
index 57f44f41d5..57f0c58af7 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkEtlJobHandler.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkEtlJobHandler.java
@@ -117,6 +117,8 @@ public class SparkEtlJobHandler {
sparkConfigs.put("spark.yarn.stage.dir", jobStageHdfsPath);
}
+LOG.info("submit etl spark job, sparkConfigs:{}", sparkConfigs);
+
try {
byte[] configData = etlJobConfig.configToJson().getBytes("UTF-8");
BrokerUtil.writeFile(configData, jobConfigHdfsPath, brokerDesc);
@@ -154,7 +156,8 @@ public class SparkEtlJobHandler {
Process process = launcher.launch();
handle.setProcess(process);
if (!FeConstants.runningUnitTest) {
-SparkLauncherMonitor.LogMonitor logMonitor =
SparkLauncherMonitor.createLogMonitor(handle);
+SparkLauncherMonitor.LogMonitor logMonitor =
+SparkLauncherMonitor.createLogMonitor(handle,
sparkConfigs);
logMonitor.setSubmitTimeoutMs(GET_APPID_TIMEOUT_MS);
logMonitor.setRedirectLogPath(logFilePath);
logMonitor.start();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkLauncherMonitor.java
b/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkLauncherMonitor.java
index 012b7ad6d5..4cf388f53b 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkLauncherMonitor.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/load/loadv2/SparkLauncherMonitor.java
@@ -20,6 +20,8 @@ package org.apache.doris.load.loadv2;
import com.google.common.base.Preconditions;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
+import org.apache.commons.collections.MapUtils;
+import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.yarn.api.records.FinalApplicationStatus;
import org.apache.hadoop.yarn.api.records.YarnApplicationState;
import org.apache.logging.log4j.LogManager;
@@ -32,14 +34,15 @@ import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.util.List;
+import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class SparkLauncherMonitor {
private static final Logger LOG =
LogManager.getLogger(SparkLauncherMonitor.class);
-public static LogMonitor createLogMonitor(SparkLoadAppHandle handle) {
-return new LogMonitor(handle);
+public static LogMonitor createLogMonitor(SparkLoadAppHandle handle,
Map resourceSparkConfig) {
+return new LogMonitor(handle, resourceSparkConfig);
}
private static SparkLoadAppHandle.State fromYarnState(YarnApplicationState
yarnState) {
@@ -80,18 +83,29 @@ public class SparkLauncherMonitor {
// 5min
private static final long DEFAULT_SUBMIT_TIMEOUT_MS = 30L;
+private static final String SUBMIT_TIMEOUT_KEY =
"spark.submit.timeout";
-public LogMonitor(SparkLoadAppHandle handle) {
+public LogMonitor(SparkLoadAppHandle handle, Map
resourceSparkConfig) {
this.handle = handle;
this.process = handle.getProcess();
this.isStop = false;
-setSubmitTimeoutMs(DEFAULT_SUBMIT_TIMEOUT_MS);
+
+if (MapUtils.isNotEmpty(resourceSparkConfig)
+&&
StringUtils.isNotEmpty(resourceSparkConfig.get(SUBMIT_TIMEOUT_KEY))) {
+
setSubmitTimeoutMs(Long.parseLong(resourceSparkConfig.get(SUBMIT_TIMEOUT_KEY)));
+} else {
+setSubmitTimeoutMs(DEFAULT_SUBMIT_TIMEOUT_MS);
+}
}
public void setSubmitTimeoutMs(long submitTimeoutMs) {
this.submitTimeoutMs = submitTimeoutMs;
}
+public long getSubmitTimeoutMs() {
+return submitTimeoutMs;
+}
+
public
[doris] branch master updated (efcc65a0d3 -> e0b20f0437)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from efcc65a0d3 [feature-wip](workload-group) Support for workload group Authentication (#20242) add e0b20f0437 [feature](function) add ip function ipv4numtostring (alias inet_ntoa) (#20936) No new revisions were added by this update. Summary of changes: be/src/vec/common/format_ip.cpp| 78 ++ be/src/vec/common/format_ip.h | 95 be/src/vec/functions/function_ip.cpp | 29 + be/src/vec/functions/function_ip.h | 120 + be/src/vec/functions/simple_function_factory.h | 2 + .../sql-functions/ip-functions/INET_NTOA.md| 70 .../ip-functions/IPV4-NUM-TO-STRING.md | 70 docs/sidebars.json | 8 ++ .../sql-functions/ip-functions/INET_NTOA.md| 71 .../ip-functions/IPV4-NUM-TO-STRING.md | 71 gensrc/script/doris_builtins_functions.py | 8 ++ .../ip_functions/test_ip_functions.out | 11 ++ .../ip_functions/test_ip_functions.groovy | 25 + 13 files changed, 658 insertions(+) create mode 100644 be/src/vec/common/format_ip.cpp create mode 100644 be/src/vec/common/format_ip.h create mode 100644 be/src/vec/functions/function_ip.cpp create mode 100644 be/src/vec/functions/function_ip.h create mode 100644 docs/en/docs/sql-manual/sql-functions/ip-functions/INET_NTOA.md create mode 100644 docs/en/docs/sql-manual/sql-functions/ip-functions/IPV4-NUM-TO-STRING.md create mode 100644 docs/zh-CN/docs/sql-manual/sql-functions/ip-functions/INET_NTOA.md create mode 100644 docs/zh-CN/docs/sql-manual/sql-functions/ip-functions/IPV4-NUM-TO-STRING.md create mode 100644 regression-test/data/query_p0/sql_functions/ip_functions/test_ip_functions.out create mode 100644 regression-test/suites/query_p0/sql_functions/ip_functions/test_ip_functions.groovy - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (1f032a551d -> 0fce7b9011)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 1f032a551d [Improve](array-functions) support array first function (#20397) add 0fce7b9011 [fix](http) Let the sdk find the httpclient package determined (#20205) No new revisions were added by this update. Summary of changes: fe/fe-core/src/main/java/org/apache/doris/fs/obj/S3ObjStorage.java | 2 ++ 1 file changed, 2 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [vectorized](function) suppoort date_trunc function truncate week mode (#18334) (#20530)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 385f79b98c [vectorized](function) suppoort date_trunc function
truncate week mode (#18334) (#20530)
385f79b98c is described below
commit 385f79b98c003414a4e631446a064fc97097545b
Author: yuxuan-luo <119841515+yuxuan-...@users.noreply.github.com>
AuthorDate: Wed Jun 7 14:44:36 2023 +0800
[vectorized](function) suppoort date_trunc function truncate week mode
(#18334) (#20530)
support date_trunc could truncate week eg:
select date_trunc('2023-4-3 19:28:30', 'week');
(cherry picked from commit 50e6c4216a26ec6a667060bf1bae6fc6dee217df)
Co-authored-by: zhangstar333
<87313068+zhangstar...@users.noreply.github.com>
---
be/src/vec/functions/function_timestamp.cpp | 2 ++
be/src/vec/runtime/vdatetime_value.cpp | 17 +
2 files changed, 19 insertions(+)
diff --git a/be/src/vec/functions/function_timestamp.cpp
b/be/src/vec/functions/function_timestamp.cpp
index 704fc8f3ff..ad8af1d676 100644
--- a/be/src/vec/functions/function_timestamp.cpp
+++ b/be/src/vec/functions/function_timestamp.cpp
@@ -298,6 +298,8 @@ struct DateTrunc {
null_map[i] = !dt.template datetime_trunc();
} else if (std::strncmp("second", str_data, 6) == 0) {
null_map[i] = !dt.template datetime_trunc();
+} else if (std::strncmp("week", str_data, 4) == 0) {
+null_map[i] = !dt.template datetime_trunc();
} else {
null_map[i] = 1;
}
diff --git a/be/src/vec/runtime/vdatetime_value.cpp
b/be/src/vec/runtime/vdatetime_value.cpp
index d5086ab6b0..a9e98b2ef8 100644
--- a/be/src/vec/runtime/vdatetime_value.cpp
+++ b/be/src/vec/runtime/vdatetime_value.cpp
@@ -1691,6 +1691,14 @@ bool VecDateTimeValue::datetime_trunc() {
_hour = 0;
break;
}
+case WEEK: {
+_second = 0;
+_minute = 0;
+_hour = 0;
+TimeInterval interval(DAY, weekday(), true);
+date_add_interval(interval);
+break;
+}
case MONTH: {
_second = 0;
_minute = 0;
@@ -2704,6 +2712,15 @@ bool DateV2Value::datetime_trunc() {
date_v2_value_.hour_ = 0;
break;
}
+case WEEK: {
+date_v2_value_.microsecond_ = 0;
+date_v2_value_.second_ = 0;
+date_v2_value_.minute_ = 0;
+date_v2_value_.hour_ = 0;
+TimeInterval interval(DAY, weekday(), true);
+date_add_interval(interval);
+break;
+}
case MONTH: {
date_v2_value_.microsecond_ = 0;
date_v2_value_.second_ = 0;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Improvement](meta) support return brief info of restore job (#20653)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new ee0e2b40da [Improvement](meta) support return brief info of restore
job (#20653)
ee0e2b40da is described below
commit ee0e2b40da2a3d698222e3e6f4c989c35820ad84
Author: Yulei-Yang
AuthorDate: Tue Jun 13 10:47:31 2023 +0800
[Improvement](meta) support return brief info of restore job (#20653)
---
.../sql-reference/Show-Statements/SHOW-RESTORE.md | 8 +-
.../sql-reference/Show-Statements/SHOW-RESTORE.md | 8 +-
fe/fe-core/src/main/cup/sql_parser.cup | 4 +++
.../org/apache/doris/analysis/ShowRestoreStmt.java | 29 --
.../java/org/apache/doris/backup/RestoreJob.java | 26 +--
.../java/org/apache/doris/qe/ShowExecutor.java | 7 +-
6 files changed, 70 insertions(+), 12 deletions(-)
diff --git
a/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
b/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
index eb31e01866..d18a3f219b 100644
--- a/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
+++ b/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
@@ -37,7 +37,7 @@ This statement is used to view RESTORE tasks
grammar:
SQL
-SHOW RESTORE [FROM DB_NAME]
+SHOW [BRIEF] RESTORE [FROM DB_NAME]
illustrate:
@@ -68,6 +68,12 @@ illustrate:
Status: If the job fails, display the failure message
Timeout: Job timeout, in seconds
+
+
+2. brief: only show key information of RESTORE tasks, columns
RestoreObjs, Progress, TaskErrMsg will not show
+
+
+
### Example
1. View the latest RESTORE task under example_db.
diff --git
a/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
b/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
index 371d9725a7..ac7c8df8c9 100644
--- a/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
+++ b/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-RESTORE.md
@@ -37,7 +37,7 @@ SHOW RESTORE
语法：
```SQL
-SHOW RESTORE [FROM DB_NAME]
+SHOW [BRIEF] RESTORE [FROM DB_NAME]
```
说明：
@@ -68,6 +68,12 @@ SHOW RESTORE [FROM DB_NAME]
Status： 如果作业失败，显示失败信息
Timeout：作业超时时间，单位秒
+
+
+2. brief: 仅返回精简格式的 RESTORE 任务信息，不包含 RestoreObjs, Progress, TaskErrMsg
三列
+
+
+
### Example
1. 查看 example_db 下最近一次 RESTORE 任务。
diff --git a/fe/fe-core/src/main/cup/sql_parser.cup
b/fe/fe-core/src/main/cup/sql_parser.cup
index b97a72f2b7..3d0f8dd392 100644
--- a/fe/fe-core/src/main/cup/sql_parser.cup
+++ b/fe/fe-core/src/main/cup/sql_parser.cup
@@ -3929,6 +3929,10 @@ show_param ::=
{:
RESULT = new ShowRestoreStmt(db, parser.where);
:}
+| KW_BRIEF KW_RESTORE opt_db:db opt_wild_where
+{:
+RESULT = new ShowRestoreStmt(db, parser.where, true);
+:}
| KW_BROKER
{:
RESULT = new ShowBrokerStmt();
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowRestoreStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowRestoreStmt.java
index 739cb504ad..3c1b34e49a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowRestoreStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowRestoreStmt.java
@@ -35,6 +35,7 @@ import org.apache.doris.qe.ShowResultSetMetaData;
import com.google.common.base.Strings;
import com.google.common.collect.ImmutableList;
+import java.util.List;
import java.util.function.Predicate;
public class ShowRestoreStmt extends ShowStmt {
@@ -45,17 +46,31 @@ public class ShowRestoreStmt extends ShowStmt {
.add("SnapshotFinishedTime").add("DownloadFinishedTime").add("FinishedTime").add("UnfinishedTasks")
.add("Progress").add("TaskErrMsg").add("Status").add("Timeout")
.build();
+public static final ImmutableList BRIEF_TITLE_NAMES = new
ImmutableList.Builder()
+
.add("JobId").add("Label").add("Timestamp").add("DbName").add("State")
+
.add("AllowLoad").add("ReplicationNum").add("ReplicaAllocation").add("ReserveReplica")
+
.add("ReserveDynamicPartitionEnable").add("CreateTime").add("MetaPreparedTime")
+
.add("SnapshotFinishedTime").add("DownloadFinishedTime").add("FinishedTime").add("UnfinishedTasks")
+.add("Status").add("Timeout")
+.build();
private String dbName;
private Expr where;
private String labelValue;
private boolean i
[doris] branch master updated: [Feature] (json)add json_contains function (#20824)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 97135a1cbb [Feature] (json)add json_contains function (#20824)
97135a1cbb is described below
commit 97135a1cbb2d3483479964515e10a0fde8fc551c
Author: yuxuan-luo <119841515+yuxuan-...@users.noreply.github.com>
AuthorDate: Fri Jun 16 15:10:12 2023 +0800
[Feature] (json)add json_contains function (#20824)
---
be/src/vec/functions/function_json.cpp | 127 +
.../sql-functions/json-functions/json_contains.md | 69 +++
.../sql-functions/json-functions/json_contains.md | 69 +++
gensrc/script/doris_builtins_functions.py | 1 +
.../json_functions/test_json_function.out | 21
.../json_functions/test_json_function.groovy | 9 ++
6 files changed, 296 insertions(+)
diff --git a/be/src/vec/functions/function_json.cpp
b/be/src/vec/functions/function_json.cpp
index e4f955ae0a..4d2f0f3614 100644
--- a/be/src/vec/functions/function_json.cpp
+++ b/be/src/vec/functions/function_json.cpp
@@ -18,6 +18,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -935,6 +936,131 @@ public:
}
};
+class FunctionJsonContains : public IFunction {
+public:
+static constexpr auto name = "json_contains";
+static FunctionPtr create() { return
std::make_shared(); }
+
+String get_name() const override { return name; }
+
+size_t get_number_of_arguments() const override { return 3; }
+
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const
override {
+return make_nullable(std::make_shared());
+}
+
+bool use_default_implementation_for_nulls() const override { return false;
}
+
+bool json_contains_object(const rapidjson::Value& target,
+ const rapidjson::Value& search_value) {
+if (!target.IsObject() || !search_value.IsObject()) {
+return false;
+}
+
+for (auto itr = search_value.MemberBegin(); itr !=
search_value.MemberEnd(); ++itr) {
+if (!target.HasMember(itr->name) ||
!json_contains(target[itr->name], itr->value)) {
+return false;
+}
+}
+
+return true;
+}
+
+bool json_contains_array(const rapidjson::Value& target, const
rapidjson::Value& search_value) {
+if (!target.IsArray() || !search_value.IsArray()) {
+return false;
+}
+
+for (auto itr = search_value.Begin(); itr != search_value.End();
++itr) {
+bool found = false;
+for (auto target_itr = target.Begin(); target_itr != target.End();
++target_itr) {
+if (json_contains(*target_itr, *itr)) {
+found = true;
+break;
+}
+}
+if (!found) {
+return false;
+}
+}
+
+return true;
+}
+
+bool json_contains(const rapidjson::Value& target, const rapidjson::Value&
search_value) {
+if (target == search_value) {
+return true;
+}
+
+if (target.IsObject() && search_value.IsObject()) {
+return json_contains_object(target, search_value);
+}
+
+if (target.IsArray() && search_value.IsArray()) {
+return json_contains_array(target, search_value);
+}
+
+return false;
+}
+
+Status execute_impl(FunctionContext* context, Block& block, const
ColumnNumbers& arguments,
+size_t result, size_t input_rows_count) override {
+const IColumn& col_json =
*(block.get_by_position(arguments[0]).column);
+const IColumn& col_search =
*(block.get_by_position(arguments[1]).column);
+const IColumn& col_path =
*(block.get_by_position(arguments[2]).column);
+
+auto null_map = ColumnUInt8::create(input_rows_count, 0);
+
+const ColumnString* col_json_string =
check_and_get_column(col_json);
+const ColumnString* col_search_string =
check_and_get_column(col_search);
+const ColumnString* col_path_string =
check_and_get_column(col_path);
+
+if (!col_json_string || !col_search_string || !col_path_string) {
+return Status::RuntimeError("Illegal column should be
ColumnString");
+}
+
+auto col_to = ColumnVector::create();
+auto& vec_to = col_to->get_data();
+size_t size = col_json.size();
+vec_to.resize(size);
+
+for (size_t i = 0; i < input_rows_count; ++i) {
+if (col_json.is_null_at(i) || col_search.is_null_at(i) ||
col_path.is_null_at(i)) {
+null_map->get_data()[i] = 1;
+
[doris] branch master updated: [fix](restore) fix bug when replay restore and reserve dynamic partition (#17326)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bada731390 [fix](restore) fix bug when replay restore and reserve
dynamic partition (#17326)
bada731390 is described below
commit bada7313907a7e17a52ca7297f482717e43bb4c5
Author: xueweizhang
AuthorDate: Tue Mar 7 10:13:08 2023 +0800
[fix](restore) fix bug when replay restore and reserve dynamic partition
(#17326)
when replay restore a table with reserve_dynamic_partition_enable=true,
must registerOrRemoveDynamicPartitionTable with isReplay=true, or maybe
cause
OBSERVER can not replay restore auditlog success.
---
fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
index 6f3a77ba66..1fd3de6352 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/backup/RestoreJob.java
@@ -1464,7 +1464,7 @@ public class RestoreJob extends AbstractJob {
// set all restored partition version and version hash
// set all tables' state to NORMAL
-setTableStateToNormal(db, true);
+setTableStateToNormal(db, true, isReplay);
for (long tblId : restoredVersionInfo.rowKeySet()) {
Table tbl = db.getTableNullable(tblId);
if (tbl == null) {
@@ -1632,7 +1632,7 @@ public class RestoreJob extends AbstractJob {
Database db = env.getInternalCatalog().getDbNullable(dbId);
if (db != null) {
// rollback table's state to NORMAL
-setTableStateToNormal(db, false);
+setTableStateToNormal(db, false, isReplay);
// remove restored tbls
for (Table restoreTbl : restoredTbls) {
@@ -1709,7 +1709,7 @@ public class RestoreJob extends AbstractJob {
LOG.info("finished to cancel restore job. is replay: {}. {}",
isReplay, this);
}
-private void setTableStateToNormal(Database db, boolean committed) {
+private void setTableStateToNormal(Database db, boolean committed, boolean
isReplay) {
for (String tableName : jobInfo.backupOlapTableObjects.keySet()) {
Table tbl =
db.getTableNullable(jobInfo.getAliasByOriginNameIfSet(tableName));
if (tbl == null) {
@@ -1743,7 +1743,7 @@ public class RestoreJob extends AbstractJob {
}
if (committed && reserveDynamicPartitionEnable) {
if (DynamicPartitionUtil.isDynamicPartitionTable(tbl)) {
-
DynamicPartitionUtil.registerOrRemoveDynamicPartitionTable(db.getId(), olapTbl,
false);
+
DynamicPartitionUtil.registerOrRemoveDynamicPartitionTable(db.getId(), olapTbl,
isReplay);
Env.getCurrentEnv().getDynamicPartitionScheduler().createOrUpdateRuntimeInfo(tbl.getId(),
DynamicPartitionScheduler.LAST_UPDATE_TIME,
TimeUtils.getCurrentFormatTime());
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [Feature](add bitmap udaf) add the bitmap intersection and difference set for mixed calculation of udaf (#15588) (#19778)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 00bd7afb93 [Feature](add bitmap udaf) add the bitmap intersection and
difference set for mixed calculation of udaf (#15588) (#19778)
00bd7afb93 is described below
commit 00bd7afb93e128950418f3d721aa4c2e9d014601
Author: yuxuan-luo <119841515+yuxuan-...@users.noreply.github.com>
AuthorDate: Thu May 18 14:59:26 2023 +0800
[Feature](add bitmap udaf) add the bitmap intersection and difference set
for mixed calculation of udaf (#15588) (#19778)
Co-authored-by: zhbinbin <16679214+zhbin...@users.noreply.github.com>
Co-authored-by: zhangbinbin05
(cherry picked from commit ff9e03e2bf8ca5b4cdd6fe44b129db868e84ddb4)
---
be/src/util/bitmap_expr_calculation.h | 216 +
be/src/util/bitmap_intersect.h | 5 +-
.../aggregate_function_orthogonal_bitmap.cpp | 19 ++
.../aggregate_function_orthogonal_bitmap.h | 97 +
.../apache/doris/analysis/FunctionCallExpr.java| 4 +-
.../apache/doris/catalog/AggregateFunction.java| 1 +
.../java/org/apache/doris/catalog/FunctionSet.java | 43
7 files changed, 381 insertions(+), 4 deletions(-)
diff --git a/be/src/util/bitmap_expr_calculation.h
b/be/src/util/bitmap_expr_calculation.h
new file mode 100644
index 00..24784cc957
--- /dev/null
+++ b/be/src/util/bitmap_expr_calculation.h
@@ -0,0 +1,216 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+#pragma once
+#include
+#include
+
+#include "util/bitmap_intersect.h"
+
+namespace doris {
+
+// Compute the intersection union difference set of two or more bitmaps
+// Usage: orthogonal_bitmap_parse_calculate(bitmap_column, filter_column,
input_string)
+// Example: orthogonal_bitmap_expr_calculate(user_id, event, '(A|B)&(C-D)'),
meaning find the intersection union difference set of user_id in all A/B/C/D 4
bitmaps
+// Operation symbol:
+// the operator '|' stands for union, the operator '&' stands for
intersection, the operator '-' indicates the difference set, the operator '^'
stands for xor
+class BitmapExprCalculation : public BitmapIntersect {
+public:
+BitmapExprCalculation() = default;
+
+explicit BitmapExprCalculation(const char* src) { deserialize(src); }
+
+void bitmap_calculation_init(std::string& input_str) {
+_polish = reverse_polish(input_str);
+std::string bitmap_key;
+for (int i = 0; i < _polish.length(); i++) {
+char c = _polish.at(i);
+if (c != '&' && c != '|' && c != '^' && c != '-' && c != ' ' && c
!= '\\') {
+bitmap_key += c;
+} else if (i != 0 && _polish.at(i - 1) == '\\') {
+bitmap_key += c;
+} else if (c == '\\') {
+continue;
+} else {
+if (bitmap_key.length() > 0) {
+add_key(bitmap_key);
+bitmap_key.clear();
+}
+}
+}
+if (bitmap_key.length() > 0) {
+add_key(bitmap_key);
+bitmap_key.clear();
+}
+}
+
+BitmapValue bitmap_calculate() {
+std::stack values;
+std::string bitmap_key;
+for (int i = 0; i < _polish.length(); i++) {
+char c = _polish.at(i);
+if (c == ' ') {
+if (bitmap_key.length() > 0) {
+values.push(_bitmaps[bitmap_key]);
+bitmap_key.clear();
+}
+} else if (c != '&' && c != '|' && c != '^' && c != '-' && c !=
'\\') {
+bitmap_key += c;
+} else if (i != 0 &&am
[doris] branch master updated (1ef85ae1f2 -> 45a3bb87c4)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 1ef85ae1f2 [Improvement](join) Support nested loop outer join (#13965) add 45a3bb87c4 [docs](recover) modify recover doc (#13904) No new revisions were added by this update. Summary of changes: .../Database-Administration-Statements/RECOVER.md | 86 -- .../Database-Administration-Statements/RECOVER.md | 86 -- 2 files changed, 161 insertions(+), 11 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Enhancement](restore) new add the property of reserve_replica to restore statement (#11942)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 53b79d5a8c [Enhancement](restore) new add the property of
reserve_replica to restore statement (#11942)
53b79d5a8c is described below
commit 53b79d5a8c6981c11e7b16ac7432dd357f2d6fc9
Author: xueweizhang
AuthorDate: Tue Sep 6 10:32:21 2022 +0800
[Enhancement](restore) new add the property of reserve_replica to restore
statement (#11942)
Add a new property called 'reserve_replica', which means you can
get a table with same partitions with the same replication num
as before the backup.
Co-authored-by: Stalary
Co-authored-by: camby <104178...@qq.com>
---
.../Backup-and-Restore/RESTORE.md | 1 +
.../Backup-and-Restore/RESTORE.md | 1 +
.../org/apache/doris/analysis/RestoreStmt.java | 18
.../org/apache/doris/analysis/ShowRestoreStmt.java | 2 +-
.../org/apache/doris/backup/BackupHandler.java | 2 +-
.../java/org/apache/doris/backup/RestoreJob.java | 51 ++
.../java/org/apache/doris/catalog/OlapTable.java | 17 ++--
.../org/apache/doris/backup/RestoreJobTest.java| 2 +-
8 files changed, 79 insertions(+), 15 deletions(-)
diff --git
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
index 5d54c6c24a..5ed19eb5d7 100644
---
a/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
+++
b/docs/en/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
@@ -57,6 +57,7 @@ illustrate:
- PROPERTIES currently supports the following properties:
- "backup_timestamp" = "2018-05-04-16-45-08": Specifies which time version
of the corresponding backup to restore, required. This information can be
obtained with the `SHOW SNAPSHOT ON repo;` statement.
- "replication_num" = "3": Specifies the number of replicas for the restored
table or partition. Default is 3. If restoring an existing table or partition,
the number of replicas must be the same as the number of replicas of the
existing table or partition. At the same time, there must be enough hosts to
accommodate multiple replicas.
+ - "reserve_replica" = "true": Default is false. When this property is true,
the replication_num property is ignored and the restored table or partition
will have the same number of replication as before the backup. Supports
multiple tables or multiple partitions within a table with different
replication number.
- "timeout" = "3600": The task timeout period, the default is one day. in
seconds.
- "meta_version" = 40: Use the specified meta_version to read the previously
backed up metadata. Note that this parameter is used as a temporary solution
and is only used to restore the data backed up by the old version of Doris. The
latest version of the backup data already contains the meta version, no need to
specify it.
diff --git
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
index c9ef66be9a..2f912c6b92 100644
---
a/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
+++
b/docs/zh-CN/docs/sql-manual/sql-reference/Data-Definition-Statements/Backup-and-Restore/RESTORE.md
@@ -56,6 +56,7 @@ PROPERTIES ("key"="value", ...);
- PROPERTIES 目前支持以下属性：
- "backup_timestamp" = "2018-05-04-16-45-08"：指定了恢复对应备份的哪个时间版本，必填。该信息可以通过
`SHOW SNAPSHOT ON repo;` 语句获得。
- "replication_num" =
"3"：指定恢复的表或分区的副本数。默认为3。若恢复已存在的表或分区，则副本数必须和已存在表或分区的副本数相同。同时，必须有足够的 host 容纳多个副本。
+ - "reserve_replica" = "true"：默认为 false。当该属性为 true 时，会忽略 replication_num
属性，恢复的表或分区的副本数将与备份之前一样。支持多个表或表内多个分区有不同的副本数。
- "timeout" = "3600"：任务超时时间，默认为一天。单位秒。
- "meta_version" = 40：使用指定的 meta_version 来读取之前备份的元数据。注意，该参数作为临时方案，仅用于恢复老版本
Doris 备份的数据。最新版本的备份数据中已经包含 meta version，无需再指定。
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
index cb9b4e3bb4..a98e6f2891 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/RestoreStmt.java
@@ -36,11 +36,13 @@ public class RestoreStmt extends AbstractBackupStmt {
private static final String PROP_REPLICATION_NUM = "replication_num
(doris) branch branch-2.0 updated: [branch-2.0](prepared statement) Handle unsigned numeric type in prepared statement in legacy planner (#36376)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 59d8403c239 [branch-2.0](prepared statement) Handle unsigned numeric
type in prepared statement in legacy planner (#36376)
59d8403c239 is described below
commit 59d8403c239eddd6b55d3b2809ffe7e9f37047c8
Author: xy720 <22125576+xy...@users.noreply.github.com>
AuthorDate: Tue Jun 18 19:05:21 2024 +0800
[branch-2.0](prepared statement) Handle unsigned numeric type in prepared
statement in legacy planner (#36376)
---
.../org/apache/doris/analysis/DateLiteral.java | 2 +-
.../org/apache/doris/analysis/DecimalLiteral.java | 2 +-
.../org/apache/doris/analysis/FloatLiteral.java| 2 +-
.../java/org/apache/doris/analysis/IntLiteral.java | 9 +-
.../org/apache/doris/analysis/LargeIntLiteral.java | 5 ++
.../org/apache/doris/analysis/LiteralExpr.java | 97 +++---
.../org/apache/doris/analysis/PlaceHolderExpr.java | 25 +-
.../org/apache/doris/analysis/PrepareStmt.java | 1 +
.../org/apache/doris/analysis/StringLiteral.java | 2 +-
.../apache/doris/common/util/ByteBufferUtil.java | 34
.../java/org/apache/doris/planner/ScanNode.java| 4 +-
.../java/org/apache/doris/qe/ConnectProcessor.java | 3 +-
.../doris/rewrite/RewriteInPredicateRule.java | 7 +-
regression-test/data/insert_p0/prepare_insert.out | 1 +
.../suites/insert_p0/prepare_insert.groovy | 17 +++-
.../suites/prepared_stmt_p0/prepared_stmt.groovy | 2 +-
16 files changed, 149 insertions(+), 64 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
index 9baa583f115..a4908e37b2e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
@@ -1893,7 +1893,7 @@ public class DateLiteral extends LiteralExpr {
}
@Override
-public void setupParamFromBinary(ByteBuffer data) {
+public void setupParamFromBinary(ByteBuffer data, boolean isUnsigned) {
int len = getParmLen(data);
if (type.getPrimitiveType() == PrimitiveType.DATE) {
if (len >= 4) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
index 3a3a8f8f66a..f568cadd1d7 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
@@ -408,7 +408,7 @@ public class DecimalLiteral extends LiteralExpr {
}
@Override
-public void setupParamFromBinary(ByteBuffer data) {
+public void setupParamFromBinary(ByteBuffer data, boolean isUnsigned) {
int len = getParmLen(data);
BigDecimal v = null;
try {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
index bbcfafb2890..55925821bda 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
@@ -271,7 +271,7 @@ public class FloatLiteral extends LiteralExpr {
}
@Override
-public void setupParamFromBinary(ByteBuffer data) {
+public void setupParamFromBinary(ByteBuffer data, boolean isUnsigned) {
if (type.getPrimitiveType() == PrimitiveType.FLOAT) {
value = data.getFloat();
return;
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
index 88fe549a047..dfefae8f7db 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
@@ -21,6 +21,7 @@ import org.apache.doris.catalog.PrimitiveType;
import org.apache.doris.catalog.Type;
import org.apache.doris.common.AnalysisException;
import org.apache.doris.common.NotImplementedException;
+import org.apache.doris.common.util.ByteBufferUtil;
import org.apache.doris.qe.ConnectContext;
import org.apache.doris.thrift.TExprNode;
import org.apache.doris.thrift.TExprNodeType;
@@ -364,19 +365,19 @@ public class IntLiteral extends LiteralExpr {
}
@Override
-public void setupParamFromBinary(ByteBuffer data) {
+public void setupParamFromBinary(ByteBuffer data, boolean isUnsigned) {
switch (type.getPrimitiveType()) {
case TINYINT:
value = data.get();
break;
case SMALLINT:
-value = data.getChar();
+value = !isUnsigned ? dat
(doris) branch branch-2.0 updated: [fix](planner) fix no data issue when use datetimev1/datatimev2 & datev2 as function coalesce's parameter in legacy planner (#36583)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new bb9d532e653 [fix](planner) fix no data issue when use
datetimev1/datatimev2 & datev2 as function coalesce's parameter in legacy
planner (#36583)
bb9d532e653 is described below
commit bb9d532e653a131e7a3883062f1bfb9ab382623a
Author: Yulei-Yang
AuthorDate: Thu Jun 20 15:22:54 2024 +0800
[fix](planner) fix no data issue when use datetimev1/datatimev2 & datev2 as
function coalesce's parameter in legacy planner (#36583)
---
gensrc/script/doris_builtins_functions.py | 2 +
.../conditional_functions/test_coalesce_new.groovy | 101 +
2 files changed, 103 insertions(+)
diff --git a/gensrc/script/doris_builtins_functions.py
b/gensrc/script/doris_builtins_functions.py
index 9fa38174be1..ea09c5473c1 100644
--- a/gensrc/script/doris_builtins_functions.py
+++ b/gensrc/script/doris_builtins_functions.py
@@ -1452,8 +1452,10 @@ visible_functions = {
[['coalesce'], 'FLOAT', ['FLOAT', '...'], 'CUSTOM'],
[['coalesce'], 'DOUBLE', ['DOUBLE', '...'], 'CUSTOM'],
[['coalesce'], 'DATETIME', ['DATETIME', '...'], 'CUSTOM'],
+[['coalesce'], 'DATETIMEV2', ['DATETIME', 'DATEV2', '...'], 'CUSTOM'],
[['coalesce'], 'DATE', ['DATE', '...'], 'CUSTOM'],
[['coalesce'], 'DATETIMEV2', ['DATETIMEV2', '...'], 'CUSTOM'],
+[['coalesce'], 'DATETIMEV2', ['DATETIMEV2', 'DATEV2', '...'],
'CUSTOM'],
[['coalesce'], 'DATEV2', ['DATEV2', '...'], 'CUSTOM'],
[['coalesce'], 'DECIMALV2', ['DECIMALV2', '...'], 'CUSTOM'],
[['coalesce'], 'DECIMAL32', ['DECIMAL32', '...'], 'CUSTOM'],
diff --git
a/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
new file mode 100644
index 000..194849a3c63
--- /dev/null
+++
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
@@ -0,0 +1,101 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_coalesce_new") {
+// test parameter:datetime, datev2
+sql """
+admin set frontend config ("enable_date_conversion"="false")
+"""
+sql """
+admin set frontend config ("disable_datev1"="false")
+"""
+sql """
+drop table if exists test_cls
+"""
+
+sql """
+CREATE TABLE `test_cls` (
+`id` int(11) NOT NULL COMMENT '',
+`name` varchar(32) NOT NULL COMMENT '',
+`dt` datetime NOT NULL
+) ENGINE=OLAP
+UNIQUE KEY(`id`)
+DISTRIBUTED BY HASH(`id`) BUCKETS 2
+PROPERTIES(
+"replication_allocation" = "tag.location.default: 1"
+);
+"""
+
+sql """
+insert into test_cls values (1,'Alice','2023-06-01
12:00:00'),(2,'Bob','2023-06-02 12:00:00'),(3,'Carl','2023-05-01 14:00:00')
+"""
+
+sql """
+SET enable_nereids_planner=false
+"""
+def result1 = try_sql """
+
(doris) branch branch-2.0 updated: [fix](planner) fix result wrong caused by mixed use multiple columns with type of datetime/datetimev2/date/datev2 of function coalesce (#36640)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 6d9492da62c [fix](planner) fix result wrong caused by mixed use
multiple columns with type of datetime/datetimev2/date/datev2 of function
coalesce (#36640)
6d9492da62c is described below
commit 6d9492da62ca0dce2043312d439faf62503c7db8
Author: Yulei-Yang
AuthorDate: Fri Jun 21 16:21:45 2024 +0800
[fix](planner) fix result wrong caused by mixed use multiple columns with
type of datetime/datetimev2/date/datev2 of function coalesce (#36640)
---
.../org/apache/doris/analysis/FunctionCallExpr.java | 8
.../conditional_functions/test_coalesce_new.groovy | 20
2 files changed, 28 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
index 1a52c965818..3f92c959670 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
@@ -1557,6 +1557,14 @@ public class FunctionCallExpr extends Expr {
argTypes[i] = assignmentCompatibleType;
}
}
+} else if (assignmentCompatibleType.isDateV2OrDateTimeV2()) {
+for (int i = 0; i < childTypes.length; i++) {
+if (assignmentCompatibleType.isDateV2OrDateTimeV2()
+&&
!childTypes[i].equals(assignmentCompatibleType)) {
+uncheckedCastChild(assignmentCompatibleType, i);
+argTypes[i] = assignmentCompatibleType;
+}
+}
}
fn = getBuiltinFunction(fnName.getFunction(), argTypes,
Function.CompareMode.IS_NONSTRICT_SUPERTYPE_OF);
diff --git
a/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
index 194849a3c63..a223c5d4cbd 100644
---
a/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
+++
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
@@ -51,6 +51,14 @@ suite("test_coalesce_new") {
select dt from test_cls where coalesce (dt,
str_to_date(concat('202306', '01'), '%Y%m%d')) >= '2023-06-01'
"""
assertEquals(result1.size(), 2);
+def result11 = try_sql """
+select dt from test_cls where coalesce (dt, dt,
str_to_date(concat('202306', '01'), '%Y%m%d')) >= '2023-06-01'
+"""
+assertEquals(result11.size(), 2);
+def result12 = try_sql """
+select dt from test_cls where coalesce (dt,
str_to_date(concat('202306', '01'), '%Y%m%d'), str_to_date(concat('202306',
'01'), '%Y%m%d')) >= '2023-06-01'
+"""
+assertEquals(result12.size(), 2);
// test parameter:datetimev2, datev2
@@ -88,6 +96,15 @@ suite("test_coalesce_new") {
select dt from test_cls_dtv2 where coalesce (dt,
str_to_date(concat('202306', '01'), '%Y%m%d')) >= '2023-06-01'
"""
assertEquals(result2.size(), 2);
+def result21 = try_sql """
+select dt from test_cls_dtv2 where coalesce (dt, dt,
str_to_date(concat('202306', '01'), '%Y%m%d')) >= '2023-06-01'
+"""
+assertEquals(result21.size(), 2);
+def result22 = try_sql """
+select dt from test_cls_dtv2 where coalesce (dt,
str_to_date(concat('202306', '01'), '%Y%m%d'), str_to_date(concat('202306',
'01'), '%Y%m%d')) >= '2023-06-01'
+"""
+assertEquals(result22.size(), 2);
+
sql """
drop table test_cls
@@ -98,4 +115,7 @@ suite("test_coalesce_new") {
sql """
admin set frontend config ("disable_datev1"="false")
"""
+sql """
+admin set frontend config ("enable_date_conversion"="true")
+"""
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [fix](nereids) fix wrong resut of function ifnull/coalesce caused by mixed use multiple columns with type of datetime/datetimev2/date/datev2 of function coalesce whe
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new bff66079a1b [fix](nereids) fix wrong resut of function ifnull/coalesce
caused by mixed use multiple columns with type of
datetime/datetimev2/date/datev2 of function coalesce when enable nereids
(#36688)
bff66079a1b is described below
commit bff66079a1bde8a6f6764d0800054d1267f0d204
Author: Yulei-Yang
AuthorDate: Sat Jun 22 07:52:58 2024 +0800
[fix](nereids) fix wrong resut of function ifnull/coalesce caused by mixed
use multiple columns with type of datetime/datetimev2/date/datev2 of function
coalesce when enable nereids (#36688)
---
.../expressions/functions/SearchSignature.java | 5 ++
.../conditional_functions/test_coalesce_new.groovy | 65 ++
2 files changed, 70 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
index 496b51d9caf..a3fb3083078 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
@@ -23,6 +23,8 @@ import org.apache.doris.nereids.exceptions.AnalysisException;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.literal.Literal;
import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.nereids.types.DateTimeType;
+import org.apache.doris.nereids.types.DateTimeV2Type;
import org.apache.doris.nereids.types.DateType;
import org.apache.doris.nereids.types.DateV2Type;
import org.apache.doris.nereids.types.DecimalV3Type;
@@ -195,6 +197,9 @@ public class SearchSignature {
nonStrictMatched++;
if (sigArgType instanceof DateV2Type && realType instanceof
DateType) {
dateToDateV2Count++;
+} else if (sigArgType instanceof DateTimeV2Type && (realType
instanceof DateTimeType
+|| realType instanceof DateV2Type || realType
instanceof DateType)) {
+dateToDateV2Count++;
}
}
}
diff --git
a/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
index a223c5d4cbd..cf19442a210 100644
---
a/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
+++
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
@@ -60,6 +60,40 @@ suite("test_coalesce_new") {
"""
assertEquals(result12.size(), 2);
+//test enable_date_conversion=true and enable_nereids
+sql """
+admin set frontend config ("enable_date_conversion"="true")
+"""
+sql """
+SET enable_nereids_planner=true
+"""
+sql """
+SET enable_fallback_to_original_planner=false
+"""
+def result13 = try_sql """
+select dt from test_cls where coalesce (dt,
str_to_date(concat('202306', '01'), '%Y%m%d')) >= '2023-06-01'
+"""
+assertEquals(result13.size(), 2);
+def result14 = try_sql """
+select dt from test_cls where coalesce (dt, dt,
str_to_date(concat('202306', '01'), '%Y%m%d')) < '2023-06-01'
+"""
+assertEquals(result14.size(), 1);
+def result15 = try_sql """
+select dt from test_cls where coalesce (dt,
str_to_date(concat('202306', '01'), '%Y%m%d'), str_to_date(concat('202306',
'01'), '%Y%m%d')) >= '2023-06-02'
+"""
+assertEquals(result15.size(), 1);
+def result16 = try_sql """
+select dt from test_cls where ifnull(dt, str_to_date(concat('202306',
'01'), '%Y%m%d')) >= '2023-06-01'
+"""
+assertEquals(result16.size(), 2);
+def result17 = try_sql """
+select dt from test_cls where coalesce(str_to_date(concat('202306',
'01'), '%Y%m%d'),dt) < '2023-06-02'
+"""
+assertEquals(result17.size(), 3);
+def result18 = try_sql """
+select dt from test_cls where ifnull(str_t
(doris) branch branch-2.0 updated (2971efd194f -> 087ee366de7)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch branch-2.0 in repository https://gitbox.apache.org/repos/asf/doris.git from 2971efd194f [release](version) change branch-2.0 version to 2.0.12 (#36626) add 087ee366de7 [fix] (nereids) fix nereids unable to recognize default value function in the insert statement (#36654) No new revisions were added by this update. Summary of changes: .../doris/nereids/rules/analysis/BindSink.java | 31 +++--- 1 file changed, 27 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [bug](prepared statement) fix some bugs in legacy planner (#36731)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 2a1bc567d65 [bug](prepared statement) fix some bugs in legacy planner
(#36731)
2a1bc567d65 is described below
commit 2a1bc567d65f947f0c793307e97656b04944581f
Author: xy720 <22125576+xy...@users.noreply.github.com>
AuthorDate: Mon Jun 24 16:41:25 2024 +0800
[bug](prepared statement) fix some bugs in legacy planner (#36731)
---
.../org/apache/doris/analysis/BoolLiteral.java | 3 +
.../org/apache/doris/analysis/DateLiteral.java | 3 +
.../org/apache/doris/analysis/DecimalLiteral.java | 3 +
.../org/apache/doris/analysis/FloatLiteral.java| 3 +
.../java/org/apache/doris/analysis/IntLiteral.java | 3 +
.../org/apache/doris/analysis/LargeIntLiteral.java | 3 +
.../org/apache/doris/analysis/LiteralExpr.java | 1 +
.../org/apache/doris/analysis/NullLiteral.java | 3 +
.../org/apache/doris/analysis/StringLiteral.java | 3 +
.../doris/rewrite/RewriteInPredicateRule.java | 10 +-
.../data/prepared_stmt_p0/prepared_stmt.out| 52 +++---
.../prepared_stmt_p0/prepared_stmt_in_list.out | 129 ++
.../suites/prepared_stmt_p0/prepared_stmt.groovy | 23 +++
.../prepared_stmt_p0/prepared_stmt_in_list.groovy | 194 +
14 files changed, 409 insertions(+), 24 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/BoolLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/BoolLiteral.java
index c37a620235f..2221ae1de82 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/BoolLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/BoolLiteral.java
@@ -86,6 +86,9 @@ public class BoolLiteral extends LiteralExpr {
@Override
public int compareLiteral(LiteralExpr expr) {
+if (expr instanceof PlaceHolderExpr) {
+return this.compareLiteral(((PlaceHolderExpr) expr).getLiteral());
+}
if (expr instanceof NullLiteral) {
return 1;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
index a4908e37b2e..b68f3fc83da 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DateLiteral.java
@@ -623,6 +623,9 @@ public class DateLiteral extends LiteralExpr {
@Override
public int compareLiteral(LiteralExpr expr) {
+if (expr instanceof PlaceHolderExpr) {
+return this.compareLiteral(((PlaceHolderExpr) expr).getLiteral());
+}
if (expr instanceof NullLiteral) {
return 1;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
index f568cadd1d7..afbe6c0c6cd 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/DecimalLiteral.java
@@ -240,6 +240,9 @@ public class DecimalLiteral extends LiteralExpr {
@Override
public int compareLiteral(LiteralExpr expr) {
+if (expr instanceof PlaceHolderExpr) {
+return this.compareLiteral(((PlaceHolderExpr) expr).getLiteral());
+}
if (expr instanceof NullLiteral) {
return 1;
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
index 55925821bda..2eabd21c01f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FloatLiteral.java
@@ -117,6 +117,9 @@ public class FloatLiteral extends LiteralExpr {
@Override
public int compareLiteral(LiteralExpr expr) {
+if (expr instanceof PlaceHolderExpr) {
+return this.compareLiteral(((PlaceHolderExpr) expr).getLiteral());
+}
if (expr instanceof NullLiteral) {
return 1;
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
index dfefae8f7db..25f8ab753b2 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/IntLiteral.java
@@ -238,6 +238,9 @@ public class IntLiteral extends LiteralExpr {
@Override
public int compareLiteral(LiteralExpr expr) {
+if (expr instanceof PlaceHolderExpr) {
+return this.compareLiteral(((PlaceHolderExpr) expr).getLiteral());
+}
if (expr instanceof NullLiteral) {
return 1;
(doris) branch branch-2.1 updated: [fix](planner) fix wrong resut of function ifnull/coalesce caused by … (#36727)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 8fc70e32bcb [fix](planner) fix wrong resut of function ifnull/coalesce
caused by … (#36727)
8fc70e32bcb is described below
commit 8fc70e32bcba0a6bdd6e9d5208cb9fc522dfd7aa
Author: Yulei-Yang
AuthorDate: Tue Jun 25 17:32:04 2024 +0800
[fix](planner) fix wrong resut of function ifnull/coalesce caused by …
(#36727)
---
.../apache/doris/analysis/FunctionCallExpr.java| 8 +
.../expressions/functions/SearchSignature.java | 5 +
.../conditional_functions/test_coalesce_new.groovy | 186 +
3 files changed, 199 insertions(+)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
index 4aaaf6508d8..a3ad5ef7e9b 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/FunctionCallExpr.java
@@ -1591,6 +1591,14 @@ public class FunctionCallExpr extends Expr {
argTypes[i] = assignmentCompatibleType;
}
}
+} else if (assignmentCompatibleType.isDateV2OrDateTimeV2()) {
+for (int i = 0; i < childTypes.length; i++) {
+if (assignmentCompatibleType.isDateV2OrDateTimeV2()
+&&
!childTypes[i].equals(assignmentCompatibleType)) {
+uncheckedCastChild(assignmentCompatibleType, i);
+argTypes[i] = assignmentCompatibleType;
+}
+}
}
fn = getBuiltinFunction(fnName.getFunction(), argTypes,
Function.CompareMode.IS_NONSTRICT_SUPERTYPE_OF);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
index 1898c93a3b0..4d6bb658356 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/SearchSignature.java
@@ -23,6 +23,8 @@ import org.apache.doris.nereids.exceptions.AnalysisException;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.literal.Literal;
import org.apache.doris.nereids.types.DataType;
+import org.apache.doris.nereids.types.DateTimeType;
+import org.apache.doris.nereids.types.DateTimeV2Type;
import org.apache.doris.nereids.types.DateType;
import org.apache.doris.nereids.types.DateV2Type;
import org.apache.doris.nereids.types.DecimalV3Type;
@@ -191,6 +193,9 @@ public class SearchSignature {
nonStrictMatched++;
if (sigArgType instanceof DateV2Type && realType instanceof
DateType) {
dateToDateV2Count++;
+} else if (sigArgType instanceof DateTimeV2Type && (realType
instanceof DateTimeType
+|| realType instanceof DateV2Type || realType
instanceof DateType)) {
+dateToDateV2Count++;
}
}
}
diff --git
a/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
new file mode 100644
index 000..834dcbd16b5
--- /dev/null
+++
b/regression-test/suites/query_p0/sql_functions/conditional_functions/test_coalesce_new.groovy
@@ -0,0 +1,186 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_coalesce_new") {
+// test parameter:datetime, datev2
+sql """
+admin set frontend config ("enable_date_conversion"="false")
+"""
+sql """
+
(doris) branch master updated: [Improvement](multicatalog) support read tencent dlc table on lakefs (#36823)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 507dfd8b7ed [Improvement](multicatalog) support read tencent dlc table
on lakefs (#36823)
507dfd8b7ed is described below
commit 507dfd8b7ed73d7d0cfd211fe64edde3cf916c9e
Author: Yulei-Yang
AuthorDate: Thu Jun 27 12:02:51 2024 +0800
[Improvement](multicatalog) support read tencent dlc table on lakefs
(#36823)
---
.../main/java/org/apache/doris/common/FeConstants.java| 1 +
.../java/org/apache/doris/common/util/LocationPath.java | 15 +++
.../doris/datasource/property/PropertyConverter.java | 3 ++-
.../doris/datasource/property/PropertyConverterTest.java | 2 +-
4 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
b/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
index 9567304be2f..1c24ca69d4f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
@@ -74,6 +74,7 @@ public class FeConstants {
public static final String FS_PREFIX_BOS = "bos";
public static final String FS_PREFIX_COS = "cos";
public static final String FS_PREFIX_COSN = "cosn";
+public static final String FS_PREFIX_LAKEFS = "lakefs";
public static final String FS_PREFIX_OBS = "obs";
public static final String FS_PREFIX_OFS = "ofs";
public static final String FS_PREFIX_GFS = "gfs";
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
index dd1641126bf..eccb483578a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
@@ -62,6 +62,7 @@ public class LocationPath {
COSN, // Tencent
OFS, // Tencent CHDFS
GFS, // Tencent GooseFs,
+LAKEFS, // used by Tencent DLC
OSS, // Alibaba,
OSS_HDFS, // JindoFS on OSS
JFS, // JuiceFS,
@@ -163,6 +164,10 @@ public class LocationPath {
locationType = LocationType.COSN;
this.location = location;
break;
+case FeConstants.FS_PREFIX_LAKEFS:
+locationType = LocationType.COSN;
+this.location = normalizedLakefsPath(location);
+break;
case FeConstants.FS_PREFIX_VIEWFS:
locationType = LocationType.VIEWFS;
this.location = location;
@@ -277,6 +282,15 @@ public class LocationPath {
}
}
+private static String normalizedLakefsPath(String location) {
+int atIndex = location.indexOf("@dlc");
+if (atIndex != -1) {
+return "lakefs://" + location.substring(atIndex + 1);
+} else {
+return location;
+}
+}
+
public static Pair getFSIdentity(String location,
String bindBrokerName) {
LocationPath locationPath = new LocationPath(location);
FileSystemType fsType = (bindBrokerName != null) ?
FileSystemType.BROKER : locationPath.getFileSystemType();
@@ -351,6 +365,7 @@ public class LocationPath {
case GCS:
// ATTN, for COSN, on FE side, use HadoopFS to access, but on
BE, use S3 client to access.
case COSN:
+case LAKEFS:
// now we only support S3 client for object storage on BE
return TFileType.FILE_S3;
case HDFS:
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
index fd0c8846029..9b9a92f7f32 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
@@ -195,7 +195,7 @@ public class PropertyConverter {
return OBSFileSystem.class.getName();
} else if (fsScheme.equalsIgnoreCase("oss")) {
return AliyunOSSFileSystem.class.getName();
-} else if (fsScheme.equalsIgnoreCase("cosn")) {
+} else if (fsScheme.equalsIgnoreCase("cosn") ||
fsScheme.equalsIgnoreCase("lakefs")) {
return CosFileSystem.class.getName();
} else {
return S3AFileSystem.class.getName();
@@ -361,6 +361,7 @@ public class PropertyConverter {
cosProperties.put(CosNConfigKeys.COSN_ENDPOINT_SUFFIX_KEY,
props.get(Co
(doris) branch branch-2.0 updated: [Improvement](multicatalog) support read tencent dlc on lakefs (#36807)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 17873d00871 [Improvement](multicatalog) support read tencent dlc on
lakefs (#36807)
17873d00871 is described below
commit 17873d00871c6a517a2af371b8d2b24b4f97
Author: Yulei-Yang
AuthorDate: Thu Jun 27 12:03:16 2024 +0800
[Improvement](multicatalog) support read tencent dlc on lakefs (#36807)
---
.../main/java/org/apache/doris/common/FeConstants.java| 1 +
.../java/org/apache/doris/common/util/LocationPath.java | 15 +++
.../doris/datasource/property/PropertyConverter.java | 3 ++-
.../doris/datasource/property/PropertyConverterTest.java | 2 +-
4 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
b/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
index 3012f6a62e7..1afa12856f0 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
@@ -73,6 +73,7 @@ public class FeConstants {
public static final String FS_PREFIX_BOS = "bos";
public static final String FS_PREFIX_COS = "cos";
public static final String FS_PREFIX_COSN = "cosn";
+public static final String FS_PREFIX_LAKEFS = "lakefs";
public static final String FS_PREFIX_OBS = "obs";
public static final String FS_PREFIX_OFS = "ofs";
public static final String FS_PREFIX_GFS = "gfs";
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
index fd7da29e519..a307ff63699 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
@@ -61,6 +61,7 @@ public class LocationPath {
COSN, // Tencent
OFS, // Tencent CHDFS
GFS, // Tencent GooseFs,
+LAKEFS, // used by Tencent DLC
OSS, // Alibaba,
OSS_HDFS, // JindoFS on OSS
JFS, // JuiceFS,
@@ -158,6 +159,10 @@ public class LocationPath {
locationType = LocationType.COSN;
this.location = location;
break;
+case FeConstants.FS_PREFIX_LAKEFS:
+locationType = LocationType.COSN;
+this.location = normalizedLakefsPath(location);
+break;
case FeConstants.FS_PREFIX_VIEWFS:
locationType = LocationType.VIEWFS;
this.location = location;
@@ -270,6 +275,15 @@ public class LocationPath {
}
}
+private static String normalizedLakefsPath(String location) {
+int atIndex = location.indexOf("@dlc");
+if (atIndex != -1) {
+return "lakefs://" + location.substring(atIndex + 1);
+} else {
+return location;
+}
+}
+
public static Pair getFSIdentity(String location,
String bindBrokerName) {
LocationPath locationPath = new LocationPath(location);
FileSystemType fsType = (bindBrokerName != null) ?
FileSystemType.BROKER : locationPath.getFileSystemType();
@@ -337,6 +351,7 @@ public class LocationPath {
case GCS:
// ATTN, for COSN, on FE side, use HadoopFS to access, but on
BE, use S3 client to access.
case COSN:
+case LAKEFS:
// now we only support S3 client for object storage on BE
return TFileType.FILE_S3;
case HDFS:
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
index c10594c4977..a2ea2cee47e 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
@@ -185,7 +185,7 @@ public class PropertyConverter {
return OBSFileSystem.class.getName();
} else if (fsScheme.equalsIgnoreCase("oss")) {
return AliyunOSSFileSystem.class.getName();
-} else if (fsScheme.equalsIgnoreCase("cosn")) {
+} else if (fsScheme.equalsIgnoreCase("cosn") ||
fsScheme.equalsIgnoreCase("lakefs")) {
return CosFileSystem.class.getName();
} else {
return S3AFileSystem.class.getName();
@@ -341,6 +341,7 @@ public class PropertyConverter {
cosProperties.put(CosNConfigKeys.COSN_ENDPOINT_SUFFIX_KEY,
props.get(Co
(doris) branch branch-2.1 updated: [Improvement](multicatalog) support read tencent dlc table on lakefs (#36891)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 8a1ebba1cc9 [Improvement](multicatalog) support read tencent dlc table
on lakefs (#36891)
8a1ebba1cc9 is described below
commit 8a1ebba1cc90b2b4d05dea596d03b66fbd1bbe79
Author: Yulei-Yang
AuthorDate: Thu Jun 27 14:03:48 2024 +0800
[Improvement](multicatalog) support read tencent dlc table on lakefs
(#36891)
bp #36823
---
.../main/java/org/apache/doris/common/FeConstants.java| 1 +
.../java/org/apache/doris/common/util/LocationPath.java | 15 +++
.../doris/datasource/property/PropertyConverter.java | 3 ++-
.../doris/datasource/property/PropertyConverterTest.java | 2 +-
4 files changed, 19 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
b/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
index a502d79e032..f137c4cab49 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/FeConstants.java
@@ -71,6 +71,7 @@ public class FeConstants {
public static final String FS_PREFIX_BOS = "bos";
public static final String FS_PREFIX_COS = "cos";
public static final String FS_PREFIX_COSN = "cosn";
+public static final String FS_PREFIX_LAKEFS = "lakefs";
public static final String FS_PREFIX_OBS = "obs";
public static final String FS_PREFIX_OFS = "ofs";
public static final String FS_PREFIX_GFS = "gfs";
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
index dd1641126bf..eccb483578a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/util/LocationPath.java
@@ -62,6 +62,7 @@ public class LocationPath {
COSN, // Tencent
OFS, // Tencent CHDFS
GFS, // Tencent GooseFs,
+LAKEFS, // used by Tencent DLC
OSS, // Alibaba,
OSS_HDFS, // JindoFS on OSS
JFS, // JuiceFS,
@@ -163,6 +164,10 @@ public class LocationPath {
locationType = LocationType.COSN;
this.location = location;
break;
+case FeConstants.FS_PREFIX_LAKEFS:
+locationType = LocationType.COSN;
+this.location = normalizedLakefsPath(location);
+break;
case FeConstants.FS_PREFIX_VIEWFS:
locationType = LocationType.VIEWFS;
this.location = location;
@@ -277,6 +282,15 @@ public class LocationPath {
}
}
+private static String normalizedLakefsPath(String location) {
+int atIndex = location.indexOf("@dlc");
+if (atIndex != -1) {
+return "lakefs://" + location.substring(atIndex + 1);
+} else {
+return location;
+}
+}
+
public static Pair getFSIdentity(String location,
String bindBrokerName) {
LocationPath locationPath = new LocationPath(location);
FileSystemType fsType = (bindBrokerName != null) ?
FileSystemType.BROKER : locationPath.getFileSystemType();
@@ -351,6 +365,7 @@ public class LocationPath {
case GCS:
// ATTN, for COSN, on FE side, use HadoopFS to access, but on
BE, use S3 client to access.
case COSN:
+case LAKEFS:
// now we only support S3 client for object storage on BE
return TFileType.FILE_S3;
case HDFS:
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
index 425ea6cdcfe..9dea5eb3802 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/property/PropertyConverter.java
@@ -188,7 +188,7 @@ public class PropertyConverter {
return OBSFileSystem.class.getName();
} else if (fsScheme.equalsIgnoreCase("oss")) {
return AliyunOSSFileSystem.class.getName();
-} else if (fsScheme.equalsIgnoreCase("cosn")) {
+} else if (fsScheme.equalsIgnoreCase("cosn") ||
fsScheme.equalsIgnoreCase("lakefs")) {
return CosFileSystem.class.getName();
} else {
return S3AFileSystem.class.getName();
@@ -354,6 +354,7 @@ public class PropertyConverter {
cosProperties.put(CosNConfigKeys.COSN_ENDPOINT_SUFFIX_KEY,
(doris) branch branch-2.0 updated: [fix](multicatalog) fix npe issue when alter property for a non-exist catalog (#36951)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 54592d048a0 [fix](multicatalog) fix npe issue when alter property for
a non-exist catalog (#36951)
54592d048a0 is described below
commit 54592d048a0c3e0eff698d0c8a062f9e382063db
Author: Yulei-Yang
AuthorDate: Fri Jun 28 18:52:43 2024 +0800
[fix](multicatalog) fix npe issue when alter property for a non-exist
catalog (#36951)
---
fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java | 2 +-
.../suites/external_table_p2/hive/test_external_catalog_hive.groovy | 4
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
index d0c8f91800d..ac809812260 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
@@ -348,10 +348,10 @@ public class CatalogMgr implements Writable,
GsonPostProcessable {
writeLock();
try {
CatalogIf catalog = nameToCatalog.get(stmt.getCatalogName());
-Map oldProperties = catalog.getProperties();
if (catalog == null) {
throw new DdlException("No catalog found with name: " +
stmt.getCatalogName());
}
+Map oldProperties = catalog.getProperties();
if (stmt.getNewProperties().containsKey("type") &&
!catalog.getType()
.equalsIgnoreCase(stmt.getNewProperties().get("type"))) {
throw new DdlException("Can't modify the type of catalog
property with name: " + stmt.getCatalogName());
diff --git
a/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy
b/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy
index ef2b5a32e42..45b0b7dc9ca 100644
---
a/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy
+++
b/regression-test/suites/external_table_p2/hive/test_external_catalog_hive.groovy
@@ -152,5 +152,9 @@ suite("test_external_catalog_hive", "p2") {
"""
exception "Failed to init access controller: bound must be
positive"
}
+
+ sql """DROP CATALOG if exists ctl_not_exist_not_exist"""
+def res4 = sql """ALTER CATALOG ctl_not_exist_not_exist SET PROPERTIES
('s3.access_key' = '')"""
+assertTrue(res4.contains("No catalog found with name"))
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [fix](multicatalog) make lastdbofcatalog a session variable (#38117)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new d146ebf0dc6 [fix](multicatalog) make lastdbofcatalog a session
variable (#38117)
d146ebf0dc6 is described below
commit d146ebf0dc6df9355dcbdfaff85983def51fcac0
Author: Yulei-Yang
AuthorDate: Tue Jul 23 16:45:10 2024 +0800
[fix](multicatalog) make lastdbofcatalog a session variable (#38117)
bp #37828
---
.../main/java/org/apache/doris/catalog/Env.java| 6 ++---
.../org/apache/doris/datasource/CatalogMgr.java| 29 ++
.../java/org/apache/doris/qe/ConnectContext.java | 17 +
.../hive/test_external_catalog_hive.groovy | 17 +++--
4 files changed, 48 insertions(+), 21 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
index d6ce141ca0a..c4f0bed4dd6 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Env.java
@@ -4466,7 +4466,7 @@ public class Env {
this.alter.getClusterHandler().cancel(stmt);
}
-// Switch catalog of this sesseion.
+// Switch catalog of this session.
public void changeCatalog(ConnectContext ctx, String catalogName) throws
DdlException {
CatalogIf catalogIf = catalogMgr.getCatalogNullable(catalogName);
if (catalogIf == null) {
@@ -4478,11 +4478,11 @@ public class Env {
if (StringUtils.isNotEmpty(currentDB)) {
// When dropped the current catalog in current context, the
current catalog will be null.
if (ctx.getCurrentCatalog() != null) {
-
catalogMgr.addLastDBOfCatalog(ctx.getCurrentCatalog().getName(), currentDB);
+ctx.addLastDBOfCatalog(ctx.getCurrentCatalog().getName(),
currentDB);
}
}
ctx.changeDefaultCatalog(catalogName);
-String lastDb = catalogMgr.getLastDB(catalogName);
+String lastDb = ctx.getLastDBOfCatalog(catalogName);
if (StringUtils.isNotEmpty(lastDb)) {
ctx.setDatabase(lastDb);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
index d4e89732ea8..8671f1f9f12 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogMgr.java
@@ -84,8 +84,6 @@ public class CatalogMgr implements Writable,
GsonPostProcessable {
private final Map idToCatalog = Maps.newConcurrentMap();
// this map will be regenerated from idToCatalog, so not need to persist.
private final Map nameToCatalog =
Maps.newConcurrentMap();
-// record last used database of every catalog
-private final Map lastDBOfCatalog =
Maps.newConcurrentMap();
// Use a separate instance to facilitate access.
// internalDataSource still exists in idToDataSource and nameToDataSource
@@ -119,7 +117,9 @@ public class CatalogMgr implements Writable,
GsonPostProcessable {
if (catalog != null) {
catalog.onClose();
nameToCatalog.remove(catalog.getName());
-lastDBOfCatalog.remove(catalog.getName());
+if (ConnectContext.get() != null) {
+ConnectContext.get().removeLastDBOfCatalog(catalog.getName());
+}
Env.getCurrentEnv().getExtMetaCacheMgr().removeCache(catalog.getId());
if (!Strings.isNullOrEmpty(catalog.getResource())) {
Resource catalogResource =
Env.getCurrentEnv().getResourceMgr().getResource(catalog.getResource());
@@ -167,14 +167,6 @@ public class CatalogMgr implements Writable,
GsonPostProcessable {
ErrorCode.ERR_UNKNOWN_CATALOG));
}
-public void addLastDBOfCatalog(String catalog, String db) {
-lastDBOfCatalog.put(catalog, db);
-}
-
-public String getLastDB(String catalog) {
-return lastDBOfCatalog.get(catalog);
-}
-
public List getCatalogIds() {
return Lists.newArrayList(idToCatalog.keySet());
}
@@ -274,7 +266,9 @@ public class CatalogMgr implements Writable,
GsonPostProcessable {
replayDropCatalog(log);
Env.getCurrentEnv().getEditLog().logCatalogLog(OperationType.OP_DROP_CATALOG,
log);
-lastDBOfCatalog.remove(stmt.getCatalogName());
+if (ConnectContext.get() != null) {
+
ConnectContext.get().removeLastDBOfCatalog(stmt.getCatalogName());
+}
} finally {
writeUnlock();
}
@@ -297,10 +291,13 @@ public class CatalogMgr implements Writable,
GsonPostProcessable {
replayAlterCatalogName
(doris) branch master updated (cb7e8bf1aa6 -> c6622ea16aa)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from cb7e8bf1aa6 [fix](simd-json-reader) fix SIMD json reader lose data and
support stream parser (#35781)
add c6622ea16aa [Improvement](coldheat) add statment to show objects which
use storage policy (#35741)
No new revisions were added by this update.
Summary of changes:
fe/fe-core/src/main/cup/sql_parser.cup | 8 +
...etStmt.java => ShowStoragePolicyUsingStmt.java} | 50 ---
.../java/org/apache/doris/policy/PolicyMgr.java| 106 +
.../java/org/apache/doris/qe/ShowExecutor.java | 8 +
.../test_show_storage_policy_using.groovy | 164 +
5 files changed, 312 insertions(+), 24 deletions(-)
copy
fe/fe-core/src/main/java/org/apache/doris/analysis/{DiagnoseTabletStmt.java =>
ShowStoragePolicyUsingStmt.java} (62%)
create mode 100644
regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [Improvement](coldhot) add statement to show objects which use storage policy (#35839)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 0585de12b55 [Improvement](coldhot) add statement to show objects which
use storage policy (#35839)
0585de12b55 is described below
commit 0585de12b559608c0041da190dd1164a50165cdc
Author: Yulei-Yang
AuthorDate: Tue Jun 4 19:24:31 2024 +0800
[Improvement](coldhot) add statement to show objects which use storage
policy (#35839)
---
fe/fe-core/src/main/cup/sql_parser.cup | 8 +
.../doris/analysis/ShowStoragePolicyUsingStmt.java | 78 ++
.../java/org/apache/doris/policy/PolicyMgr.java| 110 ++
.../java/org/apache/doris/qe/ShowExecutor.java | 8 +
.../test_show_storage_policy_using.groovy | 164 +
5 files changed, 368 insertions(+)
diff --git a/fe/fe-core/src/main/cup/sql_parser.cup
b/fe/fe-core/src/main/cup/sql_parser.cup
index 856a8cb3fb8..d7526a8706f 100644
--- a/fe/fe-core/src/main/cup/sql_parser.cup
+++ b/fe/fe-core/src/main/cup/sql_parser.cup
@@ -3944,6 +3944,14 @@ show_stmt ::=
{:
RESULT = new ShowPolicyStmt(PolicyTypeEnum.STORAGE, null, null);
:}
+| KW_SHOW KW_STORAGE KW_POLICY KW_USING
+{:
+RESULT = new ShowStoragePolicyUsingStmt(null);
+:}
+| KW_SHOW KW_STORAGE KW_POLICY KW_USING KW_FOR ident:policy
+{:
+RESULT = new ShowStoragePolicyUsingStmt(policy);
+:}
;
show_param ::=
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
new file mode 100644
index 000..56598a194ee
--- /dev/null
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
@@ -0,0 +1,78 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+
+import lombok.Getter;
+
+/**
+ * Show objects where storage policy is used
+ * syntax:
+ * SHOW STORAGE POLICY USING [for policy_name]
+ **/
+public class ShowStoragePolicyUsingStmt extends ShowStmt {
+
+public static final ShowResultSetMetaData RESULT_META_DATA =
+ShowResultSetMetaData.builder()
+.addColumn(new Column("PolicyName",
ScalarType.createVarchar(100)))
+.addColumn(new Column("Database",
ScalarType.createVarchar(20)))
+.addColumn(new Column("Table", ScalarType.createVarchar(20)))
+.addColumn(new Column("Partitions",
ScalarType.createVarchar(60)))
+.build();
+@Getter
+private final String policyName;
+
+public ShowStoragePolicyUsingStmt(String policyName) {
+this.policyName = policyName;
+}
+
+@Override
+public void analyze(Analyzer analyzer) throws UserException {
+super.analyze(analyzer);
+
+// check auth
+if
(!Env.getCurrentEnv().getAccessManager().checkGlobalPriv(ConnectContext.get(),
PrivPredicate.ADMIN)) {
+
ErrorReport.reportAnalysisException(ErrorCode.ERR_SPECIFIC_ACCESS_DENIED_ERROR,
"ADMIN");
+}
+}
+
+@Override
+public String toSql() {
+StringBuilder sb = new StringBuilder();
+sb.append("SHOW STORAGE POLICY USING");
+if (policyName != null) {
+sb.append(" FOR ").append(policyName);
+}
+
+return sb.toString();
+}
+
+@Override
+public ShowResultSetMetaData getMetaData() {
+return RESULT_META_DATA;
+}
+}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/poli
(doris) branch branch-2.0 updated: [Improvement](coldhot) add statement to show objects which use storage policy (#35836)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new dc04778c290 [Improvement](coldhot) add statement to show objects which
use storage policy (#35836)
dc04778c290 is described below
commit dc04778c290b0fbda86a5776008c1bc9f167094e
Author: Yulei-Yang
AuthorDate: Tue Jun 4 19:24:43 2024 +0800
[Improvement](coldhot) add statement to show objects which use storage
policy (#35836)
---
fe/fe-core/src/main/cup/sql_parser.cup | 8 +
.../doris/analysis/ShowStoragePolicyUsingStmt.java | 78 ++
.../java/org/apache/doris/policy/PolicyMgr.java| 110 ++
.../java/org/apache/doris/qe/ShowExecutor.java | 8 +
.../test_show_storage_policy_using.groovy | 164 +
5 files changed, 368 insertions(+)
diff --git a/fe/fe-core/src/main/cup/sql_parser.cup
b/fe/fe-core/src/main/cup/sql_parser.cup
index 5da2e9dc8cc..a9c33472534 100644
--- a/fe/fe-core/src/main/cup/sql_parser.cup
+++ b/fe/fe-core/src/main/cup/sql_parser.cup
@@ -3769,6 +3769,14 @@ show_stmt ::=
{:
RESULT = new ShowPolicyStmt(PolicyTypeEnum.STORAGE, null, null);
:}
+| KW_SHOW KW_STORAGE KW_POLICY KW_USING
+{:
+RESULT = new ShowStoragePolicyUsingStmt(null);
+:}
+| KW_SHOW KW_STORAGE KW_POLICY KW_USING KW_FOR ident:policy
+{:
+RESULT = new ShowStoragePolicyUsingStmt(policy);
+:}
;
show_param ::=
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
new file mode 100644
index 000..56598a194ee
--- /dev/null
+++
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ShowStoragePolicyUsingStmt.java
@@ -0,0 +1,78 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.analysis;
+
+import org.apache.doris.catalog.Column;
+import org.apache.doris.catalog.Env;
+import org.apache.doris.catalog.ScalarType;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.UserException;
+import org.apache.doris.mysql.privilege.PrivPredicate;
+import org.apache.doris.qe.ConnectContext;
+import org.apache.doris.qe.ShowResultSetMetaData;
+
+import lombok.Getter;
+
+/**
+ * Show objects where storage policy is used
+ * syntax:
+ * SHOW STORAGE POLICY USING [for policy_name]
+ **/
+public class ShowStoragePolicyUsingStmt extends ShowStmt {
+
+public static final ShowResultSetMetaData RESULT_META_DATA =
+ShowResultSetMetaData.builder()
+.addColumn(new Column("PolicyName",
ScalarType.createVarchar(100)))
+.addColumn(new Column("Database",
ScalarType.createVarchar(20)))
+.addColumn(new Column("Table", ScalarType.createVarchar(20)))
+.addColumn(new Column("Partitions",
ScalarType.createVarchar(60)))
+.build();
+@Getter
+private final String policyName;
+
+public ShowStoragePolicyUsingStmt(String policyName) {
+this.policyName = policyName;
+}
+
+@Override
+public void analyze(Analyzer analyzer) throws UserException {
+super.analyze(analyzer);
+
+// check auth
+if
(!Env.getCurrentEnv().getAccessManager().checkGlobalPriv(ConnectContext.get(),
PrivPredicate.ADMIN)) {
+
ErrorReport.reportAnalysisException(ErrorCode.ERR_SPECIFIC_ACCESS_DENIED_ERROR,
"ADMIN");
+}
+}
+
+@Override
+public String toSql() {
+StringBuilder sb = new StringBuilder();
+sb.append("SHOW STORAGE POLICY USING");
+if (policyName != null) {
+sb.append(" FOR ").append(policyName);
+}
+
+return sb.toString();
+}
+
+@Override
+public ShowResultSetMetaData getMetaData() {
+return RESULT_META_DATA;
+}
+}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/poli
(doris) branch branch-2.1 updated: [fix](storage_policy) fix cannot cancel a partition's storage policy (#35890)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new af31e96c4e5 [fix](storage_policy) fix cannot cancel a partition's
storage policy (#35890)
af31e96c4e5 is described below
commit af31e96c4e583c3df95de5be378c94d8a29d4ea0
Author: Yulei-Yang
AuthorDate: Wed Jun 5 15:53:26 2024 +0800
[fix](storage_policy) fix cannot cancel a partition's storage policy
(#35890)
---
.../main/java/org/apache/doris/alter/Alter.java| 6 +
.../apache/doris/common/util/PropertyAnalyzer.java | 10 -
.../test_show_storage_policy_using.groovy | 26 ++
3 files changed, 36 insertions(+), 6 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
index 1d408b6799f..68ac144de1a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
@@ -779,6 +779,12 @@ public class Alter {
// check currentStoragePolicy resource exist.
Env.getCurrentEnv().getPolicyMgr().checkStoragePolicyExist(currentStoragePolicy);
partitionInfo.setStoragePolicy(partition.getId(),
currentStoragePolicy);
+} else {
+if (partition.getRemoteDataSize() > 0) {
+throw new AnalysisException(
+"Cannot cancel storage policy for partition which is
already on code storage.");
+}
+partitionInfo.setStoragePolicy(partition.getId(), "");
}
// 4.4 analyze new properties
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
index 07236da2ee3..d9eae228739 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
@@ -244,11 +244,9 @@ public class PropertyAnalyzer {
} else if (key.equalsIgnoreCase(PROPERTIES_STORAGE_COOLDOWN_TIME))
{
DateLiteral dateLiteral = new DateLiteral(value,
ScalarType.getDefaultDateType(Type.DATETIME));
cooldownTimestamp =
dateLiteral.unixTimestamp(TimeUtils.getTimeZone());
-} else if (!hasStoragePolicy &&
key.equalsIgnoreCase(PROPERTIES_STORAGE_POLICY)) {
-if (!Strings.isNullOrEmpty(value)) {
-hasStoragePolicy = true;
-newStoragePolicy = value;
-}
+} else if (key.equalsIgnoreCase(PROPERTIES_STORAGE_POLICY)) {
+hasStoragePolicy = true;
+newStoragePolicy = value;
}
} // end for properties
@@ -277,7 +275,7 @@ public class PropertyAnalyzer {
cooldownTimestamp = DataProperty.MAX_COOLDOWN_TIME_MS;
}
-if (hasStoragePolicy) {
+if (hasStoragePolicy && !"".equals(newStoragePolicy)) {
// check remote storage policy
StoragePolicy checkedPolicy =
StoragePolicy.ofCheck(newStoragePolicy);
Policy policy =
Env.getCurrentEnv().getPolicyMgr().getPolicy(checkedPolicy);
diff --git
a/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
b/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
index ca94b09077e..6f0413f63ae 100644
---
a/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
+++
b/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
@@ -156,6 +156,32 @@ suite("test_show_storage_policy_using") {
"""
assertTrue(show_result.size() >= 4)
+// test cancel a partition's storage policy
+sql """
+ALTER TABLE partition_with_multiple_storage_policy MODIFY PARTITION
(`p201701`) SET ("storage_policy"="")
+"""
+show_result = sql """
+show storage policy using for ${policy_name}
+"""
+assertEquals(show_result.size(), 1)
+assertTrue(show_result[0][2].equals("table_with_storage_policy_1"))
+
+sql """
+ALTER TABLE partition_with_multiple_storage_policy MODIFY PARTITION
(`p201701`) SET ("storage_policy"="${policy_name}")
+"""
+show_result = sql """
+show storage policy using for ${policy_name}
+"""
+assertEquals(show_result.size(), 2)
+
+sql """
+ALTER TABLE partition_w
(doris) branch branch-2.0 updated: [fix](coldhot) fix cannot cancel storage policy of partition (#35885)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 18fd1cfb489 [fix](coldhot) fix cannot cancel storage policy of
partition (#35885)
18fd1cfb489 is described below
commit 18fd1cfb489e8af570f4fe9db9258ad5a6272363
Author: Yulei-Yang
AuthorDate: Wed Jun 5 15:53:39 2024 +0800
[fix](coldhot) fix cannot cancel storage policy of partition (#35885)
---
.../main/java/org/apache/doris/alter/Alter.java| 6 +
.../apache/doris/common/util/PropertyAnalyzer.java | 10 -
.../test_show_storage_policy_using.groovy | 26 ++
3 files changed, 36 insertions(+), 6 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
index 4e0fe1bbfcd..bcfcfad81c1 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
@@ -830,6 +830,12 @@ public class Alter {
// check currentStoragePolicy resource exist.
Env.getCurrentEnv().getPolicyMgr().checkStoragePolicyExist(currentStoragePolicy);
partitionInfo.setStoragePolicy(partition.getId(),
currentStoragePolicy);
+} else {
+if (partition.getRemoteDataSize() > 0) {
+throw new AnalysisException(
+"Cannot cancel storage policy for partition which is
already on code storage.");
+}
+partitionInfo.setStoragePolicy(partition.getId(), "");
}
// 4.4 analyze new properties
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
index 686495ef73d..b922ae84984 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/util/PropertyAnalyzer.java
@@ -220,11 +220,9 @@ public class PropertyAnalyzer {
} else if (key.equalsIgnoreCase(PROPERTIES_STORAGE_COOLDOWN_TIME))
{
DateLiteral dateLiteral = new DateLiteral(value,
ScalarType.getDefaultDateType(Type.DATETIME));
cooldownTimestamp =
dateLiteral.unixTimestamp(TimeUtils.getTimeZone());
-} else if (!hasStoragePolicy &&
key.equalsIgnoreCase(PROPERTIES_STORAGE_POLICY)) {
-if (!Strings.isNullOrEmpty(value)) {
-hasStoragePolicy = true;
-newStoragePolicy = value;
-}
+} else if (key.equalsIgnoreCase(PROPERTIES_STORAGE_POLICY)) {
+hasStoragePolicy = true;
+newStoragePolicy = value;
}
} // end for properties
@@ -253,7 +251,7 @@ public class PropertyAnalyzer {
cooldownTimestamp = DataProperty.MAX_COOLDOWN_TIME_MS;
}
-if (hasStoragePolicy) {
+if (hasStoragePolicy && !"".equals(newStoragePolicy)) {
// check remote storage policy
StoragePolicy checkedPolicy =
StoragePolicy.ofCheck(newStoragePolicy);
Policy policy =
Env.getCurrentEnv().getPolicyMgr().getPolicy(checkedPolicy);
diff --git
a/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
b/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
index ca94b09077e..6f0413f63ae 100644
---
a/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
+++
b/regression-test/suites/cold_heat_separation_p2/test_show_storage_policy_using.groovy
@@ -156,6 +156,32 @@ suite("test_show_storage_policy_using") {
"""
assertTrue(show_result.size() >= 4)
+// test cancel a partition's storage policy
+sql """
+ALTER TABLE partition_with_multiple_storage_policy MODIFY PARTITION
(`p201701`) SET ("storage_policy"="")
+"""
+show_result = sql """
+show storage policy using for ${policy_name}
+"""
+assertEquals(show_result.size(), 1)
+assertTrue(show_result[0][2].equals("table_with_storage_policy_1"))
+
+sql """
+ALTER TABLE partition_with_multiple_storage_policy MODIFY PARTITION
(`p201701`) SET ("storage_policy"="${policy_name}")
+"""
+show_result = sql """
+show storage policy using for ${policy_name}
+"""
+assertEquals(show_result.size(), 2)
+
+sql """
+ALTER TABLE partition_with_multiple_storage_policy MODIFY PAR
(doris) branch branch-2.1 updated (a42b06a1683 -> 4d5db6fee3d)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch branch-2.1 in repository https://gitbox.apache.org/repos/asf/doris.git from a42b06a1683 [branch-2.1][test](jdbc catalog) Change the db2 image address and repair test (#35967) add 4d5db6fee3d [fix](proc) fix keyword case sensitive issue for stmt show_frontends_disks (#35919) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/doris/analysis/ShowFrontendsStmt.java | 4 ++-- .../main/java/org/apache/doris/common/proc/FrontendsProcNode.java | 2 +- regression-test/suites/node_p0/test_frontend.groovy | 6 ++ 3 files changed, 9 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated (28c36bf9036 -> 9832838d181)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 28c36bf9036 [fix](profile) Fix read load profile failed (#38437) add 9832838d181 [fix](multicatalog) fix hadoop authenticator not inited for existing hms catalog (#38475) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/datasource/hive/HMSExternalCatalog.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-1.2-lts updated: [fix](es) fix querying the index (#36319)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new c9d31e0d2f1 [fix](es) fix querying the index (#36319)
c9d31e0d2f1 is described below
commit c9d31e0d2f154391f8cfff356dca93dbf7158305
Author: lw112 <131352377+felixw...@users.noreply.github.com>
AuthorDate: Fri Jun 14 18:57:10 2024 +0800
[fix](es) fix querying the index (#36319)
---
.../java/org/apache/doris/external/elasticsearch/EsUtil.java | 12
1 file changed, 8 insertions(+), 4 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java
b/fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java
index f2980133528..514c06206c3 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/external/elasticsearch/EsUtil.java
@@ -39,6 +39,9 @@ import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
+import java.util.Spliterator;
+import java.util.Spliterators;
+import java.util.stream.StreamSupport;
/**
* Util for ES, some static method.
@@ -110,15 +113,16 @@ public class EsUtil {
// remove dynamic templates, for ES 7.x and 8.x
checkNonPropertiesFields(mappings, arrayFields);
String firstType = mappings.fieldNames().next();
-if (!"properties".equals(firstType)) {
-// If type is not passed in takes the first type.
+if (StreamSupport.stream(Spliterators
+.spliteratorUnknownSize(mappings.fieldNames(),
Spliterator.ORDERED), false)
+.anyMatch(s -> s.contains("properties"))) {
+return mappings;
+} else {
ObjectNode firstData = (ObjectNode) mappings.get(firstType);
// check for ES 6.x and before
checkNonPropertiesFields(firstData, arrayFields);
return firstData;
}
-// Equal 7.x and after
-return mappings;
} else {
if (mappings.has(mappingType)) {
ObjectNode jsonData = (ObjectNode) mappings.get(mappingType);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.0 updated: [Improvement](hms catalog) support show_create_database for hms catalog (#36294)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-2.0
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.0 by this push:
new 36a43317dbb [Improvement](hms catalog) support show_create_database
for hms catalog (#36294)
36a43317dbb is described below
commit 36a43317dbb2e7cca4a8b2700e69f336c4be4c32
Author: Yulei-Yang
AuthorDate: Fri Jun 14 19:07:57 2024 +0800
[Improvement](hms catalog) support show_create_database for hms catalog
(#36294)
---
.../datasource/hive/PooledHiveMetaStoreClient.java | 14 +++
.../java/org/apache/doris/qe/ShowExecutor.java | 25 +---
.../hadoop/hive/metastore/HiveMetaStoreClient.java | 7 +++-
.../export_p0/test_show_create_database.groovy | 46 ++
4 files changed, 84 insertions(+), 8 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/PooledHiveMetaStoreClient.java
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/PooledHiveMetaStoreClient.java
index 8023a74e31a..b8c515f45d3 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/PooledHiveMetaStoreClient.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/datasource/hive/PooledHiveMetaStoreClient.java
@@ -43,6 +43,7 @@ import
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient;
import org.apache.hadoop.hive.metastore.api.ColumnStatisticsObj;
import org.apache.hadoop.hive.metastore.api.CurrentNotificationEventId;
import org.apache.hadoop.hive.metastore.api.DataOperationType;
+import org.apache.hadoop.hive.metastore.api.Database;
import org.apache.hadoop.hive.metastore.api.FieldSchema;
import org.apache.hadoop.hive.metastore.api.LockComponent;
import org.apache.hadoop.hive.metastore.api.LockResponse;
@@ -188,6 +189,19 @@ public class PooledHiveMetaStoreClient {
}
}
+public Database getDatabase(String dbName) {
+try (CachedClient client = getClient()) {
+try {
+return client.client.getDatabase(dbName);
+} catch (Exception e) {
+client.setThrowable(e);
+throw e;
+}
+} catch (Exception e) {
+throw new HMSClientException("failed to get database %s from hms
client", e, dbName);
+}
+}
+
public Table getTable(String dbName, String tblName) {
try (CachedClient client = getClient()) {
try {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
b/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
index e7350a1c30a..4726d036619 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/qe/ShowExecutor.java
@@ -991,15 +991,28 @@ public class ShowExecutor {
private void handleShowCreateDb() throws AnalysisException {
ShowCreateDbStmt showStmt = (ShowCreateDbStmt) stmt;
List> rows = Lists.newArrayList();
-DatabaseIf db =
ctx.getCurrentCatalog().getDbOrAnalysisException(showStmt.getDb());
+
StringBuilder sb = new StringBuilder();
-sb.append("CREATE DATABASE
`").append(ClusterNamespace.getNameFromFullName(showStmt.getDb())).append("`");
-if (db.getDbProperties().getProperties().size() > 0) {
-sb.append("\nPROPERTIES (\n");
-sb.append(new PrintableMap<>(db.getDbProperties().getProperties(),
"=", true, true, false));
-sb.append("\n)");
+CatalogIf catalog = ctx.getCurrentCatalog();
+if (catalog instanceof HMSExternalCatalog) {
+String simpleDBName =
ClusterNamespace.getNameFromFullName(showStmt.getDb());
+org.apache.hadoop.hive.metastore.api.Database db =
((HMSExternalCatalog) catalog).getClient()
+.getDatabase(simpleDBName);
+sb.append("CREATE DATABASE `").append(simpleDBName).append("`")
+.append(" LOCATION '")
+.append(db.getLocationUri())
+.append("'");
+} else {
+DatabaseIf db = catalog.getDbOrAnalysisException(showStmt.getDb());
+sb.append("CREATE DATABASE
`").append(ClusterNamespace.getNameFromFullName(showStmt.getDb())).append("`");
+if (db.getDbProperties().getProperties().size() > 0) {
+sb.append("\nPROPERTIES (\n");
+sb.append(new
PrintableMap<>(db.getDbProperties().getProperties(), "=", true, true, false));
+sb.append("\n)");
+}
}
+
rows.add(Lists.newArrayList(ClusterNamespace.getNameFromFullName(showStmt.getDb()),
sb.toString()));
resultSet = new ShowResultSet(showStmt.getMet
[incubator-doris] branch branch-0.15 updated: [Feature] add support for tencent chdfs (#8964)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch branch-0.15
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/branch-0.15 by this push:
new a90302b549 [Feature] add support for tencent chdfs (#8964)
a90302b549 is described below
commit a90302b549617d854f6d9792b465b0b31c7b723f
Author: wucheng
AuthorDate: Tue Apr 12 15:37:31 2022 +0800
[Feature] add support for tencent chdfs (#8964)
Co-authored-by: chengwu
---
.../java/org/apache/doris/analysis/ExportStmt.java | 5 +-
.../org/apache/doris/analysis/StorageBackend.java | 3 +
.../doris/broker/hdfs/FileSystemManager.java | 135 -
gensrc/thrift/Types.thrift | 1 +
4 files changed, 141 insertions(+), 3 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
index 275dc51bf1..095dbbd68a 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
@@ -238,8 +238,9 @@ public class ExportStmt extends StatementBase {
String schema = uri.getScheme();
if (type == StorageBackend.StorageType.BROKER) {
if (schema == null || (!schema.equalsIgnoreCase("bos") &&
!schema.equalsIgnoreCase("afs")
-&& !schema.equalsIgnoreCase("hdfs"))) {
-throw new AnalysisException("Invalid export path. please
use valid 'HDFS://', 'AFS://' or 'BOS://' path.");
+&& !schema.equalsIgnoreCase("hdfs") &&
!schema.equalsIgnoreCase("ofs"))) {
+throw new AnalysisException("Invalid export path. please
use valid 'HDFS://', 'AFS://' , 'BOS://', "
++ "or 'ofs://' path.");
}
} else if (type == StorageBackend.StorageType.S3) {
if (schema == null || !schema.equalsIgnoreCase("s3")) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
index 933153b728..70b10b4f06 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
@@ -111,6 +111,7 @@ public class StorageBackend extends StorageDesc implements
ParseNode {
public enum StorageType {
BROKER("Doris Broker"),
S3("Amazon S3 Simple Storage Service"),
+OFS("Tencent CHDFS"),
// the following is not used currently
HDFS("Hadoop Distributed File System"),
LOCAL("Local file system");
@@ -132,6 +133,8 @@ public class StorageBackend extends StorageDesc implements
ParseNode {
return TStorageBackendType.S3;
case HDFS:
return TStorageBackendType.HDFS;
+case OFS:
+return TStorageBackendType.OFS;
case LOCAL:
return TStorageBackendType.LOCAL;
default:
diff --git
a/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
b/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
index 6cedd70449..526c914c7e 100644
---
a/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
+++
b/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
@@ -66,6 +66,7 @@ public class FileSystemManager {
// supported scheme
private static final String HDFS_SCHEME = "hdfs";
private static final String S3A_SCHEME = "s3a";
+private static final String CHDFS_SCHEME = "ofs";
private static final String USER_NAME_KEY = "username";
private static final String PASSWORD_KEY = "password";
@@ -152,7 +153,9 @@ public class FileSystemManager {
brokerFileSystem = getDistributedFileSystem(path, properties);
} else if (scheme.equals(S3A_SCHEME)) {
brokerFileSystem = getS3AFileSystem(path, properties);
-} else {
+} else if (scheme.equals(CHDFS_SCHEME)) {
+brokerFileSystem = getChdfsFileSystem(path, properties);
+}else {
throw new
BrokerException(TBrokerOperationStatusCode.INVALID_INPUT_FILE_PATH,
"invalid path. scheme is not supported");
}
@@ -421,6 +424,136 @@ public class FileSystemManager {
}
[incubator-doris] branch master updated: [Feature] add support for tencent chdfs (#8963)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 6af1c52e13 [Feature] add support for tencent chdfs (#8963)
6af1c52e13 is described below
commit 6af1c52e135de8b96f02942e0492313b446440f1
Author: wucheng
AuthorDate: Tue Apr 12 16:02:42 2022 +0800
[Feature] add support for tencent chdfs (#8963)
Co-authored-by: chengwu
---
.../java/org/apache/doris/analysis/ExportStmt.java | 6 +-
.../org/apache/doris/analysis/StorageBackend.java | 3 +
.../doris/broker/hdfs/FileSystemManager.java | 133 +
gensrc/thrift/Types.thrift | 1 +
4 files changed, 140 insertions(+), 3 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
index 48fb782d24..80dcf2798e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExportStmt.java
@@ -236,9 +236,9 @@ public class ExportStmt extends StatementBase {
String schema = uri.getScheme();
if (type == StorageBackend.StorageType.BROKER) {
if (schema == null || (!schema.equalsIgnoreCase("bos") &&
!schema.equalsIgnoreCase("afs")
-&& !schema.equalsIgnoreCase("hdfs"))) {
-throw new AnalysisException("Invalid export path. please use
valid 'HDFS://', 'AFS://' or 'BOS://' " +
-"path.");
+&& !schema.equalsIgnoreCase("hdfs") &&
!schema.equalsIgnoreCase("ofs"))) {
+throw new AnalysisException("Invalid export path. please use
valid 'HDFS://', 'AFS://' , 'BOS://', "
++ "or 'ofs://' path.");
}
} else if (type == StorageBackend.StorageType.S3) {
if (schema == null || !schema.equalsIgnoreCase("s3")) {
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
index 933153b728..70b10b4f06 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/StorageBackend.java
@@ -111,6 +111,7 @@ public class StorageBackend extends StorageDesc implements
ParseNode {
public enum StorageType {
BROKER("Doris Broker"),
S3("Amazon S3 Simple Storage Service"),
+OFS("Tencent CHDFS"),
// the following is not used currently
HDFS("Hadoop Distributed File System"),
LOCAL("Local file system");
@@ -132,6 +133,8 @@ public class StorageBackend extends StorageDesc implements
ParseNode {
return TStorageBackendType.S3;
case HDFS:
return TStorageBackendType.HDFS;
+case OFS:
+return TStorageBackendType.OFS;
case LOCAL:
return TStorageBackendType.LOCAL;
default:
diff --git
a/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
b/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
index 2c00a2ae96..dc94720465 100644
---
a/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
+++
b/fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java
@@ -67,6 +67,7 @@ public class FileSystemManager {
private static final String HDFS_SCHEME = "hdfs";
private static final String S3A_SCHEME = "s3a";
private static final String KS3_SCHEME = "ks3";
+private static final String CHDFS_SCHEME = "ofs";
private static final String USER_NAME_KEY = "username";
private static final String PASSWORD_KEY = "password";
@@ -163,6 +164,8 @@ public class FileSystemManager {
brokerFileSystem = getS3AFileSystem(path, properties);
} else if (scheme.equals(KS3_SCHEME)) {
brokerFileSystem = getKS3FileSystem(path, properties);
+} else if (scheme.equals(CHDFS_SCHEME)) {
+brokerFileSystem = getChdfsFileSystem(path, properties);
}else {
throw new
BrokerException(TBrokerOperationStatusCode.INVALID_INPUT_FILE_PATH,
"invalid path. scheme is not supported");
@@ -490,6 +493,136 @@ public class FileSystemManager {
}
[incubator-doris-spark-connector] 01/01: Remove redundant variable and functions which could lead to compile fail
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch lide-reed-patch-1
in repository
https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git
commit 94ad64503146928b9e056652baa769a8d1b4afb9
Author: lide
AuthorDate: Tue Apr 19 10:44:54 2022 +0800
Remove redundant variable and functions which could lead to compile fail
---
.../src/main/java/org/apache/doris/spark/rest/models/Schema.java | 9 -
1 file changed, 9 deletions(-)
diff --git
a/spark-doris-connector/src/main/java/org/apache/doris/spark/rest/models/Schema.java
b/spark-doris-connector/src/main/java/org/apache/doris/spark/rest/models/Schema.java
index a236325..87fc55f 100644
---
a/spark-doris-connector/src/main/java/org/apache/doris/spark/rest/models/Schema.java
+++
b/spark-doris-connector/src/main/java/org/apache/doris/spark/rest/models/Schema.java
@@ -25,7 +25,6 @@ public class Schema {
private int status = 0;
private String keysType;
private List properties;
-private String keysType;
public Schema() {
properties = new ArrayList<>();
@@ -59,14 +58,6 @@ public class Schema {
this.properties = properties;
}
-public String getKeysType() {
-return keysType;
-}
-
-public void setKeysType(String keysType) {
-this.keysType = keysType;
-}
-
public void put(String name, String type, String comment, int scale, int
precision, String aggregation_type) {
properties.add(new Field(name, type, comment, scale, precision,
aggregation_type));
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-spark-connector] branch lide-reed-patch-1 created (now 94ad645)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch lide-reed-patch-1 in repository https://gitbox.apache.org/repos/asf/incubator-doris-spark-connector.git at 94ad645 Remove redundant variable and functions which could lead to compile fail This branch includes the following new commits: new 94ad645 Remove redundant variable and functions which could lead to compile fail The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: avoiding a corrupt image file when there is image.ckpt with non-zero … (#9180)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5b9a1a2a5d avoiding a corrupt image file when there is image.ckpt with
non-zero … (#9180)
5b9a1a2a5d is described below
commit 5b9a1a2a5dd7c2216452864e9cf17d5534523837
Author: dataroaring <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Mon Apr 25 17:01:01 2022 +0800
avoiding a corrupt image file when there is image.ckpt with non-zero …
(#9180)
* avoiding a corrupt image file when there is image.ckpt with non-zero size
For now, saveImage writes data to image.ckpt via an append FileOutputStream,
when there is a non-zero size file named image.ckpt, a disaster would happen
due to a corrupt image file. Even worse, fe only keeps the lastest image
file
and removes others.
BTW, image file should be synced to disk.
It is dangerous to only keep the latest image file, because an image file is
validated when generating the next image file. Then we keep an non validated
image file but remove validated ones. So I will issue a pr which keeps at
least
2 image file.
* append other data after MetaHeader
* use channel.force instead of sync
---
fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java | 9 +++--
fe/fe-core/src/main/java/org/apache/doris/common/MetaFooter.java | 1 +
fe/fe-core/src/main/java/org/apache/doris/common/MetaHeader.java | 5 +
fe/fe-core/src/main/java/org/apache/doris/common/MetaWriter.java | 8 +---
4 files changed, 18 insertions(+), 5 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
index a670dd2aeb..86d6763b24 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
@@ -1966,8 +1966,13 @@ public class Catalog {
}
public void saveImage(File curFile, long replayedJournalId) throws
IOException {
-if (!curFile.exists()) {
-curFile.createNewFile();
+if (curFile.exists()) {
+if (!curFile.delete()) {
+throw new IOException(curFile.getName() + " can not be
deleted.");
+}
+}
+if (!curFile.createNewFile()) {
+throw new IOException(curFile.getName() + " can not be created.");
}
MetaWriter.write(curFile, this);
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/MetaFooter.java
b/fe/fe-core/src/main/java/org/apache/doris/common/MetaFooter.java
index 9df82aa9b7..426cd7b3d8 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/MetaFooter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/MetaFooter.java
@@ -98,6 +98,7 @@ public class MetaFooter {
long endIndex = raf.length();
raf.writeLong(endIndex - startIndex);
MetaMagicNumber.write(raf);
+raf.getChannel.force(true);
}
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/MetaHeader.java
b/fe/fe-core/src/main/java/org/apache/doris/common/MetaHeader.java
index 5617a85448..ba91b04b5d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/MetaHeader.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/MetaHeader.java
@@ -75,10 +75,15 @@ public class MetaHeader {
}
public static long write(File imageFile) throws IOException {
+if (imageFile.length() != 0) {
+throw new IOException("Meta header has to be written to an empty
file.");
+}
+
try (RandomAccessFile raf = new RandomAccessFile(imageFile, "rw")) {
raf.seek(0);
MetaMagicNumber.write(raf);
MetaJsonHeader.write(raf);
+raf.getChannel.force(true);
return raf.getFilePointer();
}
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/MetaWriter.java
b/fe/fe-core/src/main/java/org/apache/doris/common/MetaWriter.java
index 6b9d9aa791..387a50be3d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/MetaWriter.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/MetaWriter.java
@@ -94,14 +94,15 @@ public class MetaWriter {
public static void write(File imageFile, Catalog catalog) throws
IOException {
// save image does not need any lock. because only checkpoint thread
will call this method.
-LOG.info("start save image to {}. is ckpt: {}",
imageFile.getAbsolutePath(), Catalog.isCheckpointThread());
-
+LOG.info("start to save image to {}. is ckpt: {}",
imageFile.getAbsolutePath(), Catalog.isCheckpointThread());
final Referenc
[incubator-doris] branch master updated: keep at least one validated image file (#9192)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 687421b43f keep at least one validated image file (#9192)
687421b43f is described below
commit 687421b43f0c22588b56929df375ef77af43349f
Author: dataroaring <98214048+dataroar...@users.noreply.github.com>
AuthorDate: Mon Apr 25 19:32:43 2022 +0800
keep at least one validated image file (#9192)
* rename ImageSeq to LatestImageSeq in Storage
* keep at least one validated image file
---
.../java/org/apache/doris/catalog/Catalog.java | 4 +-
.../doris/httpv2/controller/HaController.java | 2 +-
.../org/apache/doris/httpv2/meta/MetaService.java | 2 +-
.../org/apache/doris/httpv2/rest/ShowAction.java | 2 +-
.../java/org/apache/doris/master/Checkpoint.java | 8 ++--
.../java/org/apache/doris/persist/MetaCleaner.java | 2 +-
.../java/org/apache/doris/persist/Storage.java | 55 --
.../java/org/apache/doris/persist/StorageTest.java | 8 ++--
8 files changed, 46 insertions(+), 37 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
index 86d6763b24..09a06a1596 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
@@ -1531,7 +1531,7 @@ public class Catalog {
private void getNewImage(Pair helperNode) throws
IOException {
long localImageVersion = 0;
Storage storage = new Storage(this.imageDir);
-localImageVersion = storage.getImageSeq();
+localImageVersion = storage.getLatestImageSeq();
try {
URL infoUrl = new URL("http://"; + helperNode.first + ":" +
Config.http_port + "/info");
@@ -1617,7 +1617,7 @@ public class Catalog {
LOG.info("image does not exist: {}", curFile.getAbsolutePath());
return;
}
-replayedJournalId.set(storage.getImageSeq());
+replayedJournalId.set(storage.getLatestImageSeq());
MetaReader.read(curFile, this);
}
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/httpv2/controller/HaController.java
b/fe/fe-core/src/main/java/org/apache/doris/httpv2/controller/HaController.java
index cc983ad6e8..515c933e86 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/httpv2/controller/HaController.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/httpv2/controller/HaController.java
@@ -134,7 +134,7 @@ public class HaController {
Map checkPoint = new HashMap<>();
Storage storage = new Storage(Config.meta_dir + "/image");
checkPoint.put("Name", "Version");
-checkPoint.put("Value", storage.getImageSeq());
+checkPoint.put("Value", storage.getLatestImageSeq());
list.add(checkPoint);
long lastCheckpointTime =
storage.getCurrentImageFile().lastModified();
Date date = new Date(lastCheckpointTime);
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
b/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
index 22796efb42..48d0e4720f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/httpv2/meta/MetaService.java
@@ -111,7 +111,7 @@ public class MetaService extends RestBaseController {
try {
Storage currentStorageInfo = new
Storage(imageDir.getAbsolutePath());
StorageInfo storageInfo = new
StorageInfo(currentStorageInfo.getClusterID(),
-currentStorageInfo.getImageSeq(),
currentStorageInfo.getEditsSeq());
+currentStorageInfo.getLatestImageSeq(),
currentStorageInfo.getEditsSeq());
return ResponseEntityBuilder.ok(storageInfo);
} catch (IOException e) {
return ResponseEntityBuilder.internalError(e.getMessage());
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/ShowAction.java
b/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/ShowAction.java
index e08ebe5fd7..e3543db693 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/ShowAction.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/httpv2/rest/ShowAction.java
@@ -242,7 +242,7 @@ public class ShowAction extends RestBaseController {
feInfo.put("is_ready",
String.valueOf(Catalog.getCurrentCatalog().isReady()));
Storage storage = new Storage(Config.meta_dir + "/image");
-feInfo.put("last_checkpoint_version",
String.valueOf(storage.getImageSeq()));
+feInfo.put(&quo
[incubator-doris] branch master updated: [Enhancement] (image) check image validity as soon as generated (#9011)
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new bdf915abd4 [Enhancement] (image) check image validity as soon as
generated (#9011)
bdf915abd4 is described below
commit bdf915abd4e0d65e913d4caf9a8ea3ced88e85ab
Author: Henry2SS <45096548+henry...@users.noreply.github.com>
AuthorDate: Mon Apr 25 19:35:41 2022 +0800
[Enhancement] (image) check image validity as soon as generated (#9011)
* load newly generated image file as soon as generated to check if it is
valid.
* delete the latest invalid image file
* fix
* fix
* get filePath from saveImage() to ensure deleting the correct file while
exception happens
* fix
Co-authored-by: wuhangze
---
.../java/org/apache/doris/catalog/Catalog.java | 4 ++-
.../java/org/apache/doris/master/Checkpoint.java | 34 +-
.../java/org/apache/doris/persist/MetaCleaner.java | 11 +++
3 files changed, 47 insertions(+), 2 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
index 09a06a1596..e5e2914857 100755
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
@@ -1950,7 +1950,8 @@ public class Catalog {
}
// Only called by checkpoint thread
-public void saveImage() throws IOException {
+// return the latest image file's absolute path
+public String saveImage() throws IOException {
// Write image.ckpt
Storage storage = new Storage(this.imageDir);
File curFile = storage.getImageFile(replayedJournalId.get());
@@ -1963,6 +1964,7 @@ public class Catalog {
curFile.delete();
throw new IOException();
}
+return curFile.getAbsolutePath();
}
public void saveImage(File curFile, long replayedJournalId) throws
IOException {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java
b/fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java
index 7865817a82..2d3c3ac271 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/master/Checkpoint.java
@@ -35,6 +35,8 @@ import org.apache.doris.system.Frontend;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
+import com.google.common.base.Strings;
+
import java.io.IOException;
import java.io.OutputStream;
import java.net.HttpURLConnection;
@@ -111,6 +113,8 @@ public class Checkpoint extends MasterDaemon {
catalog = Catalog.getCurrentCatalog();
catalog.setEditLog(editLog);
createStaticFieldForCkpt();
+boolean exceptionCaught = false;
+String latestImageFilePath = null;
try {
catalog.loadImage(imageDir);
catalog.replayJournal(checkPointVersion);
@@ -119,13 +123,25 @@ public class Checkpoint extends MasterDaemon {
checkPointVersion, catalog.getReplayedJournalId()));
}
catalog.fixBugAfterMetadataReplayed(false);
-catalog.saveImage();
+latestImageFilePath = catalog.saveImage();
replayedJournalId = catalog.getReplayedJournalId();
+
+// destroy checkpoint catalog, reclaim memory
+catalog = null;
+Catalog.destroyCheckpoint();
+destroyStaticFieldForCkpt();
+
+// Load image to verify if the newly generated image file is valid
+// If success, do all the following jobs
+// If failed, just return
+catalog = Catalog.getCurrentCatalog();
+catalog.loadImage(imageDir);
if (MetricRepo.isInit) {
MetricRepo.COUNTER_IMAGE_WRITE_SUCCESS.increase(1L);
}
LOG.info("checkpoint finished save image.{}", replayedJournalId);
} catch (Throwable e) {
+exceptionCaught = true;
e.printStackTrace();
LOG.error("Exception when generate new image file", e);
if (MetricRepo.isInit) {
@@ -137,6 +153,22 @@ public class Checkpoint extends MasterDaemon {
catalog = null;
Catalog.destroyCheckpoint();
destroyStaticFieldForCkpt();
+// if new image generated && exception caught, delete the latest
image here
+// delete the newest image file, cuz it is invalid
+if ((!Strings.isNullOrEmpty(latestImageFilePath)) &&
exceptionCaught) {
+MetaCleaner cleaner = new MetaCleaner(Config.meta_dir +
"/image");
+try {
+
[incubator-doris-website] branch master created (now e6b8fd2)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git. at e6b8fd2 first commit This branch includes the following new commits: new e6b8fd2 first commit The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] 01/01: blank branch
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git commit dab11cd691c044ce6579d08adc59eed5fc5a1d37 Author: lide AuthorDate: Tue Sep 25 19:34:55 2018 +0800 blank branch --- README.md | 43 +++ 1 file changed, 43 insertions(+) diff --git a/README.md b/README.md new file mode 100644 index 000..dec66d3 --- /dev/null +++ b/README.md @@ -0,0 +1,43 @@ +# Apache Incubator Website + +## Prerequisites + +The website is built using JBake and a Groovy template. +The builds for the website do require internet access. + +- Install JBake from http://jbake.org/download.html +- Create an environment variable `JBAKE_HOME` pointing to your JBake installation +- Ensure that you have a JVM locally, e.g. [OpenJDK](http://openjdk.java.net/install/) + +## Clone the Source code + +If you have write access to incubator, you should clone it from https://git-wip-us.apache.org/repos/asf/incubator.git +otherwise you can fork from our github mirror https://github.com/apache/incubator and raise a pull request. + +## Building & Running the site + +There is a custom `bake.sh` file that is used to build the website. +You can call it with any of the [arguments you would pass to jbake](http://jbake.org/docs/2.5.1/#bake_command). +The easiest way to use it is to run `./bake.sh -b -s` +This will start up JBake in a watching mode as you make changes it will refresh after a short period of time. +While working with it locally, you'll notice that the site URLs redirect to `incubator.apache.org`; +to change this edit `jbake.properties` and uncomment the line referencing `localhost` + +## Automatic build and publishing - Jenkins Setup + +Commits to the `master` branch are automatically checked out and built using `build_site.sh`. + +The corresponding jenkins job can be found at [https://builds.apache.org/view/H-L/view/Incubator/job/Incubator%20Site/](https://builds.apache.org/view/H-L/view/Incubator/job/Incubator%20Site/) + +The result of the website build are pushed to the `asf-site` branch which are then published automatically using `gitwcsub` + +## Asciidoctor + +Most of the pages in the site are written using Asciidoctor. +While it is a form of asciidoc it does have some [syntax differences that are worth reviewing](http://asciidoctor.org/docs/asciidoc-syntax-quick-reference/) + +## Groovy Templates + +The site templates are written in groovy scripts. +Even though the files end with `.gsp` they are not GSP files and do not have access to tag libraries. +You can run custom code in them, similar to what is done in [homepage.gsp](templates/homepage.gsp) . - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site created (now dab11cd)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git. at dab11cd blank branch This branch includes the following new commits: new dab11cd blank branch The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: fixed build error
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new d9f1f4b fixed build error
d9f1f4b is described below
commit d9f1f4b237dea873efed2a0fc9173ce4be355669
Author: lide
AuthorDate: Wed Sep 26 16:43:06 2018 +0800
fixed build error
---
bake.sh | 4 ++--
jbake.properties | 6 +++---
2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/bake.sh b/bake.sh
index 2ad5cfb..faa65c2 100755
--- a/bake.sh
+++ b/bake.sh
@@ -1,3 +1,3 @@
#!/bin/bash
-
-java -Djavax.xml.accessExternalDTD=http -jar "${JBAKE_HOME}/jbake-core.jar" $@
+export JBAKE_OPTS="-Djavax.xml.accessExternalDTD=http"
+jbake $@
diff --git a/jbake.properties b/jbake.properties
index e1a4662..546ebe6 100644
--- a/jbake.properties
+++ b/jbake.properties
@@ -1,5 +1,5 @@
-site.host=https://incubator.apache.org
-#site.host=http://localhost:8820
+#site.host=https://incubator.apache.org
+site.host=http://doris.incubator.apache.org
render.tags=false
render.sitemap=true
template.homepage.file=homepage.gsp
@@ -18,4 +18,4 @@ template.policy.file=policy.gsp
render.index=false
index.file=index.html
content.folder=pages
-destination.folder=content
\ No newline at end of file
+destination.folder=content
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: restore bake.sh
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 902e746 restore bake.sh 902e746 is described below commit 902e7460345c8883e7b2cf2343e7a7b480b5892e Author: lide AuthorDate: Sat Sep 29 12:37:15 2018 +0800 restore bake.sh --- bake.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bake.sh b/bake.sh index 1e009c5..faa65c2 100755 --- a/bake.sh +++ b/bake.sh @@ -1,3 +1,3 @@ #!/bin/bash export JBAKE_OPTS="-Djavax.xml.accessExternalDTD=http" -jbake -b -s $@ +jbake $@ - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated (a86fc58 -> 5bcc114)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git. from a86fc58 updated menu, homepage, faq and sql_reference add 5bcc114 updated Documents No new revisions were added by this update. Summary of changes: assets/images/architecture.png | Bin 97686 -> 176951 bytes pages/guides/sql_reference.md | 4507 pages/guides/tutorial.md | 706 --- 3 files changed, 5213 deletions(-) delete mode 100644 pages/guides/sql_reference.md delete mode 100644 pages/guides/tutorial.md - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: change architecute from png to jpg
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new b5a1a00 change architecute from png to jpg b5a1a00 is described below commit b5a1a0043baf990e28634f87a977cf0f8aa1fe8e Author: lide AuthorDate: Wed Oct 24 15:02:41 2018 +0800 change architecute from png to jpg --- assets/images/architecture.jpg | Bin 0 -> 62356 bytes pages/index.ad | 2 +- 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/assets/images/architecture.jpg b/assets/images/architecture.jpg new file mode 100644 index 000..239f85b Binary files /dev/null and b/assets/images/architecture.jpg differ diff --git a/pages/index.ad b/pages/index.ad index 197fa60..ab7f568 100644 --- a/pages/index.ad +++ b/pages/index.ad @@ -25,7 +25,7 @@ Doris is a MPP-based interactive SQL data warehousing for reporting and analysis Doris’s implementation consists of two daemons: Frontend (FE) and Backend (BE). -image::architecture.png[] +image::architecture.jpg[] **Frontend daemon** consists of query coordinator and catalog manager. Query coordinator is responsible for receiving users’ sql queries, compiling queries and managing queries execution. Catalog manager is responsible for managing metadata such as databases, tables, partitions, replicas and etc. Several frontend daemons could be deployed to guarantee fault-tolerance, and load balancing. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 5a620fb Automatic Site Publish by git-site-role 5a620fb is described below commit 5a620fb3dcb251a7f1c45521bf5694254348a585 Author: lide AuthorDate: Wed Oct 24 15:05:21 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml| 4 ++-- content/images/architecture.jpg | Bin 0 -> 62356 bytes content/index.html | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index 1bf5cf2..d5ff7c4 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.incubator.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Wed, 24 Oct 2018 14:41:41 +0800 -Wed, 24 Oct 2018 14:41:41 +0800 +Wed, 24 Oct 2018 15:05:15 +0800 +Wed, 24 Oct 2018 15:05:15 +0800 diff --git a/content/images/architecture.jpg b/content/images/architecture.jpg new file mode 100644 index 000..239f85b Binary files /dev/null and b/content/images/architecture.jpg differ diff --git a/content/index.html b/content/index.html index c410981..3796555 100644 --- a/content/index.html +++ b/content/index.html @@ -100,7 +100,7 @@ - http://doris.incubator.apache.org/../images/architecture.png"; alt="architecture"> + http://doris.incubator.apache.org/../images/architecture.jpg"; alt="architecture"> - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: change README.md
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 6924c51 change README.md 6924c51 is described below commit 6924c51113a252181fbdfdb885570e305a236703 Author: lide AuthorDate: Wed Oct 24 17:55:04 2018 +0800 change README.md --- README.md | 115 +++--- 1 file changed, 87 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index dec66d3..86d9d5f 100644 --- a/README.md +++ b/README.md @@ -1,43 +1,102 @@ -# Apache Incubator Website +# Doris 网站修改编辑说明 -## Prerequisites +目前第一个版本主要内容来自于 github 的wiki。 -The website is built using JBake and a Groovy template. -The builds for the website do require internet access. +## 1 基本信息 +### 1.1 网站地址 -- Install JBake from http://jbake.org/download.html -- Create an environment variable `JBAKE_HOME` pointing to your JBake installation -- Ensure that you have a JVM locally, e.g. [OpenJDK](http://openjdk.java.net/install/) +<http://doris.apache.org> -## Clone the Source code +<http://doris.incubator.apache.org> -If you have write access to incubator, you should clone it from https://git-wip-us.apache.org/repos/asf/incubator.git -otherwise you can fork from our github mirror https://github.com/apache/incubator and raise a pull request. +### 1.2 网站源码 -## Building & Running the site +<https://github.com/apache/incubator-doris-website> -There is a custom `bake.sh` file that is used to build the website. -You can call it with any of the [arguments you would pass to jbake](http://jbake.org/docs/2.5.1/#bake_command). -The easiest way to use it is to run `./bake.sh -b -s` -This will start up JBake in a watching mode as you make changes it will refresh after a short period of time. -While working with it locally, you'll notice that the site URLs redirect to `incubator.apache.org`; -to change this edit `jbake.properties` and uncomment the line referencing `localhost` +## 2 编辑修改网站 -## Automatic build and publishing - Jenkins Setup +### 2.1 先决条件 -Commits to the `master` branch are automatically checked out and built using `build_site.sh`. +网站是通过 JBake 工具和一个 Groovy 模板建立的。 +因此必须事先安装 JBake 和 JDK。 -The corresponding jenkins job can be found at [https://builds.apache.org/view/H-L/view/Incubator/job/Incubator%20Site/](https://builds.apache.org/view/H-L/view/Incubator/job/Incubator%20Site/) +JBake 工具下载安装地址和相应说明请参考： +<https://jbake.org/download.html> -The result of the website build are pushed to the `asf-site` branch which are then published automatically using `gitwcsub` +### 2.2 下载源码到本地 -## Asciidoctor +github地址： -Most of the pages in the site are written using Asciidoctor. -While it is a form of asciidoc it does have some [syntax differences that are worth reviewing](http://asciidoctor.org/docs/asciidoc-syntax-quick-reference/) +<https://github.com/apache/incubator-doris-website> + +### 2.3 编辑修改 + +此网站的源码中将内容和模板分开来。 + +网页的形式是通过 templates 中定义的 gsp 文件模板规定的。 + +网页的内容则存放在 pages 目录中。 + +一般编辑修改直接修改 pages 中的文件就行。其中 Documents 菜单中的内容是 pages/guides 中的内容决定的。 + +网站源码中比较重要的目录文件列出如下： + +``` +. +├── README.md // 网站源码的说明 +├── assets // 这里主要是css和图片素材资源 +│ ├── css +│ └── images // 所有的图片：jpg或者png +│ └── apache-incubator-logo.png +├── bake.sh // 本地编译和启动网站脚本 +├── build_site.sh // 自动编译和提交，会调用bake.sh +├── incubator-site-groovy.iml +├── jbake.properties// JBake 属性定义 +├── pages // 网站内容： ad和md分别对应ASCII码文档和markdown文档 +│ ├── community.ad +│ ├── downloads.ad +│ ├── faq.ad +│ ├── guides // 当前的配置下，这个目录下的文件会自动加载，参看menu.gsp +│ │ ├── sql_reference.ad +│ │ └── tutorial.ad +│ ├── index.ad// 主页 +│ └── whoweare.ad +└── templates // 模板所在目录 +├── archive.gsp +├── feed.gsp +├── footer.gsp +├── guide.gsp +├── header.gsp +├── homepage.gsp +├── menu.gsp // 这个是主菜单配置 +├── page.gsp +├── simplepage.gsp // 目前大部分网页依赖的模板 +├── sitemap.gsp +└── tags.gsp + +``` + +### 2.4 在本地测试 + +通过下面的命令可以编译，并在本地启动网站服务，之后就可以通过 http://localhost:8820/ 访问了。 + +``` +sh bake.sh -b -s . +``` + + +## 3 上线 +### 3.1 提交修改 +上线的源代码必须 push 到 github 下面仓库的 master 分支上去： + +<https://github.com/apache/incubator-doris-website> + +两种方式：自己提交或者提交 pull request + +目前有两个分支：master 和 asf-site +前者是源代码，后者是部署目录。 + +### 3.2 编译和上线 +sh build_site.sh 编译网站自动提交上线，之后就可以查看更新了。 -## Groovy Templates -The site templates are written in groovy scripts. -Even though the files end with `.gsp` they are not GSP files and d
[incubator-doris-website] branch master updated: updated downloads file
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 7002a90 updated downloads file 7002a90 is described below commit 7002a90d76bdf1c2296310c758f4d518cdd0689e Author: lide AuthorDate: Fri Oct 26 17:04:36 2018 +0800 updated downloads file --- pages/downloads.ad | 66 -- pages/downloads.md | 133 + 2 files changed, 133 insertions(+), 66 deletions(-) diff --git a/pages/downloads.ad b/pages/downloads.ad deleted file mode 100644 index 155c868..000 --- a/pages/downloads.ad +++ /dev/null @@ -1,66 +0,0 @@ -//Licensed under the Apache License, Version 2.0 (the "License"); -//you may not use this file except in compliance with the License. -//You may obtain a copy of the License at -// -//http://www.apache.org/licenses/LICENSE-2.0 -// -//Unless required by applicable law or agreed to in writing, software -//distributed under the License is distributed on an "AS IS" BASIS, -//WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -//See the License for the specific language governing permissions and -//limitations under the License. -= Apache Doris Downloads -Apache Doris Downloads -2018-09-29 -:jbake-type: simplepage -:jbake-status: published -:idprefix: -:toc: -:imagesdir: ../images/ - -= Stable release -The current Doris stable release is 0.8.2. This version is recommended for production use. - -* link:https://github.com/apache/incubator-doris/releases/download/v0.8.2/palo-0.8.2-release-20180824.tar.gz[palo-0.8.2-release-20180824.tar.gz] 184 MB - -* link:https://github.com/apache/incubator-doris/archive/v0.8.2.zip[Source code (zip)] - -* link:https://github.com/apache/incubator-doris/archive/v0.8.2.tar.gz[Source code (tar.gz)] - -== Change Log - -1. Performance improvement -* Support vectorized execution engine. -* Support aggregation operator parallelization. -* Optimized storage engine for more efficient data fetching. - -2. Table-level privileges support -* User can grant fine-grained privileges on specified tables to specified user from specified host. - -3. Backup and restore -* User can backup snapshot of data to HDFS and restore snapshot to other Palo cluster. - -4. Broker supports HDFS HA and Hadoop kerberos authentication. - -5. Using [BRPC](https://github.com/brpc/brpc) instead of the old RPC framework. - -6. A lot of bugs fixed. - -7. Other changes: - -* Rewrite http server in Backend using Libevent to replace Mongoose. -* Remove mysql code to avoid copyright problem. -* More metrics to expose internal situation of Palo system. -* Same Frontend can be removed and added again without the changing port. -* Change the format of Frontend's audit log for parsing it more conveniently. - -== Compatibility - -**This release version DOES NOT support rolling upgrade, and CAN NOT rollback after upgrade. So it is HIGHLY RECOMMENDED to backup your data and meta data before upgrading to this release version. Or upgrade it in your test/pre-online Palo cluster to make sufficient test.** - -|Item|Content| -|---|---| -|Forward Compatibility | 0.8.1, 0.8.0 | -|Rolling Upgrade | No | -|Rollback | No | - diff --git a/pages/downloads.md b/pages/downloads.md new file mode 100644 index 000..5f4e705 --- /dev/null +++ b/pages/downloads.md @@ -0,0 +1,133 @@ +title=Apache Doris Downloads +date=2018-10-26 +type=simplepage +status=published +~~ + + + +## Current Release + +### 0.8.2.1 + +NOTICE: + +These is not an official Apache release version of Doris. We still same have works to do release a real Apache version. +But this does not affect the use of this release version in a production environment. + + Download: + +* [palo-0.8.2.1-release-20181026.tar.gz](http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181026.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-26T08:50:53Z/-1/host/3538578971c0a1e7cbc8e355f86a8bd55b278c211ff8be366679167227e9c129) [183MB] +* source code is associate with commit `4f6f857` + + Change Log + +1. Added: + + * Add 2 new proc '/current\_queries' and '/current\_backend\_instances' to monitor the current running queries. + * Add a manual compaction api on Backend to trigger cumulative or base compaction manually. + * Add Frontend config 'max\_bytes\_per\_broker\_scanner' to limit to bytes per one broker scanner. This is to limit the memory cost of a single broker load job. + * Add Frontend config 'max\_unfinished\_load\_job' to limit load job number: if number of running load jobs exceed the limit, no more load job is allowed to be submmitted. + * exposure backend info to user w
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 790ff1e Automatic Site Publish by git-site-role 790ff1e is described below commit 790ff1e3f32a6d62f4e5110cd24dda1470e08beb Author: lide AuthorDate: Fri Oct 26 17:06:10 2018 +0800 Automatic Site Publish by git-site-role --- content/downloads.html | 202 ++--- content/feed.xml | 4 +- content/sitemap.xml| 6 +- 3 files changed, 131 insertions(+), 81 deletions(-) diff --git a/content/downloads.html b/content/downloads.html index eb7a25b..a176966 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -77,82 +77,132 @@ Apache Doris Downloads - - - Table of Contents - - - Stable release - -Change Log -Compatibility - - - -Stable release - - The current Doris stable release is 0.8.2. This version is recommended for production use. - - - - https://github.com/apache/incubator-doris/releases/download/v0.8.2/palo-0.8.2-release-20180824.tar.gz";>palo-0.8.2-release-20180824.tar.gz 184 MB - https://github.com/apache/incubator-doris/archive/v0.8.2.zip";>Source code (zip) - https://github.com/apache/incubator-doris/archive/v0.8.2.tar.gz";>Source code (tar.gz) - - - - Change Log - - - - Performance improvement - - -Support vectorized execution engine. -Support aggregation operator parallelization. -Optimized storage engine for more efficient data fetching. - - - Table-level privileges support - - -User can grant fine-grained privileges on specified tables to specified user from specified host. - - - Backup and restore - - -User can backup snapshot of data to HDFS and restore snapshot to other Palo cluster. - - - Broker supports HDFS HA and Hadoop kerberos authentication. - Using [BRPC](https://github.com/brpc/brpc"; class="bare">https://github.com/brpc/brpc) instead of the old RPC framework. - A lot of bugs fixed. - Other changes: - - -Rewrite http server in Backend using Libevent to replace Mongoose. -Remove mysql code to avoid copyright problem. -More metrics to expose internal situation of Palo system. -Same Frontend can be removed and added again without the changing port. -Change the format of Frontend’s audit log for parsing it more conveniently. - - - - - - - - Compatibility - - - This release version DOES NOT support rolling upgrade, and CAN NOT rollback after upgrade. So it is HIGHLY RECOMMENDED to backup your data and meta data before upgrading to this release version. Or upgrade it in your test/pre-online Palo cluster to make sufficient test. - - - |Item|Content| |---|---| |Forward Compatibility | 0.8.1, 0.8.0 | |Rolling Upgrade | No | |Rollback | No | - - - + +Current Release +0.8.2.1 +NOTICE: +These is not an official Apache release version of Doris. We still same have works to do release a real Apache version. But this does not affect the use of this release version in a production environment. +Download: + + http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181026.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-26T08:50:53Z/-1/host/3538578971c0a1e7cbc8e355f86a8bd55b278c211ff8be366679167227e9c129";>palo-0.8.2.1-release-20181026.tar.gz [183MB] + source code is associate with commit 4f6f857 + +Change Log + + Added: + + Add 2 new proc ‘/current_queries’ and ‘/current_backend_instances’ to monitor the current running queries. + Add a manual compaction api on Backend to trigger cumulative or base compaction manually. + Add Frontend config ‘max_bytes_per_broker_scanner’ to limit to bytes per one broker scanner. This is to limit the memory cost of a single broker load job. + Add Frontend config ‘max_unfinished_load_job’ to limit load job number: if number of running load jobs exceed the limit, no more load job is allowed to be submmitted. + exposure backend info to user when encounter error on Backend. for debugging it more convenient. + Add 3 new metrics of Backends: host_fd_metrics, process_fd_metrics and process_thread_metrics, to monitor open file number and thread number. + Support getting column size and precision info of table or view using JDBC. + + Updated: + + Hide password and other sensitive information in log and audit log. + Change the promethues type name GAUGE to lowercase, to fit the latest promethues version. + Backend ip saved in
[incubator-doris-website] branch master updated (7002a90 -> 52d28af)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git. from 7002a90 updated downloads file new 48ef4c9 test new 52d28af updated downloads.md The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: README.md | 1 - pages/downloads.md | 40 2 files changed, 20 insertions(+), 21 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] 02/02: updated downloads.md
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git commit 52d28af5058228e9ef64edb87ec8229960943488 Author: lide AuthorDate: Fri Oct 26 17:23:07 2018 +0800 updated downloads.md --- pages/downloads.md | 40 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/pages/downloads.md b/pages/downloads.md index 5f4e705..ec96de8 100644 --- a/pages/downloads.md +++ b/pages/downloads.md @@ -22,7 +22,7 @@ limitations under the License.--> NOTICE: -These is not an official Apache release version of Doris. We still same have works to do release a real Apache version. +These is not an official Apache release version of Doris. We still have some works to do to release a real Apache version. But this does not affect the use of this release version in a production environment. Download: @@ -35,43 +35,43 @@ But this does not affect the use of this release version in a production environ 1. Added: * Add 2 new proc '/current\_queries' and '/current\_backend\_instances' to monitor the current running queries. - * Add a manual compaction api on Backend to trigger cumulative or base compaction manually. - * Add Frontend config 'max\_bytes\_per\_broker\_scanner' to limit to bytes per one broker scanner. This is to limit the memory cost of a single broker load job. - * Add Frontend config 'max\_unfinished\_load\_job' to limit load job number: if number of running load jobs exceed the limit, no more load job is allowed to be submmitted. - * exposure backend info to user when encounter error on Backend. for debugging it more convenient. - * Add 3 new metrics of Backends: host\_fd\_metrics, process\_fd\_metrics and process\_thread\_metrics, to monitor open file number and thread number. + * Add a manual compaction api on Backend to trigger cumulative and base compaction manually. + * Add Frontend config 'max\_bytes\_per\_broker\_scanner' to limit the loading bytes per one broker scanner. This is to limit the memory cost of a single broker load job. + * Add Frontend config 'max\_unfinished\_load\_job' to limit load job number. If number of running load jobs exceed the limit, no more load job is allowed to be submmitted. + * Exposure backend info to user when encounter errors on Backend, for debugging it more convenient. + * Add 3 new metrics of Backends: host\_fd\_metrics, process\_fd\_metrics and process\_thread\_metrics, to monitor open files number and threads number. * Support getting column size and precision info of table or view using JDBC. 2. Updated: - * Hide password and other sensitive information in log and audit log. - * Change the promethues type name GAUGE to lowercase, to fit the latest promethues version. - * Backend ip saved in FE will be compared with BE's local ip when doing heartbeat, to avoid false positive heartbeat response. + * Hide password and other sensitive information in fe.log and fe.audit.log. + * Change the promethues type name 'GAUGE' to lowercase, to fit the latest promethues version. + * Backend ip saved in FE will be compared with BE's local ip when heartbeating, to avoid false positive heartbeat response. * Using version\_num of tablet instead of calculating nice value to select cumulative compaction candicates. 3. Fixed - * privilege logic error: + * Fix privilege logic error: 1. No one can set root password expect for root user itself. 2. NODE\_PRIV cannot be granted. - 3. ADMIN\_PRIV and GRANT\_PRIV can only be granted or revoked on *.*. + 3. ADMIN\_PRIV and GRANT\_PRIV can only be granted or revoked on \*.\*. 4. No one can modifly privs of default role 'operator' and 'admin'. 5. No user can be granted to role 'operator'. - * missing password and auth check when handling mini load request in Frontend. - * DomainResolver should start after Frontends transfer to a certain ROLE, not in Catalog construction methods. - * read null data twice + * Missing password and auth check when handling mini load request in Frontend. + * DomainResolver should start after Frontend transfering to a certain ROLE, not in Catalog construction methods. + * Fix a bug that read null data twice: When reading data with a null value, in some cases, the same data will be read twice by the storage engine, -resulting in a wrong result.The reason for this problem is that when splitting, +resulting in a wrong result. The reason for this problem is that when splitting, and the start key is the mini
[incubator-doris-website] 01/02: test
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git commit 48ef4c9bea53b89a4a1c3bdf1126316dc2c21bb5 Author: lide AuthorDate: Fri Oct 26 17:15:52 2018 +0800 test --- README.md | 1 - 1 file changed, 1 deletion(-) diff --git a/README.md b/README.md index 86d9d5f..1549e09 100644 --- a/README.md +++ b/README.md @@ -99,4 +99,3 @@ sh bake.sh -b -s . ### 3.2 编译和上线 sh build_site.sh 编译网站自动提交上线，之后就可以查看更新了。 - - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new d094fc4 Automatic Site Publish by git-site-role d094fc4 is described below commit d094fc43581f7915b43296212190846f441ce731 Author: lide AuthorDate: Fri Oct 26 17:24:03 2018 +0800 Automatic Site Publish by git-site-role --- content/downloads.html | 38 +++--- content/feed.xml | 4 ++-- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/content/downloads.html b/content/downloads.html index a176966..d78955b 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -91,7 +91,7 @@ limitations under the License.--> Current Release 0.8.2.1 NOTICE: -These is not an official Apache release version of Doris. We still same have works to do release a real Apache version. But this does not affect the use of this release version in a production environment. +These is not an official Apache release version of Doris. We still have some works to do to release a real Apache version. But this does not affect the use of this release version in a production environment. Download: http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181026.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-26T08:50:53Z/-1/host/3538578971c0a1e7cbc8e355f86a8bd55b278c211ff8be366679167227e9c129";>palo-0.8.2.1-release-20181026.tar.gz [183MB] @@ -102,42 +102,42 @@ limitations under the License.--> Added: Add 2 new proc ‘/current_queries’ and ‘/current_backend_instances’ to monitor the current running queries. - Add a manual compaction api on Backend to trigger cumulative or base compaction manually. - Add Frontend config ‘max_bytes_per_broker_scanner’ to limit to bytes per one broker scanner. This is to limit the memory cost of a single broker load job. - Add Frontend config ‘max_unfinished_load_job’ to limit load job number: if number of running load jobs exceed the limit, no more load job is allowed to be submmitted. - exposure backend info to user when encounter error on Backend. for debugging it more convenient. - Add 3 new metrics of Backends: host_fd_metrics, process_fd_metrics and process_thread_metrics, to monitor open file number and thread number. + Add a manual compaction api on Backend to trigger cumulative and base compaction manually. + Add Frontend config ‘max_bytes_per_broker_scanner’ to limit the loading bytes per one broker scanner. This is to limit the memory cost of a single broker load job. + Add Frontend config ‘max_unfinished_load_job’ to limit load job number. If number of running load jobs exceed the limit, no more load job is allowed to be submmitted. + Exposure backend info to user when encounter errors on Backend, for debugging it more convenient. + Add 3 new metrics of Backends: host_fd_metrics, process_fd_metrics and process_thread_metrics, to monitor open files number and threads number. Support getting column size and precision info of table or view using JDBC. Updated: - Hide password and other sensitive information in log and audit log. - Change the promethues type name GAUGE to lowercase, to fit the latest promethues version. - Backend ip saved in FE will be compared with BE’s local ip when doing heartbeat, to avoid false positive heartbeat response. + Hide password and other sensitive information in fe.log and fe.audit.log. + Change the promethues type name ‘GAUGE’ to lowercase, to fit the latest promethues version. + Backend ip saved in FE will be compared with BE’s local ip when heartbeating, to avoid false positive heartbeat response. Using version_num of tablet instead of calculating nice value to select cumulative compaction candicates. Fixed - privilege logic error: + Fix privilege logic error: No one can set root password expect for root user itself. NODE_PRIV cannot be granted. - ADMIN_PRIV and GRANT_PRIV can only be granted or revoked on .. + ADMIN_PRIV and GRANT_PRIV can only be granted or revoked on *.*. No one can modifly privs of default role ‘operator’ and ‘admin’. No user can be granted to role ‘operator’. - missing password and auth check when handling mini load request in Frontend. - DomainResolver should start after Frontends transfer to a certain ROLE, not in Catalog construction methods. - read null data twice When reading data with a null value, in some cases, the same data will be read twice by the storage engine, resulting in a wrong result.The reason for this problem is that when splitting, and the start key is the minimum value, the data with null is read. - fixed a mem leak of using ByteBu
[incubator-doris-website] branch master updated: Update 0.8.2.1 release download package
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 1ad9eff Update 0.8.2.1 release download package 1ad9eff is described below commit 1ad9efff7b984e7e116f8f2b0954f8203e368d7b Author: lide AuthorDate: Sat Oct 27 17:11:20 2018 +0800 Update 0.8.2.1 release download package --- pages/downloads.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/pages/downloads.md b/pages/downloads.md index ec96de8..1d49a1d 100644 --- a/pages/downloads.md +++ b/pages/downloads.md @@ -27,13 +27,14 @@ But this does not affect the use of this release version in a production environ Download: -* [palo-0.8.2.1-release-20181026.tar.gz](http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181026.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-26T08:50:53Z/-1/host/3538578971c0a1e7cbc8e355f86a8bd55b278c211ff8be366679167227e9c129) [183MB] -* source code is associate with commit `4f6f857` +* [palo-0.8.2.1-release-20181027.tar.gz](http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181027.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-27T09:06:12Z/-1/host/d9ebd01de7e266946e6dd1f2fb6951e92d4a8cd0a7b437253f9e0dcc3efe7039) [213.5MB] +* source code is associate with commit `e453fc` Change Log 1. Added: + * Using MAVEN instead of ANT to build Frontend and Apache HDFS broker project. * Add 2 new proc '/current\_queries' and '/current\_backend\_instances' to monitor the current running queries. * Add a manual compaction api on Backend to trigger cumulative and base compaction manually. * Add Frontend config 'max\_bytes\_per\_broker\_scanner' to limit the loading bytes per one broker scanner. This is to limit the memory cost of a single broker load job. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new a6db786 Automatic Site Publish by git-site-role a6db786 is described below commit a6db7869eec221d4d69b905c8d845efa4aaad84c Author: lide AuthorDate: Sat Oct 27 17:11:59 2018 +0800 Automatic Site Publish by git-site-role --- content/downloads.html | 5 +++-- content/feed.xml | 4 ++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/content/downloads.html b/content/downloads.html index d78955b..16d9338 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -94,13 +94,14 @@ limitations under the License.--> These is not an official Apache release version of Doris. We still have some works to do to release a real Apache version. But this does not affect the use of this release version in a production environment. Download: - http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181026.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-26T08:50:53Z/-1/host/3538578971c0a1e7cbc8e355f86a8bd55b278c211ff8be366679167227e9c129";>palo-0.8.2.1-release-20181026.tar.gz [183MB] - source code is associate with commit 4f6f857 + http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181027.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-27T09:06:12Z/-1/host/d9ebd01de7e266946e6dd1f2fb6951e92d4a8cd0a7b437253f9e0dcc3efe7039";>palo-0.8.2.1-release-20181027.tar.gz [213.5MB] + source code is associate with commit e453fc Change Log Added: + Using MAVEN instead of ANT to build Frontend and Apache HDFS broker project. Add 2 new proc ‘/current_queries’ and ‘/current_backend_instances’ to monitor the current running queries. Add a manual compaction api on Backend to trigger cumulative and base compaction manually. Add Frontend config ‘max_bytes_per_broker_scanner’ to limit the loading bytes per one broker scanner. This is to limit the memory cost of a single broker load job. diff --git a/content/feed.xml b/content/feed.xml index 87e026a..b5325e9 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.incubator.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Fri, 26 Oct 2018 17:23:45 +0800 -Fri, 26 Oct 2018 17:23:45 +0800 +Sat, 27 Oct 2018 17:11:54 +0800 +Sat, 27 Oct 2018 17:11:54 +0800 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: rollback the release of 0.8.2.1, it not a right way to release a version in apache way
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new f5bbb07 rollback the release of 0.8.2.1, it not a right way to release a version in apache way f5bbb07 is described below commit f5bbb078b2ac6bec8baa1e207afc93689e19a123 Author: lide AuthorDate: Sat Oct 27 17:41:09 2018 +0800 rollback the release of 0.8.2.1, it not a right way to release a version in apache way --- pages/downloads.md | 64 ++ 1 file changed, 2 insertions(+), 62 deletions(-) diff --git a/pages/downloads.md b/pages/downloads.md index 1d49a1d..8cc568e 100644 --- a/pages/downloads.md +++ b/pages/downloads.md @@ -18,71 +18,11 @@ limitations under the License.--> ## Current Release -### 0.8.2.1 - -NOTICE: - -These is not an official Apache release version of Doris. We still have some works to do to release a real Apache version. -But this does not affect the use of this release version in a production environment. - - Download: - -* [palo-0.8.2.1-release-20181027.tar.gz](http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181027.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-27T09:06:12Z/-1/host/d9ebd01de7e266946e6dd1f2fb6951e92d4a8cd0a7b437253f9e0dcc3efe7039) [213.5MB] -* source code is associate with commit `e453fc` - - Change Log - -1. Added: - - * Using MAVEN instead of ANT to build Frontend and Apache HDFS broker project. - * Add 2 new proc '/current\_queries' and '/current\_backend\_instances' to monitor the current running queries. - * Add a manual compaction api on Backend to trigger cumulative and base compaction manually. - * Add Frontend config 'max\_bytes\_per\_broker\_scanner' to limit the loading bytes per one broker scanner. This is to limit the memory cost of a single broker load job. - * Add Frontend config 'max\_unfinished\_load\_job' to limit load job number. If number of running load jobs exceed the limit, no more load job is allowed to be submmitted. - * Exposure backend info to user when encounter errors on Backend, for debugging it more convenient. - * Add 3 new metrics of Backends: host\_fd\_metrics, process\_fd\_metrics and process\_thread\_metrics, to monitor open files number and threads number. - * Support getting column size and precision info of table or view using JDBC. - -2. Updated: - - * Hide password and other sensitive information in fe.log and fe.audit.log. - * Change the promethues type name 'GAUGE' to lowercase, to fit the latest promethues version. - * Backend ip saved in FE will be compared with BE's local ip when heartbeating, to avoid false positive heartbeat response. - * Using version\_num of tablet instead of calculating nice value to select cumulative compaction candicates. - -3. Fixed - - * Fix privilege logic error: - 1. No one can set root password expect for root user itself. - 2. NODE\_PRIV cannot be granted. - 3. ADMIN\_PRIV and GRANT\_PRIV can only be granted or revoked on \*.\*. - 4. No one can modifly privs of default role 'operator' and 'admin'. - 5. No user can be granted to role 'operator'. - * Missing password and auth check when handling mini load request in Frontend. - * DomainResolver should start after Frontend transfering to a certain ROLE, not in Catalog construction methods. - * Fix a bug that read null data twice: -When reading data with a null value, in some cases, the same data will be read twice by the storage engine, -resulting in a wrong result. The reason for this problem is that when splitting, -and the start key is the minimum value, the data with null is read. - * Fixed a mem leak of using ByteBuf when parsing auth info of http request. - * Backend process should crash if failed to saving tablet's header. - * Should remove fd from map when input stream or output stream is closed in Broker process. - * Predicates should not be pushed down to subquery which contains limit clause. - * Fix the formula of calculating BE load score. - * Fix a bug that in some edge cases, non-master Fontend may wait for a unnecessary long timeout after forwarding cmd to Master FE. - * Fix a bug that granting privs on more than one table does not work. - * Support 'Insert into' table which contains HLL columns. - * ExportStmt's toSql() method may throw NullPointer Exception if table does not exist. - * Remove unnecessary 'get capacity' operation to avoid IO impact. - - Compatibility - -This release
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 8a9a16a Automatic Site Publish by git-site-role 8a9a16a is described below commit 8a9a16af0bca8fdc99d722486510f49317e43ca4 Author: lide AuthorDate: Sat Oct 27 17:41:37 2018 +0800 Automatic Site Publish by git-site-role --- content/downloads.html | 58 ++ content/feed.xml | 4 ++-- 2 files changed, 4 insertions(+), 58 deletions(-) diff --git a/content/downloads.html b/content/downloads.html index 16d9338..954cb8a 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -89,63 +89,9 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.--> Current Release -0.8.2.1 -NOTICE: -These is not an official Apache release version of Doris. We still have some works to do to release a real Apache version. But this does not affect the use of this release version in a production environment. -Download: - - http://palo-opensource-bj.bj.bcebos.com/palo-0.8.2.1-release-20181027.tar.gz?authorization=bce-auth-v1/069fc2786e464e63a5f1183824ddb522/2018-10-27T09:06:12Z/-1/host/d9ebd01de7e266946e6dd1f2fb6951e92d4a8cd0a7b437253f9e0dcc3efe7039";>palo-0.8.2.1-release-20181027.tar.gz [213.5MB] - source code is associate with commit e453fc - -Change Log - - Added: - - Using MAVEN instead of ANT to build Frontend and Apache HDFS broker project. - Add 2 new proc ‘/current_queries’ and ‘/current_backend_instances’ to monitor the current running queries. - Add a manual compaction api on Backend to trigger cumulative and base compaction manually. - Add Frontend config ‘max_bytes_per_broker_scanner’ to limit the loading bytes per one broker scanner. This is to limit the memory cost of a single broker load job. - Add Frontend config ‘max_unfinished_load_job’ to limit load job number. If number of running load jobs exceed the limit, no more load job is allowed to be submmitted. - Exposure backend info to user when encounter errors on Backend, for debugging it more convenient. - Add 3 new metrics of Backends: host_fd_metrics, process_fd_metrics and process_thread_metrics, to monitor open files number and threads number. - Support getting column size and precision info of table or view using JDBC. - - Updated: - - Hide password and other sensitive information in fe.log and fe.audit.log. - Change the promethues type name ‘GAUGE’ to lowercase, to fit the latest promethues version. - Backend ip saved in FE will be compared with BE’s local ip when heartbeating, to avoid false positive heartbeat response. - Using version_num of tablet instead of calculating nice value to select cumulative compaction candicates. - - Fixed - - Fix privilege logic error: - - No one can set root password expect for root user itself. - NODE_PRIV cannot be granted. - ADMIN_PRIV and GRANT_PRIV can only be granted or revoked on *.*. - No one can modifly privs of default role ‘operator’ and ‘admin’. - No user can be granted to role ‘operator’. - - Missing password and auth check when handling mini load request in Frontend. - DomainResolver should start after Frontend transfering to a certain ROLE, not in Catalog construction methods. - Fix a bug that read null data twice: When reading data with a null value, in some cases, the same data will be read twice by the storage engine, resulting in a wrong result. The reason for this problem is that when splitting, and the start key is the minimum value, the data with null is read. - Fixed a mem leak of using ByteBuf when parsing auth info of http request. - Backend process should crash if failed to saving tablet’s header. - Should remove fd from map when input stream or output stream is closed in Broker process. - Predicates should not be pushed down to subquery which contains limit clause. - Fix the formula of calculating BE load score. - Fix a bug that in some edge cases, non-master Fontend may wait for a unnecessary long timeout after forwarding cmd to Master FE. - Fix a bug that granting privs on more than one table does not work. - Support ‘Insert into’ table which contains HLL columns. - ExportStmt’s toSql() method may throw NullPointer Exception if table does not exist. - Remove unnecessary ‘get capacity’ operation to avoid IO impact. - - -Compatibility -This release version support rolling upgrade ONLY from 0.8.2. The compatibility is same as 0.8.2. +Comming soon… History Release -0.8.2 +0.8.2 (20180824) Download https://github.com/apache/incubator-dor
[incubator-doris-website] branch master updated: add contributing.md to Documents
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 2ab825d add contributing.md to Documents
2ab825d is described below
commit 2ab825daf8257f2452887f42fef0fd98edd36ca4
Author: lide
AuthorDate: Thu Nov 1 10:24:40 2018 +0800
add contributing.md to Documents
---
pages/guides/contributing.md | 158 +++
1 file changed, 158 insertions(+)
diff --git a/pages/guides/contributing.md b/pages/guides/contributing.md
new file mode 100644
index 000..536cc23
--- /dev/null
+++ b/pages/guides/contributing.md
@@ -0,0 +1,158 @@
+title=Contributing
+date=2018-11-01
+type=guide
+status=published
+~~
+
+
+
+
+# Contributing to Apache Doris (incubating)
+
+Thanks for your interest in contributing.
+As you see from [GitHub] (https://github.com/apache/incubator-doris), Apache
Doris (incubating)’s codebase consists of two parts: frontend (FE) which
written in JAVA and backend (BE) which writte in C++.
+
+## How to contribute?
+
+Most of the contributions that we receive are code contributions, but you can
+also contribute to the documentation or simply report solid bugs
+for us to fix.
+
+When you contribute code, you affirm that the contribution is your original
work and that you license the work to the project under the project’s open
source license. Whether or not you state this explicitly, by submitting any
copyrighted material via pull request, email, or other means you agree to
license the material under the project’s open source license and warrant that
you have the legal authority to do so.
+
+## GitHub Workflow
+
+1. Fork the apache/incubator-doris repository into your GitHub account
+
+<https://github.com/apache/incubator-doris/fork>
+
+1. Clone your fork of the GitHub repository
+
+```sh
+git clone g...@github.com:/doris.git
+```
+
+replace `` with your GitHub username.
+
+1. Add a remote to keep up with upstream changes
+
+```
+git remote add upstream https://github.com/apache/incubator-doris.git
+```
+
+If you already have a copy, fetch upstream changes
+
+```
+git fetch upstream master
+```
+
+1. Create a feature branch to work in
+
+```
+git checkout -b feature-xxx remotes/upstream/master
+```
+
+1. _Before submitting a pull request_ periodically rebase your changes
+(but don't do it when a pull request is already submitted)
+
+```
+git pull --rebase upstream master
+```
+
+1. Before submitting a pull request, combine ("squash") related commits into a
single one
+
+```
+git rebase upstream/master
+```
+
+1. Submit a pull-request
+
+```
+git push origin feature-xxx
+```
+
+Go to your Doris fork main page
+
+```
+https://github.com//doris
+```
+
+If you recently pushed your changes GitHub will automatically pop up a
+`Compare & pull request` button for any branches you recently pushed to.
If you
+click that button it will automatically offer you to submit your
pull-request
+to the apache/incubator-doris repository.
+
+- Give your pull-request a meaningful title.
+- In the description, explain your changes and the problem they are
solving.
+
+1. Addressing code review comments
+
+Address code review comments by committing changes and pushing them to
your feature
+branch.
+
+```
+git push origin feature-xxx
+```
+
+### If your pull request shows conflicts with master
+ If your pull request shows conflicts with master, merge master into your
feature branch:
+
+
+ ```
+ git merge upstream/master
+ ```
+
+ and resolve the conflicts. After resolving conflicts, push your branch again:
+
+ ```
+ git push origin feature-xxx
+ ```
+
+ _Avoid rebasing and force pushes after submitting a pull request,_ since
these make it
+ difficult for reviewers to see what you've changed in response to their
reviews. The Doris
+ committer that merges your change will rebase and squash it into a single
commit before
+ committing it to master.
+
+## How to report a bug?
+
+* **Ensure the bug was not already reported** by searching on GitHub under
[Issues](https://github.com/apache/incubator-doris/issues).
+
+* If you're unable to find an open issue addressing the problem, [open a new
one](https://github.com/apache/incubator-doris/issues/new). Be sure to include
a **title and clear description**, as much relevant information as possible,
and a **code sample** or an **executable test case** demonstrating the expected
behavior that is not occurring.
+
+
+## How to add a new feature or change an existing one
+
+_Before making any significant changes, please [open an
issue](https://github.com/apache/incubator-doris/issues)._ Discussing your
proposed
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 04cc7de Automatic Site Publish by git-site-role 04cc7de is described below commit 04cc7dedb879f979ae7b3e326304293711df8c1f Author: lide AuthorDate: Thu Nov 1 10:27:39 2018 +0800 Automatic Site Publish by git-site-role --- content/archive.html | 2 + content/community.html| 2 + content/downloads.html| 2 + content/faq.html | 2 + content/feed.xml | 4 +- content/guides/contributing.html | 185 ++ content/guides/sql_reference.html | 2 + content/guides/tutorial.html | 2 + content/index.html| 2 + content/sitemap.xml | 5 ++ content/whoweare.html | 2 + 11 files changed, 208 insertions(+), 2 deletions(-) diff --git a/content/archive.html b/content/archive.html index e08d246..3e0496e 100644 --- a/content/archive.html +++ b/content/archive.html @@ -50,6 +50,8 @@ Documents + http://doris.incubator.apache.org/guides/contributing.html";>Contributing + http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial diff --git a/content/community.html b/content/community.html index 9e73e10..592d107 100644 --- a/content/community.html +++ b/content/community.html @@ -50,6 +50,8 @@ Documents + http://doris.incubator.apache.org/guides/contributing.html";>Contributing + http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial diff --git a/content/downloads.html b/content/downloads.html index 954cb8a..3a5518e 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -50,6 +50,8 @@ Documents + http://doris.incubator.apache.org/guides/contributing.html";>Contributing + http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial diff --git a/content/faq.html b/content/faq.html index 0c6109e..ff6827c 100644 --- a/content/faq.html +++ b/content/faq.html @@ -50,6 +50,8 @@ Documents + http://doris.incubator.apache.org/guides/contributing.html";>Contributing + http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial diff --git a/content/feed.xml b/content/feed.xml index 97c717b..e564844 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.incubator.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Sat, 27 Oct 2018 17:41:32 +0800 -Sat, 27 Oct 2018 17:41:32 +0800 +Thu, 1 Nov 2018 10:27:34 +0800 +Thu, 1 Nov 2018 10:27:34 +0800 diff --git a/content/guides/contributing.html b/content/guides/contributing.html new file mode 100644 index 000..ab80c05 --- /dev/null +++ b/content/guides/contributing.html @@ -0,0 +1,185 @@ + + + + +Contributing + + + + + + + + + + + + + + + + + + +https://www.apache.org/favicon.ico";> + + + + + + + + + + +Toggle navigation + + + + + http://doris.incubator.apache.org";>Apache Doris + + + + + Documents + + + http://doris.incubator.apache.org/guides/contributing.html";>Contributing + + http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference + + http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial + + + + http://doris.incubator.apache.org/community.html";>Community + http://doris.incubator.apache.org/downloads.html";>Downloads + h
[incubator-doris-website] branch master updated: update Community by add the link of contributing.md
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 18bdb4c update Community by add the link of contributing.md 18bdb4c is described below commit 18bdb4c96b40c4e6a5904742c861027ecdfe2b92 Author: lide AuthorDate: Thu Nov 1 10:36:25 2018 +0800 update Community by add the link of contributing.md --- jbake.properties | 2 +- pages/community.ad | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/jbake.properties b/jbake.properties index 7f0294e..5d11b5f 100644 --- a/jbake.properties +++ b/jbake.properties @@ -1,5 +1,5 @@ #site.host=https://incubator.apache.org -site.host=http://doris.incubator.apache.org +site.host=http://doris.apache.org render.tags=false render.sitemap=true template.homepage.file=homepage.gsp diff --git a/pages/community.ad b/pages/community.ad index 52745c2..6012b23 100644 --- a/pages/community.ad +++ b/pages/community.ad @@ -33,6 +33,8 @@ Most discussion about Doris happens over email and GitHub. = 2. Contributing Doris is a community-led project and we are delighted to receive contributions of anything from minor fixes to big new features. +See detail: link:http://doris.incubator.apache.org/guides/contributing.html[How to Contributing] + == 2.1 What to work on If you have an itch to scratch, then by all means do that! Fixing bugs you run into, or adding features you need, are both immensely helpful. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 6f0e1e1 Automatic Site Publish by git-site-role 6f0e1e1 is described below commit 6f0e1e16ac518aa855ff2e14d3fa1366fc0933dd Author: lide AuthorDate: Thu Nov 1 10:41:27 2018 +0800 Automatic Site Publish by git-site-role --- content/archive.html | 14 +++--- content/community.html| 17 ++--- content/downloads.html| 14 +++--- content/faq.html | 14 +++--- content/feed.xml | 8 content/guides/contributing.html | 14 +++--- content/guides/sql_reference.html | 14 +++--- content/guides/tutorial.html | 14 +++--- content/index.html| 16 content/sitemap.xml | 16 content/whoweare.html | 14 +++--- 11 files changed, 79 insertions(+), 76 deletions(-) diff --git a/content/archive.html b/content/archive.html index 3e0496e..32e8a42 100644 --- a/content/archive.html +++ b/content/archive.html @@ -42,7 +42,7 @@ - http://doris.incubator.apache.org";>Apache Doris + http://doris.apache.org";>Apache Doris @@ -50,17 +50,17 @@ Documents - http://doris.incubator.apache.org/guides/contributing.html";>Contributing + http://doris.apache.org/guides/contributing.html";>Contributing - http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference + http://doris.apache.org/guides/sql_reference.html";>SQL reference - http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial + http://doris.apache.org/guides/tutorial.html";>Tutorial - http://doris.incubator.apache.org/community.html";>Community - http://doris.incubator.apache.org/downloads.html";>Downloads - http://doris.incubator.apache.org/faq.html";>FAQs + http://doris.apache.org/community.html";>Community + http://doris.apache.org/downloads.html";>Downloads + http://doris.apache.org/faq.html";>FAQs diff --git a/content/community.html b/content/community.html index 592d107..32efdba 100644 --- a/content/community.html +++ b/content/community.html @@ -42,7 +42,7 @@ - http://doris.incubator.apache.org";>Apache Doris + http://doris.apache.org";>Apache Doris @@ -50,17 +50,17 @@ Documents - http://doris.incubator.apache.org/guides/contributing.html";>Contributing + http://doris.apache.org/guides/contributing.html";>Contributing - http://doris.incubator.apache.org/guides/sql_reference.html";>SQL reference + http://doris.apache.org/guides/sql_reference.html";>SQL reference - http://doris.incubator.apache.org/guides/tutorial.html";>Tutorial + http://doris.apache.org/guides/tutorial.html";>Tutorial - http://doris.incubator.apache.org/community.html";>Community - http://doris.incubator.apache.org/downloads.html";>Downloads - http://doris.incubator.apache.org/faq.html";>FAQs + http://doris.apache.org/community.html";>Community + http://doris.apache.org/downloads.html";>Downloads + http://doris.apache.org/faq.html";>FAQs @@ -124,6 +124,9 @@ Doris is a community-led project and we are delighted to receive contributions of anything from minor fixes to big new features. + + See detail: http://doris.incubator.apache.org/guides/contributing.html";>How to Contributing + 2.1 What to work on diff --git a/content/downloads.html b/content/downloads.html index 3a5518e..e1f7eec 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -42,7 +42,7 @@ - http://doris.incubator.apache.org";>Apache Doris + http://doris.apache.org";>Apache Doris @@ -50,
[incubator-doris-website] branch master updated: update contributing.md
This is an automated email from the ASF dual-hosted git repository.
lide pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 15d5e9a update contributing.md
15d5e9a is described below
commit 15d5e9ab37a2bbe52dd161e7f24320f8e6f3b5da
Author: lide
AuthorDate: Thu Nov 1 13:22:27 2018 +0800
update contributing.md
---
pages/guides/contributing.md | 165 +--
1 file changed, 82 insertions(+), 83 deletions(-)
diff --git a/pages/guides/contributing.md b/pages/guides/contributing.md
index 536cc23..4fd7a40 100644
--- a/pages/guides/contributing.md
+++ b/pages/guides/contributing.md
@@ -16,13 +16,12 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
express or implied.
See the License for the specific language governing permissions and
limitations under the License.-->
-
# Contributing to Apache Doris (incubating)
Thanks for your interest in contributing.
As you see from [GitHub] (https://github.com/apache/incubator-doris), Apache
Doris (incubating)’s codebase consists of two parts: frontend (FE) which
written in JAVA and backend (BE) which writte in C++.
-## How to contribute?
+## 1. How to contribute?
Most of the contributions that we receive are code contributions, but you can
also contribute to the documentation or simply report solid bugs
@@ -30,109 +29,109 @@ for us to fix.
When you contribute code, you affirm that the contribution is your original
work and that you license the work to the project under the project’s open
source license. Whether or not you state this explicitly, by submitting any
copyrighted material via pull request, email, or other means you agree to
license the material under the project’s open source license and warrant that
you have the legal authority to do so.
-## GitHub Workflow
+## 2. GitHub Workflow
+
+### 2.1 How to fork and pull request
-1. Fork the apache/incubator-doris repository into your GitHub account
+1) Fork the apache/incubator-doris repository into your GitHub account
<https://github.com/apache/incubator-doris/fork>
-1. Clone your fork of the GitHub repository
+2) Clone your fork of the GitHub repository
+
-```sh
-git clone g...@github.com:/doris.git
-```
+```
+git clone g...@github.com:/doris.git
+```
+Replace `` with your GitHub username.
-replace `` with your GitHub username.
-1. Add a remote to keep up with upstream changes
+3) Add a remote to keep up with upstream changes
-```
-git remote add upstream https://github.com/apache/incubator-doris.git
-```
+```
+git remote add upstream https://github.com/apache/incubator-doris.git
+```
-If you already have a copy, fetch upstream changes
+If you already have a copy, fetch upstream changes
-```
-git fetch upstream master
-```
+```
+git fetch upstream master
+```
-1. Create a feature branch to work in
+4) Create a feature branch to work in
-```
-git checkout -b feature-xxx remotes/upstream/master
-```
+```
+git checkout -b feature-xxx remotes/upstream/master
+```
-1. _Before submitting a pull request_ periodically rebase your changes
+5) Before submitting a pull request periodically rebase your changes
(but don't do it when a pull request is already submitted)
-```
-git pull --rebase upstream master
-```
+```
+git pull --rebase upstream master
+```
+
+6) Before submitting a pull request, combine ("squash") related commits into a
single one
+
+```
+git rebase upstream/master
+```
+
+7) Submit a pull-request
+
+```
+git push origin feature-xxx
+```
+
+Go to your Doris fork main page
+
+```
+https://github.com//doris
+```
-1. Before submitting a pull request, combine ("squash") related commits into a
single one
+If you recently pushed your changes GitHub will automatically pop up a
`Compare & pull request` button for any branches you recently pushed to. If you
click that button it will automatically offer you to submit your pull-request
to the apache/incubator-doris repository.
-```
-git rebase upstream/master
-```
+Give your pull-request a meaningful title.
-1. Submit a pull-request
+In the description, explain your changes and the problem they are solving.
-```
-git push origin feature-xxx
-```
+8) Addressing code review comments
-Go to your Doris fork main page
+Address code review comments by committing changes and pushing them to your
feature branch.
-```
-https://github.com//doris
-```
+```
+git push origin feature-xxx
+```
-If you recently pushed your changes GitHub will automatically pop up a
-`Compare & pull request` button for any branches you recently pushed to.
If you
-click that button it will automatically offer you to submit your
pull-request
-to the apache/incubator-doris
[incubator-doris-website] 01/02: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git commit 3fbc0833d115308f3e253ece66f6d685f36cb663 Author: lide AuthorDate: Thu Nov 1 13:24:36 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 +- content/guides/contributing.html | 109 ++- 2 files changed, 63 insertions(+), 50 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index 17a9669..6b65316 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Thu, 1 Nov 2018 10:41:21 +0800 -Thu, 1 Nov 2018 10:41:21 +0800 +Thu, 1 Nov 2018 13:24:29 +0800 +Thu, 1 Nov 2018 13:24:29 +0800 diff --git a/content/guides/contributing.html b/content/guides/contributing.html index e0d70c7..de9e457 100644 --- a/content/guides/contributing.html +++ b/content/guides/contributing.html @@ -96,70 +96,83 @@ See the License for the specific language governing permissions and limitations under the License.--> Contributing to Apache Doris (incubating) Thanks for your interest in contributing. As you see from https://github.com/apache/incubator-doris";>GitHub, Apache Doris (incubating)’s codebase consists of two parts: frontend (FE) which written in JAVA and backend (BE) which writte in C++. -How to contribute? +1. How to contribute? Most of the contributions that we receive are code contributions, but you can also contribute to the documentation or simply report solid bugs for us to fix. When you contribute code, you affirm that the contribution is your original work and that you license the work to the project under the project’s open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project’s open source license and warrant that you have the legal authority to do so. -GitHub Workflow - - Fork the apache/incubator-doris repository into your GitHub account https://github.com/apache/incubator-doris/fork";>https://github.com/apache/incubator-doris/fork - Clone your fork of the GitHub repository git clone g...@github.com:<username>/doris.git - replace <username> with your GitHub username. - Add a remote to keep up with upstream changes git remote add upstream https://github.com/apache/incubator-doris.git - If you already have a copy, fetch upstream changes git fetch upstream master - - Create a feature branch to work in git checkout -b feature-xxx remotes/upstream/master - - Before submitting a pull request periodically rebase your changes (but don’t do it when a pull request is already submitted) git pull --rebase upstream master - - Before submitting a pull request, combine (“squash”) related commits into a single one git rebase upstream/master - - Submit a pull-request git push origin feature-xxx - Go to your Doris fork main page https://github.com/<username>/doris - If you recently pushed your changes GitHub will automatically pop up a Compare & pull request button for any branches you recently pushed to. If you click that button it will automatically offer you to submit your pull-request to the apache/incubator-doris repository. - - Give your pull-request a meaningful title. - In the description, explain your changes and the problem they are solving. - - Addressing code review comments Address code review comments by committing changes and pushing them to your feature branch. git push origin feature-xxx - - -If your pull request shows conflicts with master +2. GitHub Workflow +2.1 How to fork and pull request +1) Fork the apache/incubator-doris repository into your GitHub account +<https://github.com/apache/incubator-doris/fork>; + +2) Clone your fork of the GitHub repository +git clone g...@github.com:<username>/doris.git + +Replace <username> with your GitHub username. +3) Add a remote to keep up with upstream changes +git remote add upstream https://github.com/apache/incubator-doris.git + +If you already have a copy, fetch upstream changes +git fetch upstream master + +4) Create a feature branch to work in +git checkout -b feature-xxx remotes/upstream/master + +5) Before submitting a pull request periodically rebase your changes (but don’t do it when a pull request is already submitted) +git pull --rebase upstream master + +6) Before submitting a pull request, combine (“squash”) related commits into a single one +git rebase upstream/master + +7) Submit a pull-request +git push origin feature-xxx + +Go to your Doris fork main page +https://github.com/<username&
[incubator-doris-website] branch asf-site updated (6f0e1e1 -> c04be83)
This is an automated email from the ASF dual-hosted git repository. lide pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git. from 6f0e1e1 Automatic Site Publish by git-site-role new 3fbc083 Automatic Site Publish by git-site-role new c04be83 Automatic Site Publish by git-site-role The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/feed.xml | 4 +- content/guides/contributing.html | 109 ++- 2 files changed, 63 insertions(+), 50 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] 02/02: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git commit c04be830038f23fc32eb829e8fc3e77e7684a06e Author: lide AuthorDate: Thu Nov 1 13:25:30 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index 6b65316..e05d6be 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Thu, 1 Nov 2018 13:24:29 +0800 -Thu, 1 Nov 2018 13:24:29 +0800 +Thu, 1 Nov 2018 13:25:26 +0800 +Thu, 1 Nov 2018 13:25:26 +0800 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: update contributing.md
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 390e976 update contributing.md 390e976 is described below commit 390e97640c916c08a8eb39a1e3165e6b7d2f28ff Author: lide AuthorDate: Thu Nov 1 13:35:55 2018 +0800 update contributing.md --- pages/guides/contributing.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/pages/guides/contributing.md b/pages/guides/contributing.md index 4fd7a40..682a41c 100644 --- a/pages/guides/contributing.md +++ b/pages/guides/contributing.md @@ -16,8 +16,6 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.--> -# Contributing to Apache Doris (incubating) - Thanks for your interest in contributing. As you see from [GitHub] (https://github.com/apache/incubator-doris), Apache Doris (incubating)’s codebase consists of two parts: frontend (FE) which written in JAVA and backend (BE) which writte in C++. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new c66f06c Automatic Site Publish by git-site-role c66f06c is described below commit c66f06c15db3eb30b2be173ca519b7c5aabffd6a Author: lide AuthorDate: Thu Nov 1 13:37:31 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- content/guides/contributing.html | 1 - 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index e05d6be..afcd794 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Thu, 1 Nov 2018 13:25:26 +0800 -Thu, 1 Nov 2018 13:25:26 +0800 +Thu, 1 Nov 2018 13:37:26 +0800 +Thu, 1 Nov 2018 13:37:26 +0800 diff --git a/content/guides/contributing.html b/content/guides/contributing.html index de9e457..6a5dd42 100644 --- a/content/guides/contributing.html +++ b/content/guides/contributing.html @@ -94,7 +94,6 @@ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.--> -Contributing to Apache Doris (incubating) Thanks for your interest in contributing. As you see from https://github.com/apache/incubator-doris";>GitHub, Apache Doris (incubating)’s codebase consists of two parts: frontend (FE) which written in JAVA and backend (BE) which writte in C++. 1. How to contribute? Most of the contributions that we receive are code contributions, but you can also contribute to the documentation or simply report solid bugs for us to fix. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: add how-to-add wechat to contributing.md
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new a9e81c2 add how-to-add wechat to contributing.md a9e81c2 is described below commit a9e81c2493a9f9d23f4a227a35e556ce58add465 Author: lide AuthorDate: Wed Nov 7 16:24:56 2018 +0800 add how-to-add wechat to contributing.md --- pages/policy/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pages/policy/contributing.md b/pages/policy/contributing.md index bacd109..87e058b 100644 --- a/pages/policy/contributing.md +++ b/pages/policy/contributing.md @@ -34,7 +34,7 @@ status=published 初次来到 Doris 社区，您可以： - 订阅我们的 [邮件列表](subscribe-mail-list.html)； -- 加入 Doris [微信群](use-wechat.html) 随时提问； +- 加入 Doris 微信群(加微信号：morningman-cmy, 备注：加入Doris群) 随时提问； - 进入 Doris 的 [Gitter](use-gitter.html) 聊天室； 通过以上方式及时了解 Doris 项目的开发动态并为您关注的话题发表意见。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 5f34678 Automatic Site Publish by git-site-role 5f34678 is described below commit 5f34678ba9e832943f37b3c13e25475ee86a9c7e Author: lide AuthorDate: Wed Nov 7 16:25:59 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- content/policy/contributing.html | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index 4fe98e8..2c95381 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Wed, 7 Nov 2018 16:20:42 +0800 -Wed, 7 Nov 2018 16:20:42 +0800 +Wed, 7 Nov 2018 16:25:54 +0800 +Wed, 7 Nov 2018 16:25:54 +0800 diff --git a/content/policy/contributing.html b/content/policy/contributing.html index 086b684..b2e1016 100644 --- a/content/policy/contributing.html +++ b/content/policy/contributing.html @@ -131,7 +131,7 @@ 初次来到 Doris 社区，您可以： 订阅我们的 邮件列表； - 加入 Doris 微信群 随时提问； + 加入 Doris 微信群(加微信号：morningman-cmy, 备注：加入Doris群) 随时提问； 进入 Doris 的 Gitter 聊天室； 通过以上方式及时了解 Doris 项目的开发动态并为您关注的话题发表意见。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: fix an error of title
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new c5ddfd6 fix an error of title c5ddfd6 is described below commit c5ddfd6a9da19971140a20c700f63817737059ae Author: lide AuthorDate: Wed Nov 7 16:39:05 2018 +0800 fix an error of title --- pages/guides/create-load-and-delete.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pages/guides/create-load-and-delete.md b/pages/guides/create-load-and-delete.md index 7dda7e2..5102c16 100644 --- a/pages/guides/create-load-and-delete.md +++ b/pages/guides/create-load-and-delete.md @@ -1,4 +1,4 @@ -title=Doris 文档 +title=建表、数据导入与删除 date=2018-11-06 type=guide status=published @@ -360,4 +360,4 @@ Doris 目前可以通过两种方式删除数据：`DELETE FROM` 语句和 `ALTE 1. 执行 delete 命令后，依然可以查询到数据。 -该问题通常可能是：1）delete 命令中指定的 Partition 不对，不包含对应的删除条件的数据。2）where 后面的条件格式不正确。如删除整型时，条件值添加了引号；或删除日期数据时，没有添加引号等。 \ No newline at end of file +该问题通常可能是：1）delete 命令中指定的 Partition 不对，不包含对应的删除条件的数据。2）where 后面的条件格式不正确。如删除整型时，条件值添加了引号；或删除日期数据时，没有添加引号等。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 103e23e Automatic Site Publish by git-site-role 103e23e is described below commit 103e23ebb78a2de85495d27846200bdffacfc2cd Author: lide AuthorDate: Wed Nov 7 16:39:52 2018 +0800 Automatic Site Publish by git-site-role --- content/archive.html | 2 +- content/community.html | 2 +- content/downloads.html | 2 +- content/faq.html | 2 +- content/feed.xml | 4 ++-- content/guides/backup-restore.html | 2 +- content/guides/best-practices.html | 2 +- content/guides/create-load-and-delete.html | 6 +++--- content/guides/data-model-rollup-prefix-index.html | 2 +- content/guides/deploy-upgrade.html | 2 +- content/guides/documents.html | 2 +- content/guides/getting-start.html | 2 +- content/guides/install.html| 2 +- content/guides/metadata-design.html| 2 +- content/guides/monitor-alert.html | 2 +- content/guides/multi-tenant.html | 2 +- content/guides/privilege.html | 2 +- content/guides/sql_reference.html | 2 +- content/index.html | 2 +- content/policy/contributing.html | 2 +- content/policy/pull-request.html | 2 +- content/policy/subscribe-mail-list.html| 2 +- content/policy/use-gitter.html | 2 +- content/whoweare.html | 2 +- 24 files changed, 27 insertions(+), 27 deletions(-) diff --git a/content/archive.html b/content/archive.html index 7eb14af..a537141 100644 --- a/content/archive.html +++ b/content/archive.html @@ -62,7 +62,7 @@ http://doris.apache.org/guides/monitor-alert.html";>监控和报警 - http://doris.apache.org/guides/create-load-and-delete.html";>Doris 文档 + http://doris.apache.org/guides/create-load-and-delete.html";>建表、数据导入与删除 http://doris.apache.org/guides/multi-tenant.html";>多租户设计 diff --git a/content/community.html b/content/community.html index 21bcc5f..3c1b467 100644 --- a/content/community.html +++ b/content/community.html @@ -62,7 +62,7 @@ http://doris.apache.org/guides/monitor-alert.html";>监控和报警 - http://doris.apache.org/guides/create-load-and-delete.html";>Doris 文档 + http://doris.apache.org/guides/create-load-and-delete.html";>建表、数据导入与删除 http://doris.apache.org/guides/multi-tenant.html";>多租户设计 diff --git a/content/downloads.html b/content/downloads.html index f64db66..7d5678a 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -62,7 +62,7 @@ http://doris.apache.org/guides/monitor-alert.html";>监控和报警 - http://doris.apache.org/guides/create-load-and-delete.html";>Doris 文档 + http://doris.apache.org/guides/create-load-and-delete.html";>建表、数据导入与删除 http://doris.apache.org/guides/multi-tenant.html";>多租户设计 diff --git a/content/faq.html b/content/faq.html index 010a8b2..eea49ca 100644 --- a/content/faq.html +++ b/content/faq.html @@ -62,7 +62,7 @@ http://doris.apache.org/guides/monitor-alert.html";>监控和报警 - http://doris.apache.org/guides/create-load-and-delete.html";>Doris 文档 + http://doris.apache.org/guides/create-load-and-delete.html";>建表、数据导入与删除 http://doris.apache.org/guides/multi-tenant.html";>多租户设计 diff --git a/content/feed.xml b/content/feed.xml index 2c95381..ce6511f 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Wed, 7 Nov 2018 16:25:54 +0800 -Wed, 7 Nov 2018 16:25:54 +0800 +Wed, 7 Nov 2018 16:39:46 +0800 +Wed, 7 Nov 2018 16:39:46 +0800 diff --git a/content/guides/backup-restore.html b/content/guides/backup-restore.html index 0a12e16..f9b6c6c 100644 --- a/content/guides/backup-restore.ht
[incubator-doris-website] branch master updated: add how-to-write commit message
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 0d68457 add how-to-write commit message 0d68457 is described below commit 0d6845777844f55e4c24ae305b13fd117f7d05e4 Author: lide AuthorDate: Fri Nov 9 15:28:34 2018 +0800 add how-to-write commit message --- pages/policy/pull-request.md | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/pages/policy/pull-request.md b/pages/policy/pull-request.md index 41e1013..e2d5818 100644 --- a/pages/policy/pull-request.md +++ b/pages/policy/pull-request.md @@ -74,9 +74,19 @@ status=published  - （3）填写 comment + （3）填写 Commit Message 这里请填写 comment 的总结和详细内容，然后点击 `Create pull request` 进行创建。 +关于如何写 Commit Message，下面列出了一些 Tips： + +* 请用英文 动词 + 宾语 的形式，动词不用过去式，语句用祈使句； +* 消息主题（Subject）和具体内容（Body）都要写，它们之间要有空行分割（GitHub PR界面上分别填写即可）。 +* 消息主题长度不要超过**50**个字符； +* 消息内容每行不要超过 72 个字符，超过的需要手动换行； +* 消息内容用于解释做了什么、为什么做以及怎么做的。 + +更详细的内容请参考 <https://chris.beams.io/posts/git-commit> +  （4）完成创建 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 2244cbe Automatic Site Publish by git-site-role 2244cbe is described below commit 2244cbe7a97e0f472c952101eea36592af8ec689 Author: lide AuthorDate: Fri Nov 9 15:29:33 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- content/policy/pull-request.html | 11 ++- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index ce6511f..5d0088d 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Wed, 7 Nov 2018 16:39:46 +0800 -Wed, 7 Nov 2018 16:39:46 +0800 +Fri, 9 Nov 2018 15:29:28 +0800 +Fri, 9 Nov 2018 15:29:28 +0800 diff --git a/content/policy/pull-request.html b/content/policy/pull-request.html index 4cd47a4..1713719 100644 --- a/content/policy/pull-request.html +++ b/content/policy/pull-request.html @@ -151,8 +151,17 @@ git push origin <your_branch_name> （2）准备分支 这时候，会出现 Create pull request 按钮，如果没有请检查是否正确选择了分支，也可以点击 “compare across forks” 重新选择 repo 和分支。 http://doris.apache.org/../images/create-pr.png"; alt="create PR"> -（3）填写 comment +（3）填写 Commit Message 这里请填写 comment 的总结和详细内容，然后点击 Create pull request 进行创建。 +关于如何写 Commit Message，下面列出了一些 Tips： + + 请用英文 动词 + 宾语 的形式，动词不用过去式，语句用祈使句； + 消息主题（Subject）和具体内容（Body）都要写，它们之间要有空行分割（GitHub PR界面上分别填写即可）。 + 消息主题长度不要超过50个字符； + 消息内容每行不要超过 72 个字符，超过的需要手动换行； + 消息内容用于解释做了什么、为什么做以及怎么做的。 + +更详细的内容请参考 https://chris.beams.io/posts/git-commit";>https://chris.beams.io/posts/git-commit http://doris.apache.org/../images/create-pr2.png"; alt="create PR"> （4）完成创建 创建成功后，您可以看到 Doris 项目需要 review，您可以等待我们 review 和合入，您也可以直接联系我们。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: update how-to-write commit message
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 2d020b1 update how-to-write commit message 2d020b1 is described below commit 2d020b19f6ec216f03a7c176e7c1d6581d8aae2e Author: lide AuthorDate: Fri Nov 9 15:43:48 2018 +0800 update how-to-write commit message --- pages/policy/pull-request.md | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/pages/policy/pull-request.md b/pages/policy/pull-request.md index e2d5818..034b3f6 100644 --- a/pages/policy/pull-request.md +++ b/pages/policy/pull-request.md @@ -80,10 +80,12 @@ status=published 关于如何写 Commit Message，下面列出了一些 Tips： * 请用英文 动词 + 宾语 的形式，动词不用过去式，语句用祈使句； -* 消息主题（Subject）和具体内容（Body）都要写，它们之间要有空行分割（GitHub PR界面上分别填写即可）。 +* 消息主题（Subject）和具体内容（Body）都要写，它们之间要有空行分隔（GitHub PR界面上分别填写即可）; * 消息主题长度不要超过**50**个字符； -* 消息内容每行不要超过 72 个字符，超过的需要手动换行； -* 消息内容用于解释做了什么、为什么做以及怎么做的。 +* 消息内容每行不要超过**72**个字符，超过的需要手动换行； +* 消息内容用于解释做了什么、为什么做以及怎么做的； +* 消息主题第一个字母要**大写**，句尾**不要**有句号； +* 消息内容中写明关联的issue，例如 #233; 更详细的内容请参考 <https://chris.beams.io/posts/git-commit> - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 9c90986 Automatic Site Publish by git-site-role 9c90986 is described below commit 9c90986083ae422eb09ff76a1105635f1d0a56d3 Author: lide AuthorDate: Fri Nov 9 15:44:41 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- content/policy/pull-request.html | 8 +--- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index 5d0088d..d1aceba 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Fri, 9 Nov 2018 15:29:28 +0800 -Fri, 9 Nov 2018 15:29:28 +0800 +Fri, 9 Nov 2018 15:44:36 +0800 +Fri, 9 Nov 2018 15:44:36 +0800 diff --git a/content/policy/pull-request.html b/content/policy/pull-request.html index 1713719..da30459 100644 --- a/content/policy/pull-request.html +++ b/content/policy/pull-request.html @@ -156,10 +156,12 @@ git push origin <your_branch_name> 关于如何写 Commit Message，下面列出了一些 Tips： 请用英文 动词 + 宾语 的形式，动词不用过去式，语句用祈使句； - 消息主题（Subject）和具体内容（Body）都要写，它们之间要有空行分割（GitHub PR界面上分别填写即可）。 + 消息主题（Subject）和具体内容（Body）都要写，它们之间要有空行分隔（GitHub PR界面上分别填写即可）; 消息主题长度不要超过50个字符； - 消息内容每行不要超过 72 个字符，超过的需要手动换行； - 消息内容用于解释做了什么、为什么做以及怎么做的。 + 消息内容每行不要超过72个字符，超过的需要手动换行； + 消息内容用于解释做了什么、为什么做以及怎么做的； + 消息主题第一个字母要大写，句尾不要有句号； + 消息内容中写明关联的issue，例如 #233; 更详细的内容请参考 https://chris.beams.io/posts/git-commit";>https://chris.beams.io/posts/git-commit http://doris.apache.org/../images/create-pr2.png"; alt="create PR"> - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: update how-to-write commit message
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new 72c81f7 update how-to-write commit message 72c81f7 is described below commit 72c81f7c9e6d46c9bbcafe89756cbe88825021a8 Author: lide AuthorDate: Fri Nov 9 15:53:02 2018 +0800 update how-to-write commit message --- pages/policy/pull-request.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/pages/policy/pull-request.md b/pages/policy/pull-request.md index 034b3f6..4b837a6 100644 --- a/pages/policy/pull-request.md +++ b/pages/policy/pull-request.md @@ -85,7 +85,7 @@ status=published * 消息内容每行不要超过**72**个字符，超过的需要手动换行； * 消息内容用于解释做了什么、为什么做以及怎么做的； * 消息主题第一个字母要**大写**，句尾**不要**有句号； -* 消息内容中写明关联的issue，例如 #233; +* 消息内容中写明关联的issue(如果有)，例如 #233; 更详细的内容请参考 <https://chris.beams.io/posts/git-commit> @@ -213,8 +213,8 @@ $ git add -u （7）填写 message 并提交到本地的新建分支上 ``` -$ git commit -m "fix a typo" -[my_branch 55e0ba2] fix a typo +$ git commit -m "Fix a typo" +[my_branch 55e0ba2] Fix a typo 1 files changed, 2 insertions(+), 2 deletions(-) ``` - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 1f9c4db Automatic Site Publish by git-site-role 1f9c4db is described below commit 1f9c4db228ed4de395e71783b2927d432cacafb8 Author: lide AuthorDate: Fri Nov 9 15:54:01 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- content/policy/pull-request.html | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index d1aceba..14f4e89 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Fri, 9 Nov 2018 15:44:36 +0800 -Fri, 9 Nov 2018 15:44:36 +0800 +Fri, 9 Nov 2018 15:53:55 +0800 +Fri, 9 Nov 2018 15:53:55 +0800 diff --git a/content/policy/pull-request.html b/content/policy/pull-request.html index da30459..63af51c 100644 --- a/content/policy/pull-request.html +++ b/content/policy/pull-request.html @@ -161,7 +161,7 @@ git push origin <your_branch_name> 消息内容每行不要超过72个字符，超过的需要手动换行； 消息内容用于解释做了什么、为什么做以及怎么做的； 消息主题第一个字母要大写，句尾不要有句号； - 消息内容中写明关联的issue，例如 #233; + 消息内容中写明关联的issue(如果有)，例如 #233; 更详细的内容请参考 https://chris.beams.io/posts/git-commit";>https://chris.beams.io/posts/git-commit http://doris.apache.org/../images/create-pr2.png"; alt="create PR"> @@ -243,8 +243,8 @@ $ git branch $ git add -u （7）填写 message 并提交到本地的新建分支上 -$ git commit -m "fix a typo" -[my_branch 55e0ba2] fix a typo +$ git commit -m "Fix a typo" +[my_branch 55e0ba2] Fix a typo 1 files changed, 2 insertions(+), 2 deletions(-) （8）将分支推到 GitHub 远端自己的 repo 中 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new f887b1c Automatic Site Publish by git-site-role f887b1c is described below commit f887b1c1e8ee2a8af60412ef25d817760e6103f3 Author: lide AuthorDate: Fri Nov 9 16:04:01 2018 +0800 Automatic Site Publish by git-site-role --- content/feed.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/feed.xml b/content/feed.xml index 14f4e89..401b80d 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml"; rel="self" type="application/rss+xml" /> JBake Bootstrap Template en-gb -Fri, 9 Nov 2018 15:53:55 +0800 -Fri, 9 Nov 2018 15:53:55 +0800 +Fri, 9 Nov 2018 16:03:54 +0800 +Fri, 9 Nov 2018 16:03:54 +0800 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch master updated: Add doris log to menu
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new eb6cfb6 Add doris log to menu eb6cfb6 is described below commit eb6cfb6ac828e0fc4ab2ac87d02ee044b6c01b0e Author: lide AuthorDate: Mon Nov 12 11:10:22 2018 +0800 Add doris log to menu --- assets/images/doris-logo-1.png| Bin 0 -> 14169 bytes assets/images/incubator_feather_egg_logo.png | Bin 30207 -> 0 bytes assets/images/incubator_feather_egg_logo_bw.png | Bin 37359 -> 0 bytes assets/images/incubator_feather_egg_logo_bw_crop.png | Bin 56218 -> 0 bytes assets/images/incubator_feather_egg_logo_red.png | Bin 39245 -> 0 bytes assets/images/incubator_feather_egg_logo_red_crop.png | Bin 68766 -> 0 bytes assets/images/incubator_feather_egg_logo_sm.png | Bin 17961 -> 0 bytes assets/images/incubator_feather_egg_logo_wb.png | Bin 38316 -> 0 bytes assets/images/incubator_feather_egg_logo_wb_crop.png | Bin 59188 -> 0 bytes templates/menu.gsp| 3 +-- 10 files changed, 1 insertion(+), 2 deletions(-) diff --git a/assets/images/doris-logo-1.png b/assets/images/doris-logo-1.png new file mode 100644 index 000..7dbcae6 Binary files /dev/null and b/assets/images/doris-logo-1.png differ diff --git a/assets/images/incubator_feather_egg_logo.png b/assets/images/incubator_feather_egg_logo.png deleted file mode 100644 index 338169e..000 Binary files a/assets/images/incubator_feather_egg_logo.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_bw.png b/assets/images/incubator_feather_egg_logo_bw.png deleted file mode 100644 index 0f647ed..000 Binary files a/assets/images/incubator_feather_egg_logo_bw.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_bw_crop.png b/assets/images/incubator_feather_egg_logo_bw_crop.png deleted file mode 100644 index 377e4e3..000 Binary files a/assets/images/incubator_feather_egg_logo_bw_crop.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_red.png b/assets/images/incubator_feather_egg_logo_red.png deleted file mode 100644 index 33dabdf..000 Binary files a/assets/images/incubator_feather_egg_logo_red.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_red_crop.png b/assets/images/incubator_feather_egg_logo_red_crop.png deleted file mode 100644 index e7bc1a7..000 Binary files a/assets/images/incubator_feather_egg_logo_red_crop.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_sm.png b/assets/images/incubator_feather_egg_logo_sm.png deleted file mode 100644 index 759252f..000 Binary files a/assets/images/incubator_feather_egg_logo_sm.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_wb.png b/assets/images/incubator_feather_egg_logo_wb.png deleted file mode 100644 index 144742a..000 Binary files a/assets/images/incubator_feather_egg_logo_wb.png and /dev/null differ diff --git a/assets/images/incubator_feather_egg_logo_wb_crop.png b/assets/images/incubator_feather_egg_logo_wb_crop.png deleted file mode 100644 index 3ec9f6d..000 Binary files a/assets/images/incubator_feather_egg_logo_wb_crop.png and /dev/null differ diff --git a/templates/menu.gsp b/templates/menu.gsp index 77c83cb..40a4235 100644 --- a/templates/menu.gsp +++ b/templates/menu.gsp @@ -37,9 +37,8 @@ + https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> - https://incubator.apache.org/images/incubator_feather_egg_logo_sm.png"; alt="The Apache Software Foundation Incubator" border="0" style="margin-top: 2px" height="100"> - https://www.apache.org/foundation/contributing.html";>https://www.apache.org/images/SupportApache-small.png"; height="75" width="75"> - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new d46aae1 Automatic Site Publish by git-site-role d46aae1 is described below commit d46aae19ada508662b106d0d89916fe0030bd2c7 Author: lide AuthorDate: Mon Nov 12 11:11:47 2018 +0800 Automatic Site Publish by git-site-role --- content/archive.html | 3 +-- content/community.html | 3 +-- content/downloads.html | 3 +-- content/faq.html | 3 +-- content/feed.xml | 4 ++-- content/guides/backup-restore.html | 3 +-- content/guides/best-practices.html | 3 +-- content/guides/create-load-and-delete.html | 3 +-- content/guides/data-model-rollup-prefix-index.html | 3 +-- content/guides/deploy-upgrade.html | 3 +-- content/guides/documents.html | 3 +-- content/guides/getting-start.html | 3 +-- content/guides/install.html| 3 +-- content/guides/metadata-design.html| 3 +-- content/guides/monitor-alert.html | 3 +-- content/guides/multi-tenant.html | 3 +-- content/guides/privilege.html | 3 +-- content/guides/sql_reference.html | 3 +-- content/images/doris-logo-1.png| Bin 0 -> 14169 bytes content/index.html | 3 +-- content/policy/contributing.html | 3 +-- content/policy/pull-request.html | 3 +-- content/policy/subscribe-mail-list.html| 3 +-- content/policy/use-gitter.html | 3 +-- content/whoweare.html | 3 +-- 25 files changed, 25 insertions(+), 48 deletions(-) diff --git a/content/archive.html b/content/archive.html index a537141..d79ff87 100644 --- a/content/archive.html +++ b/content/archive.html @@ -101,9 +101,8 @@ + https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> - https://incubator.apache.org/images/incubator_feather_egg_logo_sm.png"; alt="The Apache Software Foundation Incubator" border="0" style="margin-top: 2px" height="100"> - https://www.apache.org/foundation/contributing.html";>https://www.apache.org/images/SupportApache-small.png"; height="75" width="75"> diff --git a/content/community.html b/content/community.html index 3c1b467..2fd5b3e 100644 --- a/content/community.html +++ b/content/community.html @@ -101,9 +101,8 @@ + https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> - https://incubator.apache.org/images/incubator_feather_egg_logo_sm.png"; alt="The Apache Software Foundation Incubator" border="0" style="margin-top: 2px" height="100"> - https://www.apache.org/foundation/contributing.html";>https://www.apache.org/images/SupportApache-small.png"; height="75" width="75"> diff --git a/content/downloads.html b/content/downloads.html index 7d5678a..05bf3f7 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -101,9 +101,8 @@ + https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> - https://incubator.apache.org/images/incubator_feather_egg_logo_sm.png"; alt="The Apache Software Foundation Incubator" border="0" style="margin-top: 2px" height="100"> - https://www.apache.org/foundation/contributing.html";>https://www.apache.org/images/SupportApache-small.png"; height="75" width="75"> diff --git a/content/faq.html b/content/faq.html index eea49ca..6a4b104 100644 --- a/content/faq.html +++ b/content/faq.html @@ -101,9 +101,8 @@ + https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0"
[incubator-doris-website] branch master updated: Update doris logos under menu
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/master by this push: new c7f8930 Update doris logos under menu c7f8930 is described below commit c7f8930cb44fa9d1abe3e2883161944f671b3a13 Author: lide AuthorDate: Mon Nov 12 13:34:05 2018 +0800 Update doris logos under menu --- assets/images/doris-logo-2.png | Bin 0 -> 50200 bytes templates/menu.gsp | 4 ++-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/assets/images/doris-logo-2.png b/assets/images/doris-logo-2.png new file mode 100644 index 000..82e3224 Binary files /dev/null and b/assets/images/doris-logo-2.png differ diff --git a/templates/menu.gsp b/templates/menu.gsp index 40a4235..7a9354e 100644 --- a/templates/menu.gsp +++ b/templates/menu.gsp @@ -37,8 +37,8 @@ - -https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> +https://www.apache.org/";>https://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" height="200"> + - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-website] branch asf-site updated: Automatic Site Publish by git-site-role
This is an automated email from the ASF dual-hosted git repository. lide pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git The following commit(s) were added to refs/heads/asf-site by this push: new ce6819a Automatic Site Publish by git-site-role ce6819a is described below commit ce6819a6e6127b02a54371615f41cd1ea242234c Author: lide AuthorDate: Mon Nov 12 13:35:12 2018 +0800 Automatic Site Publish by git-site-role --- content/archive.html | 4 ++-- content/community.html | 4 ++-- content/downloads.html | 4 ++-- content/faq.html | 4 ++-- content/feed.xml | 4 ++-- content/guides/backup-restore.html | 4 ++-- content/guides/best-practices.html | 4 ++-- content/guides/create-load-and-delete.html | 4 ++-- content/guides/data-model-rollup-prefix-index.html | 4 ++-- content/guides/deploy-upgrade.html | 4 ++-- content/guides/documents.html | 4 ++-- content/guides/getting-start.html | 4 ++-- content/guides/install.html| 4 ++-- content/guides/metadata-design.html| 4 ++-- content/guides/monitor-alert.html | 4 ++-- content/guides/multi-tenant.html | 4 ++-- content/guides/privilege.html | 4 ++-- content/guides/sql_reference.html | 4 ++-- content/images/doris-logo-2.png| Bin 0 -> 50200 bytes content/index.html | 4 ++-- content/policy/contributing.html | 4 ++-- content/policy/pull-request.html | 4 ++-- content/policy/subscribe-mail-list.html| 4 ++-- content/policy/use-gitter.html | 4 ++-- content/whoweare.html | 4 ++-- 25 files changed, 48 insertions(+), 48 deletions(-) diff --git a/content/archive.html b/content/archive.html index d79ff87..59b9699 100644 --- a/content/archive.html +++ b/content/archive.html @@ -101,8 +101,8 @@ - -https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> +https://www.apache.org/";>https://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" height="200"> + diff --git a/content/community.html b/content/community.html index 2fd5b3e..64979dc 100644 --- a/content/community.html +++ b/content/community.html @@ -101,8 +101,8 @@ - -https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> +https://www.apache.org/";>https://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" height="200"> + diff --git a/content/downloads.html b/content/downloads.html index 05bf3f7..6ebb942 100644 --- a/content/downloads.html +++ b/content/downloads.html @@ -101,8 +101,8 @@ - -https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> +https://www.apache.org/";>https://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" height="200"> + diff --git a/content/faq.html b/content/faq.html index 6a4b104..d170b10 100644 --- a/content/faq.html +++ b/content/faq.html @@ -101,8 +101,8 @@ - -https://www.apache.org/";>http://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" width="250"> +https://www.apache.org/";>https://www.apache.org/img/asf_logo.png"; alt="The Apache Software Foundation" border="0" style="margin-top: 2px" height="200"> + diff --git a/content/feed.xml b/content/feed.xml index 05f5c9f..ba45c89 100644 --- a/content/feed.xml +++ b/content/feed.xml @@ -6,8 +6,8 @@ http://doris.apache.org/feed.xml";
