kangpinghuang opened a new pull request #1379: Add rle page 2
URL: https://github.com/apache/incubator-doris/pull/1379
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitH
morningman opened a new pull request #1380: Fix publish version task timeout
URL: https://github.com/apache/incubator-doris/pull/1380
Publish version task using wrong timeout config.
This is an automated message from the Apach
imay merged pull request #1380: Fix publish version task timeout
URL: https://github.com/apache/incubator-doris/pull/1380
This is an automated message from the Apache Git Service.
To respond to the message, please log on to G
morningman opened a new pull request #1381: Fix column nullable property in
show create table stmt result
URL: https://github.com/apache/incubator-doris/pull/1381
Since the default nullable of a column changed to NOT NULL,
result of show table stmt should show the correct nullable proper
imay opened a new issue #1382: Add StorageRowIterator to uniform read in storage
URL: https://github.com/apache/incubator-doris/issues/1382
Currently in storage engine, there is too many interface to read data.
Including followings
1. SegmentReader which is used to read data from Se
wuyunfeng opened a new pull request #1383: Modify some method for external
invoking and add extra sink
URL: https://github.com/apache/incubator-doris/pull/1383
This is an automated message from the Apache Git Service.
To res
imay opened a new issue #1384: Add new SegmentIterator
URL: https://github.com/apache/incubator-doris/issues/1384
according to #1382 , we should implement a new `SegmetIterator` for new
format segment.
This class will leverage statistics information and bloom filter to reduce
data load.
imay opened a new issue #1385: Add SegmentWriter for new format segment
URL: https://github.com/apache/incubator-doris/issues/1385
We should create a new SegmentWriter to write data in new format. This will
arrange all kinds of data into one file. These include
1. Column data.
2
imay opened a new issue #1386: Add new SegmentGroupIterator
URL: https://github.com/apache/incubator-doris/issues/1386
Now we have SegmentGroup who have some segments, and these segment is
ordered. And we have another class named Rowset who is container of
SegmentGroup. But I think we can
imay opened a new issue #1387: Add new SegmentGroupWriter
URL: https://github.com/apache/incubator-doris/issues/1387
This class is same as CoumnDataWriter. we create one in order to not modify
origin code. This class will write data into ordered segments in new format.
This class wil
imay opened a new issue #1388: Add new ShortKeyIndexBuilder
URL: https://github.com/apache/incubator-doris/issues/1388
In current code, short key index is written to file directly. In new format
version, index will written into segment file. So we need a
ShortKeyIndexBuilder which will enc
imay opened a new issue #1389: Support bloom filter in new format segment
URL: https://github.com/apache/incubator-doris/issues/1389
We should support bloom filter in new format segment, so we can convert old
data into new format without impact query's performance.
This will add Bloo
imay opened a new issue #1390: Support zone map for column
URL: https://github.com/apache/incubator-doris/issues/1390
we should support zone map for column include min/max value for column page.
When scan data, we can use this information to eliminate data loading.
We should add a Zo
imay opened a new issue #1391: Add column page cache
URL: https://github.com/apache/incubator-doris/issues/1391
We should add column page cache which can help use reduce our CPU/IO use
when read same column data.
#1305
---
imay opened a new issue #1392: Support column page compress
URL: https://github.com/apache/incubator-doris/issues/1392
we should support page compress. for example plain binary encode page, we
can use compress to make it smaller in size.
#1305
--
kangkaisen opened a new issue #1393: Schema change job could not finished
URL: https://github.com/apache/incubator-doris/issues/1393
**Describe the bug**
Just now, our user found a schema change job run a long time:

gaodayue commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r297619170
##
File path: be/src/olap/rowset/segment_v2/rle_page.h
##
@@ -0,0 +1,255 @@
+// Licensed to the Apache Software Foundation (ASF)
kangpinghuang opened a new pull request #1397: fix garbage collect bug
URL: https://github.com/apache/incubator-doris/pull/1397
This is an automated message from the Apache Git Service.
To respond to the message, please log o
gaodayue commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r297644227
##
File path: be/src/olap/types.cpp
##
@@ -62,6 +62,7 @@ TypeInfoResolver::TypeInfoResolver() {
add_mapping();
add_map
gaodayue commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r297633133
##
File path: be/src/olap/rowset/segment_v2/rle_page.h
##
@@ -0,0 +1,255 @@
+// Licensed to the Apache Software Foundation (ASF)
gaodayue opened a new pull request #1398: Fix a bug of v2 ColumnReader when
reading not-null column
URL: https://github.com/apache/incubator-doris/pull/1398
This is an automated message from the Apache Git Service.
To respon
zhaomingzhu opened a new issue #1399: llvm 无法编译
URL: https://github.com/apache/incubator-doris/issues/1399
环境:
1)OS centos7.2
2)gcc --version
gcc (GCC) 5.4.0
3)cmake --version
cmake version 3.4.3
3)rpm -qa|grep libstdc
**libstdc++-devel-4.8.5-36.el7.x86_64
libstdc++-4
imay commented on issue #1399: llvm 无法编译
URL:
https://github.com/apache/incubator-doris/issues/1399#issuecomment-505913192
0.9.0是老版本了,建议使用0.10版本。
还有就是建议使用我们的Docker环境进行编译
This is an automated message from the Apache Git
imay merged pull request #1398: Fix a bug of v2 ColumnReader when reading
not-null column
URL: https://github.com/apache/incubator-doris/pull/1398
This is an automated message from the Apache Git Service.
To respond to the m
yiguolei opened a new pull request #1400: Acquire rs readers at the beginning
of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400
This is an automated message from the Apache Git Service.
To respond t
kangpinghuang commented on a change in pull request #1400: Acquire rs readers
at the beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298002733
##
File path: fe/src/main/java/org/apache/doris/common/Config.java
##
@@ -483,9
kangpinghuang commented on a change in pull request #1400: Acquire rs readers
at the beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298000905
##
File path: be/src/olap/reader.cpp
##
@@ -452,7 +452,8 @@ OLAPStatus Reader::_
kangpinghuang commented on a change in pull request #1400: Acquire rs readers
at the beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298000144
##
File path: be/src/exec/olap_scanner.cpp
##
@@ -474,6 +488,13 @@ Status OlapSc
imay commented on a change in pull request #1400: Acquire rs readers at the
beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298015789
##
File path: be/src/exec/olap_scanner.cpp
##
@@ -150,6 +150,20 @@ Status OlapScanner::_i
yiguolei commented on a change in pull request #1400: Acquire rs readers at the
beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298016412
##
File path: fe/src/main/java/org/apache/doris/common/Config.java
##
@@ -483,9 +483,
yiguolei commented on a change in pull request #1400: Acquire rs readers at the
beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298016335
##
File path: be/src/exec/olap_scanner.cpp
##
@@ -150,6 +150,20 @@ Status OlapScanner
yiguolei commented on a change in pull request #1400: Acquire rs readers at the
beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298016514
##
File path: be/src/olap/reader.cpp
##
@@ -452,7 +452,8 @@ OLAPStatus Reader::_captu
yiguolei commented on a change in pull request #1400: Acquire rs readers at the
beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298016738
##
File path: be/src/exec/olap_scanner.cpp
##
@@ -474,6 +488,13 @@ Status OlapScanner
kangpinghuang commented on a change in pull request #1400: Acquire rs readers
at the beginning of the olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400#discussion_r298018184
##
File path: be/src/olap/reader.cpp
##
@@ -452,7 +452,8 @@ OLAPStatus Reader::_
imay merged pull request #1400: Acquire rs readers at the beginning of the
olapscanner
URL: https://github.com/apache/incubator-doris/pull/1400
This is an automated message from the Apache Git Service.
To respond to the mess
chaoyli merged pull request #1397: fix garbage collect bug
URL: https://github.com/apache/incubator-doris/pull/1397
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
morningman opened a new pull request #1401: Add log for unstable table when
doing alter job
URL: https://github.com/apache/incubator-doris/pull/1401
Add a INFO level log when encountering unstable table in alter job
This is a
kangpinghuang opened a new pull request #1402: Fix gc coredump
URL: https://github.com/apache/incubator-doris/pull/1402
This is an automated message from the Apache Git Service.
To respond to the message, please log on to Git
wuyunfeng opened a new pull request #1403: Add some field for extrernal service
URL: https://github.com/apache/incubator-doris/pull/1403
This is an automated message from the Apache Git Service.
To respond to the message, ple
kangpinghuang commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r298135072
##
File path: be/src/olap/rowset/segment_v2/rle_page.h
##
@@ -0,0 +1,255 @@
+// Licensed to the Apache Software Foundation
kangpinghuang commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r298135107
##
File path: be/src/olap/rowset/segment_v2/rle_page.h
##
@@ -0,0 +1,255 @@
+// Licensed to the Apache Software Foundation
kangpinghuang commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r298140649
##
File path: be/src/olap/rowset/segment_v2/rle_page.h
##
@@ -0,0 +1,255 @@
+// Licensed to the Apache Software Foundation
kangpinghuang commented on a change in pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#discussion_r298140673
##
File path: be/src/olap/types.cpp
##
@@ -62,6 +62,7 @@ TypeInfoResolver::TypeInfoResolver() {
add_mapping();
ad
EmmyMiao87 opened a new pull request #1404: Use load job v2 in default
URL: https://github.com/apache/incubator-doris/pull/1404
This is an automated message from the Apache Git Service.
To respond to the message, please log o
EmmyMiao87 opened a new pull request #1405: Get table name from DataSourceInfo
instead of DataDesc
URL: https://github.com/apache/incubator-doris/pull/1405
This is an automated message from the Apache Git Service.
To respond
gaodayue commented on issue #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379#issuecomment-506347272
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please l
imay commented on a change in pull request #1404: Use load job v2 in default
URL: https://github.com/apache/incubator-doris/pull/1404#discussion_r298196602
##
File path: fe/src/main/java/org/apache/doris/qe/DdlExecutor.java
##
@@ -116,10 +116,10 @@ public static void execut
imay merged pull request #1403: Add some field for extrernal service
URL: https://github.com/apache/incubator-doris/pull/1403
This is an automated message from the Apache Git Service.
To respond to the message, please log on
imay merged pull request #1379: Add rle page
URL: https://github.com/apache/incubator-doris/pull/1379
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
wuyunfeng opened a new pull request #1406: Add schema_hash for
TTabletVersionInfo and modify some field name
URL: https://github.com/apache/incubator-doris/pull/1406
TScanRangeParams require the schema_hash
This is an automat
wuyunfeng closed pull request #1406: Add schema_hash for TTabletVersionInfo and
modify some field name
URL: https://github.com/apache/incubator-doris/pull/1406
This is an automated message from the Apache Git Service.
To res
EmmyMiao87 opened a new issue #1407: The broker load is cancelled with error
'tablet writer writer failed'
URL: https://github.com/apache/incubator-doris/issues/1407
**Describe the bug**
The broker load is cancelled with reason 'tablet writer writer failed'. The
olap engine find the exi
EmmyMiao87 edited a comment on issue #1407: The broker load is cancelled with
error 'tablet writer writer failed'
URL:
https://github.com/apache/incubator-doris/issues/1407#issuecomment-506586717
Also, I close the compress of snappy.
---
EmmyMiao87 commented on issue #1407: The broker load is cancelled with error
'tablet writer writer failed'
URL:
https://github.com/apache/incubator-doris/issues/1407#issuecomment-506586717
Also, I close the compross of snappy.
--
kangpinghuang commented on issue #1382: Add StorageRowIterator to uniform read
in storage
URL:
https://github.com/apache/incubator-doris/issues/1382#issuecomment-506594469
I think StorageRowwiseIterator is a better name because it returns RowBlock,
not a row
-
kangpinghuang opened a new issue #1408: Add dict encoding page
URL: https://github.com/apache/incubator-doris/issues/1408
related to #1305
add dict encoding page for binary/string type data.
This is an automated messag
yiguolei commented on a change in pull request #1401: Add report version for
publish task
URL: https://github.com/apache/incubator-doris/pull/1401#discussion_r298472317
##
File path: fe/src/main/java/org/apache/doris/master/MasterImpl.java
##
@@ -559,9 +559,17 @@ private v
kangpinghuang opened a new pull request #1409: Add dict page
URL: https://github.com/apache/incubator-doris/pull/1409
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHu
HangyuanLiu opened a new pull request #1410: Fix binary plain page relocate bug
URL: https://github.com/apache/incubator-doris/pull/1410
Fix binary plain page relocate bug
This is an automated message from the Apache Git Servi
mccoding opened a new issue #1411: can't connect doris by metabse
URL: https://github.com/apache/incubator-doris/issues/1411
**Describe the bug**
can't connect doris by metabse
**To Reproduce**
Steps to reproduce the behavior:
1. add a new database in metabse
2. you will
morningman commented on a change in pull request #1401: Add report version for
publish task
URL: https://github.com/apache/incubator-doris/pull/1401#discussion_r298566509
##
File path: fe/src/main/java/org/apache/doris/master/MasterImpl.java
##
@@ -559,9 +559,17 @@ private
imay merged pull request #1401: Add report version for publish task
URL: https://github.com/apache/incubator-doris/pull/1401
This is an automated message from the Apache Git Service.
To respond to the message, please log on t
EmmyMiao87 opened a new pull request #1412: Fix the core in dpp sink by sum of
int128
URL: https://github.com/apache/incubator-doris/pull/1412
This is an automated message from the Apache Git Service.
To respond to the messa
imay merged pull request #1412: Fix the core in dpp sink by sum of int128
URL: https://github.com/apache/incubator-doris/pull/1412
This is an automated message from the Apache Git Service.
To respond to the message, please lo
imay merged pull request #1410: Fix binary plain page relocate bug
URL: https://github.com/apache/incubator-doris/pull/1410
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
imay merged pull request #1404: Use load job v2 in default
URL: https://github.com/apache/incubator-doris/pull/1404
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
imay merged pull request #1405: Get table name from DataSourceInfo instead of
DataDesc
URL: https://github.com/apache/incubator-doris/pull/1405
This is an automated message from the Apache Git Service.
To respond to the mess
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298788108
##
File path:
extension/spark-sink/src/main/resources/META-INF/services/org.apache.spark.sql.sources.Dat
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298789848
##
File path:
extension/spark-sink/src/main/scala/org/apache/spark/sql/palo/PaloSinkProvider.scala
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298790079
##
File path:
extension/spark-sink/src/main/scala/org/apache/spark/sql/palo/PaloWriter.scala
##
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298790322
##
File path:
extension/spark-sink/src/main/scala/org/apache/spark/sql/palo/PaloBulkLoadTask.scala
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298790368
##
File path:
extension/spark-sink/src/main/scala/org/apache/spark/sql/palo/PaloBulkLoadTask.scala
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298790424
##
File path:
extension/spark-sink/src/main/scala/org/apache/spark/sql/palo/PaloWriter.scala
##
LiangchangZ commented on a change in pull request #1332: Doris sink for Spark
Structured Streaming
URL: https://github.com/apache/incubator-doris/pull/1332#discussion_r298790440
##
File path:
extension/spark-sink/src/main/scala/org/apache/spark/sql/palo/PaloBulkLoadTask.scala
imay opened a new pull request #1413: Add short key index builder, decoder,
iterator
URL: https://github.com/apache/incubator-doris/pull/1413
#1388
This is an automated message from the Apache Git Service.
To respond to the
morningman opened a new pull request #1414: Collect all documents to Doris code
base
URL: https://github.com/apache/incubator-doris/pull/1414
Move the documents on Doris GitHub wiki to Doris code base.
And all documents since now should be edited and save in Doris code base.
---
imay merged pull request #1414: Collect all documents to Doris code base
URL: https://github.com/apache/incubator-doris/pull/1414
This is an automated message from the Apache Git Service.
To respond to the message, please log
chaoyli merged pull request #1402: Fix gc coredump
URL: https://github.com/apache/incubator-doris/pull/1402
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
EmmyMiao87 opened a new pull request #1415: Change error msg of mini load when
PUBLISH_TIMEOUT
URL: https://github.com/apache/incubator-doris/pull/1415
This is an automated message from the Apache Git Service.
To respond to
zhaomingzhu commented on issue #1399: llvm 无法编译
URL:
https://github.com/apache/incubator-doris/issues/1399#issuecomment-507140831
使用最新版本编译还是报错,不知道问题出在哪
1) In file included from
/export/incubator-doris-0.10.0-rc04/be/src/util/bfd_parser.cpp:18:0:
/export/incubator-doris-0.10.0-rc04/b
imay commented on issue #1399: llvm 无法编译
URL:
https://github.com/apache/incubator-doris/issues/1399#issuecomment-507142582
You can install libbfd with `yum install elfutils-devel`
This is an automated message from the Apache
imay edited a comment on issue #1399: llvm 无法编译
URL:
https://github.com/apache/incubator-doris/issues/1399#issuecomment-507142582
You can install libbfd with `yum install binutils-devel zlib-devel
libcurl-devel elfutils-devel`
--
zttaotao2013 opened a new issue #1416: ETL_SUBMIT_FAIL problem
URL: https://github.com/apache/incubator-doris/issues/1416
我搭建了3fe + 3be + 3broker的doris测试环境,计划对存储于hdfs目录中的数据使用Broker进行批量入库,我定义了一个任务:
load label data_load_1
data infile
(
"hdfs://192.168.0.2:8020/doris/data/*.nb"
imay commented on issue #1416: ETL_SUBMIT_FAIL problem
URL:
https://github.com/apache/incubator-doris/issues/1416#issuecomment-507160933
当前broker导入需要把数据都加载到内存中,然后完成排序操作再写入到磁盘中。
我们新版本的导入已经可以不需要全部加载到内存中了,这个值是可以改大的。你可以修改max_bytes_per_broker_scanner的值
---
imay merged pull request #1415: Change error msg of mini load when
PUBLISH_TIMEOUT
URL: https://github.com/apache/incubator-doris/pull/1415
This is an automated message from the Apache Git Service.
To respond to the message,
zttaotao2013 commented on issue #1416: ETL_SUBMIT_FAIL problem
URL:
https://github.com/apache/incubator-doris/issues/1416#issuecomment-507166559
是在fe或be的配置文件中改么?我用mysql set 命令提示没有这个变量
This is an automated message from the Apa
imay commented on issue #1416: ETL_SUBMIT_FAIL problem
URL:
https://github.com/apache/incubator-doris/issues/1416#issuecomment-507168539
> 是在fe或be的配置文件中改么?我用mysql set 命令提示没有这个变量
fe.conf
max_bytes_per_broker_scanner = x
gaodayue commented on a change in pull request #1413: Add short key index
builder, decoder, iterator
URL: https://github.com/apache/incubator-doris/pull/1413#discussion_r298932728
##
File path: be/src/olap/short_key_index.h
##
@@ -0,0 +1,183 @@
+// Licensed to the Apache S
gaodayue commented on a change in pull request #1413: Add short key index
builder, decoder, iterator
URL: https://github.com/apache/incubator-doris/pull/1413#discussion_r298902873
##
File path: be/src/olap/short_key_index.h
##
@@ -0,0 +1,183 @@
+// Licensed to the Apache S
gaodayue commented on a change in pull request #1413: Add short key index
builder, decoder, iterator
URL: https://github.com/apache/incubator-doris/pull/1413#discussion_r298863266
##
File path: be/src/olap/short_key_index.h
##
@@ -0,0 +1,183 @@
+// Licensed to the Apache S
gaodayue commented on a change in pull request #1413: Add short key index
builder, decoder, iterator
URL: https://github.com/apache/incubator-doris/pull/1413#discussion_r298914147
##
File path: be/src/olap/olap_index.cpp
##
@@ -50,41 +53,64 @@ MemIndex::~MemIndex() {
gaodayue commented on a change in pull request #1413: Add short key index
builder, decoder, iterator
URL: https://github.com/apache/incubator-doris/pull/1413#discussion_r298932675
##
File path: be/src/olap/short_key_index.h
##
@@ -0,0 +1,183 @@
+// Licensed to the Apache S
1101 - 1200 of 7706 matches
Mail list logo