[Discuss][Random Distribution Aggregate Table Query] Make consistent query result on Aggregate Keys table for both hash distribution and random distribution.

2022-06-26 Thread 蔡聪辉
Hi, all devs. Now Doris has supported random distribution for Duplicate Keys table and Aggregate Keys table without replace(replace_if_not_null) type column to solve data skew problem. But now, It brings some problem when we do some speical queris on aggregate table with random distribution.

Re:Re: [Discuss] Prepare to release 1.1

2022-06-26 Thread 陈明雨
Hi all, The dev-1.0.1 is now closed. I'm just waiting for one last PR to resolve the outer join bug. -- 此致!Best Regards 陈明雨 Mingyu Chen Email: morning...@apache.org 在 2022-06-24 10:06:05,"zhg yang" 写道: >+1 we should maintain a version with bug fix but without new features >Thanks >Yang

Re: [Discuss][Random Distribution Aggregate Table Query] Make consistent query result on Aggregate Keys table for both hash distribution and random distribution.

2022-06-26 Thread 张家峰
nice work! In the past few days, two or three users have raised the issue of data skew. This solution is very good. I prefer the second of the two solutions, so that users will not perceive it. 蔡聪辉 于2022年6月26日周日 21:49写道: > > > > Hi, all devs. Now Doris has supported random distribution for Dupli

Re: Re: Remove alpha rowset or segment v1 related code.

2022-06-26 Thread zhg yang
It is now possible to detect the format of the tablet in the database at that time through ADMIN SHOW TABLET STORAGE FORMAT, but there is no tools to explicitly convert v1 to v2, we need to provide such a tool before removing the code Thanks Yang Zhengguo 陈明雨 于2022年6月26日周日 11:12写道: > Hi Guolei

Re: [Discuss][DSIP] Support Multi-table materialized views

2022-06-26 Thread zhg yang
The full update does not increase the complexity of the user's use. Under normal circumstances, the user also needs to create a view, and the normal view also needs to be calculated in time. Using a multi-table materialized view is equivalent to caching some results. In order to reduce the user's d

Re: [Discuss][DSIP] Support Multi-table materialized views

2022-06-26 Thread zhg yang
And pre-request feature may be we should improve the performance of insert into select ,but this may be not pre, post may be also fine Thanks Yang Zhengguo 陈明雨 于2022年6月26日周日 11:55写道: > I have read the DSIP, and I think the Instructions for use are clearer, > but we need more detail designs.

Re: [Discuss][Random Distribution Aggregate Table Query] Make consistent query result on Aggregate Keys table for both hash distribution and random distribution.

2022-06-26 Thread zhg yang
I support solution 2. We should provide the same query interface for different tables. The wrong query result is mainly caused by the query planning not taking into account the new data shuffle method. This problem is more like a bug than a feature, because I I think it's still a matter of fixing t

[Discuss] Release v1.2 tick-off

2022-06-26 Thread 陈明雨
Hi all, The code branch for version 1.1 has been frozen. Version 1.1 can be regarded as a bug fix and optimized version of v1.0, and many new features that have been developed are not included in version 1.1. And these functions are also the functions that users desperately need. I recommend to

Re: Re: [Discuss][DSIP] Support Merge-On-Write implementation for UNIQUE KEY data model

2022-06-26 Thread Chen Zhang
Hi Devs, I've update the DISP last weekend, if you are interest on this feature, welcome to review and comment, thanks https://cwiki.apache.org/confluence/display/DORIS/DSIP-018%3A+Support+Merge-On-Write+implementation+for+UNIQUE+KEY+data+model Best Chen Zhang 在 2022年6月24日 +0800 10:13,zhg yang ,写

Re: [Discuss][Random Distribution Aggregate Table Query] Make consistent query result on Aggregate Keys table for both hash distribution and random distribution.

2022-06-26 Thread ling miao
I support solution2. Although such queries with mismatched aggregation types are really meaningless in real scenarios, based on Doris's consistent design style, we should allow users to submit such queries instead of banning them, just like we now allow users to directly query ```select * from agg_

Re: [Discuss] Release v1.2 tick-off

2022-06-26 Thread 张家峰
1. Remove string type Are we going to remove the string type in version 1.2? If so, what will be the impact on users who have already used the string type, and how to make it compatible 2. New Flink CDC Connector Can it be released together with 1.1, this function has been tested in 1.1, it should

Re: [Discuss][Random Distribution Aggregate Table Query] Make consistent query result on Aggregate Keys table for both hash distribution and random distribution.

2022-06-26 Thread 王博
For Agg table, data distribution way should keep the consistent semantics, from this perspective, I think solution 2 is better,this reduces the cost of user understanding. I have another question that whether rewrite sql and add a shuffle may cause performance drop in some scenes. 蔡聪辉 于2022年6月26

Re: [Discuss][Random Distribution Aggregate Table Query] Make consistent query result on Aggregate Keys table for both hash distribution and random distribution.

2022-06-26 Thread ling miao
> I have another question that whether rewrite sql and add a shuffle may cause performance drop in some scenes. Yes, but that's the price random tables have to pay. Random's agg table means that it can no longer perform complete pre-aggregation operations. So there will be a loss of code performan

Re: [Discuss] Release v1.2 tick-off

2022-06-26 Thread GuoLei Yi
I am willing to be the release manager. 张家峰 于2022年6月27日周一 12:31写道: > 1. Remove string type > Are we going to remove the string type in version 1.2? > If so, what will be the impact on users who have already used the string > type, and how to make it compatible > > 2. New Flink CDC Connector > Ca

Re: [Discuss] Release v1.2 tick-off

2022-06-26 Thread zhg yang
why remove string type ? Thanks Yang Zhengguo 陈明雨 于2022年6月27日周一 11:52写道: > Hi all, > The code branch for version 1.1 has been frozen. > Version 1.1 can be regarded as a bug fix and optimized version of v1.0, > and many new features that have been developed are not included in version > 1.1. >