xinyiZzz opened a new issue #6272: URL: https://github.com/apache/incubator-doris/issues/6272
When planning the Join type, the estimated memory size required for Broadcast Join HashTable is inaccurate, which is found in the TPC-DS test. The reason for OOM is that during FE planning, it is estimated that the memory value required by Broadcast Join to construct HashTable is too small, which leads to OOM when constructing HashTable after planning into Broadcast Join. Reasons that can be passed before opening Join Reorder, is that the value of `avgRowSize` in the right table of Join is larger than the actual value. Because `Cardinality * avgRowSize > memory limit`, it is planned as Shuffle Join, thus avoiding OOM. At present, `avgRowSize` is equal to the average data size of one row of the right table, that is `the amount of data in the right table/the number of rows`, but the actual Join in the HashTable construction is part of the column of the right table. Explain results are as follows:  Solution: Modify the estimation algorithm of the memory required to construct the HashTable when planning the Join type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
