kangkaisen commented on pull request #3601: URL: https://github.com/apache/incubator-doris/pull/3601#issuecomment-631209330
> > > Large bitmap will need use StringVal to allocate large memory, which is large than MAX_INT. > > > The overflow will cause serialization failure of bitmap. > > > > > > @kangpinghuang Hi. you mean the bitmap size is larger than 2G? are you sure which is reasonable？ > > yes, There is a user who use the bitmap to do the count distinct. In his situation, the dimension is about 30billion, the bitmap I see is about 3G. I See, Thanks. Have you tested this PR in prod env a long time? I think simply change the StringVal size from int to uint64_t maybe have a lot of issues: Such as, How do we handle network transfer？Mempool could handle 3G huge memory allocate well? the StringVal and Slice exchange could work well? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
