weizuo93 commented on pull request #5488: URL: https://github.com/apache/incubator-doris/pull/5488#issuecomment-810298145
> @weizuo93 You are right. I am wrong. This PR could go ahead. (But store some load job meta in FE and store some load job meta in BE, which seems not very good) @kangkaisen Thanks for your comment. There is code overlap between this PR and PR #5452(https://github.com/apache/incubator-doris/pull/5452). After discussion with Mingyu Chen, we first push forward PR #5452. About storing stream load meta in BE, it's a tradeoff. If BE reports the stream load result to FE when each stream load job finished, it may result in too much stress for FE RPC. So the stream load results are temporarily stored on BE and then sent to FE by batch. After finished data sending, stream load meta will be removed from BE soon. Do you have any better idea? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
