Jackie-Jiang commented on issue #7791: URL: https://github.com/apache/pinot/issues/7791#issuecomment-977312406
Just went over the code, and we are actually using the proper metadata push API (the other way is letting the controller download the segment and extract the metadata). The inefficiency comes from not using the metadata from the local generated segment. Ideally, we should follow these steps when generating and pushing the segments: 1. Generate the segments from the raw data 2. Cache the segment metadata from the local files 3. Tar the segments and upload to deep store 4. Push segments using the cached metadata -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
