KKcorps commented on PR #10034: URL: https://github.com/apache/pinot/pull/10034#issuecomment-1366669679
> > Doesn't this introduce inconsistency between the two steps. Assume the following scenario - > > > > * Generate one segment tar using StandaloneJobRunner, we also generate the corresponding metadata tar for it > > * Replaced that segment tar in the output dir with some other tar file with the same name > > * We trigger metadata push which pushes the old metadata tar, however the segment tar file contains different data > > You are 100% correct. I can make the segment creation job to delete the metadata tarball as well if not exists. But if users intend to manually update the segment tarball, then it's pretty hard to detect that. Thinking out loud here, but may be we can introduce a crc check and keep segment tar crc in the metadata tar? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
