kkrugler commented on issue #7791: URL: https://github.com/apache/pinot/issues/7791#issuecomment-978344683
Hi @richardstartin - could you explain what you mean by "this works everywhere"? That makes me feel like I don't really understand what you're proposing, and/or that you don't understand what I'm proposing. Streaming unpack of files should also "work everywhere", and should be highly performant, with minimal code changes. As to the mismatch scenario, I'm assuming the idea is to store segments in directory A, and associated metadata in directory B. If an ops person copies updated segments to an archive dir, and forgets to copy the associated metadata, you would have miss-matched data. My previous experience dealing with Hadoop map files, which had an index and an associated data file, taught me the dangers of having to keep files in sync. But maybe there's a very different approach you're proposing that I'm missing, so some clarification would be great. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
