lnbest0707-uber commented on PR #16494: URL: https://github.com/apache/pinot/pull/16494#issuecomment-3207106754
> @lnbest0707-uber can you please add problem and solution description just specific to this PR in the PR description. If I understand correctly, this PR adds logic to consume from new topic type and this topic will have data which was skipped before by controller. > > 1. If above is correct can you tell how is this topic created in real time in upstream. > 2. Also why we have to make changes to LLCSegmentName? The PR description is exactly for this PR only. You can forget about the offset auto reset story. The PR is about multi-topic ingestion. Previously, we uses the 10000 number to pad/map between stream partition Id to Pinot partition Id. (e.g. partition 1 of the second topic would have 10001 partition id in Pinot). With this PR, we support a special type of topic in multi-topic ingestion. And the mapping between stream partition id and Pinot partition id would be `<topic_name>__<stream_partitionId>`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
