rohityadav1993 commented on PR #13837: URL: https://github.com/apache/pinot/pull/13837#issuecomment-2323451696
@kishoreg , extracting partitionId from segmentName is a reuse of [LLCSegmentName](https://github.com/apache/pinot/blob/master/pinot-common/src/main/java/org/apache/pinot/common/utils/LLCSegmentName.java) convention in UploadedReatimeSegmentName introduced in PR #13107 A call was taken to utilise segmentName encoding to reduce overhead of querying Zk. Yes, we can support batch-only upsert table. I had a few top of my thoughts: - Upsert table is essentially a realtime table and canont be created without a stream config. - Offline upsert tables would the right classification though behind the scenes it can be a realtime table with disabled stream consumption. - Many batch workloads have the possibility to deduplicate data during offline ingestion so offline tables are still usable if we overwrite all the data. Real advantage will be for usecases with incremental data ingestion with upsert requirement. I anticipate a few usecases coming up in future for us and will be happy to enhance batch ingestion for upserts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
