navina opened a new issue, #9854: URL: https://github.com/apache/pinot/issues/9854
In Pulsar's `StreamMetadataProvider` implementation, we dynamically create a `Consumer` pulsar client with random subscription id to fetch metadata. We only close the consumer and do not clean up the subscription. This results in unused subscriptions in the source for a topic. This issue is aggravated as the realtime engine instantiates a new metadata provider based on idle timeout. **Possible Fix**: a. We should delete the subscription each time we close the consumer. b. Alternatively, I think we can use Pulsar's admin client to fetch all the metadata we need (See ([`PulsarAdmin#topics()#examinMessage`](https://pulsar.apache.org/docs/2.10.x/admin-api-topics/#examine-messages)) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@pinot.apache.org For additional commands, e-mail: commits-h...@pinot.apache.org
