RS146BIJAY commented on issue #13387: URL: https://github.com/apache/lucene/issues/13387#issuecomment-2605229476
Or to elaborate more ```Searching across the N separate shards as if they were a single index is also possible via MultiReader``` will require separate Lucene indexes for different group. This approach has following concern: * Total number of shards required will be quite high (number of tenants * number of shards). This will cause huge load on the cluster manager node to schedule these shards on the data nodes of the clusters. Also there is only single thread on master node which handles all the operations including the deciders and make allocation decisions, allocation deciders execution may continue to block these threads which may prevent execution of certain high priority tasks like applying/sending cluster state update, index create, etc. * Another issue with this approach is skewness. For eg: For status code based grouping, 5xx and 4xx shards group receive more traffic than shrads containing 2xx documents. This will cause skewness on the node containing hot shards. Therefore was looking to support multi tenancy within a Lucene index itself via a combined view rather than maintaining separate Lucene index for different tenant and searching across N shards via a MultiReader. Let me know if this make sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org For additional commands, e-mail: issues-h...@lucene.apache.org
