On 4/13/2019 9:29 PM, vishal patel wrote:
2> In production, lots of documents come for indexing within a second.If i do hard commit interval to 60 seconds then in less times open searchers when hard commit execute. Is it ohk for performance?
The autoCommit configuration should have openSearcher set to false. That way there will be no searchers opening on the hard commit. It is opening the searcher that is expensive, so doing hard commits without opening a new searcher is normally VERY fast and uses very few resources. When openSearcher is false, that commit will NOT make index changes visible.
3> soft commit 60 second we can not do as of now because our product implement like NRT after indexing instant show that changes.
As I just said, it is opening the searcher that is expensive. If that happens extremely frequently, which it would have to in order to achieve instant NRT, that's going to be opening a LOT of new searchers. This will absolutely kill performance. If you remove all cache warming, it will be better, but probably still bad.
Unless the index is extremely small, it is not usually possible to achieve a goal of "documents must be visible within one second after indexing".
If you are getting the "Overlapping onDecSearchers" warning, it means you have commits that open a new searcher happening too quickly. Let's say that it takes five seconds to finish a commit that opens a new searcher. I do not know how long it takes on your index, but I've seen it take much longer, so that's the example I'm going with. If you have such commits happening once a second, then you'll the warning will be logged because you will end up with five new searchers opening at the same time, Running them all simultaneously might make them take even longer -- the problem compounds itself.
I don't have any idea why you're having replicas go down, other than maybe this: All the commit activity (opening new searchers) might be keeping the system so busy that queries are taking long enough to reach timeouts.
Thanks, Shawn
