On 2/23/2018 2:40 PM, Tom Peters wrote: > I included the last 25 lines from the logs from each of the five nodes during > that time period.
Did you perhaps grep for "ERROR" and include the last 25 lines of that? This excludes the vast majority of the information from the error, information that be very important for troubleshooting. Each error is potentially dozens of lines long, and only the first line will have "ERROR" on it. > I _think_ I'm running into issues with bulking up deleteByQuery. Quick > background: we have objects in our system that may have multiple availability > windows. So when we index an object, will store it as separate documents each > with their own begins and expires date. At index time we don't know if the > all of the windows are still valid or not, so we remove all of them with a > deleteByQuery (e.g. deleteByQuery=object_id:12345) and then index one or more > documents. The deleteByQuery operation does not coexist well with other simultaneous operations. I am not familiar with the details of WHY this is the case, I only know that it IS the case. I had some big problems when I first started out. Ultimately, I ended up making sure that if a background optimize operation was underway, that I skipped the update process, which included deletes. I strongly recommend replacing deleteByQuery with a two step process that sends the query as a standard query, retrieves all the uniqueKey values from that query, and then issues deleteById calls. This should result in much smoother operation that can coexist with other operations, merging in particular. > I ran an isolated test a number of times where I indexed 1500 documents in > this manner (deletes then index). In Solr 3.4, it takes about 15s to > complete. In Solr 7.1, it's taking about 5m. If I remove the deleteByQuery, > the indexing times are nearly identical. SolrCloud did not exist in Solr version 3.4. It was introduced in version 4.0. Did you mean version 4.3? If you really did mean 3.4, then a direct comparison with SolrCloud on 7.1 is not possible. SolrCloud works a little differently. Thanks, Shawn
