Re: use of filter queries in Lucene/Solr Alpha40 and Beta4.0
Hoss, I'm so happy you realized the problem because I was quite worried about it!! Let me know if I can provide support with testing it. The last two days I was busy with migrating a bunch of hosts which should -hopefully- be finished today. Then I have again the infrastructure for running tests Günter On 09/05/2012 11:19 PM, Chris Hostetter wrote: : Subject: Re: use of filter queries in Lucene/Solr Alpha40 and Beta4.0 Günter, This is definitely strange The good news is, i can reproduce your problem. The bad news is, i can reproduce your problem - and i have no idea what's causing it. I've opened SOLR-3793 to try to get to the bottom of this, and included some basic steps to demonstrate the bug using the Solr 4.0-BETA example data, but i'm really not sure what the problem might be... https://issues.apache.org/jira/browse/SOLR-3793 -Hoss -- Universität Basel Universitätsbibliothek Günter Hipler Projekt SwissBib Schoenbeinstrasse 18-20 4056 Basel, Schweiz Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103 e-mailguenter.hip...@unibas.ch URL:www.swissbib.org /http://www.ub.unibas.ch/
Re: use of filter queries in Lucene/Solr Alpha40 and Beta4.0
Erick, thanks for response! Our use case is very straight forward and basic. - no cloud infrastructure - XMLUpdateRequest - handler (transformed library bibliographic data which is pushed by the post.jar component). For deletions I used to use the solrJ component until two month ago but because of the difficulties I read about I changed back to the basic procedure with XML documents - around 18 million documents, no distributed shards - once the basic use case is stable and maintainable we are heading forward to the more fancy things ;-) Yonik provided a patch (https://issues.apache.org/jira/browse/SOLR-3793) yesterday morning. I'm going to run tests once it is part of the nightly builds. By now, if I'm not wrong (https://builds.apache.org/job/Solr-Artifacts-4.x/), the last build doesn't contain it. Best wishes from Basel, Günter On 09/07/2012 07:09 AM, Erick Erickson wrote: Guenter: Are you using SolrCloud or straight Solr? And were you updating in batches (i.e. updating multiple docs at once from SolrJ by using the server.add(doclist) form)? There was a bug in this process that caused various docs to show up in various shards differently. This has been fixed in 4x, any nightly build should have the fix. I'm absolutely grasping at straws here, but this was a weird case that I happen to know about... Hossman: of course this all goes up in smoke if you can reproduce this with any recent compilation of the code. FWIW Erick On Wed, Sep 5, 2012 at 11:29 PM, guenter.hip...@unibas.ch wrote: Hoss, I'm so happy you realized the problem because I was quite worried about it!! Let me know if I can provide support with testing it. The last two days I was busy with migrating a bunch of hosts which should -hopefully- be finished today. Then I have again the infrastructure for running tests Günter On 09/05/2012 11:19 PM, Chris Hostetter wrote: : Subject: Re: use of filter queries in Lucene/Solr Alpha40 and Beta4.0 Günter, This is definitely strange The good news is, i can reproduce your problem. The bad news is, i can reproduce your problem - and i have no idea what's causing it. I've opened SOLR-3793 to try to get to the bottom of this, and included some basic steps to demonstrate the bug using the Solr 4.0-BETA example data, but i'm really not sure what the problem might be... https://issues.apache.org/jira/browse/SOLR-3793 -Hoss -- Universität Basel Universitätsbibliothek Günter Hipler Projekt SwissBib Schoenbeinstrasse 18-20 4056 Basel, Schweiz Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103 e-mailguenter.hip...@unibas.ch URL:www.swissbib.org /http://www.ub.unibas.ch/ -- Universität Basel Universitätsbibliothek Günter Hipler Projekt SwissBib Schoenbeinstrasse 18-20 4056 Basel, Schweiz Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103 e-mailguenter.hip...@unibas.ch URL:www.swissbib.org /http://www.ub.unibas.ch/
Re: use of filter queries in Lucene/Solr Alpha40 and Beta4.0
feedback to patch: I used build #85 (Revision: 1382192) to test the same use case (build up an initial index of 18 Mio and run updates with around 200.000 documents) result: the use of fq to drill down facets is now consistent! (available under http://sb-tp1.swissbib.unibas.ch) Thanks for providing a quick patch!! -Günter On 09/07/2012 05:09 PM, Erick Erickson wrote: Thank the guys who actually fixed it! Thanks for bringing this up, and please let us know if Yonik's patch fixes your problem Best Erick On Thu, Sep 6, 2012 at 11:39 PM, guenter.hip...@unibas.ch wrote: Erick, thanks for response! Our use case is very straight forward and basic. - no cloud infrastructure - XMLUpdateRequest - handler (transformed library bibliographic data which is pushed by the post.jar component). For deletions I used to use the solrJ component until two month ago but because of the difficulties I read about I changed back to the basic procedure with XML documents - around 18 million documents, no distributed shards - once the basic use case is stable and maintainable we are heading forward to the more fancy things ;-) Yonik provided a patch (https://issues.apache.org/jira/browse/SOLR-3793) yesterday morning. I'm going to run tests once it is part of the nightly builds. By now, if I'm not wrong (https://builds.apache.org/job/Solr-Artifacts-4.x/), the last build doesn't contain it. Best wishes from Basel, Günter On 09/07/2012 07:09 AM, Erick Erickson wrote: Guenter: Are you using SolrCloud or straight Solr? And were you updating in batches (i.e. updating multiple docs at once from SolrJ by using the server.add(doclist) form)? There was a bug in this process that caused various docs to show up in various shards differently. This has been fixed in 4x, any nightly build should have the fix. I'm absolutely grasping at straws here, but this was a weird case that I happen to know about... Hossman: of course this all goes up in smoke if you can reproduce this with any recent compilation of the code. FWIW Erick On Wed, Sep 5, 2012 at 11:29 PM, guenter.hip...@unibas.ch wrote: Hoss, I'm so happy you realized the problem because I was quite worried about it!! Let me know if I can provide support with testing it. The last two days I was busy with migrating a bunch of hosts which should -hopefully- be finished today. Then I have again the infrastructure for running tests Günter On 09/05/2012 11:19 PM, Chris Hostetter wrote: : Subject: Re: use of filter queries in Lucene/Solr Alpha40 and Beta4.0 Günter, This is definitely strange The good news is, i can reproduce your problem. The bad news is, i can reproduce your problem - and i have no idea what's causing it. I've opened SOLR-3793 to try to get to the bottom of this, and included some basic steps to demonstrate the bug using the Solr 4.0-BETA example data, but i'm really not sure what the problem might be... https://issues.apache.org/jira/browse/SOLR-3793 -Hoss -- Universität Basel Universitätsbibliothek Günter Hipler Projekt SwissBib Schoenbeinstrasse 18-20 4056 Basel, Schweiz Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103 e-mailguenter.hip...@unibas.ch URL:www.swissbib.org /http://www.ub.unibas.ch/ -- Universität Basel Universitätsbibliothek Günter Hipler Projekt SwissBib Schoenbeinstrasse 18-20 4056 Basel, Schweiz Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103 e-mailguenter.hip...@unibas.ch URL:www.swissbib.org /http://www.ub.unibas.ch/ -- Universität Basel Universitätsbibliothek Günter Hipler Projekt SwissBib Schoenbeinstrasse 18-20 4056 Basel, Schweiz Tel.: + 41 (0)61 267 31 12 Fax: ++41 61 267 3103 e-mailguenter.hip...@unibas.ch URL:www.swissbib.org /http://www.ub.unibas.ch/
