RE: Performance stats for indeces with over 10MM documents

2008-01-03 Thread Alex Benjamen
we currently use a relational system, and it doesn't perform. Also, even though a lot of our queries are structured, we do combine them with text search, so for instance, there could be an additional clause which is a free text search for a favorite TV show -- I had exactly

Re: Performance stats for indeces with over 10MM documents

2008-01-03 Thread Walter Underwood
I had exactly the same thought. That query is not an information retrieval (text search) query. It is data retrieval and would work great on a relational database. wunder On 1/2/08 9:53 PM, "John Stewart" <[EMAIL PROTECTED]> wrote: > Alex, > > Not to be a pain, but the response I had when looki

Re: Performance stats for indeces with over 10MM documents

2008-01-02 Thread Mike Klaas
On 2-Jan-08, at 9:52 PM, Alex Benjamen wrote: Thanks for the input, it's really valueable. Several forum users have suggested using fq to separate the caching of filters, and I can immediately see how this would help. I'm changing the code right now and going to run some benchmarks, hopeful

Re: Performance stats for indeces with over 10MM documents

2008-01-02 Thread John Stewart
Alex, Not to be a pain, but the response I had when looking at the query was, why not do this in a SQL database, which is designed precisely to process this sort of request at speed? I've noticed that people sometimes try to get Solr to act as a generalized information store -- I'm not sure that'

RE: Performance stats for indeces with over 10MM documents

2008-01-02 Thread Alex Benjamen
Mike, Thanks for the input, it's really valueable. Several forum users have suggested using fq to separate the caching of filters, and I can immediately see how this would help. I'm changing the code right now and going to run some benchmarks, hopefully see a big gain just from that > - use

Re: Performance stats for indeces with over 10MM documents

2008-01-02 Thread Mike Klaas
On 2-Jan-08, at 5:47 PM, Alex Benjamen wrote: gender:m AND status:(2 || 8 || 6 || 3) AND age:(26 || 27 || 28 || 29) AND orientation:3 gender:f AND age:(27 || 28 || 29 || 30 || 31 || 32 || 33 || 34 || 35 || 36 ) AND orientation:2 AND photos:y gender:f AND (activity:y) AND age:(28 || 29 || 30

Re: Performance stats for indeces with over 10MM documents

2008-01-02 Thread Erik Hatcher
Are you faceting? Please provide the exact lines logged in Solr's console for the offending queries - that would show us _exactly_ what you're hitting Solr with, not just the q parameter as you seem to have provided. Erik On Jan 2, 2008, at 8:47 PM, Alex Benjamen wrote: JDS: Th

RE: Performance stats for indeces with over 10MM documents

2008-01-02 Thread Alex Benjamen
JDS: > That's too slow. Can you provide more details about your schema, queries etc? Ofcourse - I'm using the standard config which comes with solr, and I've added the following fields :

Re: Performance stats for indeces with over 10MM documents

2008-01-02 Thread John Stewart
Alex, That's too slow. Can you provide more details about your schema, queries etc? jds On Jan 2, 2008 7:28 PM, Alex Benjamen <[EMAIL PROTECTED]> wrote: > Hi, > > I'm very interested in sharing performance stats with those who have indeces > that > contain more than 10MM documents. It seems th