Hello Yonik.
I’ve made few tests more today. Here are the results:
Start thread every 0.1->0.5 sec.
Each thread waits 2->10 sec before starting a new query
Each thread runs 5 min.
With FastSolrLRU
FullText :
- 10 users : 998 queries / Average time 0.037 sec
- 50 users : 4819 queries / Average time 0.049 sec
- 100 users : 6506 queries / Average time 1.272 sec
FullText + Facets :
- 10 users : 860 queries / Average time 0.511 sec
- 50 users : 1335 queries / Average time 8.264 sec
- 100 users : 1358 queries / Average time 18.703 sec
With SolrLRU
FullText :
- 50 users : 4803 queries / Average time 0.053 sec
- 100 users : 6088 queries / Average time 1.683 sec
FullText + Facets :
- 10 users : 553 queries / Average time 2.45 sec
- 50 users : 623 queries / Average time 21.352 sec
- 100 users : 679 queries / Average time 41.33 sec
Actually, the FastLRUCache is much more efficient than the LRUCache, but we
still have performance problem when we increase the number of users.:-/
Another remark, why do we have better performance when we use parallel
instances of SOLR that use the same index on the same machine?
Thanks for your help,
Regards,
Cédric
Yonik Seeley wrote:
>
> Your problem is most likely the time it takes to facet on those
> multi-valued fields.
> Help is coming within the month I'd estimate, in the form of faster
> faceting for multivalued fields where the number of values per
> document is low.
>
> Until then, you might be able to get some smaller speedups by trying
> the new FastLRUCache in Solr. Download the latest nightly build and
> change solr.LRUCache to solr.FastLRUCache in your solrconfig.xml.
>
> unrelated: you may not want a huge document cache like you have now,
> unless you still have enough RAM free for the OS to cache most of the
> index files.
>
> -Yonik
>
> On Thu, Nov 6, 2008 at 1:18 PM, Cedric Houis <[EMAIL PROTECTED]> wrote:
>>
>> Hi Solr's users, and first of all, sorry for my bad english =^D !!!
>>
>> We are experimenting Solr with the intention of using it in a web site.
>>
>> To test solr's performance, we have made a little program that simulates
>> users on the web site.
>> This program starts N threads (users), with 0.5 second pause between each
>> start.
>> Each thread makes a query, and wait between 2 and 10 seconds before make
>> another query.
>> Each query is randomly created with a list of 1500 words.
>>
>> When we simulate 1 user, a query takes about 0.3 second to execute.
>> When we simulate 10 users, a query takes about 2 second to execute.
>> When we simulate 50 users, a query takes about 60 seconds to execute.
>>
>> When we simulate 6*(10 users) with 6 instances of Solr on the same
>> machine,
>> a query takes 8 seconds to execute.
>>
>> You probably see where my problem is…
>>
>> Here is our configuration :
>>
>> Hardware :
>>
>> Debian etch, quad core, 6Go ram, 10000 rpm disk, jetty.
>> Windows 2003, quad core, 4Go ram, 10000 rpm disk, jetty and tomcat.
>>
>> Schema :
>>
>> <field name="searchable" type="boolean" indexed="true"
>> stored="true"/>
>> <field name="publishdate" type="date" indexed="true"
>> stored="true"/>
>> <field name="expirationdate" type="date" indexed="true"
>> stored="true"/>
>> <field name="site" type="string" indexed="true" stored="true"/>
>> <field name="section" type="text" indexed="true" stored="false"
>> multiValued="true"/>
>> <field name="byline" type="text" indexed="true" stored="false"
>> multiValued="true"/>
>> <field name="workflowstate" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>>
>> <field name="nDocumentId" type="string" indexed="true"
>> stored="true"/>
>> <field name="nLanguage" type="string" indexed="true"
>> stored="false"/>
>>
>> <field name="nFullText" type="text" indexed="true" stored="false"
>> termVectors="true"/>
>> <field name="nObjectXml" type="string" indexed="false"
>> stored="true"
>> multiValued="false"/>
>>
>> <field name="nGeographicLocations" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>> <field name="nOrganizationNames" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>> <field name="nPersonNames" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>> <field name="nCategories" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>> <field name="nSimpleConcepts" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>> <field name="nComplexConcepts" type="string" indexed="true"
>> stored="false" multiValued="true"/>
>> <field name="nSummary" type="text_ws" indexed="true"
>> stored="false"/>
>>
>> Solr config :
>>
>> <filterCache class="solr.LRUCache" size="1048576"
>> initialSize="4096"
>> autowarmCount="1024"/>
>> <queryResultCache class="solr.LRUCache" size="16384"
>> initialSize="4096"
>> autowarmCount="128"/>
>> <documentCache class="solr.LRUCache" size="1048576"
>> initialSize="512"
>> autowarmCount="0"/>
>>
>> <listener event="newSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst>
>> <str name="q">*:*</str>
>> <str name="fq">searchable:true</str>
>> <str name="sort">publishdate asc</str>
>> </lst>
>> <lst>
>> <str name="q">*:*</str>
>> <str name="facet.field">nPersonNames</str>
>> <str name="facet.field">nOrganizationNames</str>
>> <str name="facet.field">nGeographicLocations</str>
>> <str name="facet.field">nCategories</str>
>> <str name="fq">searchable:true</str>
>> </lst>
>> </arr>
>> </listener>
>> <listener event="firstSearcher" class="solr.QuerySenderListener">
>> <arr name="queries">
>> <lst>
>> <str name="q">*:*</str>
>> <str name="fq">searchable:true</str>
>> <str name="sort">publishdate asc</str>
>> </lst>
>> <lst>
>> <str name="q">*:*</str>
>> <str name="facet.field">nPersonNames</str>
>> <str name="facet.field">nOrganizationNames</str>
>> <str name="facet.field">nGeographicLocations</str>
>> <str name="facet.field">nCategories</str>
>> <str name="fq">searchable:true</str>
>> </lst>
>> </arr>
>> </listener>
>>
>> Data :
>>
>> 367380 documents
>>
>> nGeographicLocations : 39298 distincts values
>> nPersonNames : 325142 distincts values
>> nOrganizationNames : 130681 distincts values
>> nCategories : 929 distincts values
>> nSimpleConcepts : 110198 distincts values
>> nComplexConcepts : 1508141 distincts values
>> Each of those fields has a maximum of 7 values per document.
>>
>> Query :
>>
>> http://olympe:1006/solr/select
>> ?sort=publishdate+asc
>> &facet=true
>> &facet.field=nPersonNames
>> &facet.field=nOrganizationNames
>> &facet.field=nGeographicLocations
>> &facet.field=nCategories
>> &f.nPersonNames.facet.limit=10
>> &f.nOrganizationNames.facet.limit=10
>> &f.nGeographicLocations.facet.limit=10
>> &f.nCategories.facet.limit=10
>> &version=2.2
>> &wt=json
>> &json.nl=map
>> &q=nFullText%3A"Random words"+AND+searchable%3Atrue
>> &start=0
>> &rows=10
>>
>> Every help will be appreciated because I don't know how to optimize the
>> query time response…:-(
>>
>> Regards,
>>
>> Cédric Houis
>> --
>> View this message in context:
>> http://www.nabble.com/Very-bad-performance-tp20366783p20366783.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
--
View this message in context:
http://www.nabble.com/Very-bad-performance-tp20366783p20385748.html
Sent from the Solr - User mailing list archive at Nabble.com.
