On Thu, Dec 4, 2008 at 8:52 AM, sunnyfr <[EMAIL PROTECTED]> wrote: > > When I run my stress test ..sending multi thread ... around 100/sec I don't > start indexation at all ...
If you can't go higher than 100 requests / sec and the CPUs arent at 100% then the possibilities are: - If the index is bigger than free memory the OS can use to cache, then cache misses (at the OS level) can cause CPU to go lower - these cache mises are most likely to happen when retrieving stored fields for hits. - You can also be network IO bound if you are doing requests from a different machine. - Internal locking contention... pretty much every system will reach a peak number of requests/sec and then start declining as you add more concurrent requests. If you haven't yet, try a nightly build from December - the index-level locking should be improved under high load for non-Windows systems. -Yonik > maybe my cache ??? will check that > > > Yonik Seeley wrote: >> >> On Thu, Dec 4, 2008 at 8:36 AM, sunnyfr <[EMAIL PROTECTED]> wrote: >>> When I check my CPU, all my CPU are not full, how can I change this ? >> >> If this is while you are indexing, then it simply means that you are >> not feeding documents to Solr fast enough (use multiple threads to >> send to Solr, and send multiple documents in each update request if >> possible). If CPU utilization is still low, then it means you are IO >> (disk) bound... if you want to go faster, get faster disks. >> >> -Yonik >> >>> Do I have to change a parameter ?? >>> >>> Thanks a lot , >>> Johanna >>> >>> >>> Walter Underwood wrote: >>>> >>>> Try running your submits while watching a CPU load meter. >>>> Do this on a multi-CPU machine. >>>> >>>> If all CPUs are busy, you are running as fast as possible. >>>> >>>> If one CPU is busy (around 50% usage on a dual-CPU system), >>>> parallel submits might help. >>>> >>>> If no CPU is 100% busy, the bottleneck is probably disk >>>> or network. >>>> >>>> wunder >>>> >>>> On 2/20/07 10:46 AM, "Jack L" <[EMAIL PROTECTED]> wrote: >>>> >>>>> Thanks to all who replied. It's encouraging :) >>>>> >>>>> The numbers vary quite a bit though, from 13 docs/s (Burkamp) >>>>> to 250 docs/s (Walter) to 1000 docs/s I understand the results also >>>>> depend >>>>> on the doc size and hardware. >>>>> >>>>> I have a question for Erik: you mentioned "single threaded indexer" >>>>> (below). I'm not familiar with solr at all and did a search on solr >>>>> wiki for "thread" and didn't find anything. Is it so that I can >>>>> actually configure solr to be single-threaded and multi-threaded? >>>>> >>>>> And I'm not sure what you meant by parallelizing the indexer? >>>>> Running multiple instances of the indexer, or multiple instances >>>>> of solr? >>>>> >>>>> Thanks, >>>>> >>>>> Jack >>>>> >>>>>> My largest Solr index is currently at 1.4M and it takes a max of 3ms >>>>>> to add a document (according to Solr's console), most of them 1ms. >>>>>> My single threaded indexer is indexing around 1000 documents per >>>>>> minute, but I think I can get this number even faster by >>>>>> parallelizing the indexer. >>>>> >>>>> >>>>> __________________________________________________ >>>>> Do You Yahoo!? >>>>> Tired of spam? Yahoo! Mail has the best spam protection around >>>>> http://mail.yahoo.com >>>> >>>> >>>> >>> >>> -- >>> View this message in context: >>> http://www.nabble.com/solr-performance-tp9055437p20833521.html >>> Sent from the Solr - User mailing list archive at Nabble.com. >>> >>> >> >> > > -- > View this message in context: > http://www.nabble.com/solr-performance-tp9055437p20833790.html > Sent from the Solr - User mailing list archive at Nabble.com. > >
