Re: Solr Shards multi core slower then single big core
My query is
SolrQuery sQuery = new SolrQuery(query.getQueryStr());
sQuery.setQueryType("dismax");
sQuery.setRows(100);
if (!query.isSearchOnDefaultField()) {
sQuery.setParam("qf", queryFields.toArray(new
String[queryFields.size()]));
}
sQuery.setFields(visibleFields.toArray(new
String[visibleFields.size()]));
if(query.isORQuery())
{
sQuery.setParam("mm","1");
}
My search is
dismax
explicit
0.01
localhost:9090/solr/book1,localhost:9090/solr/book2,localhost:9090/solr/book3,localhost:9090/solr/book4,localhost:9090/solr/book5,localhost:9090/solr/book6
text^2.0
title item_id author titleMinusAuthor
4
*:*
text features name
0
name
regex
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Shards-multi-core-slower-then-single-big-core-tp3979115p3979243.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Shards multi core slower then single big core
Thanks Erick for the reply.
I have 6 cores which doesn't contain duplicated data. every core has some
unique data. What I thought was when I read it would read parallel 6 cores
and join the result and return the query. And this would be efficient then
reading one big core.
My question is wouldn't Solr read in parallel from shards when a query is
fired to it ?
Please let me know If i am assuming something which is wrong.
Thanks ,
Arjit
On Sun, May 13, 2012 at 12:44 AM, Erick Erickson [via Lucene] <
ml-node+s472066n3982950...@n3.nabble.com> wrote:
> One of the points of sharding is to use more _machines_. Running multiple
> shards on a single machine is not magically going to make things faster.
> In
> fact I'd expect your process to consume more resources since the
> cores are now not sharing common data (i.e. having a single word
> in more than one core will use two instances of that word).
>
> Best
> Erick
>
> On Fri, May 11, 2012 at 3:38 AM, arjit <[hidden
> email]<http://user/SendEmail.jtp?type=node&node=3982950&i=0>>
> wrote:
>
> > My query is
> > SolrQuery sQuery = new SolrQuery(query.getQueryStr());
> >sQuery.setQueryType("dismax");
> >
> >
> >sQuery.setRows(100);
> >
> >if (!query.isSearchOnDefaultField()) {
> >sQuery.setParam("qf", queryFields.toArray(new
> > String[queryFields.size()]));
> >}
> >sQuery.setFields(visibleFields.toArray(new
> > String[visibleFields.size()]));
> >
> >if(query.isORQuery())
> >{
> >sQuery.setParam("mm","1");
> >}
> >
> > My search is
> >
> >
> >
> > dismax
> > explicit
> > 0.01
> > >
> name="shards">localhost:9090/solr/book1,localhost:9090/solr/book2,localhost:9090/solr/book3,localhost:9090/solr/book4,localhost:9090/solr/book5,localhost:9090/solr/book6
>
> >
> >
> > text^2.0
> >
> >
> >
> >
> >
> >title item_id author titleMinusAuthor
> >
> >
> > 4
> > *:*
> >
> > text features name
> >
> > 0
> >
> > name
> > regex
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Solr-Shards-multi-core-slower-then-single-big-core-tp3979115p3979243.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
>
> --
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Solr-Shards-multi-core-slower-then-single-big-core-tp3979115p3982950.html
> To unsubscribe from Solr Shards multi core slower then single big core, click
> here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3979115&code=YXJqaXQyOTJAZ21haWwuY29tfDM5NzkxMTV8MTIwOTQwMDU4MA==>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context:
http://lucene.472066.n3.nabble.com/Solr-Shards-multi-core-slower-then-single-big-core-tp3979115p3983601.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Solr Shards multi core slower then single big core
Robert can you tell what you mean when you say "We do a lot of faceting so maybe that is why since facets can be built in parallel on different threads/cores". I am novice in solr. Can you tell me where Can i read about it ? Thanks , Arjit On Mon, May 14, 2012 at 8:54 PM, Robert Stewart [via Lucene] < ml-node+s472066n3983692...@n3.nabble.com> wrote: > We used to have one large index - then moved to 10 shards (7 million docs > each) - parallel search across all shards, and we get better performance > that way. We use a 40 core box with 128GB ram. We do a lot of faceting so > maybe that is why since facets can be built in parallel on different > threads/cores. We also have indexes on fast local disks (6 15K RPM disks > using raid stripes). > > > On May 14, 2012, at 10:42 AM, Michael Della Bitta wrote: > > > Hi, all, > > > > I've been running into murmurs about this idea elsewhere: > > > > > http://stackoverflow.com/questions/8698762/run-multiple-big-solr-shard-instances-on-one-physical-machine > > > > > http://java.dzone.com/articles/optimizing-solr-or-how-7x-your?mz=33057-solr_lucene > > > > Michael > > > > On Mon, May 14, 2012 at 10:29 AM, Otis Gospodnetic > > <[hidden email] <http://user/SendEmail.jtp?type=node&node=3983692&i=0>> > wrote: > >> Hi Kuli, > >> > >> As long as there are enough CPUs with spare cycles and disk IO is not a > bottleneck, this works faster. This was 12+ months ago. > >> > >> Otis > >> > >> Performance Monitoring for Solr / ElasticSearch / HBase - > http://sematext.com/spm > >> > >> > >> > >>> > >>> From: Michael Kuhlmann <[hidden > >>> email]<http://user/SendEmail.jtp?type=node&node=3983692&i=1>> > > >>> To: [hidden email]<http://user/SendEmail.jtp?type=node&node=3983692&i=2> > >>> Sent: Monday, May 14, 2012 10:21 AM > >>> Subject: Re: Solr Shards multi core slower then single big core > >>> > >>> Am 14.05.2012 16:18, schrieb Otis Gospodnetic: > >>>> Hi Kuli, > >>>> > >>>> In a client engagement, I did see this (N shards on 1 beefy box with > lots of RAM and CPU cores) be faster than 1 big index. > >>>> > >>> > >>> I want to believe you, but I also want to understand. Can you explain > >>> why? And did this only happen for single requests, or even under heavy > load? > >>> > >>> Greetings, > >>> Kuli > >>> > >>> > >>> > > > > -- > If you reply to this email, your message will be added to the discussion > below: > > http://lucene.472066.n3.nabble.com/Solr-Shards-multi-core-slower-then-single-big-core-tp3979115p3983692.html > To unsubscribe from Solr Shards multi core slower then single big core, click > here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=3979115&code=YXJqaXQyOTJAZ21haWwuY29tfDM5NzkxMTV8MTIwOTQwMDU4MA==> > . > NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-Shards-multi-core-slower-then-single-big-core-tp3979115p3983697.html Sent from the Solr - User mailing list archive at Nabble.com.
