High facet.limit (with only 2-3 actual facets) -> Massive bandwidth consumption in DistributedSearch

Hi guys, I've just experienced an odd issue today with the following setup: Test 1: 20 Shards facet.limit=2 (returned facets 2-3) about 5-6MB network traffic Resp Time ca 3sec Test 2: 20 Shards facet.limit=100 (returned facets 2-3) only a few kb network traffic Resp Time ca 0.02sec Does

Re: High facet.limit (with only 2-3 actual facets) -> Massive bandwidth consumption in DistributedSearch

yep - facet.mincount=1 Am Donnerstag, 8. September 2011 um 21:37 schrieb Michael Ryan: > Are you using facet.mincount in the query? > > -Michael

Re: High facet.limit (with only 2-3 actual facets) -> Massive bandwidth consumption in DistributedSearch

Now that is quite interesting indeed and sounds like a bug to me. Including facets with a count of 0 we have a few 100k which then apparently get transferred. hmhmhm Can anyone with more knowledge of the facet component maybe chime in why the miscount is removed? Am Donnerstag, 8. September

Re: High facet.limit (with only 2-3 actual facets) -> Massive bandwidth consumption in DistributedSearch

In our case it's clearly the wrong tradeoff :) I'm going to patch our Solr for now, but either - a config option - a facet.whatever param - or reversing the tradeoff should be done in my eyes. Am Donnerstag, 8. September 2011 um 22:34 schrieb Yonik Seeley: > So this is bad if you have a hi

strange performance issue with many shards on one server

Hi, I am experiencing a strange issue doing some load tests. Our setup: - 2 server with each 24 cpu cores, 130GB of RAM - 10 shards per server (needed for response times) running in a single tomcat instance - each query queries all 20 shards (distributed search) - each shard holds about 1.5

Re: strange performance issue with many shards on one server

ueries which contains numbers or > special characters. > if you don't know it: > http://www.hathitrust.org/blogs/large-scale-search/tuning-search-performance > Regards > Vadim > > > 2011/9/28 Frederik Kraus (mailto:frederik.kr...@gmail.com)> > > > Hi, >

Re: strange performance issue with many shards on one server

ecause of index rebuilt) nope > -do you replicate your index? nope > > Regards > Vadim > > > > 2011/9/28 Frederik Kraus (mailto:frederik.kr...@gmail.com)> > > > Hi Vladim, > > > > the thing is, that those exact same queries, that take longer duri

Re: strange performance issue with many shards on one server

e.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:662) Am Mittwoch, 28. September 2011 um 13:53 schrieb Frederik Kraus: > > > Am Mittwoch, 28

Re: strange performance issue with many shards on one server

> connections. > > 3. We needed to optimize our configuration for Jetty, Ubuntu, JVM GC, etc. > > There are lots of knobs to twiddle here, for better or worse. > > -- Ken > > On Sep 28, 2011, at 5:21am, Frederik Kraus wrote: > > > I just had a look at the thread-

Re: strange performance issue with many shards on one server

Am Mittwoch, 28. September 2011 um 16:40 schrieb Toke Eskildsen: > On Wed, 2011-09-28 at 12:58 +0200, Frederik Kraus wrote: > > - 10 shards per server (needed for response times) running in a single > > tomcat instance > > Have you tested that sharding actually decrea

Re: strange performance issue with many shards on one server

Yep, I'm not getting more than 50-60% CPU during those load tests. Am Mittwoch, 28. September 2011 um 23:01 schrieb Jaeger, Jay - DOT: > Yes, that thread waits (in the sense that nothing useful gets done), but > during that time, from the perspective of the applications and OS, that CPU > is

Re: mixing version of solr

No, that won't work as the index format has changed. On Donnerstag, 3. März 2011 at 20:03, Ofer Fort wrote: > Hey all, > I have a master slave using the same index folder, the master only writes, > and the slave only reads. > Is it possible to use different versions of solr for those two servers?

DIH - Multiple Cores / Consistent Hashing

Hi Guys, I'm currently working on a project with quite a few shards/cores etc. and ideally want to use the DIH to the indexing. Is there any consistent hashing method available, other than the modulo way of selecting only specific documents. Thanks, Fred.

Re: Extra facet query from within a custom search component

Haaa fantastic! Thanks a lot! Fred. On Donnerstag, 28. April 2011 at 22:21, Erick Erickson wrote: > Have you looked at: http://wiki.apache.org/solr/TermsComponent? > > Best > Erick > > On Thu, Apr 28, 2011 at 2:44 PM, Frederik Kraus > wrote: > > Hi Guys, > >

force "0" results from within a search component?

Hi guys, another question on custom search components: Is there any way to force the response to be "0 results" from within a search component (and break out of the component chain)? I'm doing some checks in my first-component and in some cases would like to stop processing the request and jus

Re: Huge performance drop in distributed search w/ shards on the same server/container

i 2011 at 12:57, Grant Ingersoll wrote: > Is that 10 different Tomcat instances or are you using multicore? How are you > testing? > > On May 13, 2011, at 6:08 AM, Frederik Kraus wrote: > > > Hi, > > > > I'm having some serious problems

Re: Huge performance drop in distributed search w/ shards on the same server/container

Any ideas? On Freitag, 13. Mai 2011 at 13:19, Frederik Kraus wrote: > One Tomcat with multicore. I have a list of about 2mio "real" queries that > I'm firing at the cluster with jmeter. Reason for splitting up the index in > rather small parts is that the maximum respo

DIH / dynamic fields / ...

Hi, I'm currently stuck with a (probably straightforward) problem concerning DIH and dynamic fields. I'm having a DB-Datasource with one of the columns (metaXml) containing an xml string looking something like this: The looks something like this: … … In my schema.xml