Re: Partition Question

2012-05-12 Thread Erick Erickson
No, this isn't what sharding is all about. Sharding is taking a single logical index and splitting it up amongst a number of physical units, often on individual machines. "Load and unload partitions dynamically" doesn't make any sense when talking about shards. So let's back up. You could create y

Re: Partition Question

2012-05-09 Thread Yuval Dotan
Thanks Lance There is already a clear partition - as you assumed, by date. My requirement is for the best setup for: 1. A *single machine* 2. Quickly changing index - so i need to have the option to load and unload partitions dynamically Do you think that the sharding model that solr offers is t

Re: Partition Question

2012-05-09 Thread Michael Kuhlmann
Am 08.05.2012 23:23, schrieb Lance Norskog: Lucene does not support more 2^32 unique documents, so you need to partition. Just a small note: I doubt that Solr supports more than 2^31 unique documents, as most other Java applications that use int values. Greetings, Kuli

Re: Partition Question

2012-05-08 Thread Lance Norskog
Lucene does not support more 2^32 unique documents, so you need to partition. In Solr this is done with Distributed Search: http://www.lucidimagination.com/search/link?url=http://wiki.apache.org/solr/DistributedSearch First, you have to decide a policy for which documents go to which 'shard'. It i

Re: Partition Question

2012-05-08 Thread Yuval Dotan
Hi Can someone please guide me to the right way to partition the solr index? On Mon, May 7, 2012 at 11:41 AM, Yuval Dotan wrote: > Hi All > Jan, thanks for the reply - answers for your questions are located below > Please update me if you have ideas that can solve my problems. > > First, some co

Re: Partition Question

2012-05-06 Thread Jan Høydahl
Hi, First you need to investigate WHY faceting and querying is too slow. What exactly do you mean by slow? Can you please tell us more about your setup? * How large documents and how many fields? * What kind of queries? How many hits? How many facets? Have you studies &debugQuery=true output? * D