Shalin, First of all each entity data is unrelated so it makes sense to use solr core concept as per your suggestion.
But Since you are suggesting putting each entity index on same box will consume CPU so does it make sense to add boxes based on number of entities considering i will have to add replication boxes also amounting a huge cost. This is what i am thinking after your suggestion - Have separate boxes for each entity and then inside each entity do some partitioning based on round robin or some strategy. With this if i am searching on any entity data then i will just require to reach a box for that entity. Now since i am doing a partitioning inside an entity also how i will do search for data so that i got merged result from each partition in a single entity box. If i doing this type of partitioning than which functionality of solr i will use ... is it http://wiki.apache.org/solr/IndexPartitioning ? My actual concern is performance irrespective of implementation design considering a good scaling logic also for future . On Mon, Jul 6, 2009 at 3:16 PM, Shalin Shekhar Mangar < shalinman...@gmail.com> wrote: > On Mon, Jul 6, 2009 at 3:05 PM, Sumit Aggarwal <sumit.kaggar...@gmail.com > >wrote: > > > Hi Shalin, > > Yes i want to achieve a logical separation of indexes for performance > > reason > > also else index size will keep on growing as i have 8 different entities. > I > > am already partitioning all these entities to different servers also on > > which i will be doing search based on distributed search by solr using > > shards and collecting merged results from 3 different servers. You > > mentioned > > i wont achieve putting all partitions on the same box , why is that so? > > > This is because each shard will compete for CPU and disk if you put them on > the same box. Logical separation and partitioning for performance are two > different things. You should partition if one Solr instance is not able to > hold the complete index or if it is not giving you the desired performance. > You can use multiple cores if the data is unrelated and you wouldn't need > to > search on all of them. > > In your case, the primary reason is performance, so it makes sense to put > each shard on a separate box. > > > > While reading solr core it says solr core is used for different > > applications > > only.... My search on different entities is also a type of different > > applications theoritically .... > > > > Does solr provides any good support for index partitioning. > > > No. Partitioning is not done by Solr. So you should decide your > partitioning > scheme: round robin, fixed hashing, random etc. Once you have partitioned > your data, a distributed search helps you search over all the shards in one > go. > > -- > Regards, > Shalin Shekhar Mangar. > -- Cheers.... Sumit
