Re: Solr Cloud and Multiple Indexes

2015-11-08 Thread Modassar Ather
As per my understanding if the data getting indexed is completely different and does not fall into same schema they can be segregated for indexing. But if they fit into same schema then it is better to keep them in same index and if the index size grows then switch to SolrCloud as it has lots of be

Re: Solr Cloud and Multiple Indexes

2015-11-08 Thread Salman Ansari
Just to give you a context of what I am talking about, I am collecting data from different sources (such as articles, videos etc.). Moreover, I will be doing enrichment on the data such as Entity Extraction. From my previous experiment with Solr what I was doing is dumping all articles, videos meta

Re: Solr Cloud and Multiple Indexes

2015-11-05 Thread Modassar Ather
Thanks for your response. I have already gone through those documents before. My point was that if I am using Solr Cloud the only way to distribute my indexes is by adding shards? and I don't have to do anything manually (because all the distributed search is handled by Solr Cloud). Yes as per my

Re: Solr Cloud and Multiple Indexes

2015-11-05 Thread Salman Ansari
Thanks for your response. I have already gone through those documents before. My point was that if I am using Solr Cloud the only way to distribute my indexes is by adding shards? and I don't have to do anything manually (because all the distributed search is handled by Solr Cloud). What is the Xm

Re: Solr Cloud and Multiple Indexes

2015-11-05 Thread Modassar Ather
SolrCloud makes the distributed search easier. You can find details about it under following link. https://cwiki.apache.org/confluence/display/solr/How+SolrCloud+Works You can also refer to following link: https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud >Fro

Re: Solr Cloud and Multiple Indexes

2015-11-05 Thread Salman Ansari
Here is the current info How much memory is used? Physical memory consumption: 5.48 GB out of 14 GB. Swap space consumption: 5.83 GB out of 15.94 GB. JVM-Memory consumption: 1.58 GB out of 3.83 GB. What is your index size? I have around 70M documents distributed on 2 shards (so each shard has 35M

Re: Solr Cloud and Multiple Indexes

2015-11-05 Thread Modassar Ather
What is your index size? How much memory is used? What type of queries are slow? Are there GC pauses as they can be a cause of slowness? Are document updates/additions happening in parallel? The queries are very slow to run so I was thinking to distribute the indexes into multiple indexes and cons