Thank you everyone for your responses. Increased the hard commit to 10mins and autoSoftCommit to 10 secs. (I wont really need a real time get - tweaked the app code to cache the doc and use the app side cached version instead of fetching it from Solr) Will watch it for a day or two and clock the throughput.
For this deployment the peak is throughout the day as more data keeps streaming in - there are no direct users with search queries here (as of now) - but every incoming doc is compared against the existing set of docs in Solr - to check whether its a new one or an updated version of an existing one and only then the doc is inserted/updated. Right now its adding about 1100 docs a minute (~20 docs a second) [But thats because it has to run a search before to determine whether its an insert/update] Also, since there are already 18 JVMs per machine - How do I go about merging these existing cores under just 1 JVM? Would it be that I'd need to create 1 Solr instance with 18 cores inside and then migrate data from these separate JVMs into the new instance? -- View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-Scale-Struggle-tp4150592p4150810.html Sent from the Solr - User mailing list archive at Nabble.com.
