Re: Replication in Master Slave Solr setup

2018-03-18 Thread Shawn Heisey
On 3/17/2018 8:06 PM, vracks wrote: Basic Questions about the Replication in Master Slave Solr Setup. 1) Can Master push the changes to Slaves using the replication handler Replication is always pull -- the slave asks the master if there's anything to copy. 2) If the Answer to the above qu

Indexing multi level Nested JSON

2018-03-18 Thread Zheng Lin Edwin Yeo
Hi, I have this sample multi level Nested JSON, with 2 level of child Documents. [ { "id": "1", "title_s": "Solr adds block join support", "contenttype_s": "parentDocument", "_childDocuments_": [ { "id": "3", "comments_s": "SolrCloud supports it too!", "_ch

Re: collection reload leads to OutOfMemoryError

2018-03-18 Thread Erick Erickson
Hendrik: There are a limited number of threads that load in parallel when starting up, depends on the configuration. The defaults are 3 threads in stand-alone and 8 in Cloud (see: NodeConfig.java) public static final int DEFAULT_CORE_LOAD_THREADS = 3; public static final int DEFAULT_CORE_LOAD_THR

Re: Some performance questions....

2018-03-18 Thread Walter Underwood
> On Mar 17, 2018, at 3:23 AM, Deepak Goel wrote: > > Sorry for being rude. But the ' results ' please, not the ' road to the > results ' We have 15 different search collections, all different sizes and all with different kinds of queries. Here are the two major ones. 22 million docs 32 server

Re: collection reload leads to OutOfMemoryError

2018-03-18 Thread Hendrik Haddorp
I increased the metaspace size to 2GB. This way I could do multiple rounds of reloading all collections already. The GC logs do show now an almost stable metaspace size. So maybe I did just set the limits too low. Still a bit odd that reloading the collections results in a higher memory usage.

Re: Looking for design ideas

2018-03-18 Thread Rick Leir
Steve Does a document have a different URL when it is in a personal DB? I suspect the easiest solution is to use just one index. You can have a field containing an integer identifying the personal DB. For public, set this to zero. Call it DBid. Update the doc to change this and the URL when th

collection reload leads to OutOfMemoryError

2018-03-18 Thread Hendrik Haddorp
Hi, I did a simple test on a three node cluster using Solr 7.2.1. The JVMs (Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 1.8.0_162 25.162-b12) have about 6.5GB heap and 1.5GB metaspace. In my test I have 1000 collections with only 1000 simple documents each. I'm then triggering collec

Re: Looking for design ideas

2018-03-18 Thread Rahul Singh
I’ve worked on something similar - data set was 100m documents with thousands of users. The ranking is relative in each index. Eg. What is #1 , #2, #3 is only 1,2,3 in that index. Your challenge will in the user interface result display: how to merge results in a way that the relevant results a

Looking for design ideas

2018-03-18 Thread Steven White
Hi everyone, I have a design problem that i"m not sure how to solve best so I figured I share it here and see what ideas others may have. I have a DB that hold documents (over 1 million and growing). This is known as the "Public" DB that holds documents visible to all of my end users. My applic