Re: DIH delta import with cache 5.3.1 issue

2017-06-20 Thread Sujay Bawaskar
Hi, Did not encounter this issue with solr 6.x. But delta import with cache executes nested query for every element encountered in parent query. Since this select does not have where clause because we are using cache, it takes long time. So delta import witch cache is very slow. My observation is

Re: DIH delta import with cache 5.3.1 issue

2017-03-16 Thread Sujay Bawaskar
Thanks Alex. I will test it with 5.4 and 6.4 and let you know. On Thu, Mar 16, 2017 at 7:40 PM, Alexandre Rafalovitch wrote: > You have nested entities and accumulate the content of the inner > entities in the outer one with caching on an inner one. Your > description sounds like the inner cache

Re: DIH delta import with cache 5.3.1 issue

2017-03-16 Thread Alexandre Rafalovitch
You have nested entities and accumulate the content of the inner entities in the outer one with caching on an inner one. Your description sounds like the inner cache is not reset on the next iteration of the outer loop. This may be connected to https://issues.apache.org/jira/browse/SOLR-7843 (Fixe

Re: DIH delta import with cache 5.3.1 issue

2017-03-16 Thread Sujay Bawaskar
This behaviour is for delta import only. One document get field values of all documents. These fields are child entities which maps column to multi valued fields. On Thu, Mar 16, 2017 at 6:35 PM, Alexandre Rafalovitch wrote: > Could you give a bit more details. Do y

Re: DIH delta import with cache 5.3.1 issue

2017-03-16 Thread Alexandre Rafalovitch
Could you give a bit more details. Do you mean one document gets the content of multiple documents? And only on delta? Regards, Alex On 16 Mar 2017 8:53 AM, "Sujay Bawaskar" wrote: Hi, We are using DIH with cache(SortedMapBackedCache) with solr 5.3.1. We have around 2.8 million documents i

DIH delta import with cache 5.3.1 issue

2017-03-16 Thread Sujay Bawaskar
Hi, We are using DIH with cache(SortedMapBackedCache) with solr 5.3.1. We have around 2.8 million documents in solr and total index size is 4 GB. DIH delta import is dumping all values of mapped columns to their respective multi valued fields. This is causing size of one solr document upto 2 GB. I