Re: Data Import Handler, also "Real Time" index updates

2017-03-05 Thread Damien Kamerman
You could configure the dataimporthandler to not delete at the start (either do a delta or set the preimportdeltequery), and set a postimportdeletequery if required. On Saturday, 4 March 2017, Alexandre Rafalovitch wrote: > Commit is index global. So if you have overlapping timelines and commit

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Alexandre Rafalovitch
Commit is index global. So if you have overlapping timelines and commit is issued, it will affect all changes done to that point. So, the aliases may be better for you. You could potentially also reload a cure with changes solrconfig.XML settings, but that's heavy on caches. Regards, Alex On

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Sales
> > You have indicated that you have a way to avoid doing updates during the > full import. Because of this, you do have another option that is likely > much easier for you to implement: Set the "commitWithin" parameter on > each update request. This works almost identically to autoSoftCommit,

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Shawn Heisey
On 3/3/2017 10:17 AM, Sales wrote: > I am not sure how best to handle this. We use the data import handle to > re-sync all our data on a daily basis, takes 1-2 hours depending on system > load. It is set up to commit at the end, so, the old index remains until it’s > done, and, we lose no access

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Sales
> On Mar 3, 2017, at 11:30 AM, Erick Erickson wrote: > > One way to handle this (presuming SolrCloud) is collection aliasing. > You create two collections, c1 and c2. You then have two aliases. when > you start "index" is aliased to c1 and "search" is aliased to c2. Now > do your full import to

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Erick Erickson
One way to handle this (presuming SolrCloud) is collection aliasing. You create two collections, c1 and c2. You then have two aliases. when you start "index" is aliased to c1 and "search" is aliased to c2. Now do your full import to "index" (and, BTW, you'd be well advised to do at least a hard co

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Sales
> > On Mar 3, 2017, at 11:22 AM, Alexandre Rafalovitch wrote: > > On 3 March 2017 at 12:17, Sales > wrote: >> When we enabled those, during the index, the data disappeared since it kept >> soft committing during the import process, > > This part does not quite make sense. Could you expand on

Re: Data Import Handler, also "Real Time" index updates

2017-03-03 Thread Alexandre Rafalovitch
On 3 March 2017 at 12:17, Sales wrote: > When we enabled those, during the index, the data disappeared since it kept > soft committing during the import process, This part does not quite make sense. Could you expand on this "data disappeared" part to understand what the issue is. The main issue