Do you have IDs defined? How do you expect Sold to know they are duplicate
records? Maybe the issue is there somewhere.
Regards,
Alex
On 13 Dec 2012 15:17, "Dikchant Sahi" <contacts...@gmail.com> wrote:
> Hi Alex,
>
> You got my point right. What I see is merge adds duplicate document. Is
> there a way to overwrite existing document in one core by another. Can
> merge operation lead to data corruption, say in case when the core on
> client had uncommitted changes.
>
> What would be a better solution for my requirement, merge or indexing
> XML/JSON?
>
> Regards,
> Dikchant
>
> On Thu, Dec 13, 2012 at 6:39 PM, Alexandre Rafalovitch
> <arafa...@gmail.com>wrote:
>
> > Not sure I fully understood this and maybe you already cover that by
> > 'merge', but if you know what you gave the client last time, you can just
> > build a differential as a second core, then on client mount that second
> > core and merge it into the first one (e.g. with DIH).
> >
> > Just a thought.
> >
> > Regards,
> > Alex.
> >
> > Personal blog: http://blog.outerthoughts.com/
> > LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch
> > - Time is the quality of nature that keeps events from happening all at
> > once. Lately, it doesn't seem to be working. (Anonymous - via GTD book)
> >
> >
> >
> > On Thu, Dec 13, 2012 at 5:28 PM, Dikchant Sahi <contacts...@gmail.com
> > >wrote:
> >
> > > Hi Erick,
> > >
> > > Sorry for creating the confusion. By slave, I mean the indexes on
> client
> > > machine will be replica of the master and in not same as the slave in
> > > master-slave model. Below is the detail:
> > >
> > > The system is being developed to support search facility on 1000s of
> > > system, a majority of which will be offline.
> > >
> > > The idea is that we will have a search system which will be sold
> > > on subscription basis. For each of the subscriber, we will copy the
> > master
> > > index to their local machine, over a drive or CD. Now, if a subscriber
> > > comes after 2 months and want the updates, we just want to provide the
> > > deltas for 2 month as the volume of data is huge. For this we can think
> > of
> > > two approaches:
> > > 1. Fetch the documents which are less than 2 months old in JSON format
> > > from master Solr. Copy it to the subscriber machine
> > > and index those documents. (copy through cd / memory sticks)
> > > 2. Create separate indexes for each month on our master machine. Copy
> the
> > > indexes to the client machine and merge. Prior to merge we need to
> delete
> > > records which the new index has, to avoid duplicates.
> > >
> > > As long as the setup is new, we will copy the complete index and
> restart
> > > Solr. We are not sure of the best approach for copying the deltas.
> > >
> > > Thanks,
> > > Dikchant
> > >
> > >
> > >
> > > On Thu, Dec 13, 2012 at 3:52 AM, Erick Erickson <
> erickerick...@gmail.com
> > > >wrote:
> > >
> > > > This is somewhat confusing. You say that box2 is the slave, yet
> they're
> > > not
> > > > connected? Then you need to copy the <solr home>/data index from box
> 1
> > to
> > > > box 2 manually (I'd have box2 solr shut down at the time) and restart
> > > Solr.
> > > >
> > > > Why can't the boxes be connected? That's a much simpler way of going
> > > about
> > > > it.
> > > >
> > > > Best
> > > > Erick
> > > >
> > > >
> > > > On Tue, Dec 11, 2012 at 1:04 AM, Dikchant Sahi <
> contacts...@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi Walter,
> > > > >
> > > > > Thanks for the response.
> > > > >
> > > > > Commit will help to reflect changes on Box1. We are able to achieve
> > > this.
> > > > > We want the changes to reflect in Box2.
> > > > >
> > > > > We have two indexes. Say
> > > > > Box1: Master & DB has been setup. Data Import runs on this.
> > > > > Box2: Slave running.
> > > > >
> > > > > We want all the updates on Box1 to be merged/present in index on
> > Box2.
> > > > Both
> > > > > the boxes are not connected over n/w. How can be achieve this.
> > > > >
> > > > > Please let me know, if am not clear.
> > > > >
> > > > > Thanks again!
> > > > >
> > > > > Regards,
> > > > > Dikchant
> > > > >
> > > > > On Tue, Dec 11, 2012 at 11:22 AM, Walter Underwood <
> > > > wun...@wunderwood.org
> > > > > >wrote:
> > > > >
> > > > > > You do not need to manage online and offline indexes. Commit when
> > you
> > > > are
> > > > > > done with your updates and Solr will take care of it for you. The
> > > > changes
> > > > > > are not live until you commit.
> > > > > >
> > > > > > wunder
> > > > > >
> > > > > > On Dec 10, 2012, at 9:46 PM, Dikchant Sahi wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > How can we do delta update of offline indexes?
> > > > > > >
> > > > > > > We have the master index on which data import will be done. The
> > > index
> > > > > > > directory will be copied to slave machine in case of full
> update,
> > > > > through
> > > > > > > CD as the slave/client machine is offline.
> > > > > > > So, what should be the approach for getting the delta to the
> > > slave. I
> > > > > can
> > > > > > > think of two approaches.
> > > > > > >
> > > > > > > 1.Create separate indexes of the delta on the master machine,
> > copy
> > > it
> > > > > to
> > > > > > > the slave machine and merge. Before merging the indexes on the
> > > client
> > > > > > > machine, delete all the updated and deleted documents in client
> > > > machine
> > > > > > > else merge will add duplicates. So along with the index, we
> need
> > to
> > > > > > > transfer the list of documents which has been updated/deleted.
> > > > > > >
> > > > > > > 2. Extract all the documents which has changed since a
> particular
> > > > time
> > > > > in
> > > > > > > XML/JSON and index it in client machine.
> > > > > > >
> > > > > > > The size of indexes are huge, so we cannot rollover index
> > > everytime.
> > > > > > >
> > > > > > > Please help me with your take and challenges you see in the
> above
> > > > > > > approaches. Please suggest if you think of any other better
> > > approach.
> > > > > > >
> > > > > > > Thanks a ton!
> > > > > > >
> > > > > > > Regards,
> > > > > > > Dikchant
> > > > > >
> > > > > > --
> > > > > > Walter Underwood
> > > > > > wun...@wunderwood.org
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
