Hi, Thanks.
That was my second option.
But I was hoping that the master and slaves could find that out for
themselves. As now I have to also have my 'updater software' know about
all the slaves (and maybe even their state). Which it previously had no
idea about.
This way I can't just plugin a 'empty' slave that knows where it's
master is and have it pull in all the required cores and indexes.
Thijs
On 8-12-2009 14:25, Joe Kessel wrote:
Hi,
In my environment I create cores on the fly, then replicate the core to all of the slaves. I first
create the core on the master and persist the solr.xml via the CoreAdmin API. I then do the same on
each of my slaves. After loading / committing / optimizing the data on the master I send the
replication request to each of the slaves. So each slave's replication handler
http://<slave_host_port>/solr/<core_name>/replication gets a request to fetch index which
includes the master url http://<master_host_port>/solr/<core_name>/replication;
The slave's solrconf.xml has no mention of the master as it is all done
progromatically. You need to specify the core name in the url, and if you
haven't created the core on the master it will result in error.
I don't create a new core everytime I update, but I do have the slaves fetch
the index after every update. My first attempt to set the polling did not seem
to work, and have not had a chance to revisit. I have not found a way to
persist the solrconfig.xml with the updates to the slave list, so the control /
management is within my application.
Hope this highlevel overview helps.
Joe
Date: Tue, 8 Dec 2009 12:42:12 +0100
From: vonk.th...@gmail.com
To: solr-user@lucene.apache.org
Subject: Re: How to setup dynamic multicore replication
But the slave never gets the message that a core is created...
at least not in my setup...
So it never starts replicating...
On 8-12-2009 12:13, Noble Paul നോബിള് नोब्ळ् wrote:
On Tue, Dec 8, 2009 at 2:43 PM, Thijs<vonk.th...@gmail.com> wrote:
Hi
I need some help setting up dynamic multicore replication.
We are changing our setup from a replicated single core index with multiple
document types, as described on the wiki[1], to a dynamic multicore setup.
We need this so that we can display facets with a zero count that are unique
to the document 'type'.
So when indexing new documents we want to create new cores on the fly using
the CoreAdminHandler through SolrJ.
What I can't figure out is how I setup solr.xml and solrconfig.xml so that a
core automatically is also replicated from the master to it's slaves once
it's created.
I have a solr.xml that starts like this:
<?xml version='1.0' encoding='UTF-8'?>
<solr persistent="true">
<cores adminPath="/admin/cores">
</cores>
</solr>
and the replication part of solrconfig.xml
master:
<requestHandler name="/replication" class="solr.ReplicationHandler">
<lst name="master">
<str name="replicateAfter">startup</str>
<str name="replicateAfter">optimize</str>
<str name="confFiles">schema.xml</str>
</lst>
</requestHandler>
slave:
<requestHandler name="/replication" class="solr.ReplicationHandler">
<lst name="slave">
<str name="masterUrl">http://localhost:8081/solr/replication</str>
<str name="pollInterval">00:00:20</str>
</lst>
</requestHandler>
I think I should change the masterUrl in the slave configuration to
something like:
<str
name="masterUrl">http://localhost:8081/solr/${solr.core.name}/replication</str>
So that the replication automatically finds the correct core replication
handler.
if you have dynamically created cores this is the solution.
But how do I tell the slaves a new core is created, and that is should start
replicating those to?
Thanks in advance.
Thijs
[1]
http://wiki.apache.org/solr/MultipleIndexes#Flattening_Data_Into_a_Single_Index
_________________________________________________________________
Windows Live Hotmail is faster and more secure than ever.
http://www.microsoft.com/windows/windowslive/hotmail_bl1/hotmail_bl1.aspx?ocid=PID23879::T:WLMTAGL:ON:WL:en-ww:WM_IMHM_1:092009
