Real-time replication

[John Reuning]

Apologies if this has been covered.  I searched the archives and didn't 
see a thread on this topic.


Has anyone experimented with a near real-time replication scheme similar 
to RDBMS replication?  There's large efficiency in using rsync to copy 
the lucene index files to slaves, but what if you want index changes to 
propagate in a few seconds instead of a few minutes?


Is it feasible to make a solr manager take update requests and send them 
to slaves as it receives them?  (I guess maybe they're not really slaves 
in this case.)  The manager might issue commits every 10-30 seconds to 
reduce the write load.  Write overhead still exists on all read servers, 
but at least the read requests are spread across the pool.


Thanks,

-John R.

GET_SCORES flag in SolrIndexSearcher

[John Reuning]

The scores list in DocIterator is null after a successful query. 
There's a flag in SolrIndexSearcher, GET_SCORES, that looks like it 
should trigger setting the scores array for the resulting DocList, but I 
can't figure out how to set it.  Any suggestions?  I'm using the svn 
trunk code.


Many thanks,

-John R.

Re: GET_SCORES flag in SolrIndexSearcher

[John Reuning]

Ah ha, perfect.  That worked brilliantly.  In case anyone is interested, 
it turns out that defining "id score" as the field list for the standard 
request handler in solrconfig.xml does the same thing.


  

 
   explicit
   id score

  

Thanks for the help,

-jrr

Henrib wrote:

I believe that keeping you code as is but initializing the query parameters
should do the trick:

HashMap params = new HashMap();
params.add("fl", "id score"); // field list is id & score
...
Regards

Re: GET_SCORES flag in SolrIndexSearcher

[John Reuning]

So, I found the following in QueryParsing::parseSort

  if( "score".equals(part) ) {
if (top) {
  // If thre is only one thing in the list, just do the regular 
thing...

  if( parts.length == 1 ) {
return null; // do normal scoring...
  }
  lst[i] = SortField.FIELD_SCORE;
}
else {
  lst[i] = new SortField(null, SortField.SCORE, true);
}

Besides the typo :), the only problem for what I want is the fact that 
it returns null for a default score sort instead of setting 
SortField.FIELD_SCORE.  I want a default score desc sort, but I want the 
scores from the lucene Hits object.  Is the only way to get score values 
to modify the solr code?


Thanks,

-jrr

Chris Hostetter wrote:

: The scores list in DocIterator is null after a successful query. There's a
: flag in SolrIndexSearcher, GET_SCORES, that looks like it should trigger
: setting the scores array for the resulting DocList, but I can't figure out how
: to set it.  Any suggestions?  I'm using the svn trunk code.

Can you elaborate (ie: paste some code examples) on how you are aquiring 
your DocList ... what method are you calling on SolrIndexSearcher? what 
arguments are you passing it?


NOTE: the SolrIndexSearcher.getDocList* methods may choose to build 
the DocList from a DocSet unless:

a) you use a sort that inlcudes score
or  b) you use a method sig that takes a flags arg and explicitly set 
   the GET_SCORES mask on your flags arg.





-Hoss

Re: GET_SCORES flag in SolrIndexSearcher

[John Reuning]

My first pass was to implement the embedded solr example:

--
MultiCore mc = MultiCore.getRegistry();
SolrCore core = mc.getCore(mIndexName);

SolrRequestHandler handler = core.getRequestHandler("");
HashMap params = new HashMap();

SolrQueryRequest request = new LocalSolrQueryRequest(core, query, 
"standard", 0, 100, params);

SolrQueryResponse response = new SolrQueryResponse();
core.execute(handler, request, response);

DocList docs = (DocList) response.getValues().get("response");
--

Is the only way to access scores to call directly to SolrIndexSearcher? 
 I was wondering if there's a solr config option I'm missing somewhere 
that tells the SolrIndexSearcher to retain lucene scores.  I'll keep 
digging.  Maybe there's a way to set a LocalSolrQueryRequest param that 
passes the right info through to SolrIndexSearcher?


Thanks,

-jrr

Chris Hostetter wrote:

: The scores list in DocIterator is null after a successful query. There's a
: flag in SolrIndexSearcher, GET_SCORES, that looks like it should trigger
: setting the scores array for the resulting DocList, but I can't figure out how
: to set it.  Any suggestions?  I'm using the svn trunk code.

Can you elaborate (ie: paste some code examples) on how you are aquiring 
your DocList ... what method are you calling on SolrIndexSearcher? what 
arguments are you passing it?


NOTE: the SolrIndexSearcher.getDocList* methods may choose to build 
the DocList from a DocSet unless:

a) you use a sort that inlcudes score
or  b) you use a method sig that takes a flags arg and explicitly set 
   the GET_SCORES mask on your flags arg.





-Hoss

MultiCore unregister

[John Reuning]

For the MultiCore experts, is there an acceptable or approved way to 
close and unregister a single SolrCore?  I'm interested in stopping 
cores, manipulating the solr directory tree, and reregistering them.


Thanks,

-John R.

Re: MultiCore unregister

[John Reuning]

I was hoping that a feature was lurking about and not yet added to the 
patch.  How about something like this?  Should it throw an exception if 
the core isn't found in the map?


Thanks,

-jrr


--- MultiCore.java.orig 2007-11-07 23:09:32.0 -0500
+++ MultiCore.java  2007-11-07 23:14:08.0 -0500
@@ -125,6 +125,25 @@
 }
   }

+  /**
+   * Stop and unregister a core of the given name
+   *
+   * @param name
+   */
+  public void shutdown ( String name )
+  {
+if ( name == null || name.length() == 0 ) {
+  throw new RuntimeException("Invalid core name.");
+}
+synchronized ( cores ) {
+  SolrCore core = cores.get(name);
+  if ( core != null ) {
+cores.remove(name);
+core.close();
+  }
+}
+  }
+
   @Override
   protected void finalize() {
 shutdown();


Ryan McKinley wrote:

Nothing yet... but check:
https://issues.apache.org/jira/browse/SOLR-350

ryan


John Reuning wrote:
For the MultiCore experts, is there an acceptable or approved way to 
close and unregister a single SolrCore?  I'm interested in stopping 
cores, manipulating the solr directory tree, and reregistering them.


Thanks,

-John R.

Re: Multiple indexes

[John Reuning]

I've had good luck with MultiCore, but you have to sync trunk from svn 
and apply the most recent patch in SOLR-350.


https://issues.apache.org/jira/browse/SOLR-350

-jrr

Jae Joo wrote:

Hi,

I am looking for the way to utilize the multiple indexes for signle sole
instance.
I saw that there is the patch 215  available  and would like to ask someone
who knows how to use multiple indexes.

Thanks,

Jae Joo