If you for instance use SolrJ and the HttpSolrServer, you could for
instance add logic to your querying making your searches more efficient!
That is partially the idea of sharding, right? :) So if the user wants to
search for a log file in June, your application knows that June logs are
stored on the second box, and hence will redirect the search to that box.
Alternatively if he wants to search for logs spanning two boxes, you
merely add the shards parameter to your query and just include the path to
those to shards in question. I'm not really sure about how solr handles
the merging of results etc and wether or not the requests are done in
paralell or sequentially, but I do know that you could easily manage this
on your own through java if you want to. (Simply setting up one
HttpSolrServer in your code for each shard, and searching them in
parallell in separate threads. => then reducing the results afterwards).
Have a look at http://wiki.apache.org/solr/DistributedSearch for more info.
You could also take a look at Hadoop. (http://hadoop.apache.org/)
regards,
Aleks
On Mon, 24 Nov 2008 06:24:51 +0100, souravm <[EMAIL PROTECTED]> wrote:
Hi,
Looking for some insight on distributed search.
Say I have an index distributed in 3 boxes and the index contains time
and text data (typical log file). Each box has index for different
timeline - say Box 1 for all Jan to April, Box 2 for May to August and
Box 3 for Sep to Dec.
Now if I try to search for a text string, will the search would happen
in parallel in all 3 boxes or sequentially?
Regards,
Sourav
**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended
solely
for the use of the addressee(s). If you are not the intended recipient,
please
notify the sender by e-mail and delete the original message. Further,
you are not
to copy, disclose, or distribute this e-mail or its contents to any
other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys
has taken
every reasonable precaution to minimize this risk, but is not liable for
any damage
you may sustain as a result of any virus in this e-mail. You should
carry out your
own virus checks before opening the e-mail or attachment. Infosys
reserves the
right to monitor and review the content of all messages sent to or from
this e-mail
address. Messages sent to or from this e-mail address may be stored on
the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***
--
Aleksander M. Stensby
Senior software developer
Integrasco A/S
www.integrasco.no
