Re: More on topic of Meta-search/Federated Search with Solr

2013-09-09 Thread Jakub Skoczen
Hi Dan, You might want to take a look at pazpar2 [1], an open-source, federated search engine with first-class support for SOLR (with addition to standard information retrieval protocols like Z39.50/SRU). [1] http://www.indexdata.com/pazpar2 On Thu, Sep 5, 2013 at 9:55 PM, Paul Libbrecht wrote

Re: More on topic of Meta-search/Federated Search with Solr

2013-09-05 Thread Paul Libbrecht
Hello list, A student of a friend of mine made his masters on that topic, especially about federated ranking. I have copied his text here: http://direct.hoplahup.net/tmp/FederatedRanking-Koblischke-2009.pdf Feel free to contact me to contact Robert Koblischke for questions. Pa

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-28 Thread Dan Davis
On Mon, Aug 26, 2013 at 9:06 PM, Amit Jha wrote: > Would you like to create something like > http://knimbus.com > I work at the National Library of Medicine. We are moving our library catalog to a newer platform, and we will probably include articles. The article's content and meta-data are

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-28 Thread Dan Davis
On Tue, Aug 27, 2013 at 3:33 AM, Bernd Fehling < bernd.fehl...@uni-bielefeld.de> wrote: > Years ago when "Federated Search" was a buzzword we did some development > and > testing with Lucene, FAST Search, Google and several other Search Engines > according Federated Search in Library context. > Th

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-28 Thread Dan Davis
On Tue, Aug 27, 2013 at 2:03 AM, Paul Libbrecht wrote: > Dan, > > if you're bound to federated search then I would say that you need to work > on the service guarantees of each of the nodes and, maybe, create > strategies to cope with bad nodes. > > paul > +1 I'll think on that.

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-27 Thread Bernd Fehling
, ... The trick with Federated Search is to combine the results. We offered three options to the users search surface: - RoundRobin - Relevancy - PseudoRandom -- View this message in context: http://lucene.472066.n3.nabble.com/More-on-topic-of-Meta-search-Federated-Search-with-Solr-tp4085167p40

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-26 Thread Paul Libbrecht
Dan, if you're bound to federated search then I would say that you need to work on the service guarantees of each of the nodes and, maybe, create strategies to cope with bad nodes. paul Le 26 août 2013 à 22:57, Dan Davis a écrit : > First answer: > > My employer is a library and do not have

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-26 Thread Amit Jha
Hi, I would suggest for the following. 1. Create custom search connectors for each individual sources. 2. Connector will responsible to query the source of any type web, gateways etc. and get the results & write the top N results to a solr. 3. Query the same keyword to solr and display the resu

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-26 Thread Dan Davis
One more question here - is this topic more appropriate to a different list? On Mon, Aug 26, 2013 at 4:38 PM, Dan Davis wrote: > I have now come to the task of estimating man-days to add "Blended Search > Results" to Apache Solr. The argument has been made that this is not > desirable (see Jo

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-26 Thread Dan Davis
First answer: My employer is a library and do not have the license to harvest everything indexed by a "web-scale discovery service" such as PRIMO or Summon.If our design automatically relays searches entered by users, and then periodically purges results, I think it is reasonable from a licens

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-26 Thread Paul Libbrecht
Why not simply create a meta search engine that indexes everything of each of the nodes.? (I think one calls this harvesting) I believe that this the way to avoid all sorts of performance bottleneck. As far as I could analyze, the performance of a federated search is the performance of the leas

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-26 Thread Dan Davis
I have now come to the task of estimating man-days to add "Blended Search Results" to Apache Solr. The argument has been made that this is not desirable (see Jonathan Rochkind's blog entries on Bento search with blacklight). But the estimate remains.No estimate is worth much without a desig

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-22 Thread Dan Davis
You are right, but here's my null hypothesis for studying the impact on relevance.Hash the query to deterministically seed random number generator.Pick one from column A or column B randomly. This is of course wrong - a query might find two non-relevant results in corpus A and lots of rele

Re: More on topic of Meta-search/Federated Search with Solr

2013-08-18 Thread Erick Erickson
The lack of global TF/IDF has been answered in the past, in the sharded case, by "usually you have similar enough stats that it doesn't matter". This pre-supposes a fairly evenly distributed set of documents. But if you're talking about federated search across different types of documents, then wh

More on topic of Meta-search/Federated Search with Solr

2013-08-16 Thread Dan Davis
I've thought about it, and I have no time to really do a meta-search during evaluation. What I need to do is to create a single core that contains both of my data sets, and then describe the architecture that would be required to do blended results, with liberal estimates. >From the perspective o