XJoin, a way to use external data sources with Solr

Hi all, We've recently built a patch that allows you to incorporate external data into Solr queries - this is for data that for some reason you can't index directly. My colleague has written a couple of blog posts illustrating how to use it in an e-commerce setting, hope you find them useful:

Nested documents and many-many relation

Hi, Am trying to investigate the possibility of using Block Join query parser in a many-to-many relation scenario. Observation is that when a document is added as a child to more than one parent document (I use Solrj to do this), I seem to get two copies of the child document. Can this be avoid

Re: implement exact match for one of the search fields only?

Jan sorry If i insist, but I really don't see the benefit. If you use the edismax and a copy field not tokenised, you can apply the boost you want to the exact match, and the query will simply be q=foo. And you obtain exactly what you do without the markers. But please, explain me how your solutio

Re: implement exact match for one of the search fields only?

Hi Did you try? Sadly, the pf param will not apply to a non-analyzed field, and is not added if query consists of one term only. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 29. jan. 2016 kl. 11.14 skrev Alessandro Benedetti : > > Jan sorry If i insist, but I re

Re: implement exact match for one of the search fields only?

Derek, the link is not available to the public. Did you restart and re-index after changing the field type? -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com > 29. jan. 2016 kl. 04.21 skrev Derek Poh : > > Hi Erick and all > > Yes I am trying to apply the same search t

Fwd: Highlights on Expanded results

Hi, Solr expanded results are not highlighted and there are no utilities to use fragment and highlight explicitly, any alternative to support highlight expand results feature ? Please let me know if we can use the highlight handlers explicitly. Thanks in advance. Regards, Anil

Re: implement exact match for one of the search fields only?

Hi Derek, What if it does not match other fields but just exact match. From original question I assume it should return such results. It seems to me that you are AND-ing your fields and that is the reason why your query is not returning anything. Can you try just exact match field and see if i

Re: collection aliasing

Hi How can a full index rebuild can be done ? -- View this message in context: http://lucene.472066.n3.nabble.com/collection-aliasing-tp4252527p4254077.html Sent from the Solr - User mailing list archive at Nabble.com.

Restoring backups of solrcores

Hi I wanted to restore the core into a new one in solrcloud by following the below link. https://cwiki.apache.org/confluence/display/solr/Making+and+Restoring+Backups+of+SolrCores But I couldnt get the success value when restore is given. It cant be implemented for solrcloud? If so, then what is

Re: How much JVM should we allocate

On 1/28/2016 10:24 PM, Midas A wrote: > CPU : 4 > physical memory : 48 GB > > > and we are only have solr on this server . How much JVM can be allocate to > run server smoothly. We don't know. You haven't provided any information about your index or how you use it. Even if you do provide that i

Re: collection aliasing

On 1/29/2016 5:00 AM, vidya wrote: > How can a full index rebuild can be done ? Here is the "not exactly helpful" wiki page on this topic: https://wiki.apache.org/solr/HowToReindex Thanks, Shawn

Re: Highlights on Expanded results

I'm wondering how difficult it would be to get the highlighter working on expanded results. I think it's worth creating a jira request for this. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jan 29, 2016 at 5:43 AM, Anil wrote: > Hi, > > Solr expanded results are not highlighted and ther

Exporting Score value from export handler

Hi, I would like to issue a query and get the ID and Score for each matching document. There may be lots of results so I wanted to use the export handler, but unfortunately the current version of Solr doesn't seem to export the Score - I read the comments on https://issues.apache.org/jira/brow

Re: Highlights on Expanded results

Maybe as a DocTransformer? > On Jan 29, 2016, at 08:23, Joel Bernstein wrote: > > I'm wondering how difficult it would be to get the highlighter working on > expanded results. I think it's worth creating a jira request for this. > > Joel Bernstein > http://joelsolr.blogspot.com/ > >> On Fri, J

Re: Exporting Score value from export handler

Exporting scores would be a great feature to have. I don't believe it will add too much complexity to export and sort by score. The main consideration has been memory consumption for every large export sets. The export feature powers SQL queries that are unlimited in Solr 6. So adding scores to exp

Re: Highlights on Expanded results

Yeah, a DocTransformer would seem to be a good route to go. Not sure how difficult it would be to do highlighting through the doc transformer. Joel Bernstein http://joelsolr.blogspot.com/ On Fri, Jan 29, 2016 at 8:32 AM, Erik Hatcher wrote: > Maybe as a DocTransformer? > > > On Jan 29, 2016, at

Re: Highlights on Expanded results

There is an open JIRA for this - https://issues.apache.org/jira/browse/SOLR-3479 > On Jan 29, 2016, at 8:50 AM, Joel Bernstein wrote: > > Yeah, a DocTransformer would seem to be a good route to go. Not sure how > difficult it would be to do h

dynamic fields | Naming conventions

Hi, We have decided to use dynamic fileds in solr for some of our use-case. What we found in examples that these are declared as: It is based upon suffix approach. What we are think to implement is: According to, second approach would be more human readable and maintainable

Re: dynamic fields | Naming conventions

Personally, I’d stick with Solr’s “built-in” dynamic field definitions just keep things smooth for future developers and ease of matching up what you’ll see on this list and via other support channels. You can use field aliasing if you know which dynamic field you want… fl=price:price_f sort of

NLP Search with Solr

Hi I am in the Facebook advertising domain, trying to build a natural language search engine using Apache Solr and IBM Watson Retrieve and Rank service. I currently have my data in a Postgresql database, where the tables are named by company, fb_ad_revenue, targeting_info, etc. which are updated d

Re: implement exact match for one of the search fields only?

Still not following. Why you would like to use pf for exact search ( which is different from phrase search) ? Let's summarise : pf Phrase Fields: boosts the score of docume

Re: NLP Search with Solr

To be fair, that part does not seem NLP search at all. Can you explain more why it is NLP ? Pretty sure you can get what you want playing properly with Json Faceting[1]. You can model your Solr schema in an appropriate way and then playing with range queries and Json faceting at different levels.

Calculating tf-idf

Dear all, I am working on a research project in which I create an OS tool which tries to detect "bad" and "good" records in a metadata collection (such as a library catalog, museum database etc. -- you can find more info here http://pkiraly.github.io/). This is not the first project of that kind,

Re: Restoring backups of solrcores

It sounds like collection aliasing would be a perfect fit for you - https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api4 On Fri, Jan 29, 2016 at 6:24 AM, vidya wrote: > Hi > > I wanted to restore the core into a new one in solrcloud by following the > below link.

Re: Nested documents and many-many relation

If you wish to change, add, or delete a child or change the parent you must do an add of the entire block again with both the parent and all children. This is because the efficiency of Block Join comes from the documents being adjacent in Lucene and segments are immutable in Lucene, so the entire b

Re: Highlights on Expanded results

Another jira for the same, https://issues.apache.org/jira/browse/SOLR-6690 Hi joel, could you please share some information on DocTransformer? any sample example ? Regards, Anil On 29 January 2016 at 19:26, Erik Hatcher wrote: > There is an open JIRA for this - > https://issues.apache.org/jir

Re: Calculating tf-idf

I found the solution: https://wiki.apache.org/solr/TermVectorComponent. I did not know that before, but that's exactly what I need. Regards, Péter 2016-01-29 16:09 GMT+01:00 Péter Király : > Dear all, > > I am working on a research project in which I create an OS tool which > tries to detect "bad

Re: SolrCloud replicas out of sync

Tomás, Good find, but I don’t think the rate of updates was high enough during the network outage to create the overrun situation described in the ticket. I did notice that one of the proposed fixes, https://issues.apache.org/jira/browse/SOLR-8586, is an entire-index consistency check between

Re: Nested documents and many-many relation

Probably if you are interested in a many-to-many relation, you could be interested in the query time join. it has been the first type of join integrated in Solr. It allow you to avoid redundancies. It's slower than block join, but it doesn't force you to any specific indexing approach. It became le

Re: collection aliasing

bq: How can a full index rebuild can be done ? > The points to the current collection (collection1) > create a new collection (collection 2) > full index to collection2 > point the alias at collection2 Best, Erick On Fri, Jan 29, 2016 at 5:20 AM, Shawn Heisey wrote: > On 1/29/2016 5:00 AM, vi

Re: How much JVM should we allocate

And adding to Shawn's comment you want to have as little JVM as possible, see: http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html Best, Erick On Fri, Jan 29, 2016 at 5:02 AM, Shawn Heisey wrote: > On 1/28/2016 10:24 PM, Midas A wrote: >> CPU : 4 >> physical memory : 48 GB >>

Re: User-defined properties and configsets

These are system properties, right? They go in the startup for all of your Solr instances scattered about your cluster. The bin/solr script has a -a option for passing additional stuff to the JVM... Best, Erick On Thu, Jan 28, 2016 at 11:50 PM, Georg Sorst wrote: > Any takers? > > Georg Sorst

Re: Nested documents and many-many relation

Hello, This implies that an indexing extracts cliques of bipartite graph. Then, every clique goes as a single block with a sentinel parent document. And this parent document can carry incidence matrix as, let's say, binary docvalues. Then, a bunch of custom components can to handle this model. On

Re: How much JVM should we allocate

Ultimately, your JVM heap size will likely be somewhere in the 4GB to 12GB range. Alas, you have to use trial and error to size it. If it's too small you will hit OOM or performance degradation due to frequent GC. If it's too large you will accumulate way too much garbage before a GC hits and then

RE: How to convert string field to date

Thanks steve. Workaround 2 is working fine. Thanks again. --sreenivasa kallu -Original Message- From: Steve Rowe [mailto:sar...@gmail.com] Sent: Thursday, January 28, 2016 6:03 PM To: solr-user@lucene.apache.org Subject: Re: How to convert string field to date Try workaround 2, I did an

Re: Mysql data import issue

Thanks Gora for your Suggestions. Since my table contains lot of fields, and all the other fields have same name mapping for solr and mysql, I thought I can give mapping for the one which is different and leave remaining as is. But is not selecting the id field in the returned query the only way to