Query regarding solr plugin.

2011-04-22 Thread rajini maski
One doubt regarding adding the solr plugin. I have a new java file created that includes few changes in SynonymFilterFactory.java. I want this java file to be added to solr instance. I created a package as : org.apache.pco.search This includes OrcSynonymFilterFactory java class extends

Multi-word Solr Synonym issue

2011-04-22 Thread Pla Gong
I am trying to do a simple mapping of a 2 word term to a 1 word term and it does not work. See my configuration at the bottom of the email. My scenario is that I have a term called "pond care" and I want to map it to the term "fountain". So whenever a user enters the term "pond care" in the search

Re: How to return score without using _val_

2011-04-22 Thread Em
Hi, did you have a look at the query()-function mentioned in the Wiki? It sounds like something you should give a try! Regards, Em Bill Bell wrote: > > I know that the _val_ is the only thing influencing the score. > > The fq is just to limit also by those queries. > > What I am asking is if

Re: solr sorting problem

2011-04-22 Thread Pratik
Were you able to get it work .. if yes how ? I'm having almost the same problem. I used the " fieldType name="alphaOnlySort" class="solr.TextField" as in the sample schema.xml , to define a field named "alphaname". Then copied from one of the fields name "foodDescUS" to "alphaname". When i

Re: Indexing 20M documents from MySQL with DIH

2011-04-22 Thread Erick Erickson
{{{A custom indexer, so that's a fairly common practice? So when you are dealing with these large indexes, do you try not to fully rebuild them when you can? It's not a nightly thing, but something to do in case of a disaster? Is there a difference in the performance of an index that was built all

Re: Query regarding solr plugin.

2011-04-22 Thread Erick Erickson
First I appreciate your writeup of the problem, it's very helpful when people take the time to put in the details I can't reconcile these two things: {{{ as org.apache.solr.common.SolrException: Error loading class 'pointcross.orchSynonymFilterFactory' at}}} This seems to indicate that your

Re: How to return score without using _val_

2011-04-22 Thread Yonik Seeley
On Fri, Apr 22, 2011 at 12:26 AM, Bill Bell wrote: > I know that the _val_ is the only thing influencing the score. What creates the score is the main query. There are tons of ways to build up that main query in different ways. So the answer to your question is "yes", you can influence the score

Re: solr sorting problem

2011-04-22 Thread Erick Erickson
Let's see the query you submit. This looks like a typo or an improperly specified field name ":foodDesc" Best Erick On Fri, Apr 22, 2011 at 8:18 AM, Pratik wrote: > Were you able to get it work .. if yes how ? > I'm having almost the same problem. > > I used the " fieldType name="alphaOnlyS

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Ofer Fort
Nobody? Am I the only one in need of upgrading an index that was created with 1.4.1? Thanks for any info Ofer On Friday, April 22, 2011, Ofer Fort wrote: > Hi all, > While doing some tests, I realized that an index that was created with > solr 1.4.1 is readable by solr 3.1, but nt readable by so

Re: Multi-word Solr Synonym issue

2011-04-22 Thread Otis Gospodnetic
Hi, Maybe you are doing query-time synonym expansion? Try changing that to do index-time synonym expansion. See http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: h

Re: Solr search based on list of terms. Order by max(score) for each term.

2011-04-22 Thread Otis Gospodnetic
Hi, You didn't say much about how your backend is configured, so it's hard to tell, but I imagine you could have multiple fields based on the same original data and one of those fields could be highly boosted (via dismax/edismax) field for exact matches. Otis Sematext :: http://sematex

Re: testing of stemming

2011-04-22 Thread Otis Gospodnetic
Bryan, Have a look at page 111 of Lucene in Action 2, section 4.1. Is that the sort of thing you are after? If so, we may have some code that produced that in the LIA2 source code download... You could also just write a small app/script that calls (via HTTP/SolrJ) one of the Solr analysis re

Re: Solr indexing size for a particular document.

2011-04-22 Thread Otis Gospodnetic
Rahul, Here's a suggestion: Write a simple app that uses *Lucene* to create N indices, one for each of the documents you want to test. Then you can look at their sizes on disk. Not sure if it's super valuable to see sizes of individual documents, but you can do it as described above. Of course

Re: Query performance

2011-04-22 Thread Otis Gospodnetic
Charles, Grab Solr nightly build and try that. Should be much faster. n.b. you don't need 10 in your config any more. (although this looks like a config from your master, not slave, if you are using that sort of setup) Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nu

Re: Solr indexing size for a particular document.

2011-04-22 Thread rahul
thanks for all your inputs. On Fri, Apr 22, 2011 at 8:36 PM, Otis Gospodnetic-2 [via Lucene] < ml-node+2851624-1936255218-340...@n3.nabble.com> wrote: > Rahul, > > Here's a suggestion: > Write a simple app that uses *Lucene* to create N indices, one for each of > the > documents you want to tes

Re: Understanding the DisMax tie parameter

2011-04-22 Thread Otis Gospodnetic
Thanks Tom! I think I've seen a good explanation of tie from Hoss once something that described the background for tie beyond "it's good for breaking score ties between two documents". For example, what are the scenarios where one can expect or fear scoring ties between multiple documents

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Otis Gospodnetic
Hi Ofer, We recently helped a customer go through just such an upgrade (or maybe even from 1.3.*). We used a tool that read data from one index and indexed it to the new index without having to reindex the data from the original sources. All fields in the source index were obviously stored.

RE: Solr - Multi Term highlighting issue

2011-04-22 Thread Ramanathapuram, Rajesh
Does anybody has other suggestions? thanks & regards, Rajesh Ramana Enterprise Applications, Turner Broadcasting System, Inc. 404.878.7474 -Original Message- From: Ramanathapuram, Rajesh [mailto:rajesh.ramanathapu...@turner.com] Sent: Wednesday, April 20, 2011 2:51 PM To: solr-user@lu

Re: partial optimize does not reduce the segment number to maxNumSegments

2011-04-22 Thread Otis Gospodnetic
Hi Renee, Here's what I'd do: * Check how many open files your system is set up for (ulimit -n). You likely want to increase that (1024 seems to be a common default under Linux, and in the past I've set that to 30k+ without issues) * Look at your mergeFactor. If it's high, consider lowering i

Localized alphabetical order

2011-04-22 Thread Ben Preece
As someone who's new to Solr/Lucene, I'm having trouble finding information on sorting results in localized alphabetical order. I've ineffectively searched the wiki and the mail archives. I'm thinking for example about Hawai'ian, where mīka (with an i-macron) comes after mika (i without the ma

Re: Localized alphabetical order

2011-04-22 Thread Robert Muir
please see http://wiki.apache.org/solr/UnicodeCollation In general the idea is similar to how this is handled in databases, you can index collation keys into a sort field at analysis time, then you just do a standard solr sort. However, I am not sure if your JRE provides a "haw" Locale for the Ha

Re: Localized alphabetical order

2011-04-22 Thread Peter Keegan
On Fri, Apr 22, 2011 at 12:33 PM, Ben Preece wrote: > As someone who's new to Solr/Lucene, I'm having trouble finding information > on sorting results in localized alphabetical order. I've ineffectively > searched the wiki and the mail archives. > > I'm thinking for example about Hawai'ian, where

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Ofer Fort
Thanks Otis, but this is not my case. Most of my fields are not stored , but I do have the original data in case I need to reindex. My question is do I need to? If my 1.4.1 can be read by 3.1, I assume 3.1 can continue to write to it? In that case, I continue assuming that 4.0 will know how to read

DIH Transform XML?

2011-04-22 Thread Matt Galvin
Hello, First post here... I spent some time researching this but can't seem to find the answer I am looking for... I have a MySQL DB that I have Solr indexing and all is well. However, one field I need to index is a text field that contains XML stored in the DB. I read up on DIH Transformers a b

Re: Localized alphabetical order

2011-04-22 Thread Bently Preece
Thank you. This looks like the right direction. I see the docs say ICUCollationKeyFilterFactory is deprecated in favor of ICUCollationField. So ... I'd implement a subclass of ICUCollationField, and use that as the fieldtype in schema.xml. And this means - what? - that I'd also implement a cust

Re: Need to create dyanamic indexies base on different document workspaces

2011-04-22 Thread Marc Sturlese
In case you need to create lots of indexes and register/unregister fast, there is work on the way http://wiki.apache.org/solr/LotsOfCores -- View this message in context: http://lucene.472066.n3.nabble.com/Need-to-create-dyanamic-indexies-base-on-different-document-workspaces-tp2845919p2852410.ht

Re: Localized alphabetical order

2011-04-22 Thread Robert Muir
On Fri, Apr 22, 2011 at 2:37 PM, Bently Preece wrote: > Thank you.  This looks like the right direction. > > I see the docs say ICUCollationKeyFilterFactory is deprecated in favor of > ICUCollationField.  So ... I'd implement a subclass of ICUCollationField, > and use that as the fieldtype in sche

Re: Localized alphabetical order

2011-04-22 Thread Bently Preece
What if there is no standard localization already? The case I'm specifically interested in is Ojibwe. So should I really be researching how the JRE does localization instead of Solr? On Fri, Apr 22, 2011 at 2:01 PM, Robert Muir wrote: > On Fri, Apr 22, 2011 at 2:37 PM, Bently Preece wrote: >

Ant is not working in Eclipse

2011-04-22 Thread Em
Hello list, there is a problem with the SVN-Checkout of the current Solr-version, I think. I can run ant eclipse, it does not show any errors (needed 20 seconds the first time and 0.9 seconds afterwards). However, the classpath-files were not set properly. A click on refresh did not show the expe

Re: Ant is not working in Eclipse

2011-04-22 Thread Em
I have to correct myself. I just tried to copy the files manually to the correct destinations. It showed that those files already are there (however, they did not show up in the terminal). What else could be responsible for the fact that the click on refresh does not show a developer-like view?

Re: Localized alphabetical order

2011-04-22 Thread Robert Muir
On Fri, Apr 22, 2011 at 3:09 PM, Bently Preece wrote: > What if there is no standard localization already?  The case I'm > specifically interested in is Ojibwe. > this is standard? to sort a field with a specific locale, you have to tell it the locale you want. if you use the ICU implementation y

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Otis Gospodnetic
Regardless of what anyone here says, you need to try it. 3.1 should be able to read 1.4.1, yes. One the format is switched to 3.1, you can't go back and read it with 1.4.1. This is why you want to upgrade your Slaves first, then your Master (if you have them -- I remember we spoke a while back

Re: Index upgrade from 1.4.1 to 3.1 and 4.0

2011-04-22 Thread Ofer Fort
Thanks, I'll do the procedure on my test env and update the community, if anybody already went through the process, I would lov to here about it On Friday, April 22, 2011, Otis Gospodnetic wrote: > Regardless of what anyone here says, you need to try it. > 3.1 should be able to read 1.4.1, yes.

Re: Localized alphabetical order

2011-04-22 Thread Bently Preece
Thanks. I get it now. I meet with our language experts again on Monday. I'll ask them about submitting localization info to the CLDR. Thanks again. -Ben On Fri, Apr 22, 2011 at 2:44 PM, Robert Muir wrote: > On Fri, Apr 22, 2011 at 3:09 PM, Bently Preece wrote: > > What if there is no stand

RE: term position question from analyzer stack for WordDelimiterFilterFactory

2011-04-22 Thread Robert Petersen
I can repeatedly demonstrate this in my dev environment, where I get entirely different results searching for AppleTV vs. appletv and I really just don't get it. I set up a specific sku in dev with AppleTV in its title to experiment with. What can I provide to help diagnose? I need to make this w

Re: Solr - Multi Term highlighting issue

2011-04-22 Thread Koji Sekiguchi
How are your hl.fl fields defined in schema.xml? Koji -- http://www.rondhuit.com/en/ (11/04/23 1:23), Ramanathapuram, Rajesh wrote: Does anybody has other suggestions? thanks& regards, Rajesh Ramana Enterprise Applications, Turner Broadcasting System, Inc. 404.878.7474 -Original Message

Re: term position question from analyzer stack for WordDelimiterFilterFactory

2011-04-22 Thread Yonik Seeley
On Fri, Apr 22, 2011 at 8:24 PM, Robert Petersen wrote: > I can repeatedly demonstrate this in my dev environment, where I get > entirely different results searching for AppleTV vs. appletv You originally said "I cannot get a match between AppleTV on the indexing side and appletv on the search si

Re: DIH Transform XML?

2011-04-22 Thread Ahmet Arslan
I have a MySQL DB that I have Solr indexing and all is well. However, one field I need to index is a text field that contains XML stored in the DB. I read up on DIH Transformers a bit and I am wondering... is there a way to have solr DIH either transform the XML data or strip the XML out of the