Re: Solr like for autocomplete field?

I implemented the edge ngrams solution and it's an awesome one compared to any other that I could think of because I can index more than just text (other metadata) that can be used to *rank* the autocomplete results eventually getting to rank by the probability of selection which is, after all, wha

Re: xpath processing

The XPathEP has the option to run a real XSL script at some point in its processing chain. I guess you could make an XSL that pulls your fields out into a simpler XML in the /a/b/c format that the XPath parser supports. On Tue, Nov 2, 2010 at 5:37 PM, wrote: > > > http://www.loc.gov/mods/v3";

Re: Solr like for autocomplete field?

And the SpellingComponent. There's nothing to help you with phrases. On Tue, Nov 2, 2010 at 11:21 AM, Erick Erickson wrote: > Also, you might want to consider TermsComponent, see: > > http://wiki.apache.org/solr/TermsComponent > > Also, note that there's an autosuggestcomponent, that's recently

Re: Possible memory leaks with frequent replication

Isn't that what this code does? onDeckSearchers++; if (onDeckSearchers < 1) { // should never happen... just a sanity check log.error(logid+"ERROR!!! onDeckSearchers is " + onDeckSearchers); onDeckSearchers=1; // reset } else if (onDeckSearchers > maxWarm

Re: How to use polish stemmer - Stempel - in schema.xml?

Here's the problem: Solr is a little dumb about these Filter classes, and so you have to make a Factory object for the Stempel Filter. There are a lot of other FilterFactory classes. You would have to just copy one and change the names to Stempel and it might actually work. This will take some So

Re: Disk usage per-field

The Lucene CheckIndex program opens an index and reads many types of data from it. It's easy to start with it and change that to count up the space used by terms and store data for field X. On Tue, Nov 2, 2010 at 5:51 AM, Muneeb Ali wrote: > > Hi, > > I am currently benchmarking solr index with d

Re: how to get TermVectorComponent using xml , vs. SOLR-949

TVC is in Solr 1.4 onwards. It is configured in example/solr/conf/solrconfig.xml as 'tvrh'. This is not a solr/url thing, so you have to say solr/select?q=word&'qt=tvrh' and look at the bottom of the xml. On Tue, Nov 2, 2010 at 5:34 AM, Will Milspec wrote: > Hi all, > > This seems a basic questio

Re: using HebMorph

I don't know the paths in the Solr package for Ubuntu. In the Solr apache release, you go to the example/ directory. The example/solr directory needs a new lib directory, and you copy the jars to there. Then run 'java -jar start.jar' still in the example/ directory. Solr should start. Now, you nee

Re: Ensuring stable timestamp ordering

memory's cheap! (I know processing it is not' though ) Dennis Gearon Signature Warning It is always a good idea to learn from your own mistakes. It is usually a better idea to learn from others’ mistakes, so you do not have to make them yourself. from 'http://blogs.techrepub

Re: xpath processing

http://www.loc.gov/mods/v3"; xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="http://www.loc.gov/mods/v3 http://www.loc.gov/standards/mods/v3/mods-3-0.xsd";> Any place I hang my hat is home

Re: Query question

My impression was that city:Chicago^10 +Romantic +View would do what you want (with the standard lucene query parser and default operator OR), and I'm not sure about this, but I have a feeling that the version with "Boolean" operators AND/OR and parens might actually net out to the same thing

Re: Influencing scores on values in multiValue fields

Thanks Mike for your suggestion. It did take me down the correct route. I basically created another multiValue field of type 'string' and boosted that. To get the partial matches to avoid the length normalisation I had the 'text' type multiValue field to omitNorms. The results look as per expected

Re: Stored or indexed?

IMO, the very, very best way to increase your grasp of all things Solr is to try to answer questions on this list. Folks are pretty gentle about correcting mistaken posts. And I certainly remember any advice I've given that's been corrected . Besides, if you try to answer the things you *do* u

Re: Querying Solr using dismax, requested field not showing up in debug score boosts

Ok, here is the query cleaned up a bit: solr/select/?q=mattress& q.op=AND& qt=dismax&fl=name%2Cdescription%2Cgroup_id%2Clowest_price%2Cnum_child_products%2Craw_category_string%2Ccategory_id%2Cparent_category_id%2Cstr_brand%2Cgrandparent_category_id%2Cgrandparent_category_name%2Cparent_category_na

Re: Query question

> Erick, that query would return all restaurants in Chicago, > whether they > matched Romantic View or not. Although the scores should > sort relevant > results to the top, the results would still contain a lot > of things I wasn't > interested in. How about this one? +(city:Chicago^1000 OR (*:*

RE: Stored or indexed?

Thanks for the great info! I appreciate everybody's help in getting started with Solr, hopefully I'll be able to get my stuff working and move on to more difficult questions. :) -Original Message- From: Elizabeth L. Murnane [mailto:emurn...@architexa.com] Sent: Friday, October 29, 2010 1

Updating last_modified field when using DIH

Hello everyone! I would like to ask you a question about DIH and delta import. I am trying to sync Solr with a PostgreSQL database and I have a field "ent_lastModified" of type "timestamp without timezone". Here is my xml file: Full-import works fine, but whe

Re: Querying Solr using dismax, requested field not showing up in debug score boosts

First, you should show us the query, as well as the debug output, it often helps to have a second set of eyes... Where are you specifying the qf? Under any circumstance it would be helpful to see the definition of the request handler you're using. Because as it stands, the best I can say is that

Re: Solr like for autocomplete field?

Also, you might want to consider TermsComponent, see: http://wiki.apache.org/solr/TermsComponent Also, note that there's an autosuggestcomponent, that's recently been committed. Best Erick On Tue, Nov 2, 2010 at 1:56 PM, PeterKerk wrote: > > I have a city field. Now when a user starts typing

Re: Query question

I... Need... more... coffee. On Tue, Nov 2, 2010 at 11:31 AM, kenf_nc wrote: > > Jonathan, Dismax is something I've been meaning to look into, and bq does > seem to fit the bill, although I'm worried about this line in the wiki > :TODO: That latter part is deprecated behavior but still wor

Querying Solr using dismax, requested field not showing up in debug score boosts

I'm storing a set of products in solr as ducuments. I'm separating out the name, description, keywords, and product category name into separate fields so that I can boost them independently using the dismax handler. All the fields are stored as "text" in the same way. I'm passing these four field

Re: Solr like for autocomplete field?

We used the filters talked about at Lucid Imagination for our site, it seems to work pretty well: http://www.lucidimagination.com/blog/2009/09/08/auto-suggest-from-popular-queries-using-edgengrams/ Your mileage might vary, but its a pretty good place to start. Matt On 11/2/2010 1:56 PM, Peter

Solr like for autocomplete field?

I have a city field. Now when a user starts typing in a city textbox I want to return found matches (like Google). So for example, user types "new", and I will return "new york", "new hampshire" etc. my schema.xml my current url: http://localhost:8983/solr/db/select/?indent=on&facet=true&q=*

Re: Possible memory leaks with frequent replication

It's definitely a known 'issue' that you can't replicate (or do any other kind of index change, including a commit) at a faster frequency than your warming queries take to complete, or you'll wind up with something like you've seen. It's in some documentation somewhere I saw, for sure. The ad

Re: Possible memory leaks with frequent replication

On Tue, Nov 2, 2010 at 12:32 PM, Simon Wistow wrote: > On Mon, Nov 01, 2010 at 05:42:51PM -0700, Lance Norskog said: >> You should query against the indexer. I'm impressed that you got 5s >> replication to work reliably. > > That's our current solution - I was just wondering if there was anything

Re: Possible memory leaks with frequent replication

On Mon, Nov 01, 2010 at 05:42:51PM -0700, Lance Norskog said: > You should query against the indexer. I'm impressed that you got 5s > replication to work reliably. That's our current solution - I was just wondering if there was anything I was missing. Thanks!

IndexableBinaryStringTools (was FieldCache)

Hi, > > [...] I tried to use IndexableBinaryStringTools to re-encode my 11 byte > > array. The size was increased to 7 characters (= 14 bytes) > > which is still a gain of more than 50 percent compared to the UTF8 > > encoding. BTW: I found no sample how to use the > > IndexableBinaryStringTools c

Re: Highlighting and maxBooleanClauses limit

On Tue, Nov 2, 2010 at 11:26 AM, Koji Sekiguchi wrote: > (10/11/02 23:14), Ken Stanley wrote: > >> I've noticed in the stack trace that this exception occurs when trying to >> build the query for the highlighting; I've confirmed this by copying the >> params and changing hl=true to hl=false. Unfo

Re: Highlighting and maxBooleanClauses limit

Hmm, i'm not sure it's the highlighter alone. Depending on the query it can also get triggered by the spellcheck component. See below what happens with a maxBoolean = 16. HTTP ERROR: 500 maxClauseCount is set to 16 org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to

Re: Query question

Don't worry about that line. It just means that one particular kind of 'default' behavior in bq shouldn't be relied upon, if you don't entirely understand that behavior they're saying is deprecated (as I don't either!) anyway, don't worry about it, just supply an explicit boost in your bq. b

Re: Query question

Jonathan, Dismax is something I've been meaning to look into, and bq does seem to fit the bill, although I'm worried about this line in the wiki :TODO: That latter part is deprecated behavior but still works. It can be problematic so avoid it. It still seems to be the closest to what I want howe

Re: Phrase Query Problem?

Indeed something doesn't seem right about that, quotes are for phrases, you are right, and I get confused even thinking about what happens when you try to "escape" spaces like that. I think there's something odd going on with your URI-escaping in general. Here's what the string should actually

Re: Highlighting and maxBooleanClauses limit

(10/11/02 23:14), Ken Stanley wrote: I've noticed in the stack trace that this exception occurs when trying to build the query for the highlighting; I've confirmed this by copying the params and changing hl=true to hl=false. Unfortunately, when using debugQuery=on, I do not see any details on wha

Re: Dynamically create new core

To create the core, the folder with the confs must already exist and has to be placed in the proper place (inside the solr home). Once you run the create core action, this core will we added to solr.xml and dinamically loaded. -- View this message in context: http://lucene.472066.n3.nabble.com/D

Re: Query question

Do you want something like (Romantic AND View) OR city:Chicago^10? Best Erick On Tue, Nov 2, 2010 at 10:45 AM, kenf_nc wrote: > > I can't seem to find the right formula for this. I have a need to build a > query where one of the fields should boost the score, but not affect the > query if there

Re: Phrase Query Problem?

On 11/2/2010 9:21 AM, Ken Stanley wrote: On Tue, Nov 2, 2010 at 8:19 AM, Erick Ericksonwrote: That's not the response I get when I try your query, so I suspect something's not quite right with your test... But you could also try putting parentheses around the words, like mykeywords:(Compliance

Re: Query question

I think you'll find the dismax request handler helpful in general, it supports more flexible query wrangling like that. With the dismax request handler, I think the bq (boost query) parameter will do what you need, eg: bq=city:Chicago^5.0 The ^5.0 is how much boost you want, you can play aro

Query question

I can't seem to find the right formula for this. I have a need to build a query where one of the fields should boost the score, but not affect the query if there isn't a match. For example, if I have documents with restaurants, name, address, cuisine, description, etc. I want to search on, say,

Re: Slave replication with custom dataDir

This is a log dump, please be aware that this only appears in my log if I have the following enabled in config. /var/lib/solr/data ... snip ... http://10.1.2.196:8080/solr/replication 00:00:20 Log ouput 03/11/2010 1:23:47 AM org.apache.solr.servlet.SolrDispatchFilter init SEVERE: Could

Slave replication with custom dataDir

Hey guys, I have 2 instances of Solr running, one as a master, one as a slave. Both have /var/lib/solr/data The master works fine, the slave dies with a huge set of stack traces. The Solr wiki says that replication must match the if it's custom, but how do I actually set that?

Highlighting and maxBooleanClauses limit

By default, the solrconfig.xml has maxBooleanClauses set to 1024, which in my opinion should be more than enough clauses in general. Recently, we have been noticing errors in our Catalina log: SEVERE: org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 2048. As a temporar

Re: How to use polish stemmer - Stempel - in schema.xml?

Sorry, I am not Java programmer at all. I would appreciate more verbose (or step by step) help. 2010/11/2 Bernd Fehling : > > So you call org.getopt.solr.analysis.StempelTokenFilterFactory. > In this case I would assume a file StempelTokenFilterFactory.class > in your directory org/getopt/solr/ana

Re: How to use polish stemmer - Stempel - in schema.xml?

So you call org.getopt.solr.analysis.StempelTokenFilterFactory. In this case I would assume a file StempelTokenFilterFactory.class in your directory org/getopt/solr/analysis/. And a class which extends the BaseTokenFilterFactory rigth? ... public class StempelTokenFilterFactory extends BaseTokenF

Re: Phrase Query Problem?

On Tue, Nov 2, 2010 at 8:19 AM, Erick Erickson wrote: > That's not the response I get when I try your query, so I suspect > something's not quite right with your test... > > But you could also try putting parentheses around the words, like > mykeywords:(Compliance+With+Conduct+Standards) > > Best

Re: How to use polish stemmer - Stempel - in schema.xml?

This is what stempel-1.0.jar consist of after jar -xf: jgod...@ubuntu:~/apache-solr-1.4.1/ifaq/lib$ ls -R org/ org/: egothor getopt org/egothor: stemmer org/egothor/stemmer: Cell.class Diff.classGener.class MultiTrie2.class Optimizer2.class Reduce.classRow.classTestAll.cla

Re: Problem with phrase matches in Solr

I will. Thanks Darren -Moazzam On Mon, Nov 1, 2010 at 1:15 PM, wrote: > Take a look at term proximity and phrase query. > > http://wiki.apache.org/solr/SolrRelevancyCookbook > >> Hey guys, >> >> I have a solr index where i store information about experts from >> various fields. The thing is whe

Re: How to use polish stemmer - Stempel - in schema.xml?

Hi Jakub, if you unzip your stempel-1.0.jar do you have the required directory structure and file in there? org/getopt/stempel/lucene/StempelFilter.class Regards, Bernd Am 02.11.2010 13:54, schrieb Jakub Godawa: > Erick I've put the jar files like that before. I also added the > directive and pu

Re: How to use polish stemmer - Stempel - in schema.xml?

Erick I've put the jar files like that before. I also added the directive and put the file in instanceDir/lib What is still a problem is that even the files are loaded: 2010-11-02 13:20:48 org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/home/jgodawa/apache-solr-1.4.1

Re: Disk usage per-field

Hi, I am currently benchmarking solr index with different fields to see the impact on its size/ search speed etc. A feature to find the disk usage per field of index would be really handy and save me alot of time. Do we have any updates on this? Has anyone tried writing custom code for it ? -

how to get TermVectorComponent using xml , vs. SOLR-949

Hi all, This seems a basic question: what's the best way to get TermVectorComponents. from the Solr XmL response? SolrJ does not include TermVectorComponents in its api; the SOLR-949 patch adds this ability, but after 2 years it's still not in the mainline. (And doesn't patch cleanly to the curre

Re: How to use polish stemmer - Stempel - in schema.xml?

The polish stemmer jar file needs to be findable by Solr, if you copy it to /lib and restart solr you should be set. Alternatively, you can add another directive to the solrconfig.xml file (there are several examples in that file already). I'm a little confused about not being able to find Token

Re: Phrase Query Problem?

That's not the response I get when I try your query, so I suspect something's not quite right with your test... But you could also try putting parentheses around the words, like mykeywords:(Compliance+With+Conduct+Standards) Best Erick On Tue, Nov 2, 2010 at 5:25 AM, Tod wrote: > On 11/1/2010

Re: How to use polish stemmer - Stempel - in schema.xml?

Thank you Bernd! I couldn't make it run though. Here is my problem: 1. There is a file ~/apache-solr-1.4.1/ifaq/lib/stempel-1.0.jar 2. In ~/apache-solr-1.4.1/ifaq/solr/conf/solrconfig.xml there is a directive: 3. In ~/apache-solr-1.4.1/ifaq/solr/conf/schema.xml there is fieldType: (...)

RE: Solr MySQL Adding new column to table

Your uniqueKey field is defined as id (in schema.xml) and your query doesn't return an id field. Ephraim Ofir -Original Message- From: nitin.vanaku...@gmail.com [mailto:nitin.vanaku...@gmail.com] Sent: Tuesday, November 02, 2010 12:10 PM To: solr-user@lucene.apache.org Subject: RE: Solr

RE: Solr MySQL Adding new column to table

ok. i have one more issue. i am getting following exception can you please explore on it INFO: Creating a connection for entity person with URL: jdbc:mysql://localhost:3306/example Nov 2, 2010 3:34:11 PM org.apache.solr.handler.dataimport.JdbcDataSource$1 call INFO: Time taken for getConnection(

Dynamically create new core

Hi, I have a requirement of dynamically creating new cores(master). Each core should have a replicated slave core. I am working with Java and using SolrJ as my solr client. I came across CoreAdminRequest class and looks like the way to go. CoreAdminRequest.createCore("NewCore1", "NewCore1", solr

RE: Solr MySQL Adding new column to table

Not if you use 'SELECT * FROM person' Ephraim Ofir -Original Message- From: nitin.vanaku...@gmail.com [mailto:nitin.vanaku...@gmail.com] Sent: Tuesday, November 02, 2010 11:19 AM To: solr-user@lucene.apache.org Subject: RE: Solr MySQL Adding new column to table Hi Sivaprasad, first of

RE: Solr MySQL Adding new column to table

Hi Sivaprasad, first of all thanks for your kind response. i gone through that link, if i use the dynamicField concept,still i need to alter the query in data-config.xml right! thanks Nitin -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-MySQL-Adding-new-column-to-t

Re: Phrase Query Problem?

On 11/1/2010 11:14 PM, Ken Stanley wrote: On Mon, Nov 1, 2010 at 10:26 PM, Tod wrote: I have a number of fields I need to do an exact match on. I've defined them as 'string' in my schema.xml. I've noticed that I get back query results that don't have all of the words I'm using to search with

RE: Solr MySQL Adding new column to table

You have to change the old configuration for the newly added field.Or you can use dynamic fields concept. Go through the link http://wiki.apache.org/solr/SchemaXml -Original Message- From: "nitin.vanaku...@gmail.com [via Lucene]" Sent: Tuesday, November 2, 2010 4:50am To: "sivapras

Solr MySQL Adding new column to table

Hello Techies, I am new to Solr, i am using it with mysql. Suppose i have table called person in mysql with two columns name, age and i have configured mysql in solr. now i have added a new column to person table called phoneNumber, is it possible for solr to recognize new column dynamically ? i

using HebMorph

Hi I'm trying to use HebMorph, a new Hebrew analyzer. http://github.com/itaifrenkel/HebMorph/tree/master/java/ The instructions says: 1. Download the code from here . 2. Use the hebmorph ant script