Re: solr + zookeeper

2008-10-08 Thread Shalin Shekhar Mangar
Hi Anoop, If you are willing to go a little cutting edge, you might want to look at https://issues.apache.org/jira/browse/SOLR-561 The patch is a little out of date. I shall post a new one soon. The patch allows for index and configuration replication (except for solrconfig.xml on which we are wo

RE: Using the more like this feature in solrj.

2008-10-08 Thread Bruce Ritchie
Erik, I just got this to work myself and the documentation was only partially helpful in figuring it out. Two main points on making this work via sor1j: #1 - Define the mlt handles in solrconfig.xml (it's not defined in the example solrconfig.xml I was using): #2 - with Solrj, access the mlt

Using the more like this feature in solrj.

2008-10-08 Thread Erik Holstad
Hi! Have been going though the documentation for the more like this/these feature but haven't found anything about how to use it in Solrj. Regards Erik

Re: solr + zookeeper

2008-10-08 Thread Aleksandar Bradic
Hi, In such setup, you would definitely want to use zookeeper in conjunction with Solr for at least two things : a) deploy management / (having cron-based pulls for a 128-node setup could be a nightmare without appropriate sync guarantees, coordination, etc) b) configuration distribution

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread Erik Hatcher
Maybe, and I haven't followed your details sorry, an issue is the lib directory for plugins. Under the solr home directory you can put a lib/ subdirectory with the data import handler JAR and it's dependencies (that aren't already in the WAR's WEB-INF/lib). To deploy Solr into Tomcat, you

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
On Oct 8, 2008, at 6:11 PM, Rajiv2 wrote: w/ faceting qtime is around +200ms. if your target time is 250, this will need some work... but lets ignore that for now... qtime for a standard query on the default search field is less than 100ms. Usually around 60ms. qtime for id: i

RE: Problem in using Unique key

2008-10-08 Thread Neville Burnell
Hi Con, In our project, we found it useful to construct a pseudo uniquekey based on the rowtype and row uniqueid, ie > -Original Message- > From: con [mailto:[EMAIL PROTECTED] > Sent: Wednesday, 8 October 2008 6:47 PM > To: solr-user@lucene.apache.org > Subject: Pr

Re: spellcheck: issues

2008-10-08 Thread Jason Rennie
On Wed, Oct 8, 2008 at 3:31 PM, Jason Rennie <[EMAIL PROTECTED]> wrote: > I just tried J-W and *yes* it seems to do a much better job! I'd certainly > vote for that becoming the default :) > Ack! I did some more testing and J-W results started to get weird (including suggesting "courses" for "c

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
w/ faceting qtime is around +200ms. qtime for a standard query on the default search field is less than 100ms. Usually around 60ms. qtime for id: is around 16ms. ryantxu wrote: > >> - >> >> 6727.0 >> - >> >> 6457.0 >> >> - >> >> 0.0 >> >> - > > So I take it, this is with faceting t

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
- 6727.0 - 6457.0 - 0.0 - So I take it, this is with faceting turned off... what are your timing results for a simple (not dismax query), perhaps: "id:" ryan

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread sunnyfr
Sorry but I can't choose for that very sorry. I will try to make it work with tomcat55, it used to work with solr 1.2 so it should be I don't know a parameter or ??? path or something that I miss for importing? thanks, Erik Hatcher wrote: > > Wouldn't life be simpler if you simply used S

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
yes I'm using 1.3 here are the contents of debug.. I'm only pasting the first explain since the whole debug section is very long. cleaning services cleaning services - +((DisjunctionMaxQuery((about_us:cleaning^0.7 | zip:cleaning^0.7 | (insurances:cleaners insurances:cleaning insurances:clean)

Practical number of Solr instances per machine

2008-10-08 Thread Phillip Farber
Hello everyone, What is the generally accepted number of solr instances it makes sense to run on a single machine given solr/lucene threading? Servers now commonly have 4 or 8 cpus. Obviously the more instances you run the bigger your JVM heap needs to be and that takes away from OS cache.

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
On Oct 8, 2008, at 4:03 PM, Rajiv2 wrote: and query times without faceting are... ? solr's built in faceting is "simple" and has its limits. 15M is higher than i've seen good faceting performance out of, particularly multivalued fields. Erik Hi, My facet fields are multi valued

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread Erik Hatcher
Wouldn't life be simpler if you simply used Solr's Jetty container configuration, at least to start with? Is Tomcat a requirement for some reason? You're struggling with things that "just work" out of the box with Solr, it seems, and I'm wondering why change around what works. E

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
>and query times without faceting are... ? >solr's built in faceting is "simple" and has its limits. 15M is >higher than i've seen good faceting performance out of, particularly >multivalued fields. > > Erik Hi, My facet fields are multi valued and w/o faceting the query time is about

Re: Need help with Solr Performance

2008-10-08 Thread Erik Hatcher
On Oct 8, 2008, at 3:30 PM, Rajiv2 wrote: what is your actual query? Are you doing faceting / highlighting / or anything else? I am doing faceting on 5 fields, no highlighting or anything else, debugging is also off. A basic query that I'm doing using dismax is 'cleaning services' over

Re: spellcheck: issues

2008-10-08 Thread Jason Rennie
On Wed, Oct 8, 2008 at 3:05 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: > chane is in the dictionary. For better or worse, Lucene skips words that > are in the dictionary when OMP is false. Ah, I see. I think we'll use OMP=true, which seems like a reasonable setting anyway. > Makes sense

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
>what is your actual query? >Are you doing faceting / highlighting / or anything else? I am doing faceting on 5 fields, no highlighting or anything else, debugging is also off. A basic query that I'm doing using dismax is 'cleaning services' over 15 million local business records. -- View thi

Re: Solr Spell check component - SolrJ

2008-10-08 Thread Grant Ingersoll
see the thread "spellcheck issues" going on right now between Jason Rennie and me. In there is a small sample piece of code that does spell checking through SolrJ. On Oct 8, 2008, at 12:52 PM, Sunil Raj wrote: Hi, I am trying to use SolrJ java client to do search and the basic search wo

Re: spellcheck: issues

2008-10-08 Thread Grant Ingersoll
On Oct 8, 2008, at 2:03 PM, Jason Rennie wrote: On Wed, Oct 8, 2008 at 1:24 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: Token: chane OMP: false Oct 8, 2008 1:19:56 PM org.apache.solr.core.SolrCore execute INFO: [spell] webapp=null path=/select params={q=description %3Achane &spellcheck

Re: Need help with Solr Performance

2008-10-08 Thread Ryan McKinley
what is your actual query? Are you doing faceting / highlighting / or anything else? On Oct 8, 2008, at 2:17 PM, Rajiv2 wrote: Hi, thanks for responding so quickly, 6-12 seconds seems really long and 15 million docs is nothing on a machine like this. Are you sure the issue is in Solr? How

Spellchecker extended results problem

2008-10-08 Thread Doug Macdonald
Hi, I'm having trouble with the spell check component of Solr 1.3. If I request multiple extended results from spell check with a json result format, the results get decoded into a single 'suggestion' array. For example, for the query http://localhost:8983/solr/spellCheckCompRH?q=beaar&sp

Re: Need help with Solr Performance

2008-10-08 Thread Walter Underwood
One other question: are you using real query logs or a set of unique queries? With real query logs, the caches will warm up after a while (tens of minutes) and performance will improve. With a set of unique queries, you are mostly measuring Solr cache misses. For us, that is about 4X slower, and w

Re: Need help with Solr Performance

2008-10-08 Thread Rajiv2
Hi, thanks for responding so quickly, >6-12 seconds seems really long and 15 million docs is nothing on a >machine like this. Are you sure the issue is in Solr? How are you >measuring the 6-12 seconds? I'm looking at the value in the Solr response. >Assuming it is Solr... >How often are

Re: spellcheck: issues

2008-10-08 Thread Jason Rennie
On Wed, Oct 8, 2008 at 1:24 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote: > Token: chane OMP: false > Oct 8, 2008 1:19:56 PM org.apache.solr.core.SolrCore execute > INFO: [spell] webapp=null path=/select > params={q=description%3Achane&spellcheck=true&spellcheck.onlyMorePopular=false&spellcheck.e

Re: Discarding undefined fields in query

2008-10-08 Thread Jérôme Etévé
On Tue, Oct 7, 2008 at 12:56 AM, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > : req.getSchema().getQueryAnalyzer(); > : > : I think it's in this analyzer that the undefined field error happens > : (because for instance the field 'foo' doesn't exists in the schema, > : and so it's impossible to fi

Re: Need help with Solr Performance

2008-10-08 Thread Grant Ingersoll
On Oct 8, 2008, at 11:56 AM, Rajiv2 wrote: Hi, I need some recommendations w/ some issues I'm having w/ solr search performance. Here is my index/hardware config: - CentOS on 8 quad core xeon processors @ 3.16 Ghz - 32 GB RAM - Tomcat and JAVA 1.6 - Solr 1.3 ~15 million documents . - Index

Re: spellcheck: issues

2008-10-08 Thread Grant Ingersoll
Hi Jason, Here's what I did: 1. Took your code and modified it to be that of [1] below 2. Set up your config, schema, etc. as per the EmbeddedSolrServer paths in the code (a Maven like dir structure w/ src/main/resources/ solr/spell containing your configuration. 3. Ran the code. My output

Re: Need help with Solr Performance

2008-10-08 Thread Mark Miller
I don't think you can search a 15 million doc index with any kind of query complexity beyond a low freq query term in under .25 seconds unless its a cached hit (in which case it still might not *quite* make it under .25 every time either I'd think). Would love to be proven wrong though You hav

Solr Spell check component - SolrJ

2008-10-08 Thread Sunil Raj
Hi, I am trying to use SolrJ java client to do search and the basic search works fine with the code given below. SolrServer server = new CommonsHttpSolrServer( " http://localhost:8983/solr";); SolrQuery query = new SolrQuery(). setQuery(searchKey).

Problems with WordDelimiterFilterFactory

2008-10-08 Thread Kraus, Ralf | pixelhouse GmbH
Hello, I am playing arount with WordDelimiterFilterFactory and run into some problems... When I search with "RedElectronicLed" I got this entries : "Red - Electronic - Led" => Perfect ! But I don´t get "RedElectronicLed" entries in my DB :-( Is there a chance I will get both ? "Red - Elect

Need help with Solr Performance

2008-10-08 Thread Rajiv2
Hi, I need some recommendations w/ some issues I'm having w/ solr search performance. Here is my index/hardware config: - CentOS on 8 quad core xeon processors @ 3.16 Ghz - 32 GB RAM - Tomcat and JAVA 1.6 - Solr 1.3 ~15 million documents . - Index size on disk is about 22 GB - 8 quad core xeon p

solr + zookeeper

2008-10-08 Thread Anoop Bhatti
Hello, I am evaluating Solr on a small cluster (~8 nodes) and will likely deploy it to a cluster of ~128 nodes. Of course, configuration management will be a big concern. Has anyone used Zookeeper for a Solr cluster? I'm looking for any documentation, tutorials, etc. on how to setup Zookeeper f

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread sunnyfr
More information : [EMAIL PROTECTED]:/usr/share/tomcat5.5/webapps/solr/WEB-INF/lib# ls README.committers.txt commons-io-1.3.1.jar lucene-snowball-2.4-dev.jar solr-lucene-highlighter-pom.xml.template apache-solr-common-1.4-dev.jar comm

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread sunnyfr
What you mean? except the name of the core which is books because in solrconfig.xml I've no part about DataImportHandler. thanks, Shalin Shekhar Mangar wrote: > > Is the DataImportHandler defined in the solrconfig.xml for the "video" > core? > > On Wed, Oct 8, 2008 at 7:26 PM, sunnyfr <[EMAIL

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread sunnyfr
sorry it's ok ... it's book but the same error. Shalin Shekhar Mangar wrote: > > Is the DataImportHandler defined in the solrconfig.xml for the "video" > core? > > On Wed, Oct 8, 2008 at 7:26 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > >> >> Hi, >> >> I've a wierd problem, my solr seems running.

Re: Solr 1.3 try to fire delta-import.

2008-10-08 Thread Shalin Shekhar Mangar
Is the DataImportHandler defined in the solrconfig.xml for the "video" core? On Wed, Oct 8, 2008 at 7:26 PM, sunnyfr <[EMAIL PROTECTED]> wrote: > > Hi, > > I've a wierd problem, my solr seems running. > When i go to : http://solr-test.adm.bookclub.com:8180/solr/books/admin/ > > I've a proper page

Re: spellcheck: issues

2008-10-08 Thread Jason Rennie
Hi Grant, Here are solr config files (attached) and java code (included below) to recreate the test case. Jason List> terms = new ArrayList>(); terms.add(new Pair("chanel", 834)); terms.add(new Pair("chant", 10)); terms.add(new Pair("chang", 8)); terms.add

Solr 1.3 try to fire delta-import.

2008-10-08 Thread sunnyfr
Hi, I've a wierd problem, my solr seems running. When i go to : http://solr-test.adm.bookclub.com:8180/solr/books/admin/ I've a proper page : Solr Admin (videos) solr-test.adm.bookclub.com:8180 cwd=/data/solr SolrHome=/data/solr/books/ Even http://solr-test.adm.dailymotion.com:8180/solr/video/

Re: solr 1.2 to solr 1.3, I've some error

2008-10-08 Thread sunnyfr
it's ok ..sorry it was my fault. forgot a jar. Grant Ingersoll-6 wrote: > > > On Oct 8, 2008, at 4:04 AM, sunnyfr wrote: > >> >> Hello, >> >> I've updated solr 1.2 to solr 1.3, so I've generated the new .war >> and add it >> to tomcat like I used to do and it used to work. > > Define "gene

Re: solr 1.2 to solr 1.3, I've some error

2008-10-08 Thread Grant Ingersoll
On Oct 8, 2008, at 4:04 AM, sunnyfr wrote: Hello, I've updated solr 1.2 to solr 1.3, so I've generated the new .war and add it to tomcat like I used to do and it used to work. Define "generated the new .war"? Is there something wrong with the WAR that comes w/ Solr? What steps did you

RE: upgrading solr

2008-10-08 Thread dudes dudes
thanks for your quick response.. yes it was under the right path... However, the problem was fixed by changing the ownership under index folder :) thanks again ak > From: [EMAIL PROTECTED] > To: solr-user@lucene.apache.org > Subject: Re: upgrading solr >

Re: upgrading solr

2008-10-08 Thread Alok Dhir
Check again that you put the index directory in the right place. If you're using the example config which comes with the release package, it needs to be in BASE/example/solr/data/index. On Oct 8, 2008, at 8:18 AM, dudes dudes wrote: hi again, I guess I'm doing something wrong, I have

RE: upgrading solr

2008-10-08 Thread dudes dudes
hi again, I guess I'm doing something wrong, I have downloaded the latest stable version... untar it ,,, then I have copied /index directory from the old version to the newer version version under data/ schema are the same... the old default one I have start it via java -jar start... ( s

RE: upgrading solr

2008-10-08 Thread dudes dudes
thanks, I will give it a try ,,, any probs will let you know :) ak > From: [EMAIL PROTECTED] > To: solr-user@lucene.apache.org > Subject: Re: upgrading solr > Date: Wed, 8 Oct 2008 07:29:33 -0400 > > I've upgraded solr several times from nightly to nightl

Re: upgrading solr

2008-10-08 Thread Alok Dhir
I've upgraded solr several times from nightly to nightly and then to the 1.3 release without reindexing, with no apparent ill effects. On Oct 8, 2008, at 6:12 AM, dudes dudes wrote: Hello all, I would like to upgrade to the latest Solr from its nightly version. MY understanding is to re-in

Re: Problem in using Unique key

2008-10-08 Thread con
Thanks for your update. But in that case, while doing a full-import I am getting the following error: org.apache.solr.common.SolrException: QueryElevationComponent requires the schema to have a uniqueKeyField at org.apache.solr.handler.component.QueryElevationComponent.inform(QueryElevat

upgrading solr

2008-10-08 Thread dudes dudes
Hello all, I would like to upgrade to the latest Solr from its nightly version. MY understanding is to re-index all the data after upgrading .. Is there a way that doesn't require re-indexing all the data ? for example a solr script or any third party tools ? or do I have to re-index all

solr 1.2 to solr 1.3, manage multilanguage error?

2008-10-08 Thread sunnyfr
Hi, I've updated recently solr to the last version, but I've an error : How can I manage language with this new version : My bug: Oct 8 11:32:13 solr-test jsvc.exec[29876]: Oct 8, 2008 11:32:13 AM org.apache.solr.request.XSLTResponseWriter init INFO: xsltCacheLifetimeSeconds=5 Oct 8 11:32:13 s

Re: Problem in using Unique key

2008-10-08 Thread Noble Paul നോബിള്‍ नोब्ळ्
userID I do not think there is a required attribute on uniquekey By default uniquekey is required . If you do not want to make it required remove the tag itself means no uniqueKey --Noble On Wed, Oct 8, 2008 at 1:17 PM, con <[EMAIL PROTECTED]> wrote: > > hi guys > I am indexing values from an orac

Re: still my problem with autoCommit which fire but never ending!!! no idea

2008-10-08 Thread Erik Hatcher
On Oct 8, 2008, at 3:43 AM, sunnyfr wrote: Just a question, I've updated solr1.2 to solr 1.3 but didn't change solrconfig.xml and schema.xml is that a real problem ? Personally, I'd strongly recommend (to the point of mandating) that whenever going from one version of Solr to another that th

solr 1.2 to solr 1.3, I've some error

2008-10-08 Thread sunnyfr
Hello, I've updated solr 1.2 to solr 1.3, so I've generated the new .war and add it to tomcat like I used to do and it used to work. For information I didn't change conf files : solrconfig, data-config, schema and scripts. I just changed the name multicore.xml to solr.xml And now I've this error.

Problem in using Unique key

2008-10-08 Thread con
hi guys I am indexing values from an oracle db and them performing searching. Since I have to search multiple tables, that is no way related to each other, I have changed the uniquekey constraint in schema.xml to false. userID But when I do indexing, the values from the table that does not have

Re: Tomcat session

2008-10-08 Thread JC R
could it become a performance issue at some point? JC R wrote: > > Hello, > Does somebody have an idea why a new servlet session is created for each > request to solr? > thanks > -- View this message in context: http://www.nabble.com/Tomcat-session-tp19861244p19873952.html Sent from the Sol

Re: still my problem with autoCommit which fire but never ending!!! no idea

2008-10-08 Thread sunnyfr
Hi, Just a question, I've updated solr1.2 to solr 1.3 but didn't change solrconfig.xml and schema.xml is that a real problem ? thanks, Grant Ingersoll-6 wrote: > > Hi Sunny, > > Pretty hard to say what's going on here, other than you don't have the > right Lucene libraries in your install