Re: Administrative questions

We host solr on TOMCAT. To tackle reboot. We start solr by starting Tomcat in init and using a small script (called by the tomcat rc.d script) kept at the "../SOLRHOME" which does the CHDIR and starts catalina. This works perfectly for multiple indexes hosting too. //Vikalp On Wed, Aug 13, 2008

Re: Spellcheker and Dismax both

The SpellCheckerRequestHandler is now deprecated with Solr 1.3 and it has been replaced by SpellCheckComponent. http://wiki.apache.org/solr/SpellCheckComponent On Thu, Aug 14, 2008 at 3:42 AM, anshuljohri <[EMAIL PROTECTED]> wrote: > > Hi, > > I am using dismax handler and I want to use spellche

Re: Administrative questions

On Tue, Aug 12, 2008 at 05:49:32PM -0700, Jon Drukman wrote: > 1. How do people deal with having solr start when system reboots, manage > the log output, etc. Right now I run it manually under a unix 'screen' > command with a wrapper script that takes care of restarts when it crashes. > That m

Re: concurrent optimize and update

On Tue, Aug 12, 2008 at 11:51:12AM -0400, Yonik Seeley wrote: > On Tue, Aug 12, 2008 at 11:19 AM, Jason Rennie <[EMAIL PROTECTED]> wrote: > > On Mon, Aug 11, 2008 at 6:41 PM, Yonik Seeley <[EMAIL PROTECTED]> wrote: > > > >> It's safe... the adds will block until the commit or optimize has finished.

RE: Word Gram?

Hi Brendan, What modifications have you made to ShingleFilter? Can you share them? Karl Wettin recently contributed ShingleMatrixFilter to Lucene - among other things, it can generate shingles of more than one size (check the test cases for how to do this):

Spellcheker and Dismax both

Hi, I am using dismax handler and I want to use spellchecker functionality also. Is there any way so that I could use both dismax and spellchecker request handler together. As I have configured everything related to spellcheck described in following link -- http://wiki.apache.org/solr/SpellCheck

Re: Word Gram?

aaah thanks for the vocabulary lesson: shingles == token n-grams On Aug 13, 2008, at 5:27 PM, Brendan Grainger wrote: Hi Ryan, We do basically the same thing, using a modified ShingleFilter (http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc//contrib-analyzers/org/apache/lucene/

Re: Word Gram?

This is fairly high on our to-do list. I'm inclined to index the bi-words at the same position as the first word, like synonyms. wunder On 8/13/08 2:27 PM, "Brendan Grainger" <[EMAIL PROTECTED]> wrote: > Hi Ryan, > > We do basically the same thing, using a modified ShingleFilter > (http://hudso

Re: Word Gram?

Hi Ryan, We do basically the same thing, using a modified ShingleFilter (http://hudson.zones.apache.org/hudson/job/Lucene-trunk/javadoc//contrib-analyzers/org/apache/lucene/analysis/shingle/ShingleFilter.html ). I have it set up to build 'shingles' of size 2, 3, 4, 5 which I index into separat

RE: Administrative questions

I wrote shell tasks that start, stop, and heartbeat the server and run them from cron (unix). Heartbeat means: 1) is the tomcat even running, 2) does tomcat return the Solr admin page, 3) does Solr return a search. For an indexer, 4) does solr return from a commit. Stopping the server via the tomca

Word Gram?

I'm looking for a way to get common word groups within documents. That is, what are the top two, three, ... n word groups within the index. I was messing with indexing adjacent words together (sorry about the earlier commit)... is this a reasonable approach? Any other ideas for pulling

Re: spellcheck collation

Hi Doug, On Wed, Aug 13, 2008 at 8:25 PM, Doug Steigerwald <[EMAIL PROTECTED]> wrote: > When spellchecking 'blackberri wi', the collation returned is 'blackberry > wii'. When spellchecking 'blackberr wi', the collation returned is > 'blackberrywii'. 'blackber wi' returns 'blackberrwiiwi'. It lo

Re: more multicore fun

On Aug 13, 2008, at 3:29 PM, Andrew Nagy wrote: Thanks for clarifing that Ryan - I was a bit confused too... Before 1.3 is released, you will either be able to: 1. set the dataDir from your solr.xml config I have been perusing the multicore code and found that the "default" attribute

Re: multicore /solr/update

check now. Should be fixed in trunk On Aug 13, 2008, at 3:05 PM, Doug Steigerwald wrote: I checked out the trunk about 2 hours ago. Was the last commit on the 10th supposed to fix this (r684606)? On Aug 13, 2008, at 3:00 PM, Ryan McKinley wrote: check a recent version, this issue should

RE: more multicore fun

Thanks for clarifing that Ryan - I was a bit confused too... > Before 1.3 is released, you will either be able to: > 1. set the dataDir from your solr.xml config > > I have been perusing the multicore code and found that the "default" attribute was removed. It also appears that the "dataDir

RE: more multicore fun

Doug - I had this same problem today. If you look at my post from earlier today you will see the problem. You will need to adjust the solr.data.dir value in the solrconfig.xml. Maybe this also needs to be changed in the example solrconfig.xml document? Andrew > -Original Message- > F

Re: more multicore fun

Ah, that's right. Thanks. Forgot I had to do that with our current setup in production. On Aug 13, 2008, at 3:05 PM, Ryan McKinley wrote: the dataDir is configured in solrconfig.xml With multicore it is currently a bit wonky. Currenlty, you need to configure it explicitly for each core

Re: multicore /solr/update

aaah -- I see, we need the same error logic for SolrUpdateServlet as we added for SolrServlet. I'll fix in one sec. Thanks ryan On Aug 13, 2008, at 3:05 PM, Doug Steigerwald wrote: I checked out the trunk about 2 hours ago. Was the last commit on the 10th supposed to fix this (r684606)?

Re: multicore /solr/update

I checked out the trunk about 2 hours ago. Was the last commit on the 10th supposed to fix this (r684606)? On Aug 13, 2008, at 3:00 PM, Ryan McKinley wrote: check a recent version, this issue should have been fixed in: https://issues.apache.org/jira/browse/SOLR-545 On Aug 13, 2008, at 2:22

Re: more multicore fun

the dataDir is configured in solrconfig.xml With multicore it is currently a bit wonky. Currenlty, you need to configure it explicitly for each core, but it shares the same system variables: ${solr.data.dir}, so if you use properties, you end up pointing to the same place. https://issues

Re: multicore /solr/update

check a recent version, this issue should have been fixed in: https://issues.apache.org/jira/browse/SOLR-545 On Aug 13, 2008, at 2:22 PM, Doug Steigerwald wrote: Yeah, that's the problem. Not having the core in the URL you're posting to shouldn't update any core, but it does. Doug On Aug

more multicore fun

OK. Last question for a while (hopefully), but something else with multicore seems to be wrong. $ java -jar start.jar ... INFO: [core0] Opening new SolrCore at solr/core0/, dataDir=./solr/data/ ... INFO: [core1] Opening new SolrCore at solr/core1/, dataDir=./solr/data/ ...

Re: Index size vs. number of documents

I'm surprised, as you are, by the non-linearity. Out of curiosity, what is your MaxFieldLength? By default only the first 10,000 tokens are added to a field per document. If you haven't set this higher, that could account for it. As far as I know, optimization shouldn't really affect the index siz

spellcheck collation

I've noticed a few things with the new spellcheck component that seem a little strange. Here's my document: 5 wii blackberry blackjack creative labs zen ipod video nano Some sample queries: http://localhost:8983/solr/core1/spellCheckCompRH?q=blackberri+wi&spellcheck=true&spellcheck.

Re: multicore /solr/update

Yeah, that's the problem. Not having the core in the URL you're posting to shouldn't update any core, but it does. Doug On Aug 13, 2008, at 2:10 PM, Alok K. Dhir wrote: you need to add the core to your call -- post to http://localhost:8983/solr/coreX/update On Aug 13, 2008, at 1:58 PM, Do

Re: multicore /solr/update

you need to add the core to your call -- post to http://localhost:8983/solr/coreX/update On Aug 13, 2008, at 1:58 PM, Doug Steigerwald wrote: I've got two cores (core{0|1}) both using the provided example schema (example/solr/conf/schema.xml). Posting to http://localhost:8983/solr/update ad

multicore /solr/update

I've got two cores (core{0|1}) both using the provided example schema (example/solr/conf/schema.xml). Posting to http://localhost:8983/solr/update added the example docs to the last core loaded (core1). Shouldn't this give you a 400? Doug

Re: Administrative questions

Jason Rennie wrote: On Tue, Aug 12, 2008 at 8:49 PM, Jon Drukman <[EMAIL PROTECTED]> wrote: 1. How do people deal with having solr start when system reboots, manage the log output, etc. Right now I run it manually under a unix 'screen' command with a wrapper script that takes care of restarts

Re: WordGramFilterFactory

Looks like an accidental commit. I just reverted it. -Yonik On Wed, Aug 13, 2008 at 1:39 PM, Doug Steigerwald <[EMAIL PROTECTED]> wrote: > Just checked out Solr trunk from SVN and ran 'ant dist && ant example'. > Running the example throws out errors because there is no > WordGramFilterFactory

Index size vs. number of documents

We're indexing the ocr for a large number of books. Our experimental schema is simple and id field and an ocr text field (not stored). Currently we just have two data points: 3005 documents = 723 MB index 174237 documents = 51460 MB index These indexes are not optimized. If the index size

WordGramFilterFactory

Just checked out Solr trunk from SVN and ran 'ant dist && ant example'. Running the example throws out errors because there is no WordGramFilterFactory class. We don't need it here, but is that something waiting to be committed? Doug --Snippet from schema-- positionIncrementGap="100"

Re: Best way to index without diacritics

Stripping accents doesn't quite work. The correct translation is language-dependent. In German, o-dieresis should turn into "oe", but in English, it shoulde be "o" (as in "coöperate" or "Mötley Crüe"). In Swedish, it should not be converted at all. There are other character-to-string conversions:

Indexing Only Parts of HTML Pages

I'm wondering, is there some way ("out of the box") to tell Solr that we're only interested in indexing certain parts of a page? For example, let's say I have a bunch of pages in my site that contain some common navigation elements, roughly like this: Stuff here about parts

Exception throws at creating CommonsHttpSolrServer.java in spring framework.

HI, I have created bean of CommonsHttpSolrServer by extending in class SolrHttpServer. when we run server at this time follwoing exception throws, could you help me how to create the bean id of CommonsHttpSolrServer.java. org.springframework.beans.factory.BeanCreationException: Error creatin

RE: multicore can't find the data directories

Nevermind - sorry. The data directory in my solrconfig.xml was not changed to the correct path. Now it's alive! Andrew > -Original Message- > From: Andrew Nagy [mailto:[EMAIL PROTECTED] > Sent: Wednesday, August 13, 2008 12:13 PM > To: solr-user@lucene.apache.org > Subject: multicore c

RE: Best way to index without diacritics

Hi Norberto, https://issues.apache.org/jira/browse/LUCENE-1343 :) Steve On 08/13/2008 at 12:35 AM, Norberto Meijome wrote: > On Tue, 12 Aug 2008 11:44:42 -0400 > "Steven A Rowe" <[EMAIL PROTECTED]> wrote: > > > Solr is Unicode aware. The ISOLatin1AccentFilterFactory > handles diacritics for t

multicore can't find the data directories

I am trying to setup a multicore system implementation. I just upgraded to today's snapshot and have converted my multicore.xml to solr.war and I also changed the xml to match the new schema. However, now that I have done that, Solr is not finding my data directory. With the use of multicore

Re: Administrative questions

On Tue, Aug 12, 2008 at 8:49 PM, Jon Drukman <[EMAIL PROTECTED]> wrote: > 1. How do people deal with having solr start when system reboots, manage > the log output, etc. Right now I run it manually under a unix 'screen' > command with a wrapper script that takes care of restarts when it crashes.

Re: Exception during Solr startup

On Wed, Aug 13, 2008 at 10:55 AM, Kashyap, Raghu <[EMAIL PROTECTED]> wrote: > SEVERE: java.lang.UnsupportedClassVersionError: Bad version number in > .class file This is normally a mismatch between java compiler and runtime (like using Java6 to compile and Java5 to try and run). -Yonik

RE: SOLR 1.2 Multicore configuration

Thanks Ryan, I think it would be high risk to move to solr 1.2 as our ops team have a standard 1.2 configuration. Perhaps I should ask them... Thanks, John -Original Message- From: Ryan McKinley [mailto:[EMAIL PROTECTED] Sent: 13 August 2008 16:47 To: solr-user@lucene.apache.org Subje

Re: Exception during Solr startup

Is this perhaps due to the renaming of multicore.xml to solr.xml? Erik On Aug 13, 2008, at 10:55 AM, Kashyap, Raghu wrote: Hi, Today I started seeing this exception when I started solr instance. Any ideas what might be causing this problem? INFO: xsltCacheLifetimeSeconds=5 Aug

Re: SOLR 1.2 Multicore configuration

Check: http://wiki.apache.org/solr/MultiCore If you can wait a few days, there will likely be a 1.3 release candidate out soon. On Aug 13, 2008, at 11:30 AM, McBride, John wrote: Hi, I am deploying an application across 3 geographies - and as a result will be running multiple solr instan

RE: Exception during Solr startup

I am still trying to figure out what changed. From the SCM nothing seems to have changed from yesterday. We are using the nightly build of solr from 07/22 -Raghu -Original Message- From: Grant Ingersoll [mailto:[EMAIL PROTECTED] Sent: Wednesday, August 13, 2008 10:35 AM To: solr-user@l

Re: Exception during Solr startup

Can you tell us a little bit more about your situation? What changed today? New Solr WAR? What version of Solr are you using? -Grant On Aug 13, 2008, at 10:55 AM, Kashyap, Raghu wrote: Hi, Today I started seeing this exception when I started solr instance. Any ideas what might be caus

SOLR 1.2 Multicore configuration

Hi, I am deploying an application across 3 geographies - and as a result will be running multiple solr instances on one host. I don't want to set up separate wars running on different ports as this will cause an increased number of firewall requests and require more management to track the set

RE: Solr1.3 Freeze

Thank Grant for the update. We have found that the lucene-2.4-dev libs are a bit out dated. My colleague is going to open a bug about this. Has any thought been made as to what snapshot of the lucene-2.4-dev libs will be used for solr 1.3? I also like the idea of renaming them to lucene-2.4-

Exception during Solr startup

Hi, Today I started seeing this exception when I started solr instance. Any ideas what might be causing this problem? INFO: xsltCacheLifetimeSeconds=5 Aug 13, 2008 9:20:45 AM org.apache.solr.common.SolrException log SEVERE: java.lang.UnsupportedClassVersionError: Bad version number in .

Re: Solr1.3 Freeze

We are getting there, though, and it is a high priority. :-) It will use 2.4-dev, as far as I can tell, unless the Lucene community all of a sudden decides to ship. Not to worry, however, as Hoss has explained a number of times, it is the Lucene PMC that makes releases, not Lucene Java o

RE: NOTICE: multicore.xml changed to solr.xml, format changes as well

Okay - I found the removal of the default attribute in https://svn.apache.org/viewvc/lucene/solr/trunk/src/java/org/apache/solr/core/MultiCore.java? r1=606335&r2=602003 I will update the documentation on the multicore changes. Andrew > -Original Message- > From: Chris Hostetter [mailto

Re: password protect solr URLs

You can find some starting points at http://wiki.apache.org/solr/SolrSecurity On Wed, Aug 13, 2008 at 5:20 PM, Noble Paul നോബിള്‍ नोब्ळ् < [EMAIL PROTECTED]> wrote: > Write a custom ServletFilter and apply it before the > SolrDispatchFilter for /update > Do the validation in the filter > > > On W

Re: password protect solr URLs

Write a custom ServletFilter and apply it before the SolrDispatchFilter for /update Do the validation in the filter On Wed, Aug 13, 2008 at 5:05 PM, Sunil <[EMAIL PROTECTED]> wrote: > Hi, > > I want to password protect the solr select/update/delete URLs. Any link > from where I can get some help

password protect solr URLs

Hi, I want to password protect the solr select/update/delete URLs. Any link from where I can get some help Thanks, Sunil

Re: Searching Questions

On Wed, Aug 13, 2008 at 1:56 AM, Jake Conk <[EMAIL PROTECTED]> wrote: > 4) Is there a page that will show me different querying combinations > or can someone post some more examples? You can see some examples of facet use at http://wiki.apache.org/solr/SimpleFacetParameters -- Regards, Shalin

Re: Static Fields vs Dynamic Fields

On Aug 12, 2008, at 9:46 PM, Jake Conk wrote: Is there a performance difference when using fields that are defined in my schema vs dynamic fields? No, nothing significant. Erik

Re: Searching Questions

just adding .. #1 search in a specific field: --- it is always possible using 'q.alt' --> e.g u want to search in category then your query might be like (u are searching for keyword 'comuters') http://localhost:8080/select/?q.alt=(category:(comput

Re: Searching Questions

On Tue, 12 Aug 2008 13:26:26 -0700 "Jake Conk" <[EMAIL PROTECTED]> wrote: > 1) I want to search only within a specific field, for instance > `category`. Is there a way to do this? of course. Please see http://wiki.apache.org/solr/SolrQuerySyntax (in particular, follow the link to Lucene syntax..