Re: Overlapping onDeckSearchers=2

2010-05-04 Thread revas
Thanks for the repsonse .What happens in this scenario? Does the commit happen in this case or does the search server hang or just throws an error without committing Regards Sujatha On Mon, May 3, 2010 at 11:41 PM, Chris Hostetter wrote: > : When i run 2 -3 commits parallely to diff instances

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Hello, But I see that the libraries are being loaded : INFO: Adding specified lib dirs to ClassLoader May 4, 2010 12:49:59 PM org.apache.solr.core.SolrResourceLoader replaceClassLoader INFO: Adding 'file:/C:/apache-solr-1.4.0/contrib/extraction/lib/asm-3.1.jar' to classloader May 4, 2010

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Yes, Grant. You are right. Copying the tika libraries to solr webapp, solved the issue and the content extraction works fine now. Thanks, Sandhya -Original Message- From: Sandhya Agarwal [mailto:sagar...@opentext.com] Sent: Tuesday, May 04, 2010 12:58 PM To: solr-user@lucene.apache.org

Re: Score cutoff

2010-05-04 Thread Michael Kuhlmann
Am 03.05.2010 23:32, schrieb Satish Kumar: > Hi, > > Can someone give clues on how to implement this feature? This is a very > important requirement for us, so any help is greatly appreciated. > Hi, I just implemented exactly this feature. You need to patch Solr to make this work. We at Zaland

Re: Commit takes 1 to 2 minutes, CPU usage affects other apps

2010-05-04 Thread Markus Fischer
Hi, On 04.05.2010 03:24, Mark Miller wrote: On 5/3/10 9:06 AM, Markus Fischer wrote: we recently began having trouble with our Solr 1.4 instance. We've about 850k documents in the index which is about 1.2GB in size; the JVM which runs tomcat/solr (no other apps are deployed) has been given 2GB.

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Marc Ghorayeb
Sandhya, How did you proceed?I did this:- jar -xf solr.war.- i then added all of the libs i had into the web-inf/lib folder- i then recreated the jar with jar -cvf solr.war *- replaced the war files- deleted the libs in the shared lib folder- started tomcat i'm now getting an error saying this:

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
I think this is most likely because tika-core-0.7.jar, no longer has the tika-config.xml. Die, to which we have the default tika config being loaded. This can be seen in ExtractingRequestHandler.inform() method. Hence, the parsers list is empty. I am still investigating. Thanks, Sandhya -O

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Praveen Agrawal
May be as Sandhya indicated, it was loading libs earlier, so it might be trying to load from contrib when you have deleted from there, but somehow not been 'seen' by Solr or something. May be to keep them there, as well put them in solr/lib in tomcat webapps.. I'm yet to try though.. On Tue, Ma

Re: Commit takes 1 to 2 minutes, CPU usage affects other apps

2010-05-04 Thread Peter Sturge
It might be worth checking the VMWare environment - if you're using the VMWare scsi vmdk and it's shared across multiple VMs and there's a lot of disk contention (i.e. multiple VMs are all busy reading/writing to/from the same disk channel), this can really slow down I/O operations. On Tue, May 4

Re: Overlapping onDeckSearchers=2

2010-05-04 Thread Erik Hatcher
The commit happens (twice!) causing potentially double the RAM to be used for warming two index searchers, one of which will be thrown away right after warming. It's best to avoid overlapping these warming searchers. Using Solr's autocommit capability is the easiest way to manage the sit

full-import cycle, period ?!

2010-05-04 Thread stockii
Hello. how often do you perform an full-import ? my full-import with DIH runs every night an every two hours runs a delta-import. is it really necessary to run an full import every night ? we have an shopsystem and i think that is necessary, to have always synchron data. we discuss here abo

Re: SpellChecking

2010-05-04 Thread Jan Kammer
Hi, thanks, exactly that i forgot. Now it works fine. :-) Am 03.05.2010 16:50, schrieb Michael Kuhlmann: Am 03.05.2010 16:43, schrieb Jan Kammer: Hi, It worked fine with a normal field. There must something wrong with copyfield, or why does dataimporthandler add/update no more documents?

Re: Score cutoff

2010-05-04 Thread dc tech
Michael, The cutoff filter would be very useful for us as well. We want to use it for more like this feature where only the top n similar docs tend to be reallt similar. On 5/4/10, Michael Kuhlmann wrote: > Am 03.05.2010 23:32, schrieb Satish Kumar: >> Hi, >> >> Can someone give clues on how to

Re: Commit takes 1 to 2 minutes, CPU usage affects other apps

2010-05-04 Thread Markus Fischer
On 04.05.2010 11:01, Peter Sturge wrote: It might be worth checking the VMWare environment - if you're using the VMWare scsi vmdk and it's shared across multiple VMs and there's a lot of disk contention (i.e. multiple VMs are all busy reading/writing to/from the same disk channel), this can reall

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Grant Ingersoll
Yes, it is loading the libraries, but they are in a different classloader that apparently the new way Tika loads doesn't have access to. -Grant On May 4, 2010, at 3:28 AM, Sandhya Agarwal wrote: > Hello, > > > > But I see that the libraries are being loaded : > > > > INFO: Adding specifie

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Ok. In tika 0.4 and 0.5, I see that this is how the tika config is loaded : public static TikaConfig getDefaultConfig() { InputStream stream; try { stream = TikaConfig.class.getResourceAsStream("/org/apache/tika/tika-config.xml"); return new TikaConfig(stream);

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Marc Ghorayeb
Hey, I got it to work. I just redid my steps, i had forgotten several libraries that were imported through the xml. PDF extraction seems to work once again, i have yet to find one that raises an exception! Thanks for the investigation, at least we now have a fix :) Marc

Need help in filtering records based on radius value in solr

2010-05-04 Thread KshamaPai
Hi, I am using solr with Lucene spatial 2.9.1 as per http://www.ibm.com/developerworks/java/library/j-spatial/ I want to write a query, that will retrieve records within a given radius using hsin function, and using cartesian tiers as filters. So i wrote query like this http://localhost:8983

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Praveen Agrawal
I seems to have mixed results: Here is what i did: copied new Tika/poi/jempbox/pdfbox/fontbox/log4j jars etc in contrib/extraction/lib (of-course removed old ones),. as well as in web-inf/lib of solr web app in tomcat. Now it extracts contents from some pdf, but either no content from others, or

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Praveen, Along with the tika core and parser jars, did you run "mvn dependency:copy-dependencies", to generate all the dependencies too. Thanks, Sandhya -Original Message- From: Praveen Agrawal [mailto:pkal...@gmail.com] Sent: Tuesday, May 04, 2010 4:52 PM To: solr-user@lucene.apache.o

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Praveen Agrawal
Yes Sandhya, i copied new poi/jempbox/pdfbox/fontbox etc jars too. I believe this is what you were asking. Thanks. On Tue, May 4, 2010 at 5:01 PM, Sandhya Agarwal wrote: > Praveen, > > Along with the tika core and parser jars, did you run "mvn > dependency:copy-dependencies", to generate all the

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Ok. So, I am assuming you copied all the dependencies from tika-app\target\dependency ? I tried with a number of files and don't see this issue yet. Thanks, Sandhya -Original Message- From: Praveen Agrawal [mailto:pkal...@gmail.com] Sent: Tuesday, May 04, 2010 5:06 PM To: solr-user@luc

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Praveen Agrawal
This email contained a .zip file attachment. Raytheon does not allow email attachments that are considered likely to contain malicious code. For your protection this attachment has been removed. If this email is from an unknown source, please simply delete this email. If this email was expected

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Both the files work for me, Praveen. Thanks, Sandhya From: Praveen Agrawal [mailto:pkal...@gmail.com] Sent: Tuesday, May 04, 2010 5:22 PM To: solr-user@lucene.apache.org Subject: Re: Problem with pdf, upgrading Cell another one here.. On Tue, May 4, 2010 at 5:20 PM, Praveen Agrawal mailto:pkal.

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Praveen Agrawal
This email contained a .zip file attachment. Raytheon does not allow email attachments that are considered likely to contain malicious code. For your protection this attachment has been removed. If this email is from an unknown source, please simply delete this email. If this email was expected

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Marc Ghorayeb
Praveen, Did you try the technique I wrote a little earlier?Take your solr.war, put it in a directory of its own. Execute "jar -xf solr.war", that should extract its content. Next, copy all of your libraries inside the WEB-INF/lib folder. This means all the extraction/lib files, and the lib fil

RE: Commit takes 1 to 2 minutes, CPU usage affects other apps

2010-05-04 Thread cbennett
Hi, This could also be caused by performing an optimize after the commit, or it could be caused by auto warming the caches, or a combination of both. If you are using the Data Import Handler the default for a delta import is commit and optimize, which caused us a similar problem except we were op

Short DismaxRequestHandler Question

2010-05-04 Thread MitchK
Hello community, I need a minimum should match only on some fields, not on all. Let me give you an example: title: "Breaking News: New information about Solr 1.5" category: development tag: Solr News If I am searching for "Solr development", I want to return this doc, although I defined a minim

Re: Custom SearchComponent to reset facet value counts after collapse

2010-05-04 Thread MitchK
When is the returned facet-info the expected info for your multiValued fields? Before or after your collapse? It could be possible, that you need to facet only on your multiValued fields before you are collapsing to retrive the right values. If this is the case, you need to integrate the before-co

Re: cores and SWAP

2010-05-04 Thread Tim Heckman
If it helps, I am running: solr 1.4.0 tomcat 6.0.26 java version "1.6.0_20" Java(TM) SE Runtime Environment (build 1.6.0_20-b02) Java HotSpot(TM) 64-Bit Server VM (build 16.3-b01, mixed mode) Red Hat Enterprise Linux Server release 5.4 (Tikanga) thanks, Tim On Mon, May 3, 2010 at 4:47 PM, Ti

Spatial Solr: problem with multiValued PointType

2010-05-04 Thread pointbreak+solr
I want to link documents to multiple spatial points, and filter documents based on a bounding box. I was expecting that the solr.PointType would help me with that, but run into a problem. When I create a filter, it seems that Solr matches the latitude and longitude of the PointType separately. Coul

Re: Short DismaxRequestHandler Question

2010-05-04 Thread Papiya Misra
I think you could combine the minimum set of fields into one field at the time of indexing, for example, you could concatenate 'category' and 'tag' at the time of querying (if you are using a database). On 05/04/2010 09:06 AM, MitchK wrote: Hello community, I need a minimum should match only on

Re: Short DismaxRequestHandler Question

2010-05-04 Thread MitchK
Thank you for responsing. This would be possible. However, I wouldn't like to do so, because a match in "title" should boost higher than a match in "category". -- View this message in context: http://lucene.472066.n3.nabble.com/Short-DismaxRequestHandler-Question-tp775913p776238.html Sent fr

Re: Problem with pdf, upgrading Cell

2010-05-04 Thread Praveen Agrawal
Hi Sandhya.. I must be missing something. I copied all dependencies jars to both contrib/extraction/lib and web-in/lib folders. Here is the list of jars copied: asm-3.1.jar bcmail-jdk15-1.45.jar bcprov-jdk15-1.45.jar commons-compress-1.0.jar commons-logging-1.1.1.jar dom4j-1.6.1.jar fontbox-1.1.0.

Lucidworks

2010-05-04 Thread joyce chan
Hi Does anybody know how to install LucidWorks Solr (LucidWorks.jar) without the gui installer? Or maybe to do it as a silent install? Thanks Joyce

Re: Lucidworks

2010-05-04 Thread joyce chan
Sorry, please ignore my previous message, I figured it out. (That is, use the console mode) On Tue, May 4, 2010 at 11:01 AM, joyce chan wrote: > Hi > > Does anybody know how to install LucidWorks Solr (LucidWorks.jar) without > the gui installer? Or maybe to do it as a silent install? > > Than

Re: Short DismaxRequestHandler Question

2010-05-04 Thread MitchK
I got an idea: If I would catenate all relevant fields to one large multiValued field, I could query like this: {!dismax qf='myLargeField^5'}solr development //mm is 1 (100%) if not set Additionally to that, I add a phraseQuery {!dismax qf='myLargeField^5'}solr development AND title:(solr develo

Re: cores and SWAP

2010-05-04 Thread Tim Heckman
It looks like this was not a solr issue at all. It looks like it's browser-related. When I use safari, curl or wget, I don't see the issue. When I use firefox or chrome, I do. I'll have to dig into this a little more with an http proxy to see what's going on. I have not altered the httpCaching se

Re: cores and SWAP

2010-05-04 Thread Erik Hatcher
The issue is that browsers (apparently not Safari?) will send the last- modified/etag headers to Solr and get back a 304 and your browser will simply display the last response it got. Use the force reload option from the browser (it's a habit for me now) to ensure you're actually getting a

[PECL-DEV] [ANNOUNCEMENT] solr-0.9.10 (beta) Released

2010-05-04 Thread Israel Ekpo
The new PECL package solr-0.9.10 (beta) has been released at http://pecl.php.net/. Release notes - - Increased compatibility with older systems running CentOS 4 or 5 and RHEL4 or 5 - Added ability to compile directly without having to build libcurl and libxml2 from source on older

Re: cores and SWAP

2010-05-04 Thread Tim Heckman
OK, yes, I see now. Even though the etags change when the swap happens, the last modified date on the server may be earlier than what the client has from the request prior to the swap. thank you. Tim On Tue, May 4, 2010 at 12:30 PM, Erik Hatcher wrote: > The issue is that browsers (apparently n

RE: Short DismaxRequestHandler Question

2010-05-04 Thread Naga Darbha
You may create a new field by copying the fields title, category and tag to the new field, like the following: and search against the new field. You may go for newField of type "textgen". Give it a try, Naga From: MitchK [mitc...@web.de] Sent: Tuesda

Re: Custom SearchComponent to reset facet value counts after collapse

2010-05-04 Thread MitchK
I would prefer extending the given CollapseComponent, because of performance-reasons. What you want to do sounds a bit like making things too complicate. There are two options I would prefer: 1. get the schema-information for every field you want to query against and define, whether you want to

SOLR-343 date facet mincount patch

2010-05-04 Thread Umesh_
Hi All, As per https://issues.apache.org/jira/browse/SOLR-343, the date facet mincount patch is tested. Has any one tried to apply this patch on Solr 1.4? When I tried I was able to patch 'SOLR-343.patch' but it failed for another 'DateFacetsMincountPatch.patch'. patching file src/java/org/apac

Re: How do I return all the results in an index?

2010-05-04 Thread Umesh_
querying for *:* works in Solr 1.4 as well. Did you check that your index has any data? ~Umesh -- View this message in context: http://lucene.472066.n3.nabble.com/How-do-I-return-all-the-results-in-an-index-tp777214p777239.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: How do I return all the results in an index?

2010-05-04 Thread Umesh_
Please post the query you are using. It could be something like 'http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on'. ~Umesh -- View this message in context: http://lucene.472066.n3.nabble.com/How-do-I-return-all-the-results-in-an-index-tp777214p777260.html Sent fr

inconsistency in SolrParams.get()

2010-05-04 Thread Frank Wesemann
Dear list, I recently stumpled upon this: modifiableParams = new ModifiableSolrParams( req.getParams() ); assert modifiableParams.get("key").equals( req.getParams().get("key") ); this test fails for requests built from a SimpleRequestParser or StandardRequestParser where the parameter "key" wa

Re: How do I return all the results in an index?

2010-05-04 Thread MitchK
Did you clean up the Browser-Cache? Maybe you need to restart (I am currently not sure, whether Solr caches HTTP-requests, even when you did a commit???). Kind regards - Mitch -- View this message in context: http://lucene.472066.n3.nabble.com/How-do-I-return-all-the-results-in-an-index-tp7772

Re: AutoSuggest with custom sorting

2010-05-04 Thread Chris Hostetter
First off: i would suggest that instead of doing a simple prefix search, you look into using EdgeNGrams for this sort of thing. I'm also assuming since you need custom scoring for this, you aren't going to get what you need using the TermsComponent or any other simple solution using your main

Re: Commit takes 1 to 2 minutes, CPU usage affects other apps

2010-05-04 Thread Lance Norskog
Are you accidentally building the spellchecker database on each commit? An option is to use the MergePolicy stuff to avoid merging during normal commits, but I failed to understand the interactions of configuration numbers. It's a bit of a jungle in there. On Tue, May 4, 2010 at 5:43 AM, wrote:

Re: Need help/assistance with Multicore admin/cores?action=CREATE

2010-05-04 Thread Chris Hostetter
: Shouldn't all the parameters be added to the solr.xml core2 that were yep .. it does in fact look like a bug in the solr.xml persistence code. please file a bug in Jira. : passed in from the URL? And why did the config="solrconfig.xml" get : removed from the core1 definition? i believe t

Re: Sort by membership of range query

2010-05-04 Thread Chris Hostetter
: What I can't quite figure out is how, when including all results, to sort : the results by whether they are "active" or not. In other words, have all : products within the date range appear before the products outside the date : range (or vice versa). if you want it to function as a true "sort"

Case Insensitive search while preserving case

2010-05-04 Thread dbashford
I've looked through the history and tried a lot of things but can't quite get this to work. Used this in my last attempt: What I'm looking to do is allow user's to execute case insensitive searches, which this does. "BLaH" should return all the "Blah"

Re: Case Insensitive search while preserving case

2010-05-04 Thread Ahmet Arslan
> I've looked through the history and tried a lot of things > but can't quite get > this to work. > > Used this in my last attempt: > >     class="solr.TextField" > positionIncrementGap="100"> >       >         class="solr.KeywordTokenizerFactory"/> >         class="solr.LowerCaseFilterFactor

Re: Case Insensitive search while preserving case

2010-05-04 Thread dbashford
All my fields are stored. And if my field name is "state" means that your suggestion is appending "fl=state", then no, that's not doing anything for me. =( The above config gets me part of the way to where I need to be. Storing, for instance, "Alaska" in such a way that querying for "alaska",

Re: Facets vs TermV's

2010-05-04 Thread Chris Hostetter
: Basically, what is the difference between issuing a facet field query : that returns facets with counts, : and a query with term vectors that also returns document frequency : counts for terms in a field? The FacetComponent generates counts that are relative the set of documents that match you

Re: AutoSuggest with custom sorting

2010-05-04 Thread Sean Timm
Chris Hostetter wrote: this can be accomplished by indexing a numeric field containing the "length" of the field as a number, and then doing a secondary sort on it. the fieldNorm typically takes care of this sort of thing for you, but is more of a generalized concept, and doesn't give you exac

Custom DIH variables

2010-05-04 Thread Blargy
Can someone please point me in the right direction (classes) on how to create my own custom dih variable that can be used in my data-config.xml So instead of ${dataimporter.last_index_time} I want to be able to create ${dataimporter.foo} Thanks -- View this message in context: http://lucene.47

Re: Commit takes 1 to 2 minutes, CPU usage affects other apps

2010-05-04 Thread Chris Hostetter
: Are you accidentally building the spellchecker database on each commit? ... : > This could also be caused by performing an optimize after the commit, or it : > could be caused by auto warming the caches, or a combination of both. The heart of the matter being: it's pretty much impossibl

Re: Monitoring via JMX; changing mbean names?

2010-05-04 Thread Chris Hostetter
: For example, when running multiple instances of solr in the same Tomcat : instance, each has an associated searc...@1234567 mbean. Alright, I : expect that. However, some values that I'm looking for (such as : avgRequestsPerSecond, avgTimePerRequest) are all located under the : solr/standa

Re: synonym filter problem for string or phrase

2010-05-04 Thread Chris Hostetter
: yes my default search field is text_sync. fields and fieldTypes are differnet things -- you've shows us the for "text_sync" but not the ... with out that we can't be sure you have things configured properly. you also having shown us the debugQuery output to help us know exactly what query

Re: copyField - how does it work?

2010-05-04 Thread Chris Hostetter
: : ... : : : Is the copyField valid specified in BLOCK-4? It seems it is not : populating the clubbed_string with the values of field_A and field_B. copyFields aren't chained together -- supporting that is "hard" and can lead to infinite loops, so each copyField src is co

Re: Custom SolrQueryRequest/SolrQueryResponse

2010-05-04 Thread Chris Hostetter
: Herein lies the problem from what I can tell: I don�t have any control : over SolrQueryRequest or SolrQueryResponse. My initial attempts have me : subclassing both of these to hold a List of requests and responses, with : a cursor that moves the �current� req/res each time through my handl

highlighting exact phrases bug?

2010-05-04 Thread Karthik Ram
Hi Folks, I am unable to get highlighting to work when searching for exact phrases in SOLR 1.4 A discussion about the exact same issue can be found here: http://www.mail-archive.com/solr-user@lucene.apache.org/msg27872.html Can someone please tell how to fix this? I am using the parameter hl.u

Re: highlighting exact phrases bug?

2010-05-04 Thread Mark Miller
You need to put memory.jar on the classpath along with highlighter.jar. On 5/4/10 10:38 PM, Karthik Ram wrote: Hi Folks, I am unable to get highlighting to work when searching for exact phrases in SOLR 1.4 A discussion about the exact same issue can be found here: http://www.mail-archive.com

Re: highlighting exact phrases bug?

2010-05-04 Thread Mark Miller
Hmm...this is actually an odd error if you are using the war though - this jar (lucene-memory-*.jar) should be in the webapp. On 5/4/10 10:48 PM, Mark Miller wrote: You need to put memory.jar on the classpath along with highlighter.jar. On 5/4/10 10:38 PM, Karthik Ram wrote: Hi Folks, I am un

SOLR Based Search - Response Times - what do you consider slow or fast?

2010-05-04 Thread dc tech
We are using SOLR in a production setup with a jRuby on Rails front end with about 20 different instances of SOLR running on heavy duty hardware. The setup is load balanced front end (jRoR) on a pair of machines and the SOLR backends on a different machine. We have plenty of memory and CPU and th

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Praveen, I only have the highlighted jars copied. Not sure, if we need the other jars. Also, I copied the jars directly into solr\WEB-INF\lib, like you did. Thanks, Sandhya -Original Message- From: Praveen Agrawal [mailto:pkal...@gmail.com] Sent: Tuesday, May 04, 2010 8:10 PM To:

RE: Problem with pdf, upgrading Cell

2010-05-04 Thread Sandhya Agarwal
Looks like the highlighting may not work here. Following is the list of jars I copied : asm-3.1.jar bcmail-jdk15-1.45.jar bcprov-jdk15-1.45.jar commons-compress-1.0.jar commons-logging-1.1.1.jar dom4j-1.6.1.jar fontbox-1.1.0.jar geronimo-stax-api_1.0_spec-1.0.1.jar jempbox-1.1.0.jar log4j-1.2.14.

max no of column in schema

2010-05-04 Thread Ranveer
Hi, How many column can we defined in schema. I have already around 100 column in schema.. thanks

Re: Custom DIH variables

2010-05-04 Thread Noble Paul നോബിള്‍ नोब्ळ्
you can use custom parameters from request like , ${dataimporter.request.foo}. pass the value of foo as a request param say foo=bar On Wed, May 5, 2010 at 6:05 AM, Blargy wrote: > > Can someone please point me in the right direction (classes) on how to create > my own custom dih variable that ca