server down caused by complex query

2011-11-24 Thread Jason, Kim
Hi all Nowadays our solr server is frequently down. Because our user send very long and complex queries with asterisk and near operator. Sometimes near operator exceeds 1,000 and keywords almost include asterisk. If such query is sent to server, jvm memory is full. (our jvm memory allocates 110G.)

RE: Index a null text field

2011-11-24 Thread jawedshamshedi
Hi Cody, Thanks for the reply. Please find the detail of that I am doing. Yes, I am using dataimport handler and the code snippet of it from solrconfig.xml is given below. data-config.xml The data-config.xml is give below.

Re: Huge Performance: Solr distributed search

2011-11-24 Thread Mark Miller
On Thu, Nov 24, 2011 at 12:09 PM, Artem Lokotosh wrote: > >How big are the documents you return (how many fields, avg KB per doc, > etc.)? > I have a following schema in my solr configuration name="field1" type="text" indexed="true" stored="false"/> name="field2" type="text" indexed="true" stored

remove answers with identical scores

2011-11-24 Thread Fred Zimmerman
I have a corpus that has a lot of identical or nearly identical documents. I'd like to return only the unique ones (excluding the "nearly identical" which are redirects). I notice that all the identical/nearly identicals have identical Solr scores. How can I tell Solr to throw out all the success

solrQueryParser defaultOperator

2011-11-24 Thread toto
Hi, I install Apache solr and integrate it on a drupal website. Everythings works perfectly. The default search operator is OR, so I changed it in my schema.xml as : But, it seems no working. For example, when I search : "bakery california", solr return all the results contains "bakery" OR "cali

Re: WordDelimiterFilter MultiPhraseQuery case insesitive Issue

2011-11-24 Thread Uomesh
Hi, I tried with preserveOriginal="1" and reindex too but still no result. Thanks, Umesh On Wed, Nov 23, 2011 at 5:33 PM, Shawn Heisey-4 [via Lucene] < ml-node+s472066n3532405...@n3.nabble.com> wrote: > On 11/23/2011 2:54 PM, Uomesh wrote: > > > Hi, > > > > case insesitive search is not working

RE: Index a null text field

2011-11-24 Thread Young, Cody
Hello, We'll need more information please. How are you indexing the documents? DataImportHandler? Xml Updates? Can you show us the relevant parts of your schema? (Field definition and data type for the field) Are you getting any error messages in the log files? Tell us more about your environme

Attempting to achieve something similar to PostgreSQL's pg_trgm / K-NN combo with Solr

2011-11-24 Thread Matt Patterson
Hello, I'm working on using trigrams for similarity matching on some data, where there's a canonical name and lots of personalised variants, e.g.: canonical: "My Wonderful Thing" variant: "My Wonderful Thing (for Matt Patterson)" Using the pg_trgm (http://wiki.postgresql.org/wiki/What's_new_in

Re: Huge Performance: Solr distributed search

2011-11-24 Thread Artem Lokotosh
>How big are the documents you return (how many fields, avg KB per doc, etc.)? I have a following schema in my solr configuration 27M–30M docs and 12-15 GB for each shard, 0.5KB per doc >Does performance get much better if you only request top 100, or top>10 >documents instead of top 1000?

Re: Huge Performance: Solr distributed search

2011-11-24 Thread Artem Lokotosh
>> Can you merge, e.g. 3 shards together or is it much effort for your >> team?>Yes, we can merge. We'll try to do this and review how it will works Merge does not help :(I've tried to merge two shards in one, three shards in one, but results are similar to results first configuration with 30 shar

Fwd: Clustering and FieldType

2011-11-24 Thread Geetu Ambwani
Sent from my iPhone Begin forwarded message: > From: Geetu Ambwani > Date: November 23, 2011 2:52:38 PM EST > To: solr-user-i...@lucene.apache.org > Subject: Clustering and FieldType > > Hi > Trying to use carrot2 for clustering search results. I have it setup except > it seems to treat the

highlighting performance poor with *.tar, *.gz files

2011-11-24 Thread Shyam Bhaskaran
Hi, It is observed that highlighting of search results is taking too much time especially for highlighting terms for archived files like *.gz, *.tar, *.zip. What could be the reason behind it ? Is it because these files are unzipped and then highlighted from the index during display time ? Or is

solr-user@lucene.apache.org

2011-11-24 Thread Tomasz Wegrzanowski
On 22 November 2011 14:28, Jan Høydahl wrote: > Why do you need spaces in the replacement? > > Try pattern="\+" replacement="plus" - it will cause the transformed > charstream to contain as many tokens as the original and avoid the > highlighting crash. I tried that, it still crashes. Replacin

Index a null text field

2011-11-24 Thread jawedshamshedi
Hi all, I am indexing a table that has a field by the name of solr_keywords of type text in mysql. And it contains null values also. While creating index in solr, this field is not getting indexed. Any help will be appreciated. Thanks -- View this message in context: http://lucene.472066.n3.

inconsistent JVM crash with version 4.0-SNAPSHOT

2011-11-24 Thread Lasse Aagren
Hi, We are running Solr-Lucene 4.0-SNAPSHOT (1199777M - hudson - 2011-11-09 14:58:50) on severel servers running: 64bit Debian Squeeze (6.0.3) OpenJDK6 (b18-1.8.9-0.1~squeeze1) Tomcat 6.028 (6.0.28-9+squeeze1) Some of the servers have 48G RAM and in that case java have 16G (-Xmx16g) and some o

Re: highlighting on range query

2011-11-24 Thread Rahul Mehta
Any other Suggestion. On Thu, Nov 24, 2011 at 5:30 PM, Rahul Mehta wrote: > Yes, I tried with specifiying hl.fl=field1, and field1 is indexed and > stored. > > > On Thu, Nov 24, 2011 at 5:23 PM, Ahmet Arslan wrote: > >> > oh sorry forgot to tell you that i >> > added &hl.usePhraseHighlighter=tru

Re: highlighting on range query

2011-11-24 Thread Rahul Mehta
Yes, I tried with specifiying hl.fl=field1, and field1 is indexed and stored. On Thu, Nov 24, 2011 at 5:23 PM, Ahmet Arslan wrote: > > oh sorry forgot to tell you that i > > added &hl.usePhraseHighlighter=true this > > also , but still no result is coming . > > Did you specify field1 in hl.fl p

Re: highlighting on range query

2011-11-24 Thread Ahmet Arslan
> oh sorry forgot to tell you that i > added &hl.usePhraseHighlighter=true this > also , but still no result is coming . Did you specify field1 in hl.fl parameter? Plus you need you mark field1 as indexed="true" and stored="true" to enable highlighting. http://wiki.apache.org/solr/FieldOptionsB

Re: highlighting on range query

2011-11-24 Thread Rahul Mehta
oh sorry forgot to tell you that i added &hl.usePhraseHighlighter=true this also , but still no result is coming . On Thu, Nov 24, 2011 at 5:14 PM, Ahmet Arslan wrote: > > I passed &hl.highlightMultiTerm=true in request ,* but > > still field1 is not > > coming in hightlighting.* > > > > > http:

Re: highlighting on range query

2011-11-24 Thread Ahmet Arslan
> I passed &hl.highlightMultiTerm=true in request ,* but > still field1 is not > coming in hightlighting.* > > http://localhsot:8983/solr/select?q=field1:[5000%20TO%206000]&fl=field2&hl=on&rows=5&wt=json&indent=on&hl.fl=field3&hl.highlightMultiTerm=true > As wiki says "If the SpanScorer is also

Re: highlighting on range query

2011-11-24 Thread Rahul Mehta
Hi Ahmet, I passed &hl.highlightMultiTerm=true in request ,* but still field1 is not coming in hightlighting.* http://localhsot:8983/solr/select?q=field1:[5000%20TO%206000]&fl=field2&hl=on&rows=5&wt=json&indent=on&hl.fl=field3&hl.highlightMultiTerm=true I am using solr 3.1. is i need to install

Re: Integrating Surround Query Parser

2011-11-24 Thread Rahul Mehta
Okay, thanks for reply. On Thu, Nov 24, 2011 at 2:35 PM, Erik Hatcher wrote: > > On Nov 23, 2011, at 09:56 , Ahmet Arslan wrote: > > > > >> is this is the trunk of solr 4.0 , > >> can't i implement in solr 3.1 .? > > > > Author of the patch would know answer to this. But why not use trunk? > > I

Re: highlighting on range query

2011-11-24 Thread Ahmet Arslan
> I want to have result of a range query with highlighted > Result. http://wiki.apache.org/solr/HighlightingParameters#hl.highlightMultiTerm

Re: need a way so that solr return result for misspelled terms

2011-11-24 Thread Ahmet Arslan
> Hi Erik , > I am sorry , i did not get you exactly. do you tries to say > that tilde (~) > works for single term only. > Say for ex. i have sentence like "i like solr speed for > searching." and i > try to search with slor~ , then it will not work bcoz it > "inside of phrases" > ? or i misunderst

highlighting on range query

2011-11-24 Thread Rahul Mehta
Hello, I want to have result of a range query with highlighted Result. e.g. i have this query http://localhsot:8983/solr/select?q=field1:[5000%20TO%206000]&fl=field2&hl=on&rows=5&wt=json&indent=on&hl.fl=field3 is not giving any result in hightliting. Please suggest how can i get the result? --

Re: need a way so that solr return result for misspelled terms

2011-11-24 Thread meghana
Hi Erik , I am sorry , i did not get you exactly. do you tries to say that tilde (~) works for single term only. Say for ex. i have sentence like "i like solr speed for searching." and i try to search with slor~ , then it will not work bcoz it "inside of phrases" ? or i misunderstood you. plz clari

Re: complex phrase plugin install

2011-11-24 Thread meghana
is this for wildcard search and search for misspell words. i need the same to do in my application. -- View this message in context: http://lucene.472066.n3.nabble.com/complex-phrase-plugin-install-tp3533123p3533182.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: complex phrase plugin install

2011-11-24 Thread Ahmet Arslan
> I want to install complex phrase plugin this one. > https://issues.apache.org/jira/browse/SOLR-1604?focusedCommentId=12923982&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12923982 > > I had done following step and got an error : 'mvn package' works for me. (Apac

complex phrase plugin install

2011-11-24 Thread Rahul Mehta
Hi, I want to install complex phrase plugin this one. https://issues.apache.org/jira/browse/SOLR-1604?focusedCommentId=12923982&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12923982 I had done following step and got an error : - configure maven path variable

Re: need a way so that solr return result for misspelled terms

2011-11-24 Thread Erik Hatcher
The default query parser in Solr is the "lucene" one. q=term~ But there is nothing that automatically makes terms fuzzy with the ~ at the end. (and fuzzy queries only work on individual terms, not terms inside "of phrases"). Erik On Nov 24, 2011, at 03:08 , meghana wrote: > okey,

Re: Integrating Surround Query Parser

2011-11-24 Thread Erik Hatcher
On Nov 23, 2011, at 09:56 , Ahmet Arslan wrote: > >> is this is the trunk of solr 4.0 , >> can't i implement in solr 3.1 .? > > Author of the patch would know answer to this. But why not use trunk? I spent a fair bit of time yesterday on making a 3.x compatible patch but have not completed th

Re: Autocomplete(terms) performance problem

2011-11-24 Thread roySolr
Thanks, it looks great! In the nearby future i will give it a try. -- View this message in context: http://lucene.472066.n3.nabble.com/Autocomplete-terms-performance-problem-tp3351352p3533066.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: need a way so that solr return result for misspelled terms

2011-11-24 Thread meghana
okey, i am not very much aware of it , can i use lucene query parser with solr and make this fuzzy search possible? Erik Hatcher-4 wrote > > Sure... if you're using the "lucene" query parser and put a ~ after every > term in the query :) > > But that would mean that either the users or your appl