RE: Field collapsing memory usage

2015-01-22 Thread Norgorn
Thank you for your answer. We've found out that the problem was in our SOLR spec (heliosearch 0.08). There are no crushes, after changing to 4.10.3 (although, there are lot of OOMs while handling query, it's not really strange for 1.1 bil of documents ). Now we are going to try latest Heliosearch.

WordDelimiterFilterFactory and position increment.

2015-01-22 Thread Modassar Ather
Hi, I am using WordDelimiterFilter while indexing. Parser used is edismax. Phrase search is failing for terms like "3d image". On the analysis page it shows following four tokens for *3d* and there positions. *token position* 3d 1 3 1 3d 1 d

RE: Field collapsing memory usage

2015-01-22 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > Is there any way to make 'docValues="true"' without reindexing? Depends on how brave you are :-) We recently had the same need and made https://github.com/netarchivesuite/dvenabler To my knowledge that is the only existing tool for that task an as we are the

RE: Field collapsing memory usage

2015-01-22 Thread Norgorn
Nice, thanks! If u'd like to, I'll write our results with that amazing util. -- View this message in context: http://lucene.472066.n3.nabble.com/Field-collapsing-memory-usage-tp4181092p4181159.html Sent from the Solr - User mailing list archive at Nabble.com.

RE: Field collapsing memory usage

2015-01-22 Thread Toke Eskildsen
Norgorn [lsunnyd...@mail.ru] wrote: > Nice, thanks! > If u'd like to, I'll write our results with that amazing util. By all means, please do. Good as well as bad. Independent testing is needed to ensure proper working tools. - Toke Eskildsen

Re: If I change schema.xml then reIndex is neccessary in Solr or not?

2015-01-22 Thread Vishal Swaroop
We noticed that SOLR/ Tomcat also needs a restart... is it same for you also ? Regards On Thu, Jan 22, 2015 at 2:11 AM, Nitin Solanki wrote: > Ok. Thanx > > On Thu, Jan 22, 2015 at 11:38 AM, Gora Mohanty wrote: > > > On 22 January 2015 at 11:23, Nitin Solanki wrote: > > > I *indexed* *2GB* o

Retrieving Phonetic Code as result

2015-01-22 Thread Amit Jha
Hi, I need to know how can I retrieve phonetic codes. Does solr provide it as part of result? I need codes for record matching. *following is schema fragment:*

Re: If I change schema.xml then reIndex is neccessary in Solr or not?

2015-01-22 Thread Shawn Heisey
On 1/22/2015 6:25 AM, Vishal Swaroop wrote: > We noticed that SOLR/ Tomcat also needs a restart... is it same for you > also ? For a change in solrconfig or schema to become effective, the core or collection must be reloaded, or a container restart is required. Once the change is active because o

Re: Retrieving Phonetic Code as result

2015-01-22 Thread Shawn Heisey
On 1/22/2015 6:42 AM, Amit Jha wrote: > I need to know how can I retrieve phonetic codes. Does solr provide it as > part of result? I need codes for record matching. > > *following is schema fragment:* > > class="solr.TextField" > > > > maxCodeLength="4"/> > >

Re: Retrieving Phonetic Code as result

2015-01-22 Thread Amit Jha
Hi, I need to know how can I retrieve phonetic codes. Does solr provide it as part of result? I need codes for record matching. *following is schema fragment:* Hi, Thanks for response, I can see generated MetaPhone codes using Luke. I am us

Re: Retrieving Phonetic Code as result

2015-01-22 Thread Amit Jha
Thanks for response, I can see generated MetaPhone codes using Luke. I am using solr only because it creates the phonetic code at time of indexing. Otherwise for each record I need to call Metaphone algorithm in realtime to get the codes and compare them. I think when luke can read and display it,

Re: Retrieving Phonetic Code as result

2015-01-22 Thread Alexandre Rafalovitch
What are you actually trying to do on a business level? Because this feels like an "XY Problem": https://people.apache.org/~hossman/#xyproblem Solr will generate MetaPhone during indexing, then during Query and will do the matching. It's not clear why you actually want to get those codes back to o

Re: Retrieving Phonetic Code as result

2015-01-22 Thread Erik Hatcher
Faceting returns indexed terms. So adding &facet=on&facet.field=firstname_phonetic will get you back the phonetic codes across an entire result set. If you have a single string and want the phonetic codes back, you can use the analysis request handler (document or field). For a bit more detai

Suggester Example In Documentation Not Working

2015-01-22 Thread Charles Sanders
Attempting to follow the documentation found here: https://cwiki.apache.org/confluence/display/solr/Suggester The example given in the documentation is not working. See below my configuration. I only changed the field names to those in my schema. Can anyone provide an example for this componen

Is there a way to pass in proxy settings to Solr?

2015-01-22 Thread Carl Roberts
Hi, Is there a way to pass in proxy settings to Solr? The reason that I am asking this question is that I am trying to run the DIH RSS example, and it is not working when I try to import the RSS feed URL because the code in Solr comes back with an unknown host exception due to the proxy that

Re: Is Solr a good candidate to index 100s of nodes in one XML file?

2015-01-22 Thread Carl Roberts
Thanks. I am looking at the RSS DIH example right now. On 1/21/15, 3:15 PM, Alexandre Rafalovitch wrote: Solr is just fine for this. It even ships with an example of how to read an RSS file under the DIH directory. DIH is also most likely what you will use for the first implementation. Don't

Re: Is Solr a good candidate to index 100s of nodes in one XML file?

2015-01-22 Thread Carl Roberts
Thanks for the input. I think one benefit of using Solr is also that I can provide a REST API to search the indexed records. Regards, Joe On 1/21/15, 3:17 PM, Shawn Heisey wrote: On 1/21/2015 12:53 PM, Carl Roberts wrote: Is Solr a good candidate to index 100s of nodes in one XML file? I ha

Re: Is there a way to pass in proxy settings to Solr?

2015-01-22 Thread Shawn Heisey
On 1/22/2015 9:18 AM, Carl Roberts wrote: > Is there a way to pass in proxy settings to Solr? > > The reason that I am asking this question is that I am trying to run > the DIH RSS example, and it is not working when I try to import the > RSS feed URL because the code in Solr comes back with an unk

How to query raw query String with Solrj?

2015-01-22 Thread Tim Molter
I'd like to query solr with solrj with a raw query such as: `class_id%3ABINGBONG%0ABlah%3A3232235780&sort=id+desc&rows=100`. These queseries are stored in a database and I cannot use the "builder" API offered by solrj (SolrQuery). Any suggestions?? signature.asc Description: OpenPGP digital sign

Re: Suggester Example In Documentation Not Working

2015-01-22 Thread Tomás Fernández Löbbe
I see that the docs say that the doc needs to be indexed only, but for Fuzzy or Analyzed, I think the field needs to be stored. On the other side, not sure how much sense it makes to use any of those two implementations if the field type you want to have is "string". Tomás On Thu, Jan 22, 2015 at

Re: Suggester Example In Documentation Not Working

2015-01-22 Thread Chris Hostetter
1) which version of Solr are you using? (note that the online HTML ref guide is a DRARFT that applies to 5.0 - you may want to review the specific released version of the ref guide that applies to your version of solr: http://archive.apache.org/dist/lucene/solr/ref-guide/ 2) the behavior of th

Re: How to query raw query String with Solrj?

2015-01-22 Thread Erik Hatcher
Maybe SolrQueryParsers.parseQueryString() is what you’re looking for. Erik > On Jan 22, 2015, at 9:41 AM, Tim Molter wrote: > > I'd like to query solr with solrj with a raw query such as: > `class_id%3ABINGBONG%0ABlah%3A3232235780&sort=id+desc&rows=100`. These > queseries are stored in

Re: Solr 4.10.3 start up issue

2015-01-22 Thread Chris Hostetter
: had thought to do this before - and should have; I uploaded the full : example collection configuration to ZK just now and tried again. Magic, it : worked, which left me feeling a bit glum. Well, happy that it wasn't Solr. : Now if you'll excuse me, I have a conf review to perform. if your prob

SolrCloud timing out marking node as down during startup.

2015-01-22 Thread Michael Roberts
Hi, I'm seeing some odd behavior that I am hoping someone could explain to me. The configuration I'm using to repro the issue, has a ZK cluster and a single Solr instance. The instance has 10 Cores, and none of the cores are sharded. The initial startup is fine, the Solr instance comes up and w

Re: Issue with Solr multiple sort

2015-01-22 Thread Erick Erickson
Shamik: Nice job of including the relevant information and just the relevant info! One addition to what Chris said that _may_ be relevant in future. The "string" type is totally unanalyzed, so sorting done on that field may be case-sensitive, leading to some confusion. If the schema has a "lowerc

Re: Solr Recovery process

2015-01-22 Thread Erick Erickson
Shalin: Just to see if my understanding is correct, how often would you expect <2> to occur? My assumption so far is that it would be quite rare that the leader and all replicas happened to hit autocommit points at the same time and thus it would be save to just bring down a few segments. But that

Avoiding wildcard queries using edismax query parser

2015-01-22 Thread Jorge Luis Betancourt González
Hello all, Currently we are using edismax query parser in an internal application, we've detected that some wildcard queries including "*" are causing some performance issues and for this particular case we're not interested in allowing any user to request all the indexed documents. This coul

Re: Avoiding wildcard queries using edismax query parser

2015-01-22 Thread Alexandre Rafalovitch
I suspect the special characters get caught before the analyzer chains. But what about pre-pending a custom search components? Regards, Alex. Sign up for my Solr resources newsletter at http://www.solr-start.com/ On 22 January 2015 at 16:33, Jorge Luis Betancourt González wrote: > Hell

Re: Field collapsing memory usage

2015-01-22 Thread Erick Erickson
Toke: What do you think about folding this into the Solr (or Lucene?) code base? Or is it to specialized? Not sure one way or the other, just askin' Erick On Thu, Jan 22, 2015 at 3:47 AM, Toke Eskildsen wrote: > Norgorn [lsunnyd...@mail.ru] wrote: >> Is there any way to make 'docValues="t

Re: Avoiding wildcard queries using edismax query parser

2015-01-22 Thread Jack Krupansky
The problem is that the presence of a wildcard causes Solr to skip the usual token analysis. But... you could add a "multiterm" analyzer, and then the wildcard would just get treated as punctuation. -- Jack Krupansky On Thu, Jan 22, 2015 at 4:33 PM, Jorge Luis Betancourt González < jlbetanco...@u

How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Carl Roberts
Hi, How do you query a sentence composed of multiple words in a description field? I want to search for sentence "Oracle Fusion Middleware" but when I try the following search query in curl, I get nothing: curl "http://localhost:8983/solr/nvd-rss/select?q=summary:Oracle Fusion Middleware&w

Re: How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Walter Underwood
Your query is this: summary:Oracle Fusion Middleware That searches for “Oracle” in the summary field and “Fusion” and “Middleware” in whatever your default field is. You want: summary:”Oracle Fusion Middleware” wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ On

Re: Avoiding wildcard queries using edismax query parser

2015-01-22 Thread Jorge Luis Betancourt González
I was also suspecting something like that, the odd thing was that the with the dismax parser this seems to work, I mean passing a single * in the query just like: http://localhost:8983/solr/collection1/select?q=*&wt=json&indent=true&defType=dismax Returns: { "responseHeader":{ "status":0

Re: How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Carl Roberts
Hi Walter, If I try this from my Mac shell: curl http://localhost:8983/solr/nvd-rss/select?wt=json&indent=true&q=summary:"Oracle Fusion" I don't get a response. If I try this, it works!: curl "http://localhost:8983/solr/nvd-rss/select?wt=json&indent=true&q=name:Oracle"; So I thi

Re: How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Shawn Heisey
On 1/22/2015 4:31 PM, Carl Roberts wrote: > Hi Walter, > > If I try this from my Mac shell: > > curl > http://localhost:8983/solr/nvd-rss/select?wt=json&indent=true&q=summary:"Oracle > Fusion" > > I don't get a response. Quotes are a special character to the shell on your mac, and get removed

Re: How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Carl Roberts
Thanks Shawn - I tried this but it does not work. I don't even get a response from curl when I try that format and when I look at the logging on the console for Jetty I don't see anything new - it seems that the request is not even making it to the server. On 1/22/15, 6:43 PM, Shawn Heisey w

Re: How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Erick Erickson
Have you considered using the admin/query form? Lots of escaping is done there for you. Once you have the form of the query down and know what to expect, it's probably easier to enter "escaping hell" with curl and the like And what is your schema definition for the field in question? the admin

Re: zk disconnects and failure to retry?

2015-01-22 Thread deniz
bumping an old entry... but are there any improvements on this issue? - Zeki ama calismiyor... Calissa yapar... -- View this message in context: http://lucene.472066.n3.nabble.com/zk-disconnects-and-failure-to-retry-tp4065877p4181370.html Sent from the Solr - User mailing list archive at Na

In a SolrCloud, will a solr core(shard replica) failover to its good peer when its state is not Active

2015-01-22 Thread 汤林
A solr core have several state, besides "Active", there are "Recovering", "Down", "Recovery failed" and "Gone". I know when the state is Recovering, the query or index request can be failover to its leader(the good one), but I'm not sure other state, especially the "Down" state at the solr server j

Re: zk disconnects and failure to retry?

2015-01-22 Thread Erick Erickson
Oh yes, lots in the past 8 months, the JIRAs can give details. Best, Erick On Thu, Jan 22, 2015 at 4:10 PM, deniz wrote: > bumping an old entry... but are there any improvements on this issue? > > > > - > Zeki ama calismiyor... Calissa yapar... > -- > View this message in context: > http://

Re: In a SolrCloud, will a solr core(shard replica) failover to its good peer when its state is not Active

2015-01-22 Thread Erick Erickson
As long as one replica for each shard is "active", you should be able to query the collection. You an also index to the collection and it'll all "just work", when the replicas that are not active become active they'll get the updates and catch up to the leader. This process may take quite some tim

Re: In a SolrCloud, will a solr core(shard replica) failover to its good peer when its state is not Active

2015-01-22 Thread 汤林
Thanks, Erick. You are right. My question is : When a Solr server is running, but a core(shard replica) on it is NOT "Active", for example, "Down", will the query request to it be failed over to the good replica of the same shard? Thanks! 2015-01-23 10:26 GMT+08:00 Erick Erickson : > As long as

Re: How do you query a sentence composed of multiple words in a description field?

2015-01-22 Thread Jack Krupansky
It appears that you are actually intending to query a phrase rather than a complete sentence. The former are easy - just enclose the phrase in quotes. Fielded query applies to a single term, a quoted phrase, or a parenthesized sub-query, so in your query it applied only to that first term, so Solr

Re: Avoiding wildcard queries using edismax query parser

2015-01-22 Thread Jack Krupansky
The dismax query parser does not support wildcards. It is designed to be simpler. -- Jack Krupansky On Thu, Jan 22, 2015 at 5:57 PM, Jorge Luis Betancourt González < jlbetanco...@uci.cu> wrote: > I was also suspecting something like that, the odd thing was that the with > the dismax parser this

Re: Avoiding wildcard queries using edismax query parser

2015-01-22 Thread Jorge Luis Betancourt González
Hi Jack! Yes, that was my point, I was thinking that being edismax an extended version of dismas, perhaps had a switch to turn on/off this feature or putting some limits. I've tried the multiterm approach but with no luck, the "*" keeps being treated a match all query, as far as I can see from

trying to get Apache Solr working with Dovecot.

2015-01-22 Thread Kevin Laurie
Hello, I am desperately trying to get Apache Solr to work with Dovecot FTS. I would really appreciate if someone could please help me! I have already done the following:- 1. I can ssh into my server and see that Apache Solr is up and running. ssh -t -L 8983:localhost:8983 u...@mydomain.com 2

Re: In a SolrCloud, will a solr core(shard replica) failover to its good peer when its state is not Active

2015-01-22 Thread 汤林
Thanks, Erick. >From a testing aspect, if we would like to verify the case that a query request to a "down" core on a running server will be failed over to the good core on another running server, is there any way to make a core as "down" on a running server? Thanks! We tried to change the /clust

Using tmpfs for Solr index

2015-01-22 Thread deniz
Would it boost any performance in case the index has been switched from RAMDirectoryFactory to use tmpfs? Or it would simply do the same thing like MMap? And in case it would be better to use tmpfs rather than RAMDirectory or MMap, which directory factory would be the most feasible one for this p

Re: trying to get Apache Solr working with Dovecot.

2015-01-22 Thread Alexandre Rafalovitch
Well, what does seem to happen? Which version of Solr is it? Can Dovecot contact Solr? If you put netcat listen instead of Solr on that port, it is being connected to? If it is, is Solr complaining about wrong url or anything in the log? Exceptions maybe. How far into the Dovecot->Solr path did

Re: trying to get Apache Solr working with Dovecot.

2015-01-22 Thread Kevin Laurie
Dear Alexandre, Thanks for your feedback. The solr / lucene version is 4.10.2 I am trying to figure out how to see if Dovecot and Solr can contact. Apparently when I make searches there seems to be no contact. I might try to rebuild dovecot again and see if that solves the problem. I just checked

Count total frequency of a word in a SOLR index

2015-01-22 Thread Nitin Solanki
I indexed some text_file files in Solr as it is. Applied " *StandardTokenizerFactory*" and "*ShingleFilterFactory*" on text_file field *Configuration of Schema.xml structure below :* *