How to search with single string in a multivalued string field

Hi, I have a field name called, features_mvs. Slim,Compact Large Storage,Efficient I want to search with either of the string, Slim or Compact or Large Storage or Efficient. If i give, select?q=features_mvs:Slim,Compact it is returning, but i want to search like, select?q=features_mvs:Slim H

Re: stopwords in solr

Eliminating stopwords is generally a bad idea. It means you cannot search for "vitamin a". Back in the 1970's, search engines eliminated stopwords so they could work on 16-bit machines. That isn't a problem any more. wunder On Nov 27, 2012, at 10:33 PM, Joe Zhang wrote: > that is really stran

Re: stopwords in solr

On Nov 28, 2012, at 12:33 AM, Joe Zhang wrote: > that is really strange. so basic stopwords such as "a" "the' are not > eliminated from the index? There is no list of "basic stopwords" anywhere. If you want stop words, you have to put them in the file yourself. There are not really any sensi

Re: stopwords in solr

that is really strange. so basic stopwords such as "a" "the' are not eliminated from the index? On Tue, Nov 27, 2012 at 11:16 PM, 曹霖 wrote: > justt no stopwords are considered in that case > > 2012/11/28 Joe Zhang > > > t no stopwords are considered in > > this case > > >

Re: stopwords in solr

justt no stopwords are considered in that case 2012/11/28 Joe Zhang > t no stopwords are considered in > this case >

Re: Help with sort on dynamic field and out of memory error

Erick, Thanks for your reply. So there is no easy way to get around this problem. We have a way to rework the schema by keeping a single sort field. The dynamic fields we have are like relevance_CLASSID. The current schema has a unique key NODEID and a multi-valued field CLASSID - the relevance s

Re: Help with sort on dynamic field and out of memory error

I sure don't see how this can work given the constraints. Just to hold the values, assuming that each doc holds a value in 150 fields, you have 150 * 4 * 14,000,000 or 8.4G of memory required, and you just don't have that much memory to play around with. Sharding seems silly for 14M docs, but that

Re: Odd casting error in embedded Jetty container

: > lucene-analyzers-common-4.0.0.jar from 2 places: : > : > /Users/username/.m2/repository/org/apache/lucene/lucene-analyzers-common/4.0.0 : > : > /Users/username/project/branch/workspace/Project/solr-webapp/webapp/WEB-INF/lib/ Why are identical jars getting loaded from two differnet places?

Re: Odd casting error in embedded Jetty container

Are you sure that NamedSPILoader respects -verbose flag? I would still check the filesystem access using something that does not lie (see my old email). Regards, Alex. Personal blog: http://blog.outerthoughts.com/ LinkedIn: http://www.linkedin.com/in/alexandrerafalovitch - Time is the quality

Re: Odd casting error in embedded Jetty container

Well, it was a nice theory. Siiihhh On Tue, Nov 27, 2012 at 4:49 PM, Mark Bennett wrote: > Hi Alex and Erick, > > I added -verbose which shows every class load. > > From the 12,000+ lines of output, I only see 4.0.0 jars, wrt lucene and > solr. > > lucene-analyzers-common-4.0.0.jar

Re: Extreme index size reduction on 4.1-SNAPSHOT?

On 11/27/2012 2:25 PM, Markus Jelsma wrote: Hi, please check this issue: https://issues.apache.org/jira/browse/LUCENE-4226 But it is enabled because of: https://issues.apache.org/jira/browse/LUCENE-4509 Since it's suddenly default you would have to completely wipe the index and reindex the dat

Re: Snapshot exception

Perhaps you can file a JIRA ticket with your findings? - Mark On Nov 27, 2012, at 5:31 PM, Marcin Rzewucki wrote: > Yes, I have and it works. It also works for SolrCloud, but not always. > Sometimes "missing file" error occurs. Restarting solr helps but it's not > good solution for me. > > On

Re: Snapshot exception

Yes, I have and it works. It also works for SolrCloud, but not always. Sometimes "missing file" error occurs. Restarting solr helps but it's not good solution for me. On 27 November 2012 23:09, Mark Miller wrote: > I don't know that I have ever tried a backup with SolrCloud, but I can't > think

Re: predefined variables usable in schema.xml ?

: The default solrconfig.xml seems to suggest ${solr.data.dir} can be used. : So I am hoping there is another pre-defined variable like this that points to : the solr core directory. there's nothing special about solr.data.dir ... it's used i nthe example configs as a convinient way to let you o

predefined variables usable in schema.xml ?

Is there a pre-defined variable that can be used in schema.xml to point to the solr core directory, or the conf subdirectory? I thought ${solr.home} or perhaps ${solr.solr.home} might work but they didn't (unless -Dsolr.home=/my/solr/home is supplied, that is). The default solrconfig.xml seems t

Re: Snapshot exception

I don't know that I have ever tried a backup with SolrCloud, but I can't think of any reason it should not work. Have you tried a back up with a single node Solr setup with 4x? - Mark On Nov 27, 2012, at 4:28 PM, Marcin Rzewucki wrote: > Hi, > > I have SolrCloud4x. I'd like to be able to mak

Re: Odd casting error in embedded Jetty container

Hi Alex and Erick, I added -verbose which shows every class load. >From the 12,000+ lines of output, I only see 4.0.0 jars, wrt lucene and solr. lucene-analyzers-common-4.0.0.jar from 2 places: /Users/username/.m2/repository/org/apache/lucene/lucene-analyzers-common/4.0.0 /Users/username/project

Snapshot exception

Hi, I have SolrCloud4x. I'd like to be able to make index backup on each node. When I did /replication?command=backup on one of them I got: File /data/index.20121119140848151/segments_1am does not exist What does it mean? File is not there indeed, but querying and indexing works fine. Also I don'

RE: Extreme index size reduction on 4.1-SNAPSHOT?

Hi, please check this issue: https://issues.apache.org/jira/browse/LUCENE-4226 But it is enabled because of: https://issues.apache.org/jira/browse/LUCENE-4509 Since it's suddenly default you would have to completely wipe the index and reindex the data, at least i had to, because of numerous code

Extreme index size reduction on 4.1-SNAPSHOT?

With a 4.1 snapshot from a couple of weeks ago, I saw about a 5% drop in index size compared to 3.5.0 when using the same schema. When I updated my 4.1 schema to ICUTokenizer so I could use CJKBigramFilter, my index dropped further -- about 10% less than 3.5, still using the same 4.1 snapshot.

Re: Solr 4, optimizing while doing other updates?

On 11/27/2012 1:30 PM, Yonik Seeley wrote: Expunging a single delete from a segment involves re-writing the entire segment, so it's just as bad as optimize (assuming most segments have a deletion). You might as well get the benefit of the optimize as well. I have had the same thought here.

Re: Solr 4, optimizing while doing other updates?

On Tue, Nov 27, 2012 at 3:21 PM, Shawn Heisey wrote: > but even way back then, rumblings on the mailing list said "don't optimize > for performance reasons." Count me amongst the dissenters. Optimize can make a lot of sense, and that's why it still exists. People should be careful to not assum

Re: Bulk Indexing Question

On 11/27/2012 1:07 PM, Joseph C. Trubisz wrote: When I curl a file to be indexed (in this case, as CSV), how do I know which index it’s going to, if I have multiple indexes currently being managed by Solr? For example, I have indexes for drug, company, author, abstract and I want to CSV load to

Re: Solr 4, optimizing while doing other updates?

On 11/27/2012 11:39 AM, Jack Krupansky wrote: So, if I understand your scenario correctly, you are doing a lot of deletes, but since they are occurring against "cold" data, there is isn't usually much if any query traffic for that old/cold data. In short, it sounds like the reason you are opti

Bulk Indexing Question

Greetings… I’m new to Solr, so this might be a real amateur question. When I curl a file to be indexed (in this case, as CSV), how do I know which index it’s going to, if I have multiple indexes currently being managed by Solr? For example, I have indexes for drug, company, author, abstract and

Re: SolrCloud and MoreLikeThis: SOLR-788

I wouldn't use a random code - follow the pattern of increments. - Mark On Tue, Nov 27, 2012 at 2:29 PM, Luis Cappa Banda wrote: > Hello, Mark. > > I didn´t know if adding that constant (with a random hexadecimal selected > by me) could affect to other parts of code. In fact some day it existed

Re: SolrCloud and MoreLikeThis: SOLR-788

Hello, Mark. I didn´t know if adding that constant (with a random hexadecimal selected by me) could affect to other parts of code. In fact some day it existed and currently not, so it may be a reason why that occurred. Also, copy & pasting a patch that you don´t really understand what´s doing and

Re: SolrCloud and MoreLikeThis: SOLR-788

Add it? - Mark On Nov 27, 2012, at 6:33 AM, Luis Cappa Banda wrote: > I´ve recently patched Solr 4.0 tag but it exists a conflict > with PURPOSE_GET_MLT_RESULTS final static attributed in ShardRequest.java. > The thing is that it doesn´t exist, :-/ Any tips? > > Regards, > > > - Luis Cappa.

Re: SolrCloud Performance - Indexing

Yup, DIH is not optimal for SolrCloud yet. I made a few JIRA issues a short while ago that may help. I've seen people use it with SolrCloud in the past though - and it wasn't so slow…(though I'm sure slower than a single node). Search me... - Mark On Nov 27, 2012, at 1:24 PM, Mikhail Khludnev

Re: Solr 4, optimizing while doing other updates?

So, if I understand your scenario correctly, you are doing a lot of deletes, but since they are occurring against "cold" data, there is isn't usually much if any query traffic for that old/cold data. In short, it sounds like the reason you are optimizing is to keep the memory footprint from gr

Re: Solr 4, optimizing while doing other updates?

On 11/27/2012 11:25 AM, Shawn Heisey wrote: To see whether my heap is too small, I connected jconsole remotely to a 3.5.0 server via JMX. The numbers look OK to me, I'm including a link to a jconsole screenshot. I could probably drop the heap lower, but that might cause some issues with DIH fu

RE: DIH nested entities don't work

Hi, I got things working for 7 views and it looks good! Next things is getting all the views in the index and getting my schema.xml in order. I'll let you know how it goes! Thanks!!!, Maarten -- View this message in context: http://lucene.472066.n3.nabble.com/DIH-nested-entities-don-t-work-t

Re: Solr 4, optimizing while doing other updates?

On 11/27/2012 10:36 AM, Jack Krupansky wrote: Okay, if performance isn't the reason for the optimize, what is the reason that you are using? 8GB for Java heap seems low for a 22GB index. How much Java heap seems available when the app is running? Are these three separate Solr instances/JVMs

Re: SolrCloud Performance - Indexing

It sounds like DataImportHandler will not be really performant with SolrCloud. From what I see it should essentiallly work - it sends doc to the chain, which should distribute them via DistributedUpdateProcessor. But it works synchronously - no multithreading in DIH since 4.0! Does anyone has an ex

Re: Odd casting error in embedded Jetty container

Erik & Alex, If it still sounds like jars to you two, I'll take another whack in that direction. >From previous Google searches I thought I had already eliminated that, but logging every gosh-darn-jar and reviewing the list makes sense. -- Mark Bennett / New Idea Engineering, Inc. / mbenn...@ide

Re: Solr 4, optimizing while doing other updates?

Okay, if performance isn't the reason for the optimize, what is the reason that you are using? 8GB for Java heap seems low for a 22GB index. How much Java heap seems available when the app is running? Are these three separate Solr instances/JVMs on the same machine? How many cores for the ma

Re: Inconsistent search results.

: After further analysis it was found that the cases in which the search : works as expected are where the "." is preceded by a number. Whenever, we : have an alphabet instead of number, the search on the word on right side : doesn't return results. Please note Jack's previous suggestion... >> T

Re: SolrCloud and exernal file fields

Martin, It's still not clear to me whether you solve the problem completely or partially: Does reducing number of cores free some resources for searching during commit? Does the commiting one-by-one core prevents the "freeze"? Thanks On Thu, Nov 22, 2012 at 4:31 PM, Martin Koch wrote: > Mikha

Re: SolrCloud: trying to use eDismax instead of MoreLikeThis.

Erratum: at the end of the previous mail I meant that Edismax *isn´t *as quick as desired, :-) Regards, - Luis Cappa. 2012/11/27 Luis Cappa Banda > - Edismax is as quick as desired. I finally I bet for that solution, I > would like to improve performance. Again I´m opened to hear any suggesti

Re: positions and qf parameter in (e)dismax

That is exactly the exception I would expect to see given your scenario. The fact that the situation may have been silently ignored in the past was surely an oversight that has since been corrected. My proposal would be that if the query parser (actually the code that generates PhraseQuery) en

SolrCloud: trying to use eDismax instead of MoreLikeThis.

Hello, guys. How are you? MoreLikeThis doesn´t support Distributed searchs, so I think that I could emulate a similar behaviour with eDismax features. What I´ll like to do is: *1. *I have two fields that contains important information from the document. This two fields will be used to relate cont

Re: Solr 4, optimizing while doing other updates?

On 11/27/2012 8:19 AM, Jack Krupansky wrote: From your experience with your application, how big is the delta for query time before and after a typical weekly optimize? 50%? 20%? 2%? We've never tried to measure it. If I were chasing better performance, I would be interested, but that's not t

Re: SolrCloud Performance - Indexing

To get the best speed out of SolrCloud you have to index from many clients (or threads). Even better is if you index to many nodes rather than one. Using a single thread against a single instance with replicas will be a fair amount slower with cloud than if you just used one node. - Mark On No

Delete documents in SOLR 1.4.1

Question: We have Solr 1.4.1 in Production currently. We need to delete documents identified by ids on daily basis from solr. --Following deletes by keys and delta data load works-- -- snippet of db-data-config.xml -- snippet of solrconfig.xml

Re: SolrCloud(5x) - Errors while recovering

Someone else has been seeing this on 5x as well - their must be a bug in the new file handling code (which is why it's still baking in 5x and not on 4x yet). I tried to trigger it in tests a while back, but had no look in the brief time I had. I'll try some manual tests when I get chance, as wel

RE: DIH nested entities don't work

The line numbers aren't matching up, but best I can tell, it looks like its probably getting a NPE on this line: DIHCacheTypes type = types[pkColumnIndex]; If this is the correct line, it seems the "types" array is NULL. The "types" array is populated from a properties file that gets written w

Re: Solr 4, optimizing while doing other updates?

From your experience with your application, how big is the delta for query time before and after a typical weekly optimize? 50%? 20%? 2%? -- Jack Krupansky -Original Message- From: Shawn Heisey Sent: Tuesday, November 27, 2012 9:47 AM To: solr-user@lucene.apache.org Subject: Re: Solr

Re: Using DateMath in Facet Label

Have a look at http://lucene.472066.n3.nabble.com/Tips-for-staggered-date-facets-i-e-last-24-hours-last-week-last-month-last-year-ala-google-news-td2066222.html On Tue, Nov 27, 2012 at 9:16 AM, ksmith wrote: > Hi, > > i have a requirement to display solr facet like > > Today, > Last Week, > L

Re: Solr 4, optimizing while doing other updates?

On 11/27/2012 5:46 AM, Erick Erickson wrote: To see how much of an issue it is, look at the admin>>statistics page. The delta between numDocs and maxDocs is the number of non-expunged deletes in your index. That may ease your temptation to, as Walter says, turn that knob.. I wrote a status serv

Re: Odd casting error in embedded Jetty container

I find it is best to troubleshoot path problems backwards. Run something that logs every file access (SysInternal's Process Monitor on Windows, truss/struss on *nix/Mac systems) and then grep for jars. You will see exactly which jars are loaded from where and also which other jars the same process

Re: Solr : query for multiValued Field

I will get it quickly :) thanks for advice ! Thread can be closed ^^ -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-query-for-multiValued-Field-tp4022600p4022633.html Sent from the Solr - User mailing list archive at Nabble.com.

Re: Suggester with punctuation signs

Hi! Upayavira: Hi I'm using the standard tokenizer right now, and it's working fine, but I was wondering if there is any form I can strip this punctuation marks right in the suggest requestHandler, so no need for index again. I've been doing some tests and increasing the threshold has improved

Re: Change UI using Solritas

First, solritas isn't intended to be a user-facing Solr interface, it's generally considered a prototyping tool in Solr. You're allowing unrestricted access to Solr, so I can enter ./update?stream.body=*:*&commit=true in the url and delete your entire index. Are you sure you want to do this? P

Re: How to post atomic updates using xml

So each and every one of your original fields has stored="true", not just the authorGender field? The only exception can be the dest fields in copyField directives. Best Erick On Tue, Nov 27, 2012 at 6:58 AM, Darniz wrote: > yes here is the snippet from schema.xml file > omitNorms="true"/> >

Re: Inconsistent search results.

Sorry, a correction. The first part doesn't give results. SA8182B.BA0850 --> Will have issues when searching on SA8182 -- no results. searching on BA0850 will give results. SA8182.BA0850 --> No issues will return results for BA0850 and SA8182. Regards, Sohail

RE: SolrCloud(5x) - Errors while recovering

i have that issue with some larger size of indexes only... 12 - 14K docs are working totally okay even after a node dies and then starts again but if index is bigger, somehow i keep getting the lines above - Zeki ama calismiyor... Calissa yapar... -- View this message in context: http://luc

Re: Solr replication

BTW, when I said "I think to say anything intelligent" I meant "I think for _me_ to say anything intelligent". Didn't mean to be snarky Erick On Mon, Nov 26, 2012 at 8:19 AM, Erick Erickson wrote: > "both servers are master and slave in the config file". So that means > they're polling each

Re: Solr 4, optimizing while doing other updates?

Shawn: To see how much of an issue it is, look at the admin>>statistics page. The delta between numDocs and maxDocs is the number of non-expunged deletes in your index. That may ease your temptation to, as Walter says, turn that knob.. Best Erick On Mon, Nov 26, 2012 at 8:18 PM, Walter Underwoo

Re: Odd casting error in embedded Jetty container

Sure looks like you're mixing up jars from old and new Solrs somehow. You've stripped down the classpath, but what about the solr libraries? All the things that can be defined in directives in solrconfig.xml? Not much help I know, but the best I can think of. Erick On Mon, Nov 26, 2012 at 7:38

Re: Problem with Solr 3.6.1 extracting ODT content using SolrCell's ExtractingRequestHandler

Not an issue that I know of. I expect you've got some obscure problem in your definitions, but I'm guession. Try modifying your schema so the glob pattern maps to a stored field, something like: remove all other fields except id, remove your mapping, and try it again. If you query with fl=* you sh

Re: Copying few field using copyField to non multiValued field

It Depends (tm). In this case on what you mean by "exact match". In Solr land, that has a precise meaning of a single token. So in your example it wouldn't match since you have two tokens that happen to be next to each other, not a single token"billy corgan". But if you mean that you want a match

Re: Search differences between solr 1.4.0 and 3.6.1

Using the definition you provided, I don't get the same output. Are you sure you are doing what you think? The generateNumberParts=0 keeps the '12' from making it through the filter in 1.4 and 3.6 so I suspect you're not quite doing something the same way in both. Perhaps looking at index tokeniza

Re: Inconsistent search results.

Hi, After further analysis it was found that the cases in which the search works as expected are where the "." is preceded by a number. Whenever, we have an alphabet instead of number, the search on the word on right side doesn't return results. SA8182B.BA0850 --> Will have issues when searching

Re: Using DateMath in Facet Label

Hi, i have a requirement to display solr facet like Today, Last Week, Last Month, so can you please help me how to display on frontend side like Today(5) Last Week (12) Last Month (3) i m using velocity on frontend side. i have confustion about facet field types and its value. how to manage i

Re: Using DateMath in Facet Label

Hi, i have a requirement to display solr facet like Today, Last Week, Last Month, so can you please help me how to display on frontend side like Today(5) Last Week (12) Last Month (3) i m using velocity on frontend side. i have confustion about facet field types and its value. how t

Re: error while doing partial update using curl

Thanks for the reply i checked the solr version and we do use exact version 4.0.0.2012.10.06.03.04.33 and also i have posted my solconfig.xml file snippet, and your assumption was correct, it did used XmlUpdateRequestHandler. Do you know whats the correct way to provide xml in order to do par

Re: SolrCloud and MoreLikeThis: SOLR-788

I´ve recently patched Solr 4.0 tag but it exists a conflict with PURPOSE_GET_MLT_RESULTS final static attributed in ShardRequest.java. The thing is that it doesn´t exist, :-/ Any tips? Regards, - Luis Cappa. 2012/11/27 Luis Cappa Banda > Hello, guys. > > Does anyone tested the latest > SOL

SolrCloud and MoreLikeThis: SOLR-788

Hello, guys. Does anyone tested the latest SOLR-788 MLT patch with Solr 4.0? It´s a very important Sorl feature that I would like to have in a SolrCloud environment. I use Eclipse as IDE, and I´m familiarized with Maven so it looks like I´ll spend o

Re: How to post atomic updates using xml

Are all your fields marked as "stored" in your schema? This is a requirement for atomic updates. /Martin Koch On Mon, Nov 26, 2012 at 7:58 PM, Darniz wrote: > i tried using the same logic to update a specific field and to my surprise > all my other fields were lost. i had a doc with almost 50

RE: SolrCloud(5x) - Errors while recovering

It only seems to happen if a node dies while indexing. -Original message- > From:deniz > Sent: Tue 27-Nov-2012 10:34 > To: solr-user@lucene.apache.org > Subject: RE: SolrCloud(5x) - Errors while recovering > > another update > > having 300K docs causes the same error... > > I think th

RE: SolrCloud(5x) - Errors while recovering

another update having 300K docs causes the same error... I think there is something going on with the size of the index stored... after some point replication fails... any ideas how to bypass this? - Zeki ama calismiyor... Calissa yapar... -- View this message in context: http://lucene.

RE: positions and qf parameter in (e)dismax

Hi - no we're not getting any errors because we enabled positions on all fields that are also listed in the qf-parameter. If we don't, and send a phrase query we would get an error such as: java.lang.IllegalStateException: field "h1" was indexed without position data; cannot run PhraseQuery (te

RE: DIH nested entities don't work

Hi James, I was out for a week hence the late response. I did as you suggested and i can import several views in parallel. However running the data-import which does the join gives the following errors: 27-nov-2012 10:11:24 org.apache.solr.common.SolrException log SEVERE: Exception while processi

RE: SolrCloud(5x) - Errors while recovering

Markus Jelsma-2 wrote > Seems you got this issue: > https://issues.apache.org/jira/browse/SOLR-4032 > thank you for the heads up and a surprising thing about my error.. when i use smaller size of documents, i do not get any errors at all... I dont know why but I have just tried to index only

RE: SolrCloud(5x) - Errors while recovering

Seems you got this issue: https://issues.apache.org/jira/browse/SOLR-4032 -Original message- > From:deniz > Sent: Tue 27-Nov-2012 05:04 > To: solr-user@lucene.apache.org > Subject: SolrCloud(5x) - Errors while recovering > > Here is briefly what is happening: > > I have a simple SolrCl