Re: Solr Deployment Question

Please explain how you have handled two indexes in a single VM. Is it multi core? To identify memory consumption, You need to calculate usedmemory before and after loading the indexes, basically calculate usedmemory before and after any check point you want to analyse. Their difference will give y

RE: Solr Deployment Question

But even we used a single index, we were running out of memory. What do you mean by "active"? No queries on the masters. Only one index is being processed/optimized. Also, if I may add to my same question, how can I find the amount of memory that an index would use, theoretically? i.e.: Is there

Re: Solr Deployment Question

You may use one index at a time, but both indexes are active and loaded all its terms in memory. Memory consumption will be certainly more. Regards Aditya http://www.findbestopensource.com On Fri, May 14, 2010 at 10:28 AM, Maduranga Kannangara < mkannang...@infomedia.com.au> wrote: > Hi > > We u

Solr Deployment Question

Hi We use separate JVMs to Index and Query. (Client applications will query only slaves, while master does only indexing) Recently we moved a two master indexes to a single JVM. Our memory allocation was for each index was 512Mb and 1Gb. Once we moved both indexes to a single VM, we thought it w

Re: Bitwise Operations on Integer Fields in Lucene and Solr Index

Correction, I meant to list https://issues.apache.org/jira/browse/LUCENE-2460 https://issues.apache.org/jira/browse/SOLR-1913 On Thu, May 13, 2010 at 10:13 PM, Israel Ekpo wrote: > I have created two ISSUES as new features > > https://issues.apache.org/jira/browse/LUCENE-1560 > > https://iss

Re: Best way to handle bitfields in solr...

William, This QParserPlugin should solve that problem now. Check out https://issues.apache.org/jira/browse/SOLR-1913 BitwiseQueryParserPlugin is a org.apache.solr.search.QParserPlugin that allows users to filter the documents returned from a query by performing bitwise operations between a parti

Re: Bitwise Operations on Integer Fields in Lucene and Solr Index

I have created two ISSUES as new features https://issues.apache.org/jira/browse/LUCENE-1560 https://issues.apache.org/jira/browse/SOLR-1913 The first one is for the Lucene Filter. The second one is for the Solr QParserPlugin The source code and jar files are attached and the Solr plugin is ava

Re: Long Lucene queries

No changes are needed. Just experiment with 'curl'. On Tue, May 11, 2010 at 11:52 PM, Pooja Verlani wrote: > Hi, > Thanks Eric.. > The search parameter length is a lot to be done in GET, I am thinking of > opting for POST, is it possible to do POST request to solr. Any > configuration changes or

Re: NPE When trying to commit

Also, The strange thing is that I still get this exception when i try to swap in a snapshot I have of the index from a day or two.. are these index commit points saved in some external place or so? Very strange.. -- View this message in context: http://lucene.472066.n3.nabble.com/NPE-When-trying

DIH settings

Can you please share with me your DIH settings and JDBC driver you are using. I'll start... jdbc driver = mysql-connector-java-5.1.12-bin batchSize = "-1" readOnly = "true" Would someone mind explaining what "convertType" and "transactionIsolation" actually does? The wiki doesnt really explain

Seattle Hadoop/NoSQL: Facebook, more Discussion. Thurs May 27th

We've heard your feedback from the last meetup: we're having less speakers and more discussion. Yay! http://www.meetup.com/Seattle-Hadoop-HBase-NoSQL-Meetup/ We're expecting: 1. Facebook will talk about Hive (a SQL-like language for MapReduce) 2. OpsCode will talk about cluster management with Ch

Re: SolrUser - ERROR:SCHEMA-INDEX-MISMATCH

This is probably a bad idea. You're getting by on backwards compatibility stuff, I'd really recommend that you reindex your entire corpus, possibly getting by on what you already have until you can successfully reindex. Have a look at trie fields (this is detailed in the example schema.xml). Here'

Re: SolrUser - Reindex

In general, it's hard to just answer since there are many factors to consider, not the least of which is what you want it to do. In this case, I suspect the issue is WordDelimiterFactory, it splits words on all non alphanumerics by default. It would probably be a good idea to work with the various

Bitwise Operations on Integer Fields in Lucene and Solr Index

Hello Lucene and Solr Community I have a custom org.apache.lucene.search.Filter that I would like to contribute to the Lucene and Solr projects. So I would need some direction as to how to create and ISSUE or submit a patch. It looks like there have been changes to the way this is done since the

Re: SolrUser - Reindex

I'm using the textgen fieldtype on my field as follow: . . They no remove the @ symbol. To configure to index the @ symbol i must use HTMLStripStandardTokenizerFactory ?

Re: SolrUser - ERROR:SCHEMA-INDEX-MISMATCH

Hi Erick. I put in my schema.xml fields with type string. The system go to te production, and now i see that the field must be a long field. When i change the fieldtype to long, show the error ERROR:SCHEMA-INDEX-MISMATCH when i search by solr admin. I Put "plong", and this works. This is the way

Re: bi-directional replication on solr 1.4?

It looks like SnapPuller.java doesn't allow for the possibility of the slave having a later index version than the master. It only checks whether the versions are equal. It's easy enough to add that check and prevent the index fetch when the slave has a later version (in fact I'm running it in a s

Re: SolrUser - Reindex

Probably your analyzer is removing the @ symbol, it's hard to say if you don't include the relevant parts of your schema. This page might help: http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters Best Erick On Thu, May 13, 2010

Re: SolrUser - ERROR:SCHEMA-INDEX-MISMATCH

Not at present, you must re-index your documents when you redefine your schema to change existing documents. Field updating of documents already indexed is being worked on, but it's not available yet. Best Erick On Thu, May 13, 2010 at 3:58 PM, Anderson vasconcelos < anderson.v...@gmail.com> wro

Re: Help with Embedded Server - SOLVED

Thanks for all the help Lance. I was finally able to get it working by using a more complex initialization process: [code lang="groovy"] def solrHome = ConfigurationHolder.config.universitySearchService?.solrHome ?: "" def coreName = ConfigurationHolder.config.universitySearchService?.solrCore ?:

SolrUser - Reindex

Why solr/lucene no index the Character '@' ? I send to index email fields x...@gmail.com ...and after try do search to_email:*...@*, and not found. I need to do some configuration? Thanks

SolrUser - ERROR:SCHEMA-INDEX-MISMATCH

Hi All. I have the follow fields in my schema: I need to change the index of SOLR, adding a dynamic field that will contains all values of "value" field. Its possible to get all index data and reindex, putting the values on my dynamic field? How the data was no stored, i don't find one wa

Re: Field Collapsing: How to estimate total number of hits

Finally I get it working. It seems that latest SOLR-236-trunk.patch just have some bugs. I checked out an older revision of solr trunk - rev 899572 (dtd. 2010-01-15) from http://svn.apache.org/repos/asf/lucene/solr/trunk and applied SOLR-236.patch dtd. 2010-02-01. And collapsing works fine. I ge

Re: [resolved] Config issue for deduplication

Got it with the help of Demian Katz, main developper of Vufind: The import script of Vufind was bypassing the duplication parameters while writing directly to the SOLR-Index. By deactivitating direct writing to the index and using the standard way it now works! Thanks to all who gave input!

Re: multi-valued associated fields

Hi Ahmed Thanks again for sharing your insight and experience. I will discuss the multi-core approach with members of our team. Regards Eric On Wed, May 12, 2010 at 9:24 PM, ahammad wrote: > > In our deployment, we thought that complications might arise when > attempting > to hit the Solr serv

Re: Config issue for deduplication

I use true and a different field than ID to control duplication. This is about bibliographic data coming from different sources with different IDs which may have the same content... I attached solrconfig.xml if you want to take a look. Thanks a lot! Markus Markus Jelsma schrieb: What's yo

bi-directional replication on solr 1.4?

Does bi-directional replication work in solr 1.4? In other words, if I wanted to have 2 servers that are both master and slave. Call them A and B. I would configure things so that normally, A runs a DIH periodically to rebuild the index, and then B pulls the updated index from A. The idea here is t

Re: Advancded Reading

A truly indispensable resource is Yonik's Mastering Solr 1.4 on-demand webinar: http://www.lucidimagination.com/solutions/Webinars/mastering-solr-1.4-with-yonik-seeley On Thu, May 13, 2010 at 6:04 PM, Blargy wrote: > > Does anyone know of any documentation that is more in-depth that the wik

maximum recommended document cache size

I am trying to tune my Solr setup so that the caches are well warmed after the index is updated. My documents are quite small, usually under 10k. I currently have a document cache size of about 15,000, and am warming up 5,000 with a query after each indexing. Autocommit is set at 30 seconds, and

Re: synonyms not working with copyfield

Hi, You could use a copyField against all fields and then AND the query terms given. Quite restrictive but all terms would then have to be present to match. I'm still a relative newbie to Solr so perhaps I'm horribly wrong. Cheers Nick On 13 May 2010, at 18:18, surajit wrote: > > Understood

Re: synonyms not working with copyfield

Understood and I can work with that limitation by using separate fields during indexing. However, my search interface is just a text box like Google and I need to take the query and return only those documents that match ALL terms in the query and if I am going to take the query and match it again

Advancded Reading

Does anyone know of any documentation that is more in-depth that the wiki and the Solr 1.4 book? I'm passed the basic usage of Solr and creating simple support plugins. I really want to know all about the inner workings of Solr and Lucene. Can someone recommend anything? Thanks -- View this mess

Re: grouping in fq

: >> (+category:xyz +price:[100 TO *]) -category:xyz : : this one doesn't seem to work (I'm not using a price field, but a text field : -- using price field here just for example). it never will, it's saying only things that are in category xyz and above 100 dollars can match, but anything in c

Re: synonyms not working with copyfield

: which is good, but the different fields that I copy into the copyfield need : different analysis and I no longer am able to do that. I can, of course, Fundementally, Solr can only apply a single analysis chain to all of the text in a given field -- regardless of where it may be copied from. i

Re: synonyms not working with copyfield

take a look at the DismaxRequestHandler: http://wiki.apache.org/solr/DisMaxRequestHandler -Original Message- From: surajit To: solr-user@lucene.apache.org Sent: Thu, May 13, 2010 9:52 pm Subject: Re: synonyms not working with copyfield Thanks much! I added a synonym filter

RE: confused by simple OR

Awesome that works, thanks Ahmet. -Kallin Nagelberg -Original Message- From: Ahmet Arslan [mailto:iori...@yahoo.com] Sent: Thursday, May 13, 2010 12:24 PM To: solr-user@lucene.apache.org Subject: Re: confused by simple OR > I must be missing something very > obvious here. I have a fil

Re: confused by simple OR

> I must be missing something very > obvious here. I have a filter query like so: > > (-rootdir:somevalue) > > I get results for that filter > > However, when I OR it with another term like so I get > nothing: > > ((-rootdir:somevalue) OR (rootdir:somevalue AND > someboolean:true)) > Simply

Re: synonyms not working with copyfield

Thanks much! I added a synonym filter to the copyfield and it started working which is good, but the different fields that I copy into the copyfield need different analysis and I no longer am able to do that. I can, of course, search against the individual fields instead of the copyfield, but I wa

confused by simple OR

I must be missing something very obvious here. I have a filter query like so: (-rootdir:somevalue) I get results for that filter However, when I OR it with another term like so I get nothing: ((-rootdir:somevalue) OR (rootdir:somevalue AND someboolean:true)) How is this possible? Have I gone m

RE: Config issue for deduplication

What's your solrconfig? No deduplication is overwritesDedupes = false and signature field is other than doc ID field (unique)   -Original message- From: Markus Fischer Sent: Thu 13-05-2010 17:01 To: solr-user@lucene.apache.org; Subject: Config issue for deduplication I am trying to con

Re: Config issue for deduplication

Hmm, I can't find in solrconfig.xml anything about dataimporthandler for Vufind. So I suppose, no the import function does not use this method. Import is done by a script. Maybe I do not associate dedupe with the correct requestHandler? I placed it directly after So kind of h

Re: Question on pf (Phrase Fields)

I don't know if this solution accomplished your requirements but you can use fq to do the query with only "foo" and q when you search by more terms. Marco Martínez Bautista http://www.paradigmatecnologico.com Avenida de Europa, 26. Ática 5. 3ª Planta 28224 Pozuelo de Alarcón Tel.: 91 352 59 42 2

Re: grouping in fq

>> (+category:xyz +price:[100 TO *]) -category:xyz this one doesn't seem to work (I'm not using a price field, but a text field -- using price field here just for example). Below are some other variations I tried: (+category:xyz +price:[100 TO *]) -category:xyz -- zero results (+category:xyz +pr

Question on pf (Phrase Fields)

Is there any way to configure this so it only takes after if you match more than one word? For example if I search for: "foo" it should have no effect on scoring, but if I search for "foo bar" then it should. Is this possible? Thanks -- View this message in context: http://lucene.472066.n3.nab

Re: Config issue for deduplication

> I am trying to configure automatic > deduplication for SOLR 1.4 in Vufind. I followed: > > http://wiki.apache.org/solr/Deduplication > > Actually nothing happens. All records are being imported > without any deduplication. Does "being imported" means you are using dataimporthandler? If yes you

Config issue for deduplication

I am trying to configure automatic deduplication for SOLR 1.4 in Vufind. I followed: http://wiki.apache.org/solr/Deduplication Actually nothing happens. All records are being imported without any deduplication. What am I missing? Thanks Markus I did: - create a duplicated set of records,

Re: ContentStreamUpdateRequest - out of memory on a large file

On May 12, 2010, at 1:58 PM, Christopher Baird wrote: > We're running into an out of memory problem when sending a large file to our > SOLR server using the ContentStreamUpdateRequest. It appears that this > happens because when the request method of CommonsHttpSolrServer is called > (this is ca

Re: Field Collapsing: How to estimate total number of hits

Joe, thanks for your answer. But it doesn't solve my problem. Below I gave a longer description of my problem. First of all, I checked out solr trunk revision 928303 with last change dtd. 2010-03-28. Then I applied the latest patch from SOLR-236 to get field collapsing component. After that I buil

Re: synonyms not working with copyfield

> I have indexed person names in solr using synonym expansion > and am getting a > match when I explicitly use that field in my query > (name:query). However, > when I copy that field into another field using copyfield > and search on that > field, I don't get a match. Below are excerpts from > sch

RE: Strange behavior for certain words

Hi,        Thanks for your response. Attached are the Schema.xml and sample docs that were indexed. The query and response are as below. The attachment Prodsku4270257.xml has a field "paymenttype" whose value is 'prepaid'. query: q=prepaid&start=0&rows=10&fl=*%2Cscore&qt=standard&wt=json&debugQu

Re: synonyms not working with copyfield

Hi Surajit I aint sure if this is any help, but I had a similar problem but with stop words, they were not working with dismax queries. Well to cut a long story it seems that all the querying fields need to be configured with stopwords. Maybe this has the similar affect with Synonyms confguratio

Re: Too many clauses in lucene query

> I am forming a query to boost a certain ids, the list of > ids can go till > 2000 too. I am sometimes getting the error for too many > clauses in the > boolean query and otherwise i am getting a null page. Can > you suggest any > config changes regarding this. > I am using solr 1.3. For too man