Solr 4.0 IllegalStateException: this writer hit an OutOfMemoryError; cannot commit

Hi folks, my Test-Server with Solr 4.0 from trunk(version 1292064 from late february) throws this exception... auto commit error...:java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot commit at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:26

Re: Strange behaviour with default request handler

Hey, Sorry for the late answer. Thanks for your help, i started with a wildcard after your advice, by adding it automatically when i build the request in my code. But then, i found the n-gram filter, which is much appropriate for my use case until i have no performance issue ;-) Thanks again. C

Re: Solr facet multiple constraint

Ok sorry to not be clear and thanks again for your answers. q=service%3A1+AND+publicationstatus%3ALIVE&sort=publishingdate+desc&fq=%7B%21ex%3Ddt%7D%28%28%28user%3A10%29%29%29&facet.field=%7B%21tag%3Ddt%7Duser&facet=true&facet.mincount=1 Here with this query i want to get all the documents (the el

Re: unique key

In order to support updates (which are treated as deleted + add), the unique key needs to be indexed. Tomás On Tue, Jul 10, 2012 at 6:08 AM, Sachin Aggarwal wrote: > is it possible not to index and but store the unique keyit will make > index size small. i need the unique key to b stored so

Re: unique key

in my use case i m not deleting any doc from solr i m using batch build on data and use solr as filters on data data is very large raw rows are in billions and filtered or searched query are in millions...is there any way to leave unique key from indexing On Tue, Jul 10, 2012 at 3:42 PM, Tomás Fer

Re: Loading custom update request handler on startup

Another approach would be to use an external application executed by a cron or some scheduler that would post the file to solr using the class. org.apache.solr.util.SimplePostTool SimplePostTool postTool = new SimplePostTool(new URL(SOLR_URL)); for (File file : outputDir.listFiles(/* smoe filte

Re: Problems with estrange data appended to body field

Hmmm, this doesn't really make any sense from a Solr perspective. You should never get the stemmed version back in search results, you should get the stored version, which has no analysis applied. Likewise, copyField doesn't copy the analyzed tokens, it copies the raw input to the dest field. Th

Re: SolrCloud replication question

The symlink thing sounds... complicated, but as you say you're going another route The indexing speed you're seeing is surprisingly slow, I'd get to the root of the timeouts before giving up. SolrCloud simply _can't_ be that slow by design, something about your setup is causing that I suspect.

Re: unique key

There are some specific use cases where you can skip having a unique key. See http://wiki.apache.org/solr/UniqueKey However, I would test how much space you save by not having one. On Tue, Jul 10, 2012 at 6:27 AM, Sachin Aggarwal wrote: > in my use case i m not deleting any doc from solr i m usi

Sort by date field = outofmemory?

Dear Solr Users, Each time I try to do a request with &sort=pubdate+desc I get: GRAVE: java.lang.OutOfMemoryError: Java heap space I use Solr3.6, I have around 80M docs and my request gets around 160 results. Actually for my test, i use jetty java -jar -Xmx2g start.jar PS: If I write 3

Re: Sort by date field = outofmemory?

To complete my question: after having this error, some fields (not all) aren't reachable with the same error. Le 10/07/2012 14:25, Bruno Mannina a écrit : Dear Solr Users, Each time I try to do a request with &sort=pubdate+desc I get: GRAVE: java.lang.OutOfMemoryError: Java heap space

Re: Loading custom update request handler on startup

Couldn't you do this in the init() method? You might have to kick off your background thread, but lazy initialize stuff when it fires to initialize other things that require Solr be fully ready. Note that DataImportHandler can index Solr XML files with very little configuration, and cronning

Re: PathHierarchyTokenizerFactory behavior

Modifying the field definition to solves the purpose . got it from the link http://stackoverflow.com/questions/6920506/solr-pathhierarchytokenizerfactory-facet-query -- View this message in context: http://lucene

maxNumberOfBackups does not cleanup - jira 3361

environment: solr 3.5 hello all, i have a question on this jira - https://issues.apache.org/jira/browse/SOLR-3361 the jira states that, "with "backupAfter"=commit, the backups do not get cleaned up" however - we are noticing this same issue in our environment, when using optimize. can someone

Re: Solr 4.0 IllegalStateException: this writer hit an OutOfMemoryError; cannot commit

On Tue, Jul 10, 2012 at 3:11 AM, Vadim Kisselmann wrote: > Hi folks, > my Test-Server with Solr 4.0 from trunk(version 1292064 from late > february) throws this exception... Can you run Lucene's checkIndex tool on your index? If that is clean, can you try a newer version? This could be a number

unsubsribe

-Original Message- From: Sachin Aggarwal [mailto:different.sac...@gmail.com] Sent: Tuesday, July 10, 2012 6:08 AM To: solr-user@lucene.apache.org Subject: unique key is it possible not to index and but store the unique keyit will make index size small. i need the unique key to b sto

Re: Solr 4.0 IllegalStateException: this writer hit an OutOfMemoryError; cannot commit

Hi Robert, > Can you run Lucene's checkIndex tool on your index? No, unfortunately not. This Solr should run without stoppage, an tomcat-restart is ok, but not more:) I tested newer trunk-versions a couple of months ago, but they fail all with tomcat. i would test 4.0-alpha in next days with tomc

RE: maxNumberOfBackups does not cleanup - jira 3361

I'm also certain that it would apply to both "oncommit" and "onoptimize". James Dyer E-Commerce Systems Ingram Content Group (615) 213-4311 -Original Message- From: geeky2 [mailto:gee...@hotmail.com] Sent: Tuesday, July 10, 2012 8:48 AM To: solr-user@lucene.apache.org Subject: maxNumber

RE: maxNumberOfBackups does not cleanup - jira 3361

thank you James - that is good to know. for the short-term we'll just use cron and kill backup directories that are older than x. for the long-term, we'll just migrate to 4.0 thanks again -- View this message in context: http://lucene.472066.n3.nabble.com/maxNumberOfBackups-does-not-cleanup-j

solr-user@lucene.apache.org

I don't have any experience with DIH: maybe XPathEntityProcessor doesn't use a true XML parser? You might want to try passing your documents through "xmllint -noent" (basically parse and reserialize) - that should inline the characters as UTF-8? On 07/09/2012 03:18 PM, Michael Belenki wrote:

Re: Searching for sentences containing a list of words with a configurable number of words not in the list inbetween?

Welcome! Two points: - did you choose right maillist? (let me reply to another one) - have you checked http://lucene.apache.org/core/3_6_0/queryparsersyntax.html#Proximity%20Searches? - the same in Lucene Queries api is http://lucene.apache.org/core/3_6_0/api/core/org/apache/lucene/search/PhraseQu

Re: Solr 4.0 IllegalStateException: this writer hit an OutOfMemoryError; cannot commit

it really seems that you are hitting an OOM during auto warming. can this be the case for your failure. Can you raise the JVM memory and see if you still hit the spike and go OOM? this is very unlikely a IndexWriter problem. I'd rather look at your warmup queries ie. fieldcache, FieldValueCache usa

Re: Check if input xml file is ok?

: Sometimes indexation bugs because there is a mistake in XML structure (like a : ]]> inside CDATA field) : : Is exist a "test.jar" somewhere before running the post.jar? You should be able to spot errors like that using any XML valdation tool ... perhaps "xmllint" is available on your system?

Re: unique key

No, a unique key needs to be indexed. You can delete documents by query (to avoid duplication), but you can't query on any field that is not indexed, so I guess you'll need it. On Tue, Jul 10, 2012 at 12:23 PM, Sachin Aggarwal < different.sac...@gmail.com> wrote: > today i experimented some param

Re: Synonyms and hyphens

Which version of Solr are you using? : Terms with embedded special characters are treated as phrases with spaces in : place of the special characters. So, "gb-mb" is treated as if you had enclosed : the term in quotes. take a look at "autoGeneratePhraseQueries" option on your field type ... dep

Re: Check if input xml file is ok?

Hi Chris, Thanks for this info ! Le 10/07/2012 22:31, Chris Hostetter a écrit : : Sometimes indexation bugs because there is a mistake in XML structure (like a : ]]> inside CDATA field) : : Is exist a "test.jar" somewhere before running the post.jar? You should be able to spot errors like that

solr-user@lucene.apache.org

: Somebody any idea? Solr seems to ignore the DTD definition and therefore : does not understand the entities like ü or ä that are defined in : dtd. Is it the problem? If yes how can I tell SOLR to consider the DTD : definition? Solr is just utilizing the builtin java XML parser for this, so ther

wrong results for phrase synonym

hi i m using edismax, synonym expansion=true, synonym processing index time only. Solr is from trunk synonyms.txt --- setup, set up backup, back up Problem is when we search for set up we get documents which have backup and back up along with docs that match setup and set up. We are

Re: wrong results for phrase synonym

Try entering these terms in the Solr admin analyzer UI page as well as adding &debugQuery=true to your queries and you will likely see what the issues are. A document containing setup will be indexed as if it was set and up as two separate terms. Also, there was a bug with mm in some release

Re: Sort by date field = outofmemory?

Sorting is a memory-intensive operation indeed. Not sure what you are asking, but it may very well be that your only option is to give JVM more memory. On 7/10/2012 8:25 AM, Bruno Mannina wrote: > Dear Solr Users, > > Each time I try to do a request with &sort=pubdate+desc > > I get: > GRAVE

Problem with Solr not finding a class that is in lucene-analyzers.jar

I have been running Solr with Tomcat, and I recently wrote a Quartz program that starts and stops Tomcat, starts Solr indexing jobs, and does a few other things. When I start Tomcat programmatically in this way, Solr starts initializing, and when it hits the text_ws field type in schema.xml, it

Re: Re:shard connection timeout

Hi Hans yes, that remote server is ok. actually we got this error when remote server is executing garbage collecting and that time is over about 1 minute. remote server is very busy and memory usage is high. -- View this message in context: http://lucene.472066.n3.nabble.com/Re-shard-connection-t

Query by Example

Hi, Is there a way to search by providing an example document to solr? We would set the fields of the example document to desired filter and ask Solr to bring all documents that match this example document. For example, if our document contains itemnumber, title, sku, price etc. Is it possible to

Re: Query by Example

On 11 July 2012 10:40, Sohail Aboobaker wrote: > Hi, > > Is there a way to search by providing an example document to solr? We would > set the fields of the example document to desired filter and ask Solr to > bring all documents that match this example document. For example, if our > document con

Re: Query by Example

Thanks, I will look into more like this. To answer your question, we were discussing interface for a central search service that all other components can call. We have to provide a set of parameters as interface which clients can set and ask for results. This would mean that name of each parameter

Re: Query by Example

On 11 July 2012 10:52, Sohail Aboobaker wrote: > Thanks, > > I will look into more like this. To answer your question, we were > discussing interface for a central search service that all other components > can call. We have to provide a set of parameters as interface which clients > can set and a

Re: shard connection timeout

Actually we got this error when remote server is executing garbage collecting and that time is over about 1 minute. Solr server sometimes is frozen during gc and occurred connection refused error. Our gc option is -XX:+UseParallelGC -XX:+UseParallelOldGC -XX:+AggressiveOpts Response waiting is bet

Re: Query by Example

Thanks. That is what we concluded i.e. to write a wrapper method within our service to build the query for Solr by examining the example bean. Thanks again.