Re: Too many documents Exception

2014-05-15 Thread yamazaki
Tanks, Jack. Is there a way to suppress setting this exception? For example, 2147483647 ? When this exception occurs, Index will not be read. If solrcloud is used, some data not read. shard1 documents 2^31-1 over shard2 documents 2^31-1 not over shard1 down. shard1 index is dead. -- yamazak

Re: Too many documents Exception

2014-05-13 Thread Jack Krupansky
Sent: Wednesday, May 7, 2014 8:15 PM To: solr-user@lucene.apache.org Subject: Re: Too many documents Exception Tanks, Jack. Is there a way to suppress setting this exception? For example, 2147483647 ? When this exception occurs, Index will not be read. If solrcloud is used, some data not

Re: Too many documents Exception

2014-05-13 Thread Chris Hostetter
As noted by others: you should definitely look into sharding your index -- fundementally there is no way to have that many documents in a single Lucene index. However: this is a terrible error for you to get, something in the stack should have really given you an error when you tried to add th

Re: Too many documents Exception

2014-05-12 Thread Greg Walters
Looks like you've hit an internal limitation of Lucene, see http://lucene.apache.org/core/3_0_3/fileformats.html#Limitations: When referring to term numbers, Lucene's current implementation uses a Java int to hold the term index, which means the maximum number of unique terms in any sin

Re: Too many documents Exception

2014-05-11 Thread Erick Erickson
You'll need to shard your index. This assumes that your index doesn't have a huge percentage of deleted docs. This latter shouldn't be happening since as you delete docs, the number of deleted docs should drop as segments are merged. I suspect you aren't searching on this index yet, because I'm pr

Re: Too many documents Exception

2014-05-07 Thread Jack Krupansky
Lucene only supports 2^31-1 documents in an index, so Solr can only support 2^31-1 documents in a single shard. I think it's a bug that Lucene doesn't throw an exception when more than that number of documents have been inserted. Instead, you get this error when Solr tries to read such an over