Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Christopher Gross
Thanks Hoss -- adding in the LengthFilterFactory did the trick. -- Chris On Mon, Sep 15, 2014 at 1:57 PM, Bryan Bende wrote: > I ran into this problem as well when upgrading to Solr 4.8.1... > > We had a somewhat large binary field that was "indexex=false stored=true", > but because of the copy

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Bryan Bende
I ran into this problem as well when upgrading to Solr 4.8.1... We had a somewhat large binary field that was "indexex=false stored=true", but because of the copyField copying "*" to "text" it would hit the immense term issue. In our case we didn't need this field to be indexed (parts of it were

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Chris Hostetter
: SCHEMA: : : : LOGS: : Caused by: java.lang.IllegalArgumentException: Document contains at least : one immense term in field="content" (whose UTF8 encoding is longer than the I don't think you are using the schema.xml you think you are ... that exception is *very* specific to the *INDEXED* te

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Alexandre Rafalovitch
Solr is for search. Storing fields is to make retrieval easier. When you hit an edge case, you need to step back and reconsider the price you are paying for that "easier" bit. It can play at being a "NoSQL Database", but it is not the primary use case and its behaviour at the edge-cases is not opti

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Christopher Gross
[sorry if this double posts -- I got an error on sending so I'm trying it again..] I'm storing the page content in a "string" in Solr -- for display later. I'm indexing that content into a text field (text_en_splitting) for full-text searching. I'm getting an error on the "string" portion, but pe

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Jack Krupansky
full wiki page as a string field. -- Jack Krupansky -Original Message- From: Alexandre Rafalovitch Sent: Monday, September 15, 2014 8:39 AM To: solr-user Subject: Re: Solr Exceptions -- "immense terms" May not need a script for that: http://www.solr-start.com/javadoc/solr-lu

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Alexandre Rafalovitch
ims large input values. > > -- Jack Krupansky > > -Original Message- From: Christopher Gross > Sent: Monday, September 15, 2014 7:58 AM > To: solr-user > Subject: Re: Solr Exceptions -- "immense terms" > > > Yeah -- for this part I'm just trying t

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Jack Krupansky
To: solr-user Subject: Re: Solr Exceptions -- "immense terms" Yeah -- for this part I'm just trying to store it to show it later. There was a change in Lucene 4.8.x. Before then, the exception was just being eaten...now they throw it up and don't index that document. Can

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Christopher Gross
Yeah -- for this part I'm just trying to store it to show it later. There was a change in Lucene 4.8.x. Before then, the exception was just being eaten...now they throw it up and don't index that document. Can't push the whole schema up -- but I do copy the content field into a "text" field (tex

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Michael Della Bitta
I just came back to this because I figured out you're trying to just store this text. Now I'm baffled. How big is it? :) Not sure why an analyzer is running if you're just storing the content. Maybe you should post your whole schema.xml... there could be a copyfield that's dumping the text into a

Re: Solr Exceptions -- "immense terms"

2014-09-15 Thread Michael Della Bitta
If you're using a String fieldtype, you're not indexing it so much as dumping the whole content blob in there as a single term for exact matching. You probably want to look at one of the text field types for textural content. That doesn't explain the difference in behavior between Solr versions,