Re: field compression in solr 3.6

2012-05-24 Thread Shawn Heisey
On 5/23/2012 2:48 PM, pramila_tha...@ontla.ola.org wrote: Hi Everyone, solr 3.6 does not seem to be honoring the field compress. While merging the indexes the size of Index is very big. Is there any other way to handle this to keep compression functionality? Compression support was removed

Re: Field compression

2011-04-18 Thread Smiley, David W.
It's probably not accurate to say that a lot of sites were *relying* on that feature. It's an optimization. Getting a working patch applying to trunk is on my TODO-list within the next couple months. https://issues.apache.org/jira/browse/SOLR-752 "Watch" the issue to see when I get to it. ~ Dav

Re: Field Compression

2009-06-09 Thread Michael Ludwig
Fer-Bj schrieb: for all the documents we have a field called "small_body" , which is a 60 chars max text field that were we store the "abstract" for each article. we need to display this small_body we want to compress every time. If this works like compressing individual files, the overhead

Re: Field Compression

2009-06-04 Thread Fer-Bj
Here is what we have: for all the documents we have a field called "small_body" , which is a 60 chars max text field that were we store the "abstract" for each article. We have about 8,000,000 documents indexed, and usually we display this small_body on our "listing pages". For each listing pa

Re: Field Compression

2009-06-04 Thread Grant Ingersoll
On Jun 4, 2009, at 6:42 AM, Erick Erickson wrote: It *will* cause performance issues if you load that field for a large number of documents on a particular search. I know Lucene itself has lazy field loading that helps in this case, but I don't know how to persuade SOLR to use it (it may even

Re: Field Compression

2009-06-04 Thread Erick Erickson
Warning: This is from a Lucene perspective I don't think it matters. I'm pretty sure that COMPRESS onlyapplies to *storing* the data, not putting the tokens in the index (this latter is what's serached)... It *will* cause performance issues if you load that field for a large number of document

Re: Field Compression

2009-06-04 Thread Fer-Bj
Is it correct to assume that using field compression will cause performance issues if we decide to allow search over this field? ie: if I decide to add "compressed=true" to the BODY field... and a I allow search on body... would that be a problem? At the same time: if I add compress

Re: Field Compression

2008-02-03 Thread Mike Klaas
On 3-Feb-08, at 1:34 PM, Stu Hood wrote: I just finished watching this talk about a column-store RDBMS, which has a long section on column compression. Specifically, it talks about the gains from compressing similar data together, and how lazily decompressing data only when it must be proc