Note that stored=true/false is irrelevant to the raw search time. What it _is_ relevant to is the time it takes to assemble the doc for return, if (and only if) you return that field. I claim your search time would be fast if you went ahead and stored the field, and specified an fl clause that did NOT contain the big field. Oh, and you'd have to have lazy field loading enabled too.....
FWIW, Erick On Wed, Jun 5, 2013 at 10:29 AM, Raheel Hasan <raheelhasan....@gmail.com> wrote: > some values in the field are up to a 1M as well > > > On Wed, Jun 5, 2013 at 7:27 PM, Raheel Hasan <raheelhasan....@gmail.com>wrote: > >> ok thanks for the reply.... The field having values like 60kb each.... >> >> Furthermore, I have realized that the issue is with MySQL as its not >> processing this table when a "where" is applied.... >> >> Secondly, I have turned this field to "*stored=false*" and now the "* >> select/*" is fast working again.... >> >> >> >> On Wed, Jun 5, 2013 at 6:56 PM, Shawn Heisey <s...@elyograg.org> wrote: >> >>> On 6/5/2013 3:08 AM, Raheel Hasan wrote: >>> > Hi, >>> > >>> > I am trying to index a heavy dataset with 1 particular field really too >>> > heavy... >>> > >>> > However, As I start, I get Memory warning and rollback >>> (OutOfMemoryError). >>> > So, I have learned that we can use -Xmx1024m option with java command to >>> > start the solr and allocate more memory to the heap. >>> > >>> > My question is, that since this could also become insufficient later, >>> so it >>> > the issue related to cacheing? >>> > >>> > here is my cache block in solrconfig: >>> > >>> > <filterCache class="solr.FastLRUCache" >>> > size="512" >>> > initialSize="512" >>> > autowarmCount="0"/> >>> > >>> > <queryResultCache class="solr.LRUCache" >>> > size="512" >>> > initialSize="512" >>> > autowarmCount="0"/> >>> > >>> > <documentCache class="solr.LRUCache" >>> > size="512" >>> > initialSize="512" >>> > autowarmCount="0"/> >>> > >>> > I am thinking like maybe I need to turn of the cache for >>> "documentClass". >>> > Anyone got a better idea? Or perhaps there is another issue here? >>> >>> Exactly how big is this field? Do you need this giant field returned >>> with your results, or is it just there for searching? >>> >>> Caches of size 512, especially with autowarm disabled, are probably not >>> a major cause for concern, unless the big field is big enough so that >>> 512 of them is really really huge. If that's the case, I would reduce >>> the size of your documentCache, not turn it off. >>> >>> The value of ramBufferSizeMB elsewhere in your config is more likely to >>> affect how much RAM gets used during indexing. The default for this >>> field as of Solr 4.1.0 is 100. Most people can reduce this value. >>> >>> I'm writing a reply to another thread where you are participating, with >>> info that will likely be useful for you too. Look for that. >>> >>> Thanks, >>> Shawn >>> >>> >> >> >> -- >> Regards, >> Raheel Hasan >> > > > > -- > Regards, > Raheel Hasan
