Re: SOLR cache tuning

Hi,Thanks for the replies so far. Walter: We have a few more solr cores. So the JVM is sized accordingly. I know we can separate the cores but for easier maintainability we have only one core. Also only one core is being used majority of the times.  Jorn: I dont have a particular performance num

Re: SOLR cache tuning

You should not have other processes/container running on the same node. They potentially screw up your os cache making things slow, eg if the other processes also read files etc they can remove things from Solr from the Os cache and then the os cache needs to be filled again. What performance d

Re: SOLR cache tuning

Reading all the documents is going to be slow. If you want to do that, use a database. You do NOT keep all of the index in heap. Solr doesn’t work like that. Your JVM heap is probably way too big for 2 million documents, but I doubt that is the performance issue. We use an 8 GB heap for all of

RE: Solr Cache clear

Heisey Sent: Tuesday, April 9, 2019 1:52 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cache clear On 4/9/2019 12:38 PM, Lewin Joy (TMNA) wrote: > I just tried to go to the location you have specified. I could not see a > "CACHE" . I can see the "Statistics" section

Re: Solr Cache clear

On 4/9/2019 12:38 PM, Lewin Joy (TMNA) wrote: I just tried to go to the location you have specified. I could not see a "CACHE" . I can see the "Statistics" section. I am using Solr 7.2 on solrcloud mode. If you are trying to select a *collection* from a dropdown, you will not see this. It wi

RE: Solr Cache clear

" section. I am using Solr 7.2 on solrcloud mode. thanks -Lewin -Original Message- From: Shawn Heisey Sent: Tuesday, April 9, 2019 1:01 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cache clear On 4/9/2019 11:51 AM, Lewin Joy (TMNA) wrote: > Hmm. I only tried reloading th

Re: Solr Cache clear

On 4/9/2019 11:51 AM, Lewin Joy (TMNA) wrote: Hmm. I only tried reloading the collection as a whole. Not the core reload. Where do I see the cache sizes after reload? If you do not know how to see the cache sizes, then what information are you looking at which has led you to the conclusion tha

Re: Solr Cache clear

Sent: Monday, April 8, 2019 6:45 PM > To: solr-user > Subject: Re: Solr Cache clear > > You may have warming queries to prepopulate your cache. Check your > solrconfig.xml. > > Regards, >Alex > > On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), wrote: >

RE: Solr Cache clear

Thank you for email, Alex. I have the autowarmCount set as 0. So, this shouldn't prepopulate with old cache data. -Lewin -Original Message- From: Alexandre Rafalovitch Sent: Monday, April 8, 2019 6:45 PM To: solr-user Subject: Re: Solr Cache clear You may have warming queri

RE: Solr Cache clear

Hmm. I only tried reloading the collection as a whole. Not the core reload. Where do I see the cache sizes after reload? -Lewin -Original Message- From: Shawn Heisey Sent: Monday, April 8, 2019 5:10 PM To: solr-user@lucene.apache.org Subject: Re: Solr Cache clear On 4/8/2019 2:14 PM

Re: Solr Cache clear

You may have warming queries to prepopulate your cache. Check your solrconfig.xml. Regards, Alex On Mon, Apr 8, 2019, 4:16 PM Lewin Joy (TMNA), wrote: > ** PROTECTED 関係者外秘 > How do I clear the solr caches without restarting Solr cluster? > Is there a way? > I tried reloading the collection.

Re: Solr Cache clear

On 4/8/2019 2:14 PM, Lewin Joy (TMNA) wrote: How do I clear the solr caches without restarting Solr cluster? Is there a way? I tried reloading the collection. But, it did not help. When I reload a core on a test setup (solr 7.4.0), I see cache sizes reset. What evidence are you seeing that rel

Re: Solr cache clear

On 11/21/2018 8:36 AM, Rajdeep Sahoo wrote: The problem is that we are using master slave solr configuration and for similar type of query sometime it is taking 512 ms and sometime it is 29 ms . we are observing this issue since the query modification. As part of modification we have reduced a l

Re: Solr cache clear

Hi all, The problem is that we are using master slave solr configuration and for similar type of query sometime it is taking 512 ms and sometime it is 29 ms . we are observing this issue since the query modification. As part of modification we have reduced a large no of facet.field param. In solrc

Re: Solr cache clear

Disabling or reducing autowarming can help too, in addition to cache size reduction. Edward Em ter, 20 de nov de 2018 17:29, Erick Erickson Why would you want to? This sounds like an XY problem, there's some > problem you think would be cured by clearing the cache. What is > that problem? > > B

Re: Solr cache clear

Why would you want to? This sounds like an XY problem, there's some problem you think would be cured by clearing the cache. What is that problem? Because I doubt this would do anything useful, pretty soon the caches would be filled up again and you'd be right back where you started and the real

Re: Solr cache clear

On 11/20/2018 9:25 AM, Rajdeep Sahoo wrote: Hi all, Without restarting is it possible to clear the cache? You'll need to clarify what cache you're talking about, but I think for the most part that if you reload the core (or collection if running SolrCloud) that all caches should be rebuilt em

Re: Solr cache for specific field

On 8/18/2015 7:21 AM, Norgorn wrote: > SOLR version - 4.10.3 > We have SOLR Cloud cluster, each node has documents only for several > categories. > Queries look like "...fq=cat(1 3 89 ...)&..." > So, only some nodes need to process, others can answer with zero as soon as > they check "cat". > > The

Re: Solr cache for specific field

Solr Cloud Document Routing described at https://cwiki.apache.org/confluence/display/solr/Shards+and+Indexing+Data+in+SolrCloud allows you to omit hitting certain shards, but they need to be assigned with the different prefixes beforehand. Do I get your point right? On Tue, Aug 18, 2015 at 4:57 PM

Re: Solr cache for specific field

Have you tried this with Cache=false? https://wiki.apache.org/solr/CommonQueryParameters#Caching_of_filters Because the internal representation of the field value already may be doing what you want. And the caching of non-repeating filters is what slowing it down. I would just do that as a sanity

Re: Solr cache for specific field

I'm sorry for being so unclear. The problem is in speed - while node holds only several cats, it can answer with "numFound=0", if these cats are missed in query. It looks like: node 1 - cats 1,2,3 node 2 - cats 3,4,5 node 3 - cats 50,70 ... Query "q=cat:(1 4)" QTime per node now is like node1 -

Re: Solr cache for specific field

I am not sure I understand the problem statement. Is it speed? Memory usage? Something very specific about SolrCloud? To me it seems the problem is that your 'fq' _are_ getting cached when you may not want them as the list is different every time. You could disable that cache. Or you could try Te

Re: Solr cache when using custom scoring

Mikhail, We've now override the equal & hashcode of the custom query to use this new param as well, and it works like charm. Thanks allot, Ami -- View this message in context: http://lucene.472066.n3.nabble.com/Solr-cache-when-using-custom-scoring-tp4216419p4216496.html Sent from the Solr - U

Re: Solr cache when using custom scoring

No sure I get you, the parameter is passed to solr as a string. It seems like solr use for the caching key only the query, sort and range of documents (from the doc - "This cache holds the results of previous searches: ordered lists of document IDs (DocList) based on a query, a sort, and the range

Re: Solr cache when using custom scoring

On Wed, Jul 8, 2015 at 11:30 PM, amid wrote: > The custom scoring code use a parameter which passed to the solr query, this param should be evaluated in equals() and hashcode(). isn;t it? -- Sincerely yours Mikhail Khludnev Principal Engineer, Grid Dynamics

Re: Solr cache considerations

I think the attachment got stripped. Here it is: http://www.flickr.com/photos/otis/8409088080/in/photostream Otis -- Solr & ElasticSearch Support http://sematext.com/ On Tue, Jan 22, 2013 at 12:36 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Same here - I've seen some documen

Re: Solr cache considerations

Same here - I've seen some document caches that were huge and highly utilized. Check out the screenshot of the SPM for Solr dashboard that shows pretty high hit rates on all caches. I've circled the parts to look at. ML manager may strip the attachment, of course. :) In addition to multiple in-

Re: Solr cache considerations

Hmm, interesting. I'll have to look closer... On Sun, Jan 20, 2013 at 3:50 PM, Walter Underwood wrote: > I routinely see hit rates over 75% on the document cache. Perhaps yours is > too small. Mine is set at 10240 entries. > > wunder > > On Jan 20, 2013, at 8:08 AM, Erick Erickson wrote: > >> Ab

Re: Solr cache considerations

I routinely see hit rates over 75% on the document cache. Perhaps yours is too small. Mine is set at 10240 entries. wunder On Jan 20, 2013, at 8:08 AM, Erick Erickson wrote: > About your question about document cache: Typically the document cache > has a pretty low hit-ratio. I've rarely, if ev

Re: Solr cache considerations

Wow Erick, The MMap acrtivle is a very fundamental one. Totaly changed my view. It must be mentioned in SolrPerformanceFactors in SolrWiki... I'm sorry I did not know it before. Thank you a lot. I promise to share my results then my cart will start to fly :) On Sun, Jan 20, 2013 at 6:08 PM, Erick

Re: Solr cache considerations

About your question about document cache: Typically the document cache has a pretty low hit-ratio. I've rarely, if ever, seen it get hit very often. And remember that this cache is only hit when assembling the response for a few documents (your page size). Bottom line: I wouldn't worry about this

Re: Solr cache considerations

Ok. Thank you everyone for your helpful answers. I understand that fieldValueCache is not used for resolving queries. Is there any cache that can help this basic scenario (a lot of different queries, on a small set of fields)? Does Lucene's FieldCache help (implicitly)? How can I use RAM to reduce

Re: Solr cache considerations

No, the fieldValueCache is not used for resolving queries. Only for multi-token faceting and apparently for the stats component too. The document cache maintains in memory the stored content of the fields you are retrieving or highlighting on. It'll hit if the same document matches the query multip

Re: Solr cache considerations

Unfortunately, it seems ( http://lucene.472066.n3.nabble.com/Nrt-and-caching-td3993612.html) that these caches are not per-segment. In this case, I want to (soft) commit less frequently. Am I right? Tomás, as the fieldValueCache is very similar to lucene's FieldCache, I guess it has a big contribu

Re: Solr cache considerations

I think fieldValueCache is not per segment, only fieldCache is. However, unless I'm missing something, this cache is only used for faceting on multivalued fields On Thu, Jan 17, 2013 at 8:58 PM, Erick Erickson wrote: > filterCache: This is bounded by 1M * (maxDoc) / 8 * (num filters in > cache).

Re: Solr cache considerations

filterCache: This is bounded by 1M * (maxDoc) / 8 * (num filters in cache). Notice the /8. This reflects the fact that the filters are represented by a bitset on the _internal_ Lucene ID. UniqueId has no bearing here whatsoever. This is, in a nutshell, why warming is required, the internal Lucene I

Re: Solr cache size information

Thanks a lot for these answers! Elisabeth 2011/12/4 Erick Erickson > See below: > > On Thu, Dec 1, 2011 at 10:57 AM, elisabeth benoit > wrote: > > Hello, > > > > If anybody can help, I'd like to confirm a few things about Solr's caches > > configuration. > > > > If I want to calculate cache si

Re: Solr cache size information

See below: On Thu, Dec 1, 2011 at 10:57 AM, elisabeth benoit wrote: > Hello, > > If anybody can help, I'd like to confirm a few things about Solr's caches > configuration. > > If I want to calculate cache size in memory relativly to cache size in > solrconfig.xml > > For Document cache > > size i

RE: Solr cache size information

> For Filter cache > > size in memory = size in solrconfig.xml * WHAT (the size of an id) ??? > (I > don't use facet.enum method) > As I understand it, size is the number queries that will be cached. My short experience means that the memory consumed will be data dependent. If you have a huge

Re: Solr Cache Viewing/Browsing

: used in a modified DisMaxHandler) and I was wondering if there is a way to : get at this data from the JSP pages? I then thought that it might be nice to : view more information about the respective caches like the current elements, : recently evicted etc to help debug performance issues. Has any

Re: Solr Cache

On Sat, Aug 16, 2008 at 12:04 AM, Tim Christensen <[EMAIL PROTECTED]> wrote: > We have two servers, with the same index load balanced. The indexes are > updated at the same time every day. Occasionally, a search on one server > will return different results from the other server, even though the da

Re: Solr cache statistics explanation

lookups : how many times the cache is referenced hits : how many times the cache hits hitratio : hits/lookups and for other items, see my previous mail at: http://www.nabble.com/about-cache-to10192953.html Koji Marshall Gunter wrote: Can someone point me to an in depth explanation of the Solr c