yep. Although I won't guarantee that the tlog won't be MMapped even if this is turned off.
On Tue, Oct 27, 2015 at 4:21 PM, Rallavagu <rallav...@gmail.com> wrote: > Is it related to this config? > > <!-- If true, IndexReaders will be opened/reopened from the IndexWriter > instead of from the Directory. Hosts in a master/slave setup > should have this set to false while those in a SolrCloud > cluster need to be set to true. Default: true > --> > <!-- > <nrtMode>true</nrtMode> > > this will be true by default. > > On 10/27/15 1:24 PM, Erick Erickson wrote: >> >> Hmm, the tlog. AFAIK, that's because of >> "real time get" (Yonik/Mark/Whoever, please >> correct if wrong). >> >> This is a feature where when fetching a document as >> the result of a search it insures that you get the most >> recent version whether it's been committed or not..... >> >> The tlog isn't relevant for searching however. >> >> Best, >> Erick >> >> On Tue, Oct 27, 2015 at 8:56 AM, Rallavagu <rallav...@gmail.com> wrote: >>> >>> >>> >>> On 10/27/15 8:43 AM, Erick Erickson wrote: >>>> >>>> >>>> bq: So, the updated file(s) on the disk automatically read into memory >>>> as they are Memory mapped? >>> >>> >>> Yes. >>>> >>>> >>>> >>>> Not quite sure why you care, curiosity or is there something you're >>>> trying to accomplish? >>> >>> >>> This is out of curiosity. So, I can get better understanding of Solr's >>> memory usage (heap & mmap). >>> >>>> >>>> The contents of the index's segment files are read into virtual memory >>>> by MMapDirectory as needed to satisfy queries. Which is the point of >>>> autowarming BTW. >>> >>> >>> >>> Ok. But, I have noticed that even "tlog" files are memory mapped (output >>> from "lsof") in addition to all other files under "data" directory. >>> >>>> >>>> commit in the following is either hard commit with openSearcher=true >>>> or soft commit. >>> >>> >>> >>> Hard commit is setup with openSearcher=false and softCommit is setup for >>> every 2 min. >>> >>>> >>>> Segments that have been created (closed actually) after the last >>>> commit are _not_ read at all until the next searcher is opened via >>>> another commit. Nothing is done with these new segments before the new >>>> searcher is opened which you control with your commit strategy. >>> >>> >>> >>> I see. Thanks for the insight. >>> >>>> >>>> Best, >>>> Erick >>>> >>>> On Mon, Oct 26, 2015 at 9:07 PM, Rallavagu <rallav...@gmail.com> wrote: >>>>> >>>>> >>>>> Erick, Thanks for clarification. I was under impression that >>>>> MMapDirectory >>>>> is being used for both read/write operations. Now, I see how it is >>>>> being >>>>> used. Essentially, it only reads from MMapDirectory and writes directly >>>>> to >>>>> disk. So, the updated file(s) on the disk automatically read into >>>>> memory >>>>> as >>>>> they are Memory mapped? >>>>> >>>>> On 10/26/15 8:43 PM, Erick Erickson wrote: >>>>>> >>>>>> >>>>>> >>>>>> You're really looking at this backwards. The MMapDirectory stuff is >>>>>> for Solr (Lucene, really) _reading_ data from closed segment files. >>>>>> >>>>>> When indexing, there are internal memory structures that are flushed >>>>>> to disk on commit, but these have nothing to do with MMapDirectory. >>>>>> >>>>>> So the question is really moot ;) >>>>>> >>>>>> Best, >>>>>> Erick >>>>>> >>>>>> On Mon, Oct 26, 2015 at 5:47 PM, Rallavagu <rallav...@gmail.com> >>>>>> wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> All, >>>>>>> >>>>>>> Are memory mapped files (mmap) flushed to disk during "hard commit"? >>>>>>> If >>>>>>> yes, >>>>>>> should we disable OS level (Linux for example) memory mapped flush? >>>>>>> >>>>>>> I am referring to following for mmap files for Lucene/Solr >>>>>>> >>>>>>> >>>>>>> http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html >>>>>>> >>>>>>> Linux level flush >>>>>>> >>>>>>> >>>>>>> >>>>>>> http://www.cyberciti.biz/faq/linux-stop-flushing-of-mmaped-pages-to-disk/ >>>>>>> >>>>>>> Solr's hard and soft commit >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >>>>>>> >>>>>>> Thanks in advance.
