Buongiorno,
Vedi allegato e di confermare. Password: 1234567 Grazie DIMA From: khi...@gmail.com Sent: Tue, 17 Oct 2017 15:40:50 +0000 To: solr-user@lucene.apache.org Subject: Re: solr 7.0: What causes the segment to flush I take my yesterday's comment back. I assumed that the file being written is a segment, however after letting solr run for the night. I see that the segment is flushed at the expected size:1945MB (so that file which i observed was still open for writing). Now, I have two other questions:- 1. Is there a way to not write to disk continuously and only write the file when segment is flushed? 2. With 6.5: i had ramBufferSizeMB=20G and limiting the threadCount to 12 (since LUCENE-6659 , there is no configuration for indexing thread count, so I did a local workaround to limit the number of threads in code); I had very good write throughput. But with 7.0, I am getting comparable throughput only at indexing threadcount > 50. What could be wrong ? Thanks @Erick, I checked the commit settings, both soft and hard commits are off. On Tue, Oct 17, 2017 at 3:47 AM, Amrit Sarkar wrote: > > > > In 7.0, i am finding that the file is written to disk very early on > > and it is being updated every second or so. Had something changed in 7.0 > > which is causing it? I tried something similar with solr 6.5 and i was > > able to get almost a GB size files on disk. > > > Interesting observation, Nawab, with ramBufferSizeMB=20G, you are getting > 20GB segments on 6.5 or less? a GB? > > Amrit Sarkar > Search Engineer > Lucidworks, Inc. > 415-589-9269 > www.lucidworks.com > Twitter http://twitter.com/lucidworks > LinkedIn: https://www.linkedin.com/in/sarkaramrit2 > > On Tue, Oct 17, 2017 at 12:48 PM, Nawab Zada Asad Iqbal > wrote: > > > Hi, > > > > I have tuned (or tried to tune) my settings to only flush the segment > > when it has reached its maximum size. At the moment,I am using my > > application with only a couple of threads (i have limited to one thread > for > > analyzing this scenario) and my ramBufferSizeMB=20000 (i.e. ~20GB). With > > this, I assumed that my file sizes on the disk will be at in the order of > > GB; and no segments will be flushed until the segment's in memory size >is > > 2GB. In 7.0, i am finding that the file is written to disk very early on > > and it is being updated every second or so. Had something changed in 7.0 > > which is causing it? I tried something similar with solr 6.5 and i was > > able to get almost a GB size files on disk. > > > > How can I control it to not write to disk until the segment has reached > its > > maximum permitted size (1945 MB?) ? My write traffic is 'new >only' (i.e., > > it doesn't delete any document) , however I also found following > infostream > > logs, which incorrectly say 'delete=true': > > > > Oct 16, 2017 10:18:29 PM INFO (qtp761960786-887) [ x:filesearch] > > o.a.s.c.S.Request [filesearch] webapp=/solr path=/update > > params={commit=false} status=0 QTime=21 > > Oct 16, 2017 10:18:29 PM INFO (qtp761960786-889) [ x:filesearch] > > o.a.s.u.LoggingInfoStream [DW][qtp761960786-889]: anyChanges? > > numDocsInRam=4434 deletes=true hasTickets:false > pendingChangesInFullFlush: > > false > > Oct 16, 2017 10:18:29 PM INFO (qtp761960786-889) [ x:filesearch] > > o.a.s.u.LoggingInfoStream [IW][qtp761960786-889]: nrtIsCurrent: > infoVersion > > matches: false; DW changes: true; BD changes: false > > Oct 16, 2017 10:18:29 PM INFO (qtp761960786-889) [ x:filesearch] > > o.a.s.c.S.Request [filesearch] webapp=/solr path=/admin/luke > > params={show=index&numTerms=0&wt=json} status=0 QTime=0 > > > > > > > > Thanks > > Nawab > > >
<<attachment: Atitlan.zip>>
