You have to separate out a couple of things. First, data gets written to segments _without_ the segment getting closed and _before_ you commit. What happens is that when ramBufferSizeMB in solrconfig.xml is exceeded, its contents are flushed to the currently-opened segment. The segment is _not_ closed, so the fact that you see index size varying on disk apart from commits is not surprising at all.
Second, segments are merged. During merging, a complete copy is made of the segments being merged. Only when the merge is successful are the old segments deleted (well, and the searcher holding them open is closed). This happens as a background process. All of which is a way of saying I think you're being puzzled by the extra stuff going on and can safely ignore it all... Best, Erick On Mon, Mar 10, 2014 at 9:57 AM, Furkan KAMACI <furkankam...@gmail.com> wrote: > Hi; > > Did you read here: > http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > > Thanks; > Furkan KAMACI > > > 2014-03-10 15:14 GMT+02:00 RadhaJayalakshmi <rlakshminaraya...@inautix.co.in >>: > >> Hi, >> >> Brief Description of my application: >> We have a java program which reads a flat file, and adds document to solr >> using cloudsolrserver. >> And we index for every 1000 documents(bulk indexing). >> >> And the Autocommit setting of my application is: >> <autoCommit> >> <maxDocs>100000</maxDocs> >> <openSearcher>false</openSearcher> >> </autoCommit> >> >> So after every 100,000 documents are indexed, engine should perform a >> HardCommit/AutoCommit. But still the OpenSearcher will be false. >> Once the file is fully read, we are issuing a commit() from the >> CloudSolrServer class. So this by default opens a new Searcher. >> >> Also, from the Log, i can see that three times, Autocommit is happenning. >> and Only with the last/final Autocommit, opensearcher is set to true. >> >> So, till now all looks fine and working as expected. >> >> But one strange issue i observed during the course of indexing. >> Now, as per the documentation, the data that is being indexed should first >> get written into tlog. When the Autocommit is performed, the data will be >> flushed to disk. >> So only at three times, there should have been size difference in the >> /index >> folder. All the time only the size of the /tlog folder should have been >> changing >> >> But actually happened is, all the time, i see the size of the /index folder >> getting increased in parallel to the size of the /tlog folder. >> Actually it is increasing to certain limit and coming down. Again >> increasing >> and coming down to a point. >> >> So Now the bigger doubt is have is, during hard commit, is the data getting >> written into both /index or /tlog folder?? >> >> I am using solr 4.5.1. >> >> Some one please clear me how the hardcommit works. I am asumming the >> following sequence: >> 1. Reads the data and writes to tlog >> 2. During hardcommit, flushes the data from tlog to index. If openSearcher >> is false, should not open a new searcher >> 3. In the end, once all the datas are indexed, it should open a new >> searcher. >> >> If not please explain me.. >> >> Thanks in Advance >> Radha >> >> >> >> >> >> >> >> >> -- >> View this message in context: >> http://lucene.472066.n3.nabble.com/The-way-Autocommit-works-in-solr-Wierd-tp4122558.html >> Sent from the Solr - User mailing list archive at Nabble.com. >>
