On 4/17/2013 11:56 AM, Mark Miller wrote:
There is one additional caveat - when you disable the updateLog, you have to
switch to MMapDirectoryFactory instead of NRTCachingDirectoryFactory. The NRT
directory implementation will cache a portion of a commit (including hard
commits) into RAM instead of onto disk. On the next commit, the previous one
is persisted completely to disk. Without a transaction log, you can lose data.
I don't think this is true? NRTCachingDirectoryFactory should not cache hard
commits and should be as safe as MMapDirectoryFactory is - neither of which is
as safe as using a tran log.
This is based on observations of what happens with my segment files when
I do a full-import, using autoCommit with openSearcher disabled. I see
that each autoCommit results in a full segment being written, the part
of another segment. On the next autoCommit, the rest of the files for
the last segment are written, another full segment is written, I get
another partial segment. I asked about this on the list some time ago,
and what I just told Umesh is a rehash of what I understood from Yonik's
response.
If I'm wrong, I hope someone who knows for sure can correct me.
Thanks,
Shawn
