Well, I don't know this issue to such level of granularity. Perhaps others do.
Regards, Alex. ---- Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: http://www.solr-start.com/ On 12 March 2015 at 18:57, Markus Jelsma <markus.jel...@openindex.io> wrote: > Hello Alexandre - if you, and others, allow me to be a bit lazy right now; > are there unit tests that input corrupted segments, where not the structure > but the data is affected, to the codec? > > Thanks, > Markus > > > > -----Original message----- >> From:Alexandre Rafalovitch <arafa...@gmail.com> >> Sent: Thursday 12th March 2015 23:52 >> To: solr-user <solr-user@lucene.apache.org> >> Subject: Re: SSD endurance >> >> Lucene 5 has added a lot of various CRCs to catch index corruption >> situations. I don't know if it is 'perfect', but there was certainly a >> lot of work. >> >> Regards, >> Alex. >> ---- >> Solr Analyzers, Tokenizers, Filters, URPs and even a newsletter: >> http://www.solr-start.com/ >> >> >> On 12 March 2015 at 18:39, Markus Jelsma <markus.jel...@openindex.io> wrote: >> > Thanks for sharing Toke! >> > >> > Reliability should not be a problem for a Solr cloud environment. A >> > corrupted index cannot be loaded due to exceptions so the core should not >> > enter an active state. However, what would happen if parts of the data >> > become corrupted but can still be processed by the codec? I don't even >> > know if the data has a CRC check to guard against such madness? >> > >> > Markus >> > >> > -----Original message----- >> >> From:Toke Eskildsen <t...@statsbiblioteket.dk> >> >> Sent: Thursday 12th March 2015 21:33 >> >> To: solr-user <solr-user@lucene.apache.org> >> >> Subject: SSD endurance >> >> >> >> For those who have not yet taken the leap to SSD goodness because they >> >> are afraid of flash wear, the burnout test from The Tech Report seems >> >> worth a read. The short story is that they wrote data to the drives until >> >> they wore out. All tested drives survived considerably longer than >> >> guaranteed, but 4/6 failed catastrophically when they did die. >> >> >> >> http://techreport.com/review/27909/the-ssd-endurance-experiment-theyre-all-dead >> >> >> >> I am disappointed about the catastrophic failures. One of the promises of >> >> SSDs was graceful end of life by switching to read-only mode. Some of >> >> them did give warnings before the end, but I wonder how those are >> >> communicated in a server environment? >> >> >> >> >> >> Regarding Lucene/Solr, the write pattern when updating an index is benign >> >> to SSDs: Updates are relatively bulky, rather than the evil >> >> constantly-flip-random-single-bits-and-flush pattern of databases. With >> >> segments being immutable, the bird's eye view is that Lucene creates and >> >> deletes large files, which makes it possible for the SSD's wear-leveler >> >> to select the least-used flash sectors for new writes: The write pattern >> >> over time is not too far from the one that The Tech Report tested with. >> >> >> >> - Toke Eskildsen >> >> Whose trusty old 160GB Intel X25-M reports an accumulated 36TB of writes. >> >> >>
