If you hadn't mentioned the fact you are using physical disk I would have
guessed you were using virtual disks on a SAN. I've seen this sort of thing
happen a lot there. Are there any virtual layers between the cassandra
process and the hardware? Just a reminder, fsync can be a liar and the
virtual
The dmesg command will usually show information about hardware errors.
An example from a spinning disk:
sd 0:0:10:0: [sdi] Unhandled sense code
sd 0:0:10:0: [sdi] Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
sd 0:0:10:0: [sdi] Sense Key : Medium Error [current]
Info fld=0x6fc72