On Mon, Jan 25, 2010 at 9:11 PM, J.C. Roberts <[email protected]> wrote: >DJB does great work and thinks about his code. Like every great > programmer, DJB wants his code to be as "correct" as possible within the > very well known bounding limitations (hardware, compilers, operating > systems, file system code, and so forth). Though he knows the
Could this thread please not be diverted to a discussion about the people behind the software? Otherwise flamewars and hate speech are looming. I am trying to understand the technical issues, not inter-personal quibbles. > limitations better than most, his writings intend to *CONVINCE* you of > the correctness of *his* code and methods (within said bounds), so he > doesn't elaborate on the supposedly "known" limitations and he > expects you to already understand them. > > Constantly bringing up all the limitations where things fail detracts > from the intent to convince you of correctness. Though some consider > not elaborating on the limitations as being incomplete or unfair, not > mentioning them is actually a great application of rhetoric and serves > his purpose very well. Rhetoric implies saying something. Not saying something means not using rhetoric. He is making claims about his software. The fact that what he says about queue reliability implies that FFS and hardware work as they should for the queue to be crash-proof. The fact that he does not talk much about hardware limitations isn't the same as using rhetoric. In any case this is a diversion of the thread to a different topic. > If you don't already know the limitations, then you'll get the false > impression of him claiming infallibility, and you'll be very easily Where did you see him mention infallibility? There's a difference between a crash-proof queue feature and infallibility. A long while ago someone wrote a very nice page about how qmail writes to the disk: http://untroubled.org/benchmarking/qmail-filesystems/operations.html : [quote] Critical qmail Operations A message managed by the typical qmail system goes through either two or three stages. 1. The message is either generated locally or received from a remote system and added to the queue. This stage causes the following disk write operations: 1. queue/pid/PID.TIMESTAMP.1 is created (and queue/pid is implicitly fsync'ed). 2. queue/pid/PID.TIMESTAMP.1 is linked to queue/mess/#/INODE (and queue/mess/# is implicitly fsync'ed). 3. queue/pid/PID.TIMESTAMP.1 is unlinked (and queue/pid is implicitly fsync'ed). 4. The message body is written to queue/mess/#/INODE (opened at stage #1) and explicitly fsync'ed. 5. queue/intd/INODE is created (and queue/intd is implicitly fsync'ed). 6. The message envelope is written to queue/intd/INODE and explicitly fsync'ed. 7. queue/intd/INODE is linked to queue/todo/INODE (and queue/todo is implicitly fsync'ed). In total, there are 7 synchronous disk operations done during the injection process. Of those, the synchronicity of operations 1, 3, and 5 is not required for reliability. 2. The message is processed and delivered by qmail-send. The processing stage causes the following disk write operations: 1. queue/info/#/INODE is created (and queue/info/# is implicitly fsync'ed). 2. If the message has local recipients, queue/local/#/INODE is created (and queue/local/# is implicitly fsync'ed). 3. If the message has remote recipients, queue/remote/#/INODE is created (and queue/remote/# is implicitly fsync'ed). 4. queue/info/#/INODE is written and explicitly fsync'ed. 5. If the message has local recipients, queue/local/#/INODE is written and explicitly fsync'ed. 6. If the message has remote recipients, queue/remote/#/INODE is written and explicitly fsync'ed. 7. queue/intd/INODE is unlinked by qmail-clean (and qmail/intd is implicitly fsync'ed). 8. queue/todo/INODE is unlinked by qmail-clean (and qmail/todo is implicitly fsync'ed). 9. if the message has local recipients, queue/local/#/INODE is unlinked (and queue/local/# is implicitly fsync'ed). 10. if the message has remote recipients, queue/remote/#/INODE is unlinked (and queue/remote/# is implicitly fsync'ed). 11. queue/info/#/INODE is unlinked (and queue/info/# is implicitly fsync'ed). 12. queue/mess/#/INODE is unlinked by qmail-clean (and queue/mess/# is implicitly fsync'ed). In total, there are 6, 9, or 12 synchronous disk operations done during the queue processing stage, depending on if the message had local or remote recipients. 3. For each local recipient, the message is delivered to a maildir. This stage causes the following disk write operations: 1. maildir/tmp/PID.TIMESTAMP.HOSTNAME is created (and maildir/tmp is implicitly fsync'ed). 2. The message is written to the above file and explicitly fsync'ed. 3. maildir/tmp/PID.TIMESTAMP.HOSTNAME is linked to maildir/new/PID.TIMESTAMP.HOSTNAME (and maildir/new is implicitly fsync'ed). 4. maildir/tmp/PID.TIMESTAMP.HOSTNAME is unlinked (and maildir/tmp is implicitly fsync'ed). In total, there are 4 synchronous disk operations done during maildir delivery. Of those, the synchronicity of the first and last stages is not required for reliability. [end quote] If the disks and controllers have disabled write-back cache, and the FS is mounted 'sync', how would a message get lost in a time of crash, assuming of course, as been mentioned that the hardware is doing its "best-effort" ?
