On Tue, Nov 24, 2020 at 01:44:28PM -0600, David Wright wrote: > On Mon 23 Nov 2020 at 07:24:37 (-0500), rhkra...@gmail.com wrote: > > On Monday, November 23, 2020 06:15:09 AM Sven Hartge wrote: > > > Joe <j...@jretrading.com> wrote: > > > > That's why we have IMAP, which doesn't use mbox. > > > > > > The IMAP protocal and the backend storage have no connection. > > I didn't think this thread was about the backend at all, but about > where to place the emails being fetched, and hence the protocol > to use.
IMAP isn't like fetchmail. It isn't even like POP3. It doesn't pull all the email down to the local client system and store it on disk. That's the POP3 model. The IMAP model is that the client opens a persistent connection to the IMAP server, and pulls down one message at a time, only when the client wishes to read that particular message. They stay on the server until the client asks for one to be deleted. The client does not, in general, store a permanent copy of a message. Look at the traditional email model first, with no POP3 or IMAP in the picture. Let's take two users, in two different organizations. Traditionally their names are Alice and Bob. We'll say that Alice works at Foo Corp (email domain foo.corp), and Bob is a student at Bar College (email domain bar.college). Foo Corp and Bar College both use a single Unix server with local user accounts, running a traditional Unix MTA, with mail delivered locally. Alice writes a message to Bob (bob@bar.college). After writing it and hitting the send button, the message is queued up for delivery by the MTA on her Unix system. Her MTA looks up the MX record for bar.college in DNS, and gets the name and IP address of Bob's Unix host. The two MTAs talk to each other, and the message is passed using the Internet. Now, the MTA on Bob's server delivers the message locally. It opens Bob's mailbox (or Maildir), and writes the message there. It calls a "sync" function to make sure the bytes are actually written. Now it's stored safely on disk. Bob's MTA tells Alice's MTA that everything is OK. Alice's MTA deletes its copy of the message from its queue, and everything is happy. At some point in the future, Bob will login to his account, and read his mail, using some local MUA such as elm or mailx. The message is stored on the local disk, in whatever format his MTA uses. Now, let's introduce the IMAP layer. Instead of logging in directly to the Unix server, let's say Bob is a Microsoft Windows user, and doesn't know how to ssh into his mail server. So, Bob's administrator sets up an IMAP server on the Unix host. Now, the IMAP server can read Bob's mailbox (in whatever format), and present the message to Bob's IMAP client. Bob can read the messages, reply to them, delete them, and so on. All the things that email users typically do. The MTA is responsible for the local deliveries and storage of the messages that are received. The IMAP server is responsible for presenting the already-stored messages to the IMAP client, which is responsible for interacting with the human user. As long as the MTA, the IMAP server (if any), and the local MUAs (if any) all agree on the mailbox location and format, everything works. The mbox formats store all messages in a single gigantic file. Each message's start is indicated by the presence of a specifc string at the start of a line. When the MTA delivers a message, it simply appends to the end of the mbox file. When an MUA or IMAP server wants to delete a message from the mailbox, it has to copy out the entire mailbox, minus the message that was deleted. It has to try to do this in a way that won't explode if the MTA is trying to deliver a message at the same moment. In general, this is hard, involves file locking, and sucks a lot. The Maildir format stores each message in a separate file. When the MTA wants to deliver a new message, it opens a new file (with a name chosen according to heuristics that make it extremely unlikely to be used by two different programs at the same time), writes the message there, and closes it. When an MUA or IMAP server wants to delete an existing message, it simply removes the file containing that message. It doesn't have to lock the entire mailbox, or make a copy of it. There is a lot more detail involved, of course, but that's the gist of it.
