On Tue, 2003-07-08 at 06:44, Magnus Sundberg wrote: > Hi again, > I have done some more thinking about the database structure. > I am not sure of the performance gain with a fast header table > structure. > This would be ideal with a single table and one VARCHAR column > for each _interesting_ header. How do we know what the clients > think are interesting headers?
This is why I suggested the value pair design, since we won't always know what's interesting. > The ideal world would be SQL statements like > SELECT subject, from, To FROM fastheadertable WHERE mailbox= > > The question is, how many of these multiple selects do the imap > clients issue? I have no idea, unfortunately, I don't really know how the imap clients really work. > WE DO GAIN if the clients very often request headers from > multiple mail in the mailbox, but do the clients make these > requests often? Mozilla only do this when the mailbox cache is > non existant. > > I beleive in splitting the message into two parts, header and body. > > What is the mail server performance loss to do something like > 1. SELECT header FROM headers WHERE mailbox= > 2. With some carefully written C code extract the interesting > headers. > Operation 2 can be distributed to another server. > > This last option shuffles more data, but there is only one select > statement, as well as in the first example. > With the last option we do not need to do any more client request > logging :-). > > A header, might be around 4 kbyte. How much does this cost? > > I know there are people with huge mail boxes. > > Well, any comments? My real question is what queries are being done by the server. When I use evolution and ask it to search the message body for a piece of text, does it send this request to the server? Or does it only search what is cached. Also, IMAP clients are not all we care about. We also have POP3 (less interesting) and direct clients that bypass the IMAP interface, querying the database themselves. I know several web apps do this already. Also, there is talk extending dbmail to include more exchange type functionality and implementing a SOAP interface, thus I think it's a bit foolish to optimize the design solely around IMAP. I think dbmail is great, and really has the potential to evolve into a killer groupware server. The best part is it already works well for what it does, it's not some pie in the sky project that is trying to do everything all at once. > /Magnus > > P.S. Why is a message split into multiple message blocks? Is this > because of some limitations in the SQL servers? I was told it is due to a limitation of MySQL but also due to the fact that the headers are stored in the first block, which helps to make header lookups faster. > Jesse Norell wrote: > > Hello, > > > > (moving this to dbmail-dev) > > > > > >>My vision here is a "fastheaders" table which has a message number and > >>half a dozen columns defined. On some regular basis, or upon message > >>insertion, the messageblks table is scanned and the first entry of each > >>new message number, which is the header, is parsed for these fast headers > >>and they are stored for future searches and quick/short header listings. > > > > > > An idea we've started (not completed) implimenting in weDBmail > > along these lines is dynamically parsing/caching the headers. Any > > time the message headers are requested, it'll check the "header cache" > > table and use entries if found, but if not, it'll parse the headers > > from messageblks and use the results while saving appropriate ones in > > the cache. Could make the pop3 and imap servers do that as well. > > > > > > > > > > -- > > Jesse Norell > > jesse (at) kci.net > > > > > > _______________________________________________ > > Dbmail-dev mailing list > > [email protected] > > http://twister.fastxs.net/mailman/listinfo/dbmail-dev > > > > > > > _______________________________________________ > Dbmail-dev mailing list > [email protected] > http://twister.fastxs.net/mailman/listinfo/dbmail-dev >
