mc wrote: > Hi All, > > I have finally managed to import some of the mails from our current user > base to test out dbmail. I am doing the tests with 2.0.3 and 2.1.3 but > have noticed severe performance problems with 2.1.3. The IMAP client I > used in the test is Outlook Express.
Try the svn-trunk version. Many problems were fixed since 2.1.3. > I have not consulted the > source tree yet, but I think dbmail is somehow trying to read the > contents of the whole folder into memory without releasing it after each > FETCH statements. Retrieving the whole contents of a folder may well be caused by a single fetch command. Still, memory usage should stay within sane limits. > My questions finally come: > 1) why was it necessary to change from the internal mime parser to > gmime? does the internal mime parser read everything into memory as well? The old mimeparser was buggy, did not deal well with non-latin charsets, was coded in a non-maintainable style (spagetti pattern, with many globals variables, etc) and was generally a major pain to debug. > 2) is this memory hogging a normal behavior (i.e. the memory is supposed > to be free'd when the sync finishes) or something unexpected? sounds like a bug, and one that may very well have been fixed since 2.1.3. > 3) why is 'full MIME parsing' ever needed when the imap clients only > expect to know certain meta info such as subject, sender, rfclines, ..., > so on and so forth? Why isn't only the wanted fields extracted from a > message instead of reading the whole big big message into memory and > pass it to a MIME parser? Can I consider the additional workload by the > MIME parser redundant? Extracting body structure requires mime-parsing, same for envelope data. Header fields are all stored seperately during insertion and do not required mime-parsing during retrieval any more. > 4) after all, if message parsing (or just extract some meta data from > the headers?) is really needed for each message, why isn't the parsing > done when the message is inserted? The headers could hopefully be saved > in more compat formats in the database, and no extra parsing is ever > needed when the message is retrieved. Wrong. Even now that all headers *are* stored separately during insertion, mime-parsing during fetch commands is unavoidable for many fetch commands. No imap server can operate without a worldclass mime parser. > > Just a side note, only mail folders with 'big and perhaps complex MIME > structure' (i.e. those generated by Outlook, Outlook Express, ...) > causes the mentioned problem. Folders with 'mailing list style' plain > text simple MIME structure mails are unaffected. Like I said, please try svn-trunk to see if that helps. -- ________________________________________________________________ Paul Stevens mailto:[EMAIL PROTECTED] NET FACILITIES GROUP PGP: finger [EMAIL PROTECTED] The Netherlands________________________________http://www.nfg.nl
