On Mon, March 20, 2006 18:02, Jonathan Angliss wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > > On Mon, March 20, 2006 10:12, eCet wrote: > >>> I agree with this last statement. The backend has a _lot_ to do with >>> this issue. We use an NFS appliance for a backend that we were >>> experiencing problems with (high latency) that caused the same issue >>> with large mailboxen. >> >> So if i'm understand right: You are working on this issue? >> > >>> After correction I can open a maildir mailbox with >>> 40,000+ messages successfully (after about 2 minutes). >>> > >> The opening is not an issue in this 20Kbox it opens (and sorts)in about >> 20secs to 2min. >> Threading is another question it took 5-6mins. >> > > That would be because just doing a normal sort/display doesn't involve > any work on the part of SquirrelMail to be honest. Threading is something > entirely different. Threading returns a formatted string of message ids > that then needs to be processed and the message information from the > server. This is two seperate IMAP calls. The first returns data that > looks like this: > > (1 (2 3 4)) (5) (6 (7 (8 9))) > > > This basically creates a thread that looks like this: > > > + 1 > - 2 > - 3 > - 4 > + 5 > + 6 > + 7 > - 8 > - 9 > > > These IDs then need to be passed onto the code that fetches the message > information. This list of IDs is potentially enormous, especially in > folders that have 20k messages (that's 20k message IDs that need to be > returned).
Theoraticly it involves the parsing of around 12 bytes per message (10 bytes for representing the max value for a 32 bits int and 2 bytes for a space and of a paranthesis) which is around 240KB which is about the same amount of data that has to be processed when we parse the sort response (only the parentheses are not part of the short response). Practically the amount of data to be parsed is shorter because normally the imap server does not return 10 byte string representations for uid's. Personally i don't think that the parsing of the thread response is an issue in 1.5.1 because that part is rewriten the week before we released 1.5.1. The reason for the rewrite had to do with the incapability of parsing large threadresponses in the previous thread parse code (which is the same as we use in 1.4.x). > > The second step in this function is the same used when issuing a fetch > from just a sorted list. The list of message IDs is passed to a function, > which then simply fetches a select number of messages (start message > through to the number of messages a user has displayed). > > Basically summarized, you'll probably find the hangup is in processing > the results from the THREAD call. The only way we could probably reduce > the processing time is to cut down the results by issuing a search > criteria to the THREAD command. Unfortunately that's close to impossible > as it'd probably make it impossible to go back a certain date range, or > we'd have to start doing time calculations and display results from date a > to date b, and when you get close to date b, extend the search to be from > date b to date c. > Correct. I think we should default show the thread result for the last nn days and use the paginator or something else for navigating through fixed periods of time (could be a preference). If we do not implement that in the future it will be impossible to display thread views for large mailboxes. To summarize: For 1.5.1 you can blame the imap server for the slow thread response/ broken thread response and you can blame squirrelmail for issuing a thread call for all messages instead of a thread resonse bounded by a limited period of time. Regards, Marc Groot Koerkamp. ------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 -- squirrelmail-users mailing list Posting Guidelines: http://www.squirrelmail.org/wiki/MailingListPostingGuidelines List Address: squirrelmail-users@lists.sourceforge.net List Archives: http://news.gmane.org/thread.php?group=gmane.mail.squirrelmail.user List Archives: http://sourceforge.net/mailarchive/forum.php?forum_id=2995 List Info: https://lists.sourceforge.net/lists/listinfo/squirrelmail-users
