Hi Jonas, On Tue, 05 Nov 2019 at 13:55:59 +0100, Jonas Smedegaard wrote: > $ interimap --config hb INBOX > remote: ERROR: UIDVALIDITY changed! (1571588814 != 1154884797) Need to > invalidate the UID cache. > > ..and it seems the --repair option doesn't do its job:
I guess naming that command ‘--repair’ wasn't the best choice; it's not meant to reconcile UIDVALIDITY mismatch anyway, but to perform a so-called “full synchronization” à la offlineimap to compare the UID+flag list between the local and remote mailboxes. (Unless there is a bug in interimap or the IMAPd's QRESYNC implementation there shouldn't be anything to repair, but it's still useful for integrity checking etc.) IMAP4rev1 allows servers without mechanism for storing persistent UID and UIDVALIDITY values. However synchronizing such server is out of scope for interimap(1), and I guess other IMAP-based synchronization software, cf. RFC 3501 sec. 2.3.1.1: “Persistent unique identifiers are required for a client to resynchronize its state from a previous session with the server […] The combination of mailbox name, UIDVALIDITY, and UID must refer to a single immutable message on that server forever.” There is nothing interimap can do so resolve UIDVALIDITY conflicts. It needs to invalidate its cache, because per RFC 3501 the lUID ↔ rUID mapping can't be relied upon anymore. (Rebuilding the mapping by comparing message RFC 5355 data is not reliable either, because IMAP does allow duplicates.) > local: S: * STATUS INBOX (UIDNEXT 290008 UIDVALIDITY 1571588814 HIGHESTMODSEQ > 71671) > […] > remote: S: * STATUS INBOX (UIDNEXT 280739 UIDVALIDITY 1571588814 > HIGHESTMODSEQ 74538) > […] > remote: ERROR: UIDVALIDITY changed! (1571588814 != 1154884797) Need to > invalidate the UID cache. I find it suspicious that both mailboxes have the same UIDVALIDITY value. By RFC 3501 that value is an arbitrary unsigned >0 32-bits integer. That value (like UID values) are set by the server and can't be chosen or altered by the client (otherwise interimap wouldn't need a database to keep track of the lUID ↔ rUID mapping, a mere state file would be sufficient). Dovecot doesn't pick it at random, but AFAICT uses the timestamp at which the mailbox was created. It looks kind of weird to me that both your local and remote INBOX appears to have been created at the very same time (Oct 20 18:26 CEST) while the timestamp found in the database is many years older (Aug 06 2006). The UIDVALIDITY conflict can't be caused by interimap or any other IMAP client, because that's beyond the IMAP protocol. Did you maybe run a lower-level synchronization tool like dsync(1) between these mailboxes? IIRC dsync preserve states, so it could explain why the UIDVALIDITY value was reset to the very same timestamp. (UIDVALIDITY reset could also happen when the dovecot index files are removed, but it seems unlikely that they would be set to the very same value.) It should never happen with a “normal” layered IMAP stack. Dovecot does have persistent UID values, at least when the index files are not stored on a volatile medium. Now, how to fix this? The easiest is to remove the database entry for that mailbox: `interimap --target=database delete INBOX` (this won't touch your mails, just the database) and then try to sync again. However every message in the local INBOX will be copied remotely and vice-versa, so if you have a lot of messages in both mailboxes this will create a lot of duplicates :-( This won't cause any data loss though. Another thing which seems to suggest that there is a lower-level synchronization running and causing cache corruption, is the huge spike between the local INBOX's current state and the one from the database (ie the last successful sync): UIDNEXT=9598 / HIGHESTMODSEQ=165 vs. UIDNEXT=290008 / HIGHESTMODSEQ=71671. While sudden heavy-traffic mailboxes are certainly possible, it's still a bit suspicious. The first step is to identify what's causing the cache corruption as it's likely to happen again otherwise, and that's beyond the scope of interimap and other plain IMAP clients :-P If it was indeed dsync(1) causing the cache corruption, then it's possible to avoid duplicates: you could run it once more so both sides are in sync (with matching/ overwritten UIDs, UIDVALIDITY, etc. values), then nuke *one* end along with the database (using `interimap --target=database --target=local delete INBOX`) and finally sync again. I reckon it's a bit scary though, so feel free to poke me over IRC if you'd like. -- Guilhem.
signature.asc
Description: PGP signature
