mbsync / dovecot Timeouts und UIDVALIDITY-Fehler bei vielen Mails

sia · 3 Juni 2018

Hallo Leute,

Ich nutze mbsync zum Hochladen von Mails auf einen Dovecot-Mailserver.

So sieht meine .mbsyncrc aus (Auszug):

Code:

IMAPAccount example-private
Host    example.org
User    phre4k@example.org
Pass    examplepassdovecot
SSLType STARTTLS
Port    144
Timeout 60

IMAPStore private-remote
Account   example-private

MaildirStore	private-local
Path		      ~/.mail/private/
Inbox		      ~/.mail/private/INBOX
Subfolders	  Verbatim

# ensures that dates of messages will be set correctly
CopyArrivalDate yes

Channel   upload-private
Master    :private-remote:
Slave     :private-local:
Patterns  *
Create    Both
Sync      Push
SyncState *
# important otherwise you will get 'Error: store ... does not support in-box sync state'
SyncState ~/.mail/imap-transfer

Wenn ich jetzt mbsync upload-private eingebe, bekomme ich folgende Nachrichten von mbsync:

Code:

mbsync -V upload-private                                                                                                                 :(
Reading configuration file /home/phre4k/.mbsyncrc
C: 0/1  B: 0/0  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Channel upload-private
Opening master store private-remote...
Resolving example.org... ok
Connecting to example.org (192.168.3.4:143)... 
Opening slave store private-local...
Connection is now encrypted
Logging in...
Authenticating with SASL mechanism PLAIN...
C: 0/1  B: 0/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box INBOX...
Opening slave box INBOX...
Error: channel upload-private: UIDVALIDITY of both master and slave changed
(master got 1527949126, expected 1528023553; slave got 1527953668, expected 1527960879).
C: 0/1  B: 1/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archive...
Opening slave box Archive...
Error: channel upload-private: slave Archive cannot be opened.
C: 0/1  B: 2/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives...
Opening slave box Archives...
Error: channel upload-private: master Archives cannot be opened.
C: 0/1  B: 3/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2011...
Opening slave box Archives/2011...
Loading master...
master: 398 messages, 0 recent
Loading slave...
slave: 398 messages, 0 recent
Synchronizing...
C: 0/1  B: 4/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2012...
Opening slave box Archives/2012...
Loading master...
master: 1435 messages, 0 recent
Loading slave...
slave: 1435 messages, 0 recent
Synchronizing...
C: 0/1  B: 5/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2013...
Opening slave box Archives/2013...
Error: channel upload-private: master Archives/2013 cannot be opened.
C: 0/1  B: 6/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2014...
Opening slave box Archives/2014...
Error: channel upload-private: master Archives/2014 cannot be opened.
C: 0/1  B: 7/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2015...
Opening slave box Archives/2015...
Error: channel upload-private: master Archives/2015 cannot be opened.
C: 0/1  B: 8/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2016...
Opening slave box Archives/2016...
Error: channel upload-private: master Archives/2016 cannot be opened.
C: 0/1  B: 9/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2017...
Opening slave box Archives/2017...
Loading master...
master: 145 messages, 0 recent
Loading slave...
slave: 145 messages, 0 recent
Synchronizing...
C: 0/1  B: 10/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Archives/2018...
Opening slave box Archives/2018...
Error: channel upload-private: master Archives/2018 cannot be opened.
C: 0/1  B: 11/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Drafts...
Opening slave box Drafts...
Error: channel upload-private: UIDVALIDITY of both master and slave changed
(master got 1527949129, expected 1528023555; slave got 1527953899, expected 1527960939).
C: 0/1  B: 12/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Entw&APw-rfe...
Opening slave box Entw&APw-rfe...
Error: channel upload-private: master Entw&APw-rfe cannot be opened.
C: 0/1  B: 13/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Junk...
Opening slave box Junk...
Error: channel upload-private: slave Junk cannot be opened.
C: 0/1  B: 14/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Papierkorb...
Opening slave box Papierkorb...
Error: channel upload-private: master Papierkorb cannot be opened.
C: 0/1  B: 15/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Queue...
Opening slave box Queue...
Error: channel upload-private: master Queue cannot be opened.
C: 0/1  B: 16/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent...
Opening slave box Sent...
Error: channel upload-private: UIDVALIDITY of both master and slave changed
(master got 1527949128, expected 1528023557; slave got 1527953901, expected 1527960939).
C: 0/1  B: 17/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2009...
Opening slave box Sent/2009...
Loading master...
master: 4 messages, 0 recent
Loading slave...
slave: 4 messages, 0 recent
Synchronizing...
C: 0/1  B: 18/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2010...
Opening slave box Sent/2010...
Loading master...
master: 93 messages, 0 recent
Loading slave...
slave: 93 messages, 0 recent
Synchronizing...
C: 0/1  B: 19/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2011...
Opening slave box Sent/2011...
Loading master...
master: 210 messages, 0 recent
Loading slave...
slave: 210 messages, 0 recent
Synchronizing...
C: 0/1  B: 20/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2012...
Opening slave box Sent/2012...
Loading master...
master: 713 messages, 0 recent
Loading slave...
slave: 713 messages, 0 recent
Synchronizing...
C: 0/1  B: 21/28  M: +0/0 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2013...
Opening slave box Sent/2013...
Loading master...
Loading slave...
slave: 623 messages, 0 recent
master: 404 messages, 0 recent
Warning: lost track of 98 pushed message(s)
Synchronizing...
C: 0/1  B: 22/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2014...
Opening slave box Sent/2014...
Error: channel upload-private: master Sent/2014 cannot be opened.
C: 0/1  B: 23/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2015...
Opening slave box Sent/2015...
Error: channel upload-private: master Sent/2015 cannot be opened.
C: 0/1  B: 24/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2016...
Opening slave box Sent/2016...
Error: channel upload-private: master Sent/2016 cannot be opened.
C: 0/1  B: 25/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2017...
Opening slave box Sent/2017...
Error: channel upload-private: master Sent/2017 cannot be opened.
C: 0/1  B: 26/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Spam...
Opening slave box Spam...
Error: channel upload-private: master Spam cannot be opened.
C: 0/1  B: 27/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Trash...
Opening slave box Trash...
Error: channel upload-private: UIDVALIDITY of both master and slave changed
(master got 1527949127, expected 1528023558; slave got 1527953961, expected 1527960989).
C: 1/1  B: 28/28  M: +219/219 *0/0 #0/0  S: +0/0 *0/0 #0/0

Manchmal auch:

Code:

Opening master box Archives/2015...
Opening slave box Archives/2015...
Loading master...
Loading slave...
slave: 1336 messages, 0 recent
master: 435 messages, 0 recent
Warning: lost track of 68 pushed message(s)
Synchronizing...
C: 0/1  B: 21/28  M: +652/652 *0/0 #0/0  S: +0/0 *0/0 #0/0
Opening master box Sent/2013...
Opening slave box Sent/2013...
Creating master Sent/2013...
Loading master...
master: 0 messages, 0 recent
Loading slave...
slave: 623 messages, 0 recent
Synchronizing...
C: 0/1  B: 21/28  M: +1043/1154 *0/0 #0/0  S: +0/0 *0/0 #0/0
Socket error on example.org (192.168.3.4:143): timeout.

und folgendes steht in den Dovecot-Logs:

Code:

Error: imap(phre4k@example.org)<788><HprJl7tt/lMuBQBr>: Mailbox Sent/2013: Saving mail: save: read(<imap client>) failed: zlib.read(<imap client>): unexpected EOF at 292366799

Was kann ich da machen? Ist meine Mailbox jetzt fucked?

drfuture · 4 Juni 2018

Unexpected EOF und diese Dateigrößen nutzt du das mbox format?
Dann würde ich das dringend in Maildir konvertieren. z.B. mit

You do not have permission to view link please Anmelden or Registrieren

sia · 4 Juni 2018

Also die Mails sind jetzt alle drüben, indem ich das einfach immer und immer wieder neu gestartet habe. Mein Problem ist, dass wohl viele Mails doppelt sind; wie bekomm ich die Dubletten jetzt zuverlässig weg?

Unexpected EOF und diese Dateigrößen – nutzt du das mbox format?

Nein, Maildir, siehe Config.

Mein Verdacht: isync komprimiert alles, was es rüber schiebt, in einen einzigen zlib-Stream und der ist zu groß für dovecot?

Würde jetzt [kw]DisableExtension COMPRESS=DEFLATE[/kw]

You do not have permission to view link please Anmelden or Registrieren

ausprobieren, aber meine mbsync-Version ist schon [kw]isync 1.3.0[/kw] vom 2017-10-01…

EDIT: Vielleicht liegt's auch daran, dass meine Verbindung zu schnell ist und [kw]PipelineDepth unlimited[/kw] (der Standardwert) eingestellt ist?

mbsync / dovecot Timeouts und UIDVALIDITY-Fehler bei vielen Mails

Weitere

sia

gesperrt

drfuture

Zeitreisender

sia

gesperrt