NAME
offlineimap - Synchronize mailboxes and Maildirs both ways or one either way.SYNOPSIS
offlineimap (options)
DESCRIPTION
Synchronize the accounts configured in the configuration file via IMAP. Each account has two sides. One of the side must be an IMAP server. The other side can either be a Maildir or another IMAP server.OPTIONS
-h, --helpDisplay summary of options.
--version
Output version.
-V
Output offlineimap version and additional
imaplib2 information.
--dry-run
Run in dry run mode.
Do not actually modify any store but check and print what synchronization
actions would be taken if a sync would be performed. It will not precisely
give the exact information what will happen. If e.g. we need to create a
folder, it merely outputs Would create folder X, but not how many and
which mails it would transfer.
--info
Output information on the configured email
repositories.
Useful for debugging and bug reporting. Use in conjunction with the
‘-a’ option to limit the output to a single account. This mode
will prevent any actual sync to occur and exits after it output the debug
information.
-1
Limit multithreading operations and run solely
a single-thread sync.
This effectively sets the maxsyncaccounts and all maxconnections
configuration file variables to 1 (the number).
-P <directory>
Set OfflineIMAP into profile mode.
The program will create DIR (it must not already exist). As it runs, Python
profiling information about each thread is logged into profiledir. Please
note: This option is present for debugging and optimization only, and should
NOT be used unless you have a specific reason to do so. It will significantly
decrease program performance, may reduce reliability, and can generate huge
amounts of data. This option implies the ‘-1’ option.
-a <account1[,account2[,...]]>
Overrides the accounts section in the config
file.
Allows one to specify a particular account or set of accounts to sync without
having to edit the config file.
-c <path/to/configuration_file>
Specifies a configuration file to use.
-d <type1[,type2[,...]]>
Enables debugging for OfflineIMAP.
This is useful if you are to track down a malfunction or figure out what is
going on under the hood. This option requires one or more debugtypes,
separated by commas. These define what exactly will be debugged, and so far
include options: "imap", "thread", "maildir" or
"ALL". The imap option will enable IMAP protocol stream and parsing
debugging. Note that the output may contain passwords, so take care to remove
that from the debugging output before sending it to anyone else. The maildir
option will enable debugging for certain Maildir operations. The use of any
debug option (unless "thread" is included), implies the
single-thread option ‘-1’.
-l <path/to/file.log>
Send logs to <file.log>.
-s
Send logs to syslog.
-f <folder1[,folder1[,...]]>
Only sync the specified folders.
The folder names are the untranslated foldernames of the remote repository. This
command-line option overrides any folderfilter and
folderincludes options in the configuration file.
-k <[section:]option=value
Override any configuration file option.
If "section" is omitted, it defaults to "general". Any
underscores in the section name are replaced with spaces: for instance, to
override option "autorefresh" in the "[Account Personal]"
section in the config file one would use ‘-k
Account_Personal:autorefresh=30’. Repeat this option as much as
necessary to redefine multiple options.
-o
Run only once.
Ignore any autorefresh setting in the configuration file.
-q
Run only quick synchronizations.
Ignore any flag updates on IMAP servers. If a flag on the remote IMAP changes,
and we have the message locally, it will be left untouched in a quick run.
This option is ignored if maxage is set.
-u <UI>
Specifies an alternative user interface to
use.
This overrides the default specified in the configuration file. The UI specified
with ‘-u’ will be forced to be used, even if checks determine
that it is not usable. Possible interface choices are: quiet, basic, syslog,
ttyui, blinkenlights, machineui.
--delete-folder
Delete a folder on the remote repository.
Only one account must be specified/configured for this feature to work or you
must provide one account with -a. The folder name must be provided with the
remote separators (likely /) in UTF-8 if utf8foldernames is enabled or
in IMAP otherwise. E.g.: "Remote/folder/name".
--migrate-fmd5-using-nametrans
Migrate FMD5 hashes from versions prior to
6.3.5.
The way that FMD5 hashes are calculated was changed in version 6.3.5 (now using
the nametrans folder name) introducing a regression which may lead to
re-uploading all messages. Try and fix the above regression by calculating the
correct FMD5 values and renaming the corresponding messages.
Remove dangling entries for removed accounts
or if mbnames is not enabled/used anymore.
Internally, offlineimap build intermediate mbnames files. They are added
automatically when mbnames is enabled. However, disabling accounts so they are
not synced anymore does not necessarily means they should be removed from the
file built by mbnames. It is required to start offlineimap with this CLI
option each time accounts are removed. When run, any account not in the
accounts configuration option are removed in the mbnames file.
It is possible to manually remove intermediate files in
<metadata>/mbnames/.
Notice this option honors --dry-run.
SYNCHRONIZATION PERFORMANCE
By default, we use fairly conservative settings that are safe for syncing but that might not be the best performing one. Once you got everything set up and running, you might want to look into speeding up your synchronization. Here are a couple of hints and tips on how to achieve this. 1.Synchronize more than one account.
By default we only use one connection to an IMAP server. Using 2 or even 3
speeds things up considerably in most cases. In order to synchronize more than
one account concurrently, consider starting one instance of offlineimap per
account.
Warning
enabling the maxsyncaccounts and maxconnections options is
deprecated since it’s known to have race conditions.
2.Use folderfilters.
The quickest sync is a sync that can ignore some folders. I sort my inbox into
monthly folders, and ignore every folder that is more than 2-3 months old,
this lets me only inspect a fraction of my Mails on every sync. If you
haven’t done this yet, do it :). See the folderfilter section in
offlineimap.conf.
3.The sqlite cache.
OfflineImap caches the state of the synchronisation to e.g. be able to determine
if a mail has been added or deleted on either side.
The historical status cache was a plain text file that was writing out the
complete file for each single new message (or even changed flag) to a
temporary file. If there was plenty of files in a folder this was bound to
make things slow. The latest status cache is sqlite. This saves plenty of disk
activity.
The historical plain status cache is not supported anymore but migrating from a
very old installation using the plain text cache is still supported. In this
case, you may want to delete the old cache directory in
<metadata>/Account-<account>/LocalStatus manually (the
sqlite cache stands in the LocalStatus-sqlite folder). First, make sure
you have run the new version of offlineimap for all your accounts so that the
status cache was migrated.
4.Use quick sync.
A regular sync will request all flags and all UIDs of all mails in each folder
which takes quite some time. A quick sync only compares the number of messages
in a folder on the IMAP side (it will detect flag changes on the Maildir side
of things though). A quick sync on my smallish account will take 7 seconds
rather than 40 seconds. E.g. run a cron script that does a regular sync once a
day, and does quick syncs ‘-q’ only synchronizing the ‘-f
INBOX’ in between.
5.Turn off fsync.
In the [general] section you can set fsync to True or
False. If you want to play 110% safe and wait for all operations to hit
the disk before continuing, you can set this to True. If you set it to False,
you lose some of that safety, trading it for speed.
SECURITY AND SSL
By default, OfflineIMAP will connect using any method that openssl supports, that is SSLv2, SSLv3, or TLSv1.•Certificate checking
Unfortunately, by default we will not verify the certificate of an IMAP TLS/SSL
server we connect to, so connecting by SSL is no guarantee against
man-in-the-middle attacks. While verifying a server certificate checking the
fingerprint is recommended. There is currently only one safe way to ensure
that you connect to the correct server in an encrypted manner: you can specify
a sslcacertfile setting in your repository section of offlineimap.conf
pointing to a file that contains (among others) a CA Certificate in PEM format
which validating your server certificate. In this case, we will check that:
1.The server SSL certificate is validated by
the CA Certificate.
2.The server host name matches the SSL
certificate.
3.The server certificate is not past its
expiration date.
•StartTLS
If you have not configured your account to connect via SSL anyway, OfflineImap
will still attempt to set up an SSL connection via the STARTTLS function, in
case the imap server supports it.
There is no certificate or fingerprint checking involved at all, when using
STARTTLS (the underlying imaplib library does not support this yet). This
means that you will be protected against passively listening eavesdroppers and
they will not be able to see your password or email contents. However, this
will not protect you from active attacks, such as Man-In-The-Middle attacks
which cause you to connect to the wrong server and pretend to be your mail
server.
DO NOT RELY ON STARTTLS AS A SAFE CONNECTION GUARANTEEING THE AUTHENTICITY OF
YOUR IMAP SERVER!
UNIX SIGNALS
OfflineImap listens to the unix signals SIGUSR1, SIGUSR2, SIGTERM, SIGINT, SIGHUP, SIGQUIT.•If sent a SIGUSR1 it will abort any
current (or next future) sleep of all accounts that are configured to
autorefresh. In effect, this will trigger a full sync of all accounts
to be performed as soon as possible.
•If sent a SIGUSR2 or SIGABRT, it will
stop autorefresh mode for all accounts. That is, accounts will abort
any current sleep and will exit after a currently running synchronization has
finished. This signal can be used to gracefully exit out of a running
offlineimap "daemon".
•SIGTERM, SIGINT, SIGHUP are all
treated to gracefully terminate as soon as possible. This means it will finish
syncing the current folder in each account, close keep alive connections,
remove locks on the accounts and exit.
It may take up to 10 seconds, if autorefresh option is used.
More than one SIGTERM will behave like SIGQUIT.
•If sent SIGQUIT, dumps stack traces
for all threads and tries to dump process core.
KNOWN ISSUES
•Deletions.
While in usual run the deletions are propagated. To prevent from data loss,
removing a folder makes offlineimap re-sync the folder. However, propagating
the removal of the whole content of a folder can happen in the two following
cases:
•The whole content of a folder is
deleted but the folder directory still exists.
•The parent directory of the folder was
deleted.
•SSL3 write pending.
Users enabling SSL may hit a bug about "SSL3 write pending". If so,
the account(s) will stay unsynchronised from the time the bug appeared.
Running OfflineIMAP again can help. We are still working on this bug. Patches
or detailed bug reports would be appreciated. Please check you’re
running the last stable version and send us a report to the mailing list
including the full log.
•IDLE support is incomplete and
experimental. Bugs may be encountered.
•No hook exists for "run after an
IDLE response".
Email will show up, but may not be processed until the next refresh cycle.
•nametrans may not be supported
correctly.
•IMAP IDLE <→ IMAP IDLE
doesn’t work yet.
•IDLE might stop syncing on a system
suspend/resume.
•IDLE may only work "once"
per refresh.
If you encounter this bug, please send a report to the list!
•Maildir support in Windows drive.
Maildir uses colon character (:) in message file names. Colon is however
forbidden character in windows drives. There are several workarounds for that
situation:
1.Enable file name character translation in
windows registry (not tested).
2.Use cygwin managed mount (not tested).
•not available anymore since cygwin
1.7
3.Use "maildir-windows-compatible =
yes" account OfflineIMAP configuration.
•That makes OfflineIMAP to use
exclamation mark (!) instead of colon for storing messages. Such files can be
written to windows partitions. But you will probably loose compatibility with
other programs trying to read the same Maildir.
•Exclamation mark was chosen because of
the note in http://docs.python.org/library/mailbox.html
•If you have some messages already
stored without this option, you will have to re-sync them again
•OfflineIMAP confused after system
suspend.
When resuming a suspended session, OfflineIMAP does not cleanly handles the
broken socket(s) if socktimeout option is not set. You should enable this
option with a value like 10.
•OfflineIMAP confused when mails change
while in a sync.
When OfflineIMAP is syncing, some events happening since the invocation on
remote or local side are badly handled. OfflineIMAP won’t track for
changes during the sync.
•Sharing a maildir with multiple IMAP
servers.
Generally a word of caution mixing IMAP repositories on the same Maildir root.
You have to be careful that you never use the same maildir folder for 2
IMAP servers. In the best case, the folder MD5 will be different, and you will
get a loop where it will upload your mails to both servers in turn
(infinitely!) as it thinks you have placed new mails in the local Maildir. In
the worst case, the MD5 is the same (likely) and mail UIDs overlap (likely
too!) and it will fail to sync some mails as it thinks they are already
existent.
I would create a new local Maildir Repository for the Personal Gmail and use a
different root to be on the safe side here. You could e.g. use
If you then point your local mutt, or whatever MUA you use to
‘~/mail/’ as root, it should still recognize all folders.
`~/mail/Pro' as Maildir root for the ProGmail and `~/mail/Personal' as root for the personal one.
•Edge cases with maxage causing too
many messages to be synced.
All messages from at most maxage days ago (+/- a few hours, depending on
timezones) are synced, but there are cases in which older messages can also be
synced. This happens when a message’s UID is significantly higher than
those of other messages with similar dates, e.g. when messages are added to
the local folder behind offlineimap’s back, causing them to get
assigned a new UID, or when offlineimap first syncs a pre-existing Maildir. In
the latter case, it could appear as if a noticeable and random subset of old
messages are synced.
•Offlineimap hangs.
When having unexpected hangs it’s advised to set
‘singlethreadperfolder’ to yes, especially when in
IMAP/IMAP mode (no maildir).
•Passwords in netrc.
Offlineimap doesn’t know how to retrieve passwords when more than one
account is stored in the netrc file. See
https://github.com/OfflineIMAP/offlineimap/issues/295.
•XOAUTH2
XOAUTH2 might be a bit tricky to set up. Make sure you’ve followed the
step to step guide in offlineimap.conf. The known bugs about Gmail are
tracked at https://github.com/OfflineIMAP/offlineimap/labels/Gmail.
Sometimes, you might hit one of the following error:
In such case, we had reports that generating a new refresh token from the same
client ID and secret can help.
Google documentation on "invalid_grant".
Token expiration.
See
https://developers.google.com/analytics/devguides/config/mgmt/v3/authorization
and https://developers.google.com/identity/protocols/OAuth2#expiration to know
more.
•[imap]: xoauth2handler: response
"{u’error': u’invalid_grant'}"
•oauth2handler got: {u’error':
u’invalid_grant'}
When you try to use a refresh token, the following returns you an invalid_grant error: - Your server's clock is not in sync with network time protocol - NTP. - The refresh token limit has been exceeded.
It is possible that a granted token might no longer work. A token might stop working for one of these reasons: - The user has revoked access. - The token has not been used for six months. - The user changed passwords and the token contains Gmail scopes. - The user account has exceeded a certain number of token requests. There is currently a limit of 50 refresh tokens per user account per client. If the limit is reached, creating a new token automatically invalidates the oldest token without warning. This limit does not apply to service accounts.
•"does not have message with
UID" with Microsoft servers
Microsoft IMAP servers are not compliant with the RFC. It is currently required
to folderfilter some faulting folders. See
http://www.offlineimap.org/doc/FAQ.html#exchange-and-office365 for a detailed
list.
`ERROR: IMAP server 'Server ### Remote' does not have a message with UID 'xxx'`
MAIN AUTHORS
John Goerzen, Sebastian Spaetz, Eygene Ryabinkin, Nicolas Sebrecht.
SEE ALSO
offlineimapui(7), openssl(1), signal(7), sqlite3(1). http://www.offlineimap.org
| 01/10/2023 |