dovecot - Jul 2008 - zlib plugin nad moving mails, bug?

Hi,


Recently I gzipped mails older than 30 days in our users' maildirs. I used 
find and gzip. Gzip added 'Z' at the end of the filename and everything 
seemed to work fine, users did not notice any difference in performance 
(yet :-), and I saved about 30% of our mail storage space.

There seems to be a problem when moving gzipped messages from one folder to 
another though. Dovecot drops the Z flag, moved messages are not processed by 
the zlib plugin and email clients end up showing garbage.

Is there something wrong with my config, or did I encounter a bug?

dovecot -n follows:

# 1.0.13: /etc/dovecot/dovecot.conf
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot.log
protocols: imap imaps pop3s
ssl_ca_file: /etc/certs/certum-chain.crt
ssl_cert_file: /etc/certs/wild.dfqs.pl.crt
ssl_key_file: /etc/certs/wild.dfqs.pl.key
ssl_cipher_list: ALL:!LOW:!MEDIUM
login_dir: /var/run/dovecot/login
login_executable(default): /usr/lib64/dovecot/imap-login
login_executable(imap): /usr/lib64/dovecot/imap-login
login_executable(pop3): /usr/lib64/dovecot/pop3-login
login_greeting: imap.dfqs.pl ready.
login_process_per_connection: no
login_processes_count: 5
login_max_processes_count: 32
max_mail_processes: 4096
verbose_proctitle: yes
mail_location: maildir:~/mail:INDEX=/var/lib/dovecot/indexes/%Ln
mail_debug: yes
maildir_copy_with_hardlinks: yes
maildir_copy_preserve_filename: yes
mail_executable(default): /usr/local/lib64/dovecot-post-login-imap.sh
mail_executable(imap): /usr/local/lib64/dovecot-post-login-imap.sh
mail_executable(pop3): /usr/local/lib64/dovecot-post-login-pop3.sh
mail_plugins(default): imap_quota trash quota acl zlib
mail_plugins(imap): imap_quota trash quota acl zlib
mail_plugins(pop3): quota
mail_plugin_dir(default): /usr/lib64/dovecot/plugins/imap
mail_plugin_dir(imap): /usr/lib64/dovecot/plugins/imap
mail_plugin_dir(pop3): /usr/lib64/dovecot/plugins/pop3
mail_log_max_lines_per_sec: 0
imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep
imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep
imap_client_workarounds(pop3): outlook-idle
pop3_uidl_format(default):
pop3_uidl_format(imap):
pop3_uidl_format(pop3): %08Xu%08Xv
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
namespace:
  type: private
  separator: /
  inbox: yes
namespace:
  type: public
  separator: /
  prefix: Public/
  location: 
maildir:/var/vmail/Public:CONTROL=~/Public/control:INDEX=~/Public/index
namespace:
  type: shared
  separator: /
  prefix: Shared/
  location: 
maildir:/var/vmail/Shared:CONTROL=~/Shared/control:INDEX=~/Shared/index
auth default:
  verbose: yes
  debug: yes
  worker_max_count: 512
  passdb:
    driver: pam
    args: dovecot
  userdb:
    driver: ldap
    args: /etc/dovecot/dovecot-ldap.conf
  socket:
    type: listen
    client:
      path: /var/run/dovecot/auth-client
      mode: 432
      user: root
      group: vmail
    master:
      path: /var/run/dovecot/auth-master
      mode: 432
      user: root
      group: vmail
plugin:
  quota: maildir:storage=12582912:ignore=Trash
  acl: vfile:
  trash: /etc/dovecot/dovecot-trash.conf
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080718/4253e391/attachment-0002.bin>

On Fri, 2008-07-18 at 16:14 +0200, Mateusz Kijowski
wrote:
> Recently I gzipped mails older than 30 days in our users' maildirs. I
used
> find and gzip. Gzip added 'Z' at the end of the filename and
everything
> seemed to work fine, users did not notice any difference in performance 
> (yet :-), and I saved about 30% of our mail storage space.
> 
> There seems to be a problem when moving gzipped messages from one folder to
> another though. Dovecot drops the Z flag, moved messages are not processed
by
> the zlib plugin and email clients end up showing garbage.
> 
> Is there something wrong with my config, or did I encounter a bug?
..
> maildir_copy_with_hardlinks: yes
Hardlink copying is broken with zlib plugin. v1.1 works differently and
it works there (although non-hardlink copying was broken there, fixed
only in hg).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20080718/949c6fff/attachment-0002.bin>

This was a bug in version 1.0 which is now corrected in version 1.1.rc6 
by looking at the file header instead of the Z flag.

http://dovecot.org/pipermail/dovecot-news/2008-May/000069.html

Regards,
--
Nuno

Mateusz Kijowski wrote:
> Hi,
>
>
> Recently I gzipped mails older than 30 days in our users' maildirs. I
used
> find and gzip. Gzip added 'Z' at the end of the filename and
everything
> seemed to work fine, users did not notice any difference in performance 
> (yet :-), and I saved about 30% of our mail storage space.
>
> There seems to be a problem when moving gzipped messages from one folder to
> another though. Dovecot drops the Z flag, moved messages are not processed
by
> the zlib plugin and email clients end up showing garbage.
>
> Is there something wrong with my config, or did I encounter a bug?
>
> dovecot -n follows:
>
> # 1.0.13: /etc/dovecot/dovecot.conf
> base_dir: /var/run/dovecot/
> log_path: /var/log/dovecot.log
> protocols: imap imaps pop3s
> ssl_ca_file: /etc/certs/certum-chain.crt
> ssl_cert_file: /etc/certs/wild.dfqs.pl.crt
> ssl_key_file: /etc/certs/wild.dfqs.pl.key
> ssl_cipher_list: ALL:!LOW:!MEDIUM
> login_dir: /var/run/dovecot/login
> login_executable(default): /usr/lib64/dovecot/imap-login
> login_executable(imap): /usr/lib64/dovecot/imap-login
> login_executable(pop3): /usr/lib64/dovecot/pop3-login
> login_greeting: imap.dfqs.pl ready.
> login_process_per_connection: no
> login_processes_count: 5
> login_max_processes_count: 32
> max_mail_processes: 4096
> verbose_proctitle: yes
> mail_location: maildir:~/mail:INDEX=/var/lib/dovecot/indexes/%Ln
> mail_debug: yes
> maildir_copy_with_hardlinks: yes
> maildir_copy_preserve_filename: yes
> mail_executable(default): /usr/local/lib64/dovecot-post-login-imap.sh
> mail_executable(imap): /usr/local/lib64/dovecot-post-login-imap.sh
> mail_executable(pop3): /usr/local/lib64/dovecot-post-login-pop3.sh
> mail_plugins(default): imap_quota trash quota acl zlib
> mail_plugins(imap): imap_quota trash quota acl zlib
> mail_plugins(pop3): quota
> mail_plugin_dir(default): /usr/lib64/dovecot/plugins/imap
> mail_plugin_dir(imap): /usr/lib64/dovecot/plugins/imap
> mail_plugin_dir(pop3): /usr/lib64/dovecot/plugins/pop3
> mail_log_max_lines_per_sec: 0
> imap_client_workarounds(default): outlook-idle tb-extra-mailbox-sep
> imap_client_workarounds(imap): outlook-idle tb-extra-mailbox-sep
> imap_client_workarounds(pop3): outlook-idle
> pop3_uidl_format(default):
> pop3_uidl_format(imap):
> pop3_uidl_format(pop3): %08Xu%08Xv
> pop3_client_workarounds(default):
> pop3_client_workarounds(imap):
> pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
> namespace:
>   type: private
>   separator: /
>   inbox: yes
> namespace:
>   type: public
>   separator: /
>   prefix: Public/
>   location: 
> maildir:/var/vmail/Public:CONTROL=~/Public/control:INDEX=~/Public/index
> namespace:
>   type: shared
>   separator: /
>   prefix: Shared/
>   location: 
> maildir:/var/vmail/Shared:CONTROL=~/Shared/control:INDEX=~/Shared/index
> auth default:
>   verbose: yes
>   debug: yes
>   worker_max_count: 512
>   passdb:
>     driver: pam
>     args: dovecot
>   userdb:
>     driver: ldap
>     args: /etc/dovecot/dovecot-ldap.conf
>   socket:
>     type: listen
>     client:
>       path: /var/run/dovecot/auth-client
>       mode: 432
>       user: root
>       group: vmail
>     master:
>       path: /var/run/dovecot/auth-master
>       mode: 432
>       user: root
>       group: vmail
> plugin:
>   quota: maildir:storage=12582912:ignore=Trash
>   acl: vfile:
>   trash: /etc/dovecot/dovecot-trash.conf
>

The docs at http://wiki.dovecot.org/MainConfig?highlight=%28ssl_listen%29 
aren't clear but I was wondering if I can have IMAP listening on port 143 
and port 144, for example, on the same machine?

Regards,
KAM