Nikita Koshikov
2009-Oct-28 08:30 UTC
[Dovecot] expire plugin + --exec-mail in 1.2.6 version
Hello list, After upgrade to dovecot version 1.2.6 I have noticed some quota malfunctions(users become often overquota). Looking at logs closely I have found that quota value resets to general dovecot-config( quota = maildir:Mailbox quota quota_rule = *:storage=500M quota_rule2 = Trash:storage=10%% ), after dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh command. The problem that some users have their own quotas stored in ldap. For example, user sr at domain.com have maildirsize before running expire-tool(1G value from ldap): # head -2 /data/mail/domain.com/sr/data/maildirsize 10485760000S 836423205 8285 Then I run expire-tool with mail_debug enabled: # /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh Info: Loading modules from directory: /usr/lib/dovecot/imap Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so Info: Quota root: name=Mailbox quota backend=maildir argsInfo: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) messages=0 Info: Quota warning: bytes=471859200 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Info: expire: pattern=Trash type=expunge secs=2592000 Info: expire: pattern=Spam type=expunge secs=2592000 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: koshikov.n at domain.com/Spam: seq=1 uid=1869: Expunge Info: koshikov.n at domain.com/Spam: seq=2 uid=1870: Expunge Info: koshikov.n at domain.com/Spam: seq=3 uid=1871: Expunge Info: koshikov.n at domain.com/Spam: seq=4 uid=1872: Expunge Info: koshikov.n at domain.com/Spam: seq=5 uid=1873: Expunge Info: koshikov.n at domain.com/Spam: seq=6 uid=1874: Expunge Info: koshikov.n at domain.com/Spam: seq=7 uid=1875: Expunge Info: koshikov.n at domain.com/Spam: seq=8 uid=1876: Expunge Info: koshikov.n at domain.com/Spam: seq=9 uid=1877: Expunge Info: koshikov.n at domain.com/Spam: seq=10 uid=1878: Expunge Info: koshikov.n at domain.com/Spam: seq=11 uid=1879: Expunge Info: koshikov.n at domain.com/Spam: seq=12 uid=1880: Expunge Info: koshikov.n at domain.com/Spam: timestamp 1256618680 (Tue Oct 27 06:44:40 2009) -> 1256716674 (Wed Oct 28 09:57:54 2009) Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/fsb Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/fsb/data, index=, control=, inbox=/data/mail/domain.com/fsb/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/fsb/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: fsb at domain.com/Trash: seq=1 uid=578: Expunge Info: fsb at domain.com/Trash: seq=2 uid=579: Expunge Info: fsb at domain.com/Trash: seq=3 uid=580: Expunge Info: fsb at domain.com/Trash: seq=4 uid=581: Expunge Info: fsb at domain.com/Trash: seq=5 uid=582: Expunge Info: fsb at domain.com/Trash: seq=6 uid=583: Expunge Info: fsb at domain.com/Trash: seq=7 uid=584: Expunge Info: fsb at domain.com/Trash: seq=8 uid=585: Expunge Info: fsb at domain.com/Trash: seq=9 uid=586: Expunge Info: fsb at domain.com/Trash: seq=10 uid=587: Expunge Info: fsb at domain.com/Trash: seq=11 uid=588: Expunge Info: fsb at domain.com/Trash: seq=12 uid=589: Expunge Info: fsb at domain.com/Trash: seq=13 uid=590: Expunge Info: fsb at domain.com/Trash: timestamp 1256627284 (Tue Oct 27 09:08:04 2009) -> 1256732991 (Wed Oct 28 14:29:51 2009) Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/arsen Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/arsen/data, index=, control=, inbox=/data/mail/domain.com/arsen/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/arsen/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: arsen at domain.com/Trash: seq=1 uid=14: Expunge Info: arsen at domain.com/Trash: seq=2 uid=15: Expunge Info: arsen at domain.com/Trash: seq=3 uid=16: Expunge Info: arsen at domain.com/Trash: seq=4 uid=17: Expunge Info: arsen at domain.com/Trash: seq=5 uid=18: Expunge Info: arsen at domain.com/Trash: seq=6 uid=19: Expunge Info: arsen at domain.com/Trash: seq=7 uid=20: Expunge Info: arsen at domain.com/Trash: seq=8 uid=21: Expunge Info: arsen at domain.com/Trash: seq=9 uid=22: Expunge Info: arsen at domain.com/Trash: seq=10 uid=23: Expunge Info: arsen at domain.com/Trash: seq=11 uid=24: Expunge Info: arsen at domain.com/Trash: timestamp 1256630371 (Tue Oct 27 09:59:31 2009) -> 1256733711 (Wed Oct 28 14:41:51 2009) Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: koshikov.n at domain.com/Trash: seq=1 uid=2561: Expunge Info: koshikov.n at domain.com/Trash: timestamp 1256709279 (Wed Oct 28 07:54:39 2009) -> 1256795118 (Thu Oct 29 07:45:18 2009) Info: auth input: quota_rule=*:bytes=10485760000 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/sr Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/sr/data, index=, control=, inbox=/data/mail/domain.com/sr/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/sr/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: sr at domain.com/Spam: seq=1 uid=405: Expunge Info: sr at domain.com/Spam: seq=2 uid=406: Expunge Info: sr at domain.com/Spam: seq=3 uid=407: Expunge Info: sr at domain.com/Spam: timestamp 1256710764 (Wed Oct 28 08:19:24 2009) -> 1256980000 (Sat Oct 31 11:06:40 2009) Info: kolotilo.m at domain.com/Trash: stop, expire time in future: Wed Oct 28 10:27:41 2009 And after expire-tool ends it's stuff, user sr at domain.com have(512M from dovecot-config): 524288000SS 836423205 8285 As you can see - quota reset to default value. In changes to version 1.2.6 I see - dovecot --exec-mail was broken. Is fixes to 1.2.6 touch userdb lookups from --exec-mail ? With 1.2.4 expire-tool works correct and I have no config changes to dovecot setup. If dovecot -n output is needed, please let me know. Thanks.
Timo Sirainen
2009-Oct-29 17:10 UTC
[Dovecot] expire plugin + --exec-mail in 1.2.6 version
On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote:> The problem that some users have their own quotas stored in ldap.So the quota_rule should be returned by userdb lookup from ldap, right?> Info: auth input: uid=8 > Info: auth input: gid=12 > Info: auth input: home=/data/mail/domain.com/koshikov.n > Info: auth input: mail=maildir:~/dataThis should have listed quota_rule.> If dovecot -n output is needed, please let me know.If the above doesn't help, then yeah, dovecot -n and dovecot-ldap.conf contents. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20091029/212b6228/attachment-0002.bin>
Nikita Koshikov
2009-Oct-30 09:00 UTC
[Dovecot] expire plugin + --exec-mail in 1.2.6 version
On Thu, 29 Oct 2009 13:10:37 -0400 Timo Sirainen <tss at iki.fi> wrote:> On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote: > > The problem that some users have their own quotas stored in ldap. > > So the quota_rule should be returned by userdb lookup from ldap, right?Yes, and it's working fine with deliver and general imap process, it only failed(reset to general quota_rule) after expire-rool.sh script finished his duties.> > > Info: auth input: uid=8 > > Info: auth input: gid=12 > > Info: auth input: home=/data/mail/domain.com/koshikov.n > > Info: auth input: mail=maildir:~/data > > This should have listed quota_rule. >The problematic user is sr, and it has quota_rule option(running with --test) (Thu Oct 29 07:45:18 2009) Info: auth input: quota_rule=*:bytes=10485760000 ^^^^^^^^^^^^^^^^^^^^^^^^^ returned from ldap Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/sr Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/sr/data, index=, control=, inbox=/data/mail/domain.com/sr/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/sr/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: sr at domain.com/Spam: seq=1 uid=405: Expunge Info: sr at domain.com/Spam: seq=2 uid=406: Expunge Info: sr at domain.com/Spam: seq=3 uid=407: Expunge Info: sr at domain.com/Spam: timestamp 1256710764 (Wed Oct 28 08:19:24 2009) -> 1256980000 (Sat Oct 31 11:06:40 2009)> > If dovecot -n output is needed, please let me know. > > If the above doesn't help, then yeah, dovecot -n and dovecot-ldap.conf > contents. >dovecot -n: # 1.2.6: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.11.1 log_path: /var/log/dovecot/dovecot-error.log info_log_path: /var/log/dovecot/dovecot.log protocols: imaps managesieve ssl_cert_file: /etc/ssl/dovecot/imaps.crt ssl_key_file: /etc/ssl/dovecot/imaps.key login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(managesieve): /usr/libexec/dovecot/managesieve-login login_greeting: Server ready. login_processes_count: 20 login_max_processes_count: 512 mail_max_userip_connections(default): 20 mail_max_userip_connections(imap): 20 mail_max_userip_connections(managesieve): 10 first_valid_uid: 8 last_valid_uid: 8 first_valid_gid: 12 last_valid_gid: 12 mail_drop_priv_before_exec: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(managesieve): /usr/libexec/dovecot/managesieve mail_plugins(default): quota imap_quota trash expire zlib autocreate virtual mail_plugins(imap): quota imap_quota trash expire zlib autocreate virtual mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve imap_client_workarounds(default): delay-newmail imap_client_workarounds(imap): delay-newmail imap_client_workarounds(managesieve): namespace: type: private separator: / location: maildir:~/data inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: Company/ location: virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ list: yes lda: postmaster_address: postmaster at domain.com hostname: mail.domain.com mail_plugins: quota trash expire sieve virtual quota_full_tempfail: yes sendmail_path: /usr/sbin/sendmail auth_socket_path: /var/run/dovecot/auth-master log_path: /var/log/dovecot/dovecot-deliver.log info_log_path: /var/log/dovecot/dovecot-deliver.log global_script_path: /etc/dovecot/sieve/default.sieve sieve_global_dir: /etc/dovecot/sieve auth default: mechanisms: plain login default_realm: domain.com cache_size: 10240 cache_negative_ttl: 0 user: dovecot_auth master_user_separator: * worker_max_count: 50 passdb: driver: passwd-file args: /etc/dovecot/passdb/master.pwd master: yes passdb: driver: passwd-file args: /etc/dovecot/passdb/users.pwd passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /etc/dovecot/dovecot-userdb-ldap.conf userdb: driver: passwd-file args: /etc/dovecot/passdb/users.pwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 user: mail group: dovecot_auth master: path: /var/run/dovecot/auth-master mode: 384 user: mail group: mail plugin: quota_warning: storage=90%% /etc/dovecot/plugins/quota_warning.sh 90 quota: maildir:Mailbox quota quota_rule: *:storage=500M quota_rule2: Trash:storage=10%% trash: /etc/dovecot/plugins/dovecot-trash.conf expire: Trash 30 Spam 30 expire_dict: proxy::expire autocreate: Drafts autocreate2: Sent autocreate3: Spam autocreate4: Trash autosubscribe: Drafts autosubscribe2: Sent autosubscribe3: Spam autosubscribe4: Trash sieve: ~/.dovecot.sieve sieve_dir: ~/sieve sieve_extensions: +notify sieve_before: /etc/dovecot/sieve/default.sieve dict: expire: sqlite:/etc/dovecot/plugins/expire.conf dovecot-ldap.conf: uris = ldaps://off-dc-1 ldaps://off-dc-2 dn = imap@ dnpass = qweqwe auth_bind = yes ldap_version = 3 base = OU=Company,DC=company,DC=intranet pass_attrs = =userdb_uid=8, =userdb_gid=12,=userdb_home=/data/mail/%d/%n, =userdb_mail=maildir:~/data, maxStorage=userdb_quota_rule=*:bytes=%$ pass_filter = (&(mail=%u)(!(userAccountControl:1.2.840.113556.1.4.803:=2))) user_attrs = =uid=8, =gid=12,=home=/data/mail/%d/%n, =mail=maildir:~/data, maxStorage=quota_rule=*:bytes=%$ user_filter = (&(mail=%u)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))
Nikita Koshikov
2009-Oct-30 12:03 UTC
[Dovecot] expire plugin + --exec-mail in 1.2.6 version
On Thu, 29 Oct 2009 13:10:37 -0400 Timo Sirainen <tss at iki.fi> wrote:> On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote: > > The problem that some users have their own quotas stored in ldap. > > So the quota_rule should be returned by userdb lookup from ldap, right? >One more clean example. For user koshikov.n at domain.com I change quota in AD to 600M(default user quota is 512M,). After successful login to imap account, maildirsize is(first 2 lines): 629145600S 260658345 5484 So, quota value read from ldap database and applied to user's mailbox. Then I run /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh --test: Info: Loading modules from directory: /usr/lib/dovecot/imap Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so Info: Quota root: name=Mailbox quota backend=maildir argsInfo: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) messages=0 Info: Quota warning: bytes=471859200 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Info: expire: pattern=Trash type=expunge secs=2592000 Info: expire: pattern=Spam type=expunge secs=2592000 Info: auth input: quota_rule=*:bytes=629145600 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inboxInfo: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: koshikov.n at domain.com/Spam: seq=1 uid=1933: Expunge Info: koshikov.n at domain.com/Spam: timestamp 1256902423 (Fri Oct 30 13:33:43 2009) -> 1256903611 (Fri Oct 30 13:53:31 2009) Quota_rule are different from auth section and quota root\rule section. After expire-tool expunges e-mails, the maildirsize file become: 524288000S 260686038 5487 Which is default quota_rule value, not from ldap database.