Hi, I just upgraded a 0.99 installation to 1.0rc7 (going from mbox to maildir along the way, but that's another story), and I'm seeing a rather fast fd quick; I already had to restart dovecot once, after a few hours. What is leaked are pipes, left open on the master dovecot process while the other end is long gone. I can't tell right now if it happens for every single login process spawned (either pop3 or imap), but at least a lot of them. FWIW, I'm running dovecot 1.0rc7 on NetBSD 2.0.3_STABLE. I'll try debugging tomorrow, but if anyone has an idea what happens in the meantime, I'm all hears. -- Quentin Garnier - cube at cubidou.net - cube at NetBSD.org "When I find the controls, I'll go where I like, I'll know where I want to be, but maybe for now I'll stay right here on a silent sea." KT Tunstall, Silent Sea, Eye to the Telescope, 2004. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 478 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20060904/0318e6e4/attachment.bin>
On Mon, Sep 04, 2006 at 01:23:55AM +0200, Quentin Garnier wrote:> Hi, > > I just upgraded a 0.99 installation to 1.0rc7 (going from mbox to > maildir along the way, but that's another story), and I'm seeing a > rather fast fd quick; I already had to restart dovecot once, after a > few hours. > > What is leaked are pipes, left open on the master dovecot process while > the other end is long gone. I can't tell right now if it happens for > every single login process spawned (either pop3 or imap), but at least > a lot of them. > > FWIW, I'm running dovecot 1.0rc7 on NetBSD 2.0.3_STABLE. I'll try > debugging tomorrow, but if anyone has an idea what happens in the > meantime, I'm all hears.... and LDAP authentication over a ldapi socket, in case it matters. -- Quentin Garnier - cube at cubidou.net - cube at NetBSD.org "When I find the controls, I'll go where I like, I'll know where I want to be, but maybe for now I'll stay right here on a silent sea." KT Tunstall, Silent Sea, Eye to the Telescope, 2004. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 478 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20060904/20bd1de0/attachment.bin>
On Mon, 2006-09-04 at 01:23 +0200, Quentin Garnier wrote:> Hi, > > I just upgraded a 0.99 installation to 1.0rc7 (going from mbox to > maildir along the way, but that's another story), and I'm seeing a > rather fast fd quick; I already had to restart dovecot once, after a > few hours. > > What is leaked are pipes, left open on the master dovecot process while > the other end is long gone. I can't tell right now if it happens for > every single login process spawned (either pop3 or imap), but at least > a lot of them. > > FWIW, I'm running dovecot 1.0rc7 on NetBSD 2.0.3_STABLE. I'll try > debugging tomorrow, but if anyone has an idea what happens in the > meantime, I'm all hears.Have you figured out anything yet? Pipes are created for logging. I haven't been able to reproduce this fd leak myself, but since several people have ran into "too many open files" problems I guess it could be because of fd leaks.. I don't really see how it could leak though. Or maybe there are problems with noticing that the other end of the pipe died with non-Linux OSes? For example are you using kqueue? I haven't tested it with that. Also do you see any "Sending log messages too fast, throttling.." messages in your logs? That part of the code is a bit weird, so it might have caused them. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20061009/21c53d34/attachment.bin>
On Mon, Oct 09, 2006 at 01:40:10AM +0300, Timo Sirainen wrote:> On Mon, 2006-09-04 at 01:23 +0200, Quentin Garnier wrote: > > Hi, > > > > I just upgraded a 0.99 installation to 1.0rc7 (going from mbox to > > maildir along the way, but that's another story), and I'm seeing a > > rather fast fd quick; I already had to restart dovecot once, after a > > few hours. > > > > What is leaked are pipes, left open on the master dovecot process while > > the other end is long gone. I can't tell right now if it happens for > > every single login process spawned (either pop3 or imap), but at least > > a lot of them. > > > > FWIW, I'm running dovecot 1.0rc7 on NetBSD 2.0.3_STABLE. I'll try > > debugging tomorrow, but if anyone has an idea what happens in the > > meantime, I'm all hears. > > Have you figured out anything yet? > > Pipes are created for logging. I haven't been able to reproduce this fd > leak myself, but since several people have ran into "too many open > files" problems I guess it could be because of fd leaks.. > > I don't really see how it could leak though. Or maybe there are problems > with noticing that the other end of the pipe died with non-Linux OSes? > For example are you using kqueue? I haven't tested it with that.Yes, I have identified the culprit as the kqueue code.> Also do you see any "Sending log messages too fast, throttling.." > messages in your logs? That part of the code is a bit weird, so it might > have caused them.Nope, no such thing in the logs. I haven't gone much further with the investigation; not using kqueue made the problem go away, so I moved on. However, I can tell that I was mostly able to reproduce the issue on a test server, but somehow it was harder (and the kernel was a different version of NetBSD). -- Quentin Garnier - cube at cubidou.net - cube at NetBSD.org "You could have made it, spitting out benchmarks Owe it to yourself not to fail" Amplifico, Spitting Out Benchmarks, Hometakes Vol. 2, 2005. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 478 bytes Desc: not available URL: <http://dovecot.org/pipermail/dovecot/attachments/20061009/25563784/attachment.bin>