dovecot - Jul 2005 - Very slow select() command on large mailboxes

Hi,

I'm trying to decide if I could use you mail server in our enviroment which 
has about 10k users and mailboxes size about ~1GB. 
So I made a few tests and it all worked quite fast, except that select 
command on large mailboxes. The first time I select a mailbox which has 10k 
messages it takes about 4s to process. I am deliberately trying this on a 
Celeron 850Mhz, 512 MB memory and some shitty hard-drive to maximize the 
times.
So why is this so slow? Does it index the messages on select? It's even 
slower when I have some new messages in folder, more that I have more I 
wait. So I presume it must be something with indexing? Anyway, how could I 
optimize this? Why don't you index when a mail arrives instead of on select?


Thanks for help and keep doing a great job with Dovecot.

Sebastjan
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050714/f9e0a234/attachment-0003.html>

Sebastjan Trepca wrote:
> Does it index the messages on select?
Yes.
> Why don't you index when a mail arrives instead of on
> select?
You need to use Dovecot LDA ($PREFIX/libexec/dovecot/deliver) to
get messages indexed on delivery. It works like maildrop, mail is
read from stdin and recipient is given with -d parameter.

-- 
Tomi Hakala

On Thu, 2005-07-14 at 11:59 +0200, Sebastjan Trepca
wrote:
> Hi,
> 
> I'm trying to decide if I could use you mail server in our enviroment
> which has about 10k users and mailboxes size about ~1GB. 
> So I made a few tests and it all worked quite fast, except that select
> command on large mailboxes. The first time I select a mailbox which
> has 10k messages it takes about 4s to process. I am deliberately
> trying this on a Celeron 850Mhz, 512 MB memory and some shitty
> hard-drive to maximize the times.
> So why is this so slow? Does it index the messages on select? It's
> even slower when I have some new messages in folder, more that I have
> more I wait. So I presume it must be something with indexing? Anyway,
> how could I optimize this? Why don't you index when a mail arrives
> instead of on select? 
mbox or maildir? mbox can be optimized in different ways with
1.0-stable/test releases, but maildir itself needs to be rescanned all
too often..

Maildir, its easier to export and can support hardlinks. What kind of 
optimizations did you have in mind? What's the downside of mbox? 
Problem is that version 1.0 is only test release for now.

On 7/14/05, Timo Sirainen <tss at iki.fi> wrote:
> 
> On Thu, 2005-07-14 at 11:59 +0200, Sebastjan Trepca wrote:
> > Hi,
> >
> > I'm trying to decide if I could use you mail server in our
enviroment
> > which has about 10k users and mailboxes size about ~1GB. 
> > So I made a few tests and it all worked quite fast, except that select
> > command on large mailboxes. The first time I select a mailbox which
> > has 10k messages it takes about 4s to process. I am deliberately 
> > trying this on a Celeron 850Mhz, 512 MB memory and some shitty
> > hard-drive to maximize the times.
> > So why is this so slow? Does it index the messages on select? It's
> > even slower when I have some new messages in folder, more that I have 
> > more I wait. So I presume it must be something with indexing? Anyway,
> > how could I optimize this? Why don't you index when a mail arrives
> > instead of on select?
> 
> mbox or maildir? mbox can be optimized in different ways with 
> 1.0-stable/test releases, but maildir itself needs to be rescanned all
> too often..
> 
> 
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://dovecot.org/pipermail/dovecot/attachments/20050714/7424d5a0/attachment-0001.html>