Subject: Re: Load sharing imap service

From: David L Miller <[email-masked]>
Date: Wed, 12 Feb 1997 13:12:31 -0800

[I'm cross-posting this to comp.mail.imap, please excuse the
duplication --DLM]

On Tue, 11 Feb 1997, Jeffrey Goldberg wrote:

> This is more of an imap question than a pine one, but c.m.pine
> seems like the best place to ask it.

Check to see if your server has the comp.mail.imap newsgroup.  It is
pretty new, so I don't know how widely it is being propagated...

> 
> We are running IMAP2bis Service 7.8(100) on a server called, say,
> mailboxes.our.domain

I would recommend upgrading to the latest IMAP4rev1 server (v10.164). 
Some of the newer IMAP clients don't work (at least not well) with
older servers... 

> 
> Other things are also running on that machine and we are begining to
> encounter performance problems due to memory.  At a typical midday
> time (say now) we have around 35 open imap connections.  But the
> numbers can go up to double or triple that on Monday morning.  Furthermore
> we will be moving more users over to this imap server as we reorganze
> things.  So we should be expecting double or triple our current load
> on that server.
> 
> We would like to spread the load.  One thing we don't want to
> do is have users have to know which imap server their mail happens
> to be on.  We are specifically trying to move away from that sort
> of set-up.  (For Unix clients we can put in a little shell
> script that calls pine with different pine.conf files depending
> on info we get about the user, but most clients are from MS-DOS
> systems.)
> 
> So what we would like is to have one CNAME which might point to
> multiple imap servers and let the DNS rotate the names.

You may be on the right track here.  See below...

> 
> Second we would NFS mount /var/spool/mail across these servers as
> well as the directories containing the users' non-INBOX remote folders.
> 

This is a BAD idea!  You should never NFS mount /var/spool/mail.  

> I have a couple of questions:
> 
>   (1) Does this make any sense?
>   (2) Has anyone tried this?
>   (3) Does the NFS overhead cost too much compared to the imapd savings?
>   (4) Will imap clients such as pine cope with the multiple IP addresses
>       returned by DNS?  (Actually this question presupposes stuff that
>       may not be true.)
>   (5) Will imapd locking get munged if a client breaks a connection and
>       then reconnects to another host?

Locking doesn't work over NFS, period.

>   (6) If this is not the right strategy, what is?  (In addition to
>       putting more memory into mailboxes.our.domain)

Here are some possibilities:

	- Move non-IMAP processes off of your IMAP server.  We find
	that having "single function" boxes leads to much better 
	performance without unexplained slowdowns caused by some
	obscure resource contention problem.

	- Switch to a more memory-efficient mailbox format.  Using the
	default Unix (Berkeley) mailbox format, the server has to keep
	the entire mailbox image in memory.  Other formats, like
	Tenex, only keep an index in memory.  The downside is that
	this puts a heavier load on disk access, which some systems
	don't handle too well.  

If you still need to split onto multiple servers, we have found that
giving every user a CNAME to each user and using DNS to keep track of
which server a user's folders are on works well.  You may need to
upgrade your DNS server though ;-)

> 
> -j
> --
> Jeffrey Goldberg                +44 (0)1234 750 111 x 2826
>  Cranfield Computer Centre      FAX         751 814
>  [email-masked]     http://WWW.Cranfield.ac.uk/public/cc/cc047/
> Relativism is the triumph of authority over truth, convention over justice.
> 
> 
> 

Good luck!

-- 
David L. Miller <[email-masked]> | If I were two-faced, would I be
Software Engineer, Pine Development Team | wearing this one? -- Abraham
Box 354841, University of Washington     | Lincoln
4545 15th Ave NE, Seattle WA 98105, USA  |
Phone: (206)685-6240  FAX: (206)685-4045 |