Subject: Re: Tenex and scalability

From: Mark Crispin <[email-masked]>
Date: Thu, 30 Nov 1995 23:59:30 -0800

Trey, I can answer some of your comments.

On 1 Dec 1995, Trey Harris wrote:
>    However, I am concerned that if at some point the Tenex format may
>    no longer be the path we want to take, for whatever reason, there
>    be a path of migration.  Are there programs to convert from Tenex
>    to Berkeley format?  Is the Tenex specification available so that
>    we can write our own utilities to convert its format to some other
>    in the future?  Where would I find a paper documenting this mailbox
>    style?

One of the programs in the imap-utils package, mbxcvt, can convert between
any of the mailbox formats supported by c-client.  imap-utils is on the
ftp.cac.washington.edu server.

It is possible that we will migrate to a new format I am developing called
mbx format (I'm alpha-testing it now, but it's not ready for prime time),
however tenex format will never be desupported in our toolkit.

The Pine technical notes describe tenex format.

> -- I am told that the Tenex format uses Unix filesystem semantics not
>    available under NFS.  Is this true?

Yes.  It expects that file operations which are documented to be atomic
are atomic; that when file data is updated there are no obsolete copies of
the file data in some cache; and that a working facility to apply shared
and exclusive locks (and switch between shared and exclusive) exists.
None of these are true on NFS.

NFS is great as a easy way to do FTP or rcp, but it is not a true
filesystem.

>  If so, can I expect my mailserver to
>    be able to handle enough more IMAP clients with Tenex so that I can
>    convert my whole cluster to use IMAP exclusively?

This, you will have to judge for yourself.  Our experience with RS/6000
hardware and IMAP servers suggests the following:
	1) More smaller disks are better than fewer large ones.  Better
	   to have eight 1GB spindles than a single 8GB spindle.
	2) Try to have sendmail not use the same spindle as imapd.
	   Sendmail uses a lot of disk bandwidth and you don't want it
	   fighting with your imapds.  The same thing holds for syslog
	   and other daemons.
	3) File fragmentation tends to be a problem on AIX.  Since tenex
	   files are accessed with random access I/O, they are almost
	   never rewritten (and hence defragmented).  You may need to
	   write a defragmentation tool (shudder!) to defragment mail
	   files every so often.
	4) Don't swap on the same disk that does mail files either.

>    It seems that the 50-60 imapd processes it runs now are about all
>    it can manage in the Berkeley format.  Is it likely that switching
>    to Tenex would allow the mailserver to run the 400-700 imapds
>    needed with similar performance?

It all depends upon how much memory is available.  The computer center
splits its user community (the entire campus) across a dozen RS/6000
servers.  They don't want to run out.

imapd does not need much CPU.  It's all a matter of memory and disk
bandwidth.

-- Mark --

DoD #0105, R90/6 pilot, FAX: (206) 685-4045  ICBM: N 47 39'35" W 122 18'39"
Science does not emerge from voting, party politics, or public debate.