Subject: mbox vs mbx decisions

From: <[email-masked]>
Date: Thu, 3 Jun 2004 12:20:11 -0500 (CDT)

This has probably been discussed many times over, but I cannot find
references to it...

We have historically used the mbox format and have developed tools for
interacting, analyzing, and managing inboxes.  This has worked well for
admins and users.

There has been some pressure in the recent past to enhance the user's
email experience by providing even faster access to their inbox and
simultaneous access (only one request so far on the latter).

The question is:  When does the mbx performance gains over mbox begin?

We have a 3MB limit on inbox size, it gets moved to the user's ~/Mail dir
nightly if it exceeds the limit.  The user can access their inbox and saved
mail normally via IMAP.  An access of a 3MB mbox inbox is pretty quick on
a 4-CPU Solaris 8 system with 16GB RAM (Sun V440).

Would switching from mbox to mbx significantly increase access time?
A conversion would also imply a rework of tools and procedures for about
26K accounts spread over 4 systems.

-- 
scott hollatz                                        net [email-masked]
information technology systems and services          tel +1 218 726 8851
university of minnesota duluth mn usa                fax +1 218 726 7674
                                                                         --
                                            "gabba gabba hey" - the ramones

-- 
------------------------------------------------------------------
 For information about this mailing list, and its archives, see: 
 http://www.washington.edu/imap/c-client-list.html
------------------------------------------------------------------

---

From: Mark Crispin <[email-masked]>
Date: Thu, 3 Jun 2004 11:05:36 -0700 (PDT)

> The question is:  When does the mbx performance gains over mbox begin?

Mbx performance over traditional UNIX format is in all aspects.  Whether 
or not these benefits are worth the cost to you of conversion is for you 
to decide.

Here are some of the differences between traditional UNIX format and mbx 
format:

A program opening a traditional UNIX format mailbox for read must read 
every byte in the file to locate the "From " lines.

A program opening an mbx format for read must read the 2K internal header, 
and then 64 bytes per message.

A program updating a flag in a traditional UNIX format mailbox must 
rewrite the message header holding the Status:/X-Status:/X-Keywords: 
lines, and probably must also rewrite the message body and all subsequent 
messages in the mailbox since the position of this body probably shifted. 
Because of this, c-client defers message status updates until the next 
checkpoint (check, expunge, or close) so it can do multiple updates at 
once.

A program updating a flag in an mbx format mailbox only needs to change a 
single byte of the file.  Because of this, c-client does message status 
updates immediately.

Traditional UNIX format mailboxes use UNIX-style LF-only newline 
conventions.  Consequently, texts must be converted to CRLF-format, and 
size calculations must take into account this conversion (the entire text 
must be scanned for bare LFs and the count incremented for each).  Extra 
memory is required for this conversion.

Mbx format mailboxes use Internet-style CRLF newline conventions, 
consequently no text conversions are necessary and size calculations are 
simply the difference between two pointers.

Newly-delivered messages in traditional UNIX format must be rewritten when 
first read to set up Status:, X-Status:, X-Keywords:, and X-UID: lines in 
the message header to hold message metadata.  These lines must also be 
filtered out prior to delivering header strings to the client.

Newly-delivered messages in mbx format have the message metadata in the 
internal format header, not in the message, and the space is preallocated 
so no rewrite is necessary.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.