Subject: re: IMAP Problem

From: Mark Crispin <[email-masked]>
Date: Sat, 18 Aug 2001 18:20:58 -0700 (PDT)

I am willing to bet that the Apple mail program is doing a LIST * (the
equivalent of "ls -R ~"), which in a filesystem-based server such as UW will
recursively list all files in your home directory.  Depending upon what you
have (and if if you have any symlinks that make a loop) this can take a very
long time.

It'll eventually stop at 20 levels of recursion, but that can be a very long
time.

The UW IMAP server is the reference implementation of IMAP, written by the
inventor of IMAP.  Programs that don't work well with UW IMAP are either not
fully compliant with the IMAP specification, or do dumb things like a
recursive list by default.

My suggestion is to get rid of the Apple mail program and use a good IMAP
client.

---

From: Chris Newman <[email-masked]>
Date: Mon, 20 Aug 2001 11:34:29 -0700

--On Saturday, August 18, 2001 18:20 -0700 Mark Crispin 
<[email-masked]> wrote:
> inventor of IMAP.  Programs that don't work well with UW IMAP are either
> not fully compliant with the IMAP specification, or do dumb things like a

When building a GUI IMAP client, it's just not obvious that the correct 
command set to build a standard mailbox tree widget is:

	LIST "" %
	LIST "" %/%

rather than

	LIST "" *

Especially when the developer is locked in the concept of "mail" as the 
(small) set of folders where the user has explicitly moved messages from 
their INBOX (a common misconception).  So "ignorant" may be a more 
appropriate adjective than "dumb".

I should also mention that the IETF has no concept of "reference 
implementation" and no references to the UW server show up in the RFCs.  So 
someone in a corporate environment that runs commercial servers can 
responsibly read the RFCs and still not know that testing with both 
open-source IMAP implementations is a good idea.  And the developers were 
probably told not to talk about the fact they were working on a mail client 
to people outside of Apple (standard stupid corporate behavior) so they 
didn't have a chance to ask.

> My suggestion is to get rid of the Apple mail program and use a good IMAP

One of the weaknesses of the IMAP community is that there are no IMAP 
clients with _both_ a good user interface and a good understanding of the 
protocol.  We seem to get only one or the other or a compromise.  We need 
more IMAP clients with really good user interfaces, so rather than telling 
people not to use the Mac OS X mail client, I believe it would be more 
constructive to lobby the OS X client people to fix any unwise use of the 
protocol.

Incidentally, I have contact addresses for some people in the OS X IMAP 
client team.  If we can gather evidence (e.g., a protocol trace) that 
they're doing LIST "" * when they should be doing LIST "" % and LIST "" 
%/%, I'd be glad to explain to them what they need to fix and why.

		- Chris

---

From: Mark Crispin <[email-masked]>
Date: Mon, 20 Aug 2001 11:42:41 -0700 (PDT)

> command set to build a standard mailbox tree widget is:

Yes.

> 	LIST "" %/%

Not necessarily.  I understand why some GUIs do this, but I don't accept the
reasoning because I consider that reasoning to be fundamentally flawed.  Since
there is no current mechanism to announce newly-created mailboxes, a client
must continuously issue %/% requests in order for that GUI functionality to
work right.

More specifically, to be considered "acceptable" in my book, any IMAP client
which has a "smart" directory open widget (that doesn't offer the widget if
the directory is empty) MUST immediately offer the widget when an external
agent creates an inferior in a previously-empty directory.

> 	LIST "" *

This is covered into RFC 2683, section 3.2.1.1

> implementation" and no references to the UW server show up in the RFCs.

That may be, but the UW implementation has been referred to as the reference
implementation innumerable times by a great many individuals, including you.

It is good practice, to say the least, for a developer to make an attempt to
identify and contact an existing user/developer community rather than blindly
implement from the RFCs.  I am hard-pressed to think of any Internet protocol
in which a successful implementation can be made with absolutely no contact or
interchange with the existing community.  Certainly not with Telnet, FTP,
HTTP, SMTP, and (especially!!) NNTP.

Not that the IMAP specification can't do a better job.  It can.  But, I think
that a developer who implements strictly from the relevant RFCs (which
includes 2683) will get closer to a usable implementation with IMAP than with
many other Internet protocols.

> clients with _both_ a good user interface and a good understanding of the

Pine has an excellent user interface and excellent understanding of the
protocol.  My old MailManager application for the NeXT also does (I still find
it quite usable), modulo lack of HTML support.

I can think of at least two actively-marketed GUI programs which have good
user interfaces and excellent understanding of IMAP (I won't name them, for
fear of inadvertantly bad-mouthing another that I've overlooked).

The problem is not the lack of good IMAP clients.

> %/%, I'd be glad to explain to them what they need to fix and why.

That would be a good idea.  Thanks.

If it is the program that I think it is, I offered to talk to those guys many
years ago and was rebuffed.  Maybe you'll have more success.

---

From: "Herr Doktor C. Lesser" <[email-masked]>
Date: Tue, 21 Aug 2001 05:04:03 +1000

Personally, I found the problem was the documentation was so overwhelming,
that it was far easier to look at an existing implementation (by sniffing
packets, etc) and build on that while reducing the status of the RFC to a
mere reference to be used while transalting the sniffer logs.

And I'm guessing that the people who wrote the software I was sniffing took
similar shortcuts.

Chinese whispers?

Or maybe I'm the only programmer too lazy to read RFC's completely?   :)

Cyrus

----- Original Message -----
From: "Chris Newman" <[email-masked]>
To: "c-client Interest List" <[email-masked]>
Cc: "Jeff Zacharias" <[email-masked]>
Sent: Tuesday, August 21, 2001 4:34 AM
Subject: re: IMAP Problem

> implementation" and no references to the UW server show up in the RFCs.
So

---

From: Mark Crispin <[email-masked]>
Date: Mon, 20 Aug 2001 12:58:49 -0700 (PDT)

> mere reference to be used while transalting the sniffer logs.

I sympathize.

However, that method is the perfect way to write a bad IMAP implementation.
Not to mention a bad RFC [2]822, MIME, SMTP, and NNTP implementation.  It also
inflicts the resulting misunderstandings upon the community at large, and this
combined with other efforts that did the same thing create a terrible burden.

Rather than do that, I recommend "Internet Email Protocols: A Developer's
Guide", by Kevin Johnson, published by Addison Wesley, ISBN 0-201-43288-9.  It
is an excellent pedagogical text for email protocols, and will show you how to
do things right.

By the way, I'm not getting any money for this endorsement, although I was a
reviewer.  I recommend it simply because it is a book that has been needed for
a very long time, and it would greatly simplify my life (as in having fewer
questions to answer and fewer problems to resolve) if more people read it.  If
someone ever comes out with a better book, I'll recommend it instead.

> And I'm guessing that the people who wrote the software I was sniffing took

Unfortunately, you are probably correct.  That one of the reasons why we have
so very many interoperability problems.

---

From: Jeff Zacharias <[email-masked]>
Date: Mon, 20 Aug 2001 19:42:04 -0400

> years ago and was rebuffed.  Maybe you'll have more success.

FYI, I just found out today that in next version of Mac OS X (10.1) due 
in September, Apple Mail has the ability to set the user mail directory.

Jeff