Subject: re: Object-oriented IMAP/c-client implementation?

From: [email-masked] (William Chung)
Date: Tue, 25 Jul 1995 09:51:31 -0400

On Jul 21,  1:23pm, Mark Crispin wrote:
} Subject: re: Object-oriented IMAP/c-client implementation?
} ...
} The bottom line is: it's difficult for me to see what technical benefit is
} obtained by doing this.  Ordinary C, which c-client is written in, provides a
} base that is much more portable (C meshes with C++, Obj-C, SmallTalk, Lisp,
} etc. much better than any of the OO languages mesh with any other OO
} language); it's faster, and pretty much all of the benefits of OO programming
} are already in the current c-client.
}
} It's certainly possible to re-write c-client in an OO language, and rather
} straightforward at that, but it's also a lot of work with a strong likelihood
} that you won't get any particular benefit from doing it.
} ...
}-- End of excerpt from Mark Crispin

Thanks for the detailed response Mark!

Your points about c-client already having almost all of the benefits of
OO programming are duly noted.

I was interested in an OO c-client because of a desire to mesh c-client
with some OO code.  Mixing OO code with c-client shouldn't be much of an
issue given you had no problems calling c-client from an OO implementation
of MailManager.

- William Chung.
Multimedia Messaging
IBM Research, Hawthorne, NY
[email-masked]

---

From: "Syd Logan" <[email-masked]>
Date: Mon, 3 Mar 1997 22:38:30 -0800

-----BEGIN PGP SIGNED MESSAGE-----

First, let me quote a few lines from RFC 2060:

"6.1.2 NOOP Command

...

Since any command can return a status update as untagged data, the NOOP command
can be used as a periodic poll for new messages or message status updates...

...

S: * 14 FETCH (FLAGS (\Seen \Deleted))

...

7.4.2 FETCH Response

The FETCH response returns data about a message to the client... This response
occurs as the result of a FETCH or STORE command, as well as a unilateral
server decision (e.g. flag updates)."

The above implies (to me at least) that the IMAP4 server will notify interested
parties that a message's flags have changed (interested parties = all MUAs that
have a connection to the IMAP4 server and subscribed to the mailbox containing
the message which had its flags changed).

I have a couple of IMAP4-enabled Z-Mail Unix apps open, both pointing to the
same IMAP server, and both subscribed to the same mailbox. In response to a
delete operation made in Z-Mail GUI, I call c-client's mail_setflag() routine,
passing a flag string of \Deleted. On the net, I see:

TCP:

           0: 0800 201d fb41 0800 0918 2d8e 0800 4500    .. .ûA....-...E.
          16: 004e 9df2 4000 4006 eba7 9c1b 3cd0 9c1b    .N..@.@.....<...
          32: 3c09 0cdb 008f 1fa1 cadb 71d0 6942 5018    <.........q.iBP.
          48: 8000 b76a 0000 4130 3032 3836 2055 4944    ...j..A00286 UID
          64: 2053 544f 5245 2039 202b 466c 6167 7320     STORE 9 +Flags
          80: 285c 4465 6c65 7465 6429 0d0a              (\Deleted)..

If I late send an EXPUNGE, the message is deleted (and I see the expunges on
the net too).

Similarly, if I double click a message header in the Z-Mail GUI in order to
read the message, I send a \Seen flag change, and I see this as well in the
network sniff.

What I never see is a FETCH flags sent to the other MUA sharing the mailbox. My
view of the world is that multiple clients with the same mailbox open should
display the same reality, e.g. that of the mailbox in its current state on the
server. If one client changes the flag(s) of a message in a remote mailbox,
then all clients that are connected to the mailbox should receive notification
of this change, and update their GUIs via an unsolicited FETCH FLAGS response.
Again, I gathered this from what I have read in FRC 2060 and the c-client docs,
plus what I would expect as a user. The real life situation is that of a user
that leaves work with Z-Mail Unix running, goes home and accesses the mailbox
with our Z-Mail Pro from his Windows box, and comes back to work the next day -
he should see (IMHO) the chjanges made while at home in the Z-Mail Unix GUI
(sorry for the shameless plugs of Z-Mail :-)

Perhaps I have a bug in my implementation, or I am grossly misinterpreting the
protocol, or I have a twisted reality with regard to the handling of
concurrently opened mailboxes, or there is a deficiency in c-client or the UW
imapd, or some combination of the above.

I'd appreciate any comments or guidance in this area than any of you wish to
offer.

Regards,

syd

-- 
Syd Logan
Senior Software Engineer, Z-Mail for Unix
NetManage, Inc.

http://www.users.cts.com/crash/s/slogan

My PGP public key is available at:

http://www-swiss.ai.mit.edu/~bal/pks-toplev.html - search for "Syd Logan"

---

From: Mark Crispin <[email-masked]>
Date: Tue, 4 Mar 1997 00:11:58 -0800 (PST)

There certainly are unsolicited FLAGS updates in the c-client imapd.

However, you must be using a mailbox format that supports multiple
simultaneous write access.  The standard UNIX format is not such a format; if
your mailbox is in standard UNIX format then one imapd process has write
access and all other imapds are working with a read-only snapshot of the
mailbox.  A read-only snapshot won't get any updates.

---

From: Erik Forsberg <[email-masked]>
Date: Tue, 4 Mar 1997 14:45:54 -0800 (Pacific Standard Time)

> mailbox.  A read-only snapshot won't get any updates.

This has never been my understanding of what Read-Only means. The way I 
read RFC 2060 is that a mailbox that is selected read-only means that NO
actions performed by the user that IS selecting the mailbox is allowed 
to change the permanent state of this mailbox. Other users selecting 
the same mailbox read-write certainly could change the permanent 
state and this SHOULD be reported to other users. If this was 
interpreted differently, we would end up with a situation that implies 
that as long as there is one user selecting a mailbox read-write, we 
could not allow anyone selecting the same mailbox as read-only.

It doesnt make sense that selecting a mailbox for read-only is just an 
excuse for avoiding to keep track of what the state of a mailbox is. 
That could be quite confusing for a client (as noted in the earlier msg)

----------------------
Erik Forsberg
[email-masked]

---

From: Mark Crispin <[email-masked]>
Date: Tue, 4 Mar 1997 14:46:05 -0800 (PST)

On Tue, 4 Mar 1997 14:45:54 -0800 (Pacific Standard Time), Erik Forsberg
wrote:
> This has never been my understanding of what Read-Only means.

Don't confuse "read-only" in the IMAP sense with "read-only" in the c-client
sense.

With the standard UNIX mailbox format, only ONE process is able to write into
the file and know about the current state of the file.  This is an exclusive
lock.  The only choices in such circumstances are:
 1) access is denied
 2) take a snapshot and look at the snapshot read-only.
 3) forcibly wrest the exclusive lock from its owner, causing the erstwhile
     owner to die or to become read-only looking at a snapshot as of the time
     of wresting.

Yes, this is mediocre.  That's why we don't use the standard UNIX mailbox
format at UW.  We use a format which permits shared read/write access.

> The way I
> state and this SHOULD be reported to other users.

Yes, if at all possible, you should do this.

> If this was

That's one interpretation of what happens when read-write is exclusive due to
limitations of the mail store.  It's interpretation (1) that I listed above.
But there's also (2) and (3).  c-client does (3).

> excuse for avoiding to keep track of what the state of a mailbox is.

If the mail store can't do it, it can't do it.  Either the server is allowed
to do something (such as not report updates in this state) that at least
conforms to the syntax of the protocol, or there's no server.  "We don't
support IMAP because IMAP is too inflexible for our needs.  Use our wonderful
PropertaryMAP instead."

> That could be quite confusing for a client (as noted in the earlier msg)

As I noted above, we don't use standard UNIX format for this reason and other
reasons.

---

From: "Syd Logan" <[email-masked]>
Date: Tue, 4 Mar 1997 18:03:35 -0800

-----BEGIN PGP SIGNED MESSAGE-----

On 1997-3-4 at  2:46pm Mark Crispin write:
> Subject Re: re: Unsolicited FLAGS updates

(stuff snipped)

> But there's also (2) and (3).  c-client does (3).

So, if client A starts (read-write), then client B starts, then client B will
become read-write and client A will become read-only? Option (3) above lists
"owner to die" or "become read-only". Just want to clarify which it is.

>-- End of excerpt from Mark Crispin

---

From: Mark Crispin <[email-masked]>
Date: Tue, 4 Mar 1997 21:01:15 -0800 (PST)

On Tue, 4 Mar 1997 18:03:35 -0800, Syd Logan wrote:
> So, if client A starts (read-write), then client B starts, then client B
> will become read-write and client A will become read-only?

Yes.  And client A will no longer see updates.

Like I said, this is why we don't use UNIX mailbox format.  It was never
designed for shared read/write/update access.  I only support UNIX mailbox
format for those folks who insist upon compatibility with the past.

---

From: Subu Rama <[email-masked]>
Date: Wed, 5 Mar 1997 08:45:29 -0500 (EST)

> "owner to die" or "become read-only". Just want to clarify which it is.

"Becomes read-only". e.g. the pine MUA on Unix. I start up pine in one 
window, minimize it, forget that I already have one pine session running
and start another one in another window, the first one becomes read-only.

I also see this case:
  I start up pine at work (on a HP-UX box), keep reading ...
  My wife starts up Eudora on our Mac at home (Uses POP, not IMAP)
accessing the same mailbox, and ...
  pine gives up (dies) with access errors.

My mailbox is on a Solaris 2.5 system (UNIX mailbox format, of course).

Just curious, Mark: Could you give more details on the non-UNIX
mailbox format you use at UW ?

Thank you

Subu Rama
[email-masked]

---

From: Mark Crispin <[email-masked]>
Date: Wed, 5 Mar 1997 12:10:28 -0800 (PST)

> and start another one in another window, the first one becomes read-only.

That is correct.  And the first one will not see any updates.  The only way to
see updates is to close the read-only stream and open the mailbox anew.

>   pine gives up (dies) with access errors.

This is because your system's POP server does not respect the locks that would
cause c-client to go read-only.  It just goes ahead and assumes that it can
write into the mailbox.

c-client (whether in Pine or in an imapd) sees that something happened to the
mailbox.  It doesn't know what happened, so the best that it can do is give up
and say "access error".

> mailbox format you use at UW ?

There are several choices of format that support multiple read/write, all
supported by c-client.
 . tenex and mtx format are what were used on the TOPS-20 system and by mm on
    UNIX.  The only difference is that tenex uses UNIX-style newlines and
    mtx uses Internet-style (CR/LF) newlines.  Yes, I know that the names
    should be the other way around.
 . mbx, our current favorite, is a descendent of mtx format.  It uses Internet
    style newlines (it gets a speed advantage from not having to convert
    newlines) and has unique identifier support.  It's the fastest and most
    featureful of all our existing formats.
 . mh format is what's used by the mh program.  It doesn't have full c-client
    type features and is a bit slow.
 . mx format is an advanced version with full c-client type features, but it
    is a bit slow.  I may be working on improving the performance of mx.

At the present time, there isn't much documentation about all of this except
for some older information in the Pine technical notes.  I intend to write up
some documentation in the future.

---

From: "Syd Logan" <[email-masked]>
Date: Wed, 5 Mar 1997 14:38:05 -0800

-----BEGIN PGP SIGNED MESSAGE-----

On 1997-3-5 at 12:10pm Mark Crispin write:
> Subject Re: Unsolicited FLAGS updates

This is a paradigm that initially I believe I can work with (see below) - now
that I understand it. Thanks Mark for straightening me out.

(stuff snipped)

> some documentation in the future.

That'd be great. Let me make sure I have it straight - if the server I connect
to happens to base itself upon one of the more advanced formats that you
mentioned (e.g. mx format), then both MUAs will never see stream->rdonly go to
1, and flags changes will be distributed by the server to all MUAs who have the
same mailbox open (as long as the server is doing the right thing, that is)?

---

From: Mark Crispin <[email-masked]>
Date: Wed, 5 Mar 1997 14:45:37 -0800 (PST)

On Wed, 5 Mar 1997 14:38:05 -0800, Syd Logan wrote:
> That'd be great. Let me make sure I have it straight - if the server I
> connect to happens to base itself upon one of the more advanced formats that
> you mentioned (e.g. mx format), then both MUAs will never see stream->rdonly
> go to 1, and flags changes will be distributed by the server to all MUAs who
> have the same mailbox open (as long as the server is doing the right thing,
> that is)?

Yes.

---

From: David L Miller <[email-masked]>
Date: Thu, 6 Mar 1997 14:21:22 -0800 (PST)

> same mailbox open (as long as the server is doing the right thing, that is)?

Yes, but one caveat is that expunge is disallowed in mtx, tenex, and
mbx formats when multiple streams have a folder open.  The mx format
was designed to allow expunge while a folder is open multiple times. 
Unfortunately the performance penalty of going to this format is too
severe to be practical in most situations... 

-- 
David L. Miller <[email-masked]> | 640K ought to be enough for
Software Engineer, Pine Development Team | anybody. -- Bill Gates, in 1981
Box 354841, University of Washington     |
4545 15th Ave NE, Seattle WA 98105, USA  |
Phone: (206)685-6240  FAX: (206)685-4045 |