Subject: reopening a stream with OP_HALFOPEN

From: "Tomas Pospisek's Mailing Lists" <[email-masked]>
Date: Wed, 1 Dec 2004 11:33:06 +0100 (CET)

I've got reports [1] against Debian's version of c-client 2002e that when 
opening a stream, that is not OP_HALFOPEN (such as OP_READONLY or NIL), 
c-client will actually close and open the stream again (thus requiring the 
user to re-authenticate).

Is this:

* a known problem?
* a feature with a rationale?
* fixed in c-client 2004?

Any other comment?
*t

[1] http://bugs.debian.org/257418

--
-----------------------------------------------------------
   Tomas Pospisek
   http://sourcepole.com -  Linux & Open Source Solutions
-----------------------------------------------------------
-- 
------------------------------------------------------------------
 For information about this mailing list, and its archives, see: 
 http://www.washington.edu/imap/c-client-list.html
------------------------------------------------------------------

---

From: Mark Crispin <[email-masked]>
Date: Wed, 1 Dec 2004 08:38:23 -0800 (PST)

> user to re-authenticate).

I assume that you're talking about recycling an already-open stream?

A close and open will happen if c-client determines that the new mailbox 
name is not compatible with the existing stream.

If you recycle a non-halfopen stream, and decide that you want the stream 
to be halfopen now, a close/open is required unless the server supports 
the UNSELECT capability.  That's the only reliable way to get a halfopen 
session if the server does not have UNSELECT.

If you don't care if the stream is halfopen or not, then you probably 
should not call mail_open() with the OP_HALFOPEN flag to recycle the 
stream.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: "Tomas Pospisek's Mailing Lists" <[email-masked]>
Date: Wed, 1 Dec 2004 18:10:11 +0100 (CET)

> I assume that you're talking about recycling an already-open stream?

Yes.

> A close and open will happen if c-client determines that the new mailbox name 
> is not compatible with the existing stream.
>
> If you recycle a non-halfopen stream, and decide that you want the stream to 
> be halfopen now, a close/open is required unless the server supports the 
> UNSELECT capability.  That's the only reliable way to get a halfopen session 
> if the server does not have UNSELECT.
>
> If you don't care if the stream is halfopen or not, then you probably should 
> not call mail_open() with the OP_HALFOPEN flag to recycle the stream.

I'm looking at messages in readonly mode in order to determine which ones 
need to be synchronized between two sites.

Then I'm flagging those as deleted that have been removed on one side.

After that I'm appending new messages to mailboxes at both sides. In order 
to do this without having the "Status: O" set by c-client I need to change 
"down" into OP_HALFOPEN mode.

Thanks for the answer!
*t

---

From: Mark Crispin <[email-masked]>
Date: Wed, 1 Dec 2004 09:18:56 -0800 (PST)

On Wed, 1 Dec 2004, Tomas Pospisek's Mailing Lists wrote:
> After that I'm appending new messages to mailboxes at both sides. In order to 
> do this without having the "Status: O" set by c-client I need to change 

Thanks for explaining so clearly what you're doing.  I understand the 
issue.

If the mailbox is opened readonly then there shouldn't be any "Status: O" 
set.  c-client doesn't do this; the IMAP server does.  If one of the IMAP 
servers is UW imapd, then indeed "Status: O" will happen when messages are 
appended to a mailbox that is open readwrite, since the readwrite session 
will see the messages.  But that won't happen if it's open readonly.

So, if "Status: O" is being written then I think that must be something 
that is happening in some other server.  Or perhaps you don't really have 
the mailbox open readonly (perhaps because you don't always see new mail 
in readonly sessions).

Anyway, the correct long-term fix is to upgrade to servers that support 
UNSELECT.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

---

From: "Tomas Pospisek's Mailing Lists" <[email-masked]>
Date: Wed, 1 Dec 2004 18:59:35 +0100 (CET)

On Wed, 1 Dec 2004, Mark Crispin wrote:

> Thanks for explaining so clearly what you're doing.  I understand the issue.
>
> So, if "Status: O" is being written then I think that must be something that 
> is happening in some other server.  Or perhaps you don't really have the 
> mailbox open readonly (perhaps because you don't always see new mail in 
> readonly sessions).
>

This is not an option since the tool I maintain (mailsync) wants to 
allow the user to access all forms of mailboxes uniformly, be they IMAP 
servers with or without UNSELECT or any kind of local mailboxes...

... but - are you saying that I don't even need to change into HALFOPEN 
mode in order to be able to append messages to any kind of mailbox 
without the "Status: O" flag being set? That I just have to keep the 
stream in READONLY mode?

I seem to remember you've told me otherwise last time I asked but I can't 
dig up the mail in question (here [1] is one such email, but it doesn't 
talk about READONLY).

Thanks again,
*t

[1] http://www.mail-archive.com/[email-masked]/msg00220.html

PS: I'd be nice to link to that searchable online archive from the UW
     c-client page.

---

From: Mark Crispin <[email-masked]>
Date: Wed, 1 Dec 2004 10:16:20 -0800 (Pacific Standard Time)

On Wed, 1 Dec 2004, Tomas Pospisek's Mailing Lists wrote:
> ... but - are you saying that I don't even need to change into HALFOPEN mode 
> in order to be able to append messages to any kind of mailbox without the 
> "Status: O" flag being set? That I just have to keep the stream in READONLY 
> mode?

Stream readonly status refers to operations on the opened mailbox (the 
selected mailbox in IMAP terms); it affects such operations as 
setting/clearing flags and expunging.  It does not in any way affect 
operations that work on a mailbox by name, such as append.

So I think that the answer to your question (if I understand it correctly) 
is "yes".

-- Mark --

---

From: Mark Crispin <[email-masked]>
Date: Wed, 1 Dec 2004 10:19:29 -0800 (Pacific Standard Time)

> [1] http://www.mail-archive.com/[email-masked]/msg00220.html

That message did not consider the case of the stream being opened 
readonly.

-- Mark --

---

From: "Tomas Pospisek's Mailing Lists" <[email-masked]>
Date: Thu, 2 Dec 2004 11:20:16 +0100 (CET)

On Wed, 1 Dec 2004, Mark Crispin wrote:

>> mode in order to be able to append messages to any kind of mailbox without 
>> the "Status: O" flag being set? That I just have to keep the stream in 
>> READONLY mode?
>
> So I think that the answer to your question (if I understand it correctly) is 
> "yes".

Hmm - this is starting to be interesting for me. So this means that I 
could switch from mailbox to mailbox by opening them READONLY and that 
this would not issue internally an open/close on the stream and thus not 
require a re-authentication?

So in the general case, that is for local and remote mailboxes of a random 
format supported by c-client I can:

 	foreach mailbox in mailboxes_at_a_location {
 	  stream = open(mailbox, READONLY);	# first step (read)
 	  read_mails(stream)			#

 	  stream = open(mailbox, READWRITE);	# second step (expunge)
 	  flag_removed_mails(stream);		#
 	  expunge(stream);			#

 	  stream = open(mailbox, READONLY);     # third step
 	  append(new_mails, mailbox-name);	#
 	}

Q1: If I ommit the second step, then I can guarantee that all email stati
     are maintained, especially "Status: O" will not be set by c-client or
     the IMAP server no matter what format the mailbox is in. Correct?

Q2: It would be interesting if there exists a way to delete emails in a
     mailbox (that is the second step) without affecting the "Status: "
     respectively the flags of any other email in that mailbox or any other
     mailbox (this again for the general case).

Q3: Or if there is a way to remove "Status: O" after the fact
     (afterwards). As I understand though this is not possible for the
     general case. Correct?

     If any of the two (preferable the later method - removing "Status: O")
     could be achieved, then mailsync could be implemented in a "perfect"
     way.

Thanks,
*t

---

From: Shawn Walker <[email-masked]>
Date: Thu, 02 Dec 2004 10:33:33 -0600

How do I manage custom flags for IMAP messages.  I want to be able to 
set "\Forwarded", "\ReplyToAll", etc on a message and be able to see 
them using c-client.

Thanks,
Shawn

---

From: Mark Crispin <[email-masked]>
Date: Thu, 2 Dec 2004 08:54:11 -0800 (PST)

On Thu, 2 Dec 2004, Shawn Walker wrote:
> How do I manage custom flags for IMAP messages.  I want to be able to set 
> "\Forwarded", "\ReplyToAll", etc on a message and be able to see them using 
> c-client.

You can not use "\Forwarded" or "\ReplyToAll".

You can, however, use "Forwarded" and "ReplyToAll".  These are keyword 
flags, and work just like other flags.

All flags which start with \ are reserved to the IMAP protocol for 
definition and are not available for user definition.

-- Mark --

---

From: Mark Crispin <[email-masked]>
Date: Thu, 2 Dec 2004 18:20:59 -0800 (Pacific Standard Time)

On Thu, 2 Dec 2004, Tomas Pospisek's Mailing Lists wrote:
> Hmm - this is starting to be interesting for me. So this means that I could 
> switch from mailbox to mailbox by opening them READONLY and that this would 
> not issue internally an open/close on the stream and thus not require a 
> re-authentication?

You can switch from mailbox to mailbox regardless of whether or not they 
are readonly.  The only thing that readonly does is prevent any changes 
to the mailbox (possibly including loading new messages).

> 	  append(new_mails, mailbox-name);	#

If I were you, I would try to determine an algorithm that permits just one 
readwrite open per mailbox.

>    the IMAP server no matter what format the mailbox is in. Correct?

"Status: O" is an artifact of traditional UNIX mailbox format, and is 
meaningless in other formats.  If you mean the \Recent flag, the answer is 
"yes".

>    mailbox (this again for the general case).

There isn't.

>    (afterwards). As I understand though this is not possible for the

There isn't, and correct.

-- Mark --