Subject: mailutil and MUA flags

From: Nicolas Kowalski <[email-masked]>
Date: Wed, 7 Apr 2004 20:54:20 +0200 (MEST)

Hello.

When using the mailutil utility to copy mail/move from a folder to
another, some MUA-specific flags are lost, like the Junk or label flags
that Mozilla uses.

Is this a bug/feature ?

Thanks.

PS: version in use is 2002e.

-- 
Nicolas
-- 
------------------------------------------------------------------
 For information about this mailing list, and its archives, see: 
 http://www.washington.edu/imap/c-client-list.html
------------------------------------------------------------------

---

From: Mark Crispin <[email-masked]>
Date: Wed, 7 Apr 2004 11:59:30 -0700 (PDT)

> another, some MUA-specific flags are lost, like the Junk or label flags

Mailutil tries to copy keywords, but (depending upon the mail store) some 
keywords may not copy in the IMAP server.  Newer versions of UW imapd do a 
*much* better job at preserving keywords on copy, but there remain some 
limitations.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Nicolas Kowalski <[email-masked]>
Date: Thu, 8 Apr 2004 15:22:40 +0200 (MEST)

> *much* better job at preserving keywords on copy, but there remain some

Thanks for your reply.

I just tried with 2004-RC7. mailutil (copy) gave me a lot of errors
like:

warning: Unknown flag: Junk

And the Junk flags were not copied to the destination folder. Is this
one of the limitations you wrote about ?

-- 
Nicolas

---

From: Mark Crispin <[email-masked]>
Date: Thu, 8 Apr 2004 08:14:43 -0700 (PDT)

> one of the limitations you wrote about ?

Yes.  A copy operation won't create a new keyword if the destination does 
not already have it.

-- Mark --

---

From: Mark Crispin <[email-masked]>
Date: Thu, 8 Apr 2004 08:19:32 -0700 (PDT)

On Thu, 8 Apr 2004, Mark Crispin wrote:
> Yes.  A copy operation won't create a new keyword if the destination does not 
> already have it.

To be more specific: each mailbox has a list of defined keywords used with 
that mailbox.  New keywords are defined by setting a previously-unused 
keyword in a message while you have the mailbox open.

Appending new messages to the mailbox does not define new keywords, and 
it's not clear from the IMAP specification that it should.  So if the 
source has a keyword that is not defined in the destination, that keyword 
is likely not to be copied.

-- Mark --

---

From: Rich Graves <[email-masked]>
Date: Thu, 8 Apr 2004 11:56:58 -0400 (EDT)

> it's not clear from the IMAP specification that it should.  So if the 

Hmm.

I guess this is more for the imap list, but this sounds like a bug in the
IMAP specification to me. Was there ever any working group discussion of
this? Is there any good reason *not* to want to create new keywords upon
copy?

Is there a good place in imapd we can hack "Junk" into the default keywords
available for all newly created (in our case, mbx format) mailboxes?

---

From: Mark Crispin <[email-masked]>
Date: Thu, 8 Apr 2004 10:26:19 -0700 (PDT)

> this? Is there any good reason *not* to want to create new keywords upon

I don't think that anyone ever considered it.  The actual text in RFC 3501 
says:
          The PERMANENTFLAGS list can also include the special flag \*,
          which indicates that it is possible to create new keywords by
          attempting to store those flags in the mailbox.

It's not clear whether "store those flags in the mailbox" includes flags 
being copied by a COPY command or set by an APPEND command, or if it is 
specific to an STORE command.  It's easier to argue for this with APPEND 
rather than COPY.

There is a list of all valid keywords in mailbox metadata.  That allows a 
client to build a menu of usable keywords.

Even if the specification gets changed, it will be a long time before UW 
imapd will be able to create flags in a mailbox as a result of a COPY or 
APPEND.  Creating flags requires being able to access the mailbox 
metadata, and COPY/APPEND just append to the end of the mailbox without 
touching mailbox metadata.

> available for all newly created (in our case, mbx format) mailboxes?

Yes, just write a dummy message, select the mailbox, and store the flags.

-- Mark --

---

From: Rich Graves <[email-masked]>
Date: Thu, 8 Apr 2004 15:45:14 -0400 (EDT)

> Yes, just write a dummy message, select the mailbox, and store the flags.

That's ok if you're writing a new proprietary client, but not for users 
creating mailboxes the normal way.

I meant new, empty mailboxes, i.e., this list, which seems to be specified
by rfc3501 section 2.3.2.

  Pre-authenticated user rcgraves moe.unet.brandeis.edu IMAP4rev1 2003.339 
  at Thu, 8 Apr 2004 13:32:24 -0400 (EDT)
. create foobar2
. OK CREATE completed
. select foobar2
* 0 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1081445606] UID validity status
* OK [UIDNEXT 1] Predicted next UID
* FLAGS (\Answered \Flagged \Deleted \Draft \Seen)
* OK [PERMANENTFLAGS (\* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags
. OK [READ-WRITE] SELECT completed

Yuck. Looks like those are hardcoded and do not appear in the mbx file at 
all. Only newly defined ones do. Any hope?

I do see that

. store 999999 +FLAGS (Junk)

Succeeds in creating the "Junk" flag even if no message 999999 exists. But
that seems ugly, and how would I automate it upon mailbox creation?

Incidentally, the error message you get when trying to DELETE a currently
selected mailbox is a little misleading. It's not "another" process, it's
the current process. This annoys users when they try to delete an empty
mailbox from a GUI client. In some cases it can take some serious
gymnastics to shift-click and delete a mailbox without SELECTing it. I
always assumed this was a silliness about the GUI client, but it's the
server. So should client authors "have known" that they need to unselect a
mailbox otherwise in use? What about servers (including uw imapd until
recently) that don't implement UNSELECT?

. delete foobar2
. NO DELETE failed: Mailbox foobar2 is in use by another process
. unselect
. OK UNSELECT completed
. delete foobar2
. OK DELETE completed
-- 
Rich Graves <[email-masked]>
UNet Systems Administrator

---

From: Mark Crispin <[email-masked]>
Date: Thu, 8 Apr 2004 13:40:24 -0700 (PDT)

> creating mailboxes the normal way.

That doesn't make any sense.  If your client has a list of keywords that 
it wants in all mailboxes, it is perfectly capable of acting as I suggest.

> all. Only newly defined ones do. Any hope?

Of course \ flags are hardcoded.  That's a requirement of the IMAP 
specification.  You'll notice that keywords don't start with a \.

> Succeeds in creating the "Junk" flag even if no message 999999 exists.

You can not count upon that behavior.  It's a bug that UW imapd does an 
action on a command that returns a tagged NO.  Now that I know about that 
bug, I'm fixing it.  Thank you for letting me know about it.

> mailbox from a GUI client.

Yes, some GUI clients have bugs.

> So should client authors "have known" that they need to unselect a
> mailbox otherwise in use?

Yes.  Of course.

If you think about it, it doesn't make sense to have a selected mailbox 
that has been deleted, given the other semantics of mailboxes (which are 
different from UNIX file semantics).

> What about servers (including uw imapd until

The client shouldn't have selected the mailbox first if it was going to 
delete it.

It isn't guaranteed that error messages will always be precise for every 
possible thing that goes wrong.  Often, as in this case, a 
well-intentioned effort to provide user-friendly information, as opposed 
to unintelligible (but precisely accurate) errors, can result in an error 
message that is not strictly accurate.

The library does not know what has it open, nor does it have a reasonable 
way to find out.  It just knows that it's locked as an open mailbox.  It 
guesses that the culprit is another process, since surely no program would 
be foolish enough to delete something that it has open.

The true error condition is "Mailbox locked".  Most users would have a 
completely incorrect understanding of what that means, and how to remedy 
it.

-- Mark --