Subject: More on /SEEN and /DELETED... (sigh!)

From: Terry Gray <[email-masked]>
Date: Sat, 16 Oct 1993 23:35:56 -0700 (PDT)

Folks,
At last, I have completed reviewing the recent /SEEN and /DELETED thread. 
I'm sending this response to both the imap and c-client lists, as I think
some of the issues relate to both.  In this set of messages, there seem to
have been four major themes: 

 1. Are per-session flags useful?  If so, how should they be implemented?
 2. What are IMAP "BBOARDS", anyway?
 3. What different classes of mailboxes and flag sets are there?
 4. What have those UW guys been smoking?  (Clearly they are a bunch of 
    idiots for challenging the traditional news reading paradigm that 
    messages should automatically disappear after one reading.)

I'd like to make brief comments on the first three, then concentrate my 
fire on item 4...

--------------------------------------------------------------------------

TOPIC 1: Per-session flags.  

I'm actually not a big fan of per-session flags.  I would not violently
oppose their existence, but I would tend to avoid using them if I could,
on the grounds that flags that are not preserved across sessions tend to
confuse users.  Note that the questions of where *persistent* flags might
be stored (with the mailbox, or in an auxiliary file) and who manages them
(server or client) are completely orthogonal to the question of whether
or not transient (per-session) flags should exist. 

--------------------------------------------------------------------------

TOPIC 2: BBOARDS... what are they *really*?

I think of BBOARDS as a way of defining an alternate namespace.  The
problem is that having precisely *two* namespaces defined in IMAP can't
possibly be the right general solution.  It is either too few or one too
many.  I personally believe that BBOARDS needs to be re-thought in the
context of namespace selection, especially for non-filesystem namespaces. 
We need a clean way to either (a) select amongst possibly many drivers and
their namespaces, or (b) cleverly map a diversity of them so that we can
pretend we only have one namespace.  It would not break my heart if the
present BBOARDS construct was superseded or obsoleted.

--------------------------------------------------------------------------

TOPIC 3. Classes of mailboxes; private vs. global flags.

Here are my assumptions:
 -A Mailbox = [Msg Data + Global Flags] OR [Msg Data + Private Flags]
 -Global (per-mailbox) flags may or may not be shared.
 -Private (per-user) flags, if they exist, are never shared.
 -A user views a mailbox EITHER with Global or Private flags, NEVER both.
 -A user with RW/EXPUNGE rights will use Global flags, never Private flags.
 -Global and Private flag sets may include any defined IMAP flag, however...
 -Implementation constraints may limit which flags can actually be stored.
 
Given these assumptions, we can now identify several classes of interesting 
mailboxes. I propose a taxonomy based on three questions: 

 1. Is message store RW? (In particular, does user have expunge rights?)
 2. Are Global (per-mailbox) flags shared?
 3. Do Private (per-user) flags exist?  (At all?  Partially?  Fully?)

(Note: in speaking of RW or RO, I'm talking about access rights, not
 transient RO failure modes.)

Not all combinations make sense, and given fine-grain access controls, 
one can identify even more cases.  However, the following decision 
tree leads to four mailbox categories that I believe do make sense,
and in fact are all in common use:

                       MESSAGE STORE IS RW?
                           /          \
                         Yes           No
                         /              \
            GLOBAL FLAGS SHARED?    PRIVATE FLAGS EXIST?
                     /    \             /   \ 
                   Yes     No         Yes    No
                   /        \         /        \ 

MAILBOX TYPE:     1           2      3           4

In words, these mailbox classes can be characterized as follows:

 1. A shared mailbox.  Multiple users may set and see common msg state.
 2. Standard personal mailbox.  
 3. "Pseudo-personal" mailbox.  An RO archive plus private flags.
 4. A RO mailbox/archive, with no provision for user to record pers. state.

Type 1 assumes a mailbox format that supports concurrent flag update,
but otherwise is identical to standard personal mailboxes except for
access rights.

Type 3 and 4 mailboxes are what we normally think of as "Shared but
ReadOnly", and where typically the user does not have control over when a
message disappears, nor expunge rights.  It is not required that the
message store in question be ReadOnly for everyone, only to the user
in question.

All of these mailbox types are in common use; however, C-client doesn't
yet support a fully general type 3 "pseudo personal" mailbox.  It does
support a limited form of this for news, via the .newsrc file. Note that
using .newsrc is purely a concession to interoperability with existing
newsreaders, and it would have been better if we could have just ignored
.newsrc files and used a more general private flag implementation for any
Type 3 "pseudo personal" mailbox (which we need to invent anyway). 

Some of you have made assumptions that differ from my own.  In particular:
that certain flags are inherently per-mailbox or per-user; that a user in
a particular session might see a combination of Global and Private flags;
and that a set of Private flags can in any way affect anyone else's view 
of a mailbox --or worse, lead to inadvertent expunging of messages.

In contrast to those views, I've attempted to identify a mailbox framework
which both corresponds to common practice and avoids the criticisms that
legitimately follow from some of the alternative assumptions.  The case of
"pseudo personal" mailboxes (RO message stores + Private flags) is the
most relevant to this discussion, so perhaps my most important assertions
concern those private flags: 
 -By definition they are not shared.
 -They only apply to shared-but-RO message stores.
 -They can have no effect on any other user.
 -They can include any valid IMAP flag (subject to implementation limits).
 -They are never "mixed" with a potentially-existing global flag set.

My basic contention is that for a particular mailbox session, the MUA must
choose between using [1] the possibly-shared set of "per-mailbox" flags
(if the user has RW mailbox access), and [2] (if available) a private set
of "per-user" flags that are invisibile to anyone else.  This "one or the
other" restriction follows from my strongly-held belief that no flag is
inherently only per-user or per-mailbox, plus the observation that IMAP
provides no facilities for manipulating both "per-user" and "per-maibox"
flag sets as separate entities. 

--------------------------------------------------------------------------

TOPIC 4: Use of /DELETED for news...

In teaching our MUA to read news, we chose to use the one bit available in
a .newsrc to denote /deleted rather than /seen.  Evidently this decision
has proven controversial.  Please read the entire message before reacting :)

John Myers:

    this is a dangerous model. It trains users to type "d" instead
    of "n" and will cause messages to inadvertently disappear when
    users walk into a mailbox to which they have delete access. 

I think John's adamant abhorrence of what we are doing is based on the
assumption that /deleted is necessarily a global (per mailbox) flag,
rather than a per-user flag.  If we were talking about global (per
mailbox) /deleted flags, wherein one user could naively mark /deleted, and
perhaps inadvertently expunge, a message out from under another user, I
would agree that Pine's behavior is dangerous... but we are not.  News is
one example of what I called a "Type 3, pseudo personal" mailbox, wherein
sharing flag state amongst users is neither necessary nor desirable, and
wherein the normal user is powerless to actually make messages disappear
from *other* people's views.  That is, by definition the user of private
flags does not have expunge rights on the shared message store (although
one could theoretically simulate "personal expunge" via private flags). 

Furthermore, I disagree with the implicit assertion that there are no
dangers associated with applying the traditional "news model" to personal
mailboxes, or even to news!  Messages magically and implicitly disappear,
simply by looking at them once!  I've happily lived with this behavior for
well over a decade, so I was truly unprepared for how "wrong" this model
now seems to me, since having had a chance to use the other paradigm. 

    That design is based on the narrow view that mail and news are
    fundamentally different, distinguishable things which users
    process with different methodologies. 

Nothing could be further from the truth.  This design decision was a
*direct* result of trying to blur the distinctions between the mail and
news worlds. 

    This observation is backed up by the fact that Pine "knows" the
    difference between mail and news and behaves differently
    depending on how the macro IS_NEWS evaluates. 

Some of the differences are constraints imposed by using existing .newsrc
file structure for recording private flags, rather than being fundamental 
news vs. mail differences, which we completely agree should be minimized. 

    As we've mentioned before, it is extremely dangerous to require the 
    user to change their behavior.

If you are starting from the mail reader perspective, as Pine must
necessarily do, there is no change in user behavior.  To adopt the
traditional newsreader view, in contrast, would very definitely require a
change in Pine users' behavior. 

    The "per-user \Deleted for news" model does not generalize.  As
    it trains users to do dangerous things, it would be hard to
    convince me to support it

It doesn't generalize and is dangerous if and *only* if you accept John's
premise that /DELETED is inherently global, a notion I can't agree
with in the context of Type 3 (RO + private flags) mailboxes.  What *is*
inherently "per mailbox" for all classes of mailbox is the existence or
non-existence of a particular message, but that's not the same thing.  If
one were to define /DELETED as invalid for private flag state, we would
need another flag to indicate a particular user's personal disinterest in
a particular message, since I utterly and totally reject the notion of
using /seen to mean this.  Further, I see no value to a global /Deleted
flag in a Type 3 (pseudo personal) mailbox --more or less by definition. 

Accordingly, I consider the traditional newsreading paradigm of "see it
once, then it disappears", to be dangerous if applied to personal
mailboxes. (Therefore, "it would be hard to convince me to support it".
But to not apply the same paradigm to both mail and news introduces the
very modality that John incorrectly accuses us of.)

Steve Hole: 

    Apparently, the \Deleted flag is used to indicate that a News
    message "has been seen and shouldn't show up in the list". 

No, not at all.  It is used to indicate that the user is no longer
interested in the message.  Whether or not it shows up in a particular
index view is a separate question. 

    Anyway, I contend that view removal does not and should not
    have anything to do with the physical manipulation of the
    messages.  \Delete implies physical deletion.  Changing the
    view to sort filter out \Seen messages is entirely different. 

I completely *agree* that views based on flag state are distinct from what
actually exists on disk.  From a user's perspective, the available views
DEFINE their REALITY, regardless of what exists or doesn't exist on disk.
Indeed, even whether or not a message exists to a user (say after an
expunge) can be implemented in one bit! I claim that the statement
"\Delete implies physical deletion" reflects your own personal bias about
a particular implementation strategy.  It has no foundation in either
database theory or filesystem practice. (Consider, for example, what
happens when you delete a filesystem object when its link count is >1 )

    I really don't see how anyone can say that "deleting" a News
    message to make it disappear is consistent with the way that
    it works when processing private mail OR with conventional
    news readers.  It is much better just to filter the message
    list view to show or not show read messages.  This is exactly
    the way that many newsreaders work. 

Well, OK.  I'll say it.  Or more precisely, I'll say that Pine's current
behavior is *completely* consistent between mail and news with respect to
the /deleted flag, and that as far as I can tell, this has proven to be a
definite win.  As Steve said "Consistency is *very* important."

Again, the /deleted flag reflects the user's explicit disinterest in a
message.  It is the only flag available to the user for indicating this. 
Whether messages so marked "disappear" is a function of the current view.
One might choose a view where these messages are hidden, but that is a
distinct issue.  Defining a *view* that excludes certain messages is not
at all the same as *marking* those messages as uninteresting. 

In contrast, "I really don't see how anyone can say" that an implicitly-
set /seen flag should imply the user's explicit disinterest in a message,
which is indeed --and unfortunately-- "exactly the way that many newsreaders
work." And which is why Steve and Chris' suggestion to just use /seen and
suppress those messages from the current view is exactly backwards from my
perspective.  (It also clutters the UI, because now you MUST implemnent an
"UNSEE" command...  now there's a real intuitive concept, right?)

SUMMARY:

In traditional newsreaders, a message "disappears from view" *implicitly*,
simply by having looked at it once.  One must take explicit action to 
make it re-appear.

In all the mailers I've used, the opposite is true. A message never
disappears from view without *explicit* action by the user.  This might
not be strictly true in XLView, (I can imagine view definitions that would
emulate newsreader behavior) but even there, one has a Delete button to
explicitly mark a message as no longer interesting. 

Using /deleted as the way to denote disinterest in a message for both mail
and news is completely reasonable as long as you realize that the flags
for Type 2 and 3 mailboxes are not shared across users. (And as for sharing
/deleted state in Type 1 mailboxes, "it's a feature!" )

In building a tool that deals with both mail and news, the choices are:

1. Emulate *mailer* behavior for both mail and news
2. Emulate (old) *newsreader* behavior for both mail and news
3. Have inconsistent (modal) behavior, for each.

Surely we can do better than condemning the user to option #3!
Equally clearly, a mailer that is being taught to read news cannot
abandon its traditional mailreading behavior.  For Pine, that leaves 
only option #1.

Newsrc files and flag preservation...

Part of the perceived mail-news dichotemy is actually an artifact of
having a news infrastructure that only permits a single "flag bit" in the
personal state database (the absence or presence of an article number in
one's .newsrc).  This limitation has nothing to do with news per se, but
it is a real-world constraint that tends to force us into modality
whenever accessing news (in conjunction with a newsrc).  In our case, the
modality manifests itself as ignoring /seen state in newsgroups. 

If tool interoperability was irrelevant, we could ignore .newsrc files and
invent a new mechanism that would preserve all interesting flags, suitable
for any kind of Type 3 (pseudo personal) mailbox.  We need to invent this
mechanism anyway for other Type 3 mailboxes, but alas, in our case,
compatible use of the existing .newsrc files was considered essential. 

But let's return to First Principles.  From the user view, we need a clear
way of letting the user explicitly declare that a message is no longer of
interest.  In Pine, you do that with the "D" key, which is associated with
the word "deleted".  Strictly speaking, that word is misleading (both in
our UA and in the protocol flag definition) since nothing has been deleted
yet.  Maybe the "D" should mean "dismiss" or "drop" or "dullsville"... the
point is that this is the way a user tells the system that she *probably*
doesn't want to see that message again.  Not at all the same as N, for
"show me the next message". 

In the case of news, since we wanted to use the existing .newsrc
framework, we had to choose what the meaning of an entry in the .newsrc
was going to mean.  There was effectively one bit per message to manipulate. 
We could choose one and only one flag to be preserved across sessions. 
Would it be the "seen" flag or the "deleted/dismissed/dropped/dull" flag? 

We actually tried it both ways.  Now having used both models, I'm pretty
convinced that the present behavior (saving the "D" state in .newsrc) is
best.  In fact, although I still use trn a lot, I find myself using Pine's
(very incomplete) newsreading abilities more and more just because of the
"mail like" paradigm: messages don't implicitly "disappear", they stay
around until I "dismiss" them.  I like this behavior a lot, even without
having the code to *hide* /deleted messages in place yet.  (And when we 
do, rest assured it will work uniformly for both mail and news.  This code 
does exist in test versions of Pine.)

CONCLUSION:

This 40+ message discussion has surfaced a number of interesting issues.
Unfortunately, my response seems to have become almost as long as all
those messages put together... :)   (Sorry!)

Some issues I don't claim to have any good answer for yet (e.g. namespace
selection and the role of BBOARDS).  But I think this idea of message flag
*scope* (whether a flag is private/personal or shared/global) for different
classes of mailboxes is fundamental and crucial. 

The controversy over Pine's (and c-client's) newsreading behavior revolves
around these three issues:
 -the meaning of /deleted as compared to /seen
 -the relative importance of /deleted vs. /seen, if only one can be saved.
 -whether or not /deleted is inherently global, and therefore "unsafe"

The case of a RO message store + private flags, which news is an example
of, I've dubbed a (type 3) "pseudo personal" mailbox, wherein the message
data bits are shared RO but flags (including /deleted) are not shared. 
Private (per-user) /deleted state is only sensible ("safe") when the user
is unable to expunge the actual mailbox message data, as defined above. 

To insist that /deleted must be global in all cases leaves us without a
reasonable way for a particular user to record their personal disinterest
in a shared RO message; hence we postulate the Type 3 mailbox scenario
wherein one person's full suite of flags is of no interest to anyone 
else, and cannot affect anyone else. 

Finally, I hope you will all keep an open mind to the idea that an
*implicit* /SEEN flag should *not* be taken as prima facia evidence that
the user has lost interest in a message, a state that I believe is more
appropriately denoted by an *explicit* /DELETED flag.

That's it for now.  Cheers...

-teg

---

From: John Gardiner Myers <[email-masked]>
Date: Sun, 17 Oct 1993 11:05:42 -0400 (EDT)

[There are both protocol and c-client issues here]

>  -A user views a mailbox EITHER with Global or Private flags, NEVER both.

In the Cyrus imapd, the \Seen and \Recent flags are always "Private"
and all other flags are always "Global".  Given this model is useful
(for example, user wants to track their own reading progress through a
"trouble ticket" mailbox), desired, and corresponds to what we veiw
existing practice, and given the observation that Terry mentions no
specific problems with this model, I think it's a bit premature to
assume it out of existence.

To paraphrase Terry, I think his four mailbox classes are "too few or
too many."  I think it's best to state that whether any given flag is
preserved on a per-user or per-mailbox basis is an implementation
detail.  I do, however, believe it is useless or silly for certain
flags to be preserved on a certain basis.

Whether or not the user may change the preserved state of any given
flag is an entirely different matter.

> rather than a per-user flag.

Terry completely misses the reason I consider the use of \Deleted for
news dangerous.

Given there are both "Type 1" and "Type 3" mailboxes, and given that
there might not be a line in the namespace which says "all these over
here are Type 1 and all those over there are Type 3," the user trained
by c-client's handling of \Deleted in news to type "d" when they are
"no longer interested" in a message will do so when reading the "Type
1" mailbox.  They will then inadvertently physically remove messages
before other users of the shared mailbox can read them.

> now seems to me, since having had a chance to use the other paradigm. 

Messages "magically and implicitly disappear" due to the view you
selected?  One might as well use the same arguments against kill
files.

"The other paradigm" assumes that "Type 1" mailboxes with multiple
readers don't exist.

> *direct* result of trying to blur the distinctions between the mail and

I'd say you botched it, given all the places IS_NEWS is used in Pine.

> news vs. mail differences, which we completely agree should be minimized. 

Things IS_NEWS affects, from my reading of the code:

* Whether three 'n' commands cause a CHECK command
* On 'n' command, whether to prompt to ``View next news group''
* On 'd' command, whether to prompt to ``View next group''
* Whether the 'x' command is named "eXclude" or "eXpunge"
* On 'x' command, whether to error with 
  "eXclude of deleted news not implemented yet."
* On '&' command, whether the error message is
  "Unexclude command not implemented yet" or
  "Unexclude not available for mail folders"
* Whether opening a mailbox announces it as a
  "News group" or a "Folder"
* Whether or not the titlebar says "(READONLY)"
* Whether or not un-\Seen messages are marked as "NEW" in the bar status
* Whether or not TAB will stop on un-\Seen messages
* On TAB command, whether to say
  "No more undeleted messages" or "No more new messages"

Only the last three can be considered as having anything to do with
the way c-client handles .newsrc files.  Even they could go away if
the .newsrc mapped to \Seen instead of \Deleted.

The fact that Pine will only offer the user the ability to view the
next mailbox in "news mode" is a clear example of Pine considering
News and Mail as fundamentally different.

> necessarily do, there is no change in user behavior.

The user must shift the meaning of 'D' from "physically delete
this message" to "don't show me this message, for now".  You're also
assuming that users always delete their mail when they read it.

If you start from CMU mail reader perspective, there is a definite
change in user behavior.

> To adopt the

Traditionally, users haven't read their mail in "show me only the new
messages" mode because they only ever had one mailbox which had new
messages delivered to it.  Treating mail as news is an easy paradigm
shift and "browse mode" is useful for people who like the traditional
mail model.

> I claim that the statement

\Deleted has a specific definition in the IMAP protocol specification.
That definition specifically mentions "removal by later EXPUNGE".
EXPUNGE "permanently removes all messages with the \Deleted flag set".

> message.

Again, your interpretation of the \Deleted flag does not correspond to
its definition in the IMAP protocol specificiation.

> perspective.

It seems very intuitive for a command "show me the new messages" to
imply that the user is explicitly disinterested in messages with the
\Seen flag set.

> (It also clutters the UI, because now you MUST implemnent an

It's about as intuitive as "UNDELETE".

A UI should enable the user to set or clear any non-special flag.

-- 
_.John G. Myers		Internet: [email-masked]
			LoseNet:  ...!seismo!ihnp4!wiscvm.wisc.edu!give!up

---

From: Chris Newman <[email-masked]>
Date: Sun, 17 Oct 1993 18:35:59 -0400 (EDT)

Here are a couple scenerios which show why Terry's different interpretation of
/DELETED for netnews and mail is dangerous:

The secretary for the athletic department office is administering a
publicly readable athletic bboard which is fully read-write to
athletic department administrators.  A message with a major error is
posted to the bboard and he wishes to remove it.  He goes to the
bboard, deletes the message, and expunges the bboard.  The message
disappears from his view.  Then he continues to get complaints about
this message and is confused.  He had, after all, removed it the same
way he would remove it from his mailbox.

Now suppose Jane User is reading a mailbox for her project group.
She's gotten used to hitting "d" to remove messages from her view.  So
when she's read a message, she hits "d" to remove it from her view.
Her client, as usual, expunges the messages when she leaves the
folder.  But now, none of the other people in her group can see that
message because its gone.  She did the same thing she does in news,
and it doesn't remove the message there.  She is confused and her
project is set back a few days.

----------

These two examples show why "/DELETED" must either mean "flag for
removal" or "hide".  It must not have different meanings in different
mailboxes/bboards.  However, Terry has argued eloquently that a "hide"
concept is useful and I _personally_ wouldn't object to adding an
extra optional flag for it.  I do object strongly to equating the "hide"
concept with "/DELETED" as I believe it would confuse and endanger
users.

		- Chris

---

From: Terry Gray <[email-masked]>
Date: Sun, 17 Oct 1993 15:44:16 -0700 (PDT)

> assume it out of existence.

My intent was not to preclude other scenarios; only to explain the basis
for my own thinking.

In a shared mailbox situation, one can make a case for presenting either a
"private view" or "global view" of \seen, and even \recent. In fact, one
(not me) *could* even argue for showing both, but we don't have the
machinery in IMAP to deal with both simultaneously.  Given the differences
in opinion about which flags should be considered private, and which
global, how can we proceed?  I suppose the best we can do is adopt a
laissez faire "implementation dependent" stance.  The downside of that
there will be surprises for users. The clients should all *work*, but what
the flags actually mean, in terms of scope, will differ from one server to
the next. 

> Terry completely misses the reason I consider the use of \Deleted for

I don't think so; at least John's next paragraph is consistent with what I 
understood his objection to be...
 
> before other users of the shared mailbox can read them.

One could as easily argue that C-client's handling of \deleted for
*conventional* personal mailboxes is dangerous!  And it is!!  After all,
in a personal mailbox, an individual gets to mark a msg as deleted (or
uninteresting), and since they have expunge rights, actually make it go
away.  Forget about news, what happens when such a *mail* user begins
using a shared mailbox?  Exactly the same thing that happens when people
share files in any system: either they talk, or have a locking protocol,
or sooner or later, somebody gets burned.

[Small aside to illustrate the point: in 1976 people from 18 Arpanet sites
around the US were collaborating on a major report, using MIT-Multics as
the file repository.  Much gnashing of teeth ensued when one team member
unilateraly copied the files to BBN-TENEX for spell checking, then copied
them back a few days later --obliterating much intervening work.]

I don't believe that allowing \deleted for news makes this problem any
worse than it already is, but I do think that there is an opportunity for
improving IMAP here:  it would be nice if clients could find out if a
mailbox was being shared, and/or sharable, so the UI could take pains to
warn users that their actions could affect multiple users. 
 
> selected?  One might as well use the same arguments against kill

Not at all.  Kill files are the result of *explicit* action by the user.
 
> "The other paradigm" assumes that "Type 1" mailboxes with multiple

Not at all.  Unless John wishes to legislate "normal" unshared personal
mailboxes out of existence as well (and of course he doesn't), this
argument makes no sense to me. 
 
> I'd say you botched it, given all the places IS_NEWS is used in Pine.

Reality sometimes lags intent, and Pine is still "work in progress",
especially with respect to newsreading.  News is the first, but not the
last, case of having an inherently ReadOnly message store with private
flag state. Some of the current Pine behavior, in particular having N
prompt for the next news group, but not the next mail folder, is clearly
wrong and will be changed.  In many of the other cases where "IS_NEWS" is
used, a macro named something like IS_IMMUTABLE would have been better, 
since NEWS is only one of possibly many classes of immutable mailboxes. 

> this message" to "don't show me this message, for now". 

Incorrect, both with respect to the "from" and the "to"...

A "D" does *not* mean "physically delete this message", no matter what
the protocol spec might say.  We all know that "D" has absolutely no
effect on the message other than to signal *intent* by associating a flag
with that message.  If we want "D" to *really* mean "delete data", then we
should eliminate Expunge from the protocol, and the notion of a \deleted
flag becomes nonsensical, as it becomes a flag associated with a message
that no longer exists. 

[NB: Even if we keep the \deleted "statement of intent" separate from the
action of expunging, which I certainly support, we still must decide what
\deleted means for immutable mailboxes.  A *global* \deleted flag for an
immutable message store makes no sense to me.]

Nor does "D" mean "don't show me this message for now", although that
could be a *side effect* of the currently defined view. 

[NB: An important part of this debate has to do with the relationship
between *flags* and *views*...  Although views are often defined in terms 
of flags, the concepts are distinct and we must avoid blurring them.]

> You're also

I'm clueless as to why John said this... it certainly is not true.

> shift and "browse mode" is useful for people who like the traditional

Conversely, I would claim the CMU community has not used "browse" mode for
news only because of limitations in the CMU system, namely that it only
keeps a single pointer, delimiting old and new.  I have no problem with
having different "view modes" and being able to apply them uniformly to
mail and news.  But I have a real problem with overloading the /seen flag
to imply "no future interest in this msg". 
 
> its definition in the IMAP protocol specificiation.

The protocol spec reflects a traditional terminology that was *never*
precisely accurate, and doesn't translate perfectly to the world of
immutable mailboxes and private flag sets that we are now trying to extend
IMAP and c-client to cover.  Either we forbid \deleted from being used on
immutable mailboxes, in which case we need to invent another flag that
would be redundant in the RW case, or we try to find an "enhanced"
interpretation of the existing flag that can work for both RO and RW
mailboxes. 

> imply that the user is explicitly disinterested in messages with the

It is important (to me, anyway) that we keep the concepts of "message
attribute" and "current view" distinct, because I see the lifetime of a
particular message *attribute* being generally different than the
persistence of a current view.  Both are changeable at will, but they are
not identical.  So I have *no* problem with being able to say "just show
me new messages right now"... but I object to the idea that this transient
"bulk" definition of what's interesting at the moment has anything to do
with a per-message *explicit* declaration that a particular message is of
no future interest. 
 
> It's about as intuitive as "UNDELETE".

Not *quite* that intuitive.  :) But I certainly wouldn't want to defend
delete/undelete very vigorously, since the common interpretation of those
words is clearly not what happens in fact.  Probably too late to pick a 
better word, though. 
 
-teg

---

From: Chris Newman <[email-masked]>
Date: Sun, 17 Oct 1993 19:00:08 -0400 (EDT)

> -A Mailbox = [Msg Data + Global Flags] OR [Msg Data + Private Flags]

How about, the mailbox administrator(s) use global flags and the
mailbox readers use private flags?  A useful scenerio, IMHO.

> -Global (per-mailbox) flags may or may not be shared.

Agreed.

> -Private (per-user) flags, if they exist, are never shared.

Agreed.

> -A user views a mailbox EITHER with Global or Private flags, NEVER both.

A bboard with several administrators that read the bboard probably
will want private \SEEN flags for the administrators, while sharing
the other flags.

> -A user with RW/EXPUNGE rights will use Global flags, never Private flags.

What about a user with RW but not EXPUNGE rights?  How about a user
with EXPUNGE but not APPEND rights? What about the several
administrators that read the bboard separately (e.g. a club bboard
with several officers who have RW/EXPUNGE rights)?

> -Implementation constraints may limit which flags can actually be stored.

Agreed.

		- Chris

---

From: Terry Gray <[email-masked]>
Date: Sun, 17 Oct 1993 16:05:13 -0700 (PDT)

Chris,
I believe that both of your scenarios represent misunderstandings of the
model I advocate, and also illustrate the danger of blurring the
distinction between "message attributes" and "current view".  Let me try
to explain why... 

> way he would remove it from his mailbox.

In my model, the \deleted flag is only private if the user does *not* have
expunge rights.  So in this case, both the flag and the subsequent expunge
would have had global scope and there would be no problem.  The message
really would be gone.
 
> project is set back a few days.

The mistake in this scenario is postulating that the user's motivation for
hitting "d" is to simply remove messages from the current view.  (Pine
users certainly don't have this idea, since hitting "d" does not remove a
message from the current view.  Expunge does.) As you know by now, it has
never occurred to me that D would or could be construed as purely for
transient view control.  Especially since, like you, I consider it
perfectly reasonable to postulate a *current* view that suppresses /seen
for either news or mail, without having to type "d". 

So the objection to "D" being applied to news is based on fear that the
user would not realize that "D" really associates a persistent attribute
with the message, rather than simply hiding the message, (assuming that
"D"s are hidden in some views). I claim that to the extent this might be
true, the problem already exists with personal mailboxes, and is not
significantly exacerbated by applying "D" to news as well as mail. 

I think we can agree that we must in all cases avoid leading users "down
the garden path" of believing that flags do nothing but control views.
That's why I think it so important to clearly delineate message state 
from views.

> concept with "/DELETED" as I believe it would confuse and endanger

For the reasons I cite above, I too object to equating "deleted" with
"hide", but I do not see any conflict in equating "deleted" with "of no
future interest".  Neither "Deleted" nor "of no future interest" should
imply "hidden", lest we find ourselves in yet another terminology
perversion, wherein we allow definition of a view to temporarily display
that which is hidden! 

Suppose you accept my thesis that it is useful to have a flag to let the 
user explicitly denote a message as having no future interest, and that 
this message state is orthogonal to the current view definition.

Now suppose that for whatever reason, we don't want to use "deleted" to
mean "no future interest", so we need a new flag.  My objections would be
that in a Read/Write/Expungable mailbox, the new flag is superfluous, and
that any new flag requires extra machinery to display and manipulate. 
Here's what we would have: 

MAILBOX TYPE:               RWE            RO

\marked-for-deletion        OK             Contradiction
\uninteresting              Redundant      OK

Given that the "danger of deleted" concern seems to be partly a result of
an incorrect assumption about the connection between flags and views, and
partly an intrinsic and long-standing problem that occurs whenever a
private mailbox user moves into a shared mailbox arena, does it really
make sense to have a different, but oh-so-similar flag? 

-teg

p.s. I don't mean to imply that I'm not concerned about inadvertent 
deletion of messages in shared mailboxes; I am, but I see that as a 
preexisting problem that is neither solved nor worsened by either of 
our positions.

---

From: Terry Gray <[email-masked]>
Date: Sun, 17 Oct 1993 20:38:46 -0700 (PDT)

Well, looks like I'm 3 for 6 in your book... (our Seahawks should do as
well! :)  I've deleted the 3 assumptions we agree on, and now offer a few
comments on the other 3... 

> mailbox readers use private flags?  A useful scenerio, IMHO.

I'm guessing that you are proposing this as an *addition* to the above,
rather than as a replacement.  If so, it's not needed, because I meant to
imply that my two alternatives apply to a particular session, and are not
mutually exclusive across all users, or even across sequential roles of
one user. 

More difficult for me to accept is the case of
   "Mailbox = [Msg Data + Some Global Flags + Some Private Flags]"
because I don't know how to manage this level of state complexity in a
consistent way and surprise-free way, unless each flag is defined as
*either* private *or* global, (which I really *wish* I could convince
myself was reasonable, since it simplifies the problem.)
 
> will want private \SEEN flags for the administrators, while sharing

I agree that private \seen flags can be useful (though I hold that global
\seen can also be useful).  I'm struggling with how we would keep track of
the fully general case.  A message is marked answered; did I answer it, or
did someone else on the team?  Likewise SEEN, FLAGGED, etc... How does the
user know, for any given flag in any given mailbox, what its scope is? 

Also, how does the client know when to look for a client-maintained file
of private flags, e.g. .newsrc, or its mailbox equivalent for, say, a 
remote message archive?
 
> with several officers who have RW/EXPUNGE rights)?

My assumptions do not preclude multiple users with expunge rights.  (But
if multiple people do have expunge rights, they better also have a common 
vision of who expunges and when!)

I agree that one could meaningfully have the ability to store, say, a
Flagged flag without having expunge rights.  But that leads either to (a)
asserting that some flags are inherently global, the others inherently
private --which I still can't convince myself is true, or (b) the dilemma
of having arbitrary mixtures of private and global flags --which I don't
know how to manage, especially when in some cases the private flags
will be maintained by the server, and in other cases by the client.

As a simplifying assumption, I was hoping we could key on expunge rights
to determine whether to look for (and use) private flags. If that won't
work, we need to tighten-up our thinking caps one more notch... 

-teg

---

From: Wallace Colyer <[email-masked]>
Date: Mon, 18 Oct 1993 10:39:58 -0400 (EDT)

First, I should say, if the seen flag in the .newsrc means seen, I think
the IMAPD should honor the meaning, but since that doesn't seem like it
is going to fly....

I see three different actions here:

SEEN
DELETED

and a third which I will call 

BURPLYBLOOP 

I disagree is Terry with respect to the meaning of DELETED.

To me, DELETED means marked for deletion, and when I say deletion I mean
physical removal from the mail store.  I think it is VERY important for
this to have a consistant meaning.  Why, because otherwise people are
required to know what the state of the mailstore is when they use this
flag.  When I mark a message for deletion, I expect the EXPUNGE
operation to physically remove the message so that no one else can see
it.  To have any other action on the DELETED flag, to me, is inherently
dangerous.  In netnews, if I am allowed to delete the message I expect
the server to send out a cancel and remove it from the local store (I
like who ever said this earlier). DELETED could be a personal flag and
when I do an EXPUNGE operation it may only delete the things I have
marked for deletion, but the CMU Cyrus IMAPD will certainly implement it
as global.

The SEEN flag is marked when I have seen the message.  My message viewer
may choose to not show me the message after I have seen it.  A good
message view will allow that option as one view, but allow me other
views to see the other messages in the folder. 

The BURPLYBLOOP flag says, "Don't show me the message".  Here I am not
deleting the  message.  I am not marking it as seen, because I may or
may not have seen it.  I am saying, when BURPLYBLOOP is set, don't show
me the message.  This gives the clients
all the options in the world to handle this as desired and causes no
negative behavior. They can choose to show me BURPLYBLOOP marked
messages, SEEN messages, and DELETED messages, or any combination of
those and other state.

I further support the prevailing view that the current method of using
DELETED is very very very dangerous.

---

From: John Gardiner Myers <[email-masked]>
Date: Mon, 18 Oct 1993 10:45:34 -0400 (EDT)

Terry Gray <[email-masked]> writes:
> away.

You've got to be kidding.

When a user marks a message in a personal mailbox as deleted, they
really want the message to go away.  As long as this model is kept
consistent, they will only mark messages in shared mailboxes when they
really want the message to be permanently removed.

There is also the minor fact that personal mailboxes are generally in
their own part of the namespace.

CMU has had this model (delete means really delete, UI doesn't go
through pains to identify folders as "shared") for years and has had
no problems that I am aware of.

The redefinition of "\Deleted" as "uninteresting" is part of the
c-client news philosophy and is part of what we find so dangerous.
"Uninteresting" is an attribute of the user's view, not of the message.

> Not at all.  Kill files are the result of *explicit* action by the user.

Selecting an "unseen only" view and reading messages are explicit
actions by the user.

> argument makes no sense to me. 

Then how, using your paridgm, do you mark a shared "Type 1" mailbox as
being uninteresting to you, but without actually removing the message.

> In many of the other cases where "IS_NEWS" is

There is already a separate READONLY_FOLDER macro, which didn't get used.

Put another way the uses:

Were made necessary because c-client changed the mapping of .newsrc
from \Seen to \Deleted.  Every other client is going to have to have
this knowledge of this difference between "mail" and "news", or
they're not going to interoperate with the c-client imapd.

> with that message.

You're picking nits.  To reword, ``The user must shift the meaning of
'D' from "intend to physically delete this message on later expunge"
to "don't show me this message for now."''

> immutable message store makes no sense to me.]

For immutable mailboxes, \Deleted means "intend to physically delete
this message on later expunge."  The fact that the mailbox is
immutable doesn't matter.

What does COPY mean when the destination is an immutable mailbox?

> could be a *side effect* of the currently defined view. 

You're using this "side effect" as a fundamental reason for changing the
definition of \Deleted.

> keeps a single pointer, delimiting old and new.

The CMU system still allows users to go back and view messages which
had previously been marked as old.

> But I have a real problem with overloading the /seen flag

It doesn't imply that, it implies "not interested in this message when
looking for new messages."

> interpretation of the existing flag that can work for both RO and RW

You certainly haven't reached a consensus for "enhancing" the
interpretation of the existing \Deleted flag.  Last I checked, it was
1 for, 3 against.  This "enhancement" of the meaning is actualy an
incompatible change which we consider extremely dangerous.

> So I have *no* problem with being able to say "just show

I don't understand.  How is this any different when applied to \Deleted?

> A message is marked answered; did I answer it, or

I consider this one of the key problems with Chris' suggestion that
flags be private or global depending on the user's access.  I believe
that for a given flag in a given mailbox, it should be private for
everybody or global to everybody.

---

From: Chris Newman <[email-masked]>
Date: Mon, 18 Oct 1993 12:02:37 -0400 (EDT)

> myself was reasonable, since it simplifies the problem.)

Well, this is exactly what the Cyrus imapd is going to do.  \SEEN is
always going to be private because we've found global \SEEN to be
useless in practice.  \DELETED is always going to be global so that
every user knows which messages EXPUNGE will remove.  The other flags
are going to be global because we don't consider them important enough
to merit the high cost of per-user storage.

Since flag behavior won't change between different mailbox types, I
don't believe Cyrus users will be surprised.  I think the answer is
that every good server implementation should be internally consistant
about global vs. local flags.

> of private flags, e.g. .newsrc, or its mailbox equivalent for, say, a

I believe we all agree this problem needs to be solved. This is why
CMU has suggested that the FLAGS reply exclude non-permanent flags.
If the flag isn't in the FLAGS reply, then the client can look for a
client-maintained file.

> will be maintained by the server, and in other cases by the client.

I suspect private vs. global is ultimately a religious issue and thus
must be implementation specific.

> work, we need to tighten-up our thinking caps one more notch...

It certainly won't work with the Cyrus imapd.  You can have "seen"
rights which permit storing \SEEN info on the server without having
"delete" rights which permit storing the \DELETED flag and EXPUNGE.

		- Chris

---

From: Terry Gray <[email-masked]>
Date: Mon, 18 Oct 1993 09:06:16 -0700 (PDT)

John,
The good news is that at least *some* of our difference still seems to be
a result of misunderstandings rather than convictions... :)

On Mon, 18 Oct 1993, John Gardiner Myers wrote:

> > *conventional* personal mailboxes is dangerous!  
> 

This statement says to me that I have totally failed to communicate 
and/or you have totally failed to understand what I'm trying to say.
I was not kidding.
 
> really want the message to go away.  

This is precisely what is meant by "D" in Pine in all cases, news and
mail. The only difference between the the two is whether or not you can
later successfully expunge the messages you marked for deletion.  To
account for that possibility, I attempted, obviously unwisely, to redefine
the "deleted" term so it didn't sound so contradictory in the RO case (or
more precisely, in the "no expunge rights" case.) But from the user's
perspective, the motivation for typing "D" is identical in both the mail
case and the news case. It is totally consistent.  It means "delete it (if
you can), I'm done with it". 

> As long as this model is kept

It is precisely because Pine does keep this absolute consistency in usage
that the claim of it's behavior being dangerous is bogus. 
 
> their own part of the namespace.

Shareability and RO-ness are, and should be, orthogonal to location.
 
> no problems that I am aware of.

The only difference between the CMU and UW models is that apparently you
don't permit someone to mark a message as "deleted" if they don't have
expunge rights.  The essence of my --apparently ultra controversial--
position is that I believe it is useful to let someone mark a message as
deleted even if they do not have the right to expunge it. 

If you *do* let people delete even without expunge rights, then exactly 
what is it that you are unhappy about in our position?
 
> "Uninteresting" is an attribute of the user's view, not of the message.

The "intend to delete" flag that we call \Deleted has always been an
attribute of the user's view of the message in exactly the same sense.
Whether that intent is actionable depends on having expunge rights.
 
> Selecting an "unseen only" view and reading messages are explicit

Agreed.  What I object to is not having the ability to differentiate 
between \seen and \deleted for purposes of my current view.  That is,
I want the ability to mark news messages that I'm done with as \deleted,
just as I would a mail message --even if I don't have expunge rights.
 
> being uninteresting to you, but without actually removing the message.

In a type 1 mailbox, the state is shared by definition, so when I mark a 
message as \deleted it has global effect.  The next expunge nails it.
 
> There is already a separate READONLY_FOLDER macro, which didn't get used.

Unfortunately, there are cases where folders become RO due to errors or
transient locking phenomena.  I *think* the UA needs to distinguish
between the transient RO case and the RO-because-of-access-restrictions
case, but I'd be pleased to be wrong on that.  There may also be cases 
where the READONLY_FOLDER macro should have been used...
 
> they're not going to interoperate with the c-client imapd.

These are all a result of having only one bit to play with in the .newsrc,
and having decided that it was more useful to store /deleted than /seen.
These are not fundamental news vs. mail issues.  A richer .newsrc structure
would eliminate this issue.
 
> to "don't show me this message for now."''

NO NO NO!!!  "D" does *not* mean "don't show me this message for now". 
The "D" means "I'm done with it forever.  Make it go away --if you can!".
One's current view may or may not include messages so marked. 

> What does COPY mean when the destination is an immutable mailbox?

If you don't have the rights to do the copy, you get an error, just like 
if you don't have the rights to do an expunge, you get an error.
 
> You're using this "side effect" as a fundamental reason for changing the

Definitely not.  Again, the issue is whether or not it is sensible to 
mark a message as \Deleted even when you don't have expunge rights.
I claim it is, you apparently feel otherwise.
 
> incompatible change which we consider extremely dangerous.

Look at it this way: all I've been trying to say throughout this entire
debate is that I want to be able to mark a news message as deleted, and
have that mean *exactly* the same thing as it would in the case of a mail
message, regardless of the fact that a subsequent expunge might fail or be
forbidden.   (Which of course might happen in the mail case as well.)

I'm not yet sure whether my attempt to explain this position via the 
semantics of word "deleted" was just terribly ineffective, or whether
you guys really believe it is a terrible idea to allow someone to set a 
\Deleted flag in cases where they don't have expunge rights.
 
> flags be private or global depending on the user's access.  

I can see how to deal with the case where they were *all* global or *all*
private for a given session... but the mix-n-match scenario really scares
me. Your and Chris' suggestion about the server telling the client what
flags it lets the client set might give the UA enough info to somehow
present each flag's scope, but I haven't yet thought of a way to do that
"elegantly"

> I believe

I agree that our problem would be much simpler if we could do this, but 
when I look at each flag and try to decide if there are any that are 
meaningless in either a private or a global context, I don't see any 
silly states.  Also, to legislate that certain flags are inherently 
private has far-reaching implementation consequences since there are 
servers that don't have the concept of "private flags".

-teg

---

From: Terry Gray <[email-masked]>
Date: Mon, 18 Oct 1993 10:13:37 -0700 (PDT)

On Mon, 18 Oct 1993, Chris Newman wrote:

> useless in practice.  

My experience differs.  Both private and global \seen convey useful
information... global \seen lets one quickly identify which messages have
as yet been unread by *anyone*... 

> \DELETED is always going to be global so that
> every user knows which messages EXPUNGE will remove.  

Yes, if you have expunge rights, your \deleted flag *must* be global.

-teg

---

From: Wallace Colyer <[email-masked]>
Date: Mon, 18 Oct 1993 13:29:14 -0400 (EDT)

Let me pose some simple questions which I think Chris asked earlier.

First, let me state my assumption that you cannot assume that netnews is
the only thing in the bboard namespace and netnews may exist in the
mailbox namespace as well.

How do I delete a message from my personal view not not from everyone's
view when I am in a shared folder?

How do I delete a message permanently from a mailstore so no one can see it?

Is it based on whether I perform the expunge command?  If so, how do I
delete some messages from my personal view and some from the permanent
mailstore?

-Wallace

---

From: Terry Gray <[email-masked]>
Date: Mon, 18 Oct 1993 10:32:14 -0700 (PDT)

> mailbox namespace as well.

Agreed.  We are just talking about the relationship between \DELETED and 
expunge rights, and those are orthogonal to both namespace and whether we 
are dealing with "news" or "mail".
 
> view when I am in a shared folder?

You don't, at least not if you have expunge rights to that shared folder,
because if you can expunge, your \deleted flag *must* be global.  (Whether
\deleted should be global --or even allowed-- when you do *not* have
expunge rights is a separate, and more interesting, question.)
 
> How do I delete a message permanently from a mailstore so no one can see it?

"In the normal way"... :)  Again, *if* you have expunge rights...
 
> delete some messages from my personal view and some from the permanent

If you have the *right* to expunge, the \deleted flag is necessarily
global, so in that case there is no distinction between "personal view"
and what happens to the permanent mailstore.

My premise is that \DELETED should be *consistently* available to the user
to indicate that s/he is done with the message and intends, if it is
within his/her power, to have that message go away. 

Clearly, in a shared mailbox case, *if* the user has expunge rights, their
\DELETED flag *must* be global.  So that leads us to the case of *not*
having expunge rights.  What about \deleted then?

Three choices:

  1. \deleted is disallowed (an error)
  2. \deleted is global
  3. \deleted is private

Chris' last message suggests that Cyrus chooses option #1, tying whether 
or not you can set \deleted to whether or not you have expunge rights.

Example of where #2 would be useful: I work for you, and we share a
trouble tracking folder.  You don't completely trust me :) so you don't
give me expunge rights, but you want an easy way to see whether I think a
case is closed.  When I finish a case I mark it deleted. You review my
work, and if you agree let the \deleted stand.  Periodically you expunge.
Note that Answered, while useful, does not imply that the case is closed.

Example of where #3 is useful: You have access to a remote message
archive, but no access rights other than read.  You'd like to keep track
of which messages you've seen, of course, but in addition, you'd like to
be able to use \deleted as you always do, to indicate that you are forever
done with the msg and would like it deleted --even if you are unable to
enforce that intent.  To forbid this case as an error forces a
user-behavior change... In all the cases the user would normally say
"delete" s/he now gets an error.  (It's true that, even if we allow the
private \deleted for this case, a subsequent expunge will generate an
error, but that happens in case #2 as well.) By allowing users to set
\deleted in this situation, we can come very close to providing the
semantics of a private mailbox, without actually having to replicate the
message archive (or deal with getting updates).  Not being able to expunge
seems a small price to pay for this benefit. 

-teg

---

From: Chris Newman <[email-masked]>
Date: Mon, 18 Oct 1993 15:32:12 -0400 (EDT)

Terry Gray <[email-masked]> writes:
> The essence of my --apparently ultra controversial--

I think this is a dangerous garden-path scenerio.  Here's an example:

Jane User uses the FOOBAR client which always does an EXPUNGE on exit.
She learns that hitting 'd' in her mail folder makes the message go
away.  She also learns that hitting 'd' in non-mail folders doesn't
make the message go away.  She then joins a group project with a group
RW/E folder.  Because it's another non-mail folder she assumes she can
hit 'd' and the message won't go away.  She has never encountered a case
where hitting 'd' makes the message disappear for other people, so she
assumes it won't.  FOOBAR issues its EXPUNGE and the message goes
away.

Note what you have happening here.  Your proposal forces Jane to make
distinctions between different types of mailboxes.  She made a
reasonable assumption which caused the loss of important work.  In
order to understand what happened, she needs to learn that 'd' does
different things based on "EXPUNGE" rights -- which is not a simple
concept.

So what's wrong with the scenerio:
A) Your proposal is dangerous and forces users to distinguish between
   different mailbox object types.
B) Jane was stupid for not understanding "EXPUNGE" rights.  She needs
   to learn how to properly distinguish different mailbox object types
   before she can be allowed to use a c-client program.
C) FOOBAR client was designed wrong (what should be fixed?).

		- Chris

---

From: Terry Gray <[email-masked]>
Date: Mon, 18 Oct 1993 12:36:14 -0700 (PDT)

Chris,
Your argument that my model forces users to make distinctions between
different mailbox types is bogus because the same concerns apply to the
CMU model (e.g. just having or not having delete rights in general.)

The fact is, there *are* differences in access rights, and these
differences result in errors, which result in changes in user behavior.
This is true whether \delete is forbidden, as you prefer, or whether just 
expunge is forbidden.

If your view prevails, it means that c-client will be far less useful for 
supporting the "pseudo personal" mailbox case (where the remote message 
archive is totally RO.)  

-teg

---

From: Mark Crispin <[email-masked]>
Date: Mon, 18 Oct 1993 12:36:01 -0700 (PDT)

Mind you, Terry and I don't agree 100% on these issues.  I have a completely
different perspective, based in part from dealing with this stuff for 7 years
and having a set of ``seat-of-the-pants feelings''.  I've been trying to stay
mostly out of this issue, until it reaches the ``what does this mean when it
is implemented in c-client?'' point...  ;-)  But some of your questions were
interesting enough to deserve answers from my perspective.

Agreed.

> delete some messages from my personal view and some from the permanent

These questions are based upon a presumption of a unity of shared folders.
This doesn't exist in c-client.  Instead, c-client has several different types
of shared folders, with different classes of access:
 1) BBoards: (in the IMAP sense, or ``blurdybloops'' if you prefer)
    1a) Netnews.  Per-user \Deleted flags, \Recent flags begin after the
        highest-numbered \Deleted flag, other flags are per-session.
    1b) Non-netnews mailbox formatted.  All flags are per-session.  \Recent
        set for all messages.  File may have initial settings for the per-
        session flags, but user can't change them.
    1c) Non-formatted objects.  Presented as a single message with no flags
        initially set.  Any flags are per-session.
    Note that with BBoards, there is a commonality in that there are no shared
    flags that the user can set.  At best, he can set a per-user flag.  Also,
    EXPUNGE is always meaningless with a BBoard; as CMU says, they are second
    class citizens (and useless for them).
 2) Shared Mailboxes:
    2a) read/write mailbox formatted.  Flags are global.
    2b) read-only mailbox formatted.  Similar to 1b) case.
    2c) read-only non-formatted objects.  Similar to 1c) case.
    2d) read/write non-formatted objects.  No definition yet.
    We have talked about per-user flags with mailboxes, but don't do anything
    about it yet.

It is important to realize that although you CAN access bboard objects in the
mailbox namespace by prepending a * to the name, this is an internal
convenience for c-client only.  It does not mean that bboards are considered
inside the c-client namespace.

It may have been a mistake to have a separate bboard namespace, but, at the
time this decision was made, it was presented to me as a *given* that the
namespace is separate.

My own givens were that it was unreasonable to issue an error message where
there was something arguably useful to do for the user.  It quickly became
clear that in many cases, the only useful you could do was set a per-session
flag.  I still dispute the claim that it is wrong to have per-session flags as
a capability; it is entirely up to the user interface to decide whether or not
users may have access to these.

There is also a question about whether or not it is right to recycle the
meaning of \Deleted in the BBoard namespace as Terry has done.  I believe that
it is ``not wrong''; that a fair and reasonable case can be made to do so.  I
have my own opinions about whether or not ``not wrong'' imply ``right'', but
they are irrelevant to this discussion.  So far, he arguments against this use
of \Deleted presume that this is taking place in the mailbox namespace.  It is
not.  It can not.

This also pretty well debunks the CMU claim of danger.  They need never
encounter the need to support this in their servers, because they don't use
the bboard namespace.  If, for compatibility sake, they decided to support the
bboard namespace in their servers, they could change the \Deleted request in
that namespace to a request in the mailbox namespace that makes more sense.
Similarly, if they were to support the bboard namespace in their clients for
compatibility with our servers, they would undoubtably choose to hide the fact
that a \Deleted flag is set.

As I see the REAL problem, the question is one of determining access rights.
In a more sophisticated application, you want to make a determination of what
kind of object you are dealing with and what are your access rights to that
object.  The present Pine gets along with the bboard/mailbox namespace switch
and the read-only switch to subdivide mailboxes; that is, it deals with three
different access right entities.  The present Pine doesn't distinguish between
shared read/write mailboxes and private read/write mailboxes.

It is not hard to imagine more sophisticated choices being made than what Pine
uses (after all, I did list 7 different kinds of shared mailboxes, of which 6
are supported now by c-client!), but I don't think that pulling a capability
that some people think isn't useful isn't the way to go about enabling those
choices.

I probably find myself alone in defending c-client's current model.  I want to
caution folks, though; don't forget that whatever we end up with has to be
implementable!  c-client differs from Cyrus in a very fundamental way; it is
intended to interoperate with whatever crufty infrastructure may already be in
place.  Cyrus has the luxury of starting from scratch (and yes, I covet that
luxury!).  I have heard a number of things in this discussion which would be
quite difficult to implement in the c-client context, or which would introduce
interoperability problems.

---

From: Chris Newman <[email-masked]>
Date: Mon, 18 Oct 1993 16:44:55 -0400 (EDT)

> CMU model (e.g. just having or not having delete rights in general.)

I don't think it's bogus.  Users are used to have options "grayed out"
or rejected because they "don't have permission".  The problem comes
when a user takes an action which is successful, but does different
things in different situations.

> supporting the "pseudo personal" mailbox case (where the remote message

I have already stated that the concept you suggest is useful -- I'm
just opposed to overloading \DELETED.

>From what I understand of your _intent_ for use of \DELETED in
netnews, may I suggest adding a user flag "hidden" which means the
user is not interested in seeing the message again?  All you have to
do is lobby client makers to have a default view of "UNKEYWORD hidden"
and you get what you want.  If you wait for NSEARCH, you could even
add a \HIDDEN system flag.

		- Chris

---

From: Terry Gray <[email-masked]>
Date: Mon, 18 Oct 1993 14:11:31 -0700 (PDT)

Chris,
I believe your argument is: that because Pine treats D the same for mail
and news, when users encounter a shared mailbox with RWE access, they will 
continue to use D with possibly dangerous results. 

Take news out of the equation above and you'll see the inconsistency of
that argument.  Anyone who uses a personal mailbox (where I presume
deletion is permitted) will be potentially dangerous given a shared
mailbox with RWE access, unless they are suitably educated.  There is
nothing greyed-out, but the number of people affected by the "formerly 
harmless" action could be very different. 

As to a separate flag... two points:
  -I'm not eager to embrace this solution because I fear it adds 
   needless cruft to the user-interface and/or protocol.
  -Such a flag should not be called "hidden" because "hiddenness" is a 
   property of the current view, not an attribute of a message.

Consider again the case of a shared mailbox, where an underling marks a
message as deleted when s/he is done with it, for subsequent review by a
superior who has expunge rights.  The need is not for a "hidden" flag, but
something that conveys "doneness" or "dismissal", which is exactly what
our users convey with "D".  To have both a \dismissed and a \deleted flag
seems like it would be needlessly confusing, if not error prone. 

I suspect if we had both flags available we would probably use "D Dismiss"
--> \dismissed universally, since the difference between \deleted and
"expunged" has *always* caused some confusion.  Then we would change the
eXpunge prompt to say "Delete messages marked as Dismissed?"... then we
could invent a 2nd version of expunge for greater efficiency... :)

Seriously, I could live with that outcome; it just seems unnecessary to
me. 

-teg

---

From: Chris Newman <[email-masked]>
Date: Tue, 19 Oct 1993 09:17:25 -0400 (EDT)

> continue to use D with possibly dangerous results.

Not really.  My argument is if users learn they can hit "d" (delete)
without losing the message, then they may hit "d" on a message they
don't want to lose, expecting it to stay around.  The problem is
having "d" be a non-destructive operation in some circumstances, but
not others.  I think this problem would apply to all clients
(particularly those which EXPUNGE without asking the user), not just
Pine.

> harmless" action could be very different.

When you hit "d" in a personal mailbox (on a client with
auto-expunge), the message is removed forever.  Nobody can ever see it
again.  In addition, delete is a well known concept not limited to
computers.  I claim users will not be confused if "d" does the same
thing in RWE group mailboxes.  Do Macintosh users who throw a program
in the trash when on a server machine still expect other users to see
that program?  No.  I believe the situation is equivalent.

> superior who has expunge rights.

Let's look a little closer at this example.  Suppose the underling is
marking messages with \DELETED (global flag) at the same time that her
superior is looking at the folder and about to switch to another
folder.  When the superior's client issues an EXPUNGE, it will remove
the messages without chance of review.  Therefore I claim this is an
unuseful scenerio.  If she can set \DELETED, it means messages could
be removed without her superior seeing them.  Thus \DELETED and
EXPUNGE rights are equivalent when there is a global \DELETED flag.

She should probably use either "\FLAGGED" or a user flag like "review"
to flag the message for review by her superior.

> The need is not for a "hidden" flag, but something that conveys
>"doneness" or "dismissal", which is exactly what our users convey with
>"D".

I suspect if you polled your non-technical users about what "D" means,
that you would be surprised by the results.

There is a need for a separate "slated for permanent removal" flag,
namely \DELETED.  What you seem to desire (in the case of netnews) is
a "No-longer-interesting" flag.  I think the term "dismiss" is
misleading because it implies both possible deletion and possible
change of the view.

> could invent a 2nd version of expunge for greater efficiency... :)

Do you want to be prompted with "Delete messages marked as Dismissed?"
every time you leave a netnews group where you've "dismissed" a
message?  If a cancel message is sent out for every message you've hit
"d" on in netnews, would you be getting your desire?

Unless your answer is "yes" for both these questions, then your use of
"\DELETED" in netnews really means "no-longer-interesting" which is a
very different concept from "flag for permanent removal".

		- Chris

---

From: Terry Gray <[email-masked]>
Date: Tue, 19 Oct 1993 08:24:49 -0700 (PDT)

> (particularly those which EXPUNGE without asking the user), not just

It's not a problem *if* the fact that it is non-destructive in some cases
does not alter people's motivation for using it, i.e. if it *always*
represents the *intention* to delete.  My view has been that people will
use D when they are done with the message, and this behavior would be
consistent regardless of whether or not a subsequent expunge succeeded. 

Having said that, would you agree that this is essentially a UI design
issue, and that your concern would completely go away if (for the
EXPUNGE-IS-NOT-ALLOWED class of mailboxes) the key label was something
less "final-sounding" than "Delete" ??

> Do Macintosh users who throw a program

It's a good analogy, which I agree weakens the case for using \deleted
as an inter-user communication mechanism.  Likewise, I agree that clients
with auto-expunge are not well-suited for trying to do this in a shared
environment.

Note that the trashcan metaphor may strengthen my case about user behavior
modification in the RO case, since I'm pretty certain I've gotten the Mac
error "Can't empty the trash" a time or two in my life, and that never
caused me to change the way I use Delete (I mean, the way I put things in
the trash :).  But I'm willing to stop pretending to be a psychologist if
you agree that a change in key label for the no-expunge case solves the
problem. 
 
> to flag the message for review by her superior.

This is very different from how I'd like to use \FLAGGED... but maybe a 
user flag is a better answer for the shared case.
 
> that you would be surprised by the results.

I don't doubt that "delete" means "delete" to most folks, I just doubt
that using the "Delete" word in the no-expunge case will cause people to
remap the word in there minds to mean something different. 
 
Pine is careful to use the phrase "Marked as Deleted" in order to reduce
the inherent confusion between "delete" and "expunge", and Pine users have
no basis for the idea that marking a message with "D" might make it
disappear from the current view without further action, so they don't do
"D" to simply make something disappear from the current view. 

> message?  

This would never happen.  Remember that you don't want to prompt for 
expunge if you don't have expunge rights.

(And note that these issues apply to any RO archive, not just news.)
 
> If a cancel message is sent out for every message you've hit

No, which I expect you knew.  My mindset is that people will normally
behave in a personal context, and it is important for the UI to explicitly
warn them when an action has global implications that may affect many people.
This would be true for either posting or cancelling.

-teg

---

From: Adam Treister <[email-masked]>
Date: Tue, 19 Oct 93 11:41:11 -0800

>   Pine is careful to use the phrase "Marked as Deleted" in
>   order to reduce the inherent confusion between "delete"
>   and "expunge", and Pine users have no basis for the idea
>   that marking a message with "D" might make it disappear
>   from the current view without further action, so they
>   don't do

In my mind, "Marked as Deleted" confuses the issue more.  Is it deleted or just
marked as deleted.  Its its deleted, then what's expunge?  As similar as the
concept is to the trash can on the mac, users have a tough time understanding
whats up.

I find the Delete / Expunge issue is the hardest part of writing an imap client.
I am trying to shield the user from this feature entirely, largely because of
imap's remapping message numbers on the expunge (so I want to do it when the
client is about to close down a mailbox), but also because the user doesn't care
about storage, just presentation.

My current thinking in Mailstrom is to support a Hide Deleted Messages option,
so that when the user "hits d", the message disappears.  I used to be actually
removing the message from my view (but not the mailbox), but that made the
ability to reshow the messages difficult.  Now, I'm changing it so that,
internally, hiding the message is just setting the row height of that message to
0.  This allows me to later restore the original row height and bring messages
back.  The important point is that as I read my mail and delete messages, I want
to see the list shrinking, but I may also want to look at all those messages
again.

Its also my intention to support a "Deleted Messages" folder so that hitting the
delete button will actually MOVE the message to the deleted-messages folder,
setting the delete flag in the process.  That folder can have a process
invisible to the user, which searches for messages that have been there over N
days, and delete and expunge them without user interaction.  For the user, this
is a big gain, they can get back deleted, expunged messages 30 or 60 days after
they trashed it.

I'd like to see this be a server/system feature aot just something in my client.
(Yes that means I should cross post to IMAP list) It also makes sense to have
the expiration occur on the server side, regardless of whether or not the client
has been run.  As I recall, IMSP is planning to support a "Sent-Messages" folder
as a standard element (ala Inbox).  What about raising "Deleted-Messages" to
that level?

Adam

---

From: John Gardiner Myers <[email-masked]>
Date: Tue, 19 Oct 1993 16:19:55 -0400 (EDT)

Adam Treister <[email-masked]> writes:
> client is about to close down a mailbox)

An IMAP server can (and the Cyrus IMAP server WILL) send * n EXPUNGE
notifications when processing *any* command other than FETCH, STORE,
or SEARCH.

---

From: Mark Crispin <[email-masked]>
Date: Tue, 19 Oct 1993 13:26:47 -0700 (PDT)

> client is about to close down a mailbox)

You can get expunge events even when you didn't issue an EXPUNGE, because some
other process expunged the mailbox.  It is not safe to assume that this won't
happen in the middle of your session, so you have to deal with the
renumbering.  Instead of thinking of message n as ``message number n'', think
of it as being ``the nth extant message''.

---

From: John Gardiner Myers <[email-masked]>
Date: Wed, 20 Oct 1993 12:02:11 -0400 (EDT)

> consistent regardless of whether or not a subsequent expunge succeeded. 

Your model that "people use D when they are done with the message" is
fundamentally inconsistent and interacts dangerously with the IMAP
model of "\Deleted causes message to be permanently removed upon
EXPUNGE".  "User disinterest" is inherently a per-user concept,
"permanent removal" is inherently a global concept.

> less "final-sounding" than "Delete" ??

Changing the label does not get rid of the interaction.  Changing the
label to paper over the "permanent removal" interaction is dangerous.

If you want a different label and set of sementics, use a different flag.

> remap the word in there minds to mean something different. 

If it has different semantics, it will cause users to remap the concept.

Mark Crispin <[email-masked]> writes:
> there was something arguably useful to do for the user.

I would have to disagree with this.  In user interfaces, it is better
to present a simple, clear model and to provide immediate feedback
when the user strays from that model than it is to provide every
possible feature the user might want.  I believe this concept has some
application to protocols as well.  Systems, such as the DWIM LISP's,
which attempt to provide some possibly-useful interpretation to
everything they're fed are generally reviled.

> So far, he arguments against this use of \Deleted presume that this
> is taking place in the mailbox namespace.  It is not.  It can not.
> 
> the bboard namespace.

Our arguments don't presume that this takes place in the SELECT
namespace.  Our arguments presume this creates a model that the
users will carry over to objects in the SELECT namespace.

While we won't be using servers which have objects which exist only in
the BBOARD namespace, we will be using clients which necessarily will
know how to deal with objects in the BBOARD namespace.  

To the extent where objects in the BBOARD namespace are not merely
second-class citizens but strange beasts, this will cause
interoperability problems.  Clients like Pine will not deal well with
news-like objects which exist in the SELECT namespace.  Other clients
will have the choice of not handling news-like objects in the SELECT
namespace well, not handling the BBOARD namepace well, or having a lot
of compatibility code to handle/hide the semantic differences between
the two.

> As I see the REAL problem, the question is one of determining access rights.

This is an interesting problem.  One solution would be for IMAP to
pick up IMSP's GETACL, but there's the question of whether the access
right bits in IMSP are sufficiently fine-grained.

> The present Pine doesn't distinguish between

Neither does the Cyrus imapd.  One way of stating the CMU model is
"everything is a shared mailbox, though the name 'INBOX' is magic."