Subject: [Imap-uw] IDLE in c-client

From: Mark Sirota <[email-masked]>
Date: Fri, 16 Dec 2005 10:53:26 -0500

Here at Penn we're working on a c-client based IMAP client for a
large-scale unified messaging project, which will handle voicemail for
several thousand IP telephony customers.

We're debating whether to develop an IDLE interface in the c-client or
just use a mail_ping() loop.  The client will need to monitor everyone's
mailbox in order to maintain the status of the message waiting indicator
light on the VOIP telephone.

While the mail_ping() loop is functionally identical to IDLE, it seems to
me that the tradeoff is whether to put the resource burden on the client
or the server, which may start mattering when we get into the several
thousand mailbox level, depending on the scalability of the client and
server architectures and the specifics of the IDLE implementation on the
server.

Are there any plans to implement IDLE in a future release of the c-client?
If I ended up writing it myself, would you welcome the contribution?

Thanks,
-- 
Mark Sirota, Associate Director, Network Engineering and Services
University of Pennsylvania, Information Systems and Computing
[email-masked], +1 215 573 7214

_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: Mark Crispin <[email-masked]>
Date: Fri, 16 Dec 2005 09:00:46 -0800 (PST)

> server architectures and the specifics of the IDLE implementation on the

I don't think that this is the correct view of the tradeoff, although it 
may be a factor in a server architecture in which email delivery causes 
event notification.

The usual reason for IDLE is to reduce network traffic.  It also can help 
in the case of mobile clients by reducing battery consumption.  This is a 
big issue, especially with IMAP-capable cell phones.

The flip side is that IDLE has proven disasterous if there is any sort of 
NAT or similar facility between client and server.  The inactivity causes 
the NAT mapping to be purged, and then when the server does send something 
the session gets killed.  NAT and similar facilities presume HTTP and such 
protocols in which activity is client initiated.  Server-initiated 
activity, such as IDLE, is not accomodated at all.

As a consequence, some server implementations (such as UW imapd) end up 
sending an untagged OK every two minutes or so, with the sole purpose of 
causing a TCP ACK from the client and keeping the NAT mapping alive.  The 
mobile device guys complain about this behavior (and have good reason to 
do so!) but no obvious solution has come up.

The upshot is that many people feel that IDLE was a good idea for its 
time, but not the right idea for modern needs; and that a separate "push" 
type notification mechanism is what's really needed for mobile devices.

As IMAP is fundamentally a "pull" type protocol, such notifications would 
be done by a mechanism outside of IMAP.  This is actually not a bad thing; 
if all you want to do is light the little "voice mail" light on the phone, 
you don't really need to have an IMAP session monitoring the mailbox 24/7. 
You just need a notification from whatever agent *delivers* the voice 
mail, and perhaps also some mechanism in the IMAP server when all new 
voice mail has been read to turn the little light off.  This will scale 
better, since then the IMAP session is only activated when the user 
requests it.

> If I ended up writing it myself, would you welcome the contribution?

I'll look at it, but I can't promise that I would adopt it.  Nor can I 
promise that if I adopt it, that it wouldn't have a different interface.

c-client is fundamentally synchronous in its implementation of IMAP, and 
IDLE is very much asynchronous.  It'll need careful consideration.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: Mark Sirota <[email-masked]>
Date: Fri, 16 Dec 2005 13:34:11 -0500

> better, since then the IMAP session is only activated when the user

Thanks for the suggestion, it sounds worth pursuing.  We'll look into it.

We've already got the delivery side.  It's easy when a message is played
via the telephone interface, because that's the same software that manages
the message waiting indicator.  The challenge is when someone listens to a
voice message by way of their regular IMAP e-mail client.

Perhaps whenever a message is read, the IMAP server could send a message
to an external routine that would determine whether this was the last
unread voicemail message in the mailbox.  Sounds more complex, and
requires customizations to the IMAP server, but FAR more scalable than the
"every mailbox open, selected, and idle" approach we'd been considering.

So far, the only customizations we require on the IMAP server is the
ability to trust a specific client to properly authenticate the user.
That is, the IMAP client authenticates the user, then authenticates to
the IMAP server as itself rather than as the user.  The IMAP server
authenticates the client, trusts that the client has authenticated the
user, and allows the client to access any mailbox.

This is the same approach we use for webmail, so it's not a big deal
for us.

Mark

---

From: Mark Crispin <[email-masked]>
Date: Fri, 16 Dec 2005 18:37:24 -0800 (Pacific Standard Time)

On Fri, 16 Dec 2005, Mark Sirota wrote:
> unread voicemail message in the mailbox.

I think that this depends upon the store that holds the voice mail 
messages.  Assuming that it is IMAP-friendly and holds \Seen status, then 
either the store itself can recognize that the number of not-\Seen 
messages has fallen to zero, or the driver for that store in the IMAP 
server can do something like the following in the "flag" method (the one 
called for the entire mailbox at the end of flagging, as opposed to the 
"flagmsg" method):
   for (i = 1,lite = NIL; !lite && (i <= stream->nmsgs); ++i)
     if (mail_elt (stream,i)->seen) lite = T;

Then turn the light off iff lite is NIL afterwards.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw