Subject: [Imap-uw] Negative usage and libc-client parsing

From: John Morrissey <[email-masked]>
Date: Fri, 11 Nov 2005 14:09:51 -0500

Hello all,

We use libc-client to access Courier IMAP, which stores mail in Maildir++
format. During normal operation, it's possible for Maildir++ quota usage to
become negative (less than zero bytes used), like so:

[jwm@webmail06:pts/1 ~> telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc.  See COPYING for distribution information.
a login johnmorr5 7houting
a OK LOGIN Ok.
b getquotaroot INBOX
* QUOTAROOT "INBOX" "ROOT"
* QUOTA "ROOT" (STORAGE 0 10240 MESSAGE -1 10000)
b OK GETQUOTAROOT Ok.

However, imap_parse_unsolicited() seems to assume quota usage will always be
positive and uses strtoul() to parse these values. As a result, parsing
QUOTA strings with negative values always yields "Bad quota resource list
for ROOT."

It seems like a simple matter to use strtol() and expand the isdigit()
checks to include "-" for negative values. I can submit a patch, but wanted
to get some feedback on my observations first.

john
-- 
John Morrissey          _o            /\         ----  __o
[email-masked]        _-< \_          /  \       ----  <  \,
www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: Mark Crispin <[email-masked]>
Date: Fri, 11 Nov 2005 11:35:17 -0800 (PST)

All numeric values in IMAP are unsigned; there is no such thing as a 
negative numeric value in the IMAP protocol.  IMAP has two types 
of numeric value:
  number		a value between 0 and 4,294,967,295 inclusive
  nz-number	a value between 1 and 4,294,967,295 inclusive.

The IMAP QUOTA specification (RFC 2087, page 5) defines all the numeric 
values used by quota as "number".

A change to use strtol() instead of strtoul() would break the handling of 
half of the available number space (from 2,147,483,648 to 4,294,967,295). 
Consequently, such a patch would not be accepted.

Sadly, this is (yet another) bug in Courier.  There are numerous known 
issues in which Courier fails to comply with the IMAP specification; and 
past history indicates that there is little (if any) hope that these 
issues will ever be fixed.

If Maildir format is a requirement for you, consider switching to the 
Dovecot server.  I don't know very much about it, but its author 
participates in the IMAP community and seems willing to make changes in 
his software as needed to require with the IMAP specification.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: John Morrissey <[email-masked]>
Date: Tue, 15 Nov 2005 13:04:49 -0500

> issues will ever be fixed.

Yes, I would also rather avoid trying to convince Courier's maintainer to
change this.

Would a patch be accepted that detected negative usage and reported zero in
that case while still using strtoul() to achieve the full range of an RFC
2087 number? Zero is the de facto result of negative usage, and it would
avoid throwing an error. It's almost cliched to say, but this would make
libc-client more liberal in what it accepts, without impacting compliant
implementations.

john

---

From: Mark Crispin <[email-masked]>
Date: Tue, 15 Nov 2005 12:36:36 -0800 (PST)

> Yes, I would also rather avoid trying to convince Courier's maintainer to

:-)

> libc-client more liberal in what it accepts, without impacting compliant

I don't want to do this by default, but the attached patch will do this if 
you specify "/loser" in the mailbox name.  You need to use "/loser" with 
Courier servers anyway.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

*** imap4r1.c~	2005-11-04 09:04:41.000000000 -0800
--- imap4r1.c	2005-11-15 10:53:43.000000000 -0800
***************
*** 4107,4116 ****
        if (*s != ')') for (ql = qc = mail_newquotalist (); T;
  			  qc = qc->next = mail_newquotalist ()) {
  	if ((qc->name = imap_parse_astring (stream,&s,reply,NIL)) && s &&
! 	    (*s++ == ' ') && isdigit (*s)) {
! 	  qc->usage = strtoul (s,(char **) &s,10);
! 	  if ((*s++ == ' ') && isdigit (*s)) {
! 	    qc->limit = strtoul (s,(char **) &s,10);
  				/* another resource follows? */
  	    if (*s == ' ') continue;
  				/* end of resource list? */
--- 4107,4118 ----
        if (*s != ')') for (ql = qc = mail_newquotalist (); T;
  			  qc = qc->next = mail_newquotalist ()) {
  	if ((qc->name = imap_parse_astring (stream,&s,reply,NIL)) && s &&
! 	    (*s++ == ' ') && (isdigit (*s) || (LOCAL->loser && (*s == '-')))) {
! 	  if (isdigit (*s)) qc->usage = strtoul (s,(char **) &s,10);
! 	  else if (t = strchr (s,' ')) t = s;
! 	  if ((*s++ == ' ') && (isdigit (*s) || (LOCAL->loser &&(*s == '-')))){
! 	    if (isdigit (*s)) qc->limit = strtoul (s,(char **) &s,10);
! 	    else if (t = strpbrk (s,' )')) t = s;
  				/* another resource follows? */
  	    if (*s == ' ') continue;
  				/* end of resource list? */
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: John Morrissey <[email-masked]>
Date: Wed, 16 Nov 2005 16:03:32 -0500

> you specify "/loser" in the mailbox name.  You need to use "/loser" with 

Sounds good - thanks!

Before I enable the loser flag locally, could you confirm what it enables?
>From reading the source, it looks like:

* Anything that deals with sequences passes them through
  imap_reform_sequence() to fix any reversed sequences (i.e., x:y where
  x > y).
* Unsets adr->personal if it contains '@'.
* Ignores the THREAD capability, if advertised.
* imap_send_astring() always quotes astrings.

Correct?

john

---

From: Mark Crispin <[email-masked]>
Date: Wed, 16 Nov 2005 14:22:06 -0800

> Before I enable the loser flag locally, could you confirm what it enables?

Sure.

I don't know what you mean by "enable the loser flag locally".  /loser is 
specified in the mailbox name, e.g.,
 	{server.example.com/loser}INBOX
when opening the maibox.  It should not be used unless the server is known 
to fail without it.  The code certainly should NOT be patched to set it.

> * imap_send_astring() always quotes astrings.

Correct.

In addition (this may be new in imap-2005) it also does all searches 
locally, because Courier doesn't handle non-trivial search criteria 
correctly and that in turn breaks Pine filters.

And, of course, this new thing in QUOTA responses to treat negative 
numbers as being 0.

Are you sure that you really want to use Courier?  I can understand if you 
want to use maildir format (I won't get into the theological discussion 
about maildir) but there is at least one alternative (Dovecot).

-- Mark --

---

From: John Morrissey <[email-masked]>
Date: Thu, 17 Nov 2005 15:56:04 -0500

> 	{server.example.com/loser}INBOX

Yes, sorry. I should have been more specific and said "specify the loser
flag locally in our webmail software."

> about maildir) but there is at least one alternative (Dovecot).

When we launched our webmail offering, Courier was the only IMAP server to
offer all the functionality we needed (LDAP auth, Maildir++ quotas, etc.).
I would love to move to Dovecot since it's more standards compliant and the
source is definitely cleaner, but Maildir++ quota support is currently a
third-party patch which lacks some things we need. It looks like it returns
0 usage in response to RFC 2087 quota commands (in other words, it tracks
usage, but doesn't report it). It also can't pull an updated quota limit
from your user database, which would be useful to us when a user's quota is
up- or downgraded.

Thanks again for your help, Mark. I really appreciate it.

john