Subject: Directory that is a mailbox

From: "Jean-Luc Wasmer" <[email-masked]>
Date: Thu, 21 Aug 2003 23:04:40 -0400

Hi,

The FAQ section 1.10 says:

"Some mailbox formats, including the default which is the traditional UNIX
mailbox format, are stored as a single file containing all the messages.
UNIX does not permit a name in the filesystem to be both a file and a
directory; consequently you can not have a sub-mailbox within a mailbox that
is in one of these formats.
[...]
It is always permissible to create a directory that is not a mailbox, and
have sub-mailboxes under it."

I use this a lot. Unfortunatly, if I select a directory I get an error
because it's not a mailbox.

Would it be possible to interpret a file wih a special name (eg. "_") as the
mailbox of the parent directory?

So having the following file system tree:

Clients/
    _
    ClientA
    ClientB
    ClientC

would appear to the IMAP client like

Clients
    ClientA
    ClientB
    ClientC

with Clients ClientA ClientB and ClientC four different mailboxes.

Mail in the "Clients" mailbox would be stored in the file Clients/_

Thanks,

JL

-- 
------------------------------------------------------------------
 For information about this mailing list, and its archives, see: 
 http://www.washington.edu/imap/c-client-list.html
------------------------------------------------------------------

---

From: Creighton MacDonnell <[email-masked]>
Date: Fri, 22 Aug 2003 08:31:13 -0600

I have a patch that I use on my version of IMAP that does this. I have 
attached it.

I do not vouch for its correctness (particularly regarding file locking) 
or usefulness. It also reflects all of my confgiuration changes, some of 
which have nothing to do with "Directory that is a mailbox".

I have responded in this way because I too like to have directories that 
are also mailboxes.

I have also responded because I ma tired of maintining this patch, and 
worrying that it will be too much work to retest it with each release. I 
want to encourage discussion on this list about some sort of extension 
to IMAP to provide this.

The patch hacks the "unix" mailbox driver. Instead of "_" it uses 
".mail" as a "special" file name. The "hideDotFiles = T" configuration 
option is also important. It should of course be a seperate driver, and 
leave "unix" intact.

The patch used to do the same for the "mbx" mailbox driver (which is the 
preferred format I guess). But this stopped working a couple of releases 
ago, and I could not muster the desire to fix it. I might be willing to 
contribute some of my own time to adapt the technique to the standards 
and preferences of the UW-IMAP stewards, under their instruction, if the 
result found its way into the core code. But I may not have the depth of 
experience that they would want.

-- 
Creighton MacDonnell
http://macdonnell.ca/
--- ./src/imapd/Makefile~	2002-11-18 09:48:25.000000000 -0700
+++ ./src/imapd/Makefile	2003-04-23 10:47:58.000000000 -0600
@@ -29,7 +29,7 @@
 # causes the "Manage Mail" menu item to open the given URL, e.g. to point to
 # an alternative IMAP client (e.g. Pine) or perhaps to a homebrew mail
 # account management page.
-#NSBD= -DNETSCAPE_BRAIN_DAMAGE=\"http://www.washington.edu/pine\"
+NSBD= -DNETSCAPE_BRAIN_DAMAGE=\"http://www.washington.edu/pine\"

--- ./src/osdep/unix/env_unix.c~	2003-04-16 15:03:26.000000000 -0600
+++ ./src/osdep/unix/env_unix.c	2003-04-23 11:25:10.000000000 -0600
@@ -29,7 +29,7 @@
 static char *myMailboxDir = NIL;/* mailbox directory name */
 static char *myLocalHost = NIL;	/* local host name */
 static char *myNewsrc = NIL;	/* newsrc file name */
-static char *mailsubdir = NIL;	/* mail subdirectory name */
+static char *mailsubdir = "mail";   /* mail subdirectory name */
 static char *sysInbox = NIL;	/* system inbox name */
 static char *newsActive = NIL;	/* news active file */
 static char *newsSpool = NIL;	/* news spool */
@@ -44,15 +44,15 @@
 static short anonymous = NIL;	/* is anonymous */
 static short blackBox = NIL;	/* is a black box */
 static short closedBox = NIL;	/* is a closed box */
-static short restrictBox = NIL;	/* is a restricted box */
+static short restrictBox = T;	/* is a restricted box */
 static short has_no_life = NIL;	/* is a cretin with no life */
 				/* flock() emulator is a no-op */
 static short disableFcntlLock = NIL;
-static short hideDotFiles = NIL;/* hide files whose names start with . */
+static short hideDotFiles = T;	/* hide files whose names start with . */
 				/* advertise filesystem root */
 static short advertisetheworld = NIL;
 				/* disable automatic shared namespaces */
-static short noautomaticsharedns = NIL;
+static short noautomaticsharedns = T;
 static short no822tztext = NIL;	/* disable RFC [2]822 timezone text */
 static short netfsstatbug = NIL;/* compensate for broken stat() on network
 				 * filesystems (AFS and old NFS).  Don't do
@@ -804,7 +804,7 @@
 {
   char tmp[MAILTMPLEN];
   if (!sysInbox) {		/* initialize if first time */
-    sprintf (tmp,"%s/%s",MAILSPOOL,myusername ());
+    sprintf (tmp,"%s/mail/.inbox",myhomedir ());
     sysInbox = cpystr (tmp);	/* system inbox is from mail spool */
   }
   return sysInbox;
--- ./src/osdep/unix/env_unix.h~	2002-02-22 21:03:45.000000000 -0700
+++ ./src/osdep/unix/env_unix.h	2003-04-23 10:47:58.000000000 -0600
@@ -33,8 +33,8 @@
 
 /* Subscription definitions for UNIX */
 
-#define SUBSCRIPTIONFILE(t) sprintf (t,"%s/.mailboxlist",myhomedir ())
-#define SUBSCRIPTIONTEMP(t) sprintf (t,"%s/.mlbxlsttmp",myhomedir ())
+#define SUBSCRIPTIONFILE(t) sprintf (t,"%s/mail/.mailboxlist",myhomedir ())
+#define SUBSCRIPTIONTEMP(t) sprintf (t,"%s/mail/.mlbxlsttmp",myhomedir ())

 /* dorc() options */
--- ./src/osdep/unix/unix.c~	2003-01-07 13:23:16.000000000 -0700
+++ ./src/osdep/unix/unix.c	2003-04-23 10:47:58.000000000 -0600
@@ -97,6 +97,20 @@
 				/* driver parameters */
 static long unix_fromwidget = T;
 
+/* UNIX mail generate file string
+ * Accepts: temporary buffer to write into
+ *	    mailbox name string
+ * Returns: local file string or NIL if failure
+ */
+
+char *unix_file (char *dst,char *name)
+{
+  char *s = mailboxfile (dst,name);
+  if (!s) return s;
+  if (!*s) return strcpy (dst,sysinbox ());
+  return strcat(dst,"/.mail");
+}
+
 /* UNIX mail validate mailbox
  * Accepts: mailbox name
  * Returns: our driver if name is valid, NIL otherwise
@@ -111,7 +125,7 @@
   time_t tp[2];
   errno = EINVAL;		/* assume invalid argument */
 				/* must be non-empty file */
-  if ((t = dummy_file (file,name)) && !stat (t,&sbuf)) {
+  if ((t = unix_file (file,name)) && !stat (t,&sbuf)) {
     if (!sbuf.st_size)errno = 0;/* empty file */
     else if ((fd = open (file,O_RDONLY,NIL)) >= 0) {
 				/* OK if mailbox format good */
@@ -214,10 +228,15 @@
   long ret = NIL;
   int i,fd;
   time_t ti = time (0);
-  if (!(s = dummy_file (mbx,mailbox))) {
+  if (!(s = unix_file (mbx,mailbox))) {
     sprintf (tmp,"Can't create %.80s: invalid name",mailbox);
     MM_LOG (tmp,ERROR);
   }
+				/* no hidden files or directories */
+  else if (mailbox[0] == '.' || strstr(mailbox,"/.")) {
+    sprintf (tmp,"Can't create %.80s: invalid name - leading period",mailbox);
+    MM_LOG (tmp,ERROR);
+  }
 				/* create underlying file */
   else if (dummy_create_path (stream,s,get_dir_protection (mailbox))) {
 				/* done if made directory */
@@ -284,41 +303,32 @@
   long i;
   struct stat sbuf;
   MM_CRITICAL (stream);		/* get the c-client lock */
-  if (newname && !((s = dummy_file (tmp,newname)) && *s))
+  if (newname && !((s = unix_file (tmp,newname)) && *s))
     sprintf (tmp,"Can't rename mailbox %.80s to %.80s: invalid name",
 	     old,newname);
 				/* lock out other c-clients */
-  else if ((ld = lockname (lock,dummy_file (file,old),LOCK_EX|LOCK_NB,&i)) < 0)
+  else if ((ld = lockname (lock,unix_file (file,old),LOCK_EX|LOCK_NB,&i)) < 0)
     sprintf (tmp,"Mailbox %.80s is in use by another process",old);
   else {
     if ((fd = unix_lock (file,O_RDWR,S_IREAD|S_IWRITE,&lockx,LOCK_EX)) < 0)
       sprintf (tmp,"Can't lock mailbox %.80s: %s",old,strerror (errno));
     else {
       if (newname) {		/* want rename? */
-				/* found superior to destination name? */
-	if (s = strrchr (s,'/')) {
-	  c = *++s;		/* remember first character of inferior */
-	  *s = '\0';		/* tie off to get just superior */
-				/* name doesn't exist, create it */
-	  if ((stat (tmp,&sbuf) || ((sbuf.st_mode & S_IFMT) != S_IFDIR)) &&
-	      !dummy_create_path (stream,tmp,get_dir_protection (newname))) {
-	    unix_unlock (fd,NIL,&lockx);
-	    unix_unlock (ld,NIL,NIL);
-	    unlink (lock);
-	    MM_NOCRITICAL (stream);
-	    return ret;		/* return success or failure */
-	  }
-	  *s = c;		/* restore full name */
-	}
-	if (rename (file,tmp))
-	  sprintf (tmp,"Can't rename mailbox %.80s to %.80s: %s",old,newname,
-		   strerror (errno));
-	else ret = T;		/* set success */
+        unix_unlock (fd,NIL,&lockx);	/* lock file is in directory to rename */
+        fd = -1;
+        if (!(ret = dummy_rename(stream,old,newname)))	/* rename parent directory */
+          sprintf (tmp,"Can't rename mailbox directory %.80s to %.80s: %s",old,newname,
+                   strerror (errno));
       }
       else if (unlink (file))
-	sprintf (tmp,"Can't delete mailbox %.80s: %s",old,strerror (errno));
-      else ret = T;		/* set success */
-      unix_unlock (fd,NIL,&lockx);
+        sprintf (tmp,"Can't delete mailbox \".mail\" file %.80s: %s",old,strerror (errno));
+      else {				/* delete parent directory too */
+        unix_unlock (fd,NIL,&lockx);	/* lock file is in directory to delete */
+        fd = -1;
+        if (!(ret = dummy_delete (stream,old)))
+          sprintf (tmp,"Can't delete mailbox directory %.80s: %s",old,strerror (errno));
+      }
+      if (fd >= 0) unix_unlock (fd,NIL,&lockx);
     }
     unix_unlock (ld,NIL,NIL);	/* flush the lock */
     unlink (lock);
@@ -348,7 +358,7 @@
 				/* note if an INBOX or not */
   stream->inbox = !compare_cstring (stream->mailbox,"INBOX");
 				/* canonicalize the stream mailbox name */
-  dummy_file (tmp,stream->mailbox);
+  unix_file (tmp,stream->mailbox);
 				/* flush old name */
   fs_give ((void **) &stream->mailbox);
 				/* save canonical name */
@@ -761,7 +771,7 @@
   }
   LOCAL->buf[0] = '\0';
   MM_CRITICAL (stream);		/* go critical */
-  if ((fd = unix_lock (dummy_file (file,mailbox),O_WRONLY|O_APPEND|O_CREAT,
+  if ((fd = unix_lock (unix_file (file,mailbox),O_WRONLY|O_APPEND|O_CREAT,
 		       S_IREAD|S_IWRITE,&lock,LOCK_EX)) < 0) {
     MM_NOCRITICAL (stream);	/* done with critical */
     sprintf (LOCAL->buf,"Can't open destination mailbox: %s",strerror (errno));
@@ -903,7 +913,7 @@
   i = sbuf.st_size;		/* size of scratch file */
 
   MM_CRITICAL (stream);		/* go critical */
-  if (((fd = unix_lock (dummy_file (file,mailbox),O_WRONLY|O_APPEND|O_CREAT,
+  if (((fd = unix_lock (unix_file (file,mailbox),O_WRONLY|O_APPEND|O_CREAT,
 		       S_IREAD|S_IWRITE,&lock,LOCK_EX)) < 0) ||
       !(df = fdopen (fd,"ab"))) {
     MM_NOCRITICAL (stream);	/* done with critical */
--- ./src/osdep/unix/Makefile~	2003-03-04 22:26:35.000000000 -0700
+++ ./src/osdep/unix/Makefile	2003-04-23 10:47:58.000000000 -0600
@@ -23,13 +23,13 @@
 EXTRAAUTHENTICATORS=
 EXTRADRIVERS=mbox
 PASSWDTYPE=std
-SSLTYPE=nopwd
+SSLTYPE=unix

 # Extended flags needed for SSL.  You may need to modify.
 
 SSLDIR=/usr/local/ssl
-SSLCERTS=$(SSLDIR)/certs
+SSLCERTS=/etc/imap/SSL
 SSLINCLUDE=$(SSLDIR)/include
 SSLLIB=$(SSLDIR)/lib
 
@@ -115,7 +115,7 @@
  dummy.o pseudo.o netmsg.o flstring.o fdstring.o \
  rfc822.o nntp.o smtp.o imap4r1.o pop3.o \
  unix.o mbox.o mbx.o mmdf.o tenex.o mtx.o news.o phile.o mh.o mx.o
-CFLAGS=-g
+CFLAGS=
 
 CAT=cat
 MAKE=make
@@ -613,7 +613,7 @@
 	 SPOOLDIR=/var/spool \
 	 ACTIVEFILE=/var/lib/news/active \
 	 RSHPATH=/usr/bin/rsh \
-	 BASECFLAGS="-g -fno-omit-frame-pointer -O6" \
+	 BASECFLAGS="-fno-omit-frame-pointer -O6" \
 	 BASELDFLAGS="-lcrypt"
 
 sl4:	# Secure Linux using libc4

---

From: "Jean-Luc Wasmer" <[email-masked]>
Date: Fri, 22 Aug 2003 10:59:49 -0400

> I have a patch that I use on my version of IMAP that does this. I have

I never expected to get a positive answer for my request...
I received a patch... incredible!

> want to encourage discussion on this list about some sort of extension

I'm new to this list... did you submit your patch to the maintainer of
UW-IMAP in the past?
It could be part of the default release but only activated by some
configuration option.

> ".mail" as a "special" file name.

I agree, ".mail" is more appropriate than "_".
I suggested "_" since it's what my directory mailbox file is called right
now (in order to be the first in the mailbox list).

> The patch used to do the same for the "mbx" mailbox driver

I use the unix driver.

JL

---

From: Creighton MacDonnell <[email-masked]>
Date: Fri, 22 Aug 2003 10:57:38 -0600

> I received a patch... incredible!

Let me emphasize again that I am not promising to support this or to 
ensure it works with new releases.

> It could be part of the default release but only activated by some

No. I did not.

I don't even know if the UW-IMAP devleopment team is interested in input 
of this sort from the user community. I only joined this mailing list 
recently myself.

---

From: Mark Crispin <[email-masked]>
Date: Fri, 22 Aug 2003 10:22:40 -0700 (PDT)

> of this sort from the user community. I only joined this mailing list

We intend to support some kind of mailbox-directory function in the
future as an addition (not in lieu) to existing functionality.  The exact
nature of this is still under consideration.

We encourage people such as you to develop and distribute patches, even if
we do not choose to adopt them in the official distribution.  We receive
many suggested patches, and can adopt only a small number of them; but all
suggested patches are carefully considered.

Please send a copy of the patch to me and I will look at it.  If nothing
else, I can offer suggestions for your patch so that it is likely to
remain usable in future releases.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: "Jean-Luc Wasmer" <[email-masked]>
Date: Fri, 22 Aug 2003 21:25:47 -0400

> ensure it works with new releases.

You were perfectly clear... I was just surprised to realize I was not the
only one wanting this feature.

Thanks again for the patch...

JL

---

From: Creighton MacDonnell <[email-masked]>
Date: Sun, 24 Aug 2003 12:54:08 -0600

> remain usable in future releases.

On the off chance that my patch may be useful to others, I have polished 
it up a bit.

And I would indeed appreciate your comments, Mark.

A tar ball is available at:

   http://gnudip2.sourceforge.net/imap/

The README file is available at:

   http://gnudip2.sourceforge.net/imap/patch/README

===

It should be much easier to use now.

It is free of my configuration details.

It does not change the exiting UNIX driver. It adds a new "UNIXSUB" 
driver - as the default driver. So existing folders will still be 
accessible (unless there is another folder in the same directory named 
".mail"!). Only new ones have the new structure.

It is no longer necessary to hide "dot files", and folder names 
beginning with a dot are allowed.

It is actually two patches.

The first patch is a minimal change to the IMAP base code. It is in 
"imap.diff", and these are its contents:

--- src/osdep/unix/dummy.c~	2003-03-05 17:37:40.000000000 -0700
+++ src/osdep/unix/dummy.c	2003-08-23 19:42:27.000000000 -0600
@@ -233,8 +233,11 @@
    struct stat sbuf;
    int ismx;
    char tmp[MAILTMPLEN];
+  char is_mailbox = 0;
  				/* punt if bogus name */
    if (!mailboxdir (tmp,dir,NIL)) return;
+				/* is this a mail box? */
+  is_mailbox = (mail_valid(NIL, tmp, NIL) != NIL);
    if (dp = opendir (tmp)) {	/* do nothing if can't open directory */
  				/* list it if not at top-level */
      if (!level && dir && pmatch_full (dir,pat,'/'))
@@ -274,6 +277,7 @@
  	      dummy_list_work (stream,tmp,pat,contents,level+1);
  	    break;
  	  case S_IFREG:		/* ordinary name */
+	    if (!is_mailbox)	/* ignore regular file within mailbox */
  				/* ignore all-digit names from mx */
  	    /* Must use ctime for systems that don't update mtime properly */
  	    if (!(ismx && mx_select (d)) && pmatch_full (tmp,pat,'/') &&

The "imap.diff" patch introduces into UW-IMAP the notion that a 
directory can be a mail box. And for a directory that is a mail box,it 
will ignore any regular files (i.e. not subdirectories) within that 
directory when scanning the file system. The result is that these will 
not be visible to an IMAP client. It is up to the mailbox driver for the 
directory to use these files.

It is my hope that "imap.diff", or some equivalent change will be 
accepted into UW-IMAP, whether or not the driver itself is.

The rest of the package is the actual driver, which is still really just 
a hack of the UNIX driver. I would never expect this to get into the 
UW-IMAP code. But I think it makes a case for the technique used to 
allow folders within folders. One short coming I can think of is that 
there should be something in the ".mail" files that is unique to this 
driver, which would then be used within the "VALID" macro to ensure that 
a ".mail" file for this driver is never mistaken for a UNIX driver folder.

Thanks.