Subject: [OT?] mailutil wants a cert

From: Thomas L Roche <[email-masked]>
Date: Sun, 14 Sep 2003 22:07:49 -0400

I'm using Thunderbird 0.2 on win2k (with cygwin) to access a large
IMAP account. The account has a lot of mail, and is
"well-structured": lots of folders, fairly deep tree. Unfortunately
the account is at a university from which I recently graduated, and
will soon disappear into /dev/null.

I would like to download all the mail in the account to local
filespace, but preserving the structure, and to automate the process.
I note that, when (in thunderbird) I drag the IMAP account's INBOX
onto the Local Folders, tb recreates the folder structure, but does
not copy all the mail: the contents of INBOX are copied, but not the
contents of any subfolder ... and I have a lot of subfolders :-(

mailutil was recommended to me, so I got it from

http://cygwin.com/cgi-bin2/package-cat.cgi?file=uw-imap-util/uw-imap-util-2002e-1

and noticed

(*manpages*)mailutil
> mailutil transfer [-debug] [-verbose] [-merge m] [-rwcopy] src dst
<big snip>
> ARGUMENTS

>   The arguments are standard c-client mailbox names. A variety of
>   mailbox name formats and types of mailboxes are supported by
>   c-client; examples of the most common forms of names are:

>   Name           Meaning

>   INBOX          primary incoming mail folder on the local system

>   archive/tx-project
>                  mail folder named "tx-project" in "archive"
>                  subdirectory of local filesystem home directory

>   {imapserver.foo.com}INBOX
>                  primary incoming mail folder on IMAP server
>                  system "imapserver.foo.com"

>   {imapserver.foo.com}archive/tx-project
>                  mail folder named "tx-project" in "archive"
>                  subdirectory on IMAP server system
>                  "imapserver.foo.com"

>   #news.comp.mail.misc
>                  newsgroup "comp.mail.misc" on local filesystem

So I tried

$ mailutil transfer -debug -verbose -merge prompt {me.mail.big.edu}INBOX 
/g/mail/

and got (something like :-)

[Trying IP address [xxx.xxx.xxx.xxx]]
* OK imap.big.edu Cyrus IMAP4 v2.1.11 server ready
[imap.big.edu Cyrus IMAP4 v2.1.11 server ready]
00000000 CAPABILITY
* CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ MAILBOX-REFERRALS 
NAMESPACE UIDPLUS ID NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND SORT 
THREAD=ORDEREDSUBJECT THREAD=REFERENCES IDLE STARTTLS AUTH=GSSAPI 
AUTH=KERBEROS_V4 LISTEXT LIST-SUBSC
RIBED X-NETSCAPE
00000000 OK Completed
00000001 STARTTLS
00000001 OK Begin TLS negotiation now
[Certificate failure for me.mail.big.edu: self signed certificate in 
certificate chain: /C=US/ST=Poor/L=Broke/O=Big State 
University/OU=IT/CN=BSU-IT CA/emailAddress=[email-masked]]

Unfortunately I know nothing about certificate generation. Can anyone
point me at a quick'n'dirty one-shot implementation? Or another
workaround for this problem? Or a completely different alternative?

I thought I might get around this by ssh-ing in to ssh.big.edu to run
mailutil from there, hoping that I would not need certification
"inside the firewall"; but there `which mailutil` fails. <sigh/>
I have also posted [email-masked], but based on past
experience and the current budgetary problems, I'm not holding my
breath <sigh x 2/>

If this is a FAQ, or the wrong forum for this question, please guide
me in the right direction. Otherwise, your help is appreciated.

-- 
------------------------------------------------------------------
 For information about this mailing list, and its archives, see: 
 http://www.washington.edu/imap/c-client-list.html
------------------------------------------------------------------

---

From: Mark Crispin <[email-masked]>
Date: Sun, 14 Sep 2003 19:18:38 -0700 (PDT)

On Sun, 14 Sep 2003, Thomas L Roche wrote:
> [...]

Since big.edu has a self-signed certificate, either you have to install
certificate on your local machine or you have to use the /novalidate-cert
option to disable certificate validation.  This goes inside the {}, e.g.
	{me.mail.big.edu/novalidate-cert}INBOX

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Thomas L Roche <[email-masked]>
Date: Sun, 14 Sep 2003 23:34:18 -0400

On Sun, 14 Sep 2003, Thomas L Roche wrote:
>> [My large IMAP] account has a lot of mail[,] is "well-structured"
>> [(]lots of folders, fairly deep tree[),] and will soon disappear
>> into /dev/null.

<big snip>

>> $ mailutil transfer -debug -verbose -merge prompt
>> {me.mail.big.edu}INBOX /g/mail/
>> [...]
>> [Certificate failure for me.mail.big.edu: self signed certificate
>> in certificate chain: /C=US/ST=Poor/L=Broke/O=Big State

Mark Crispin 09/14/2003 10:18 PM
> Since big.edu has a self-signed certificate, either you have to
> install certificate on your local machine or you have to use the
> /novalidate-cert option to disable certificate validation. This goes
> inside the {}, e.g.

Thanks! 

mailutil transfer -debug -verbose -merge prompt 
{me.mail.big.edu/novalidate-cert}INBOX /g/mail/

gets me authn prompts, and off we go ... until we hit substructure.
E.g.

- Copying 
{imap.big.edu:143/imap/tls/novalidate-cert/user="me"}INBOX.Drafts => 
/g/mail/INBOX/Drafts
- [Reusing connection to imap.big.edu/user="me"]
- [Mailbox is empty]
- Can't create mailbox node /g/mail/INBOX/: File exists
- alternative name: 

Naïvely I try
> /g/mail/INBOX/Drafts
but get dope-slapped

- Can't create mailbox node /g/mail/INBOX/: File exists

Is there a way to tell mailutil to just create a subdir to correspond
to the substructure? The alternative is, e.g.,

- Copying {imap.big.edu:143/imap/tls/novalidate-cert/user="me"}INBOX.GNU 
=> /g/mail/INBOX/GNU
- [Reusing connection to imap.big.edu/user="me"]
- [Mailbox is empty]
- Can't create mailbox node /g/mail/INBOX/: File exists
- alternative name: INBOX.GNU
- {imap.big.edu:143/imap/tls/novalidate-cert/readonly/user="me"}INBOX.GNU 
[empty] => INBOX.GNU
- Copying 
{imap.big.edu:143/imap/tls/novalidate-cert/user="me"}INBOX.GNU.Emacs  => 
/g/mail/INBOX/GNU/Emacs
- [Reusing connection to imap.big.edu/user="me"]
- [[UNSEEN 1]  ]
- Can't create mailbox node /g/mail/INBOX/: File exists
- alternative name: INBOX.GNU.Emacs
- 
{imap.big.edu:143/imap/tls/novalidate-cert/readonly/user="me"}INBOX.GNU.Emacs 
[5 message(s)] => INBOX.GNU.Emacs
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- Permission denied
- [Ok 5 messages(s)]

But this

0 is scarily manual. Thanks to the

> Listing mailboxes...
> 00000005 LIST INBOX *

  earlier in the session, I know that I have 84 subfolders. That's a
  lotta cut'n'paste :-( Furthermore, it ...

1 creates a flat bunch of mailbox nodes in ~. If there is no way
  around problem 0, how might one recreate (e.g. via some script) from
  this bunch of nodes something a mail client would recognize as
  having the original folder structure?

I hope I'm not being too much of a PITA. OTOH I suspect a user-
friendly solution to this problem would be highly reusable (I Am Not A
Mail Admin (you guessed ?-) but I suspect I'm not the first person to
find themselves in this situation), and probably toolworthy (i.e. for
mail-client writers).

Your assistance is appreciated! Tom Roche <[email-masked]>

---

From: Mark Crispin <[email-masked]>
Date: Tue, 16 Sep 2003 14:33:25 -0700 (Pacific Daylight Time)

You are apparently attempting to copy a hierarchy from a mail store which
has so-called "dual use" mailbox names (names that are both a mailbox and
contain other mailboxes) to one which does not.

mailutil can not resolve the resulting conflict on its own.  You have to
give it some alternative name.

In general, you can't create a subdirectory to INBOX, and you certainly
can not create a subdirectory to a mail-file type mailbox.  You have to
use some other name.

-- Mark --

---

From: David B Funk <[email-masked]>
Date: Wed, 17 Sep 2003 02:17:05 -0500 (CDT)

We've got a "horde/IMP" webmail server based upon
apache+php4+c-client (from the imap-2002d kit).

Occasionally I see the following error message show up in the
server syslog: "IMAP toolkit crash: Lock when already locked"

Any suggestions as to how to debug this?

Note that this server is just using c-client as a client
library, the actual imap server is running on a different machine
(also built from the imap-2002d kit).

Thanks,
Dave

-- 
Dave Funk                                  University of Iowa
<dbfunk (at) engineering.uiowa.edu>        College of Engineering
319/335-5751   FAX: 319/384-0549           1256 Seamans Center
Sys_admin/Postmaster/cell_admin            Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{

---

From: "Chr. von Stuckrad" <[email-masked]>
Date: Wed, 17 Sep 2003 16:49:05 +0200

> server syslog: "IMAP toolkit crash: Lock when already locked"

Just by chance I saw the same around last Saturday comig up
here. (same setup, UW-Imap on Mailhost, horde/imp webmailer)

I always assumed this to happen, if somebody runs more than one
IMPA-Connection on the same UNIX-Format-Mailbox?

E.g. runnign netscape from HOME (and leaving it online)
then going some place and using the webmailer from there
to access the same INBOX, which is a dumb default
UNIX-Spoolfile.

If I understood the doku right, only '#mh'-Boxes can be
opened by more than one connection in parallel, so one
of the connections (the older?) will abort.

Do I seee this correctly?

Stucki   (postmaster at mi.fu-berlin.de)

---

From: Mark Crispin <[email-masked]>
Date: Wed, 17 Sep 2003 08:21:03 -0700 (PDT)

> server syslog: "IMAP toolkit crash: Lock when already locked"

This is a bug in horde/IMP.

The error message "Lock when already locked" means that the application is
calling c-client recursively from a callback routine.  For example, if
the application's mm_exists() routine calls a mail_fetch_???() routine.

Recursive calls are not permitted.  It may sometimes work if the data is
in the cache, but not if c-client has to go out to the IMAP server.

I recommend that you examine all mm_???() routines in horde/IMP and look
for calls to mail_???() routines.  Any such calls are bugs.

-- Mark --

---

From: Mark Crispin <[email-masked]>
Date: Wed, 17 Sep 2003 08:23:03 -0700 (PDT)

> IMPA-Connection on the same UNIX-Format-Mailbox?

No.  "Lock when already locked" is a bug in the application; it is making
an illegal call to c-client.

If you make more than one simultaneous open to a UNIX format mailbox, the
older open will be closed.  It will not cause a "Lock when already locked"
crash.

> of the connections (the older?) will abort.

No.  The mbx, tenex, mtx, and mx formats also support shared open.  It is
only UNIX and MMDF formats which don't.

-- Mark --