Subject: Cygwin patches for 2002e

From: "Abe Backus" <[email-masked]>
Date: Tue, 21 Oct 2003 00:26:52 -0700

Hello,

I'd like to submit a couple of patches for uw-imap.

1) the flock simulator for cygwin in flockcyg.c has the same limitations as
the flock simulator for NT, so the code should only lock on one byte in the
file.  The person that brought this to my attention would prefer that the
code lock the last possible byte in the file.
2) CRAM-MD5 wasn't working on cygwin

Thanks!
-Abe

---

From: Mark Crispin <[email-masked]>
Date: Tue, 21 Oct 2003 06:31:25 -0700 (PDT)

On Tue, 21 Oct 2003, Abe Backus wrote:
> file.

This is a bug in Cygwin.  UNIX locks (either fcntl() or flock() are not
mandatory).

> 2) CRAM-MD5 wasn't working on cygwin

How can that code work without doing the impersonate?

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Mark Crispin <[email-masked]>
Date: Tue, 21 Oct 2003 16:14:46 -0700 (Pacific Daylight Time)

> code lock the last possible byte in the file.

I have accepted this change, but to lock the first byte of the file.  I do
not believe that it is safe to lock a non-existant byte of a file, since
we have no way of knowing if future changes to Cygwin or Windows would
break that.

Arguably, the real bug is in Cygwin's implementation of fcntl() locks.
Cygwin is evidentally calling LockFileEx() on Windows directly, without
attempting to implement the correct fcntl() semantics.  In one case, this
is a good thing -- it doesn't implement the fcntl() design flaw that
requires all that flocksim master/slave hackery -- but in this case it is
not a good thing.

I'm not convinced that Cygwin should use the UNIX version of c-client
instead of the native Windows version.

> 2) CRAM-MD5 wasn't working on cygwin

I don't see how it can work either.  Those logon/impersonate calls are
necessary to get access to the user files.  Removing those calls in the
CRAM-MD5 case has the effect of leaving the IMAP server running as SYSTEM,
which IMHO is not a good thing.

-- Mark --

---

From: Eduardo Chappa <[email-masked]>
Date: Tue, 21 Oct 2003 21:11:31 -0700 (PDT)

*** Mark Crispin ([email-masked]) wrote today:

:) On Tue, 21 Oct 2003, Abe Backus wrote:
:) > 1) the flock simulator for cygwin in flockcyg.c has the same
:) >    limitations as the flock simulator for NT, so the code should only
:) >    lock on one byte in the file.  The person that brought this to my
:) >    attention would prefer that the code lock the last possible byte
:) >    in the file.
:)
:) I have accepted this change, but to lock the first byte of the file.
:) I do not believe that it is safe to lock a non-existant byte of a file,
:) since we have no way of knowing if future changes to Cygwin or Windows
:) would break that.
:)
:) Arguably, the real bug is in Cygwin's implementation of fcntl() locks.
:) Cygwin is evidentally calling LockFileEx() on Windows directly, without
:) attempting to implement the correct fcntl() semantics.  In one case,
:) this is a good thing -- it doesn't implement the fcntl() design flaw
:) that requires all that flocksim master/slave hackery -- but in this
:) case it is not a good thing.
:)
:) I'm not convinced that Cygwin should use the UNIX version of c-client
:) instead of the native Windows version.

Hi Mark,

  One question, though. If your hunch is correct, I see that LockFileEx is
only implemented on Windows NT, 2000 and XP, so how does this patch work
in Windows 98?

  I have no way of testing this now, but if someone has tested/can test
this change in Windows 9X, I would be grateful for a report. I am
particularly interested in seeing how this code works when one attempts to
save a message to the same folder that it belongs, and how it works with
mbx style folders.

  It is my own personal belief that Mark is right, and Cygwin c-client
should use the windows code directly, instead of the unix code. Maybe we
could try to do this instead? What do you think Abraham?

Eduardo
http://www.math.washington.edu/~chappa/pine/

---

From: Mark Crispin <[email-masked]>
Date: Tue, 21 Oct 2003 22:02:11 -0700 (PDT)

> only implemented on Windows NT, 2000 and XP, so how does this patch work

There is no such thing as meaningful file locking in Windows 98.  Win98 is
not a real operating system.

But it is also very doubtful that one is going to be running IMAP servers
on Windows 98; or multiple instances of Pine on the same mailbox.

-- Mark --

---

From: Eduardo Chappa <[email-masked]>
Date: Wed, 22 Oct 2003 08:18:00 -0700 (PDT)

*** Mark Crispin ([email-masked]) wrote on Oct 21, 2003:

:) On Tue, 21 Oct 2003, Eduardo Chappa wrote:
:) >   One question, though. If your hunch is correct, I see that
:) > LockFileEx is only implemented on Windows NT, 2000 and XP, so how
:) > does this patch work in Windows 98?
:)
:) There is no such thing as meaningful file locking in Windows 98.
:) Win98 is not a real operating system.
:)
:) But it is also very doubtful that one is going to be running IMAP
:) servers on Windows 98; or multiple instances of Pine on the same
:) mailbox.

Hi Mark,

  I agree with your first statement, but not the second one. There is no
way I can predict what a user will do when using Pine, and you must know
very well that ignoring a possible case is one of the main reasons why
users report bugs, so I would like to offer a full featured (or as much
complete as possible)  software for people in Cygwin. This means that if
they want simultaneous access to a mailbox, they'll need to get it if the
format of the mailbox allows it. I see no reason why PC-Pine and Cygwin
Pine could not be exchanged one for the other.

  I believe it's a priority to offer access to mbx folders to any user of
Pine/imapd/popXd, for any version of Windows, and that's the trick that
must be done.

  So my question still stands, does this patch work for Windowx 9X?, can
anyone confirm or deny this?

  Thank you,

Eduardo

---

From: Mark Crispin <[email-masked]>
Date: Wed, 22 Oct 2003 09:59:26 -0700 (PDT)

On Wed, 22 Oct 2003, Eduardo Chappa wrote:
> There is no
> format of the mailbox allows it.

Once again...

Simultaneous access in the mbx format requires meaningful file locking.

There is no such thing as meaningful file locking in Windows 98.  Windows
98 is not a real operating system.  Nor are Windows 95, Windows Me,
Windows 3.1, MS-DOS, Mac OS 9 (and earlier), etc.

If you wish to have real operating system services, then you must run a
real operating system.  Windows NT, 2000, and XP are real operating
systems.  UNIX and its variants are real operating systems.  Mac OS X is a
real operating system.

Cygwin is not a real operating system either.  It is one of long series of
similar products (e.g. Yale Tools, Mint, MachTen, MKS Toolkit) which
provide UNIX-like library calls and compiler semantics on a different
operating system (in this case, Windows).  However, like all these other
products, it is (and always will be) an incomplete compromise.  In some
cases, it will provide the service by translating the call to a similar
call on the host operating system.  In other cases, it must implement that
service as best it can within its own environment.  And in still other
cases, it has to punt and provide methods into the host environment (such
as the interface into the Windows concepts of logon and impersonate).

Cygwin apparently implements fcntl() locking via the Windows LockFileEx()
call.  This implementation is *NOT* correct for UNIX, but it is close.
One of these implementation bugs is a good thing, the other is not; hence
the special handling required for Cygwin.

LockFileEx() does not exist on the MS-DOS based versions of Windows (3.1,
95, 98, Me).  If you use one of those systems, you have to live with the
fact that you don't have meaningful locking.  Life is tough.

> I see no reason why PC-Pine and Cygwin

Any favorable interaction between PC Pine and a UNIX Pine built under
Cygwin is completely coincidental and is not (nor ever will be)
guaranteed.

This is the price that you pay for using a kludge that hacks one operating
system's semantics on top of another.  They may be extremely useful
kludges, but they are (and always will be) limited.  Most people who use
such things understand these limitations.

On top of that, you are expecting services out of the MS-DOS versions of
Windows that do not exist.

> Pine/imapd/popXd, for any version of Windows, and that's the trick that

It may be a priority for you.  It is not for me; and quite frankly it is a
pipe-dream.  The MS-DOS versions of Windows do not have the capabilities
necessary to performs mbx-style shared access.  The MS-DOS versions of
Windows are dead products, and never will have those capabilities.

Nor do I want to waste much more of my time having to explain this to you.

> anyone confirm or deny this?

The question is meaningless.  There is no "works for Windows 9x" in the
sense of "operates with the full functionality of modern systems."

-- Mark --

---

From: Eduardo Chappa <[email-masked]>
Date: Wed, 22 Oct 2003 10:52:22 -0700 (PDT)

*** Mark Crispin ([email-masked]) wrote today:

:) On Wed, 22 Oct 2003, Eduardo Chappa wrote:
:) > There is no way I can predict what a user will do when using Pine,
:) > and you must know very well that ignoring a possible case is one of
:) > the main reasons why users report bugs, so I would like to offer a
:) > full featured (or as much complete as possible)  software for people
:) > in Cygwin. This means that if they want simultaneous access to a
:) > mailbox, they'll need to get it if the format of the mailbox allows
:) > it.
:)
:) Once again...
:)
:) Simultaneous access in the mbx format requires meaningful file locking.
:)
:) There is no such thing as meaningful file locking in Windows 98.
:) Windows 98 is not a real operating system.  Nor are Windows 95, Windows
:) Me, Windows 3.1, MS-DOS, Mac OS 9 (and earlier), etc.

Mark,

  Does this mean that the changes that you are accepting into C-client
will make Pine not work in Windows 9X when using mbx style folders?. I
believe you are trying to say that, but it is not completely clear. Your
answer is about locking, not about mbx style folders. I just want to be
sure that the answer to my question is yes. Can you confirm or deny this,
please?

Thanks. Have a nice day.

Eduardo

---

From: Brad Arlt <[email-masked]>
Date: Wed, 22 Oct 2003 12:37:50 -0600

> sure that the answer to my question is yes. Can you confirm or deny this,

I'll take a stab at this...

mbx needs file locking if it has any hope of mailbox integrity
if there is concurrent access - like email delivery or the mailbox
being open in two processes.

If you don't have real file locking, you can't use mbx.

There is a nice long rant/documentation on locking in the UW-imap
distro (so I presume it is in the Pine distro as well) in
docs/locking.txt

So to spell it out for you:

Windows Me/98/95/3.11/3.1/3.0, MacOS 9, and MS-DOS cannot use mbx
format without great risk of data corruption.  It may be you *can*
use mbx, but you shouldn't.

I have a feeling that the Unix mailbox format will work better for you
under these OSes.

Using Windows 2000/XP/2003 will allow you to use both Windows *and*
mbx in without these problems.
-----------------------------------------------------------------------
   __o		Bradley Arlt			Security Team Lead
 _ \<_		[email-masked]		University Of Calgary
(_)/(_) 	Joyously Canadian	 	Computer Science