Subject: [Imap-uw] build issues

From: BuildSmart <[email-masked]>
Date: Sun, 5 Mar 2006 22:40:25 -0500

Im in the process of creating a new mail server, I've been trying to  
build uw-imap and while I can build successfully, I'm not getting the  
expected results.

an older working install on
* OK [CAPABILITY IMAP4REV1 LITERAL+ SASL-IR LOGIN-REFERRALS  
AUTH=LOGIN] macftphttp.serverbox.org IMAP4rev1 2003.347 at Sun, 5 Mar  
2006 22:14:17 -0500 (EST)

current build gives:
* OK [CAPABILITY IMAP4REV1 LITERAL+ SASL-IR LOGIN-REFERRALS STARTTLS  
LOGINDISABLED] mustangrestomods.com IMAP4rev1 2004.357 at Sun, 5 Mar  
2006 22:35:47 -0500 (EST)

Because I'm using a newer OS version the older binaries don't work so  
building is a must.

I am able to build the binaries in newer the newer OS to be OS  
backward compatible so I am able to test in different versions of the  
OS.

In case you're wondering, the new OS is Mac OSX 10.4.5 and the old OS  
is 10.2.4

-- Dale
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: BuildSmart <[email-masked]>
Date: Sun, 5 Mar 2006 23:24:45 -0500

Sorry, I should have looked a little closer at the Makefile and read  
the changes to see thet the defualt was changesd to 'SSLTYPE=nopwd'

Using 'SSLTYPE=none' gave me my working build but I was wondering if  
SSL/TLS support is still available with this build method???

Is it possible to have both methods available in a single binary???

On Mar 5, 2006, at 22:40 , BuildSmart wrote:

> Im in the process of creating a new mail server, I've been trying  
> to build uw-imap and while I can build successfully, I'm not  
> getting the expected results.
>
> AUTH=LOGIN] macftphttp.serverbox.org IMAP4rev1 2003.347 at Sun, 5  
> Mar 2006 22:14:17 -0500 (EST)
>
> current build gives:
> STARTTLS LOGINDISABLED] mustangrestomods.com IMAP4rev1 2004.357 at  
> Sun, 5 Mar 2006 22:35:47 -0500 (EST)
>
> Because I'm using a newer OS version the older binaries don't work  
> so building is a must.
>
> backward compatible so I am able to test in different versions of  
> the OS.
>
> In case you're wondering, the new OS is Mac OSX 10.4.5 and the old  
> OS is 10.2.4
>
>
> -- Dale

-- Dale

---

From: Mark Crispin <[email-masked]>
Date: Sun, 5 Mar 2006 20:48:31 -0800 (PST)

On Sun, 5 Mar 2006, BuildSmart wrote:
> Sorry, I should have looked a little closer at the Makefile and read the 
> changes to see thet the defualt was changesd to 'SSLTYPE=nopwd'

That is the standard build, as required by RFC 3501.

> Using 'SSLTYPE=none' gave me my working build but I was wondering if SSL/TLS 
> support is still available with this build method???

That builds without SSL entirely; which is non-complaint with RFC 3501 and 
VERY insecure.

> Is it possible to have both methods available in a single binary???

If, by this, you mean building with SSL support, but allowing insecure 
logins without the use of SSL, that is done by SSLTYPE=unix.

However, I recommend that you use the standard SSLTYPE=nopwd.  Every 
client in the past several years has supported SSL and/or TLS.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: BuildSmart <[email-masked]>
Date: Mon, 6 Mar 2006 01:13:41 -0500

On Mar 5, 2006, at 23:48 , Mark Crispin wrote:

>> Sorry, I should have looked a little closer at the Makefile and  
>> read the changes to see thet the defualt was changesd to  
>> 'SSLTYPE=nopwd'
>
>> Using 'SSLTYPE=none' gave me my working build but I was wondering  
>> if SSL/TLS support is still available with this build method???
>
> That builds without SSL entirely; which is non-complaint with RFC  
> 3501 and VERY insecure.
>
> If, by this, you mean building with SSL support, but allowing  
> insecure logins without the use of SSL, that is done by SSLTYPE=unix.
>
> However, I recommend that you use the standard SSLTYPE=nopwd.   
> Every client in the past several years has supported SSL and/or TLS.

I'm sorry if I'm starting to sound like an idiot but I am having a  
hard time with understanding the documentation.

I can agree that most mail apps I've looked at have some checkbox for  
SSL encryption which also changes the port to communicate on.

I have an app set up that sends the password to retrieve mail to be  
read (no clue what format, only that it works), when I build with the  
nopwd option I can no longer retrieve e-mail.

When I try "telnet loalhost 993" it refuses connection (obviously not  
set to watch this port).

I have no issues with using SSL options to hide plain-text passowrds  
but pam_mysql expects the password to be plain-text when it processes  
it.

Forcing only SSL connections for mail is probably a good idea however  
the instructions for making it work are unclear to me, I understand  
drag and drop and it works but if I build with SSLTYPE=unix it's no  
longer drag and drop and I couldn't find simple explanations in the  
documentation to make it work.

My issue is probably more of configuration than anything else and  
what I do know is I have a /etc/xinet.d/imapd file with the following  
content:

service imapd
{
disable = no
type = unlisted
socket_type = stream
protocol = tcp
wait = no
user = root
groups = yes
server = /usr/libexec/imapd
port = 143
}

Aside from the port change, what else do I need to change to get mail  
working again?

What is required to have both SSL and non-SSL capabilities?

Will I need 2 files or can it be done in the existing file?

-- Dale

---

From: Mark Crispin <[email-masked]>
Date: Sun, 5 Mar 2006 22:40:02 -0800 (PST)

On Mon, 6 Mar 2006, BuildSmart wrote:
> I have an app set up that sends the password to retrieve mail to be read (no 
> clue what format, only that it works), when I build with the nopwd option I 
> can no longer retrieve e-mail.

Do you know if that application uses SSL?  What is the application?

> When I try "telnet loalhost 993" it refuses connection (obviously not set to 
> watch this port).

You can't telnet to an SSL port.  You have to use an SSL-enabled program 
such as the s_client in OpenSSL.

> I have no issues with using SSL options to hide plain-text passowrds but 
> pam_mysql expects the password to be plain-text when it processes it.

SSL has nothing to do with your password validation.  SSL encrypts the 
entire session, not just SSL.

> Forcing only SSL connections for mail is probably a good idea however the 
> instructions for making it work are unclear to me, I understand drag and drop 
> and it works but if I build with SSLTYPE=unix it's no longer drag and drop 
> and I couldn't find simple explanations in the documentation to make it work.

Did you set up separate xinetd startup files for both the IMAP port (port 
143) and the SSL IMAP port (port 993)?  You have to do that.

xinetd is a completely different program, which I neither develop nor 
maintain.  What's more, there are different ways that xinetd is set up on 
different systems.  You have to consult the xinetd documentation on your 
system for how this should be done.

> Aside from the port change, what else do I need to change to get mail working 
> again?

All you need to do is to set up the appropriate configuration files *for 
your operating system* that will start imapd on port 143 and 993 and 
ipop3d on port 110 and 995.  That's four separate server listen/startup 
rules that you have to set up.

There are too many variations for me to offer a "one shoe fits all".  The 
documentation tried to do such for inetd, but even inetd has many 
variations.  xinetd also has variations.

What's more, if you use Mac OS X, you may have to use a launchd 
configuration file instead of xinetd.  xinetd doesn't seem to work 
reliably on Mac OS X.  I don't know much about launchd, other than the 
fact that it uses XML (shudder) and follows Apple's typical contempt of 
standards.

Your best bet is to consult your system's documentation and/or the vendor 
of your system; alternatively, ask a expert on your system type.

imapd and ipop3d do no TCP I/O on their own.  All their I/O is via stdio; 
thus they are always invoked by the system's TCP listener (inetd or xinetd 
or launchd).

-- Mark --

http://panda.com/mrc

---

From: Michael Cashwell <[email-masked]>
Date: Mon, 6 Mar 2006 11:39:05 -0500

On Mar 6, 2006, at 1:40 AM, Mark Crispin wrote:

> I don't know much about launchd, other than the fact that it uses  
> XML (shudder) and follows Apple's typical contempt of standards.

Oh please! What standards? If there's a uniform and performant  
solution that addresses system-init, network daemon listers, system-  
and per-user time-scheduled execution and the like, can we have a  
link to it? It was precisely this void that prompted Apple to create  
launchd.

Branding Apple as contemptuous of standards is rich indeed from the  
author of a distribution that hard-codes configuration settings in  
deeply-buried source files. What happened to the de-facto standard of  
using runtime configuration files like dhcpd, BIND, postfix, and  
Apache do? Never have I needed to repeatedly put configuration items  
in their source files (nor recompile to change them!) and their  
settings persist with almost no effort on my part from one release to  
the next.

But more broadly, when I "don't know much about" something I make an  
effort to learn about it before criticizing. To that end, there is  
readily-available information on launchd <http://developer.apple.com/ 
macosx/launchd.html>.

> imapd and ipop3d do no TCP I/O on their own.  All their I/O is via  
> stdio; thus they are always invoked by the system's TCP listener  
> (inetd or xinetd or launchd).

I build it using the OS X (PAM) target:
	make oxp

and copy the 2 executables:
	sudo cp ./imapd/imapd  /usr/local/sbin/ ; sudo chown root:wheel /usr/ 
local/sbin/imapd
	sudo cp ./ipopd/ipop3d /usr/local/sbin/ ; sudo chown root:wheel /usr/ 
local/sbin/ipop3d

(BTW, to make SSL mean much, one should procure a server certificate.  
With just a few clients even a self-signed one is manageable.)

I use two launchd .plist files: /Library/LaunchDaemons/ 
edu.washington.<imapd | pop3d>.plist. Each handles the plain and ssl  
form of the service. Here's my imap one:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http:// 
www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>Disabled</key>
     <false/>
   <key>Label</key>
     <string>edu.washington.imapd</string>
   <key>ServiceDescription</key>
     <string>IMAP Server</string>
   <key>ProgramArguments</key>
     <array>
       <string>/usr/local/sbin/imapd</string>
     </array>
   <key>inetdCompatibility</key>
     <dict>
       <key>Wait</key>
         <false/>
     </dict>
   <key>Sockets</key>
     <dict>
       <key>plain</key>
         <dict>
           <key>SockServiceName</key>
             <string>imap</string>
           <key>SockType</key>
             <string>stream</string>
         </dict>
       <key>ssl</key>
         <dict>
           <key>SockServiceName</key>
             <string>imaps</string>
           <key>SockType</key>
             <string>stream</string>
         </dict>
   </dict>
</dict>
</plist>

Best regards,
Mike Cashwell

---

From: Mark Crispin <[email-masked]>
Date: Mon, 6 Mar 2006 11:00:11 -0800 (PST)

On Mon, 6 Mar 2006, Michael Cashwell wrote:
> Oh please! What standards? If there's a uniform and performant solution that 
> addresses system-init, network daemon listers, system- and per-user 
> time-scheduled execution and the like, can we have a link to it? It was 
> precisely this void that prompted Apple to create launchd.

Other UNIX systems have variants, each being their respective vendors' own 
idea of "improved", but along a common theme that respects the 30+ year 
UNIX tradition.  It is possible to transfer knowledge of one system to 
another; and where the changes are major, it is possible to use man to 
uncover the answer.

Apple, however, went completely off on their own, just as NeXT did with 
NetInfo (creating the system where you couldn't do an ls or much else 
because a DNS query didn't respond).

Launchd is unlike anything else.  You have to be of the XML religion to 
like it.  I've noticed that neither Linux, nor BSD, nor System V, seems to 
be making any move to adopt it.

Much is missing.  For example, nowhere in any of the Apple configuration 
tools is there any operation to get the modem port to function as an 
incoming dialup.  "man init", "man getty" and "man ttys" all tell you 
things that you already knew from UNIX, but are totally false on Mac OS X.

Bits and pieces of the traditional programs, configuration files, and man 
pages remain.  Some do nothing at all (e.g., ttys, fstab).  Others work, 
but unpredictably (whether your xinetd configuration will start properly 
at system boot time depends completely upon timing!).  Still others 
actually seem to be current (e.g., /etc/services).

A web page has a sample launchd XML equivalent of a sample xinetd file; 
but it doesn't document the set of values nor does it document the 
equivalent for fields that aren't in the sample xinetd file.

I find it ironic that you complain about imapd having a fixed 
configuration.  That's about as "uniform and performant" as you can get.

I couldn't help the problems caused by multiple system configuration 
databases (/etc files, Yellow Pages/NIS, NetInfo), multiple network 
listeners (inetd, xinetd, launchd), multiple security packages (OpenSSL, 
SSPI, PAM, SecureWare, SIA, AFS, Kerberos, POSIX,...), etc.  It wasn't so 
bad when I wrote imapd.

But I could (and did) create a single, sane, default IMAP configuration. 
In my experience, most deviations from that configuration are unnecessary, 
ill-informed, and generally cause more problems than they solve.

Many of these deviations are applied by third parties, who then do not 
disclose what they did, nor the implications of what they did, with their 
customers.  Guess who ends up answering the customers' "why doesn't it 
work the way it's documented" questions.  Most of the time, the solution 
is to replace the hacked version with a completely unmodified one.

-- Mark --

http://panda.com/mrc

---

From: Dan Pritts <[email-masked]>
Date: Mon, 6 Mar 2006 14:51:35 -0500

> be making any move to adopt it.

in many ways, it is like Solaris 10's SMF, but i haven't looked deep
under the covers of either one.  So far, i like SMF.  

danno
--
dan pritts - systems administrator - internet2
734/352-4953 office        734/834-7224 mobile
_______________________________________________
Imap-uw mailing list
[email-masked]
https://mailman1.u.washington.edu/mailman/listinfo/imap-uw

---

From: Michael Cashwell <[email-masked]>
Date: Mon, 6 Mar 2006 16:04:33 -0500

On Mar 6, 2006, at 2:00 PM, Mark Crispin wrote:

>> solution that addresses system-init, network daemon listers,  
>> system- and per-user time-scheduled execution and the like, can we  
>> have a link to it? It was precisely this void that prompted Apple  
>> to create launchd.
>
> Other UNIX systems have variants, each being their respective  
> vendors' own idea of "improved", but along a common theme that  
> respects the 30+ year UNIX tradition.  It is possible to transfer  
> knowledge of one system to another; and where the changes are  
> major, it is possible to use man to uncover the answer.
>
> Apple, however, went completely off on their own, just as NeXT did  
> with NetInfo (creating the system where you couldn't do an ls or  
> much else because a DNS query didn't respond).

I've been irritated too by delays and hangs when services (especially  
DNS) are down or during system bring-up, but I've not see this  
recently. And I've found Red Hat systems whose behavior was FAR worse  
in this regard. But these issues have more to do with poor initial  
configurations than inherent design problems, IMO.

But it's worth noting that Apple doesn't just make these things up  
for the fun of it. As an example, they have deprecated OpenSSL in  
favor of their SecureFramework in order to support new features. (In  
this case the concept of having certificates and keys on smartcards  
where OpenSSL forces certificates to be in files.)

And even in doing that they didn't just roll their own code. The  
framework is CDSA-based (Intel open-source) and they use the muscle  
card code (also open source) for much of the smartcard heavy lifting.

> Launchd is unlike anything else.  You have to be of the XML  
> religion to like it.  I've noticed that neither Linux, nor BSD, nor  
> System V, seems to be making any move to adopt it.

I'd argue that being unlike the existing, fractured solutions is a  
requirement in order to address them. That the other vendors aren't  
adopting it or doing something along the same lines is more of an  
indictment against them. Inertia is a powerful thing but that doesn't  
necessarily make it good.

But that aside, I really don't understand your aversion to XML  
plists. It's a standard way to represent key/value data and is far  
more elegant than some of the flat file formats where the ideas of  
container data structures or hierarchy were grafted on as an  
afterthought. I agree that XML is ugly to edit in raw form but it was  
not intended to be handled that way. (No one faults JPEGs for being  
hard to manipulate in a text editor!) There are decent tools for  
working with plist XML.

I certainly would prefer my daemons to use XML with their own keys  
than each rolling their own format from scratch. Regrettably, the  
latter seems to be much more common. Plists allow me to concentrate  
on the key/values I want to set and not spend time worrying about  
which daemon wants commas and which want single quotes, etc.

> Much is missing.  For example, nowhere in any of the Apple  
> configuration tools is there any operation to get the modem port to  
> function as an incoming dialup.  "man init", "man getty" and "man  
> ttys" all tell you things that you already knew from UNIX, but are  
> totally false on Mac OS X.

I freely admit that I have not done this and that some man pages are  
utterly wrong. The latter has improved considerably in the last 2  
years, however.

> Bits and pieces of the traditional programs, configuration files,  
> and man pages remain.  Some do nothing at all (e.g., ttys, fstab).   
> Others work, but unpredictably (whether your xinetd configuration  
> will start properly at system boot time depends completely upon  
> timing!).  Still others actually seem to be current (e.g., /etc/ 
> services).
>
> A web page has a sample launchd XML equivalent of a sample xinetd  
> file; but it doesn't document the set of values nor does it  
> document the equivalent for fields that aren't in the sample xinetd  
> file.

I'm not sure which web page you mean here, but man launchd has "See  
Also launchd.plist". That man page seems to document all the various  
keys, values and behaviors pretty thoroughly.

> configuration.  That's about as "uniform and performant" as you can  

I was talking about a uniform and performant daemon launcher /  
listener. Something the loose collection of init, /etc/rc, inetd,  
xinetd, and cron can hardly claim to be.

As the launchd docs note, "... processes which are not directly  
initiated by users might be launched from at least three different  
places, and configured in at least three different ways." And there  
are other side effects like having to kill processes or reboot for  
changes to take effect. launchd is intended to try to bring some  
uniformity to tasks that are largely similar.

My complaint with imapd wasn't that the configuration is fixed but  
that it's done in the source and compiled in. Using a config file  
read at startup strikes me as a widely used standard practice that  
imapd doesn't follow and maintaining my configuration across releases  
is made harder for it.

> I couldn't help the problems caused by multiple system  
> configuration databases (/etc files, Yellow Pages/NIS, NetInfo),  
> multiple network listeners (inetd, xinetd, launchd), multiple  
> security packages (OpenSSL, SSPI, PAM, SecureWare, SIA, AFS,  
> Kerberos, POSIX,...), etc.  It wasn't so bad when I wrote imapd.

Quite true, I completely agree.

> But I could (and did) create a single, sane, default IMAP  
> configuration. In my experience, most deviations from that  
> configuration are unnecessary, ill-informed, and generally cause  
> more problems than they solve.

Indeed this is largely true. I see the frequent posts where people  
complain that they cannot log into imapd using a plain-text password  
or they want SSL but don't want to mess with certificates. Often they  
really don't want to do what they think they want to do and your  
defaults are superior.

But there are many settings that are little more than personal  
preference. The two I constantly have to set are mailsubdir and  
hideDotFiles. This is to counteract the negative effects of the  
default settings in situations where users do a lot more in their  
home directories than just email. They don't like seeing their  
documents cross-pollenating with their email.

Having a sane default configuration is great but if it's implemented  
in a manner that makes perfectly legitimate settings hard(er) to  
maintain then it does come at some cost.

> Many of these deviations are applied by third parties, who then do  
> not disclose what they did, nor the implications of what they did,  
> with their customers.  Guess who ends up answering the customers'  
> "why doesn't it work the way it's documented" questions.  Most of  
> the time, the solution is to replace the hacked version with a  
> completely unmodified one.

Yes, I've certainly seen this too and seen the problems vanish when  
they just use the real distribution. Please understand, I very much  
appreciate your hard work both on the toolkit and on trying to  
educate people that standards are important!

And I'm certainly not in the camp that thinks Apple can do no wrong.  
(Don't get me started about their private, special cases relating to  
their 802.11 software!) But in my experience, they generally honor  
documented standards for data structures and protocols (unlike some  
vendors... ahem) but will branch out on their own in implementations  
of them when there's good reason.

I do see value in launchd over the discordant mess it seeks to replace.

Cheers!
-Mike Cashwell