	Lynx SSL support for certificates - README.sslcerts file

BACKGROUND:

The original README.ssl document for lynx stated:

	Note that the server... may not have a valid certificate. Lynx will not 
	complain, as it does not yet support certificates...

Such lack of support is no longer the case.  Lynx now features excellent 
certificate management through the openssl project.  There is almost no 
online documentation available regarding how to use openssl's certificate 
management with other programs, so this will accompany lynx and hopefully 
encourage good practical security for unix clients. 

Lynx relies on openssl to not only encrypt connections over https, but also to
determine whether it should even accept a certificate and establish a secure 
connection with a remote host. Because of this reliance upon openssl by lynx, 
most of this tutorial deals with how to use openssl to "install" both
commercial CA cert bundles as well as self-signed certs from trusted sources 
and, most importantly, how to get them recognized by lynx. 

While lynx on many systems will transparently accept valid certificates, not 
all systems enjoy such functionality. Further, as noted above, older versions 
of lynx do not perform any validity checks on a certificate. 

There is also the common case of wanting to trust, use and install a 
self-signed certificate from a known server source and have it be trusted by 
client programs.

Briefly, the procedure will involve confirming the default system location for
certificates, possibly setting values for SSL_CERT_DIR and SSL_CERT_FILE in 
the environment, and converting and hashing the certificates using openssl 
utilities to enable recognition.

THE CURRENT SITUATION:

Prior to lynx2.8.5dev9, lynx did not check at all for certificate validity.

Since lynx2.8.5.dev9, lynx has reported this openssl error:

SSL error:unable to get local issuer certificate-Continue? (y)

whenever an https connection was initiated and the certificate could not be
found,  for whatever reason, by openssl, and therefore lynx. 

This checking for a certificate is an enhancement to security, but rather 
tediously generates errors at each https browser request. 

The ability to turn off reporting of this error to the user was added to 
lynx2.8.5dev16 as the FORCE_SSL_PROMPT setting in lynx.cfg as noted in the
CHANGELOG:

	This lets the user decide whether to ignore prompting for questionable 
	aspects of an SSL connection.  

While this is a convenient setting to employ when using lynx to script 
https -dumps, it by definition ignores the issue of certificate validity 
altogether. Those concerned with proper certificate management and 
the maintenance of a store of updated CA certificates will be uncomfortable 
with this relaxed security setting.

PRELIMINARY PROCEDURES:

It is assumed that openssl has been installed correctly, that the default 
cert directory is /usr/local/ssl/certs, (it's often /etc/ssl/certs, but we 
need a point of departure for the discussion) and that lynx has been compiled 
--with-ssl. 

The default location for certs on your system may be different, or there may not
be one. You will have to substitute that location for /usr/local/ssl/certs in 
the following instructions, and/or set environment variables.

To determine the default location for certs on your system you may run the 
following command: 

strings libcrypto.a  | grep -in cert | less

Look in this output for SSL_CERT_DIR and SSL_CERT_FILE, and the lines just
above them. This is your default location, respectively, for certificates, 
and the CA cert bundle, cert.pem. You will need to know where libcrypto.a is 
found of course.

Example output:

<snip>
7490:/etc/ssl/certs
7491:/etc/ssl/cert.pem
7492:SSL_CERT_DIR
7493:SSL_CERT_FILE
<snip>

Other possible example output:

<snip>
31555:/usr/local/ssl/certs
31556:/usr/local/ssl/cert.pem
31557:SSL_CERT_DIR
31558:SSL_CERT_FILE
<snip>

Note that when OpenSSL is installed, the c_rehash utility is installed in a 
binary directory (default /usr/local/ssl/bin).  You will need to know where it 
is on your system. The command:

whereis c_rehash 

will probably give useful results.

Note also that there is no CA cert bundle distributed with OpenSSL. The 
OpenSSL team specifically decided NOT to do that. Getting a set of trusted 
certificates is left up to the installer. 

It is a fairly trivial procedure to pull the bundle of trusted root certs out 
of a recent version of Internet Explorer. The mod_ssl project also provides 
them.  The procedure to convert and install them is detailed later in this 
document, and if you simply need to have commercially provided certificates 
trusted by lynx, you can skip down a few lines to the INSTALLING OR UPDATING 
THE CA BUNDLE section.

INSTALLING A SELF-SIGNED CERTIFICATE:

When you would like to trust a self-signed (non-commercial) certificate you will
need to get hold of the actual file. If it's a cert local to your network you
can ask the sysadmin to make it available for download as a link on a webpage.

If such file is not human-readable it's probably DER formatted and will need to 
be converted to PEM format to allow openssl to use it.

To convert DER formatted certificates into something openssl can deal with:

Save the cert as site_name.crt in a directory. In that directory, type:

openssl x509 -inform DER -in site_name.crt -outform PEM -out site_name.pem

You can now copy this individual cert into the directory for that, usually
/usr/local/ssl/certs.  The alternative is to concatenate the individual certs 
to the cert.pem bundle in /usr/local/ssl. (Please see INSTALLING OR UPDATING 
THE CA BUNDLE below). 

The cert file will now be in an acceptable format to openssl, PEM encoded. 
However, openssl, and by extension lynx, will not know about it until that 
cert is present in a file named after the hash value of that cert, in the 
default directory /usr/local/ssl/certs.

So the next thing to do is to hash the cert using c_rehash.

INSTALLING OR UPDATING THE CA BUNDLE:

Now would be a good time to check to see if you have the bundle of CA certs
/usr/local/ssl/cert.pem, or to update them.

CA bundles are available in various places, such as the modssl distribution, 
for those who want to take that route, or you can extract the current bundle 
from a current version of Netscape or Internet Explorer (export them all from 
IE and transfer it onto your system). 

From IE it extracts as a PKCS7 file and needs to be converted with something 
like:

openssl pkcs7 -inform DER -in bundle.crt -outform PEM -out cert.pem \ 
-print_certs -text

The resulting cert.pem file should be copied to the default directory for 
bundles (usually /usr/local/ssl) and renamed to "cert.pem". It will also 
process just fine if it is present and hashed in /usr/local/ssl/certs.

We now have all of the individual certs we wish to trust in our certs 
directory, and the most recent bundle of CA certs as well. 

Confirm that you have the script c_rehash (See PRELIMINARY PROCEDURES; if it is 
not found, a copy is usually located in the tools directory of the openssl 
source tree. If you use this copy, it needs the execute bit set or it will not
run).

Run:

./c_rehash

The c_rehash utility is a perl script that runs openssl commands which creates 
the files named after the hash values of the certs in the default directory 
for certs. 

Its output looks like this:

Doing /usr/local/ssl/certs
vsignss.pem => f73e89fd.0
vsign3.pem => 7651b327.0
...more output
<snip>

All pem encoded certs in /usr/local/ssl/certs will now be recognized.

SETTING AND EXPORTING ENVIRONMENT VARIABLES:

If lynx is still not recognizing certs, environment variables may need
to be set; if on a sh type shell, the variables also need to be exported.

The environment variables SSL_CERT_DIR and SSL_CERT_FILE only need to be set 
if a non-default location is used for certificates, or if certs just can't be
found by lynx. They may be set as follows in /etc/profile, or a shell 
initialization .profile or .*shrc, if we run a non csh type shell, according 
to the results of the search for the default location for certs procedure 
(See PRELIMINARY PROCEDURES):

SSL_CERT_DIR="/usr/local/ssl/certs"
SSL_CERT_FILE="/usr/local/ssl/cert.pem"
export SSL_CERT_DIR SSL_CERT_FILE

On csh type shells, you can use:
setenv SSL_CERT_DIR "/usr/local/ssl/certs"
setenv SSL_CERT_FILE "/usr/local/ssl/cert.pem"

On some systems setting and exporting them makes all the difference. Apparently 
this is often not an issue on other systems, but this might help someone 
somewhere.

Note that the environment variable SSL_CERT_FILE applies to the cert-bundle 
if used outside of the default location (/usr/local/ssl/cert.pem) compiled 
into OpenSSL. There are issues with SSL_CERT_FILE in 0.9.6x versions of openssl.

Make sure you have FORCE_SSL_PROMPT set to PROMPT in lynx.cfg like so:

FORCE_SSL_PROMPT:PROMPT

You will now connect without error to https servers with trusted certs, but
will still get this error for untrusted certs:

SSL error:self signed certificate-Continue? (y)

A quick check confirms that these procedures have the same effect with ssl 
errors in the pine program.

--Stef Caunter <cauntes@mail.mohawkc.on.ca>
Mohawk College Department of Computer Science
Hamilton Ontario Canada
