Subject: Re: Extended ISO-8859-1 and Pine

From: John Berthels <[email-masked]>
Date: Fri, 3 Jul 1998 16:49:57 +0100 (BST)

Just to beat this to death, attached is a filter re-written in C which
does a similar job to the previously posted perl version but: 

a) doesn't require perl start-up time per bodypart display
b) recognises many more code page 850 codes
c) requires compiling (duh): try "gcc 8859filt.c -o 8859filt"

For those who aren't supremely disinterested in this, the problem this
tries to solve is the presence of character codes in the 0x80 -> 0x9f
range in bodyparts labelled as iso-8859-1.

Such bodyparts seem to be generated by mailers including Netscape 4, which
incorrectly insert these codes to represent characters from code page 850. 

This filter replaces some of these codes with 'lookalike' sequences from
the legal iso-8859-1 range.

The comment at the beginning of the program gives terse help on installing
in PINE.

[I guess the other workaround to this might be to use a font in codepage
850 (presumably one of the ones netscape uses?) and unset the PINE
variable: send-control-characters-as-is]

Have fun,

jb

-- 
John Berthels
Email: [email-masked]
X.400: /G=john/S=berthels/O=nexor/P=nexor/A=cwmail/C=gb/
#include <stdio.h>

/*
 * Pine filter to map control character sequences and
 * characters corresponding to code page 850 to iso-8859-1
 *
 * Useful since some mailers (Netscape 4? IE4?) produce code
 * page 850 but label it as iso-8859-1
 *
 * thanks to Arthur Snoke <[email-masked]> for
 * ideas + determination
 *
 * HOW TO USE:
 * In your pine setup, add the line:
 * _CHARSET(ISO-8859-1)_ /path/to/your/compiled/8859filt
 * to your 'display-filters' setting
 *
 */

/*
 * Table: position [0] maps to position 0x80 in code page 850
 * rest of table runs continuously to 0x9f
 */
static	char	*map_table[]	=	{
	"",		/* Missing */
	"",		/* Missing */
	",",		/* Looks like a comma */
	"f",		/* Italic f */
	"\"",		/* 'low' quotes */
	"...",		/* ellipsis */
	"+",		/* No good choice (footnote marker) */
	"|",		/* Ditto */
	"^",		/* Looks like a caret */
	"%o",		/* Looks like % and a small 'o' (a 'permille') */
	"S",		/* Capital S with an accent */
	"<",		/* Open angle bracket */
	"OE",		/* Capital OE dipthong */
	"",		/* Missing */
	"",		/* Missing */
	"",		/* Missing */
	"",		/* Missing */
	"'",		/* Open single quote */
	"'",		/* Close single quote */
	"\"",		/* Open double quote */
	"\"",		/* Close double quote */
	"*",		/* Bullet */
	"-",		/* Dash */
	"-",		/* Long dash */
	"~",		/* Tidle/accent character */
	"(tm)",		/* Trade mark symbol */
	"s",		/* s with an accent */
	">",		/* Close angle bracket */
	"oe",		/* oe dipthong */
	"Y",		/* Capital Y with umlaut */
};

int main( int argc, char **argv )
{
	int	c;
	char	*str;

	while( (c = getchar()) != EOF )	{
		if( c == '^' )	{
			c = getchar();
			if( c != EOF && c >= 0xc0 && c <= 0xff )	{
				/*
				 * We have a ctrl char
				 */
				str = map_table[c - 0xc0];
				printf( "%s", str );
			} else	{
				putchar( '^' );
				putchar( c );
			}
		} else if( c >= 0x80 && c <= 0x9f )	{
			/*
			 * Non-printable iso-8859-1, try cp850
			 */
			str = map_table[c - 0x80];
			printf( "%s", str );
		} else	{
			putchar( c );
		}
	}

	return 0;
}