Subject: International character sets

From: [email-masked] (Olav Georg Karlsen)
Date: 9 Mar 1995 11:43:50 +0100

I am using Pine 3.91 (in Norway)
I would like to use the norwegian characters æøå and ÆØÅ. (They might not appear
right, but should be oslash, aring, etc. (ae, oe, aa).
I have therefore configured Pine to use:

character-set          = iso-8859-1                                             

When i recieve mail from a person with these letters in his/her name, it
appears like this:

From: =?iso-8859-1?Q?Bj=F8rn_Ole_Fredrik_=D8deg=E5rd?=
     <[email-masked]>

When i type these letters in the message body or receive mail containing them, they
do not appear at all on screen, but the  letters ARE there, because the recipient can
read them in mail from me, and when i export my incoming mail, they appear in the
text file.

Any solutions?

Olav Karlsen

---

From: [email-masked] (Barry Bouwsma)
Date: 10 Mar 1995 04:40:21 GMT

>right, but should be oslash, aring, etc. (ae, oe, aa).

    I'm seeing this on a US-ASCII-only terminal, but I think they're right.
They seem like the correct 7-bit equivalents to me for the 8-bit ISO 8859-1
characters, not a regional variant of ISO 646.  (I see f, x, and e.)
I'm also posting this with a newsreader which does not make use of MIME
headers or Q-P encoding, so I know several people will not see the correct
information.  Bear with me in spite of this, please.

>From: =?iso-8859-1?Q?Bj=F8rn_Ole_Fredrik_=D8deg=E5rd?=

    Pine does not at present decode this MIME header information, specified
in some RFC which escapes me -- been away from the qomputer too long.
Don't worry, this will be supported sometime, though I have no word as
to whether it will make it into 3.92.

>read them in mail from me, and when i export my incoming mail, they appear in the

    This is weird, that you can see them in exported mail, meaning you have
the correct font, but not within Pine -- usually you'll see things properly
with Pine so long as you have an appropriate font.
    Is there any other information you can give?  Like, perhaps, you use
Pine on an X terminal, in a separate window (and thus a different font) from
the window you view the exported mail?  Do you see a space where the special
characters should be, or are they just missing (Bjrn Ole Fredrikdegrd, for
example, in the From: header you quoted above)?  Give a bit more information
about the specifics of your setup to help track down what's failing, please.
-- 
 Barry Bouwsma, will crash computers for chocolate and dental care
    Please direct mail to  <[email-masked]> (MIME-aware)

---

From: [email-masked] (Alan J Flavell)
Date: Sun, 12 Mar 1995 20:03:55 GMT

In article <[email-masked]>
[email-masked] (Robert E Adams) writes:
 
>Do not use them. Use only the standard character set for
>your terminal type, based on an old mainframe terminal.
 
Just keep chipping away at those stone tablets: these
quill pens and papyrus are highly overrated...
 
>They do not recognize more modern character sets, and will
>not print them properly.
 
That's one of the purposes of MIME's quoted-printable,
to exchange documents between consenting 8-bit platforms in
a way that will not be harmed by 7-bit data paths.
 
It stands to reason that if your platform does not support
the character glyphs in question, then you cannot display
them, directly.  But it is not completely clear from what the
poster wrote that their terminal or terminal emulation is
fundamentally incapable of displaying these characters: maybe
they have simply failed to set it up correctly (as happens if
I set my terminal emulator up for UK-ASCII and tell PINE
wrongly that it is set for US-ASCII).
 
Check the manual for your terminal (or terminal emulation
software) to find out how to put it into ISO8859-1 mode.
it's no use configuring PINE to use ISO8859-1 unless the
terminal does the correct thing when receiving this code.
 
In general, if you and your correspondent both have
proper compliant platforms, you have every right to want to
interchange mail using all your available characters, and PINE
(with MIME encoding) is an excellent way to do that,
 
Don't expect it to work on usenet, though.  MIME compliance
is a rarity, so far, in news reader software.
 
As for the poster's comments about national characters
being included in email addresses.  What is legal in
email addresses is laid down by RFC822.  I don't suppose you
are allowed to have national characters as elements in email
addresses.  This part of the mail interchange is not included
in the MIME encoding.
 
>I always adapt. If I am talking British money, I write
>out 'pounds sterling' or use # or LB that most people there
>would understand. Their pound sterling symbol is likely not
>to print or display on the receiving end.
 
PINE with MIME do this perfectly fine.  PINE switches into
quoted-printable, declares in the header that it's using
ISO8859-1, and represents our pound-sterling sign as =A3.
 
LB is definitely wrong, by the way - that's for pounds
weight, not Pounds Sterling, which is best abbreviated as
GBP (GB being the ISO code for United Kingdom), although
UKL is also understood by those with a little classical education.
However, I digress.
 
At the receiving end if the platform is compliant it displays
the pound sterling.  If it does not support MIME it displays
=A3 (the hexadecimal escape code for the ISO8859-1 code point).
If it supports MIME but does not support ISO8859-1 (for
example, it only supports US-ASCII), PINE displays a warning.
 
Your advice about avoiding national characters is still valid
for many situations.  Comp.mail.pine is one of the few places
where this advice can be deprecated...    ;-)
 
regards

---

From: [email-masked] (Alan J Flavell)
Date: Mon, 13 Mar 1995 11:42:18 GMT

In article <[email-masked]>
[email-masked] (Barry Bouwsma) writes:
 
>In a previous article, [email-masked] (Alan J Flavell) says:
...
>>                                               (as happens if
>    Hmmm, to my knowledge there is no such MIME charset defined as UK-ASCII.
 
Please, read what I wrote.  I can set up my terminal emulator for UK-ASCII
(it is in fact identical to 7bit US-ASCII with the exception of the
hash codepoint, which in UK mode displays as pound-sterling).  This has
nothing to do with MIME charsets or PINE configuration parameter values -
neither MIME nor PINE had been invented at the time this misbegotten
7bit code was defined.

---

From: [email-masked] (Barry Bouwsma)
Date: 13 Mar 1995 04:15:57 GMT

In a previous article, [email-masked] (Alan J Flavell) says:

>But it is not completely clear from what the

    Hmmm, to my knowledge there is no such MIME charset defined as UK-ASCII.
Maybe a version of ISO 646 that goes under a different name...

This got me thinking, remembering that very old Pines would display any
characters not covered by US-ASCII as an underscore if you did not configure
Pine with the same character set with which the message was tagged, unless
you specifically told Pine to display all characters.
    I don't know if this might have been the problem, and I hope most
sites would have upgraded to 3.8* or 3.9* to eliminate this problem,
but I know there are some which haven't.
    The original poster wrote that the characters were visible when
viewing the mail *outside* of Pine, but not while in Pine, which is what
I find peculiar -- the later Pines pass all data to the display regardless
of your charset emulation; the worst they do is display a warning.

>addresses.  This part of the mail interchange is not included

    You may include non-US-ASCII characters in headers, in the manner
quoted in the original post.  This is covered by MIME, and there is a
particular RFC whose number I've forgotten which describes this.  (Could
it be RFC1522?)  But no 8-bit data may appear in a header, so it must
be encoded in a manner similar to Q-P or BASE64.  I believe Eudora is one
of a few mailers which do this.

---

From: [email-masked] (Robert E Adams)
Date: Sun, 12 Mar 1995 18:14:26 GMT

Olav Georg Karlsen ([email-masked]) wrote:

: I am using Pine 3.91 (in Norway)
: I would like to use the norwegian characters æøå and ÆØÅ. (They might not appear
: right, but should be oslash, aring, etc. (ae, oe, aa).
: I have therefore configured Pine to use:

: character-set          = iso-8859-1                                             

: When i recieve mail from a person with these letters in his/her name, it
: appears like this:

: From: =?iso-8859-1?Q?Bj=F8rn_Ole_Fredrik_=D8deg=E5rd?=
:      <[email-masked]>

: When i type these letters in the message body or receive mail containing them, they
: do not appear at all on screen, but the  letters ARE there, because the recipient can
: read them in mail from me, and when i export my incoming mail, they appear in the
: text file.

: Any solutions?

: Olav Karlsen

Bob,

%% Bob Adams [email-masked] %%%%% HRD   %%%%%%%%  PUB REL   %%%%%%
% Adams Assoc   (902) 456-0761    %    TRAINING        DACUM          %
%      [email-masked]        %    EDITING         HTML           %
%      Inexpensive services       %    GRAPHICS        GAME ANIMS ART %
%   provided over the internet    % Space LEASING  Real Est PROFORMAS %
%%%%%%%%%%% http://www.cfn.cs.dal.ca/~ac200/adams.html  %%%%%%%%%%%%%%%
            Visit my home page for CIAU Mens Basketball

---

From: Roman Czyborra <[email-masked]>
Date: Thu, 16 Mar 1995 18:30:50 +0100

> > there is no such MIME charset defined as UK-ASCII.

> This has nothing to do with MIME charsets or PINE configuration
> parameter values - neither MIME nor PINE had been invented at the
> time this misbegotten 7bit code was defined.

Barry misunderstood you.  You weren't going to send out UK-ASCII.  But
you could send out ISO-646-GB which is the MIME label (RFC 1700) for
that old British standard 4730.  My users would be able to read that.

---

From: [email-masked] (Barry Bouwsma)
Date: 17 Mar 1995 18:35:19 GMT

In a previous article, [email-masked] (Alan J Flavell) says:

>[I said...]
>>    In other words, with Pine, the only way to see the correct characters
>>that are intended to be sent is for the sending device to match the
>>display device.  (Probably PC-Pine has some translation, ...
> 
>Well, now that question is on the table, so I would be interested
>to hear the answer...

    I can't say for sure about ISO Latin-1 support with CP850.  I do know
that I chose to run PC-Pine as a batch file which first loaded an ISO 8859-2
code page on the PC (this was in Slovakia and Czechia), then ran Pine, then
restored the original code page, so no translation was needed.  I'm pretty
sure the support for Latin-2 and PC code pages is not there, though (please
correct me if I'm wrong).

    The FTP servers mentioned in my previous posts also contain fonts
and utilities from the Kermit distribution which can be used for this...
ftp://ccsun.tuke.sk/pub/font-archive/pc/  coll or utils for unzipped
ftp://ftp.vszbr.cz/pub/pcfonts/

-- 
 Barry Bouwsma, will crash computers for chocolate and dental care
    Please direct mail to  <[email-masked]> (MIME-aware)
                driving a 300 baud Trabant on the Datenautobahn
When in Ann Arbor, Michigan this week, visit the 33rd Ann Arbor 16mm Film Fest

---

From: [email-masked] (Alan J Flavell)
Date: Fri, 17 Mar 1995 17:45:43 GMT

In article <[email-masked]>
[email-masked] (Barry Bouwsma) writes:
 
(I wrote)          ...                              and at the other
>>end will need to do its best to map those onto the display used
>>by the receiving platform.
>
>    To avoid further misunderstanding, let me point out that Pine
>has no routines for translating a particular character set as defined
>in incoming mail, or outgoing mail for that matter, to a different
>character set of a display device.  Pine simply passes through all
>data as the 8-bit characters they are.
 
Thanks for making that clear.  I was trying to provoke someone
into saying what I had been unable to find clearly stated in the
documentation.     ;-)
 
>    It would be nice if one could configure the display charset and the
>trnasport (mail) character set differently, but that would add quite a
>bit of extra code and tables to translate from one charset to another.
 
This would represent a valuable facility in some situations, in the
sense that the recipient cannot control the sender's choice of
code, and must make do with what they've received.
 
In my previous posting I had drafted a question asking how _PC_ Pine
(which I don't use and am not familiar with) deals with the PC character
set(s), but then I took it out again, thinking that this could be a
topic of its own...

 
>      ..    Kermit which supports a variety of code pages and charsets.)
 
MS Kermit (the last I looked at it) did indeed seem to support more
code pages and characters than most other terminal emulations added
together.
 
>    If your character set doesn't match that with which the mail was set,
>you'll see some characters wrong
 
Quite so.
 
Thanks for clearing up those various points.  Best regards

---

From: [email-masked] (Barry Bouwsma)
Date: 17 Mar 1995 13:53:08 GMT

In a previous article, [email-masked] (Alan J Flavell) says:

>[...]  PINE
>is rather more ambitious than this, in that it can use a range of
>encodings on the wire (as declared in the header) and at the other

    In other words, with Pine, the only way to see the correct characters
that are intended to be sent is for the sending device to match the
display device.  (Probably PC-Pine has some translation, unless I'm
thinking of Kermit which supports a variety of code pages and charsets.)

    The charsets that should be used for mail, and thus those for which
Pine should be configured, are outlined in RFC1700, although use of
regional variants of ISO 646 is deprecated in favor of ISO 8859-1 which
covers a wide variety of western European languages.

    If your character set doesn't match that with which the mail was set,
you'll see some characters wrong -- like the British pound for the US
$ dollar sign with your UK-ASCII, or the currency symbol (sorta a cross
between o and x) on the otherwise-US-ASCII Tesla VT100 clones I used in
Slovakia.  Anyone want to volunteer to write code to perform translation
from a mail MIME charset to a different display charset?  It would be
very useful...

>Well, I now discover that if I consult PINE's online help
>information for its character code parameter, it talks exclusively
>about its effect on outgoing mail, and does not at all mention  the
>effect on the display of incoming mail.  This seems to me something
>of a shortcoming in the information.

    Pine uses the character set you define in the configuration to tag
the message for the recipient.  When you receive mail with a different
tagging, Pine cautions you that some characters may be different, but
goes ahead and passes the characters to your display.  It would be up
to you, if possible, to change your charset to match that specified in
the incoming mail, which is relatively easy to do if running X with an
escape sequence.

    Any volunteers to write code to send X sequences to change the font
used when differently tagged mail comes in?  There are a variety of
MIME-useful charset fonts available from
ftp://ftp.vszbr.cz/pub/X11-fonts/   or
ftp://ccsun.tuke.sk/pub/font-archive/X11/
for people running X desiring support for additional languages...

---

From: [email-masked] (Nelson Chin)
Date: 18 Mar 1995 00:07:30 GMT

where in RFC 1700, says which characters sets are registered to be used
with MIME? i thought IANA takes care of MIME registered character
sets?

Nelson

Roman Czyborra ([email-masked]) wrote:

: Barry misunderstood you.  You weren't going to send out UK-ASCII.  But
: you could send out ISO-646-GB which is the MIME label (RFC 1700) for
: that old British standard 4730.  My users would be able to read that.

---

From: David L Miller <[email-masked]>
Date: Fri, 17 Mar 1995 13:56:00 -0800

-----BEGIN PGP SIGNED MESSAGE-----

On Fri, 17 Mar 1995, Alan J Flavell wrote:

> Date: Fri, 17 Mar 1995 10:40:38 GMT
> From: Alan J Flavell <[email-masked]>
> Newgroups: comp.mail.pine
> Subject: Re: International character sets
> 
>  
> I have to say frankly that I was approaching the problem with
> a mindset somewhat conditioned by WWW, where (on http) there is
> only one transfer code defined (ISO-8859-1) and it is the job
> of the software to map to and from the display codes used by the
> local platforms (be they PC-850, "EBCDIC", or whatever).  PINE
> by the receiving platform.  Which (and we come back to my original
> point) is often capable of being configured by a separate mechanism,
> that is not known to PINE, and might include codes that have no
> defined name in MIME.
>  

Actually, Pine is less ambitious since it does not do any translation. 
The character-set parameter tells Pine what to _assume_ the user's
terminal is set to use for a character set.  We may investigate some form
of translation capabilities in the future, but probably not for the next
release... 

> of a shortcoming in the information.

The only effect on incoming mail is to display a warning about potential
problems at the top of a message when the character-set parameter and the
charset in the message do not match... 

|\ |  |\/|  David L. Miller    [email-masked]  (206) 685-6240
|/ |_ |  |  Software Engineer, Pine Development Team   (206) 685-4045 (FAX)
University of Washington, Networks & Distributed Computing, JE-20
4545 15th Ave NE, Seattle WA 98105, USA                 finger for PGP key

---

From: Roman Czyborra <[email-masked]>
Date: Thu, 16 Mar 1995 18:30:50 +0100

---

From: [email-masked] (Barry Bouwsma)
Date: 17 Mar 1995 13:53:08 GMT