Subject: international character sets

From: [email-masked]
Date: Mon, 25 Apr 1994 08:38:31 +0200

Helmut Brammerts                  [email-masked]
Seminar fuer Sprachlehrforschung  FAX:  +49 234 7094 138
Ruhr-Universitaet Bochum          Tel.: +49 234 700 5748 / 3815
 ---------------------------------------------------------------------------
In our international project ("International E-Mail Tandem Network" - with
bilingual communication between language learners) we need international
character sets and MIME compliant e-mail tools like PINE.

But we do not really understand PINE's behaviour with respect to character
sets. We expected 4 steps:

1. PINE (sender) knows about platform specific codes (PC code-pages, Windows'
   ANSI, etc.) and maps them to a more general character set like ISO-8859-1.
2. PINE (sender) encodes the message for transport (Base64 or whatever).
3. PINE (receiver) decodes the message.
4. PINE (receiver) maps the general character set (ISO-8859-1 ..) to the
   corresponding platform (Unix, Mac, ...) specific character set.

But we suspect that pine doesn't behave like that. When we send a message
from pc-pine and read it using pine on a NeXt (console or telnet), all
special characters (umlauts, accented letters etc.) are mapped to
nonsense letters.

.pinerc/PINERC say "character-set=ISO-8859-1" on both machines. Is there
anything else we have to do?

Thanks in advance for your help.
Helmut

---

From: David L Miller <[email-masked]>
Date: Tue, 26 Apr 1994 15:05:30 -0700 (PDT)

Helmut,

PC-Pine includes mapping tables for some of the IBM codepages to and from
ISO-8859-1 and ISO-8859-5, but the other platforms assume that the terminal
or emulation is using the character set specified by character-set in your
.pinerc file.  

>From the Pine Technical Notes:

          In the case of PC-Pine, the  character  values
          cannot  be  passed  thru to the display device
          unaltered  since  MS-DOS  uses  various   non-
          standard character sets called "Code Pages".  

          The mapping between DOS Code Page and standard
          character    set    is   controlled   by   the
          "character-set" variable in  the  PINERC  file
          and  the  PC's  installed  Code Page.  PC-Pine
          will automatically map  common  characters  in
          IBM  Code Pages 437, 850, 860, 863, and 865 to
          ISO-8859-1  and  back  when  the  PINERC   has
          "character-set=ISO-8859-1".   Pine  will  also
          map common characters for IBM Code Page 866 to
          ISO-8859-5  and  back when "character-set=ISO-
          8859-5".  The mappings are bi-directional, and
          applied  to  all saved text attachments in the
          defined character set, messages exported, etc.

          Alternatively, the translation tables  can  be
          configured  externally and applied at run time
          whenever the "character-set=" variable is  set
          to  something  other  then "US-ASCII" (the de-
          fault).  PC-Pine looks in the text file point-
          ed  to by the environment variable "ISO_TO_CP"
          for the table to use for mapping text matching
          the type defined by the "character-set=" vari-
          able into the local Code Page value.   PC-Pine
          looks  in  the text file pointed to by the en-
          vironment variable "CP_TO_ISO" for  the  table
          to use for mapping text in the local Code Page
          into   outbound   text   tagged    with    the
          "character-set=" variable's value.

          A text file containing a character set mapping
          table  is  expected  to  contain  256 elements
          where  each  element  is  a   decimal   number
          separated from the next element by white-space
          (space, tab or newline, but no commas!).   The
          index  of the element is the character's value
          in the source character set, and the element's
          value  is  the corresponding character's value
          in the destination character set.

Does that help?

--DLM

|\ |  |\/|  David L. Miller    [email-masked]  (206) 685-6240
|/ |_ |  |  Software Engineer, Pine Development Team   (206) 685-4045 (FAX)
University of Washington, Networks & Distributed Computing, JE-20
4545 15th Ave NE, Seattle WA 98105, USA