Subject: Why Content-Transfer-Encoding: should be passed "as is"

From: Tomas Kullman - ICL ProSystems <[email-masked]>
Date: Sun, 30 Oct 1994 19:12:41 +0100

Hej everybody!

The issue is: Should an IMAP server pass the original string value of
the Content-Transfer-Encoding: header field? Or not? And why not?

Here's a clip from my letter to Mark Crispin dated 26 june 1994:

> > What do you mean by ``encodings not recognized by the daemon''?  I 
> > am a little bit confused here.
> 
> Ah. Well, when the daemon analyze a message and prepare to transfer the
> structure it matches the 'Transfer-Encoding:' header field against a
> static array of strings corresponding to possible encodings. In case of
> no match the daemon mark this as "UNKNOWN". Sofar so good. The problem
> occurs when transferring of the structure takes place. The encoding is
> still marked "UNKNOWN" when arriving to the client.
> My opinion is that the daemon should NOT censor this kind of information
> at the client. At least not when the MIME specification allows
> 'X-myencoding' as a valid encoding.
> Can you see my point of view? The reason why we discovered this at all
> was because we were experimenting with uuencode inside MIME marking it
> with "X-UUENCODE". I know uuencode is a sensitive issue among pro-MIME
> people, but we had to deal with the reality and lot of customers was
> demanding a swift solution fast.

Attached to this message are two messages I sent 27 & 28 june to Mark C.
The second message contains a correct patch file that solves the problem.

Mark, I hope you don't feel bad about this letter. Back then I think you
already left office for vacation and neither of us raised the issue again.
I just want this to be settled. I would like to have comments from other
persons as well.
--
Tomas Kullman <[email-masked]>
Internet Ambassador @ ICL ProSystems AB

Subject: Proposed modification to imap-3.3
From: Tomas Kullman <[email-masked]>
To: [email-masked]
Date: Mon, 27 Jun 1994 13:45:25 -0100

Hi Mark.

In an earlier letter to you I was pointing out that our MUA wanted
the IMAP daemon to pass Content-Transfer-Encoding: value "as is".

We have been made a modification to the c-client and imapd sources
to support that. It has been in production (that is, used at our
customer sites) for quite a while now. I thought it could be a good
idea to incorporate this into the "official" version, if it's okey
with you. I have been cleaning up the modification to a minimum,
incorporated it in the latest version (imap-3.3 17 June 1994) and
finally made a diff on that. See separate text part.

I would like you to comment this and how you feel about to apply
this to the "official" version.

About the c-client modifications for our MUA, I'll be back later.

	Tomas
Jun 27 11:40 1994  diff -crl imap-3.3.orig/ANSI/c-client/mail.c imap-3.3/ANSI/c-client/mail.c Page 1

*** imap-3.3.orig/ANSI/c-client/mail.c	Fri Jun 10 10:25:58 1994
--- imap-3.3/ANSI/c-client/mail.c	Mon Jun 27 11:32:07 1994
***************
*** 1428,1433 ****
--- 1428,1434 ----
  {
    body->type = TYPETEXT;	/* content type */
    body->encoding = ENC7BIT;	/* content encoding */
+   body->encoding_string = NIL;	/* tomku : content encoding string */
    body->subtype = body->id = body->description = NIL;
    body->parameter = NIL;
    body->contents.text = NIL;	/* no contents yet */
***************
*** 1489,1494 ****
--- 1490,1497 ----
  
  void mail_free_body_data (BODY *body)
  {
+   /* tomku : give away encoding string */
+   if (body->encoding_string) fs_give ((void **) &body->encoding_string);
    if (body->subtype) fs_give ((void **) &body->subtype);
    mail_free_body_parameter (&body->parameter);
    if (body->id) fs_give ((void **) &body->id);
Jun 27 11:40 1994  diff -crl imap-3.3.orig/ANSI/c-client/mail.h imap-3.3/ANSI/c-client/mail.h Page 1

*** imap-3.3.orig/ANSI/c-client/mail.h	Fri Jun 10 08:34:03 1994
--- imap-3.3/ANSI/c-client/mail.h	Mon Jun 27 11:15:13 1994
***************
*** 203,208 ****
--- 203,209 ----
  BODY {
    unsigned short type;		/* body primary type */
    unsigned short encoding;	/* body transfer encoding */
+   char *encoding_string;	/* tomku : body transfer encoding string */
    char *subtype;		/* subtype string */
    PARAMETER *parameter;		/* parameter list */
    char *id;			/* body identifier */
Jun 27 11:40 1994  diff -crl imap-3.3.orig/ANSI/c-client/rfc822.c imap-3.3/ANSI/c-client/rfc822.c Page 1

*** imap-3.3.orig/ANSI/c-client/rfc822.c	Fri Apr 15 08:08:55 1994
--- imap-3.3/ANSI/c-client/rfc822.c	Mon Jun 27 11:25:13 1994
***************
*** 307,313 ****
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     body_encodings[body->encoding]);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
--- 307,315 ----
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     /* tomku : we already know the encoding string */
! 	     /*body_encodings[body->encoding]);*/
! 	     body->encoding_string);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
***************
*** 733,738 ****
--- 735,742 ----
      else if (!strcmp (name+1,"RANSFER-ENCODING")) {
  				/* flush out any confusing whitespace */
        if (t = strchr (ucase (s),' ')) *t = '\0';
+       /* tomku : remember original encoding string */
+       body->encoding_string = cpystr(s);
  				/* search for body encoding */
        for (i = 0; (i < ENCOTHER) && strcmp (s,body_encodings[i]); i++);
        body->encoding = i;	/* set body type */
Jun 27 11:40 1994  diff -crl imap-3.3.orig/ANSI/imapd/imapd.c imap-3.3/ANSI/imapd/imapd.c Page 1

*** imap-3.3.orig/ANSI/imapd/imapd.c	Wed Jun  8 23:44:13 1994
--- imap-3.3/ANSI/imapd/imapd.c	Mon Jun 27 11:36:27 1994
***************
*** 1002,1008 ****
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       pstring ((char *) body_encodings[body->encoding]);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:
--- 1002,1010 ----
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       /* tomku : already know encoding string */
!       /*pstring ((char *) body_encodings[body->encoding]);*/
!       pstring (body->encoding_string);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:
Jun 27 11:40 1994  diff -crl imap-3.3.orig/non-ANSI/c-client/mail.c imap-3.3/non-ANSI/c-client/mail.c Page 1

*** imap-3.3.orig/non-ANSI/c-client/mail.c	Fri Jun 10 10:26:58 1994
--- imap-3.3/non-ANSI/c-client/mail.c	Mon Jun 27 11:30:24 1994
***************
*** 1560,1565 ****
--- 1560,1566 ----
  {
    body->type = TYPETEXT;	/* content type */
    body->encoding = ENC7BIT;	/* content encoding */
+   body->encoding_string = NIL;	/* tomku : content encoding string */
    body->subtype = body->id = body->description = NIL;
    body->parameter = NIL;
    body->contents.text = NIL;	/* no contents yet */
***************
*** 1623,1628 ****
--- 1624,1631 ----
  void mail_free_body_data (body)
  	BODY *body;
  {
+   /* tomku : give away encoding string */
+   if (body->encoding_string) fs_give ((void **) &body->encoding_string);
    if (body->subtype) fs_give ((void **) &body->subtype);
    mail_free_body_parameter (&body->parameter);
    if (body->id) fs_give ((void **) &body->id);
Jun 27 11:40 1994  diff -crl imap-3.3.orig/non-ANSI/c-client/mail.h imap-3.3/non-ANSI/c-client/mail.h Page 1

*** imap-3.3.orig/non-ANSI/c-client/mail.h	Fri Jun 10 08:33:56 1994
--- imap-3.3/non-ANSI/c-client/mail.h	Mon Jun 27 11:15:36 1994
***************
*** 203,208 ****
--- 203,209 ----
  BODY {
    unsigned short type;		/* body primary type */
    unsigned short encoding;	/* body transfer encoding */
+   char *encoding_string;	/* tomku : body transfer encoding string */
    char *subtype;		/* subtype string */
    PARAMETER *parameter;		/* parameter list */
    char *id;			/* body identifier */
Jun 27 11:40 1994  diff -crl imap-3.3.orig/non-ANSI/c-client/rfc822.c imap-3.3/non-ANSI/c-client/rfc822.c Page 1

*** imap-3.3.orig/non-ANSI/c-client/rfc822.c	Fri Apr 15 08:34:10 1994
--- imap-3.3/non-ANSI/c-client/rfc822.c	Mon Jun 27 11:23:39 1994
***************
*** 327,333 ****
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     body_encodings[body->encoding]);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
--- 327,335 ----
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     /* tomku : we already know the encoding string */
! 	     /*body_encodings[body->encoding]);*/
! 	     body->encoding_string);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
***************
*** 770,775 ****
--- 772,779 ----
  				/* flush out any confusing whitespace */
        if (t = strchr (ucase (s),' ')) *t = '\0';
  				/* search for body encoding */
+       /* tomku : remember original encoding string */
+       body->encoding_string = s;
        for (i = 0; (i < ENCOTHER) && strcmp (s,body_encodings[i]); i++);
        body->encoding = i;	/* set body type */
      }
Jun 27 11:40 1994  diff -crl imap-3.3.orig/non-ANSI/imapd/imapd.c imap-3.3/non-ANSI/imapd/imapd.c Page 1

*** imap-3.3.orig/non-ANSI/imapd/imapd.c	Wed Jun  8 23:48:00 1994
--- imap-3.3/non-ANSI/imapd/imapd.c	Mon Jun 27 11:35:50 1994
***************
*** 1029,1035 ****
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       pstring ((char *) body_encodings[body->encoding]);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:
--- 1029,1037 ----
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       /* tomku : already know encoding string */
!       /*pstring ((char *) body_encodings[body->encoding]);*/
!       pstring (body->encoding_string);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:

Subject: Re: Proposed modification to imap-3.3
From: Tomas Kullman <[email-masked]>
To: [email-masked]
Date: Tue, 28 Jun 1994 02:26:53 -0100

Hi Again.

I was cleaning up little too much.
I was ignorant enough not testing messages defaulting to 7BIT.
I re-introduced a test for 7BIT in which case we lookup the string
as before. Otherwise we pull out the original string (imapd.c:1005).

I send you a fixed diff text part.
This time in <cr><lf> style.
(just for my own sake)

	Tomas
Jun 27 18:25 1994  diff -crl imap-3.3.orig/ANSI/c-client/mail.c imap-3.3/ANSI/c-client/mail.c Page 1

*** imap-3.3.orig/ANSI/c-client/mail.c	Fri Jun 10 10:25:58 1994
--- imap-3.3/ANSI/c-client/mail.c	Mon Jun 27 11:32:07 1994
***************
*** 1428,1433 ****
--- 1428,1434 ----
  {
    body->type = TYPETEXT;	/* content type */
    body->encoding = ENC7BIT;	/* content encoding */
+   body->encoding_string = NIL;	/* tomku : content encoding string */
    body->subtype = body->id = body->description = NIL;
    body->parameter = NIL;
    body->contents.text = NIL;	/* no contents yet */
***************
*** 1489,1494 ****
--- 1490,1497 ----
  
  void mail_free_body_data (BODY *body)
  {
+   /* tomku : give away encoding string */
+   if (body->encoding_string) fs_give ((void **) &body->encoding_string);
    if (body->subtype) fs_give ((void **) &body->subtype);
    mail_free_body_parameter (&body->parameter);
    if (body->id) fs_give ((void **) &body->id);
Jun 27 18:25 1994  diff -crl imap-3.3.orig/ANSI/c-client/mail.h imap-3.3/ANSI/c-client/mail.h Page 1

*** imap-3.3.orig/ANSI/c-client/mail.h	Fri Jun 10 08:34:03 1994
--- imap-3.3/ANSI/c-client/mail.h	Mon Jun 27 11:15:13 1994
***************
*** 203,208 ****
--- 203,209 ----
  BODY {
    unsigned short type;		/* body primary type */
    unsigned short encoding;	/* body transfer encoding */
+   char *encoding_string;	/* tomku : body transfer encoding string */
    char *subtype;		/* subtype string */
    PARAMETER *parameter;		/* parameter list */
    char *id;			/* body identifier */
Jun 27 18:26 1994  diff -crl imap-3.3.orig/ANSI/c-client/rfc822.c imap-3.3/ANSI/c-client/rfc822.c Page 1

*** imap-3.3.orig/ANSI/c-client/rfc822.c	Fri Apr 15 08:08:55 1994
--- imap-3.3/ANSI/c-client/rfc822.c	Mon Jun 27 11:25:13 1994
***************
*** 307,313 ****
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     body_encodings[body->encoding]);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
--- 307,315 ----
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     /* tomku : we already know the encoding string */
! 	     /*body_encodings[body->encoding]);*/
! 	     body->encoding_string);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
***************
*** 733,738 ****
--- 735,742 ----
      else if (!strcmp (name+1,"RANSFER-ENCODING")) {
  				/* flush out any confusing whitespace */
        if (t = strchr (ucase (s),' ')) *t = '\0';
+       /* tomku : remember original encoding string */
+       body->encoding_string = cpystr(s);
  				/* search for body encoding */
        for (i = 0; (i < ENCOTHER) && strcmp (s,body_encodings[i]); i++);
        body->encoding = i;	/* set body type */
Jun 27 18:26 1994  diff -crl imap-3.3.orig/ANSI/imapd/imapd.c imap-3.3/ANSI/imapd/imapd.c Page 1

*** imap-3.3.orig/ANSI/imapd/imapd.c	Wed Jun  8 23:44:13 1994
--- imap-3.3/ANSI/imapd/imapd.c	Mon Jun 27 14:24:46 1994
***************
*** 1002,1008 ****
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       pstring ((char *) body_encodings[body->encoding]);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:
--- 1002,1010 ----
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       pstring (body->encoding
! 	       ? body->encoding_string /* tomku : valid when not 7BIT */
! 	       : (char *) body_encodings[body->encoding]);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:
Jun 27 18:26 1994  diff -crl imap-3.3.orig/non-ANSI/c-client/mail.c imap-3.3/non-ANSI/c-client/mail.c Page 1

*** imap-3.3.orig/non-ANSI/c-client/mail.c	Fri Jun 10 10:26:58 1994
--- imap-3.3/non-ANSI/c-client/mail.c	Mon Jun 27 11:30:24 1994
***************
*** 1560,1565 ****
--- 1560,1566 ----
  {
    body->type = TYPETEXT;	/* content type */
    body->encoding = ENC7BIT;	/* content encoding */
+   body->encoding_string = NIL;	/* tomku : content encoding string */
    body->subtype = body->id = body->description = NIL;
    body->parameter = NIL;
    body->contents.text = NIL;	/* no contents yet */
***************
*** 1623,1628 ****
--- 1624,1631 ----
  void mail_free_body_data (body)
  	BODY *body;
  {
+   /* tomku : give away encoding string */
+   if (body->encoding_string) fs_give ((void **) &body->encoding_string);
    if (body->subtype) fs_give ((void **) &body->subtype);
    mail_free_body_parameter (&body->parameter);
    if (body->id) fs_give ((void **) &body->id);
Jun 27 18:26 1994  diff -crl imap-3.3.orig/non-ANSI/c-client/mail.h imap-3.3/non-ANSI/c-client/mail.h Page 1

*** imap-3.3.orig/non-ANSI/c-client/mail.h	Fri Jun 10 08:33:56 1994
--- imap-3.3/non-ANSI/c-client/mail.h	Mon Jun 27 11:15:36 1994
***************
*** 203,208 ****
--- 203,209 ----
  BODY {
    unsigned short type;		/* body primary type */
    unsigned short encoding;	/* body transfer encoding */
+   char *encoding_string;	/* tomku : body transfer encoding string */
    char *subtype;		/* subtype string */
    PARAMETER *parameter;		/* parameter list */
    char *id;			/* body identifier */
Jun 27 18:26 1994  diff -crl imap-3.3.orig/non-ANSI/c-client/rfc822.c imap-3.3/non-ANSI/c-client/rfc822.c Page 1

*** imap-3.3.orig/non-ANSI/c-client/rfc822.c	Fri Apr 15 08:34:10 1994
--- imap-3.3/non-ANSI/c-client/rfc822.c	Mon Jun 27 14:01:09 1994
***************
*** 327,333 ****
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     body_encodings[body->encoding]);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
--- 327,335 ----
    strcpy (*dst += strlen (*dst),"\015\012");
    if (body->encoding)		/* note: encoding 7BIT never output! */
      sprintf (*dst += strlen (*dst),"Content-Transfer-Encoding: %s\015\012",
! 	     /* tomku : we already know the encoding string */
! 	     /*body_encodings[body->encoding]);*/
! 	     body->encoding_string);
    if (body->id) sprintf (*dst += strlen (*dst),"Content-ID: %s\015\012",
  			 body->id);
    if (body->description)
***************
*** 769,774 ****
--- 771,778 ----
      else if (!strcmp (name+1,"RANSFER-ENCODING")) {
  				/* flush out any confusing whitespace */
        if (t = strchr (ucase (s),' ')) *t = '\0';
+       /* tomku : remember original encoding string */
+       body->encoding_string = cpystr(s);
  				/* search for body encoding */
        for (i = 0; (i < ENCOTHER) && strcmp (s,body_encodings[i]); i++);
        body->encoding = i;	/* set body type */
Jun 27 18:26 1994  diff -crl imap-3.3.orig/non-ANSI/imapd/imapd.c imap-3.3/non-ANSI/imapd/imapd.c Page 1

*** imap-3.3.orig/non-ANSI/imapd/imapd.c	Wed Jun  8 23:48:00 1994
--- imap-3.3/non-ANSI/imapd/imapd.c	Mon Jun 27 14:24:07 1994
***************
*** 1029,1035 ****
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       pstring ((char *) body_encodings[body->encoding]);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:
--- 1029,1037 ----
        putchar (' ');
        pstring (body->description);
        putchar (' ');
!       pstring (body->encoding
! 	       ? body->encoding_string /* tomku : valid when not 7BIT */
! 	       : (char *) body_encodings[body->encoding]);
        printf (" %d",body->size.bytes);
        switch (body->type) {	/* extra stuff depends upon body type */
        case TYPEMESSAGE:

---

From: Mark Crispin <[email-masked]>
Date: Sun, 30 Oct 1994 12:06:04 -0800 (PST)

Tomas -

This change was incorporated into c-client on June 30, 1994.  imap-3.3 is
an obsolete version of the c-client.  The current distribution is
imap-3.5 (used in Pine 3.91), the current beta (used for Pine 3.92 
development) is imap-3.6.BETA, and the current development is imap-4 (not 
used at all because it doesn't compile yet...).

Thank you for your interest in c-client.

-- Mark --