Subject: Attached text always BASE64 encoded?

From: Joe Ilacqua <[email-masked]>
Date: Sat, 23 Oct 1993 13:17:10 -0400

I've just put up 3.87 (from 3.01).  Some of my users are
complaining that attached text files are being encoded in BASE64 and
this seems be the case.  I realize the 3.01 use BASE64 if it found any
8 bit charaters or very long lines, but that is not the case here.  If
there any way to change this behavior?

->Spike

---

From: Terry Gray <[email-masked]>
Date: Sat, 23 Oct 1993 10:25:42 -0700 (PDT)

> there any way to change this behavior?

Spike,
This change was a very conscious decision; and hence is a feature, not a 
bug :)  We thought very hard about this one, so let me share the rationale.

Pine provides two ways of sending text files:
  1. By *inclusion*, using the ^R function in the composer
  2. By *attachment*, using ^J or the attchmnt header line.

Text *inclusion* should be used if the recipient does not have MIME support.
Text *attachment* should be used if the recipient *does* have MIME support.

But why change the *attachment* behavior for text files?  Because we
*strongly* believe that any MIME attachment should arrive intact, without
modification of any kind.  The only way we have to guarantee the integrity
of the attachment is to Base64 encode it.  (Q-P encoding is not sufficient.)

Using MIME attachments should be as safe as doing a file transfer.  Once
we learned that q-p encoding was not sufficient in all cases to prevent
modification of the text attachment during transit through gateways or
delivery, we were not willing to jeopardize the reputation of MIME in
general and Pine in particular by using the more "readable" encoding,
especially given that the ^R alternative was always available when
absolute integrity was not needed. 

Note that this problem completely disappears when your correspondents
upgrade to MIME-aware mailers, as we hope they will soon, so that they 
can fully participate in the Internet mail world.

-teg

---

From: [email-masked] (Barry Shein)
Date: Sat, 23 Oct 1993 14:33:30 -0400

Thanks for the response, now to offer (possibly) some help...

We're experiencing that reported wild looping when someone hangs up on
Pine (3.87), I used Sun's trace to see what it's looping on (this only
reports syscalls, but perhaps that's useful) and here's what it looks
like:

write (1, "\33[22;1H\33[K", 10) = -1 EIO (I/O error)
write (1, "\33[22;29H\33[7m[ \7Unknown command: ".., 39) = -1 EIO (I/O error)
write (1, "\33[6;18H", 7) = -1 EIO (I/O error)
sigblock (0) = 0
sigstack (0, 0xf7ffead4) = 0
setitimer (0, 0xf7ffead0, 0xf7ffeac0) = 0
getpid () = 20467
kill (20467, 1) = - SIGHUP (1)
sigcleanup () = 0
setitimer (0, 0xf7ffead0, 0xf7ffeac0) = 0
gettimeofday (0xf7ffeae8, 0) = 0
write (1, "\33[22;1H\33[K", 10) = -1 EIO (I/O error)
write (1, "\33[22;29H\33[7m[ \7Unknown command: ".., 39) = -1 EIO (I/O error)
write (1, "\33[6;18H", 7) = -1 EIO (I/O error)

Perhaps one easy fix is that someone, somewhere needs to check for
that error return on the write()? Also, not that it is trying to
kill() itself with a SIGHUP.

Anyhow, perhaps that's helpful.

        -Barry Shein

Software Tool & Die    | [email-masked]          | uunet!world!bzs
Purveyors to the Trade | Voice: 617-739-0202        | Login: 617-739-WRLD

---

From: Ralph Sims <[email-masked]>
Date: Sat, 23 Oct 1993 11:36:08 -0700 (PDT)

> > there any way to change this behavior?
 
[ good discussion deleted ]
 
> can fully participate in the Internet mail world.

However, pine is designed for folks that have little awareness of
unix/ms-dos tricks.  If a pine user uploads a file prepared with an
ms-dos editor that leaves a a ctrl-z EOF, or a ctrl-m at the end of
each line, pine makes the determination that this is a binary file,
suitable for BASE64 encoding.  This was documented by one of our users,
Doug Palm, and brought to the development team's attention.  It was
indicated by Dave Miller (I think) that the 'feature' will be
re-designed in the upcoming 3.88 maintenance release, in order to ease
the pain.

We should not assume that pine's users know how to strip these control
characters from uploaded files, or use zip/unzip to remove them, in
order to make them 'true' ascii files, compliant to unix format.  While
this may have been planned as a 'feature', it is rapidly becoming an
adminstrative/helpdesk headache.  Messages posted with pine traverse
many networks, including CompuServe, which is arguably one of the
largest single mail systems around; CIS's mail interfaces do not
provide for MIME attachments, it appears.  While the rest of the world
ramps up to MIME, pine is producing messages that can't be using the
recipient's {traditional,only available} mail reader.

---

From: Terry Gray <[email-masked]>
Date: Sat, 23 Oct 1993 11:35:35 -0700 (PDT)

Thanks Barry!

This SIGHUP looping bug is one of several reasons why a pine 3.88 (mtce 
release) will appear much sooner than we planned :(

Probably before the end of next week...

-teg

---

From: Terry Gray <[email-masked]>
Date: Sat, 23 Oct 1993 11:43:23 -0700 (PDT)

Ralph,
The question of when to encode *in-line* text, as opposed to *attachments*
is definitely distinct, and 3.87 appears to be too aggressive in when it 
encodes in-line text.

I personally feel very strongly that Pine is doing the right thing in 
always encoding *attachments*, but I do not feel that Pine should be 
imposing MIME-ness on everyone else for *in-line* text.  

I need to talk with the team about exactly what will be done, but your
(and Doug's) input on this was received loud and clear. 

-teg

---

From: Ralph Sims <[email-masked]>
Date: Sat, 23 Oct 1993 21:29:47 -0700 (PDT)

> imposing MIME-ness on everyone else for *in-line* text.  

Perhaps a suggestion:  when text is yanked in with a CTRL-R, then
process it as-is--do not invoke encoding.  If text is sent as an
attachment, then use whatever encoding scheme prevails.  Or, when text
is yanked in, ask if this should be encoded or processed as-is.  I can
see times where both features would be useful.

---

From: Terry Gray <[email-masked]>
Date: Sat, 23 Oct 1993 21:33:31 -0700 (PDT)

Ralph,
There probably doesn't want to be any distinction between what is typed 
in directly and what is included via ^R... the question is what 
conditions should trigger q-p or b64 encoding of the inline text. 

We don't want Pine to be overly obnoxious to non-MIME users, but at the 
same time we want to fulfil the requirements of the MIME RFCs, and make 
sure that Pine does the right thing for our non-US-ASCII friends.

We have been using an algorithmic approach to triggering the inline 
encoding, but the algorithm needs refinement.  Stay tuned...

-teg