Subject: Re: filtering, once and for all

From: [email-masked] (Trey Harris)
Date: 7 May 1995 22:43:05 GMT

In article <[email-masked]>,
Bob Brody <[email-masked]> wrote:
>Define the problem: I subscribe to many listservs, but when respective
>mail arrives, all I see as far as sender is concerned is a person's name
>(or ename, whatever).  I *do not* see that it's from xyz listserv.
>Consequently my inbox, newmail, etc., is inundated with hundreds of
>mailings daily and I have no way of telling if an email is via listserv
>or something specifically sent to me one-to-one.  Subject info isn't
>necessarily helpful in differentiating listserv from personal email.

You're suffering from a very common misunderstanding of the way the list
distribution programs work.  The misunderstanding is because it is
initially counterintuitive, but once you think about it, I think you'll 
understand.

You see, when we talk about email that we have received because we are on
a list, we say, "oh, I just got mail from the ziffle-bang list".  But
really, the ziffle-bang list didn't write anything.  A *person*, say Jane
Smith from Kenosha, sent that mail, and in actuality the mail is *from*
Jane.  All the ziffle-bang list did was propogate it to many different 
people, much the same as if you had received the mail yourself and 
bounced it to a bunch of your friends.

It's not the From header that is important in determining whether the 
mail is coming from a list, it's the *TO* header.  What you really want 
to know is not who the mail is from, but whether it is to you, or to you 
and ten thousand of your closest pals on the ziffle-bang list.

This always surprises me when I hear people struggling with this 
backwards way of doing things, mostly because I've been using and running 
lists for so long that this seems only natural.  But if you think about 
it, from the list's point of reference, this *is* natural.  The mail is 
coming *from* Jane.  It's going *to* the list, which you happen to be 
on.  Lists aren't publications, but it's easy to fall into the trap of 
thinking of them as such.

So, check the To line instead of the From line.  Better yet (and this is 
what I do because it's so easy) look to the left of the numbers in your 
folder index.  Mail to a list should have only the "N" for new messages.  
But mail that is specifically to you should have a plus next to it, too.  
If it doesn't, then that's something that should be fixed in Pine 3.92.  
But probably it works, and you just haven't noticed the little plus sign.
-- 
Trey Harris                              http://sunsite.unc.edu/harris
  System Administrator, Project Isis, Office of Information Technology
                       The University of North Carolina at Chapel Hill

---

From: [email-masked] (Bob Brody)
Date: 8 May 1995 01:26:41 GMT

>thinking of them as such.

Well, if that's the way I should think of it, then that's how I will,
I'm easy.  But I have 'ta tell 'ya, it's not how I see it.  When I
recieve a magazine in the mail, say, Smithsonian, or Wired, Mad (my
favorite) or whatever, there are lots of contributing articles to the
gist of the magazine.  These articles are not "from" so-&-so, they're
"by" so-&-so, and they're not sent to "me" they're sent to the
magazine.  I receive so-&-so's article "from" the magazine, not from
so-&-so.

Consequently, where I use Internet services that don't present an index
of mail via Pine, I get an Inbox that might look like this:

    [email-masked]             Filing deadlines
    outdoor-L@ulkyvm              trash bins          
    outdoor-L@ulkyvm              RE: trash bins           
    Shannon Brody                 Hi, Dad
    ODP-L@tamvm1                  Slant drilling mishap

Whereas via Pine:

    Brent Wilson                  Filing deadlines
    Ellen Smith                   trash bins          
    Scott McIntyre                RE: trash bins           
    Shannon Brody                 Hi, Dad
    Bill Whitman                  Slant drilling mishap

And when I have many Inbox screens filled with sometimes hundreds of
mails arrived, it's just much easier to note the real source of the
mail, the list, to distinguish what's what and from where.  From "whom"
is not the source lest it's a personal email and to present just a
person's name doesn't help when it's coming from a listserv because the
mail is really not "to me."  So I respectfully disagree with your view
as to the origin of an email. 

>So, check the To line instead of the From line. 

But the To line is not what's listed in my Inbox index screen, the
person the mail is from is what's listed.  OTOH, if there's a Pine
setting that will show me the To: line rather than the From: line when
I view my Inbox index, then perhaps that's precisely what I'm after and
I haven't configured Pine properly to do it.

>Better yet (and this is 

Now we're in sync.  I was over-looking this and indeed you're right,
the plus sign distinguishes personal email and I thank you for the tip
(I wonder if I can perform a sort that will organize personal email
from everything else; anyway, no matter, looking for the plus sign is
very helpful, thanks).

Remains, however, to be able to distinguish various listserv mailings
but in fact I'm not as concerned about that now that I'm reminded of
the efficacy of the plus sign.  Thanks for the reply.

Regards, Bob

---

From: [email-masked] (Bob Brody)
Date: 7 May 1995 03:40:16 GMT

So I simply must be able to separate mail if only visually in order to
spot personal email from listserv and/or junk email.  Some services I've
used in the past list listserv mail as sent from the listserv, not by
name of who wrote the note, followed by subject/thread info.  That makes
it very easy to tell which is from which listserv or if personal email.
But I'm not able to do that with Pine.

Suggestions/instructions/documentation on how to fix this problem will
be most appreciated.  I read a faq on filtering, by Nancy McGough, but
the instructions do not work via many ISP's, including this one,
Primenet.  No fixit suggestions from her end and evasive techno-
gibberish from Primenet's end as to why her .forward file's string
doesn't work here, etc., etc., etc.

So I need help on this because I find myself either subscribing to
different services just to subscribe to a single listserv thus always
knowing which mail is which, or unsubscribing from listservs because I
just can't distill personal email from hundreds of listserv posts when
everything is presented by personal sender name.

Clearly (no pun intended, well, maybe so <g>) there's got to be an
expeditious way to deal with this.  I frustratingly seek your experience
if you've figured it out.

Bob
[email-masked]

---

From: [email-masked] (Bob Brody)
Date: 8 May 1995 06:26:08 GMT

Phil Tompkins ([email-masked]) wrote:
> 
>When you display your list of mail, press the semi-colon key,
>which is the select command.  When you are asked for criteria,
>choose T (text), then A (all text), then key in the name of
>one of your listservs.
> 
>Then press A (apply next command to all selected mail).
>Then you can either save all selected messages to another folder,
>or you can export to a file in your home directory.
> 
>I use export.  Since I have communication software on my PC with
>buffer capture to disk, I exit Pine, go to shell, turn on buffer
>capture,then cat the file I exported to.   Then I read the stuff
>off line.
> 
>A little awkward, but when Pine comes out with a scripting ability
>it will be just fine.

A "little" awkward??  I've got more than a dozen listservs to deal
with.  I'm familiar with the approach you mention but I'm not one who
gets off on fondling my keyboard.  I just wanna see what mail I've got,
especially in the morning when I first check for anything urgent before
going about my business.  OTOH, I've been thinking my PC's comm program
has a fine scripting language and I might very well be able to automate
the procedure you've outlined.  I use its scripting language for
automating a number of online actions.  I think I can massage it for
this purpose as well.  

Ideally, I'd just like to see listserv mail amid real email show up
like this:

      outdoor-l             (size)     new campsites
      outdoor-l             (size)     trail bikes
      biotech               (size)     a breakthrough
      law-net               (size)     tort reform
      law-net               (size)     RE: tort reform
      Jessica Brody         (size)     I need $$, Pop
      archeaology-l         (size)     scrolls translation

etc.  While I appreciate what was said about how another views lists
mail, I view 2-way lists as a kind of interactive magazine.  I would
like to see my mail listed by way of the list and of course respective
article subjects.  Probably no need to have more than the list's name
which prefixes the @ sign.  I'm afraid I can't see this as any simpler
or easier to distinguish which mail is which at instant glance.  The
previous explanation given me as to how listserv mail should be
considered simply strikes me as time consuming if not arcane.

 bob
 [email-masked]

---

From: Phil Tompkins <[email-masked]>
Date: Sun, 7 May 1995 22:32:38 -0400

> (or ename, whatever).  I *do not* see that it's from xyz listserv.

etc.

Try this:
Go into Setup.  Choose configure.  Activate the aggegate command
option.  Exit Pine, then go back in.

 

Phil

---

From: [email-masked] (Barry Bouwsma)
Date: 8 May 1995 12:30:35 GMT

>      archeaology-l         (size)     scrolls translation

    In this case you want Pine to display your index not by the From: field,
but rather by the Sender: field (wherein the LISTSERV name can be found).
Doesn't appear to be an option to select this, although you can select how
messages should be saved -- whether by the list name, or by the author: the
former corresponds to the by-sender rule, and the latter to by-from.

    I agree that it would be a desirable feature for one to select whether
the Sender: or the From: field is displayed in the index, and maybe the Pine
team will add this to 3.92, to give you the behavior shown above.

-- 
 Barry Bouwsma, will crash computers for chocolate and dental care
    Please direct mail to  <[email-masked]> (MIME-aware)
In a blindfolded taste test, 4 of 5 newsreaders could not tell the
difference between Spam and Velveeta...

---

From: [email-masked] (Erin A. O'Neill)
Date: 9 May 1995 13:28:35 -0700

You all posted some nice info on mail headers & pine but if I understand 
the question right it's how do you filter ?

In the pine docs & manual it says you gotta get another software to 
filter & most sites offer the elm filter (at the Unix prompt type 
which filter) I set up a filter to send my listEmail to different folders 
to separate my work Email from my list Email. If you are interested in 
the gory details send me Email & I'll send you the how-tos.

erin
[email-masked]
-- 
erin
[email-masked]

---

From: [email-masked] (Trey Harris)
Date: 8 May 1995 19:26:10 GMT

In article <[email-masked]>,
>as to the origin of an email. 

See, that's the clincher.  It isn't coming *from* a listserv, it's coming 
*from* Brent Wilson and Ellen Smith and Scott McIntyre, just like "Hi, 
Dad" is coming from Shannon Brody.

The difference is that in some mail-user agents, if the To: line is not
specifically to you (and the Cc: line also is not specifically to you)
then the index field that usually shows "From" shows To: instead.  Another
possibility is that the mail-user agent you're used to uses the "From "
line rather than the "From:" header (which is actually strictly against 
RFC822, since the "From " line is a Unix convention rather than anything 
having to do with the mail-transport protocols).

Either way, it's really a matter of preference, but I prefer the way Pine 
does it because if I'm on, say, the Perl Developers list, I'd definitely 
want to immediately read a post from Larry Wall, the author of Perl, 
entitled "New feature for Perl?".  I wouldn't feel the same immediacy to 
read a post with the same subject line from someone I've never heard of, 
since they're probably just making a suggestion.

Perhaps choosing the behavior (either a toggle button to switch between 
To: lines and From: lines, or a pine config variable that would allow the 
user to pick between the above behaviors) would be a nice addition to Pine?

>I haven't configured Pine properly to do it.

Well, yes.  I didn't mean you should *look* at the To: header with your 
eyes per se, just "look" at it with whatever tools.  So, for instance 
(assuming you have enable-aggregate-command-set checked in your config 
variables), you can type ";" (Select), "T" (Text), "T" (To), 
"ziffle-bang" (name of list), <return>, "Z" (Zoom), and now you'll see 
all the messages in the ziffle-bang list.

>the efficacy of the plus sign.  Thanks for the reply.

Just keep doing the above for each list, and you can read each list in 
sequence rather than all jumbled together.  To select your personal mail, 
just type "brody" instead of a list name in the above.
-- 
Trey Harris                              http://sunsite.unc.edu/harris
  System Administrator, Project Isis, Office of Information Technology
                       The University of North Carolina at Chapel Hill

---

From: Wayne Wilson <[email-masked]>
Date: 11 May 1995 11:39:08 GMT

>But probably it works, and you just haven't noticed the little plus sign.
  The plus sign helps but is not totally sufficient.  In my office, we nearly 
alwasys include other people in mail, especially mail that talks about business and 
setting up meetings.  So it is rather rare for me to get any mail with a plus sign 
on it at all.  I still have the problem of not knowing if the mail is from a list or 
not.  Filtering will be the only hope, given the nature of lists.  I might very well 
have to change e-mail packages soon just to get filtering.

---

From: [email-masked] (Bob Brody)
Date: 11 May 1995 09:33:48 GMT

>the gory details send me Email & I'll send you the how-tos.

Thank you for the file.  Your quick-start instructions took all of
three minutes to implement and then I began to read the full length
doc.  About 30 minutes into it I was curious so suspended reading and
did a quick check of mail.  Wonders, but there it was.  All mail ever
so neatly residing in respective file/folders I want incoming mail to
be filtered into (a few listservs are for business so I created a single
file/folder BIZ for those lists to collectively go into and makes for
much faster skimming in the morning when I haven't time to spare, then
separate files/folders for more casual listservs, each file/folder
named apropos the list.  Personal email, not desiring to filter it,
goes to the Inbox as before.)  Excellent.

I use NEWMAIL to notify me of newmail while online and modified its
command string to include my BIZ mail folder as well.  Works fine.

Quite a difference now, to load Pine and just see my personal email,
then go to respective folders to view listserv mail.

Again, thanks.

P.S.  What I don't understand is why it's taken so long to find this
very easy information.  I've asked everywhere and nobody has a straight
answer.  System Administrators, tech support, UNIX gurus, nada, when
'lo, I finally receive the info and in some 15 lines of explanation and
3 minutes later, it's installed and working fine.  I dunno.  Sometimes
I wonder about this "Information Highway."  Anyway, pardon my spleen, I
was just exhaling the pent up frustration I was experiencing now that
it's over.

---

From: [email-masked] (Roberts Freimuts)
Date: 12 May 1995 12:31:47 GMT

FILTER, ELM, PROCMAIL etc....?

Where can these programs be gotten so that we can all try them?

Roberts Freimuts

---

From: [email-masked] (Bob Brody)
Date: 11 May 1995 22:38:50 GMT

I'm afraid I disagree with your attitude.  You seem to think everyone
should work the way you do and that anything else is, uhh, organized
procastination.  Bull.  Software ulitmately belongs to the user and
flexibility is paramount.  Now that I'm able to fully filter incoming
mail, I have already seen how much FASTER certain mail is tended to,
and how much EASIER it is to deal with it all.  You get 200 mails a
day?  I get that from but one listserv and within there is professional
needs to be dealt with.  Similarly, some mail can be filtered/forwarded
to staff for faster expediting.  Another thing I can now do is auto
filter/collate mail by days of week (and even times of day) to go
directly into daily reports folders that automatically become available
via Web page links per day of week.  No need for physically doing this
anymore, no depending on somebody always doing it even if they over
sleep or are sick or have an emergency to tend to, whatever.

So I respectfully disagree with you on this.  You tell me how I'm
supposed to "think of email", tell another they should read thousands
of mail in the same way you do, and so on.  I believe instead people
should work as THEY prefer and via their own zone.

Cordially, Bob

BTW, the filtering agent I'm using is FILTER and works just fine
through any of the mail programs since as you say it's not the mail
agent, but the filter agent; the mail agent then offers up the mail.
That's of course best since it allows us Pine users to use Pine, Elm
users to use Elm, Mail users to use Mail, and everybody's happy.

---

From: [email-masked] (Trey Harris)
Date: 11 May 1995 14:48:53 GMT

In article <[email-masked]>,
Wayne Wilson  <[email-masked]> wrote:
>  The plus sign helps but is not totally sufficient.  In my office,
>we nearly alwasys include other people in mail, especially mail that
>talks about business and setting up meetings.  So it is rather rare
>for me to get any mail with a plus sign on it at all.  I still have
>the problem of not knowing if the mail is from a list or not.
>Filtering will be the only hope, given the nature of lists.  I might
>very well have to change e-mail packages soon just to get filtering.

And why would you change email user agents?  Simply out of spite for
Pine not doing what you want?  And what mail user agent, pray tell,
will give you this filtering that you so desire?  I can't think of
one.

One last time, and listen VERY CAREFULLY: mail-filtering agents and
mail-user agents are TOTALLY SEPARATE.  Different animals, they aren't
used in the same way, don't get invoked the same way, and most
certainly do not do the jobs of the other.  You're not going to use
procmail anytime soon to read your mail (though I once had a wizardly
friend who set up procmail to find him on the campus network and pop
up a window every time a mail arrived).  And you won't, ever, filter
your mail when you're not logged in using Pine (unless the development
group has something up their sleeves they're not telling us
about... :)

People whining about their humongous amounts of mail...a few years
back when I posted a now-published essay, I got 2,000 individual
responses in the course of four days. I dealt with it, not by using a
filtering agent, but by staying up late and answering each and every
positive email, and throwing out all the negative. :)

And today I get an average of two hundred messages a day...I just have
everything land in one INBOX and I deal with it.  How? If I have time,
then I read everything.  Big deal, no fuss.  If I have less time, then
I move all my religious mailing-list mail (which is 60% of my email
these days...Unitarians are motormouths!) to another folder for later
reading, using the (;) Select command.  If I really don't have time,
then I just throw away all the list mail and I select mail that is To:
or Cc:'d explicitly to me to read immediately, and set aside the rest
for later.

I still have a few hundred messages always in my INBOX (if you're more
organized, there's no need to have more than a couple dozen, but, hey,
I'm a sysadmin), but my method at least means that all the mail at the
front of the INBOX is mostly mailing list or junk mail, while the mail
at the latest end is mostly directed to me.

When I've used filtering agents, it's been either to throw away mail I
absolutely, positively, never want to see again (like a chain-letter
that I kept getting again, and again, and again--I just auto-forwarded
it to the postmaster at the originating site) or when I've wanted
certain mail to change media--become a fax or a pager message.

I don't see where filtering agents give you any more time.  They just
give you organization, and automated procrastination. :)  And that's a
great boon for some, but for others it isn't really needed.
-- 
Trey Harris                              http://sunsite.unc.edu/harris
  System Administrator, Project Isis, Office of Information Technology
                       The University of North Carolina at Chapel Hill

---

From: [email-masked] (Trey Harris)
Date: 12 May 1995 03:01:06 GMT

In article <[email-masked]>,
Bob Brody <[email-masked]> wrote:
>procastination.  Bull.  [....]

I think this is one of those cases where the inability to read intonation 
into USENET posts is causing flamage where none should exist.

I don't think that everyone should work the way I do.  Neither do I see no
need for filtering programs.  After all, I am writing one, and I wouldn't
do that if I thought they had no use (even though I won't likely use it
much myself).  I'm not a fan of offering things to my users that I don't
think will help them.

I'm simply saying that mail filtering is not a panacea to solve all 
problems, and that for some people, filtering isn't the best option.  I'm 
more fluid in my mailreading than the rigid rules that filtering 
provides; and if I find I'm not reading a list, week after week, and it's 
piling up, then I unsubscribe.

For some people, however, it increases organization drastically.  I 
recognize that, and because of that, I'm trying to get a filtering agent 
that will meet their needs.

>users to use Elm, Mail users to use Mail, and everybody's happy.

Filter unfortunately suffers from some problems of its own, and even the 
filter developers suggest that serious mail-filter users should use 
procmail instead.  But if it does what you want, and you're not 
experiencing mail locking problems, then you should use it.

I should note here that I will soon be removing Elm, Mail, etc. from my
machines.  Not because I don't think they should use the other interfaces
(to the contrary, I have several news readers, web browsers, editors,
etc., even though that increases our work) but because Pine is the only
freely-distributable interface using IMAP, and we're going to remove local
filesystem access to mail very soon (we already have for all our Pine
users).  Is anyone working on backends to Elm, etc., that use IMAP like
Pine does? 
-- 
Trey Harris                              http://sunsite.unc.edu/harris
  System Administrator, Project Isis, Office of Information Technology
                       The University of North Carolina at Chapel Hill

---

From: [email-masked] (Maretta Holden)
Date: Tue, 16 May 1995 22:35:21 GMT

> users to use Elm, Mail users to use Mail, and everybody's happy.

What is FILTER, on which systems does it run and where can it be obtained?

-- 
Maretta T. Holden                   | (206) 662-0232
Boeing Defense & Space Group        | (206) 662-0115 FAX
PO Box 3707 MS 4C-63                |  [email-masked]
Seattle, WA 98124                   |  [email-masked]
--------------------------------------------------------------------------
Postings are personal opinions only, not those of the Boeing Company.

---

From: [email-masked] (Trey Harris)
Date: 12 May 1995 03:01:06 GMT

---

From: [email-masked] (Trey Harris)
Date: 15 May 1995 19:31:14 GMT

In article <[email-masked]>,
Wayne Wilson  <[email-masked]> wrote:
>>mail-user agents are TOTALLY SEPARATE. 
>
>And I assume, Mail servers are totally separate as well.  So that we have a 
>three component architecture:  MTA's, MFA's and MUA's.  This is a new slant
>on mail architecture, I am uncertain where the MFA's are in the process.  If
>I understand things they reside with the server.

Hmmm...I don't think that MFAs are a third prong, because unlike MTAs and 
MUAs, MFAs are optional.

Whether MFAs reside on the server depends on whether you want the 
filtering to happen immediately upon receipt of mail, or at any time up 
to the point when you start your MUA.  If immediate filtering is a 
requirement, then yes, the functions must reside on the server.  On the 
other hand, if you could live with doing the filtering when you log in, 
then you could use procmail (if you have /usr/spool/mail access) or the 
program that I'm writing to filter your mail at that time.  Then it 
wouldn't matter whether it lived on the server side or the client side.

>  Now if only I had control over the server side of mail,
>I might be able to insert something between Pine and my IMAP connection.
>I and a rapidly growing number of folks don't have any say nor
>ability to implement anything on the server side, we simply subscribe to 
>an internet service provider who offers either an IMAP or POP service.

Then you'll have to get your service provider to establish some route for
you to use MFAs.  This thread is reminiscent of how people a few years ago
who used ftp-mail gateways, usenet-mail gateways, web-mail gateways, etc.,
wanted to know why a X-mail gateway didn't exist, where X could be telnet,
MUD, IRC, or some other interactive service.  If you don't like the amount
of access your service provider gives you, then you should agitate for
more access, or find another service provider.  These days, Unix shell
accounts are so cheap that if you want it, you can probably get it. 

>  You must live in an environment where you have control of the server side?

No.  You just need to be in an environment where your provider has 
yielded you enough control of your mail (via .forward files and MFAs) 
where you can do it yourself.  If they haven't, then you can switch to 
any mail user agent, but your level of access won't get any better.
-- 
Trey Harris                              http://sunsite.unc.edu/harris
  System Administrator, Project Isis, Office of Information Technology
                       The University of North Carolina at Chapel Hill