Subject: Converting VMS Folders

From: [email-masked] (Kevin D. Joseph)
Date: 22 Jun 1995 13:58:21 GMT

Hi,

   I would like to know if there is an easy way to convert/copy my email
folders used with VAX mail for use with Pine on an SGI unix box.  Any
help would be greatly appreciated!

Thanks,

Kevin Joseph
[email-masked]

---

From: [email-masked] (Alan J Flavell)
Date: Thu, 22 Jun 1995 16:30:21 GMT

>help would be greatly appreciated!
 
Theoretically (i.e meaning that I believe it would work, but I have not
actually moved folders around this way myself) you could either run
PINE under VMS and have it contact IMAPD on the SGI box, or run IMAPD
on VMS and and have your unix PINE make contact with it.
 
For experiences running Yehavi's port of VMS PINE, see:
 
http://ppewww.ph.gla.ac.uk/%7Eflavell/vms-pine.html
 
The price is right!

---

From: [email-masked] (Hugh McIntyre)
Date: 23 Jun 1995 16:41:16 GMT

|> In article <[email-masked]>
|> [email-masked] (Kevin D. Joseph) writes:
|>  
|> >   I would like to know if there is an easy way to convert/copy my email
|> >folders used with VAX mail for use with Pine on an SGI unix box.  Any
|> >help would be greatly appreciated!

The way I converted VMS mail folders to Unix format was to use the "vmsmail.awk"
program attached below.  There are also several other old news articles with
other possibilities.  Note: this stuff comes with no guarantees, and a bit of
fine-tuning may be needed.  Also, it assumes you want to create a "standard"
Unix mailbox, not MMDF or MH or whatever format.

Basically, for each VMS folder:

   a) Start up VMS mail and use "EXTRACT/ALL somefile".
   b) Copy this file to your Unix system.
   c) Run "vmsmail.awk" or whatever program.
   d) You now have a Unix mail version of the folder.

Hugh.

-- 
| Hugh McIntyre                                      |     [email-masked]
| SGS-Thomson Microelectronics Ltd, 1000 Aztec West, | or: [email-masked]
| Almondsbury, Bristol, BS12 4SQ, UK.                | or: [email-masked]
| Tel: +44 (0)1454 611443,  FAX: +44 (0)1454 620688  |

---------------------------------------------------------------------------
Article: 6993 of comp.unix.programmer
Path: inmos.co.uk!inmos!uknet!pipex!uunet!haven.umd.edu!darwin.sura.net!newsserver.jvnc.net!netnews.upenn.edu!wang
From: [email-masked] (jia wang)
Newsgroups: comp.unix.ultrix,comp.mail.multi-media,comp.unix.programmer
Subject: Re: vms mail -> unix mail (summary)
Message-ID: <[email-masked]>
Date: 17 Jun 93 14:33:26 GMT
Sender: [email-masked]
Followup-To: comp.unix.ultrix
Organization: Wharton Center for Academic Computing
Lines: 704
Nntp-Posting-Host: anvil.wharton.upenn.edu
Originator: [email-masked]

In article <[email-masked]>, [email-masked] (jia wang) writes:
|> 
|> 
|> Hello, all the unix and vms experts out there,
|> 
|> I have a vms-unix mail question which needs your help. I would like
|> to transfer all the files under the [.mail] directory on a VAX
|> machine, which includes all *.mai files, to a unix machine. And is
|> it possible to access these mail messages from unix with all the
|> folder information preserved?  I was told that someone has done
|> that successfully. I understand that some conversion or programming
|> could be involved, and will appreciate any hint.
|> 
|> Thank you very much. Also could you common copy my account when you
|> reply if possible, thanks again.
|> 
|> Jia
|> 
|> 

I would like to thank the following people for responding my mail,
and providing me useful programs, shellscripts and suggestions ...
Under the help of them, this problem is solved very quickly.

Rob Riepel 		([email-masked]) 
Mike Khaw  		(khawparcplace.com)
Russ Jones 		([email-masked])
H.A. Kippenhan Jr.  	([email-masked])
Mark Haidl            	([email-masked])
Stephen W. Thompson  	([email-masked])
Marco Luchini 		([email-masked])

Thanks a lot! And sorry that I posted multiple copies of this message 
to multiple newsgroups.  

Jia
==================== summary ==============================================
>From [email-masked] Wed Jun 16 13:52:17 1993
Posted-Date: Wed, 16 Jun 93 10:52:02 -0700
Received-Date: Wed, 16 Jun 93 13:52:17 EDT
Date: Wed, 16 Jun 93 10:52:02 -0700
From: [email-masked] (Russ Jones)
To: [email-masked] (jia wang)
Subject: Re: vms mail --> unix mail
Status: R

Jia,

This attached script is what I and many other people inside of Digital use.  
It only does a folder at a time, but it works. :-)

Russ Jones
Digital Equipment Corporation
------------------------------ cut here ---------------------------------------

#! /bin/sh
##
## Extensively rewritten by Bob Heiney on Thu Jan 21 11:56:36 PST 1993
## to make the script actually work with real VMS mail files, and to
## fix many lame bugs.
##
# $Header: /u3/acs/jdpeek/.bin/RCS.Z/vmsmail2mh,v 1.2 90/03/22 06:53:59 jdpeek Exp $
###	vmsmail2mh - split VMS mail messages from EXTRACT/ALL into MH folder
###	Usage: vmsmail2mh [+folder] [...vmsmsg-files]
##
##	AN EASY WAY TO TRANSFER VMS MAIL MESSAGES TO UNIX IS BY EXTRACTING
##	ALL THE MESSAGES FROM A VMS MAIL FOLDER INTO A FILE... USE:
##		MAIL> EXTRACT/ALL
##		_file: FILENAME
##	THEN, YOU CAN ftp THEM TO THE SYSTEM RUNNING MH.
##
##	WHEN YOU GET THEM TO THE MH SYSTEM, vmsmail2mh WILL READ THROUGH
##	THE FILE (OR STANDARD INPUT, IF YOU DON'T GIVE A FILENAME):
##		- THE MESSAGES ARE SEPARATED BY FORMFEEDS (CONTROL-L).
##		  vmsmail2mh WILL CUT THEM INTO SEPARATE MH MESSAGES
##		  IN THE CURRENT FOLDER (OR +folder IF YOU GIVE ONE).
##		- THE FORMAT OF THE VMS MAIL MESSAGE IS DIFFERENT.
##			* Subj: LINES ARE CONVERTED TO Subject: LINES.
##			* From: LINES ARE SPLIT INTO From: AND Date: LINES.
##		  ONLY THE FIRST Subj: AND From: IN EACH MESSAGE IS CONVERTED.
##
##	BUG: VMS ADDRESSES LIKE:
##		SMTP%"user@host"
##		HOST::USER
##	AREN'T CONVERTED.  IN FACT, *NO* ADDRESSES ARE CONVERTED.

#	NOTE: PROGRAM HASN'T BEEN TESTED THOROUGHLY.
# 	CHECK THE MESSAGES!
#	Placed in the public domain.  Use at your own risk.
#		--Jerry Peek, 22 March 1990

dfltprot=600	# DEFAULT MESSAGE PROTECTION (IF NONE IN MH PROFILE)
folopts="-fast -nolist -nototal -nopack -norecurse"
mh=/usr/bin/mh
scanopts="-noclear -noheader -noreverse"

case "$1" in
[@+]?*)
	# IF $1 DOESN'T EXIST, folder WILL CREATE IT (SIGH).
	# THAT'S BECAUSE stdout IS REDIRECTED AWAY FROM TERMINAL.
	if $mh/folder $folopts "$1" </dev/null >/dev/null
	then
		# GET PATHNAME OF FOLDER, LAST MESSAGE NUMBER:
		folpath="`$mh/mhpath`" || exit
		firstmsg="`$mh/scan -format '%(msg)' last`" || firstmsg=0
	else
		echo "`basename $0`: no folder.  quitting." 1>&2
		exit 1
	fi
	;;
*)	echo "Usage: `basename $0` +folder|@folder [file]
	('$1' doesn't start with + or @.)" 1>&2
	exit 1
	;;
esac

if [ -n "$2" -a \( ! -r "$2" \) ]
then
	echo "`basename $0`: quitting: can't read VMS file '$2'." 1>&2
	exit 1
fi

# GET PROTECTION MODE FROM MH PROFILE (IF NONE, USE $dfltprot):
#msgprot="`$mhprofile -b msg-protect`" || msgprot=$dfltprot
msgprot=$dfltprot

# MAKE SHELL ARCHIVE FILE ON awk'S STANDARD OUTPUT.
# PIPE IT INTO sh TO CREATE THE MESSAGE FILES...
nawk "BEGIN {
	folpath=\"$folpath\" # STORE AS STRING, WITH QUOTES
	msgprot=$msgprot     # STORE AS NUMBER (NO QUOTES)
	msgnum=$firstmsg"'   # CHANGE FROM DOUBLE- TO SINGLE-QUOTES
	inmsg = 1
	sawff = 0
	suppressff = 1
}
# PROCESS MESSAGES.  EACH MESSAGE STARTS WITH CONTROL-L
{
	# MESSAGE STARTS WITH CTRL-L ON A LINE BY ITSELF.
	# PRINT SHELL COMMANDS AND RESET FLAGS:
	if ($1 ~ /^From:/ && sawff) {
	  if (!suppressff) {
		printf "END-OF-%s/%d\n", folpath, msgnum
		# SET PROTECTION (INEFFICIENT; SHOULD DO ALL MSGS. AT ONCE)
		printf "chmod %d %s/%d\n\n", msgprot, folpath, msgnum
	  }
	  suppressff = 0
	  inmsg = 0
	}
	else
	{
	  sawff = 0
	}
	# AT START OF MESSAGE, PRINT START OF ARCHIVE AND SET FLAGS:
	if (inmsg == 0) {
		inmsg = 1
		msgnum += 1
		printf "/bin/cat > %s/%d << \\END-OF-%s/%d\n", \
			folpath, msgnum, folpath, msgnum
		didsubj = 0
		didfrom = 0
	}
	# TURN Subj: INTO Subject: AND SET FLAG TO MAKE SURE WE DO NOT
	# TRASH ANY OTHER Subj: LINES (LIKE IN FORWARDED MESSAGES).
	# STRIP DATE (14-MAR-1990 15:31:12.02) OFF END OF THE From: LINE
	# AND MOVE IT TO LINE OF ITS OWN, THEN SET FLAG LIKE ABOVE:
	if ($1 ~ /^Subj:/ && didsubj == 0) {
		$1 = "Subject:"
		didsubj = 1
		print $0
	}
	else if ($1 ~ /^From:/ && didfrom == 0) {
		printf "%s %s", $1, $2
		if (NF > 4) {
			printf " ("
			for (i = 3; i <= NF - 2; i++) {
				if (i!=3) { printf " " }
				printf "%s", $i
			}
			printf ")"
		}
		printf "\n"
		printf "Date: %s %s\n", $(NF - 1), $NF
		didfrom = 1
	}
	else if ($0 ~ /^\f$/ && inmsg == 1) {
		sawff=1
		next
	}
	else
		print
}
END {
		printf "END-OF-%s/%d\n", folpath, msgnum
		# SET PROTECTION (INEFFICIENT; SHOULD DO ALL MSGS. AT ONCE)
		printf "chmod %d %s/%d\n\n", msgprot, folpath, msgnum
}' $2 |		# IF NO $2, WILL READ STANDARD INPUT...
/bin/sh -e	# EXIT ON ANY ERROR

exit

>From [email-masked] Wed Jun 16 13:07:40 1993
Posted-Date: Wed, 16 Jun 93 10:06:29 PDT
Received-Date: Wed, 16 Jun 93 13:07:39 EDT
Date: Wed, 16 Jun 93 10:06:29 PDT
From: [email-masked] (Mike Khaw)
To: [email-masked]
Subject: Re: vms mail --> unix mail
Newsgroups: comp.mail.elm
Status: R

You may be able to use the attached awk script. I last used it about 4
years ago when I last dealt with a VMS machine. With any luck, mail file
formats on VMS haven't changed since then and you can use the script
as is. Otherwise, it'll at least serve as a starting point. Awk isn't
too difficult to learn.

--- documentation file for vmsmail.awk ---
The awk script "vmsmail.awk" converts mail headers in a VMS 3.5 "MAIL.MAI" file
to a format that the Unix mail program can understand.

A VMS mail message begins with a header of the form

	From: USER dd-MMM-yyyy hh:mm *

where the asterisk indicates a new (unread) mail message

A Unix mail message begins with

	From USER www Mmm dd hh:mm:ss yyyy

where Www is a 3 letter abbreviation of the day of the week (first letter
capitalized), and Mmm is the month counterpart.

New Unix mail messages are unmarked.  Old (read) messages are marked with a
line of the form

	Status: RO

following the "Subj:" header line, which both VMS and Unix mail have.

Finally, an ASCII form-feed precedes each VMS mail message, but Unix mail
messages are separated by a blank line following each message.

Therefore, what the awk script has to do is
	(1) find the beginning of a VMS mail message (i.e., distinguish it
		from formfeeds and lines beginning "From:" within the body
		of a mail message), and change the message separator from
		a formfeed to a blank line.
	2) figure out what day of the week a given date corresponds to,
		and reformat the "From" header accordingly.
	3) if the message is an old/read one, insert the "Status: RO" line
		following the "Subj:" line.

The awk script starts out by setting up (a) an array of day-of-week names
indexed by integers in the range 0 to 6 inclusive (this allows modulo
arithmetic computations), (b) an array of Unix-mail format month names
indexed by VMS-mail format month names, and (c) an array showing what day of
the week the first of each month was in 1972 (a safe baseline year because
it precedes the release dates of VMS 3.5 and 4.2bsd Unix), indexed by Unix-
format month name.  The awk "BEGIN" clause has a function analogous to compile-
time initialization in C (see the awk tutorial, section 2.1).

The body of the script contains 4 pattern-match directed clauses (see section
1.2 in the awk tutorial), where the last clause is a "default" that matches
any line in the input, because it has no pattern to match.  The first 3 clauses
each look for a line beginning with a particular token.  Because awk attempts
to match each input line to EVERY clause, each clause ends with "next" to
force awk to go on to the next line of input.

Let's examine the simplest clause first and work towards the most complex.

1) Whenever a line begins with a formfeed (^L), the script simply sets a flag
	called "pagesep" (page separator).  The "^L" gets printed only if the
	next line is not the beginning of a new mail message; therefore, the
	script does not act on it immediately.  (It can't check "the previous
	line" because a VMS mail file can begin with "^L" as its first line.)

2) If a line does not begin with "From:", "Subj:" or "^L", the script checks
	to see if there is a deferred "^L" that is within the body of a mail
	message (pagesep = 1 and newmsg = 0; newmsg = 1 as long as the script
	is dealing with mail headers), and if so prints the "^L" and unsets
	"pagesep" before it prints the current line.

3) If the line begins with the token "Subj:", it gets printed.  Then if the
	"newmsg" flag is still on, indicating that this is a header line,
	the string "status" gets printed and the script unsets "newmsg".
	"status" is set in the remaining clause:

4) If an input line begins with the token "From:", the script first checks to
	see if "pagesep" is set.  If not, this line is just a line within the
	body of a mail message, so it gets printed.  Otherwise, the script
	prints a blank line message-separator, unsets "pagesep", and sets
	"newmsg" to 1 to indicate that it found the start of a new message.
	Then, it sets the string variable "status" to "Status: RO" if the input
	line didn't have an asterisk (fewer than 5 tokens), or empty otherwise.
	Next, it extracts the date, month and year, translating VMS' all-caps
	month name to the Unix equivalent.  Finally, it calculates the day of
	the week, and prints the Unix-format "From" line.
--- documentation file for vmsmail.awk ---

--- vmsmail.awk ---
# awk program to reformat VMS mail header for Unix mail
BEGIN	{ sun = 0;	dname[sun] = "Sun"
	  mon = 1;	dname[mon] = "Mon"
	  tue = 2;	dname[tue] = "Tue"
	  wed = 3;	dname[wed] = "Wed"
	  thu = 4;	dname[thu] = "Thu"
	  fri = 5;	dname[fri] = "Fri"
	  sat = 6;	dname[sat] = "Sat"
	  mname["JAN"] = "Jan";	firstof["Jan"] = sun
	  mname["FEB"] = "Feb";	firstof["Feb"] = wed
	  mname["MAR"] = "Mar";	firstof["Mar"] = wed
	  mname["APR"] = "Apr";	firstof["Apr"] = sat
	  mname["MAY"] = "May";	firstof["May"] = mon
	  mname["JUN"] = "Jun";	firstof["Jun"] = thu
	  mname["JUL"] = "Jul";	firstof["Jul"] = sat
	  mname["AUG"] = "Aug";	firstof["Aug"] = tue
	  mname["SEP"] = "Sep";	firstof["Sep"] = fri
	  mname["OCT"] = "Oct";	firstof["Oct"] = sun
	  mname["NOV"] = "Nov";	firstof["Nov"] = wed
	  mname["DEC"] = "Dec";	firstof["Dec"] = fri
	}
$1 == "From:" {
	if (!pagesep)		# not a new mail header
	  {
		print
		next
	  }

	print ""		# substitute blank line for formfeed
	pagesep = 0
	newmsg = 1
	# unread mail has an asterisk as a 5th field
	if (NF < 5) status = "Status: RO"	# Unix flags OLD messages
	else status = ""
	# reformat date/time from DD-MMM-YYYY HH:MM to Day Mmm DD HH:MM:00 YYYY
	dash = index($3, "-")
	date = substr($3, 1, dash - 1)
	month = substr($3, dash + 1, 3); month = mname[month]
	year = substr($3, dash + 5, 4)
	# find day-of-week (baseline = Jan 1, 1972, a Saturday)
	y = year - 1972
	leapdays = int(y/4) - int(y/100) + int(y/400) + 1
	d = y + leapdays + firstof[month] + date - 1 + sat
	if ((year % 4) == 0 || (year % 400) == 0)
		if (month == "Jan" || month == "Feb")
			--d	# adjust for date in Jan/Feb of a leap year
	if (date < 10) date = " " date
	print "From", $2, dname[d % 7], month, date, $4 ":00", year
	next
  }
$1 == "Subj:"	{
	print
	if (newmsg)	# "Status" after "Subj"
	  {
		print status
		newmsg = 0
	  }
	next
  }
$1 == ""	{ pagesep = 1; next }	# delay formfeeds

  {		# ordinary line
	if (pagesep && !newmsg)
	  {
		pagesep = 0
		print ""
	  }
	print $0
	next
  }
--- vmsmail.awk ---
-- 
Michael Khaw	[email-masked] (or [email-masked])
ParcPlace Systems, Sunnyvale, CA	PRODUCT INFO: [email-masked]

>From [email-masked] Wed Jun 16 12:41:35 1993
Posted-Date: Wed, 16 Jun 93 09:41:32 -0700
Received-Date: Wed, 16 Jun 93 12:41:35 EDT
Date: Wed, 16 Jun 93 09:41:32 -0700
From: [email-masked] (Rob Riepel)
To: [email-masked] (jia wang)
Subject: vms mail --> unix mail
Status: Re

Here's a VMS fortran program to convert VMS mail files to unix mail files.

Message  4:
>From [email-masked] Wed Jun 16 14:34:28 1993
Posted-Date: Wed, 16 Jun 1993 13:34:25 -0500 (CDT)
Received-Date: Wed, 16 Jun 93 14:34:27 EDT
Received: by anvil.wharton.upenn.edu (5.57/Ultrix3.0-C)
        id AA22812; Wed, 16 Jun 93 14:33:30 -0400
Date: Wed, 16 Jun 1993 13:34:25 -0500 (CDT)
From: "H.A. Kippenhan Jr." <[email-masked]>
To: [email-masked]
Message-Id: <[email-masked]>
Subject: VMS mail <---> Unix mail
Status: RO

        Hi Jia:
        The only way you would be able to preserve the folder information
        is to place each folder in a separate Unix file.  The VMS mail
        file (MAIL.MAI) is an indexed sequential file.  Short mail messages
        (1,536 or less bytes in length) are stored in MAIL.MAI.  There is
        a fourth block associated with each message that contains the
        overhead associated with the indexed file; one portion of that
        overhead is a secondary key -- the folder name.

        Since Unix doesn't support indexed files, it would seem very
        difficult to transport the folder concept from VMS to Unix.
        I suppose you could make separate files for each folder (I believe
        that the Berkeley mail tool does this), but that implies some
        file gymnastics on the Unix side.

        I hope this is of some help!

        Best regards

        - Kipp -
 {~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
 { H.A. Kippenhan Jr.               |   Internet:   [email-masked] }
 { National HEPnet Management       |   HEPnet/NSI DECnet:  FNDCD::KIPPENHAN }
 { Fermi National Accelerator Lab.  |   BITnet:       [email-masked] }
 { P.O. Box 500   MS: FCC-3E/368    |   Telephone:            (708) 840-8068 }
 { Batavia, Illinois 60510          |   FAX:                  (708) 840-8463 }
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~