Subject: Re: automatic processing of ftp'ed files ...

From: Dave Jevans <[email-masked]>
Date: Tue, 2 Dec 1997 12:21:26 -0800 (PST)

>I want to ftp a file to a remote site where a program will automatically
>process it once it's transferred. What are the different ways that the
>remote program can know that the transfer is complete ? (I obviously don't want

You should use FileDrive EX.  It hass all the wu-ftpd features
plus security and *Active Agents*.   Active Agents lets you
define automated actions to run on files.   You can define them
to run at start of transfer, failure of transfer, or completed file transfer.

wu-ftpd does not have this capability.  It doesn't know when a file
has completed transferring.

checkitout at www.filedrive.com

DJ

---

From: Anders Thulin <[email-masked]>
Date: Wed, 3 Dec 1997 09:02:53 +0100 (MET)

*** Subu Rama (whose name is mistaken for a listserv command) writes: 

> remote program can know that the transfer is complete ? (I obviously don't
> want the program to pick up the file in the midst of the transfer)

  Here's one method that works pretty well:

  1) Ensure that you have a distinct log message (syslog or tranfer log) when
the
     transfer is completed successfully. That's your trigger. You may need to
     add log messages to the code for this.

  2) Get swatch. It's log watching program that looks for user-specified
patterns
     in a log file, and executes a user-specified command when such a pattern
     appears. It's very useful for ordinary sysadm work. You'll need perl to
run it.

  3) Set up swatch to watch the logfile where the trigger message appears, and
     set up a pattern:action rule for that particular transfer.

  As you hace access to perl pattern matching, you can set up rather complex
matches.

  Also, this will start the action as soon as the log message appears.
If you don't need that quick response, you can also feed swatch the logs
that have been collected since last time, and so run the necessary actions
at a time that you decide.

  If you'd rather not use perl, it's not too difficult to write a small C
program that does much the same thing by using the regexp package.

hope this helps,

Anders Thulin       [email-masked]        013-23 55 32
Telia Engineering AB, Teknikringen 6, S-583 30 Linkoping, Sweden

Anders Thulin       [email-masked]        013-23 55 32
Telia Engineering AB, Teknikringen 6, S-583 30 Linkoping, Sweden

---

From: [email-masked] (Alan J Rosenthal)
Date: Wed, 3 Dec 1997 11:47:16 -0500

Ftp it to a different file name, then rename it to the expected file name as
a second ftp command.  Then as soon as the file exists (under the expected
name), it's complete.
Renames in unix are atomic.

---

From: Dave Jevans <[email-masked]>
Date: Wed, 3 Dec 1997 10:40:03 -0800 (PST)

>> This doesn't deal with the crux of the issue, however, which 
>> is determining if the file is completely there before processing,
>> or renaming.    wu-ftpd doesn't know if the file is completely there.

>That is true, i couldnt think of a defacto way to figure out if a upload
>was completed fully and successfully. But, you can usually beware of a few
>things. One is, were there any errors? If there were read() type errors or
>something similar, you can be pretty sure the file was not complete. And
ofcourse.. if it was cancled midway through. Altho this wouldnt be a
>perfect fix.. id bet it would catch more than half of incomplete xfers. :)

Yep.  Thats what we do if you are using a regular FTP client.
Check for broken pipes, ABORT commands, etc.   This does get
many of the incompleted xfers.   If the server talks to a client
that supports our extensions, then we enable the more reliable
message checksumming to determine if the file is completely
uploaded.

This is very important if you are using FTP to implement things
like EDI or EFT or various kinds or remote document processing.
In those cases it become pretty critical to know if the
file is completely there before triggering a processing stage.

The problem is that the silly FTP protocol relies on a
read() returning 0 bytes to indicate the end of a file transfer.
There are many failure conditions that look like this.  :-(

DJ