Subject: Re: Wrong completion status

From: Phil Schwan <[email-masked]>
Date: Thu, 5 Nov 1998 07:30:46 -0500

On Nov 04, Oliver Billmann wrote:
> I just played around with the completion status and ran into a problem:
> I aborted a PUT command and the transfer was logged as complete! Is this
> a bug or is there just no way to determine this correctly? (A PUT hitting
> the quota hardlimit is logged as incomplete...)
> 
> I'm running VR10 on a Linux box.

I've run into a few different situations and haven't really had time
to correct them (nor do I really know how to go about doing so):

* Every once in a while I'll see a RETR that is incomplete being
marked as complete.  I have no idea why.  It doesn't happen often. 

* Cases such as yours--where a STOR is marked as incomplete.  I think
it depends on the client, to be honest.  If the client, when receiving
your signal to abort, closes up the transfer nicely, it's probably
recorded as complete, because the ftp server was given that
impression.  If the client just severs the pipe and lets the server
clean up the mess, it's probably recorded as incomplete.

I've made a note to look at it, and I will when I get a minute, but
it's not at the extreme high-end of my to-do list :).

-Phil

------------------------------------------------------------------
"In film you will find four basic story lines. Man versus man, man
versus nature, nature versus nature, and dog versus vampire."
        - Steven Spielberg

---

From: Gregory A Lundberg <[email-masked]>
Date: Thu, 5 Nov 1998 08:16:37 -0500 (EST)

> it's not at the extreme high-end of my to-do list :).

I, too, have received a report of false-complete status being logged.  
This is the first I've heard of false-incomplete, but it does not surprise
me.  The complete/incomplete status means simply no errors were reported
in the transfer.  We could just leave it at that and say it's a client/OS
problem (which it is).

At this point, I can see the problem (even in my own logs) but cannot
reproduce it.  I do not know which clients, if any, consistently cause the
problem.  And I do not know if it is OS dependant (although I expect some
cases are).  All I do know is this is one of the problems with TCP; we got
the right bytes in the right order but if there was no protocol problem in
termination we have no way of knowing if the sequence was complete .. at
some point we need more information which FTP does not provide.

On RETR, I'll be adding a check that the full file was sent which should
reduce the number of false-complete reports.  For STOR I'm not sure
there's much which can be done; we don't know anything more than the OS
runtime calls claimed no errors occurred.

It seems to me Apache is a bit more stable in this respect.  I intend to
take a look at what, if anything, they've done to make the reporting more
correct.  Remember, however, that HTTP has a ContentLength header which
can be used to help determine if a request was completely received.

If accurate transmission is your goal, I would suggest making use of the
SITE CHECKSUM extensions.  They will allow your client to determine
whether the file was properly transmitted far better than reliance upon
the complete status in the logs.

In closing, I want to stress this has been happening all along.  It's just
now, with Phil's patch, we're recording the fact.  Hey, nobody said the
Emperor _ever_ had any clothes!

-- 

Gregory A Lundberg		Senior Partner, VRnet Company
1441 Elmdale Drive              [email-masked]
Kettering, OH 45409-1615 USA    1-800-809-2195

---

From: [email-masked] (Oliver Billmann)
Date: Mon, 9 Nov 1998 15:54:36 +0100

>> clean up the mess, it's probably recorded as incomplete.

Well, its the Linux command line ftp client, and it says the following
when pressing CTRL-C:

send aborted
waiting for remote to finish abort
226 Transfer complete.

This leaves the impression, that the client sends some kind of notification
to the server (at least if one [like me :] overlooks the 226 line, which
leaves a complete different impression...).

In the meantime I tested this with a SunOS client as well, with the same
result. On WinNT, CTRL-C kills the ftp client and the connection is closed.
The server doesn't even log the partial transfer! Just a "FTP session closed"
in the syslog...

In article <[email-masked]>,

> This is the first I've heard of false-incomplete, but it does not surprise

No, the incomplete I mentioned (when hitting the quota hardlimit) is
correct, at least IMO ;-)

> problem.

With the Linux command line ftp client, it happens every time (at least it
happened the last 10 times I tried).

> the complete status in the logs.

I thought this was a nice informational feature for the users. But because
it's not 100% accurate, I'll probably drop it and take a look at the
checksum extension...

Maybe this behaviour should be documented somewhere...

> Emperor _ever_ had any clothes!

You're absolutely right. If you can come up with a solution, it'd be great.
If not, well, before VR8 (or was it VR9?) we didn't know about this and we
lived happily after all :-]

Oliver

-- 
                                \\\///
      +--------------------+     \\//     +-----------------------------+
      | Oliver Billmann    |    ( .. )    |    [email-masked] |
      +--------------------+-ooO-(__)-Ooo-+-----------------------------+