Subject: problems with ftp + tar + compression

From: Bill Wyatt <[email-masked]>
Date: Fri, 12 Dec 1997 16:22:18 -0500

I'm having a problem with a Linux wu-ftpd sometimes dropping the last
part of a transmission.

System: Linux 133MHx Pentium, 160 MB RAM
        Slackware 3.2 (kernel 2.0.29)
        wu-ftpd Version wu-2.4.2-academ[BETA-15]
        GNU tar 1.12
        GNU gzip 1.2.4

The problem I'm encountering existed with the orginal Slackware
installation (wu-ftpd 2.4, tar 1.11.8, gzip ?), so it's a longstanding
problem.

The problem is that a few large (>100MB) transfers across the Internet are
truncated at the very end, with no error message. The tar files are
blocked at 10k, its default, and the amount that is dropped seems to be
consistently 28,672 bytes, which is 30k - 2k.  That number seems
significant, although I don't know why.

The problem occurs only (so far as I know) when using both the tar and
gzip options, i.e. I do a "get  dirname.tar.gz". Also, it only occurs
when I'm sending the data across the relatively slow Internet - i.e.
at about 100Kbytes/sec or less. If I do the same get from another machine
on the Linux machine's local net, there's no problem.

If I create the *.tar.gz file on the remote system and then ftp that,
there is again no problem.  

The error is not 100% consistent, in that, with repeated attempts,
a particular directory usually, but not always, fails to transfer 
correctly. Other directories seem to always transfer ok.

So, it seems like there's some sort of buffer flushing problem, what with
the ftpd process calling the tar, which calls gzip, etc., but my quick
look at the various routines certainly shows nothing obvious. Any ideas or
solutions would be appreciated.

Bill Wyatt  ([email-masked])
    Smithsonian Astrophysical Observatory  (Cambridge, MA, USA)