Subject: Re: File Locking Suggestion

From: [email-masked] (Schoolfield W. R. (Bill))
Date: Wed, 6 Nov 96 12:11:41 CST

> 
> [ deleted ]
> 
> > Here's our problem: 
> > 
> > We have a script under the control of
> > cron that executes periodically. The script's job is to
> > retrieve any deposited files, perform logging and other
> > file processing.
> > 
> > We very often run into problems when we try to "get" a
> > file that is still being "put" (The script detects this
> > situation by comparing the local file size against the 
> > file size on the ftp server-BUT NOT before a very large
> > transfer is attempted).
> > 
> > I have solved this problem by using file locking (lockf 
> > calls). I lock files being "put" and I check files for a
> > lock before a "get". 
> > 
> > The changes are minor; only one source file is changed
> > (wftpd.c).
> > 
> > My questions are this:
> > 
> > 	Is this a OK thing to do? Is it general enough? 
> > 	Where might it fail?
> > 
> > and if the answer is yes...
> > 
> > 	Who do I submit these changes (ideas) to?
> > 
> > Thanks for those who reply. 
> > 
> 
> Ho !! not at all _AU CONTRAIRE_
> This will break the monotony of the usual "_put_-a_and_RTFM" answers.
> After all, this is a place for general discussions and
> bouncing new ideas ;-)
> 
> Where you can submit the patches ? ... I don't know
> But according to ``koos V.D.H'' it's
> 
> KVDH>"[email-masked]"
> KVDH>"For as far as I know, wu-ftpd is not an official project of academ.com.
> KVDH>Therefore things like release dates don't mean much."
> 
> 
> Usually I don't care for new versions. If someone has a new
> idea|patch and it can be of use. I implemente it on my side.
> 
> Now for the locking scheme. Altought I think it will
> be of seldom use to me. I'm curious about those issues.
> 
> 1-
>   Security: I'll never move around blindly files that anybody
> could put in my Ftp server. So to do a cron jobs like the one
> you have, .. send bad vibes to my spine as a sys adm.
> 

The ftp access is not anonymous. Each vendor has its own 
login. The received files are ALL handled with a single script.
Everything is logged and all file are interogated to determine
what format they are in. So... I think I'm ok on the security
front. 

> 2-
>   What kind of lock do you use: advisory|mandatory ?
> 

Advisory.

> 3-
>  if mandatory. What do you do about deadlocks ?
>    -How to prevent malicious attacks: a user putting a lock on the
>     file
>    -Runaway process still having the lock
>    -You'll be suprise of how many broken implementations of Mandatory
>     locks ;)
> 
> 4-
>  if advisory. then what's the point really. All process
>  must implement the scheme... not transparent. In fact
>  the use of "overwrite no" should suffice.
>

Well, the only process that should interact with these files
is wuftpd. So while not fool proof, in practice, I think it
works quite well (particularily for remote ftp servers beyond
the firewall). 
 
> 5-
>  is it {,non}blocking ?

If a file is locked, you just get a 450 message saying the
file is in use. So its non blocking. 
 
> 6-
>  what about those who share ~ftp across machines ?
> 

Don't quite understand your question here. 

> 7-
>  and if your cron job is running on an other machine
>  then ftpd ? I guess you must have some lock deamon
>  to share locks :(
> 

Nope. The locks are server side only. 

> 8-
>  protability issue ? lockf() is not on all platform
>  some use fcntl() to perform locking.
> 

Yea, I know that I need ifdefs for the machines that don't
do locking my way. 

> 9-
>   Well that's enough ;-)
> 
> 10-
>   I'm still very curious to see the code
>

I inserted only ten lines of code. As I look at it I realize
it may need a little work. I did test it though: haven't found anything
wrong with it yet.

I can send it to you if you really want it.
 
Bill
 
> --
> bonne journe'e, alain

----------------------------------------------------------------------------
| Bill Schoolfield                               [email-masked] |
| Union Pacific Resources                                     817-877-6790 |
| P.O. Box 7                                                               |
| Fort Worth, TX 76101                                                     |
----------------------------------------------------------------------------