Subject: Thread safety..

From: Andy Fiddaman <[email-masked]>
Date: Wed, 1 Sep 2004 18:33:37 +0000 (GMT)

Hi List,

I am busy working on integrating the c-client library with an application,
for the moment just to use the MIME parser but it will be extended to use
IMAP.

The application is threaded so I was looking to see how thread-safe the
library is. In general, it looks as if it will be okay if I'm careful
(e.g. don't manipulate driver parameters from a thread),
but there are a couple of places where race conditions could occur. For
example, there is code at the start of unix_header() which populates a
global STRINGLIST if it hasn't been done before.

I'm a bit stuck for what to do here as I don't want to modify the library
code, and I don't want to stick a big lock around all calls into it, is
there any recommended way around this ?

If I were to modify the code, I'd move the 'once only code' out of the
unix_header() function into one called build_unix_hlines() or something
then I could just call this from my application before going threaded.

Any thoughts appreciated,

Thanks,

Andy

-- 
------------------------------------------------------------------
 For information about this mailing list, and its archives, see: 
 http://www.washington.edu/imap/c-client-list.html
------------------------------------------------------------------

---

From: Mark Crispin <[email-masked]>
Date: Wed, 1 Sep 2004 12:11:47 -0700 (PDT)

On Wed, 1 Sep 2004, Andy Fiddaman wrote:
> library is.

The real issue is your C library.  A common problem is that many C 
libraries do not have a thread-safe strtok (meaning that it keeps the 
state per-thread), select(), or host name routines.  Many C library 
implementors weasel their way out of it by making you use other calls such 
as strtok_r().

> In general, it looks as if it will be okay if I'm careful

It has been said that "threads are the crack cocaine of programming";
they are powerful, poorly-understood by many of their users, and have a 
substantial mythology.

There is *nothing* wrong with altering a global in threaded code as long 
as you accept that that alteration affects every thread.  This is a very 
useful attribute (beloved of those of us who wrote multi-forked code with 
shareable writable pages on TOPS-20).  It is something that must be 
thoroughly understood; but it is not something to fear.

In general, when you change a driver parameter, the desired effect is to 
change the global behavior of the driver.  What this means in general is 
that if you want to change a driver parameter, you go ahead and do it. 
Usually, you'll be doing this at startup or under control of the user 
interface, instead of deep within some thread.

> global STRINGLIST if it hasn't been done before.

This is an excellent example.  That global STRINGLIST is global for a 
reason.  It should be a global constant, but it's complex enough to need 
code to build it.

The only concern is whether another thread can run and join that code 
while the STRINGLIST is being built.  For this, you need to understand how 
your thread library works.  If thread switching only happens on I/O, you 
are safe since the only thing that code does is assignment and malloc().

If the threads really are separate kernel processes running in the same 
address space (as in Linux), then a simple mutex will do the trick. 
Here's one that will build even on systems which don't have threading:
 	static long lock = -1;
 	 . . .
 	if (++lock) sleep (1);
 	 . . . protected code . . .
 	lock = -1;
It'll take over 100 years for the lock to overflow with that sleep() 
there, so it's alright to keep testing it.  On the other hand, this may 
not work if prefix increment is non-atomic; the C specification implies 
that it is, but it makes no definite statement.

I'm not adverse to adding this sort of a mutex to this and the other 
places that initialize globals that are intended to be constants once set.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Andy Fiddaman <[email-masked]>
Date: Wed, 1 Sep 2004 21:26:36 +0000 (GMT)

On Wed, 1 Sep 2004, Mark Crispin wrote:
; There is *nothing* wrong with altering a global in threaded code as long as
; you accept that that alteration affects every thread.  This is a very useful
; attribute (beloved of those of us who wrote multi-forked code with shareable
; writable pages on TOPS-20).  It is something that must be thoroughly
; understood; but it is not something to fear.

As I understand it, changing a global variable in threaded code without a
lock can be a bad thing if another thread reads from the global while it
is being changed (that may not be the case on a real-life system). However,
as you say, changing driver parameters etc. would usually be done at the
initialisation stage.

; > but there are a couple of places where race conditions could occur. For
; > example, there is code at the start of unix_header() which populates a
; > global STRINGLIST if it hasn't been done before.
;
; This is an excellent example.  That global STRINGLIST is global for a reason.
; It should be a global constant, but it's complex enough to need code to build
; it.
;
; The only concern is whether another thread can run and join that code while
; the STRINGLIST is being built.  For this, you need to understand how your
; thread library works.  If thread switching only happens on I/O, you are safe
; since the only thing that code does is assignment and malloc().

I'm using Solaris so I can't rely on switching only happening on I/O -
I'm also developing on a 4-way system so potentially four kthreads can be
runnable and on a processor at once.

Ideally, I'd want the code to be as portable as possible so I'm keen to make
as few assumptions as I can.

; I'm not adverse to adding this sort of a mutex to this and the other places
; that initialize globals that are intended to be constants once set.

That would be ideal!

Thanks,

Andy

---

From: David Morris <[email-masked]>
Date: Wed, 1 Sep 2004 15:26:52 -0700 (PDT)

> that it is, but it makes no definite statement.

There is no way for a C implementation to insure this is atomic unless the
hardware has an instruction to perform the operation atomically. If you
know your hardware has the facility, they you need to verify that your
compiler won't optimize the value into a register, etc.

When the hardware doesn't provide an atomic operation, the compiler would
have to generate mutex or other locking code around the variable, very
expensive considering that there is no way for a compiler to know if a
global value is modifed by other operations.

With modern desk top and larger computers, fetching or storing a 32 bit
value should be atomic. A trick I use to guard one-time code is:

   if (!initialized) {
      getmutex()
      if (!initialized) { // real test guarded by mutex
        do heavy lifting
        initialized=TRUE
      }
      freemutex()
   }

This approach is a light weight test for the common case that
initialization is complete but is 100% accurate since the test is repeated
under protection of the mutex.

Another trick, where an atomic data object is built, if building
multiple copies of the object won't waste too much memory (or for
languages such as Java with garbage collection):

if (!object) {
   tempObject=makeTheObject()
   object = tempObject
}

In this case, the object may be initialized multiple times until one of
the threads actually stores the result.

Dave Morris

---

From: Vadim Zeitlin <[email-masked]>
Date: Fri, 3 Sep 2004 00:52:14 +0200 (Romance Daylight Time)

On Wed, 1 Sep 2004 15:26:52 -0700 (PDT) David Morris <[email-masked]> wrote:

DM> There is no way for a C implementation to insure this is atomic unless the
DM> hardware has an instruction to perform the operation atomically.

 This is true, but Win32 and pthreads both allow to do it so for most of
the modern platforms supported by cclient it is possible to do it.

DM> With modern desk top and larger computers, fetching or storing a 32 bit
DM> value should be atomic. A trick I use to guard one-time code is:
DM> 
DM>    if (!initialized) {
DM>       getmutex()
DM>       if (!initialized) { // real test guarded by mutex
DM>         do heavy lifting
DM>         initialized=TRUE
DM>       }
DM>       freemutex()
DM>    }
DM> 
DM> This approach is a light weight test for the common case that
DM> initialization is complete but is 100% accurate since the test is repeated
DM> under protection of the mutex.

 Although this mostly does work in practice please see an article in a
recent (August I believe) Dr Dobbs journal issue which explains why this
doesn't have to always work, i.e. is not portable. Basically, an optimizing
compiler is perfectly free to reorder the "do heavy lifting" and
"initialization=TRUE" lines which could result in using uninitialized
object. To ensure that it does work as expected, a memory barrier needs to
be inserted between these 2 lines -- and, of course, there is no portable
way to do this in C neither.

 Regards,
VZ