Subject: Re: On multithreaded cclient

From: "Alex Shvedov" <[email-masked]>
Date: Thu, 10 Dec 1998 16:35:23 -0500

Sorry, Mark, but you still seem not to be 100% right.

1. gethostbyname is not thread-safe (experimental result), and - as opposed
to strtok - I don't know its thread-safe equivalent (if any). In AIX there
are none; it's a statement, not a question.
So, without some non-context free changes library's code IS NOT thread safe,
at least in AIX case, and this is also a statment, not a question.

2. Socket read/writes in tcp_unix.c ARE NOT thread-safe; threads confuse
each other sockets if IO calls come at the same time (I forgot which global
they use to talk to each other, but confusion is an experimental result,
mutexed version works). This is also a statement, not a question.

3. Call-backs are obviously not thread-safe. And this appeared to be the
biggest problem, by the way, because mutexed serializing would be more than
idiotic here.

Why everyone talks about poor strtok, I wonder? I had no problems with it,
and strtok->strtok_r is a *context-free* substitution anyway.

Don't understand *BEGIN FLAME*-*END FLAME* piece at all.
Yes, some good guy made strtok not safe in 3000 BC. Yes, they invented a
safe substitute only 2000 years later (though in AIX case it's wrong that
"thread-safe strtok() alternative that requires you to keep track of which
thread you are").
Yes, all this is bad.
Do I like POSIX? No.
Was it me who invented process-wide synchronous signals, just to give an
example? No.

Well, and what about it?

>On Thu, 10 Dec 1998 11:04:05 -0500, Alex Shvedov wrote:
>> It's a long story to describe them in detail here and now, but the test
is
>> really simple: create two threads that talk to the same server (or to two
>> different servers) and try to open these two streams. If you'll manage to
do
>> this, I'll eat my hat (lucky not to have one).
>
>As far as I know, the problem is that you need a C library with a
thread-safe
>version of strtok().  If your C library's strtok() isn't thread-safe, you
need
>to replace it with one that is.  strtok() is used throughout c-client,
>particularly in the IMAP code.
>
>*BEGIN FLAME*
>Some C libraries have a thread-safe strtok() alternative that requires you
to
>keep track of which thread you are for the global used by strtok().  It's a
>great example of people who don't understand the concept of good software
>engineering; if the application can keep track of per-thread context, a
>library routine can do so as well.  Since c-client is itself a library,
it's
>no more work for the C library to do it than c-client.  Since the problem
is
>in the C library, the C library should fix it rather than making every
program
>in the world do so.
>
>It isn't as if these are new concepts either.  Perhaps if UNIX geeks paid
more
>attention to what other operating systems were doing 10-20 years ago
instead
>of trying to kill them all off, we wouldn't be reinventing all this stuff
>today.
>*END FLAME*
>
>Even with this problem, c-client is definitely thread-safe as long as a
thread
>can not be scheduled while another thread is inside c-client.  I have an
>application that does this.
>

---

From: Mark Crispin <[email-masked]>
Date: Thu, 10 Dec 1998 21:56:47 -0800 (PST)

> are none; it's a statement, not a question.

It sounds to me like you should be complaining to IBM to fix their C library.
There's no reason why gethostbyname() shouldn't be thread-safe.  If it isn't,
that means they're using the old BSD gethostnamadr.c which does all its work
in globals.

> mutexed version works).

Once again, are you sure that this isn't an AIX specific problem?

> 3. Call-backs are obviously not thread-safe.

Why it it "obvious"?  The callback arguments contain enough information; it's
up to you to write the callback routines correctly.

> and strtok->strtok_r is a *context-free* substitution anyway.

Not all systems have strtok_r().  Most of the systems that I develop on do
not.  In any case, strtok_r() is not a compatible interface; it requires the
main program to keep track of the pointer.  That should be maintained by the
thread mechanism in per-thread context.

Quite frankly, if the cost of making c-client thread-safe on AIX (or any other
system) turns out to be breaking c-client for every system and application
that doesn't use threads, it isn't going to happen.  Any proposed thread
solutions MUST be compatible with non-threading, and furthermore MUST NOT
introduce untoward burdens in the non-threading case.

What this means is that the burden is on those who want threads to come up
with compatible interfaces.  c-client has numerous examples of how to do this.
Note for example that the general OS environment is BSD-like; on SVR4 systems,
the BSD environment is simulated through emulation functions and macros.

---

From: Karsten Ballueder <[email-masked]>
Date: Fri, 11 Dec 1998 10:58:00 +0000 (GMT)

> that means they're using the old BSD gethostnamadr.c which does all its work

Neither is Solars's gethostbyname() functino thread safe, you need the 
_r version for it.

 > Once again, are you sure that this isn't an AIX specific problem?

On Solaris, they are thread safe.

I believe actually checking for thread safety and writing code that works
around such problems would be more helpful than ranting about some
OS's bad design.

Fortunately these problems are realtively easy to solve. Don't know
about read/write though.

 > up to you to write the callback routines correctly.

I agree with this, the callback happens in the thread of that folder,
it doesn't cross threads. So the callback can easily send a signal to
the main thread in some thread-safe way.

 > > and strtok->strtok_r is a *context-free* substitution anyway.

So is gethostbyname() / gethostbyname_r().

 > thread mechanism in per-thread context.

How about jsut replacing it with strsep(). It's thread-safe and
context-free and I have an implementation of it that we could use on
systems that don't support it. Of course, it depends on how often you
use strtok(), but replacing strtok(0 with strsep() is pretty
straightforward. 
 
 > introduce untoward burdens in the non-threading case.

100% agreed. I don't think that adding thread-safety would lead to any 
serious problems. In the worst case it would bring some mutex tests
which can be made conditional on some preprocessor define.

 > with compatible interfaces.  c-client has numerous examples of how to do this. 

What do you mean by compatible interfaces? If Alex has some patches
which make the existing c-client library thread safe by introducing
mutex to control access to critical code, that doesn't mean any new
interfaces. I think it's worth having a look at it.
-- 
 Karsten Ballüder http://www.phy.hw.ac.uk/~karsten/ mailto:[email-masked]
 Physics Department, Heriot-Watt University, Edinburgh, EH14 4AS, Scotland
                     Tel. +44-(0)-131-4513068  Fax. +44-(0)-131-4513136
           "In a world without fences, who needs Gates?"