WiredTiger: Multithreading

| Version 3.1.0

Fork me on GitHub Join my user group
Multithreading

All WT_CONNECTION methods are thread safe, and WT_CONNECTION handles can be shared between threads. Applications typically open a single connection to each database, per process. Multi-threaded programs must wait for all other threads to exit before closing the WT_CONNECTION handle because that will implicitly close all other handles. Alternatively, a thread can set an event handler using WT_CONNECTION::open_session to be notified when a WT_CONNECTION close is in progress.

WT_SESSION and WT_CURSOR methods are not thread safe and WT_SESSION and WT_CURSOR handles cannot be accessed concurrently by multiple threads. Applications typically open one WT_SESSION handle for each thread accessing a database, and then one or more WT_CURSOR handles within the session.

WT_SESSION and WT_CURSOR methods may be accessed by different threads serially (for example, a pool of threads managed by the application with a set of shared session or cursor handles). There is no thread-local state in WiredTiger, and no built-in synchronization of session or cursor handles, either, so if multiple threads access a session or cursor handle, access must be serialized by the application.

WT_ASYNC_OP methods are not thread-safe, and must be accessed by only a single thread at a time. WT_ASYNC_OP methods may be accessed by different threads serially (and that is expected to happen when the asynchronous callback function runs).

# Code samples

The code below is taken from the complete example program ex_thread.c.

This is an example of a thread entry point. A new session is opened for the thread and used for all operations within that thread.

static WT_THREAD_RET
scan_thread(void *conn_arg)
{
WT_CONNECTION *conn;
WT_CURSOR *cursor;
WT_SESSION *session;
int ret;
const char *key, *value;
conn = conn_arg;
error_check(conn->open_session(conn, NULL, NULL, &session));
error_check(session->open_cursor(
session, "table:access", NULL, NULL, &cursor));
/* Show all records. */
while ((ret = cursor->next(cursor)) == 0) {
error_check(cursor->get_key(cursor, &key));
error_check(cursor->get_value(cursor, &value));
printf("Got record: %s : %s\n", key, value);
}
if (ret != WT_NOTFOUND)
fprintf(stderr,
"WT_CURSOR.next: %s\n", session->strerror(session, ret));
return (WT_THREAD_RET_VALUE);
}

Here is the main function that starts the threads. It opens a single connection, shared between the threads, and closes the connection after waiting for all of the threads to exit.

int
main(int argc, char *argv[])
{
WT_CONNECTION *conn;
WT_SESSION *session;
WT_CURSOR *cursor;
wt_thread_t threads[NUM_THREADS];
int i;
home = example_setup(argc, argv);
error_check(wiredtiger_open(home, NULL, "create", &conn));
error_check(conn->open_session(conn, NULL, NULL, &session));
error_check(session->create(session, "table:access",
"key_format=S,value_format=S"));
error_check(session->open_cursor(
session, "table:access", NULL, "overwrite", &cursor));
cursor->set_key(cursor, "key1");
cursor->set_value(cursor, "value1");
error_check(cursor->insert(cursor));
error_check(session->close(session, NULL));
for (i = 0; i < NUM_THREADS; i++)
error_check(
__wt_thread_create(NULL, &threads[i], scan_thread, conn));
for (i = 0; i < NUM_THREADS; i++)
error_check(__wt_thread_join(NULL, threads[i]));
error_check(conn->close(conn, NULL));
return (EXIT_SUCCESS);
}