WiredTiger: Encryptors

| Version 3.1.0

Fork me on GitHub Join my user group
Encryptors

# Overview of Encryption in WiredTiger

Custom encryption engines may be used to extend WiredTiger. WiredTiger does not currently offer builtin support for any particular encryption algorithm. Example encryption code is provided to demonstrate how encryption extensions are created.

Warning
  The encryption infrastructure included in WiredTiger, when used with a strong encryption algorithm, is intended to protect data stored in files (that is, encryption at rest). The table content (keys, values), the metadata pertaining to data (table, index, column names, and other configuration information) as well as the database log files are encrypted on disk. Decryption occurs when the data is read into memory; thus an attacker having the ability to directly read system memory will have access to unencrypted data. Many systems may also page memory to a backing disk under load. Access to any such paging or swap devices must be considered when planning the security of a system.

The encryption extension must be loaded in the wiredtiger_open call. See Extending WiredTiger for details on how extensions are loaded. Also, encryption is specified using `encryption=` in the configuration for the wiredtiger_open call. This configuration establishes the encryption algorithm and keys to be used for database log files and a subset of the WiredTiger metadata files. By default, this same encryption is also used for all data files. We call this the system encryption.

It is also possible to use different encryption options when individual data files are first created, using the `encryption=` configuration in the WT_SESSION::create call. Such options override the default (system) encryption that was indicated in the wiredtiger_open call for the individual data file. It is possible to turn encryption off for individual files, to use a different encryptor, or to specify a different `keyid`.

Overriding the system encryption for a table does not override the system encryption for indices on that table, nor does it override the system encryption for column groups specified on that table. Encryption for column groups and indices must specified when they are created, if they are to be different than the system encryption.

It is an error to specify encryption in a WT_SESSION::create call when it was not specified in the wiredtiger_open call. This prevents accidental exposure of the file's data in log files, which would be written in the clear in such a scenario.

Warning
  When using separate keys for individual data files or tables, the key used for the system encryption continues to have fundamental importance. The database log, protected by the system encryption, contains a shared stream of changes to all data files. Thus, if the system key is exposed, even when per-file keys are not exposed, an attacker can read database log files, and hence has access to data in individual files.

# Encryption keyid and secretkey

Two parameters, `keyid` and `secretkey`, may be specified when configuring encryption for wiredtiger_open to allow the possibility of varying the algorithm according to different keys.

The configuration parameter `encryption=(keyid=identifier)` may be used in wiredtiger_open or WT_SESSION::create calls. This is intended to reference a key stored using a Key Management Solution (KMS). The `keyid` given to wiredtiger_open is stored in the clear in WiredTiger configuration files; it should never contain sensitive information. As an example, with a `keyid` of `"customerABC"`, the encryptor would consult the KMS to return a key previously stored for `"customerABC"`. The encryptor will use the returned key when applying the encryption. To effectively use the `keyid`, a custom encryptor must implement the WT_ENCRYPTOR::customize callback. It is during `customize` that the encryptor has an opportunity to use the `keyid` to fetch the actual key. The `customize` function is called on the first use of a `keyid`, and the same customized encryptor will be used with each use of the same `keyid`.

The configuration parameter `encryption=(secretkey)` is used only in the wiredtiger_open call. The value of the secretkey is never stored on disk in any form, so it must always be provided when WiredTiger is reopened (again, with the wiredtiger_open call). The secretkey is available to the encryptor during the WT_ENCRYPTOR::customize callback, during which the encryptor may be customized to keep the secretkey or a transformation of it for use during the WT_ENCRYPTOR::encrypt and WT_ENCRYPTOR::decrypt callbacks.

If a `secretkey` is used, it must be provided using the `-E` option when using the `wt` utility. Specifying `keyid` is not needed with the `wt` utility, as the `keyid` is stored in the clear on disk by WiredTiger. Any additional `keyid` values needed to decrypt data files are stored in WiredTiger metadata using the system encryptor.

# Custom encryption engines

WiredTiger may be extended by adding custom encryption engines that we call encryptors. Custom encryptors must be coded in the C language. Once packaged, they can be used in any language.

See Extending WiredTiger for general details on extending WiredTiger, and see WT_ENCRYPTOR for the encryptor interface.

Custom encryptors are registered by calling WT_CONNECTION::add_encryptor, this creates an encryptor name that may be referenced using the `encryption=(name=...` configuration string in the wiredtiger_open or WT_SESSION::create call.

# Encryption examples

There are two kinds of example code with overlapping functionality. A simple, self contained encryption example is in ex_encrypt.c. This example includes a small encryptor that rotates letters in the alphabet by a fixed amount, based on the value of `keyid`. This example also shows how encryption is configured within an application. The second set of examples are in `ext/encryptors`. These are encryptors only (no application level code), showing how encryptors might be packaged in a loadable shared library. nop_encrypt.c merely copies its input to its output. rotn_encrypt.c is an extended version of the example that rotates the alphabet. It adds a twist in that a `secretkey` can be specified, changing the rotation per letter. The Python test suite uses the rotn encryptor to help test the encryption framework.

Please note that these samples are for demonstration use only. They do not provide any security.