OSSP var -- Variable Expansion var_rc_t, var_t, var_config_t, var_syntax_t, var_cb_value_t, var_cb_operation_t. var_create, var_destroy, var_config, var_unescape, var_expand, var_formatv, var_format, var_strerror. var_id. OSSP var is a flexible, full-featured and fast variable construct expansion library. It supports a configurable variable construct syntax very similar to the style found in many scripting languages (like "@"name, ${name"}", $(name")", etc.) and provides both simple scalar (${name"}") and array (${name"["index"]}") expansion, plus optionally one or more post-operations on the expanded value (${name":"op[":"op...]]"}"). The supported post-operations are length determination, case conversion, defaults, positive and negative alternatives, sub-strings, regular expression based substitutions, character translations, and padding. Additionally, a meta-construct plus arithmetic expressions for index and range calculations allow (even nested) iterations over array variable expansions (.."["..${name"[#+1]}".."]"..). The actual variable value lookup is performed through a callback function, so OSSP var can expand arbitrary values. A string expanded through OSSP var can consist of arbitrary text characters plus one or more of the following special syntax constructs which are expanded by OSSP var. Character with the octal value NNN (N: 0,...,7). Character with the hexadecimal value NN or the characters denoted by grouped hexadecimal numbers NNMM... (N, M: 0,...,9,["aA"],...,["fF"]). Tabulator, Carriage Return and Newline character. Ordinary character "\" and x. Contents of scalar variable name. Contents of array variable name at position index. For index full arithmetic expressions are allowed. Length of "$"name. "$"name, converted to all lower-case or all upper-case. If "$"name is not empty string and not undefined, then "$"name, else word (default value). If "$"name is empty string, then empty string, else word (positive alternative). If "$"name is not empty string, then empty string, else word (negative alternative). Substring of "$"name starting at position start with length characters. Substring of "$"name starting at position start and ending at position end (inclusive). "$"name after replacing characters matching pattern with string. By default, case-sensitive regular expression matching is performed and only the first occurance of pattern is replaced. Flag ""i"" switches to case insensitive matching; flag ""t"" switches to plain text pattern; flag ""g"" switches to replacements of all occurances; flag ""m"" switches to multi-line matching (That is, change ""^"" and ""$"" from matching the start or end of the string to matching the start or end of any line). "$"name after replacing all characters found in the ochars character class by the corresponding character in the nchars character class. "$"name after padding to width with string. Original contents of name is either left justified (flag ""l""), centered (flag ""c""), or right justified (flag ""r""). "$"name after passing it to an application-supplied function func. The optional argument arg is passed to the function, too. By default no such functions are defined. Repeat expansion of body as long as at least one array variable does not expand to the empty string (first variant) or exactly (end-start)/step times (second variant). In both cases the character ""#"" is expanded in "body" as the current loop index (0,... for first variant and start,...,end with stepping step for second variant). index of array variable lookups. For start, step and end, full arithmetic expressions are allowed. This loop construct can be nested, too. In this case an inner loop is fully repeated for each iteration of the outer loop. Additionally, arithmetic expressions are supported in both start, step, end and index parts of variable constructs in body. All the variable syntax constructs supported by OSSP var follow the same grammatical form. For completeness and reference reasons, the corresponding grammar is given in an extended BNF: input ::= ( TEXT | variable | INDEX_OPEN input INDEX_CLOSE (loop_limits)? )* variable ::= DELIM_INIT (name|expression) name ::= (NAME_CHARS)+ expression ::= DELIM_OPEN (name|variable)+ (INDEX_OPEN num_exp INDEX_CLOSE)? (':' command)* DELIM_CLOSE command ::= '-' (TEXT_EXP|variable)+ | '+' (TEXT_EXP|variable)+ | 'o' NUMBER ('-'|',') (NUMBER)? | '#' | '*' (TEXT_EXP|variable)+ | 's' '/' (TEXT_PATTERN)+ '/' (variable|TEXT_SUBST)* '/' ('m'|'g'|'i'|'t')* | 'y' '/' (variable|TEXT_SUBST)+ '/' (variable|TEXT_SUBST)* '/' | 'p' '/' NUMBER '/' (variable|TEXT_SUBST)* '/' ('r'|'l'|'c') | '%' (name|variable)+ ('(' (TEXT_ARGS)? ')')? | 'l' | 'u' num_exp ::= operand | operand ('+'|'-'|'*'|'/'|'%') num_exp operand ::= ('+'|'-')? NUMBER | INDEX_MARK | '(' num_exp ')' | variable loop_limits ::= DELIM_OPEN (num_exp)? ',' (num_exp)? (',' (num_exp)?)? DELIM_CLOSE NUMBER ::= ('0'|...|'9')+ TEXT_PATTERN::= (^('/'))+ TEXT_SUBST ::= (^(DELIM_INIT|'/'))+ TEXT_ARGS ::= (^(DELIM_INIT|')'))+ TEXT_EXP ::= (^(DELIM_INIT|DELIM_CLOSE|':'))+ TEXT ::= (^(DELIM_INIT|INDEX_OPEN|INDEX_CLOSE))+ DELIM_INIT ::= '$' DELIM_OPEN ::= '{' DELIM_CLOSE ::= '}' INDEX_OPEN ::= '[' INDEX_CLOSE ::= ']' INDEX_MARK ::= '#' NAME_CHARS ::= 'a'|...|'z'|'A'|...|'Z'|'0'|...|'9' Notice that the grammar definitions of DELIM_INIT, DELIM_OPEN, DELIM_CLOSE, INDEX_OPEN, INDEX_CLOSE, INDEX_MARK and NAME_CHARS correspond to the default syntax configuration only. They can be changed through the API (see var_syntax_t). The following is a detailed description of the OSSP var ISO-C language Application Programming Interface (API):

TYPES
The OSSP var API consists of the following ISO-C data types: This is an exported enumerated integer type describing the return code of all API functions. On success, every API functions returns "VAR_OK". On error, they return "VAR_ERR_XXX". For a list of all possible return codes see var.h. Their corresponding describing text can be determined with var_strerror. This is an opaque data type representing a variable expansion context. Only pointers to this abstract data type are used in the API. This is an exported enumerated integer type describing configuration parameters for var_config. Currently "VAR_CONFIG_SYNTAX" (for configuring the syntax via var_syntax_t) and "VAR_CONFIG_CB_VALUE" (for configuring the callback for value lookups via var_cb_value_t) are defined. This is an exported structural data type describing the variable construct syntax. It is passed to var_config on "VAR_CONFIG_SYNTAX" and consists of the following members (directly corresponding to the upper-case non-terminals in the grammar above): char escape; /* default: '\\' */ char delim_init; /* default: '$' */ char delim_open; /* default: '{' */ char delim_close; /* default: '}' */ char index_open; /* default: '[' */ char index_close; /* default: ']' */ char index_mark; /* default: '#' */ char *name_chars; /* default: "a-zA-Z0-9_" */ All members are single character constants, except for name_chars which is a character class listing all valid characters. As an abbreviation the construct "x"-"y" is supported which means all characters from x to y (both included) in the underlying character set. This is an exported function pointer type for variable value lookup functions. Such a callback function cb has to be of the following prototype: var_rc_t *cb(var_t *var, void *ctx, const char *var_ptr, size_t var_len, int var_idx, const char **val_ptr, size_t *val_len, size_t *val_size); This function will be called by var_expand internally whenever it has to resolve the contents of a variable. Its parameters are: This is the passed-through argument as passed to var_expand as the first argument. This can be used in the callback function to distinguish the expansion context or to resolve return codes, etc. This is the passed-through argument as passed to var_config on "VAR_CONFIG_CB_VALUE" as the forth argument. This can be used to provide an internal context to the callback function through var_expand. This is a pointer to the name of the variable whose contents var_expand wishes to resolve. Please note that the string is NOT necessarily terminated by a "NUL" ('"\0"') character. If the callback function needs it "NUL"-terminated, it has to copy the string into an a temporary buffer of its own and "NUL"-terminate it there. This is the length of the variable name at var_ptr. This determines which entry of an array variable to lookup. If the variable specification that led to the execution of the lookup function did not contain an index, zero (0) is provided by default as var_idx. If var_idx is less than zero, the callback should return the number of entries in the array variable. If var_idx is greater or equal zero, it should return the specified particular entry. It is up to the callback to decide what to return for an index not equal to zero if the underlying variable is a scalar. This is a pointer to the location where the callback function should store the pointer to the resolved value of the variable. This is a pointer to the location where the callback function should store the length of the resolved value of the variable. This is a pointer to the location where the callback function should store the size of the buffer that has been allocated to hold the value of the resolved variable. If no buffer has been allocated by the callback at all, because the variable uses some other means of storing the contents -- as in the case of getenv(3), where the system provides the buffer for the string --, this should be set to zero (0). In case a buffer size greater than zero is returned by the callback, var_expand will make use of that buffer internally if possible. It will also free(3) the buffer when it is not needed anymore, so it is important that it was previously allocated with malloc(3) by the callback. The return code of the lookup function cb is interpreted by var_expand according to the following convention: "VAR_OK" means success, that is, the contents of the variable has been resolved successfully and the val_ptr, val_len, and val_size variables have been filled with appropriate values. A return code "VAR_ERR_XXX" means that the resolving failed, such as a system error or lack of resources. In the latter two cases, the contents of val_ptr, val_len and val_size is assumed to be undefined. Hence, var_expand will not free(3) any possibly allocated buffers, the callback must take care of this itself. If a callback returns the special "VAR_ERR_UNDEFINED_VARIABLE" return code, the behavior of var_expand depends on the setting of its force_expand parameter. If force_expand has been set, var_expand will pass-through this error to the caller. If force_expand has not been set, var_expand will copy the expression that caused the lookup to fail verbatim into the output buffer so that an additional expanding pass may expand it later. If the callback returns an "VAR_ERR_XXX", var_expand will fail with this return code. If the cause for the error can not be denoted by an error code defined in var.h, callback implementors should use the error code "VAR_ERR_CALLBACK" (which is currently defined to -64). It is guaranteed that no error code smaller than "VAR_ERR_CALLBACK" is ever used by any OSSP var API function, so if the callback implementor wishes to distinguish between different reasons for failure, he subtract own callback return codes from this value, i.e., return ("VAR_ERR_CALLBACK" - n) (n >= 0) from the callback function. This is an exported function pointer type for variable value operation functions. Such a callback function cb has to be of the following prototype: var_rc_t *cb(var_t *var, void *ctx, const char *op_ptr, size_t op_len, const char *arg_ptr, size_t arg_len, const char *val_ptr, size_t val_len, const char **out_ptr, size_t *out_len, size_t *out_size); This function will be called by var_expand internally whenever a custom operation is used. Its parameters are: This is the passed-through argument as passed to var_expand as the first argument. This can be used in the callback function to distinguish the expansion context or to resolve return codes, etc. This is the passed-through argument as passed to var_config on "VAR_CONFIG_CB_OPERATION" as the forth argument. This can be used to provide an internal context to the callback function through var_expand. This is a pointer to the name of the operation which var_expand wishes to perform. Please note that the string is NOT necessarily terminated by a "NUL" ('"\0"') character. If the callback function needs it "NUL"-terminated, it has to copy the string into an a temporary buffer of its own and "NUL"-terminate it there. This is the length of the variable name at op_ptr. This is a pointer to the optional argument string to the operation. If no argument string or an empty argument string was supplied this is "NULL". This is the length of the arg_ptr. This is a pointer to the value of the variable which the operation wants to adjust. This is the length of the val_ptr. This is a pointer to the location where the callback function should store the pointer to the adjusted value. This is a pointer to the location where the callback function should store the length of the adjusted value of the variable. This is a pointer to the location where the callback function should store the size of the buffer that has been allocated to hold the adjusted value of the variable. If no buffer has been allocated by the callback at all, because the variable uses some other means of storing the contents, this should be set to zero (0). In case a buffer size greater than zero is returned by the callback, var_expand will make use of that buffer internally if possible. It will also free(3) the buffer when it is not needed anymore, so it is important that it was previously allocated with malloc(3) by the callback.

FUNCTIONS
The OSSP var API consists of the following ISO-C functions: Create a new variable expansion context and store it into var. Destroy the variable expansion context var. Configure the variable expansion context var. The variable argument list depends on the mode identifier: This overrides the syntax configuration in var with the one provided in syntax. The complete structure contents is copied, so the caller is allowed to immediately destroy syntax after the var_config call. The default is the contents as shown above under the type description of var_syntax_t. This overrides the syntax configuration in var with the one provided The default is "NULL" for cb and ctx. At least "NULL" for cb is not valid for proper operation of var_expand, so the caller has to configure the callback before variable expansions can be successfully performed. This expands escape sequences found in the input buffer src_ptr/src_len. The dst_ptr/dst_len point to a output buffer, into which the expanded data is copied if processing is successful. The size of this buffer must be at least src_len+1 characters. The reason is that var_unescape always adds a terminating "NUL" ('"\0"') character at the end of the output buffer, so that you can use the result comfortably with other C library routines. The supplied dst_ptr either has to be point to a pre-allocated buffer or is allowed to point to src_ptr (because the unescaping operation is guarrantied to either keep the size or reduce the size of the input). The parameter all is a boolean flag that modifies the behavior of var_unescape. If is set to true (any value except zero), var_unescape will expand any escape sequences it sees, even those that it does not know about. This means that ""\1"" will become "1", even though ""\1"" has no special meaning to var_unescape. If all is set to false (the value zero), such escape sequences will be copied verbatim to the output buffer. The quoted pairs supported by var_unescape are ""\t"" (tabulator), ""\r"" (carriage return), ""\n"" (line feed), ""\NNN"" (octal value), ""\xNN"" (hexadecimal value), and ""\x{NNMM..}"" (grouped hexadecimal values). This is the heart of OSSP var. It expands all syntax constructs in src_ptr/src_len and stores them in an allocated buffer returned in dst_ptr/dst_len. The output buffer dst_ptr/dst_len is allocated by var_expand using the system call malloc(3), thus it is the caller's responsibility to free(3) that buffer once it is no longer used anymore. The output buffer for convenience reasons is always "NUL"-terminated by var_expand, but this "NUL" character is not counted for dst_len. The dst_len pointer can be specified as "NULL" if you are not interested in the output buffer length. The force_expand flag determines how var_expand deals with undefined variables (indicated by the callback function through the return code "VAR_ERR_UNDEFINED_VARIABLE"). If it is set to true (any value except zero), var_expand will fail with error code "VAR_ERR_UNDEFINED_VARIABLE" whenever an undefined variable is encountered. That is, it just passes-through the return code of the callback function. If set to false (value zero), var_expand will copy the expression it failed to expand verbatim into the output buffer, in order to enable you to go over the buffer with an additional pass. Generally, if you do not plan to use multi-pass expansion, you should set force_expand to true in order to make sure no unexpanded variable constructs are left over in the buffer. If var_expand fails with an error, dst_ptr will point to src_ptr and dst_len will contain the number of characters that have been consumed from src_ptr before the error occurred. In other words, if an error occurs, dst_ptr/dst_len point to the last parsing location in src_ptr/src_len before the error occurred. The only exceptions for this error semantics are: on "VAR_ERR_INVALID_ARGUMENT" and "VAR_ERR_OUT_OF_MEMORY" errors, dst_ptr and dst_len are undefined. This is a high-level function on top of var_expand which expands simple printf(3)-style constructs before expanding the complex variable constructs. So, this is something of a combination between sprintf(3) and var_expand. It expands simple "%s" (string, type ""char *""), "%d" (integer number, type ""int"") and "%c" (character, type ""int"") constructs in fmt. The values are taken from the variable argument vector ap. After this expansion the result is passed through var_expand by passing through the var, dst_ptr and force_expand arguments. The final result is a malloc(3)'ed buffer provided in dst_ptr which the caller has to free(3) later. This is just a wrapper around var_formatv which translates the variable argument list into "va_list". This can be used to map any var_rc_t return codes (as returned by all the OSSP var API functions) into a clear-text message describing the reason for failure in prose. Please note that errors coming from the callback, such as "VAR_ERR_CALLBACK" and those based on it, cannot be mapped and will yield the message ""unknown error"".

VARIABLES
For maximum power and flexibility, you usually want to combine var_unescape and var_expand. That is, you will want to use var_unescape to turn all escape sequences into their real representation before you call var_expand for expanding variable constructs. This way the user can safely use specials like ""\n"" or ""\t"" throughout the template and achieve the desired effect. These escape sequences are particularly useful if search-and-replace or transpose actions are performed on variables before they are expanded. Be sure, though, to make the first var_unescape pass with the all flag set to false, or the routine will also expand escape sequences like ""\1"", which might have a special meaning (regular expression back-references) in the var_expand pass to follow. Once, all known escape sequences are expanded, expand the variables with var_expand. After that, you will want to have a second pass with var_unescape and the flag all set to true, to make sure all remaining escape sequences are expanded. Also, the var_expand pass might have introduced now quoted pairs into the output text, which you need to expand to get the desired effect. OSSP var can be optionally built with support for exception handling via OSSP ex (see http://www.ossp.org/pkg/lib/ex/). For this it has to be configured with the GNU Autoconf option "--with-ex". The difference then is that the OSSP var API functions throw exceptions instead of returning "VAR_ERR_XXX" return codes. The thrown exceptions can be identified as OSSP var exceptions by checking the exception attribute "ex_class". It is the OSSP var API symbol "var_id" for all OSSP var exceptions. The "ex_object" attribute is always "NULL". The "ex_value" attribute is the "var_rc_t" which forced the throwing of the exception. Exception throwing can be suppressed with "ex_shield" only. The following simple but complete program illustrates the full usage of OSSP var. It accepts a single argument on the command line and expands this in three steps (unescaping known escape sequences, expanding variable constructs, unescaping new and unknown escape sequences). The value lookup callback uses the process environment to resolve variables. #include <stdio.h> #include <stdlib.h> #include <string.h> #include "var.h" static var_rc_t lookup( var_t *var, void *ctx, const char *var_ptr, size_t var_len, int var_idx, const char **val_ptr, size_t *val_len, size_t *val_size) { char tmp[256]; if (var_idx != 0) return VAR_ERR_ARRAY_LOOKUPS_ARE_UNSUPPORTED; if (var_len > sizeof(tmp) - 1) return VAR_ERR_OUT_OF_MEMORY; memcpy(tmp, var_ptr, var_len); tmp[var_len] = '\0'; if ((*val_ptr = getenv(tmp)) == NULL) return VAR_ERR_UNDEFINED_VARIABLE; *val_len = strlen(*val_ptr); *val_size = 0; return VAR_OK; } static void die(const char *context, var_t *var, var_rc_t rc) { char *error; var_strerror(var, rc, &error); fprintf(stderr, "ERROR: %s: %s (%d)\n", context, error, rc); exit(1); } int main(int argc, char *argv[]) { var_t *var; var_rc_t rc; char *src_ptr; char *dst_ptr; size_t src_len; size_t dst_len; var_syntax_t syntax = { '\\', '$', '{', '}', '[', ']', '#', "a-zA-Z0-9_" }; /* command line handling */ if (argc != 2) die("command line", NULL, VAR_ERR_INVALID_ARGUMENT); src_ptr = argv[1]; src_len = strlen(src_ptr); fprintf(stdout, "input: \"%s\"\n", src_ptr); /* establish variable expansion context */ if ((rc = var_create(&var)) != VAR_OK) die("create context", NULL, rc); if ((rc = var_config(var, VAR_CONFIG_SYNTAX, &syntax)) != VAR_OK) die("configure syntax", var, rc); if ((rc = var_config(var, VAR_CONFIG_CB_VALUE, lookup, NULL)) != VAR_OK) die("configure callback", var, rc); /* unescape known escape sequences (in place) */ if ((rc = var_unescape(var, src_ptr, src_len, src_ptr, src_len+1, 0)) != VAR_OK) die("unescape known escape sequences", var, rc); src_len = strlen(src_ptr); fprintf(stdout, "unescaped: \"%s\"\n", src_ptr); /* expand variable constructs (force expansion) */ if ((rc = var_expand(var, src_ptr, src_len, &dst_ptr, &dst_len, 1)) != VAR_OK) { if (rc != VAR_ERR_INVALID_ARGUMENT && rc != VAR_ERR_OUT_OF_MEMORY) { fprintf(stdout, "parsing: \"%s\"\n", dst_ptr); fprintf(stdout, " %*s\n", dst_len, "^"); } die("variable expansion", var, rc); } fprintf(stdout, "expanded: \"%s\"\n", dst_ptr); /* unescape new and unknown escape sequences (in place) */ if ((rc = var_unescape(var, dst_ptr, dst_len, dst_ptr, dst_len+1, 1)) != VAR_OK) die("unescape new and unknown escape sequences", var, rc); fprintf(stdout, "output: \"%s\"\n", dst_ptr); free(dst_ptr); /* destroy variable expansion context */ if ((rc = var_destroy(var)) != VAR_OK) die("destroy context", var, rc); return 0; } Copy & paste the source code it into var_play.c (or use the version already shipped with the OSSP var source distribution), compile it with $ cc `var-config --cflags` -o var_play var_play.c `var-config --ldflags --libs` and use it to play with the various OSSP var variable expansion possibilities. The following are a few sample use cases of OSSP var variable expansions. They all assume the default syntax configuration and the following variable definitions: "$foo=foo" (a scalar), "$bar=<bar1,bar2,bar3,>" (an array), "$baz=<baz1,baz2,baz3,>" (another array), "$quux=quux" (another scalar), "$name=<foo,bar,baz,quux>" (another scalar) and "$empty=""" (another scalar). Input Output ----------------------------- -------------- $foo foo ${foo} foo ${bar[0]} bar1 ${${name[1]}[0]} bar1 ${foo:u:y/O/U/:s/(.*)/<\1>/} <FUU> ${foo:u:y/O/U/:s/(.*)/<\1>/} <FUU> ${empty:-foo} foo ${foo:+yes}${foo:*no} yes ${empty:+yes}${empty:*no} no ${foo:p/6/./l} foo... ${foo:p/6/./r} ...foo [${bar[#]}${bar[#+1]:+,}] bar1,bar2,bar3 [${bar[#-1]:+,}${bar[#]}] bar1,bar2,bar3 [${bar[#]}]{2,1,3} bar2bar3 [${bar[#]}]{1,2,3} bar1bar3 [${foo[#]}[${bar[#]}]]{1,,2} foo1bar1bar2bar3foo2bar1bar2bar3 pcre(3), regex(7), OSSP val (Value Access). OSSP var was initially written by Peter Simons <simons@crypt.to> in November 2001 under contract with the OSSP sponsor Cable & Wireless. Its API and internal code structure was revamped in February 2002 by Ralf S. Engelschall <rse@engelschall.com> to fully conform to the OSSP library standards. Before its initial public release, Ralf S. Engelschall in March 2002 finally added support for custom operations, the formatting functionality, optional multi-line matching, etc.