.TH "md_doc_help_elektra-architecture" 3elektra "Sun May 29 2016" "Version 0.8.14" "Elektra" \" -*- nroff -*-
.ad l
.nh
.SH NAME
md_doc_help_elektra-architecture \- elektra-architecture(7) -- architecture of elektra 
In this document we start to explain the implementation of Elektra\&. There are several follow-up documents which explain all details of:
.PP
.IP "\(bu" 2
\fBerror handling\fP,
.IP "\(bu" 2
\fBdata structures\fP, and
.IP "\(bu" 2
finally the \fBcore algorithm\fP\&.
.PP
.PP
We discuss problems and the solution space so that the reader can understand the rationale of how problems were solved\&.
.PP
To help readers to understand the algorithm that glues together the plugins, we first describe some details of the \fBdata structures\fP\&. Full knowledge of the \fBalgorithm\fP is not presumed to be able to develop most plugins (with the exception of \fBthe resolver\fP)\&.
.PP
Further important concepts are explained in:
.PP
.IP "\(bu" 2
\fBbootstrapping\fP
.IP "\(bu" 2
\fBgranularity\fP
.IP "\(bu" 2
\fBsync-flag\fP
.PP
.PP
The aim of the Elektra Project is to design and implement a powerful API for configuration\&. When the project started, we assumed that this goal was easy to achieve, but dealing with the semantics turned out to be a difficult problem\&. For the implementation, an ambitious solution is required because of the necessary modularity to implement flexible backends as introduced in Elektra\&. But also the design of a good API has proved to be much more difficult than expected\&.
.PP
.SS "Changes in the APIs"
.PP
From Elektra 0\&.7 to Elektra 0\&.8, we changed the API of Elektra as little as possible\&. It should be mentioned that \fCKeySet\fP is now always sorted by name\&. The function \fCksSort()\fP is now depreciated and was removed\&. The handling of removed keys was modified\&. Additionally, the API for metadata has fundamentally changed, but the old interface still works\&. These changes will be described in \fBimplementation of meta data\fP\&. However, the implementation of Elektra changed radically as discussed in \fBalgorithm\fP\&.
.PP
.SS "API Design"
.PP
API Design presents a critical craft every programmer should be aware of\&. We will shortly present some of the main design issues that matter and show how Elektra has solved them\&.
.PP
A design goal is to detect errors early\&. As easy as it sounds, as difficult it is to actually achieve this goal\&. Elektra tries to avoid the problem by checking data being inserted into \fCKey\fP and \fCKeySet\fP\&. Elektra catches many errors like invalid key names soon\&. Elektra allows plugins to check the configuration before it is written into the key database so that problematic values are never stored\&.
.PP
''Hard to use it wrong'' tends to be a more important design objective than ''Easy to use it right''\&. Searching for a stupid bug costs more time than falling into some standard traps which are explained in documentation\&. In Elektra, the data structures are robust and some efforts were taken to make misuse unlikely\&.
.PP
Another fundamental principle is that the API must hide implementation details and should not be optimised towards speed\&. In Elektra, the actual process of making configuration permanent is completely hidden\&.
.PP
''Off-by-one confusion'' is a topic of its own\&. The best is to stick to the conventions the programming language gives\&. For returning sizes of strings, it must be clear whether a terminating `'\\0'` is included or not\&. All such decisions must be consistent\&. In Elektra the terminating null is always included in the size\&.
.PP
The interface must be as small as possible to tackle problems addressed by the library\&. Internal and external APIs must be separated\&. Internal APIs in libraries shall be declared as \fCstatic\fP to prevent its export\&. In Elektra, internal names start with \fCelektra\fP opposed to the external names starting with \fCkey\fP, \fCks\fP or \fCkdb\fP\&.
.PP
Elektra always passes user context pointers, but never passes or receives a full data structure by value\&. It is impossible to be ABI{We will read more about ABI in {ABI}\&.} compatible otherwise\&. Elektra is restrictive in what it returns (strong postconditions), but as liberal as possible for what comes in (preconditions are avoided where possible)\&. In Elektra even null pointers are accepted for any argument\&.
.PP
''Free everything you allocate'' is a difficult topic in some cases\&. If Elektra cannot free space or other resources after every call, it provides a \fCclose()\fP function\&. Everything will be freed\&. The tool \fBValgrind\fP with \fBMemcheck\fP helps us locate problems\&. The whole test suite runs without any memory problems\&. The user is responsible for deleting all created \fCKey\fP and \fCKeySet\fP objects and closing the \fCKDB\fP handle\&.
.PP
As a final statement, we note that the UNIX philosophy should always be considered: ''Do only one thing, but do it in the best way\&. Write it that way that programs work together well\&.''
.PP
.SS "Modules"
.PP
Elektra's core can be compiled with a C compiler conforming to the ISO/IEC 9899:1999 standard:
.PP
.IP "\(bu" 2
One line comments,
.IP "\(bu" 2
inline functions,
.IP "\(bu" 2
\fCsnprintf()\fP
.IP "\(bu" 2
inttypes\&.h and
.IP "\(bu" 2
variable declaration at any place
.PP
.PP
are used in addition to what is already defined in the standard ISO/IEC 9899:1990, called \fBC99\fP in the following text\&. Functions not conforming to C99 are considered to be not portable enough for Elektra and are separated into plugins\&. But there is one notable exception: it must be the core's task to load plugins\&. Unfortunately, C99 does not know anything about modules\&. \fBPOSIX\fP (Portable Operating System Interface) provides \fCdlopen()\fP, but other operating systems have dissimilar APIs for that purpose\&. They sometimes behave differently, use other names for the libraries and have incompatible error reporting systems\&. Because of these requirements Elektra provides a small internal API to load such modules independently from the operating system\&. This API also hides the fact that modules must be loaded dynamically if they are not available statically\&.
.PP
Plugins are usually realised with modules\&. Modules and libraries are technically the same in most systems\&. (One exception is Mac OS X\&.) After the module is loaded, the special function plugin factory is searched for\&. This function returns a new plugin\&. With the plugin factory the actual plugins are created\&.
.PP
.SS "Static loading"
.PP
For the static loading of modules, the modules must be built-in\&. With \fCdlopen(const\fP \fCchar*\fP \fCfile)\fP POSIX provides a solution to look up such symbols by passing a null pointer for the parameter \fCfile\fP\&. Non-POSIX operating systems may not support this kind of static loading\&. Therefore, Elektra provides a C99 conforming solution for that problem: a data structure stores the pointers to the plugin factory of every plugin\&. The build system generates the source file of this data structure because it depends on built-in plugins as shown in Figure~fig:architecture}\&.
.PP
Elektra distinguishes internally between modules and plugins\&. Several plugins can be created out of a single module\&. During the creation process of the plugin, dynamic information -- like the configuration or the data handle -- is added\&.
.PP
.SS "API"
.PP
The API of [libloader\\lstinline{libloader}]{libloader} consists of the following functions:
.PP
Interface of Module System: 
.PP
.nf
    elektraModulesInit (KeySet *modules, Key *error); elektraPluginFactory
    elektraModulesLoad (KeySet *modules,
                    const char *name, Key *error);
    int elektraModulesClose (KeySet *modules, Key *error); \end{lstlisting}

.fi
.PP
.PP
\fCelektraModulesInit()\fP initialises the module cache and calls necessary operating system facilities if needed\&. {elektraModulesLoad()} does the main work by either returning a pointer to the plugin factory from cache or loading it from the operating system\&. The plugin factory creates plugins that do not have references to the module anymore\&. \fCelektraModulesClose()\fP cleans up the cache and finalises all connections with the operating system\&.
.PP
Not every plugin is loaded by \fClibloader\fP\&. For example, the \fIversion plugin\fP, which exports version information, is implemented internally\&.
.PP
.SS "Mount Point Configuration"
.PP
\fCkdb mount\fP creates a \fBmount point configuration\fP as shown in the example below\&. \fCfstab\fP is a unique name within the mount point configuration provided by the administrator\&.
.PP
Example for a mount point configuration: 
.PP
.nf
    system/elektra/mountpoints system/elektra/mountpoints/fstab
    system/elektra/mountpoints/fstab/config
    system/elektra/mountpoints/fstab/config/path=fstab
    system/elektra/mountpoints/fstab/config/struct=list FStab
    system/elektra/mountpoints/fstab/config/struct/FStab
    system/elektra/mountpoints/fstab/config/struct/FStab/device
    system/elektra/mountpoints/fstab/config/struct/FStab/dumpfreq
    system/elektra/mountpoints/fstab/config/struct/FStab/mpoint
    system/elektra/mountpoints/fstab/config/struct/FStab/options
    system/elektra/mountpoints/fstab/config/struct/FStab/passno
    system/elektra/mountpoints/fstab/config/struct/FStab/type
    system/elektra/mountpoints/fstab/errorplugins
    system/elektra/mountpoints/fstab/errorplugins/#5#resolver#resolver#
    system/elektra/mountpoints/fstab/getplugins
    system/elektra/mountpoints/fstab/getplugins/#0#resolver
    system/elektra/mountpoints/fstab/getplugins/#5#fstab#fstab#
    system/elektra/mountpoints/fstab/mountpoint /fstab
    system/elektra/mountpoints/fstab/setplugins
    system/elektra/mountpoints/fstab/setplugins/#0#resolver
    system/elektra/mountpoints/fstab/setplugins/#1#struct#struct#
    system/elektra/mountpoints/fstab/setplugins/#2#type#type#
    system/elektra/mountpoints/fstab/setplugins/#3#path#path#
    system/elektra/mountpoints/fstab/setplugins/#3#path#path#/config
    system/elektra/mountpoints/fstab/setplugins/#3#path#path#/config/path/allow=proc tmpfs none
    system/elektra/mountpoints/fstab/setplugins/#5#fstab
    system/elektra/mountpoints/fstab/setplugins/#7#resolver \end{lstlisting}

.fi
.PP
.PP
Let us look at the subkeys below the key \fCsystem/elektra/mountpoints/fstab\fP:
.PP
.IP "\(bu" 2
\fBconfig\fP: Everything below {config} is the system's configuration of the backend\&. Every plugin within the backend will find this configuration directly below {system/} in its {plugin configuration}\&. For example, 
.PP
.nf
    system/elektra/mountpoints/fstab/config/struct/FStab/mpoint

.fi
.PP

.PP
.PP
will be translated to 
.PP
.nf
    system/struct/FStab/mpoint

.fi
.PP
.PP
and inserted into the plugin configuration for all plugins in the \fCfstab\fP backend\&.
.PP
It is the place where configuration can be provided for every plugin of a backend\&. The contract checker deduces this configuration to satisfy the contract for a plugin\&. Fstab, for example, claims in a contract that it needs ''struct''\&. But the struct plugin needs a configuration to work properly\&. Fstab will provide this configuration\&. The {contract checker} writes out the configuration looking like the one in this example\&.
.PP
.IP "\(bu" 2
\fBconfig/path\fP: is a common setting needed by the resolver plugin\&. It is the relative path to a filename that is used by this backend\&. On UNIX systems, the resolver would determine the name \fC/etc/fstab\fP for system configuration\&.
.IP "\(bu" 2
\fBmountpoint\fP: is a key that represents the mount point\&. Its value is the location where the backend is mounted\&. If a mount point has an entry for both the user and the system hierarchy, it is called {cascading mount point}\&. A cascading mount point differs from two separate mount points because internally only one backend is created\&. In the example, the mount point \fC/fstab\fP means that the backend handles both \fCuser/fstab\fP and \fCsystem/fstab\fP\&. If the mount point is \fC/\fP, the backend will be mounted to all namespaces except \fCspec\fP, including both \fCuser\fP and \fCsystem\fP\&.
.IP "\(bu" 2
\fBerrorplugins\fP: presents a list of all plugins to be executed in the error case of \fC\fBkdbSet()\fP\fP which will be explained in {error situation}\&.
.IP "\(bu" 2
\fBgetplugins\fP: is a list of all plugins used when reading the configuration from the key database\&. They are executed in \fC\fBkdbGet()\fP\fP\&.
.IP "\(bu" 2
\fBsetplugins\fP: contains a list of all plugins used when storing configuration\&. They are executed in \fC\fBkdbSet()\fP\fP\&.
.PP
.PP
Each of the plugins inside the three lists may have the subkey \fCconfig\fP\&. The configuration below this subkey provides plugin specific configuration\&. This configuration appears in the user's configuration of the plugin\&. Configuration is renamed properly\&. For example, the key 
.PP
.nf
    system/elektra/mountpoints/fstab/setplugins/#3#path#path#/config/path/allow

.fi
.PP
.PP
is transformed to 
.PP
.nf
    user/path/allow

.fi
.PP
.PP
and appears in the plugin configuration of the path plugin inside the fstab backend\&.
.PP
.SS "Referencing"
.PP
The same plugin often must occur in more than one place within a backend\&. The most common use case is a plugin that has to be executed for both \fC\fBkdbGet()\fP\fP and \fC\fBkdbSet()\fP\fP\&. It must be the same plugin if it preserves state between the executions\&.
.PP
Other plugins additionally have to handle error or success situations\&. One example of exceptional intensive use is the resolver plugin\&. It is executed twice in \fC\fBkdbSet()\fP\fP\&. In \fC\fBkdbGet()\fP\fP it is also used as shown in Listing~lst:mount point configuration}\&.
.PP
[language=]{#n<name>} introduces a new plugin from the module \fCname\fP which cannot be referenced later\&. The cypher \fCn\fP appoints the actual placement of the plugin\&. [language=]{#n#<name>#<label>#} also introduces a new plugin from the module \fCname\fP and gives it the name \fClabel\fP\&. The last [language=]{#} shows that a new name is being introduced\&. [language=]{#n#<ref>} references back to a label which was introduced before\&. This configuration does not create a new plugin\&. \fCkdb mount\fP already implements the generation of these names as described above\&.
.PP
.SS "Changing Mount Point Configuration"
.PP
When the user changes the mount point configuration, without countermeasures, applications already started will continue to run with the old configuration\&. This could lead to a problem if backends in use are changed or removed\&. It is necessary to restart all such programs\&. Notification is the best way to deal with the situation\&. Changes of the mount point configuration, however, do not occur often\&. For some systems, the manual restart may also be appropriate\&.
.PP
In this situation, applications can receive warning or error information if the configuration files are moved or removed\&. The most adverse situation occurs if the sequence of locking multiple files produces a {dead lock}\&. Under normal circumstances, the sequence of locking the files is deterministic, so either all locks can be requested or another program will be served first\&. But several programs with different mount point configurations running at the same time can cause a disaster\&. The problem gets even worse, because \fCkdb mount\fP is unable to detect such situations\&. Every specific mount point configuration for itself is trouble-free\&.
.PP
But still a dead lock can arise when multiple programs run with different mount point configurations\&. Suppose we have a program \fCA\fP which uses the backends \fCB1\fP and \fCB2\fP that requests locks for the files \fCF1\fP and \fCF2\fP\&. Then the mount point configuration is changed\&. The user removes \fCB1\fP and introduces \fCB3\fP\&. \fCB3\fP is in a different path mounted after \fCB2\fP, but also accesses the same file \fCF1\fP\&. The program \fCB\fP starts after the mount point configuration is changed\&. So it uses the backends \fCB2\fP and \fCB3\fP\&. If the scheduler decides that first \fCA\fP and then \fCB\fP both successfully lock the files \fCF1\fP and \fCF2\fP, a dead lock situation happens because in the afterwards the applications \fCA\fP and \fCB\fP try to lock \fCF2\fP and \fCF1\fP\&.
.PP
A manual solution for this problem is to enable \fCkdb\fP to output a list of processes that still use old mount point configuration\&. The administrator can restart these processes\&. The preferred solution is to use notification for mount point configuration changes or simply to use a lock-free resolver\&.
.PP
Continue reading \fBwith the data structures\fP\&.