NAME¶
coroipc_overview - Overview of coroipc libraries
OVERVIEW¶
The coroipcs and coroipcc libraries provide a generically reusable very high
performance shared memory IPC sytem for client and service applications. It
supports many features including:
65536 user services and 65536 command ids per service.
Shared memory implementation for very high performance.
A synchronous request/response channel and asynchronous response channel per ipc
connection.
User defined private data per IPC connection.
Ability to call a function per service on ipc connection and disconnection.
Authenticated IPC connection with ability for developer to define which UIDs and
GIDs are valid at connection time.
Fully abstracted poll system so that any poll library may be used.
User defined selector for determining the proper function to call per service
and id.
Description of the libraries¶
There are two shared libraries available for developing IPC client applications.
The first library is coroipcs.so which is responsible for the server
implementation. This library should be linked with the server and then
initialized with coroipcs_init(3).
Once the library is initialized, it will provide service to coroipcc.so library
users.
The second library is coroipcc.so which is responsible for the client
implementation. This library should be linked with the client and requires no
initialization. This library provides communication functions for sending and
receiving synchronous requests, and also reading asynchronous message requests
from the server.
Initializing the coroipcs library¶
To use the coroipcs library, the developer creates a coroipcs_init_state
structure and populates it with function names. The functions do various
operations described in coroipcs_init(3) man page. Not all operations must be
specified. If some are missing, the corosync ipcs library will automatically
populate the structure with internal versions which provide basic
functionality.
Communicating with the coroipcc clients¶
Every ipc connection is represented by a void * pointer which uniquely
identifies the data set for the IPC connection. Each IPC connection also
contains user defined private data. To obtain this private data pointer, the
function coroipcs_private_data_get(3) function can be called.
There are two channels for communication. The primary channel is the synchronous
request/response channel. Another channel is available for out of band
asynchronous responses from the server.
To send a response on the syncronous channel, coroipcs_response_send(3) or
coroipcs_response_iov_send(3) should be used. To send a message on the
asynchronous channel, coroipcs_dispatch_send(3) or
coroipc_dispatch_iov_send(3) should be used.
The abstracted poll system¶
There are many different poll systems in use in applications. They are usually
intricately tied up in the implementation of the application and each provide
different APIs and functionality. To manage this, the coroipcs library
provides callbacks in coroipcs_init(3) which should be called when a new
connection should be added for accept system calls or to dispatch messages.
These callbacks add the relevant fd to the application's poll system. When the
application poll system triggers the callback registered by the user defined
poll adding functions, they then call either coroipc_handler_accept(3) or
coroipc_handler_dispatch(3).
Initializing the coroipcc library¶
No initialization is required in the coroipcc library.
Lifecycle of an IPC connection.¶
An IPC connection is made to the server with coroipcc_service_connect(3). This
function connects to the server and requests channels be created for
communication. To disconnect, the client either exits or executes the function
coroipcc_service_disconnect(3).
Synchronous communication¶
There are two functions for sending a request and receiving a response. The
first function coroipcc_msg_send_reply_receive(3) sends an iovector request
and receives a response. This function copies the response into the response
buffer. the second function coroipcc_msg_end_reply_receive_in_buf(3) does not
copy the response buffer and allows for zero-copy reading of the response when
the lifetime of the response buffer is known.
Asynchronous communication¶
The coroipcc_dispatch_recv(3) function receives an out-of-band asyncronous
message. Unlike the synchronous communication channel, the asynchronous
messages are queued and can provide very high out-of-band performance.
To determine when to call coroipcc_dispatch_recv(3) the corosync_fd_get(3) call
is used to obtain a file descriptor used in the
poll(2) or
select(2) system
calls.
Finally the current dispatch flow control state can be obtained with
coroipcc_flow_control_get(3).
The ipc system is tuned for very high performance while also being comletely
abstracted from the underlying poll mechanism and any internalisms required by
the server. The ipc system achieves such high performance by using shared
memory as oppossed to slower techniques such as UNIX_PF sockets.
We intend to do further development to allow syncronous requests to return
messages in an asyncronous way to avoid blocking involved in the syncronous
request/response model used today for higher throughput in some use cases.
Security¶
The ipc system uses default operating system security mechanics to ensure ipc
connections are validated. A callback used with coroipcs_init(3) is called for
every new ipc connection with the parameters of UID and GID. The callback then
determines if the UID and GID are authenticated for communication. More about
this topic can be viewed in the coroipcs_init(3) man page.
SEE ALSO¶
coroipcs_ipc_init(3),
coroipcs_ipc_exit(3),
coroipcs_private_data_get(3),
coroipcs_respone_send(3),
coroipcs_response_iov_send(3),
coroipcs_dispatch_send(3),
coroipcs_dispatch_iov_send(3),
coroipcs_refcount_inc(3),
coroipcs_refcount_dec(3),
coroipcs_handler_accept(3),
coroipcs_handler_dispatch(3),
cooripcc_service_connect(3),
coroipcc_service_disconnect(3),
coroipcc_msg_send_reply_receive(3),
coroipcc_msg_send_reply_receive_in_buf(3),
coroipcc_dispatch_recv(3),
coroipcc_fd_get(3),
coroipcc_dispatch_flow_control_get(3)