NAME¶
zco - library for manipulating zero-copy objects
SYNOPSIS¶
#include "zco.h"
typedef enum
{
ZcoFileSource = 1,
ZcoSdrSource = 2,
ZcoZcoSource = 3
} ZcoMedium;
typedef void (*ZcoCallback);
[see description for available functions]
DESCRIPTION¶
"Zero-copy objects" (ZCOs) are abstract data access representations
designed to minimize I/O in the encapsulation of application source data
within one or more layers of communication protocol structure. ZCOs are
constructed within the heap space of an SDR to which implementations of all
layers of the stack must have access. Each ZCO contains information enabling
access to the source data objects, together with (a) a linked list of zero or
more "extents" that reference portions of these source data objects
and (b) linked lists of protocol header and trailer capsules that have been
explicitly attached to the ZCO since its creation. The concatenation of the
headers (in ascending stack sequence), source data object extents, and
trailers (in descending stack sequence) is what is to be transmitted or has
been received.
Each source data object may be either a file (identified by pathname stored in a
"file reference" object in SDR heap) or an array of bytes in SDR
heap space (identified by SDR address). Each protocol header or trailer
capsule indicates the length and the address (within SDR heap space) of a
single protocol header or trailer at some layer of the stack. Note that for
some purposes the source data object for a newly added extent of a ZCO may be
specified indirectly, by reference to an extent of an existing ZCO.
The extents of multiple ZCOs may reference the same files and/or SDR source data
objects. The source data objects are reference-counted to ensure that they are
deleted automatically when (and only when) all ZCO extents that reference them
have been deleted.
Note that the safety of shared access to a ZCO is protected by the fact that the
ZCO resides in SDR and therefore cannot be modified other than in the course
of an SDR transaction, which serializes access. Moreover, extraction of data
from a ZCO may entail the reading of file-based source data extents, which may
cause file progress to be updated in one or more file reference objects in the
SDR heap. For this reason, all ZCO "transmit" and
"receive" functions must be performed within SDR transactions.
Note also that ZCO can more broadly be used as a general-purpose reference
counting system for non-volatile data objects, where a need for such a system
is identified.
The total volume of file system space that may be occupied by file-sourced ZCO
extents and the total volume of SDR heap space that may be occupied by
heap-sourced ZCO extents are system configuration parameters that may be set
by ZCO library functions. Those limits are enforced when extents are appended
to ZCOs: total ZCO file space occupancy and total ZCO heap occupancy are
updated continuously as ZCOs are created and destroyed, and the formation of a
new extent is prohibited when the length of the extent exceeds the difference
between the applicable limit and the corresponding current occupancy total.
- void zco_register_callback(ZcoCallback notify)
- This function registers the "callback" function that the ZCO
system will invoke every time a ZCO is destroyed, making ZCO file and/or
heap space available for the formation of new ZCO extents. This mechanism
can be used, for example, to notify tasks that are waiting for ZCO space
to be made available so that they can resume some communication protocol
procedure.
- void zco_unregister_callback( )
- This function simply unregisters the currently registered callback
function for ZCO destruction.
- Object zco_create_file_ref(Sdr sdr, char *pathName, char
*cleanupScript)
- Creates and returns a new file reference object, which can be used as the
source data extent location for creating a ZCO whose source data object is
the file identified by pathName. cleanupScript, if not NULL,
is invoked at the moment the last ZCO that cites this file reference is
destroyed [normally upon delivery either down to the "ZCO transition
layer" of the protocol stack or up to a ZCO-capable application]. A
zero-length string is interpreted as implicit direction to delete the
referenced file when the file reference object is destroyed. Maximum
length of cleanupScript is 255. Returns SDR location of file
reference object on success, 0 on any error.
- Object zco_revise_file_ref(Sdr sdr, Object fileRef, char *pathName, char
*cleanupScript)
- Changes the pathName and cleanupScript of the indicated file
reference. The new values of these fields are validated as for
zco_create_file_ref(). Returns 0 on success, -1 on any error.
- char *zco_file_ref_path(Sdr sdr, Object fileRef, char *buffer, int
buflen)
- Retrieves the pathName associated with fileRef and stores it in
buffer, truncating it to fit (as indicated by buflen) and
NULL-terminating it. On success, returns buffer; returns NULL on
any error.
- int zco_file_ref_xmit_eof(Sdr sdr, Object fileRef)
- Returns 1 if the last octet of the referenced file (as determined at the
time the file reference object was created) has been read by ZCO via a
reader with file offset tracking turned on. Otherwise returns zero.
- void zco_destroy_file_ref(Sdr sdr, Object fileRef)
- If the file reference object residing at location fileRef within
the indicated Sdr is no longer in use (no longer referenced by any ZCO),
destroys this file reference object immediately. Otherwise, flags this
file reference object for destruction as soon as the last reference to it
is removed.
- vast zco_get_file_occupancy(Sdr sdr)
- Returns the total number of file system space bytes occupied by ZCOs
created in this Sdr.
- void zco_set_max_file_occupancy(Sdr sdr, vast occupancy)
- Declares the total number of file system space bytes that may be occupied
by ZCOs created in this Sdr.
- vast zco_get_max_file_occupancy(Sdr sdr)
- Returns the total number of file system space bytes that may be occupied
by ZCOs created in this Sdr.
- int zco_enough_file_space(Sdr sdr, vast length)
- Returns 1 if the total remaining file system space available for ZCOs in
this Sdr is greater than length. Returns 0 otherwise.
- vast zco_get_heap_occupancy(Sdr sdr)
- Returns the total number of SDR heap space bytes occupied by ZCOs created
in this Sdr.
- void zco_set_max_heap_occupancy(Sdr sdr, vast occupancy)
- Declares the total number of SDR heap space bytes that may be occupied by
ZCOs created in this Sdr.
- vast zco_get_max_heap_occupancy(Sdr sdr)
- Returns the total number of SDR heap space bytes that may be occupied by
ZCOs created in this Sdr.
- int zco_enough_heap_space(Sdr sdr, vast length)
- Returns 1 if the total remaining SDR heap space available for ZCOs in this
Sdr is greater than length. Returns 0 otherwise.
- Object zco_create(Sdr sdr, ZcoMedium firstExtentSourceMedium, Object
firstExtentLocation, vast firstExtentOffset, vast firstExtentLength)
- Creates a new ZCO. firstExtentLocation and firstExtentLength
must either both be zero (indicating that zco_append_extent() will
be used to insert the first source data extent later) or else both be
non-zero. If firstExtentLocation is non-zero, then (a)
firstExtentLocation must be the SDR location of a file reference
object if firstExtentSourceMedium is ZcoFileSource and must
otherwise be the SDR location of the source data itself, and (b)
firstExtentOffset indicates how many leading bytes of the source
data object should be skipped over when adding the initial source data
extent to the new ZCO. On success, returns the SDR location of the new
ZCO. Returns 0 if there is insufficient ZCO space for creation of the new
ZCO; returns ((Object) -1) on any error.
- int zco_append_extent(Sdr sdr, Object zco, ZcoMedium sourceMedium, Object
location, vast offset, vast length)
- Appends the indicated source data extent to the indicated ZCO, as
described for zco_create(). Both the location and
length of the source data must be non-zero. Returns length
on success, 0 if there is insufficient ZCO space for creation of the new
source data extent, -1 on any error.
- int zco_prepend_header(Sdr sdr, Object zco, char *header, vast
length)
- int zco_append_trailer(Sdr sdr, Object zco, char *trailer, vast
length)
- void zco_discard_first_header(Sdr sdr, Object zco)
- void zco_discard_last_trailer(Sdr sdr, Object zco)
- These functions attach and remove the ZCO's headers and trailers.
header and trailer are assumed to be arrays of octets, not
necessarily text. Attaching a header or trailer causes it to be written to
the SDR. The prepend and append functions return 0 on success, -1 on any
error.
- void zco_destroy(Sdr sdr, Object zco)
- Destroys the indicated Zco. This reduces the reference counts for all
files and SDR objects referenced in the ZCO's extents, resulting in the
freeing of SDR objects and (optionally) the deletion of files as those
reference count drop to zero.
- Object zco_clone(Sdr sdr, Object zco, vast offset, vast length)
- Creates a new ZCO whose source data is a copy of a subset of the source
data of the referenced ZCO. Portions of the source data extents of the
original ZCO are copied as necessary, but no header or trailer capsules
are copied. Returns SDR location of the new ZCO on success, 0 on any
error.
- vast zco_clone_source_data(Sdr sdr, Object toZco, Object fromZco, vast
offset, vast length)
- Appends to toZco a copy of a subset of the source data of
fromZCO. Portions of the source data extents of fromZCO are
copied as necessary. Returns length on success, -1 on any
error.
- vast zco_length(Sdr sdr, Object zco)
- Returns length of entire ZCO, including all headers and trailers and all
source data extents. This is the size of the object that would be formed
by concatenating the text of all headers, trailers, and source data
extents into a single serialized object.
- vast zco_source_data_length(Sdr sdr, Object zco)
- Returns length of entire ZCO minus the lengths of all attached header and
trailer capsules. This is the size of the object that would be formed by
concatenating the text of all source data extents (including those that
are presumed to contain header or trailer text attached elsewhere) into a
single serialized object.
- void zco_start_transmitting(Object zco, ZcoReader *reader)
- Used by underlying protocol layer to start extraction of an outbound ZCO's
bytes (both from header and trailer capsules and from source data extents)
for "transmission" -- i.e., the copying of bytes into a memory
buffer for delivery to some non-ZCO-aware protocol implementation.
Initializes reading at the first byte of the total concatenated ZCO
object. Populates reader, which is used to keep track of
"transmission" progress via this ZCO reference.
Note that this function can be called multiple times to restart reading at
the start of the ZCO. Note also that multiple ZcoReader objects may be
used concurrently, by the same task or different tasks, to advance through
the ZCO independently.
- void zco_track_file_offset(ZcoReader *reader)
- Turns on file offset tracking for this reader.
- vast zco_transmit(Sdr sdr, ZcoReader *reader, vast length, char
*buffer)
- Copies length as-yet-uncopied bytes of the total concatenated ZCO
(referenced by reader) into buffer. If buffer is
NULL, skips over length bytes without copying. Returns the number
of bytes copied (or skipped) on success, 0 on any file access error, -1 on
any other error.
- void zco_start_receiving(Object zco, ZcoReader *reader)
- Used by overlying protocol layer to start extraction of an inbound ZCO's
bytes for "reception" -- i.e., the copying of bytes into a
memory buffer for delivery to a protocol header parser, to a protocol
trailer parser, or to the ultimate recipient (application). Initializes
reading of headers, source data, and trailers at the first byte of the
concatenated ZCO objects. Populates reader, which is used to keep
track of "reception" progress via this ZCO reference and is
required.
- vast zco_receive_headers(Sdr sdr, ZcoReader *reader, vast length, char
*buffer)
- Copies length as-yet-uncopied bytes of presumptive protocol header
text from ZCO source data extents into buffer. If buffer is
NULL, skips over length bytes without copying. Returns number of
bytes copied (or skipped) on success, 0 on any file access error, -1 on
any other error.
- void zco_delimit_source(Sdr sdr, Object zco, vast offset, vast
length)
- Sets the computed offset and length of actual source data in the ZCO,
thereby implicitly establishing the total length of the ZCO's concatenated
protocol headers as offset and the location of the ZCO's innermost
protocol trailer as the sum of offset and length. Offset and
length are typically determined from the information carried in received
presumptive protocol header text.
- vast zco_receive_source(Sdr sdr, ZcoReader *reader, vast length, char
*buffer)
- Copies length as-yet-uncopied bytes of source data from ZCO extents
into buffer. If buffer is NULL, skips over length
bytes without copying. Returns number of bytes copied (or skipped) on
success, 0 on any file access error, -1 on any other error.
- vast zco_receive_trailers(Sdr sdr, ZcoReader *reader, vast length, char
*buffer)
- Copies length as-yet-uncopied bytes of trailer data from ZCO
extents into buffer. If buffer is NULL, skips over
length bytes without copying. Returns number of bytes copied (or
skipped) on success, 0 on any file access error, -1 on any other
error.
- void zco_strip(Sdr sdr, Object zco)
- Deletes all source data extents that contain only header or trailer data
and adjusts the offsets and/or lengths of all remaining extents to exclude
any known header or trailer data. This function is useful when handling a
ZCO that was received from an underlying protocol layer rather than from
an overlying application or protocol layer; use it before starting the
transmission of the ZCO to another node or before enqueuing it for
reception by an overlying application or protocol layer.
SEE ALSO¶
sdr(3)