NAME¶
zco - library for manipulating zero-copy objects
SYNOPSIS¶
#include "zco.h"
typedef enum
{
ZcoFileSource = 1,
ZcoSdrSource
} ZcoMedium;
[see description for available functions]
DESCRIPTION¶
"Zero-copy objects" (ZCOs) are abstract data access representations
designed to minimize I/O in the encapsulation of application source data
within one or more layers of communication protocol structure. ZCOs are
constructed within the heap space of an SDR to which implementations of all
layers of the stack must have access. Each ZCO contains information enabling
access to the source data object, together with (a) a linked list of zero or
more "extents" that reference portions of this source data object
and (b) linked lists of protocol header and trailer capsules that have been
explicitly attached to the ZCO since its creation. The concatenation of the
headers (in ascending stack sequence), source data object extents, and
trailers (in descending stack sequence) is what is to be transmitted or has
been received.
Each source data object may be either a file (identified by pathname stored in a
"file reference" object in SDR heap) or an array of bytes in SDR
heap space (identified by SDR address). Each protocol header or trailer
capsule indicates the length and the address (within SDR heap space) of a
single protocol header or trailer at some layer of the stack.
The extents of multiple ZCOs may reference the same files and/or SDR source data
objects. The source data objects are reference-counted to ensure that they are
deleted automatically when (and only when) all ZCO extents that reference them
have been deleted.
Note that the safety of shared access to a ZCO is protected by the fact that the
ZCO resides in SDR and therefore cannot be modified other than in the course
of an SDR transaction, which serializes access. Moreover, extraction of data
from a ZCO may entail the reading of file-based source data extents, which may
cause file progress to be updated in one or more file reference objects in the
SDR heap. For this reason, all ZCO "transmit" and
"receive" functions must be performed within SDR transactions.
Note also that ZCO can more broadly be used as a general-purpose reference
counting system for non-volatile data objects, where a need for such a system
is identified.
- Object zco_create_file_ref(Sdr sdr, char *pathName, char
*cleanupScript)
- Creates and returns a new file reference object, which can
be used as the source data extent location for creating a ZCO whose source
data object is the file identified by pathName.
cleanupScript, if not NULL, is invoked at the moment the last ZCO
that cites this file reference is destroyed [normally upon delivery either
down to the "ZCO transition layer" of the protocol stack or up
to a ZCO-capable application]. A zero-length string is interpreted as
implicit direction to delete the referenced file when the file reference
object is destroyed. Maximum length of cleanupScript is 255.
Returns SDR location of file reference object on success, 0 on any
error.
- Object zco_revise_file_ref(Sdr sdr, Object fileRef, char
*pathName, char *cleanupScript)
- Changes the pathName and cleanupScript of the
indicated file reference. The new values of these fields are validated as
for zco_create_file_ref(). Returns 0 on success, -1 on any
error.
- char *zco_file_ref_path(Sdr sdr, Object fileRef, char
*buffer, int buflen)
- Retrieves the pathName associated with fileRef and
stores it in buffer, truncating it to fit (as indicated by
buflen) and NULL-terminating it. On success, returns buffer;
returns NULL on any error.
- unsigned int zco_file_ref_occupancy(Sdr sdr, Object
fileRef)
- Returns the number of bytes of SDR heap space occupied by
the ZCO file reference object identified by fileRef. If
fileRef is zero, returns the maximum possible SDR space occupancy
of any single file reference object.
- int zco_file_ref_xmit_eof(Sdr sdr, Object fileRef)
- Returns 1 if the last octet of the referenced file (as
determined at the time the file reference object was created) has been
read by ZCO via a reader with file offset tracking turned on. Otherwise
returns zero.
- void zco_destroy_file_ref(Sdr sdr, Object fileRef)
- If the file reference object residing at location
fileRef within the indicated Sdr is no longer in use (no longer
referenced by any ZCO), destroys this file reference object immediately.
Otherwise, flags this file reference object for destruction as soon as the
last reference to it is removed.
- Object zco_create(Sdr sdr, ZcoMedium
firstExtentSourceMedium, Object firstExtentLocation, unsigned int
firstExtentOffset, unsigned int firstExtentLength)
- Creates a new ZCO. firstExtentLocation and
firstExtentLength must either both be zero (indicating that
zco_append_extent() will be used to insert the first source data
extent later) or else both be non-zero. If firstExtentLocation is
non-zero, then (a) firstExtentLocation must be the SDR location of
a file reference object if firstExtentSourceMedium is ZcoFileSource
and must otherwise be the SDR location of the source data itself, and (b)
firstExtentOffset indicates how many leading bytes of the source
data object should be skipped over when adding the initial source data
extent to the new ZCO. On success, returns the SDR location of the the new
ZCO. Returns 0 on any error.
- int zco_append_extent(Sdr sdr, Object zco, ZcoMedium
sourceMedium, Object location, unsigned int offset, unsigned int
length)
- Appends the indicated source data extent to the indicated
ZCO, as described for zco_create(). Both the location and
length of the source data must be non-zero. Returns 0 on success,
-1 on any error.
- int zco_prepend_header(Sdr sdr, Object zco, char *header,
unsigned int length)
- int zco_append_trailer(Sdr sdr, Object zco, char *trailer,
unsigned int length)
- void zco_discard_first_header(Sdr sdr, Object zco)
- void zco_discard_last_trailer(Sdr sdr, Object zco)
- These functions attach and remove the ZCO's headers and
trailers. header and trailer are assumed to be arrays of
octets, not necessarily text. Attaching a header or trailer causes it to
be written to the SDR. The prepend and append functions return 0 on
success, -1 on any error.
- void zco_destroy(Sdr sdr, Object zco)
- Destroys the indicated Zco. This reduces the reference
counts for all files and SDR objects referenced in the ZCO's extents,
resulting in the freeing of SDR objects and (optionally) the deletion of
files as those reference count drop to zero.
- Object zco_clone(Sdr sdr, Object zco, unsigned int offset,
unsigned int length)
- Creates a new ZCO whose source data is a copy of a subset
of the source data of the referenced ZCO. Portions of the source data
extents of the original ZCO are copied as necessary, but no header or
trailer capsules are copied. Returns SDR location of the new ZCO on
success, 0 on any error.
- unsigned int zco_length(Sdr sdr, Object zco)
- Returns length of entire ZCO, including all headers and
trailers and all source data extents. This is the size of the object that
would be formed by concatenating the text of all headers, trailers, and
source data extents into a single serialized object.
- unsigned int zco_source_data_length(Sdr sdr, Object
zco)
- Returns length of entire ZCO minus the lengths of all
attached header and trailer capsules. This is the size of the object that
would be formed by concatenating the text of all source data extents
(including those that are presumed to contain header or trailer text
attached elsewhere) into a single serialized object.
- unsigned int zco_occupancy(Sdr sdr, Object zco)
- Returns the number of bytes of SDR heap space occupied by
the ZCO identified by zco. It includes the sizes of the ZCO, ZCO
header/trailer capsules, and ZCO extent objects themselves, and it
omits the size of all source data extent text that does not occupy
SDR heap space (i.e., source data that resides in files to which ZCO
extent objects refer).
Note that this figure may be somewhat overstated: all space occupied by the
object referenced by each SDR source data extent of a ZCO is included in
the occupancy of that ZCO, even if multiple extents reference the same
object and/or the extents of multiple cloned ZCOs reference that same
object. In ION the impact of this oversatement is minimal because (a) we
don't cite multiple parts of a single SDR object in different extents of a
ZCO and ((b) the ZCOs that are likely to be cloned are bundle payloads,
most of whose extents are likely to reference file source data
objects.
- void zco_start_transmitting(Object zco, ZcoReader
*reader)
- Used by underlying protocol layer to start extraction of an
outbound ZCO's bytes (both from header and trailer capsules and from
source data extents) for "transmission" -- i.e., the copying of
bytes into a memory buffer for delivery to some non-ZCO-aware protocol
implementation. Initializes reading at the first byte of the total
concatenated ZCO object. Populates reader, which is used to keep
track of "transmission" progress via this ZCO reference.
Note that this function can be called multiple times to restart reading at
the start of the ZCO. Note also that multiple ZcoReader objects may be
used concurrently, by the same task or different tasks, to advance through
the ZCO independently.
- void zco_track_file_offset(ZcoReader *reader)
- Turns on file offset tracking for this reader.
- int zco_transmit(Sdr sdr, ZcoReader *reader, unsigned int
length, char *buffer)
- Copies length as-yet-uncopied bytes of the total
concatenated ZCO (referenced by reader) into buffer. If
buffer is NULL, skips over length bytes without copying.
Returns the number of bytes copied (or skipped) on success, 0 on any file
access error, -1 on any other error.
- void zco_start_receiving(Object zco, ZcoReader
*reader)
- Used by overlying protocol layer to start extraction of an
inbound ZCO's bytes for "reception" -- i.e., the copying of
bytes into a memory buffer for delivery to a protocol header parser, to a
protocol trailer parser, or to the ultimate recipient (application).
Initializes reading of headers, source data, and trailers at the first
byte of the concatenated ZCO objects. Populates reader, which is
used to keep track of "reception" progress via this ZCO
reference and is required.
- int zco_receive_headers(Sdr sdr, ZcoReader *reader,
unsigned int length, char *buffer)
- Copies length as-yet-uncopied bytes of presumptive
protocol header text from ZCO source data extents into buffer. If
buffer is NULL, skips over length bytes without copying.
Returns number of bytes copied (or skipped) on success, 0 on any file
access error, -1 on any other error.
- void zco_delimit_source(Sdr sdr, Object zco, unsigned int
offset, unsigned int length)
- Sets the computed offset and length of actual source data
in the ZCO, thereby implicitly establishing the total length of the ZCO's
concatenated protocol headers as offset and the location of the
ZCO's innermost protocol trailer as the sum of offset and
length. Offset and length are typically determined from the
information carried in received presumptive protocol header text.
- int zco_receive_source(Sdr sdr, ZcoReader *reader, unsigned
int length, char *buffer)
- Copies length as-yet-uncopied bytes of source data
from ZCO extents into buffer. If buffer is NULL, skips over
length bytes without copying. Returns number of bytes copied (or
skipped) on success, 0 on any file access error, -1 on any other
error.
- int zco_receive_trailers(Sdr sdr, ZcoReader *reader,
unsigned int length, char *buffer)
- Copies length as-yet-uncopied bytes of trailer data
from ZCO extents into buffer. If buffer is NULL, skips over
length bytes without copying. Returns number of bytes copied (or
skipped) on success, 0 on any file access error, -1 on any other
error.
- void zco_strip(Sdr sdr, Object zco)
- Deletes all source data extents that contain only header or
trailer data and adjusts the offsets and/or lengths of all remaining
extents to exclude any known header or trailer data. Use this function
before concatenating with another ZCO, before starting the transmission of
a ZCO that was received from an underlying protocol layer rather than from
an overlying application or protocol layer, and before enqueuing the ZCO
for reception by an overlying application or protocol layer.
SEE ALSO¶
sdr(3)