NAME¶
IPC::Shareable - share Perl variables between processes
SYNOPSIS¶
use IPC::Shareable (':lock');
tie SCALAR, 'IPC::Shareable', GLUE, OPTIONS;
tie ARRAY, 'IPC::Shareable', GLUE, OPTIONS;
tie HASH, 'IPC::Shareable', GLUE, OPTIONS;
(tied VARIABLE)->shlock;
(tied VARIABLE)->shunlock;
(tied VARIABLE)->shlock(LOCK_SH|LOCK_NB)
or print "resource unavailable\n";
(tied VARIABLE)->remove;
IPC::Shareable->clean_up;
IPC::Shareable->clean_up_all;
CONVENTIONS¶
The occurrence of a number in square brackets, as in [N], in the text of this
document refers to a numbered note in the "NOTES".
DESCRIPTION¶
IPC::Shareable allows you to tie a variable to shared memory making it easy to
share the contents of that variable with other Perl processes. Scalars,
arrays, and hashes can be tied. The variable being tied may contain
arbitrarily complex data structures - including references to arrays, hashes
of hashes, etc.
The association between variables in distinct processes is provided by GLUE.
This is an integer number or 4 character string[1] that serves as a common
identifier for data across process space. Hence the statement
tie $scalar, 'IPC::Shareable', 'data';
in program one and the statement
tie $variable, 'IPC::Shareable', 'data';
in program two will bind $scalar in program one and $variable in program two.
There is no pre-set limit to the number of processes that can bind to data; nor
is there a pre-set limit to the complexity of the underlying data of the tied
variables[2]. The amount of data that can be shared within a single bound
variable is limited by the system's maximum size for a shared memory segment
(the exact value is system-dependent).
The bound data structures are all linearized (using Raphael Manfredi's Storable
module) before being slurped into shared memory. Upon retrieval, the original
format of the data structure is recovered. Semaphore flags can be used for
locking data between competing processes.
OPTIONS¶
Options are specified by passing a reference to a hash as the fourth argument to
the
tie() function that enchants a variable. Alternatively you can pass
a reference to a hash as the third argument; IPC::Shareable will then look at
the field named
key in this hash for the value of GLUE. So,
tie $variable, 'IPC::Shareable', 'data', \%options;
is equivalent to
tie $variable, 'IPC::Shareable', { key => 'data', ... };
Boolean option values can be specified using a value that evaluates to either
true or false in the Perl sense.
NOTE: Earlier versions allowed you to use the word
yes for true and the
word
no for false, but support for this "feature" is being
removed.
yes will still act as true (since it is true, in the Perl
sense), but use of the word
no now emits an (optional) warning and then
converts to a false value. This warning will become mandatory in a future
release and then at some later date the use of
no will stop working
altogether.
The following fields are recognized in the options hash.
- key
- The key field is used to determine the GLUE when using the
three-argument form of the call to tie(). This argument is then, in
turn, used as the KEY argument in subsequent calls to shmget() and
semget().
The default value is IPC_PRIVATE, meaning that your variables cannot be
shared with other processes.
- create
- create is used to control whether calls to tie() create new
shared memory segments or not. If create is set to a true value,
IPC::Shareable will create a new binding associated with GLUE as needed.
If create is false, IPC::Shareable will not attempt to create a new
shared memory segment associated with GLUE. In this case, a shared memory
segment associated with GLUE must already exist or the call to
tie() will fail and return undef. The default is false.
- exclusive
- If exclusive field is set to a true value, calls to tie()
will fail (returning undef) if a data binding associated with GLUE already
exists. If set to a false value, calls to tie() will succeed even
if a shared memory segment associated with GLUE already exists. The
default is false
- mode
- The mode argument is an octal number specifying the access
permissions when a new data binding is being created. These access
permission are the same as file access permissions in that 0666 is world
readable, 0600 is readable only by the effective UID of the process
creating the shared variable, etc. The default is 0666 (world readable and
writable).
- destroy
- If set to a true value, the shared memory segment underlying the data
binding will be removed when the process calling tie() exits
(gracefully)[3]. Use this option with care. In particular you should not
use this option in a program that will fork after binding the data. On the
other hand, shared memory is a finite resource and should be released if
it is not needed. The default is false
- size
- This field may be used to specify the size of the shared memory segment
allocated. The default is IPC::Shareable::SHM_BUFSIZ().
Default values for options are
key => IPC_PRIVATE,
create => 0,
exclusive => 0,
destroy => 0,
mode => 0,
size => IPC::Shareable::SHM_BUFSIZ(),
LOCKING¶
IPC::Shareable provides methods to implement application-level advisory locking
of the shared data structures. These methods are called
shlock() and
shunlock(). To use them you must first get the object underlying the
tied variable, either by saving the return value of the original call to
tie() or by using the built-in
tied() function.
To lock a variable, do this:
$knot = tie $sv, 'IPC::Shareable', $glue, { %options };
...
$knot->shlock;
or equivalently
tie($scalar, 'IPC::Shareable', $glue, { %options });
(tied $scalar)->shlock;
This will place an exclusive lock on the data of $scalar. You can also get
shared locks or attempt to get a lock without blocking. IPC::Shareable makes
the constants LOCK_EX, LOCK_SH, LOCK_UN, and LOCK_NB exportable to your
address space with the export tags ":lock", ":flock", or
":all". The values should be the same as the standard
"flock" option arguments.
if ( (tied $scalar)->shlock(LOCK_SH|LOCK_NB) ) {
print "The value is $scalar\n";
(tied $scalar)->shunlock;
} else {
print "Another process has an exlusive lock.\n";
}
If no argument is provided to "shlock", it defaults to LOCK_EX. To
unlock a variable do this:
$knot->shunlock;
or
(tied $scalar)->shunlock;
or
$knot->shlock(LOCK_UN); # Same as calling shunlock
There are some pitfalls regarding locking and signals about which you should
make yourself aware; these are discussed in "NOTES".
If you use the advisory locking, IPC::Shareable assumes that you know what you
are doing and attempts some optimizations. When you obtain a lock, either
exclusive or shared, a fetch and thaw of the data is performed. No additional
fetch/thaw operations are performed until you release the lock and access the
bound variable again. During the time that the lock is kept, all accesses are
perfomed on the copy in program memory. If other processes do not honor the
lock, and update the shared memory region unfairly, the process with the lock
will not be in sync. In other words, IPC::Shareable does not enforce the lock
for you.
A similar optimization is done if you obtain an exclusive lock. Updates to the
shared memory region will be postponed until you release the lock (or
downgrade to a shared lock).
Use of locking can significantly improve performance for operations such as
iterating over an array, retrieving a list from a slice or doing a slice
assignment.
REFERENCES¶
When a reference to a non-tied scalar, hash, or array is assigned to a
tie()d variable, IPC::Shareable will attempt to
tie() the thingy
being referenced[4]. This allows disparate processes to see changes to not
only the top-level variable, but also changes to nested data. This feature is
intended to be transparent to the application, but there are some caveats to
be aware of.
First of all, IPC::Shareable does not (yet) guarantee that the ids shared memory
segments allocated automagically are unique. The more automagical
tie()ing that happens, the greater the chance of a collision.
Secondly, since a new shared memory segment is created for each thingy being
referenced, the liberal use of references could cause the system to approach
its limit for the total number of shared memory segments allowed.
OBJECTS¶
IPC::Shareable implements
tie()ing objects to shared memory too. Since an
object is just a reference, the same principles (and caveats) apply to
tie()ing objects as other reference types.
DESTRUCTION¶
perl(1) will destroy the object underlying a tied variable when then tied
variable goes out of scope. Unfortunately for IPC::Shareable, this may not be
desirable: other processes may still need a handle on the relevant shared
memory segment. IPC::Shareable therefore provides an interface to allow the
application to control the timing of removal of shared memory segments. The
interface consists of three methods -
remove(),
clean_up(), and
clean_up_all() - and the
destroy option to
tie().
- destroy option
- As described in "OPTIONS", specifying the destroy option
when tie()ing a variable coerces IPC::Shareable to remove the
underlying shared memory segment when the process calling tie()
exits gracefully. Note that any related shared memory segments created
automagically by the use of references will also be removed.
- remove()
-
(tied $var)->remove;
Calling remove() on the object underlying a tie()d variable
removes the associated shared memory segment. The segment is removed
irrespective of whether it has the destroy option set or not and
irrespective of whether the calling process created the segment.
- clean_up()
-
IPC::Shareable->clean_up;
This is a class method that provokes IPC::Shareable to remove all shared
memory segments created by the process. Segments not created by the
calling process are not removed.
- clean_up_all()
-
IPC::Shareable->clean_up_all;
This is a class method that provokes IPC::Shareable to remove all shared
memory segments encountered by the process. Segments are removed even if
they were not created by the calling process.
EXAMPLES¶
In a file called
server:
#!/usr/bin/perl -w
use strict;
use IPC::Shareable;
my $glue = 'data';
my %options = (
create => 'yes',
exclusive => 0,
mode => 0644,
destroy => 'yes',
);
my %colours;
tie %colours, 'IPC::Shareable', $glue, { %options } or
die "server: tie failed\n";
%colours = (
red => [
'fire truck',
'leaves in the fall',
],
blue => [
'sky',
'police cars',
],
);
((print "server: there are 2 colours\n"), sleep 5)
while scalar keys %colours == 2;
print "server: here are all my colours:\n";
foreach my $c (keys %colours) {
print "server: these are $c: ",
join(', ', @{$colours{$c}}), "\n";
}
exit;
In a file called
client
#!/usr/bin/perl -w
use strict;
use IPC::Shareable;
my $glue = 'data';
my %options = (
create => 0,
exclusive => 0,
mode => 0644,
destroy => 0,
);
my %colours;
tie %colours, 'IPC::Shareable', $glue, { %options } or
die "client: tie failed\n";
foreach my $c (keys %colours) {
print "client: these are $c: ",
join(', ', @{$colours{$c}}), "\n";
}
delete $colours{'red'};
exit;
And here is the output (the sleep commands in the command line prevent the
output from being interrupted by shell prompts):
bash$ ( ./server & ) ; sleep 10 ; ./client ; sleep 10
server: there are 2 colours
server: there are 2 colours
server: there are 2 colours
client: these are blue: sky, police cars
client: these are red: fire truck, leaves in the fall
server: here are all my colours:
server: these are blue: sky, police cars
RETURN VALUES¶
Calls to
tie() that try to implement IPC::Shareable will return true if
successful,
undef otherwise. The value returned is an instance of the
IPC::Shareable class.
AUTHOR¶
Benjamin Sugars <bsugars@canoe.ca>
NOTES¶
- 1.
- If GLUE is longer than 4 characters, only the 4 most significant
characters are used. These characters are turned into integers by
unpack()ing them. If GLUE is less than 4 characters, it is space
padded.
- 2.
- IPC::Shareable provides no pre-set limits, but the system does. Namely,
there are limits on the number of shared memory segments that can be
allocated and the total amount of memory usable by shared memory.
- 3.
- If the process has been smoked by an untrapped signal, the binding will
remain in shared memory. If you're cautious, you might try
$SIG{INT} = \&catch_int;
sub catch_int {
die;
}
...
tie $variable, IPC::Shareable, 'data', { 'destroy' => 'Yes!' };
which will at least clean up after your user hits CTRL-C because
IPC::Shareable's END method will be called. Or, maybe you'd like to leave
the binding in shared memory, so subsequent process can recover the
data...
- 4.
- This behaviour is markedly different from previous versions of
IPC::Shareable. Older versions would sometimes tie() referenced
thingies, and sometimes not. The new approach is more reliable (I think)
and predictable (certainly) but uses more shared memory segments.
General Notes¶
- o
- When using shlock() to lock a variable, be careful to guard against
signals. Under normal circumstances, IPC::Shareable's END method unlocks
any locked variables when the process exits. However, if an untrapped
signal is received while a process holds an exclusive lock, DESTROY will
not be called and the lock may be maintained even though the process has
exited. If this scares you, you might be better off implementing your own
locking methods.
One advantage of using "flock" on some known file instead of the
locking implemented with semaphores in IPC::Shareable is that when a
process dies, it automatically releases any locks. This only happens with
IPC::Shareable if the process dies gracefully. The alternative is to
attempt to account for every possible calamitous ending for your process
(robust signal handling in Perl is a source of much debate, though it
usually works just fine) or to become familiar with your system's tools
for removing shared memory and semaphores. This concern should be balanced
against the significant performance improvements you can gain for larger
data structures by using the locking mechanism implemented in
IPC::Shareable.
- o
- There is a program called ipcs(1/8) (and ipcrm(1/8)) that is available on
at least Solaris and Linux that might be useful for cleaning moribund
shared memory segments or semaphore sets produced by bugs in either
IPC::Shareable or applications using it.
- o
- This version of IPC::Shareable does not understand the format of shared
memory segments created by versions prior to 0.60. If you try to tie to
such segments, you will get an error. The only work around is to clear the
shared memory segments and start with a fresh set.
- o
- Iterating over a hash causes a special optimization if you have not
obtained a lock (it is better to obtain a read (or write) lock before
iterating over a hash tied to Shareable, but we attempt this optimization
if you do not). The fetch/thaw operation is performed when the first key
is accessed. Subsequent key and and value accesses are done without
accessing shared memory. Doing an assignment to the hash or fetching
another value between key accesses causes the hash to be replaced from
shared memory. The state of the iterator in this case is not defined by
the Perl documentation. Caveat Emptor.
CREDITS¶
Thanks to all those with comments or bug fixes, especially
Maurice Aubrey <maurice@hevanet.com>
Stephane Bortzmeyer <bortzmeyer@pasteur.fr>
Doug MacEachern <dougm@telebusiness.co.nz>
Robert Emmery <roberte@netscape.com>
Mohammed J. Kabir <kabir@intevo.com>
Terry Ewing <terry@intevo.com>
Tim Fries <timf@dicecorp.com>
Joe Thomas <jthomas@women.com>
Paul Makepeace <Paul.Makepeace@realprogrammers.com>
Raphael Manfredi <Raphael_Manfredi@pobox.com>
Lee Lindley <Lee.Lindley@bigfoot.com>
Dave Rolsky <autarch@urth.org>
BUGS¶
Certainly; this is beta software. When you discover an anomaly, send an email to
me at bsugars@canoe.ca.
SEE ALSO¶
perl(1),
perltie(1),
Storable(3),
shmget(2),
ipcs(1),
ipcrm(1) and other SysV IPC man pages.