NAME¶
MCE - Many-core Engine for Perl. Provides parallel processing capabilities.
VERSION¶
This document describes MCE version 1.517
DESCRIPTION¶
Many-core Engine (MCE) for Perl helps enable a new level of performance by
maximizing all available cores. MCE spawns a pool of workers and therefore
does not fork a new process per each element of data. Instead, MCE follows a
bank queuing model. Imagine the line being the data and bank-tellers the
parallel workers. MCE enhances that model by adding the ability to chunk the
next n elements from the input stream to the next available worker.
Chunking and input data are optional in MCE. One may use MCE to run many workers
in parallel without specifying input data.
CORE MODULES¶
Three modules make up the core engine for MCE.
- MCE::Core
- Provides the core API for Many-core Engine.
- MCE::Signal
- Temporary directory creation/cleanup & signal handling.
- MCE::Util
- Public and private utility functions.
MCE ADDONS¶
Currently, there are 2 addon modules.
- MCE::Queue
- Provides a hybrid queuing implementation for MCE supporting normal queues
and priority queues from a single module. MCE::Queue exchanges data via
the core engine to enable queueing to work for both children (spawned from
fork) and threads.
- MCE::Subs
- Exports functions mapped directly to MCE's methods, e.g. mce_wid. The
module allows 3 options; :manager, :worker, :getter.
MCE MODELS¶
MCE 1.5 includes 6 models. They configure the MCE instance and tune chunk_size
and max_workers for you. Spawning and shutdown is also automatic. In essence,
the models take Many-core Engine to a new level for ease of use.
- MCE::Loop
- Provides a parallel loop utilizing MCE for building creative loops.
- MCE::Flow
- A parallel flow model for building creative applications. This makes use
of user_tasks in MCE. The author has full control when utilizing this
model. MCE::Flow is similar to MCE::Loop, but allows for multiple code
blocks to run simultaneously with a slight change to syntax.
- MCE::Grep
- Provides a parallel grep implementation similar to the native grep
function.
- MCE::Map
- Provides a parallel map model similar to the native map function.
- MCE::Step
- Provides a parallel step implementation utilizing MCE::Queue between user
tasks. MCE::Step is a spin off from MCE::Flow with a touch of MCE::Stream.
This model, introduced in 1.506, allows one to pass data from one sub-task
into the next transparently.
- MCE::Stream
- Provides an efficient parallel implementation for chaining multiple maps
and greps together through user_tasks and MCE::Queue. Like with MCE::Flow,
MCE::Stream can run multiple code blocks simultaneously with a slight
change to syntax from MCE::Map and MCE::Grep.
EXAMPLES¶
A placeholder for the examples included with the distribution.
- MCE::Examples
- Describes various demonstations for MCE including a Monte Carlo
simulation.
REQUIREMENTS¶
Perl 5.8.0 or later. PDL::IO::Storable is required in scripts running PDL and
using MCE for parallelization.
SOURCE¶
The source is hosted at <
http://code.google.com/p/many-core-engine-perl/>
AUTHOR¶
Mario E. Roy, <marioeroy AT gmail DOT com>
COPYRIGHT AND LICENSE¶
Copyright (C) 2012-2014 by Mario E. Roy
This program is free software; you can redistribute it and/or modify it under
the terms of either: the GNU General Public License as published by the Free
Software Foundation; or the Artistic License.
See <
http://dev.perl.org/licenses/> for more information.