.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.43)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "MCE 3pm"
.TH MCE 3pm "2023-09-29" "perl v5.36.0" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
MCE \- Many\-Core Engine for Perl providing parallel processing capabilities
.SH "VERSION"
.IX Header "VERSION"
This document describes \s-1MCE\s0 version 1.889
.PP
Many-Core Engine (\s-1MCE\s0) for Perl helps enable a new level of performance by
maximizing all available cores.
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
\&\s-1MCE\s0 spawns a pool of workers and therefore does not fork a new process per
each element of data. Instead, \s-1MCE\s0 follows a bank queuing model. Imagine the
line being the data and bank-tellers the parallel workers. \s-1MCE\s0 enhances that
model by adding the ability to chunk the next n elements from the input
stream to the next available worker.
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
This is a simplistic use case of \s-1MCE\s0 running with 5 workers.
.PP
.Vb 1
\& # Construction using the Core API
\&
\& use MCE;
\&
\& my $mce = MCE\->new(
\&    max_workers => 5,
\&    user_func => sub {
\&       my ($mce) = @_;
\&       $mce\->say("Hello from " . $mce\->wid);
\&    }
\& );
\&
\& $mce\->run;
\&
\& # Construction using a MCE model
\&
\& use MCE::Flow max_workers => 5;
\&
\& mce_flow sub {
\&    my ($mce) = @_;
\&    MCE\->say("Hello from " . MCE\->wid);
\& };
.Ve
.PP
The following is a demonstration for parsing a huge log file in parallel.
.PP
.Vb 1
\& use MCE::Loop;
\&
\& MCE::Loop\->init( max_workers => 8, use_slurpio => 1 );
\&
\& my $pattern  = \*(Aqsomething\*(Aq;
\& my $hugefile = \*(Aqvery_huge.file\*(Aq;
\&
\& my @result = mce_loop_f {
\&    my ($mce, $slurp_ref, $chunk_id) = @_;
\&
\&    # Quickly determine if a match is found.
\&    # Process the slurped chunk only if true.
\&
\&    if ($$slurp_ref =~ /$pattern/m) {
\&       my @matches;
\&
\&       # The following is fast on Unix, but performance degrades
\&       # drastically on Windows beyond 4 workers.
\&
\&       open my $MEM_FH, \*(Aq<\*(Aq, $slurp_ref;
\&       binmode $MEM_FH, \*(Aq:raw\*(Aq;
\&       while (<$MEM_FH>) { push @matches, $_ if (/$pattern/); }
\&       close   $MEM_FH;
\&
\&       # Therefore, use the following construction on Windows.
\&
\&       while ( $$slurp_ref =~ /([^\en]+\en)/mg ) {
\&          my $line = $1; # save $1 to not lose the value
\&          push @matches, $line if ($line =~ /$pattern/);
\&       }
\&
\&       # Gather matched lines.
\&
\&       MCE\->gather(@matches);
\&    }
\&
\& } $hugefile;
\&
\& print join(\*(Aq\*(Aq, @result);
.Ve
.PP
The next demonstration loops through a sequence of numbers with MCE::Flow.
.PP
.Vb 1
\& use MCE::Flow;
\&
\& my $N = shift || 4_000_000;
\&
\& sub compute_pi {
\&    my ( $beg_seq, $end_seq ) = @_;
\&    my ( $pi, $t ) = ( 0.0 );
\&
\&    foreach my $i ( $beg_seq .. $end_seq ) {
\&       $t = ( $i + 0.5 ) / $N;
\&       $pi += 4.0 / ( 1.0 + $t * $t );
\&    }
\&
\&    MCE\->gather( $pi );
\& }
\&
\& # Compute bounds only, workers receive [ begin, end ] values
\&
\& MCE::Flow\->init(
\&    chunk_size  => 200_000,
\&    max_workers => 8,
\&    bounds_only => 1
\& );
\&
\& my @ret = mce_flow_s sub {
\&    compute_pi( $_\->[0], $_\->[1] );
\& }, 0, $N \- 1;
\&
\& my $pi = 0.0;  $pi += $_ for @ret;
\&
\& printf "pi = %0.13f\en", $pi / $N;  # 3.1415926535898
.Ve
.SH "CORE MODULES"
.IX Header "CORE MODULES"
Four modules make up the core engine for \s-1MCE.\s0
.IP "MCE::Core" 3
.IX Item "MCE::Core"
This is the \s-1POD\s0 documentation describing the core Many-Core Engine (\s-1MCE\s0) \s-1API.\s0
Go here for help with the various \s-1MCE\s0 options. See also, MCE::Examples
for additional demonstrations.
.IP "MCE::Mutex" 3
.IX Item "MCE::Mutex"
Provides a simple semaphore implementation supporting threads and processes.
Two implementations are provided; one via pipes or socket depending on the
platform and the other using Fcntl.
.IP "MCE::Signal" 3
.IX Item "MCE::Signal"
Provides signal handling, temporary directory creation, and cleanup for \s-1MCE.\s0
.IP "MCE::Util" 3
.IX Item "MCE::Util"
Provides utility functions for \s-1MCE.\s0
.SH "MCE EXTRAS"
.IX Header "MCE EXTRAS"
There are 5 add-on modules for use with \s-1MCE.\s0
.IP "MCE::Candy" 3
.IX Item "MCE::Candy"
Provides a collection of sugar methods and output iterators for preserving
output order.
.IP "MCE::Channel" 3
.IX Item "MCE::Channel"
Introduced in \s-1MCE 1.839,\s0 provides queue-like and two-way communication
capability. Three implementations \f(CW\*(C`Simple\*(C'\fR, \f(CW\*(C`Mutex\*(C'\fR, and \f(CW\*(C`Threads\*(C'\fR are
provided. \f(CW\*(C`Simple\*(C'\fR does not involve locking whereas \f(CW\*(C`Mutex\*(C'\fR and \f(CW\*(C`Threads\*(C'\fR
do locking transparently using \f(CW\*(C`MCE::Mutex\*(C'\fR and \f(CW\*(C`threads\*(C'\fR respectively.
.IP "MCE::Child" 3
.IX Item "MCE::Child"
Also introduced in \s-1MCE 1.839,\s0 provides a threads-like parallelization module
that is compatible with Perl 5.8. It is a fork of MCE::Hobo. The difference
is using a common \f(CW\*(C`MCE::Channel\*(C'\fR object when yielding and joining.
.IP "MCE::Queue" 3
.IX Item "MCE::Queue"
Provides a hybrid queuing implementation for \s-1MCE\s0 supporting normal queues and
priority queues from a single module. MCE::Queue exchanges data via the core
engine to enable queuing to work for both children (spawned from fork) and
threads.
.IP "MCE::Relay" 3
.IX Item "MCE::Relay"
Provides workers the ability to receive and pass information orderly with zero
involvement by the manager process. This module is loaded automatically by
\&\s-1MCE\s0 when specifying the \f(CW\*(C`init_relay\*(C'\fR \s-1MCE\s0 option.
.SH "MCE MODELS"
.IX Header "MCE MODELS"
The \s-1MCE\s0 models are sugar syntax on top of the MCE::Core \s-1API.\s0 Two \s-1MCE\s0 options
(chunk_size and max_workers) are configured automatically. Moreover, spawning
workers and later shutdown occur transparently behind the scene.
.PP
Choosing a \s-1MCE\s0 Model largely depends on the application. It all boils down
to how much automation you need \s-1MCE\s0 to handle transparently. Or if you prefer,
constructing the \s-1MCE\s0 object and running using the core \s-1MCE API\s0 is fine too.
.IP "MCE::Grep" 3
.IX Item "MCE::Grep"
Provides a parallel grep implementation similar to the native grep function.
.IP "MCE::Map" 3
.IX Item "MCE::Map"
Provides a parallel map implementation similar to the native map function.
.IP "MCE::Loop" 3
.IX Item "MCE::Loop"
Provides a parallel for loop implementation.
.IP "MCE::Flow" 3
.IX Item "MCE::Flow"
Like \f(CW\*(C`MCE::Loop\*(C'\fR, but with support for multiple pools of workers. The pool
of workers are configured transparently via the \s-1MCE\s0 \f(CW\*(C`user_tasks\*(C'\fR option.
.IP "MCE::Step" 3
.IX Item "MCE::Step"
Like \f(CW\*(C`MCE::Flow\*(C'\fR, but adds a \f(CW\*(C`MCE::Queue\*(C'\fR object between each pool of
workers. This model, introduced in 1.506, allows one to pass data forward
(left to right) from one sub-task into another with little effort.
.IP "MCE::Stream" 3
.IX Item "MCE::Stream"
This provides an efficient parallel implementation for chaining multiple maps
and greps transparently. Like \f(CW\*(C`MCE::Flow\*(C'\fR and \f(CW\*(C`MCE::Step\*(C'\fR, it too supports
multiple pools of workers. The distinction is that \f(CW\*(C`MCE::Stream\*(C'\fR passes
data from right to left and done for you transparently.
.SH "MISCELLANEOUS"
.IX Header "MISCELLANEOUS"
Miscellaneous additions included with the distribution.
.IP "MCE::Examples" 3
.IX Item "MCE::Examples"
Describes various demonstrations for \s-1MCE\s0 including a Monte Carlo simulation.
.IP "MCE::Subs" 3
.IX Item "MCE::Subs"
Exports functions mapped directly to \s-1MCE\s0 methods; e.g. mce_wid. The module
allows 3 options; :manager, :worker, and :getter.
.SH "REQUIREMENTS"
.IX Header "REQUIREMENTS"
Perl 5.8.0 or later.
.SH "SOURCE AND FURTHER READING"
.IX Header "SOURCE AND FURTHER READING"
The source and examples are hosted at GitHub.
.IP "\(bu" 3
<https://github.com/marioroy/mce\-perl>
.IP "\(bu" 3
<https://github.com/marioroy/mce\-examples>
.SH "SEE ALSO"
.IX Header "SEE ALSO"
Refer to the MCE::Core documentation where the \s-1API\s0 is described.
.PP
\&\f(CW\*(C`MCE::Shared\*(C'\fR provides data sharing capabilities for \f(CW\*(C`MCE\*(C'\fR. It includes
\&\f(CW\*(C`MCE::Hobo\*(C'\fR for running code asynchronously with the \s-1IPC\s0 handled by the
shared-manager process.
.IP "\(bu" 3
MCE::Shared
.IP "\(bu" 3
MCE::Hobo
.SH "AUTHOR"
.IX Header "AUTHOR"
Mario E. Roy, <marioeroy AT gmail DOT com>
.SH "COPYRIGHT AND LICENSE"
.IX Header "COPYRIGHT AND LICENSE"
Copyright (C) 2012\-2023 by Mario E. Roy
.PP
\&\s-1MCE\s0 is released under the same license as Perl.
.PP
See <https://dev.perl.org/licenses/> for more information.