.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.43) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" ======================================================================== .\" .IX Title "Perlude 3pm" .TH Perlude 3pm "2023-02-03" "perl v5.36.0" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "SYNOPSIS" .IX Header "SYNOPSIS" If you're used to a unix shell, Windows Powershell or any language coming with the notion of streams, perl could be frustrating as functions like map and grep only works with arrays. .PP The goodness of it is that \f(CW\*(C`|\*(C'\fR is an on demand operator that can easily compose actions on potentially very large amount of data in a very memory and you can control the amount of consummed data in a friendly way. .PP Perlude gives a better \f(CW\*(C`|\*(C'\fR to Perl: as it works on scalars which can be both strings (like unix shell), numbers or references (like powershell). .PP In Perlude::Tutorial i show examples .PP The big difference is there is no \f(CW\*(C`|\*(C'\fR operator, so the generator is used as function parameter instead of lhs of the pipe (still, the ease of composition remains). So the perlude notation of .PP .Vb 1 \& seq 1000 | sed 5q .Ve .PP is .PP .Vb 1 \& take 5, range 1, 1000 .Ve .PP this code returns a new iterator you want to consume, maybe to fold it in a array, maybe to act on each lastly generated element with the keyword \f(CW\*(C`now\*(C'\fR (as \*(L"now, compute things you learnt to compute\*(R"). .PP .Vb 2 \& my @five = fold take 5, range 1, 1000; \& map {say} take 5, range 1, 1000; .Ve .PP a classical, memory aggressive, Perl code would be .PP .Vb 1 \& map {say} (1..1000)[0..4] .Ve .PP Note that .PP .Vb 1 \& map {say} (1..4)[0..1000] .Ve .PP is an error when .PP .Vb 1 \& now {say} take 1000, range 1,4 .Ve .PP Perlude stole some keywords from the Haskell Prelude (mainly) to make iterators easy to combine and consume. .PP .Vb 1 \& map say, grep /foo/, ; .Ve .PP Perlude provides \*(L"streamed\*(R" counterpart where a stream is a set (whole or partial) of results an iterator can return. .PP .Vb 1 \& now {say} filter {/foo/} lines \e*STDIN; .Ve .PP Now we'll define the concepts under Perlude. the functions provided are in the next section. .SS "an iterator" .IX Subsection "an iterator" is a function reference that can produce a list of at least one element at each calls. an exhausted iterator returns an empty list. .PP Counter is a basic example of iterator .PP .Vb 4 \& my $counter = sub { \& state $x = 0; \& $x++ \& }; .Ve .PP If you use Perl before 5.10, you can write .PP .Vb 4 \& my $counter = do { \& my $x = 0; \& sub {$x++} \& }; .Ve .PP (see \*(L"Persistent variables with closures\*(R") in the \f(CW\*(C`perldoc perlsub\*(C'\fR. .SS "an iteration" .IX Subsection "an iteration" one call of an iterator .PP .Vb 1 \& print $counter\->(); .Ve .SS "a stream" .IX Subsection "a stream" the list of all elements an iterator can produce (it may be infinite). .PP the five first elements of the stream of \f(CW$counter\fR (if it wasn't previously used) is .PP .Vb 1 \& my @top5 = map $counter\->(), 1..5; .Ve .PP the perlude counterpart is .PP .Vb 1 \& my @top5 = fold take 5, $counter; .Ve .SS "a generator" .IX Subsection "a generator" is a function that returns an iterator. .PP .Vb 5 \& sub counter ($) { \& my $x = $_[0]; \& # iterator starts here \& sub { $x++ } \& } \& \& my $iterator = counter 1; \& print $iterator\->(); .Ve .SS "a filter" .IX Subsection "a filter" is a function that take an iterator as argument and returns an iterator, applying a behavior to the elements of the stream. .PP such behavior can be removing or adding elements of the stream, exhaust it or applying a function in the elements of it. .PP some filters are Perlude counterparts of the perl \f(CW\*(C`map\*(C'\fR and \f(CW\*(C`grep\*(C'\fR, other can control the way the stream is consumed (think of them as unix shell filters). .SS "a consumer" .IX Subsection "a consumer" filters are about combining things nothing is computed as long as you don't use the stream. consumers actually starts to stream (iterate on) them (think python3 \f(CW\*(C`list()\*(C'\fR or the perl6 \f(CW&eager\fR). .SH "to sumarize" .IX Header "to sumarize" A stream is a list finished by an empty list (which makes sense if you come from a functional language). .PP .Vb 1 \& (2,4,6,8,10,()) .Ve .PP A an iterator is a function that can return the elements of an iterator one by one. A generator is a function that returns the iterator .PP .Vb 9 \& sub from_to { # the generator \& my ( $from, $to ) = @_; \& sub { # the iterator \& return () if $from > $to; \& my $r = $from; \& $from+=2; \& return $r \& } \& } .Ve .PP note that perlude authors are used to implicit notations so we're used to write more like .PP .Vb 5 \& sub { \& return if $from > $to; \& (my $r, $from) = ( $from, $from + 2 ); \& $r; \& } .Ve .PP (see the code of the \f(CW&lines\fR generator) .SH "Examples" .IX Header "Examples" find the first five zsh users .PP .Vb 5 \& my @top5 = \& fold \& take 5, \& filter {/zsh$/} \& lines "/etc/passwd"; .Ve .PP A math example: every elements of fibo below 1000 (1 element a time in memory) .PP .Vb 3 \& use Perlude; \& use strict; \& use warnings; \& \& sub fibo { \& my @seed = @_; \& sub { \& push @seed, $seed[0] + $seed[1]; \& shift @seed \& } \& } \& \& now {say} takeWhile { $_ < 1000 } fibo 1,1; .Ve .PP Used to shell? the Perlude version of .PP .Vb 1 \& yes "happy birthday" | sed 5q .Ve .PP is .PP .Vb 2 \& sub yes ($msg) { sub { $msg } } \& now {say} take 5, yes "happy birthday" .Ve .PP A sysop example: throw your shellscripts away .PP .Vb 3 \& use Perlude; \& use strictures; \& use 5.10.0; \& \& # iterator on a glob matches stolen from Perlude::Sh module \& sub ls { \& my $glob = glob shift; \& my $match; \& sub { \& return $match while $match = <$glob>; \& (); \& } \& } \& \& # show every txt files in /tmp \& now {say} ls "/tmp/*txt \& \& # remove empty files from tmp \& \& now { unlink if \-f && ! \-s } ls "/tmp/*" \& \& # something more reusable/readable ? \& \& sub is_empty_file { \-f && ! \-s } \& sub empty_files_of { filter {is_empty_file} shift } \& sub rm { now {unlink} shift } \& \& rm empty_files_of ls "/tmp/*./txt"; .Ve .SH "Function composition" .IX Header "Function composition" When relevant, i used the Haskell Prelude documentation descriptions and examples. for example, the take documentation comes from . .SH "Functions" .IX Header "Functions" .SS "generators" .IX Subsection "generators" \fIrange \f(CI$begin\fI, [ \f(CI$end\fI, [ \f(CI$step\fI ] ]\fR .IX Subsection "range $begin, [ $end, [ $step ] ]" .PP A range of numbers from \f(CW$begin\fR to \f(CW$end\fR (infinity if \f(CW$end\fR isn't set) \f(CW$step\fR by \f(CW$step\fR. .PP .Vb 3 \& range 5 # from 5 to infinity \& range 5,9 # 5, 6, 7, 8, 9 \& range 5,9,2 # 5, 7, 9 .Ve .PP \fIcycle \f(CI@set\fI\fR .IX Subsection "cycle @set" .PP infinitly loop on a set of values .PP .Vb 1 \& cycle 1,4,7 \& \& # 1,4,7,1,4,7,1,4,7,1,4,7,1,4,7,... .Ve .PP \fIrecords \f(CI$ref\fI\fR .IX Subsection "records $ref" .PP given any kind of ref that implements the \*(L"<>\*(R" iterator, returns a Perlude compliant iterator. .PP .Vb 4 \& now {print if /data/} records do { \& open my $fh,"foo"; \& $fh; \& }; .Ve .PP \fIas_open\fR .IX Subsection "as_open" .PP just easier (yet safer?) to use wrapper on the sub described in \f(CW\*(C`perldoc \-f open\*(C'\fR (also \*(L"open\*(R" in perlfunc). .PP the goal is to have an wrapper on open does a coercion (just return \f(CW@_\fR if nothing to do). so .IP "\(bu" 2 don't carre about prototype (so you can call it with an array, not only a list) .IP "\(bu" 2 return a \s-1FILEHANDLE\s0 instead of having a side effect on the first variable .IP "\(bu" 2 just return a \s-1FILEHANDLE\s0 passed as argument (so it's a coercion from \f(CW@_\fR to an open handler). .Sp .Vb 5 \& open FILEHANDLE \& open EXPR \& open MODE,EXPR \& open MODE,EXPR,LIST \& open MODE,EXPR,REF .Ve .PP \fIlines \f(CI@openargs\fI\fR .IX Subsection "lines @openargs" .PP if \f(CW$openargs[0]\fR is a string, \f(CW&open\fR \f(CW@openargs\fR (nothing done there if it's already a file handler). .PP return an iterator that chomp the records of the open file. .PP so .PP .Vb 1 \& now {say} lines "/etc/passwd" .Ve .PP can be written like .PP .Vb 8 \& now {say} apply { chomp; $_ } do { \& open my $fh, "/etc/passwd"; \& sub { \& return unless defined my $line = <$fh>; \& chomp $line; \& $line; \& } \& } .Ve .SS "filters" .IX Subsection "filters" filters are composition functions that take a stream and returns a modified stream. .PP \fIfilter \f(CI$xs\fI\fR .IX Subsection "filter $xs" .PP the Perlude counterpart of \f(CW\*(C`grep\*(C'\fR. .PP .Vb 1 \& sub odds () { filter { $_ % 2 } shift } .Ve .PP \fIapply\fR .IX Subsection "apply" .PP the Perlude counterpart of \f(CW\*(C`map\*(C'\fR. .PP .Vb 1 \& sub double { apply {$_*2} shift } .Ve .PP \fItake \f(CI$n\fI, \f(CI$xs\fI\fR .IX Subsection "take $n, $xs" .PP take \f(CW$n\fR, applied to a list \f(CW$xs\fR, returns the prefix of \f(CW$xs\fR of length \f(CW$n\fR, or \f(CW$xs\fR itself if \f(CW$n\fR > length \f(CW$xs:\fR .PP .Vb 1 \& sub top10 { take 10, shift } \& \& take 5, range 1, 10 \& # 1, 2, 3, 4, 5, () \& \& take 5, range 1, 3 \& # 1, 2, 3, () .Ve .PP \fItakeWhile \f(CI$predicate\fI, \f(CI$xs\fI\fR .IX Subsection "takeWhile $predicate, $xs" .PP takeWhile, applied to a predicate \f(CW$p\fR and a list \f(CW$xs\fR, returns the longest prefix (possibly empty) of \f(CW$xs\fR of elements that satisfy \f(CW$p\fR .PP .Vb 2 \& takeWhile { 10 > ($_*2) } range 1,5 \& # 1, 2, 3, 4 .Ve .PP \fIdrop \f(CI$n\fI, \f(CI$xs\fI\fR .IX Subsection "drop $n, $xs" .PP drop \f(CW$n\fR \f(CW$xs\fR returns the suffix of \f(CW$xs\fR after the first \f(CW$n\fR elements, or () if \f(CW$n\fR > length \f(CW$xs:\fR .PP .Vb 2 \& drop 3, range 1,5 \& # 4 , 5 \& \& drop 3, range 1,2 \& # () .Ve .PP \fIdropWhile \f(CI$predicate\fI, \f(CI$xs\fI\fR .IX Subsection "dropWhile $predicate, $xs" .PP dropWhile \f(CW$predicate\fR, \f(CW$xs\fR returns the suffix remaining after dropWhile \f(CW$predicate\fR, \f(CW$xs\fR .PP .Vb 3 \& dropWhile { $_ < 3 } unfold [1,2,3,4,5,1,2,3] # [3,4,5,1,2,3] \& dropWhile { $_ < 9 } unfold [1,2,3] # [] \& dropWhile { $_ < 0 } unfold [1,2,3] # [1,2,3] .Ve .SS "misc" .IX Subsection "misc" \fIunfold \f(CI$array\fI\fR .IX Subsection "unfold $array" .PP unfold returns an iterator on the \f(CW$array\fR ref so that every Perlude goodies can be applied. there is no side effect on the referenced array. .PP .Vb 1 \& my @lower = fold takeWhile {/data/} unfold $abstract .Ve .PP see also fold .PP \fIpairs \f(CI$hash\fI\fR .IX Subsection "pairs $hash" .PP returns an iterator on the pairs of \f(CW$hash\fR stored in a 2 items array ref. .PP .Vb 4 \& now { \& my ( $k, $v ) = @$_; \& say "$k : $v"; \& } pairs {qw< a A b B >}; .Ve .PP aims to be equivalent to .PP .Vb 4 \& my $hash = {qw< a A b B >}; \& while ( my ( $k, $v ) = each %$hash ) { \& say "$k : $v"; \& } .Ve .PP except that: .IP "\(bu" 4 pairs can use an anonymous hash .IP "\(bu" 4 can be used in streams .IP "\(bu" 4 i hate the while syntax .SS "consumers" .IX Subsection "consumers" \fInow {actions} \f(CI$xs\fI\fR .IX Subsection "now {actions} $xs" .PP read the \f(CW$xs\fR stream and execute the {actions} block with the returned element as \f(CW$_\fR until the \f(CW$xs\fR stream exhausts. it also returns the last transformed element so that it can be used to foldl. .PP (compare it to perl6 \*(L"eager\*(R" or haskell foldl) .PP \fIfold \f(CI$xs\fI\fR .IX Subsection "fold $xs" .PP returns the array of all the elements computed by \f(CW$xs\fR .PP .Vb 2 \& say join \*(Aq,\*(Aq, take 5, sub { state $x=\-2; $x+=2 } # CODE(0x180bad8) \& say join \*(Aq,\*(Aq, fold take 5, sub { state $x=\-2; $x+=2 } # 0,2,4,6,8 .Ve .PP see also unfold .PP \fInth \f(CI$xs\fI\fR .IX Subsection "nth $xs" .PP returns the nth element of a stream .PP .Vb 2 \& say fold nth 5, sub { state $x=1; $x++ } \& # 5 .Ve .PP \fIchunksOf\fR .IX Subsection "chunksOf" .PP non destructive splice alike (maybe best named as \*(L"traverse\*(R"? haskell name?). you can traverse an array by a group of copies of elements .PP .Vb 3 \& say "@$_" for fold chunksOf 3, [\*(Aqa\*(Aq..\*(Aqf\*(Aq]; \& # a b c \& # d e f .Ve .SS "Composers" .IX Subsection "Composers" \fIconcat \f(CI@streams\fI\fR .IX Subsection "concat @streams" .PP concat takes a list of streams and returns them as a unique one: .PP .Vb 1 \& concat map { unfold [split //] } split /\es*/; .Ve .PP streams every chars of the words of the text .PP \fIconcatC \f(CI$stream_of_streams\fI\fR .IX Subsection "concatC $stream_of_streams" .PP takes a stream of streams \f(CW$stream_of_streams\fR and expose them as a single one. A stream of streams is a steam that returns streams. .PP .Vb 1 \& concatC { take 3, range $_ } lines $fh .Ve .PP take 3 elements from the range started by the values of \f(CW$fh\fR, so if \f(CW$fh\fR contains (5,10), the stream is (5,6,7,10,11,12) .PP \fIconcatM \f(CI$apply\fI, \f(CI$stream\fI\fR .IX Subsection "concatM $apply, $stream" .PP applying \f(CW$apply\fR on each iterations of \f(CW$stream\fR must return a new stream. concatM expose them as a single stream. .PP .Vb 1 \& # ls is a generator for a glob \& \& sub cat { concatM {lines} ls shift } \& cat "/tmp/*.conf" .Ve .SH "Perlude companions" .IX Header "Perlude companions" some modules comes with generators so they are perfect Perlude companions (send me an example if yours does too). .ie n .SH """Path::Iterator::Rule""" .el .SH "\f(CWPath::Iterator::Rule\fP" .IX Header "Path::Iterator::Rule" .Vb 1 \& use aliased qw(Path::Iterator::Rule find); \& \& now {print} \& take 3, \& find\->new \& \-> file \& \-> size(\*(Aq>1k\*(Aq) \& \-> and( sub { \-r } ) \& \-> iter(qw( /tmp )); .Ve .PP you can use \f(CW\*(C`filter\*(C'\fR instead of \f(CW\*(C`and\*(C'\fR: .PP .Vb 7 \& now {print} \& take 3, \& filter {\-r} \& find\->new \& \-> file \& \-> size(\*(Aq>1k\*(Aq) \& \-> iter(qw( /tmp )); .Ve .ie n .SH """Path::Tiny""" .el .SH "\f(CWPath::Tiny\fP" .IX Header "Path::Tiny" .Vb 1 \& use Path::Tiny; \& \& now {print} take 3, path("/etc")\->iterator; \& \& now {print} \& take 3, \& apply {chomp;$_} \& records path("/etc/passwd")\->openr_utf8( {qw( locked 1 )}); .Ve .ie n .SH """curry""" .el .SH "\f(CWcurry\fP" .IX Header "curry" a very friendly way to write iterators. i rewrote the example from the \&\f(CW\*(C`TAP::Parser\*(C'\fR doc: .PP .Vb 1 \& use TAP::Parser; \& \& my $parser = TAP::Parser\->new( { tap => $output } ); \& \& while ( my $result = $parser\->next ) { \& print $result\->as_string; \& } .Ve .PP with Perlude .PP .Vb 6 \& now {print $_\->as_string."\en"} do { \& my $parser = \& TAP::Parser \& \-> new( { tap => path("/tmp/x")\->slurp }); \& sub { $parser\->next // () } \& } .Ve .PP with Perlude and curry .PP .Vb 4 \& now {defined and print $_\->as_string."\en"} \& TAP::Parser \& \-> new( { tap => path("/tmp/x")\->slurp }) \& \-> curry::next; .Ve .SH "TODO / CONTRIBUTONS" .IX Header "TODO / CONTRIBUTONS" feedbacks and contributions are very welcome .PP .Vb 1 \& http://github.com/eiro/p5\-perlude .Ve .IP "\(bu" 4 Improve general quality: doc, have a look on , . .IP "\(bu" 4 Explore test suite to know what isn't well tested. find bugs :) .Sp .Vb 4 \& * see range implementation # what if step 0 ? \& * pairs must support streams and array \& * provide an alternative to takeWhile to return the combo breaker \& * explore AST manipulations for further optimizations .Ve .IP "\(bu" 4 deprecate open_file and lines (or/and find a companion) as it is out of the scope of Perlude and open_file seems scary (anything to avoid the \f(CW\*(C`open\*(C'\fR prototype?). .IP "\(bu" 4 reboot \f(CW\*(C`Perl::builtins\*(C'\fR .Sp remove the hardcoded \f(CW\*(C`f\*(C'\fR namespace and use \f(CW\*(C`use aliased\*(C'\fR instead. .IP "\(bu" 4 ask for BooK and Dolmen if they mind to remove \f(CW\*(C`Perlude::Lazy\*(C'\fR as no one seems to use it anymore. .IP "\(bu" 4 \&\f(CW\*(C`Perlude::XS\*(C'\fR anyone ? .IP "\(bu" 4 Something to revert the callback mechanism: how to provide a generic syntax to use Anyevent driven streams or \*(L"callback to closures\*(R" (for example: Net::LDAP callback to treat entries onfly) .IP "\(bu" 4 provide streamers for common sources \s-1CSV, LDAP, DBI\s0 (see \f(CW\*(C`p5\-csv\-stream\*(C'\fR) .SH "KNOWN BUGS" .IX Header "KNOWN BUGS" not anymore, if you find one, please email bug-Perlude [at] rt.cpan.org. .SH "AUTHORS" .IX Header "AUTHORS" .IP "\(bu" 4 Philippe Bruhat (BooK) .IP "\(bu" 4 Marc Chantreux (eiro) .IP "\(bu" 4 Olivier Mengué (dolmen) .SH "CONTRIBUTORS" .IX Header "CONTRIBUTORS" Burak Gürsoy (cpanization) .SH "ACKNOWLEDGMENTS" .IX Header "ACKNOWLEDGMENTS" .IP "\(bu" 4 Thanks to Nicolas Pouillard and Valentin (#haskell\-fr), i leanrt a lot about streams, laziness, lists and so on. Lazyness.pm was my first attempt. .IP "\(bu" 4 The name \*(L"Perlude\*(R" is an idea from Germain Maurice, the amazing sysop of http://linkfluence.com back to early 2010. .IP "\(bu" 4 Former versions of Perlude used undef as stream terminator. After my talk at the French Perl Workshop 2011, dolmen suggested to use () as stream terminator, which makes sense not only because undef is a value but also because () is the perfect semantic to end a stream. So Book, Dolmen and myself rewrote the entire module from scratch in the hall of the hotel with a bottle of chartreuse and Cognominal. .Sp We also tried some experiments about real laziness, memoization and so on. it becomes clear now that this is hell to implement correctly: use perl6 instead :) .Sp I was drunk and and misspelled Perlude as \*(L"Perl dude\*(R" so Cognominal collected some quotes of \*(L"The Big Lebowski\*(R" and we called ourselves \*(L"the Perl Dudes\*(R". This is way my best remember of peer programming and one of the best moment i shared with my friends mongueurs.