.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.43)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "FastRaw 3pm"
.TH FastRaw 3pm "2023-06-17" "perl v5.36.0" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
PDL::IO::FastRaw \-\- A simple, fast and convenient io format for PerlDL.
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
.Vb 2
\& use PDL;
\& use PDL::IO::FastRaw;
\&
\& writefraw($pdl,"fname");         # write a raw file
\&
\& $pdl2 = readfraw("fname");       # read a raw file
\& $pdl2 = PDL\->readfraw("fname");
\&
\& gluefraw($pdlx, "fname");        # append to existing file
\& $pdlx\->gluefraw("fname");
\&
\& $pdl3 = mapfraw("fname2",{ReadOnly => 1}); # mmap a file, don\*(Aqt read yet
\&
\& $pdl4 = maptextfraw("fname3",{...}); # map a text file into a 1\-D pdl.
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
This is a very simple and fast io format for PerlDL.
The disk data consists of two files, a header metadata file
in \s-1ASCII\s0 and a binary file consisting simply of consecutive
bytes, shorts or whatever.
.PP
It is hoped that this will not only make for a simple PerlDL module
for saving and retrieving these files but also make it easy
for other programs to use these files.
.PP
The format of the \s-1ASCII\s0 header is simply
.PP
.Vb 3
\&        <typeid>
\&        <ndims>
\&        <dim0> <dim1> ...
.Ve
.PP
You should probably stick with the default header name.  You may want
to specify your own header, however, such as when you have a large
collection of data files with identical dimensions and data types.
Under these circumstances, simply specify the \f(CW\*(C`Header\*(C'\fR option in the
options hash.
.PP
The binary files are in general
\&\s-1NOT\s0 interchangeable between different architectures since the binary
file is simply dumped from the memory region of the ndarray.
This is what makes the approach efficient.
.PP
It is also possible to mmap the file which can give a large
speedup in certain situations as well as save a lot of memory
by using a disk file as virtual memory. When a file is mapped,
parts of it are read only as they are accessed in the memory
(or as the kernel decides: if you are reading the pages in order,
it may well preread some for you).
.PP
Note that memory savings and copy-on-write are operating-system
dependent \- see Core.xs and your operating system documentation
for exact semantics of whatever. Basically, if you write to a
mmapped file without \f(CW\*(C`ReadOnly\*(C'\fR, the change will be reflected
in the file immediately. \f(CW\*(C`ReadOnly\*(C'\fR doesn't really make it impossible
to write to the ndarray but maps the memory privately so the file
will not be changed when you change the ndarray. Be aware though
that mmapping a 40Mb file without \f(CW\*(C`ReadOnly\*(C'\fR spends no virtual
memory but with \f(CW\*(C`ReadOnly\*(C'\fR it does reserve 40Mb.
.SS "Example: Converting \s-1ASCII\s0 to raw"
.IX Subsection "Example: Converting ASCII to raw"
You have a whole slew of data files in \s-1ASCII\s0 from an experiment
that you ran in your lab.  You're still tweaking the analysis
and plots, so you'd like if your data could load as fast as
possible.  Eventually you'll read the data into your scripts
using \f(CW\*(C`readfraw\*(C'\fR, but the first thing you might do is create
a script that converts all the data files to raw files:
.PP
.Vb 7
\& #!/usr/bin/perl
\& # Assumes that the data files end with a .asc or .dat extension
\& # and saves the raw file output with a .bdat extension.
\& # call with
\& #  >./convert_to_raw.pl file1.dat file2.dat ...
\& # or
\& #  >./convert_to_raw.pl *.dat
\& 
\& use PDL;
\& use PDL::IO::FastRaw;  # for saving raw files
\& use PDL::IO::Misc;             # for reading ASCII files with rcols
\& while(shift) {                 # run through the entire supplied list of file names
\&         ($newName = $_) =~ s/\e.(asc|dat)/.bdat/;
\&         print "Saving contents of $_ to $newName\en";
\&         $data = rcols($_);
\&         writefraw($data, $newName);
\& }
.Ve
.SS "Example: readfraw"
.IX Subsection "Example: readfraw"
Now that you've gotten your data into a raw file format, you can
start working on your analysis scripts.  If you scripts used \f(CW\*(C`rcols\*(C'\fR
in the past, the reading portion of the script should go much,
much faster now:
.PP
.Vb 6
\& #!/usr/bin/perl
\& # My plotting script.
\& # Assume I\*(Aqve specified the files to plot on the command line like
\& #  >./plot_script.pl file1.bdat file2.bdat ...
\& # or
\& #  >./plot_script.pl *.bdat
\& 
\& use PDL;
\& use PDL::IO::FastRaw;
\& while(shift) {                 # run through the entire supplied list of file names
\&         $data = readfraw($_);
\&         my_plot_func($data);
\& }
.Ve
.SS "Example: Custom headers"
.IX Subsection "Example: Custom headers"
In the first example, I allow \f(CW\*(C`writefraw\*(C'\fR to use the standard header
file name, which would be \f(CW\*(C`file.bdat.hdr\*(C'\fR.  However, I often measure
time series that have identical length, so all of those header files
are redundant.  To fix that, I simply pass the Header option to the
\&\f(CW\*(C`writefraw\*(C'\fR command.  A modified script would look like this:
.PP
.Vb 5
\& #!/usr/bin/perl
\& # Assumes that the data files end with a .asc or .dat extension
\& # and saves the raw file output with a .bdat extension.
\& # call with
\& #  >./convert_to_raw.pl [\-hHeaderFile] <fileglob> [\-hHeaderFile] <fileglob> ...
\& 
\& use PDL;
\& use PDL::IO::FastRaw;  # for saving raw files
\& use PDL::IO::Misc;             # for reading ASCII files with rcols
\& my $header_file = undef;
\& CL_OPTION: while($_ = shift @ARGV) {   # run through the entire list of command\-line options
\&         if(/\-h(.*)/) {
\&                 $header_file = $1;
\&                 next CL_OPTION;
\&         }
\&         ($newName = $_) =~ s/\e.(asc|dat)/.bdat/;
\&         print "Saving contents of $_ to $newName\en";
\&         $data = rcols($_);
\&         writefraw($data, $newName, {Header => $header_file});
\& }
.Ve
.PP
Modifying the read script is left as an exercise for the reader.  :]
.SS "Example: Using mapfraw"
.IX Subsection "Example: Using mapfraw"
Sometimes you'll want to use \f(CW\*(C`mapfraw\*(C'\fR rather than the read/write
functions.  In fact, the original author of the module doesn't
use the read/write functions anymore, prefering to always use
\&\f(CW\*(C`mapfraw\*(C'\fR.  How would you go about doing this?
.PP
Assuming you've already saved your data into the raw format, the
only change you would have to make to the script in example 2 would
be to change the call to \f(CW\*(C`readfraw\*(C'\fR to \f(CW\*(C`mapfraw\*(C'\fR.  That's it.
You will probably see differences in performance, though I (David
Mertens) couldn't tell you about them because I haven't played
around with \f(CW\*(C`mapfraw\*(C'\fR much myself.
.PP
What if you eschew the use of \f(CW\*(C`writefraw\*(C'\fR and prefer to only use
\&\f(CW\*(C`mapfraw\*(C'\fR?  How would you save your data to a raw format?  In that
case, you would have to create a \f(CW\*(C`mapfraw\*(C'\fR ndarray with the correct
dimensions first using
.PP
.Vb 1
\& $ndarray_on_hd = mapfraw(\*(Aqfname\*(Aq, {Creat => 1, Dims => [dim1, dim2, ...]});
.Ve
.PP
Note that you must specify the dimensions and you must tell
\&\f(CW\*(C`mapfraw\*(C'\fR to create the new ndarray for you by setting the
\&\f(CW\*(C`Creat\*(C'\fR option to a true value, not \f(CW\*(C`Create\*(C'\fR (note the missing
final 'e').
.SH "FUNCTIONS"
.IX Header "FUNCTIONS"
.SS "writefraw"
.IX Subsection "writefraw"
Write a raw format binary file
.PP
.Vb 2
\& writefraw($pdl,"fname");
\& writefraw($pdl,"fname", {Header => \*(Aqheaderfname\*(Aq});
.Ve
.PP
The \f(CW\*(C`writefraw\*(C'\fR command
supports the following option:
.IP "Header" 8
.IX Item "Header"
Specify the header file name.
.SS "readfraw"
.IX Subsection "readfraw"
Read a raw format binary file
.PP
.Vb 3
\& $pdl2 = readfraw("fname");
\& $pdl2 = PDL\->readfraw("fname");
\& $pdl2 = readfraw("fname", {Header => \*(Aqheaderfname\*(Aq});
.Ve
.PP
The \f(CW\*(C`readfraw\*(C'\fR command
supports the following option:
.IP "Header" 8
.IX Item "Header"
Specify the header file name.
.SS "gluefraw"
.IX Subsection "gluefraw"
Append a single data item to an existing binary file written by
\&\*(L"writefraw\*(R". Error if dims not compatible with existing data.
.PP
.Vb 1
\&  gluefraw($file, $pdl[, $opts]);
.Ve
.SS "mapfraw"
.IX Subsection "mapfraw"
Memory map a raw format binary file (see the module docs also)
.PP
.Vb 1
\& $pdl3 = mapfraw("fname2",{ReadOnly => 1});
.Ve
.PP
The \f(CW\*(C`mapfraw\*(C'\fR command
supports the following options (not all combinations make sense):
.IP "Dims, Datatype" 8
.IX Item "Dims, Datatype"
If creating a new file or if you want to specify your own header
data for the file, you can give an array reference and a scalar,
respectively.
.IP "Creat" 8
.IX Item "Creat"
Create the file. Also writes out a header for the file.
.IP "Trunc" 8
.IX Item "Trunc"
Set the file size. Automatically enabled with \f(CW\*(C`Creat\*(C'\fR. \s-1NOTE:\s0 This also
clears the file to all zeroes.
.IP "ReadOnly" 8
.IX Item "ReadOnly"
Disallow writing to the file.
.IP "Header" 8
.IX Item "Header"
Specify the header file name.
.SS "maptextfraw"
.IX Subsection "maptextfraw"
Memory map a text file (see the module docs also).
.PP
Note that this function maps the raw format so if you are
using an operating system which does strange things to e.g.
line delimiters upon reading a text file, you get the raw (binary)
representation.
.PP
The file doesn't really need to be text but it is just mapped
as one large binary chunk.
.PP
This function is just a convenience wrapper which firsts \f(CW\*(C`stat\*(C'\fRs
the file and sets the dimensions and datatype.
.PP
.Vb 1
\& $pdl4 = maptextfraw("fname", {options}
.Ve
.PP
The options other than Dims, Datatype of \f(CW\*(C`mapfraw\*(C'\fR are
supported.
.SH "BUGS"
.IX Header "BUGS"
Should be documented better. \f(CW\*(C`writefraw\*(C'\fR and \f(CW\*(C`readfraw\*(C'\fR should
also have options (the author nowadays only uses \f(CW\*(C`mapfraw\*(C'\fR ;)
.SH "AUTHOR"
.IX Header "AUTHOR"
Copyright (C) Tuomas J. Lukka 1997.
All rights reserved. There is no warranty. You are allowed
to redistribute this software / documentation under certain
conditions. For details, see the file \s-1COPYING\s0 in the \s-1PDL\s0
distribution. If this file is separated from the \s-1PDL\s0 distribution,
the copyright notice should be included in the file.