.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.43) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "MRCAL 1" .TH MRCAL 1 "2023-01-30" "mrcal 2.2" "mrcal: camera projection, calibration toolkit" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" mrcal\-stereo \- Stereo processing .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 10 \& $ mrcal\-stereo \e \& \-\-az\-fov\-deg 90 \e \& \-\-el\-fov\-deg 90 \e \& \-\-sgbm\-block\-size 5 \e \& \-\-sgbm\-p1 600 \e \& \-\-sgbm\-p2 2400 \e \& \-\-sgbm\-uniqueness\-ratio 5 \e \& \-\-sgbm\-disp12\-max\-diff 1 \e \& \-\-sgbm\-speckle\-window\-size 200 \e \& \-\-sgbm\-speckle\-range 2 \e \& \-\-outdir /tmp \e \& left.cameramodel right.cameramodel \e \& left.jpg right.jpg \& \& Processing left.jpg and right.jpg \& Wrote \*(Aq/tmp/rectified0.cameramodel\*(Aq \& Wrote \*(Aq/tmp/rectified1.cameramodel\*(Aq \& Wrote \*(Aq/tmp/left\-rectified.png\*(Aq \& Wrote \*(Aq/tmp/right\-rectified.png\*(Aq \& Wrote \*(Aq/tmp/left\-disparity.png\*(Aq \& Wrote \*(Aq/tmp/left\-range.png\*(Aq \& Wrote \*(Aq/tmp/points\-cam0.vnl\*(Aq .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" Given a pair of calibrated cameras and pairs of images captured by these cameras, this tool runs the whole stereo processing sequence to produce disparity and range images and a point cloud array. .PP mrcal functions are used to construct the rectified system. Currently only the OpenCV \s-1SGBM\s0 routine is available to perform stereo matching, but more options will be made available with time. .PP The commandline arguments to configure the \s-1SGBM\s0 matcher (\-\-sgbm\-...) map to the corresponding OpenCV APIs. Omitting an \-\-sgbm\-... argument will result in the defaults being used in the cv2.\fBStereoSGBM_create()\fR call. Usually the cv2.\fBStereoSGBM_create()\fR defaults are terrible, and produce a disparity map that isn't great. The \-\-sgbm\-... arguments in the synopsis above are a good start to get usable stereo. .PP The rectified system is constructed with the axes .PP \&\- x: from the origin of the first camera to the origin of the second camera (the baseline direction) .PP \&\- y: completes the system from x,z .PP \&\- z: the mean \*(L"forward\*(R" direction of the two input cameras, with the component parallel to the baseline subtracted off .PP The active window in this system is specified using a few parameters. These refer to .PP \&\- the \*(L"azimuth\*(R" (or \*(L"az\*(R"): the direction along the baseline: rectified x axis .PP \&\- the \*(L"elevation\*(R" (or \*(L"el\*(R"): the direction across the baseline: rectified y axis .PP The rectified field of view is given by the arguments \-\-az\-fov\-deg and \&\-\-el\-fov\-deg. At this time there's no auto-detection logic, and these must be given. Changing these is a \*(L"zoom\*(R" operation. .PP To pan the stereo system, pass \-\-az0\-deg and/or \-\-el0\-deg. These specify the center of the rectified images, and are optional. .PP Finally, the resolution of the rectified images is given with \-\-pixels\-per\-deg. This is optional, and defaults to the resolution of the first input image. If we want to scale the input resolution, pass a value <0. For instance, to generate rectified images at half the first-input-image resolution, pass \&\-\-pixels\-per\-deg=\-0.5. Note that the Python argparse has a problem with negative numbers, so \*(L"\-\-pixels\-per\-deg \-0.5\*(R" does not work. .PP The input images are specified by a pair of globs, so we can process many images with a single call. Each glob is expanded, and the filenames are sorted. The resulting lists of files are assumed to match up. .PP There are several modes of operation: .PP \&\- No images given: we compute the rectified system only, writing the models to disk .PP \&\- No \-\-viz argument given: we compute the rectified system and the disparity, and we write all output as images on disk .PP \&\- \-\-viz geometry: we compute the rectified system, and display its geometry as a plot. No rectification is computed, and the images aren't used, and don't need to be passed in .PP \&\- \-\-viz stereo: compute the rectified system and the disparity. We don't write anything to disk initially, but we invoke an interactive visualization tool to display the results. Requires pyFLTK (homepage: and GL_image_display (homepage: .SH "OPTIONS" .IX Header "OPTIONS" .SS "\s-1POSITIONAL ARGUMENTS\s0" .IX Subsection "POSITIONAL ARGUMENTS" .Vb 5 \& models Camera models representing cameras used to capture the \& images. Both intrinsics and extrinsics are used \& images The image globs to use for the stereo. If omitted, we \& only write out the rectified models. If given, exactly \& two image globs must be given .Ve .SS "\s-1OPTIONAL ARGUMENTS\s0" .IX Subsection "OPTIONAL ARGUMENTS" .Vb 10 \& \-h, \-\-help show this help message and exit \& \-\-az\-fov\-deg AZ_FOV_DEG \& The field of view in the azimuth direction, in \& degrees. There\*(Aqs no auto\-detection at this time, so \& this argument is required (unless \-\-already\-rectified) \& \-\-el\-fov\-deg EL_FOV_DEG \& The field of view in the elevation direction, in \& degrees. There\*(Aqs no auto\-detection at this time, so \& this argument is required (unless \-\-already\-rectified) \& \-\-az0\-deg AZ0_DEG The azimuth center of the rectified images. "0" means \& "the horizontal center of the rectified system is the \& mean forward direction of the two cameras projected to \& lie perpendicular to the baseline". If omitted, we \& align the center of the rectified system with the \& center of the two cameras\*(Aq views \& \-\-el0\-deg EL0_DEG The elevation center of the rectified system. "0" \& means "the vertical center of the rectified system \& lies along the mean forward direction of the two \& cameras" Defaults to 0. \& \-\-pixels\-per\-deg PIXELS_PER_DEG \& The resolution of the rectified images. This is either \& a whitespace\-less, comma\-separated list of two values \& (az,el) or a single value to be applied to both axes. \& If a resolution of >0 is requested, the value is used \& as is. If a resolution of <0 is requested, we use this \& as a scale factor on the resolution of the first input \& image. For instance, to downsample by a factor of 2, \& pass \-0.5. By default, we use \-1 for both axes: the \& resolution of the input image at the center of the \& rectified system. \& \-\-rectification {LENSMODEL_PINHOLE,LENSMODEL_LATLON} \& The lens model to use for rectification. Currently two \& models are supported: LENSMODEL_LATLON (the default) \& and LENSMODEL_PINHOLE. Pinhole stereo works badly for \& wide lenses and suffers from varying angular \& resolution across the image. LENSMODEL_LATLON \& rectification uses a transverse equirectangular \& projection, and does not suffer from these effects. It \& is thus the recommended model \& \-\-already\-rectified If given, assume the given models and images already \& represent a rectified system. This will be checked, \& and the models will be used as\-is if the checks pass \& \-\-clahe If given, apply CLAHE equalization to the images prior \& to the stereo matching. If \-\-already\-rectified, we \& still apply this equalization, if requested. Requires \& \-\-force\-grayscale \& \-\-force\-grayscale If given, convert the images to grayscale prior to \& doing anything else with them. By default, read the \& images in their default format, and pass those \& posibly\-color images to all the processing steps. \& Required if \-\-clahe \& \-\-viz {geometry,stereo} \& If given, we visualize either the rectified geometry \& or the stereo results. If \-\-viz geometry: we construct \& the rectified stereo system, but instead of continuing \& with the stereo processing, we render the geometry of \& the stereo world; the images are ignored in this mode. \& If \-\-viz stereo: we launch an interactive graphical \& tool to examine the rectification and stereo matching \& results; the Fl_Gl_Image_Widget Python library must be \& available \& \-\-axis\-scale AXIS_SCALE \& Used if \-\-viz geometry. Scale for the camera axes. By \& default a reasonable default is chosen (see \& mrcal.show_geometry() for the logic) \& \-\-title TITLE Used if \-\-viz geometry. Title string for the plot \& \-\-hardcopy HARDCOPY Used if \-\-viz geometry. Write the output to disk, \& instead of making an interactive plot. The output \& filename is given in the option \& \-\-terminal TERMINAL Used if \-\-viz geometry. The gnuplotlib terminal. The \& default is almost always right, so most people don\*(Aqt \& need this option \& \-\-set SET Used if \-\-viz geometry. Extra \*(Aqset\*(Aq directives to pass \& to gnuplotlib. May be given multiple times \& \-\-unset UNSET Used if \-\-viz geometry. Extra \*(Aqunset\*(Aq directives to \& pass to gnuplotlib. May be given multiple times \& \-\-force, \-f By default existing files are not overwritten. Pass \& \-\-force to overwrite them without complaint \& \-\-outdir OUTDIR Directory to write the output into. If omitted, we \& user the current directory \& \-\-tag TAG String to use in the output filenames. Non\-specific \& output filenames if omitted \& \-\-disparity\-range DISPARITY_RANGE DISPARITY_RANGE \& The disparity limits to use in the search, in pixels. \& Two integers are expected: MIN_DISPARITY \& MAX_DISPARITY. Completely arbitrarily, we default to \& MIN_DISPARITY=0 and MAX_DISPARITY=100 \& \-\-valid\-intrinsics\-region \& If given, annotate the image with its valid\-intrinsics \& region. This will end up in the rectified images, and \& make it clear where successful matching shouldn\*(Aqt be \& expected \& \-\-range\-image\-limits RANGE_IMAGE_LIMITS RANGE_IMAGE_LIMITS \& The nearest,furthest range to encode in the range \& image. Defaults to 1,1000, arbitrarily \& \-\-stereo\-matcher {SGBM,ELAS} \& The stereo\-matching method. By default we use the \& "SGBM" method from OpenCV. libelas isn\*(Aqt always \& available, and must be enabled at compile\-time by \& setting USE_LIBELAS=1 during the build \& \-\-sgbm\-block\-size SGBM_BLOCK_SIZE \& A parameter for the OpenCV SGBM matcher. If omitted, 5 \& is used \& \-\-sgbm\-p1 SGBM_P1 A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-p2 SGBM_P2 A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-disp12\-max\-diff SGBM_DISP12_MAX_DIFF \& A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-pre\-filter\-cap SGBM_PRE_FILTER_CAP \& A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-uniqueness\-ratio SGBM_UNIQUENESS_RATIO \& A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-speckle\-window\-size SGBM_SPECKLE_WINDOW_SIZE \& A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-speckle\-range SGBM_SPECKLE_RANGE \& A parameter for the OpenCV SGBM matcher. If omitted, \& the OpenCV default is used \& \-\-sgbm\-mode {SGBM,HH,HH4,SGBM_3WAY} \& A parameter for the OpenCV SGBM matcher. Must be one \& of (\*(AqSGBM\*(Aq,\*(AqHH\*(Aq,\*(AqHH4\*(Aq,\*(AqSGBM_3WAY\*(Aq). If omitted, the \& OpenCV default (SGBM) is used \& \-\-write\-point\-cloud If given, we write out the point cloud as a .ply file. \& Each point is reported in the reference coordinate \& system, colored with the nearest\-neighbor color of the \& camera0 image. This is disabled by default because \& this is potentially a very large file .Ve .SH "REPOSITORY" .IX Header "REPOSITORY" .SH "AUTHOR" .IX Header "AUTHOR" Dima Kogan, \f(CW\*(C`\*(C'\fR .SH "LICENSE AND COPYRIGHT" .IX Header "LICENSE AND COPYRIGHT" Copyright (c) 2017\-2021 California Institute of Technology (\*(L"Caltech\*(R"). U.S. Government sponsorship acknowledged. All rights reserved. .PP Licensed under the Apache License, Version 2.0 (the \*(L"License\*(R"); You may obtain a copy of the License at .PP .Vb 1 \& http://www.apache.org/licenses/LICENSE\-2.0 .Ve