169 lines
6.7 KiB
Markdown
169 lines
6.7 KiB
Markdown
|
High level stitching API (Stitcher class) {#tutorial_stitcher}
|
|||
|
|
=========================================
|
|||
|
|
|
|||
|
|
@tableofcontents
|
|||
|
|
|
|||
|
|
@prev_tutorial{tutorial_hdr_imaging}
|
|||
|
|
@next_tutorial{tutorial_background_subtraction}
|
|||
|
|
|
|||
|
|
| | |
|
|||
|
|
| -: | :- |
|
|||
|
|
| Original author | Jiri Horner |
|
|||
|
|
| Compatibility | OpenCV >= 3.2 |
|
|||
|
|
|
|||
|
|
Goal
|
|||
|
|
----
|
|||
|
|
|
|||
|
|
In this tutorial you will learn how to:
|
|||
|
|
|
|||
|
|
- use the high-level stitching API for stitching provided by
|
|||
|
|
- @ref cv::Stitcher
|
|||
|
|
- learn how to use preconfigured Stitcher configurations to stitch images
|
|||
|
|
using different camera models.
|
|||
|
|
|
|||
|
|
Code
|
|||
|
|
----
|
|||
|
|
|
|||
|
|
This tutorial code's is shown lines below. You can also download it from
|
|||
|
|
[here](https://github.com/opencv/opencv/tree/4.x/samples/cpp/stitching.cpp).
|
|||
|
|
|
|||
|
|
@include samples/cpp/stitching.cpp
|
|||
|
|
|
|||
|
|
Explanation
|
|||
|
|
-----------
|
|||
|
|
|
|||
|
|
The most important code part is:
|
|||
|
|
|
|||
|
|
@snippet cpp/stitching.cpp stitching
|
|||
|
|
|
|||
|
|
A new instance of stitcher is created and the @ref cv::Stitcher::stitch will
|
|||
|
|
do all the hard work.
|
|||
|
|
|
|||
|
|
@ref cv::Stitcher::create can create stitcher in one of the predefined
|
|||
|
|
configurations (argument `mode`). See @ref cv::Stitcher::Mode for details. These
|
|||
|
|
configurations will setup multiple stitcher properties to operate in one of
|
|||
|
|
predefined scenarios. After you create stitcher in one of predefined
|
|||
|
|
configurations you can adjust stitching by setting any of the stitcher
|
|||
|
|
properties.
|
|||
|
|
|
|||
|
|
If you have cuda device @ref cv::Stitcher can be configured to offload certain
|
|||
|
|
operations to GPU. If you prefer this configuration set `try_use_gpu` to true.
|
|||
|
|
OpenCL acceleration will be used transparently based on global OpenCV settings
|
|||
|
|
regardless of this flag.
|
|||
|
|
|
|||
|
|
Stitching might fail for several reasons, you should always check if
|
|||
|
|
everything went good and resulting pano is stored in `pano`. See
|
|||
|
|
@ref cv::Stitcher::Status documentation for possible error codes.
|
|||
|
|
|
|||
|
|
Camera models
|
|||
|
|
-------------
|
|||
|
|
|
|||
|
|
There are currently 2 camera models implemented in stitching pipeline.
|
|||
|
|
|
|||
|
|
- _Homography model_ expecting perspective transformations between images
|
|||
|
|
implemented in @ref cv::detail::BestOf2NearestMatcher cv::detail::HomographyBasedEstimator
|
|||
|
|
cv::detail::BundleAdjusterReproj cv::detail::BundleAdjusterRay
|
|||
|
|
- _Affine model_ expecting affine transformation with 6 DOF or 4 DOF implemented in
|
|||
|
|
@ref cv::detail::AffineBestOf2NearestMatcher cv::detail::AffineBasedEstimator
|
|||
|
|
cv::detail::BundleAdjusterAffine cv::detail::BundleAdjusterAffinePartial cv::AffineWarper
|
|||
|
|
|
|||
|
|
Homography model is useful for creating photo panoramas captured by camera,
|
|||
|
|
while affine-based model can be used to stitch scans and object captured by
|
|||
|
|
specialized devices.
|
|||
|
|
|
|||
|
|
@note
|
|||
|
|
Certain detailed settings of @ref cv::Stitcher might not make sense. Especially
|
|||
|
|
you should not mix classes implementing affine model and classes implementing
|
|||
|
|
Homography model, as they work with different transformations.
|
|||
|
|
|
|||
|
|
Try it out
|
|||
|
|
----------
|
|||
|
|
|
|||
|
|
If you enabled building samples you can found binary under
|
|||
|
|
`build/bin/cpp-example-stitching`. This example is a console application, run it without
|
|||
|
|
arguments to see help. `opencv_extra` provides some sample data for testing all available
|
|||
|
|
configurations.
|
|||
|
|
|
|||
|
|
to try panorama mode run:
|
|||
|
|
```
|
|||
|
|
./cpp-example-stitching --mode panorama <path to opencv_extra>/testdata/stitching/boat*
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
to try scans mode run (dataset from home-grade scanner):
|
|||
|
|
```
|
|||
|
|
./cpp-example-stitching --mode scans <path to opencv_extra>/testdata/stitching/newspaper*
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
or (dataset from professional book scanner):
|
|||
|
|
```
|
|||
|
|
./cpp-example-stitching --mode scans <path to opencv_extra>/testdata/stitching/budapest*
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
|
|||
|
|
@note
|
|||
|
|
Examples above expects POSIX platform, on windows you have to provide all files names explicitly
|
|||
|
|
(e.g. `boat1.jpg` `boat2.jpg`...) as windows command line does not support `*` expansion.
|
|||
|
|
|
|||
|
|
Stitching detailed (python opencv >4.0.1)
|
|||
|
|
--------
|
|||
|
|
|
|||
|
|
If you want to study internals of the stitching pipeline or you want to experiment with detailed
|
|||
|
|
configuration you can use stitching_detailed source code available in C++ or python
|
|||
|
|
|
|||
|
|
<H4>stitching_detailed</H4>
|
|||
|
|
@add_toggle_cpp
|
|||
|
|
[stitching_detailed.cpp](https://raw.githubusercontent.com/opencv/opencv/4.x/samples/cpp/stitching_detailed.cpp)
|
|||
|
|
@end_toggle
|
|||
|
|
|
|||
|
|
@add_toggle_python
|
|||
|
|
[stitching_detailed.py](https://raw.githubusercontent.com/opencv/opencv/4.x/samples/python/stitching_detailed.py)
|
|||
|
|
@end_toggle
|
|||
|
|
|
|||
|
|
stitching_detailed program uses command line to get stitching parameter. Many parameters exists. Above examples shows some command line parameters possible :
|
|||
|
|
|
|||
|
|
boat5.jpg boat2.jpg boat3.jpg boat4.jpg boat1.jpg boat6.jpg --work_megapix 0.6 --features orb --matcher homography --estimator homography --match_conf 0.3 --conf_thresh 0.3 --ba ray --ba_refine_mask xxxxx --save_graph test.txt --wave_correct no --warp fisheye --blend multiband --expos_comp no --seam gc_colorgrad
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
Pairwise images are matched using an homography --matcher homography and estimator used for transformation estimation too --estimator homography
|
|||
|
|
|
|||
|
|
Confidence for feature matching step is 0.3 : --match_conf 0.3. You can decrease this value if you have some difficulties to match images
|
|||
|
|
|
|||
|
|
Threshold for two images are from the same panorama confidence is 0. : --conf_thresh 0.3 You can decrease this value if you have some difficulties to match images
|
|||
|
|
|
|||
|
|
Bundle adjustment cost function is ray --ba ray
|
|||
|
|
|
|||
|
|
Refinement mask for bundle adjustment is xxxxx ( --ba_refine_mask xxxxx) where 'x' means refine respective parameter and '_' means don't. Refine one, and has the following format: fx,skew,ppx,aspect,ppy
|
|||
|
|
|
|||
|
|
Save matches graph represented in DOT language to test.txt ( --save_graph test.txt) : Labels description: Nm is number of matches, Ni is number of inliers, C is confidence
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
Perform wave effect correction is no (--wave_correct no)
|
|||
|
|
|
|||
|
|
Warp surface type is fisheye (--warp fisheye)
|
|||
|
|
|
|||
|
|
Blending method is multiband (--blend multiband)
|
|||
|
|
|
|||
|
|
Exposure compensation method is not used (--expos_comp no)
|
|||
|
|
|
|||
|
|
Seam estimation estimator is Minimum graph cut-based seam (--seam gc_colorgrad)
|
|||
|
|
|
|||
|
|
you can use those arguments on command line too :
|
|||
|
|
|
|||
|
|
boat5.jpg boat2.jpg boat3.jpg boat4.jpg boat1.jpg boat6.jpg --work_megapix 0.6 --features orb --matcher homography --estimator homography --match_conf 0.3 --conf_thresh 0.3 --ba ray --ba_refine_mask xxxxx --wave_correct horiz --warp compressedPlaneA2B1 --blend multiband --expos_comp channels_blocks --seam gc_colorgrad
|
|||
|
|
|
|||
|
|
You will get :
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
For images captured using a scanner or a drone ( affine motion) you can use those arguments on command line :
|
|||
|
|
|
|||
|
|
newspaper1.jpg newspaper2.jpg --work_megapix 0.6 --features surf --matcher affine --estimator affine --match_conf 0.3 --conf_thresh 0.3 --ba affine --ba_refine_mask xxxxx --wave_correct no --warp affine
|
|||
|
|
|
|||
|
|
|
|||
|
|
|
|||
|
|
You can find all images in https://github.com/opencv/opencv_extra/tree/4.x/testdata/stitching
|