## mosaicing

mosaicing — build image mosaics

## Stability Level

Stable, unless otherwise indicated

## Synopsis

    #include <vips/vips.h>

    int                 im_lrmerge                          (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int dx,
                                                             int dy,
                                                             int mwidth);
    int                 im_tbmerge                          (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int dx,
                                                             int dy,
                                                             int mwidth);
    int                 im_lrmerge1                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int mwidth);
    int                 im_tbmerge1                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int mwidth);
    int                 im_lrmosaic                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xref,
                                                             int yref,
                                                             int xsec,
                                                             int ysec,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);
    int                 im_tbmosaic                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xref,
                                                             int yref,
                                                             int xsec,
                                                             int ysec,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);
    int                 im_lrmosaic1                        (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);
    int                 im_tbmosaic1                        (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);
    int                 im_global_balance                   (VipsImage *in,
                                                             VipsImage *out,
                                                             double gamma);
    int                 im_global_balancef                  (VipsImage *in,
                                                             VipsImage *out,
                                                             double gamma);
    int                 im_correl                           (VipsImage *ref,
                                                             VipsImage *sec,
                                                             int xref,
                                                             int yref,
                                                             int xsec,
                                                             int ysec,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             double *correlation,
                                                             int *x,
                                                             int *y);
    int                 im_remosaic                         (VipsImage *in,
                                                             VipsImage *out,
                                                             const char *old_str,
                                                             const char *new_str);
    int                 im_align_bands                      (VipsImage *in,
                                                             VipsImage *out);
    int                 im_maxpos_subpel                    (VipsImage *in,
                                                             double *x,
                                                             double *y);

## Description

These functions are useful for joining many small images together to make one large image. They can cope with unstable contrast and arbitary sub-image layout, but will not do any geometric correction. Geometric errors should be removed before using these functions.

The mosaicing functions can be grouped into layers:

The lowest level functions are `im_correl()`, `im_lrmerge()` and `im_tbmerge()`. `im_correl()` searches a large image for a small sub-image, returning the position of the best sub-image match. `im_lrmerge()` and `im_tbmerge()` join two images together left-right or up-down with a smooth seam.

Next, `im_lrmosaic()` and `im_tbmosaic()` use the search function plus the two low-level merge operations to join two images given just an approximate overlap as a start point.

The functions `im_lrmosaic1()` and `im_tbmosaic1()` are first-order analogues of the basic mosaic functions: they take two approximate tie-points and use them to rotate and scale the right-hand or bottom image before starting to join.

Finally, `im_global_balance()` can be used to remove contrast differences in a mosaic which has been assembled with these functions. It takes the mosaic apart, measures image contrast differences along the seams, finds a set of correction factors which will minimise these differences, and reassembles the mosaic. `im_remosaic()` uses the same techniques, but will reassemble the image from a different set of source images.

## Details

### im_lrmerge ()

    int                 im_lrmerge                          (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int dx,
                                                             int dy,
                                                             int mwidth);

This operation joins two images left-right (with `ref` on the left) with a smooth seam.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

`dx` and `dy` give the displacement of `sec` relative to `ref`, in other words, the vector to get from the origin of `sec` to the origin of `ref`, in other words, `dx` will generally be a negative number.

`mwidth` limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

See also: `im_lrmosaic()`, `im_tbmerge()`, `im_match_linear()`, `im_insert()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`dx` : | displacement of ref from sec
`dy` : | displacement of ref from sec
`mwidth` : | maximum seam width
Returns : | 0 on success, -1 on error

### im_tbmerge ()

    int                 im_tbmerge                          (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int dx,
                                                             int dy,
                                                             int mwidth);

This operation joins two images top-bottom (with `ref` on the top) with a smooth seam.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

`dx` and `dy` give the displacement of `sec` relative to `ref`, in other words, the vector to get from the origin of `sec` to the origin of `ref`, in other words, `dx` will generally be a negative number.

`mwidth` limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

See also: `im_lrmosaic()`, `im_lrmerge()`, `im_match_linear()`, `im_insert()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`dx` : | displacement of ref from sec
`dy` : | displacement of ref from sec
`mwidth` : | maximum seam width
Returns : | 0 on success, -1 on error

### im_lrmerge1 ()

    int                 im_lrmerge1                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int mwidth);

This operation joins two images left-right (with `ref` on the left) given a pair of tie-points. `sec` is scaled and rotated as necessary before the join.

`mwidth` limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: `im_tbmerge1()`, `im_lrmerge()`, `im_insert()`, `im_global_balance()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`xr1` : | first reference tie-point
`yr1` : | first reference tie-point
`xs1` : | first secondary tie-point
`ys1` : | first secondary tie-point
`xr2` : | second reference tie-point
`yr2` : | second reference tie-point
`xs2` : | second secondary tie-point
`ys2` : | second secondary tie-point
`mwidth` : | maximum blend width
Returns : | 0 on success, -1 on error

### im_tbmerge1 ()

    int                 im_tbmerge1                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int mwidth);

This operation joins two images top-bottom (with `ref` on the top) given a pair of tie-points. `sec` is scaled and rotated as necessary before the join.

`mwidth` limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: `im_lrmerge1()`, `im_tbmerge()`, `im_insert()`, `im_global_balance()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`xr1` : | first reference tie-point
`yr1` : | first reference tie-point
`xs1` : | first secondary tie-point
`ys1` : | first secondary tie-point
`xr2` : | second reference tie-point
`yr2` : | second reference tie-point
`xs2` : | second secondary tie-point
`ys2` : | second secondary tie-point
`mwidth` : | maximum blend width
Returns : | 0 on success, -1 on error

### im_lrmosaic ()

    int                 im_lrmosaic                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xref,
                                                             int yref,
                                                             int xsec,
                                                             int ysec,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);

This operation joins two images left-right (with `ref` on the left) given an approximate overlap.

`sec` is positioned so that the pixel (`xsec`, `ysec`) lies on top of the pixel in `ref` at (`xref`, `yref`). The overlap area is divided into three sections, 20 high-contrast points in band `bandno` of image `ref` are found in each, and each high-contrast point is searched for in `sec` using `hwindowsize` and `hsearchsize` (see `im_correl()`).

A linear model is fitted to the 60 tie-points, points a long way from the fit are discarded, and the model refitted until either too few points remain or the model reaches good agreement.

The detected displacement is used with `im_lrmerge()` to join the two images together.

`mwidth` limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: `im_lrmerge()`, `im_tbmosaic()`, `im_insert()`, `im_global_balance()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`bandno` : | band to search for features
`xref` : | position in reference image
`yref` : | position in reference image
`xsec` : | position in secondary image
`ysec` : | position in secondary image
`hwindowsize` : | half window size
`hsearchsize` : | half search size
`balancetype` : | no longer used
`mwidth` : | maximum blend width
Returns : | 0 on success, -1 on error

### im_tbmosaic ()

    int                 im_tbmosaic                         (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xref,
                                                             int yref,
                                                             int xsec,
                                                             int ysec,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);

This operation joins two images top-bottom (with `ref` on the top) given an approximate overlap.

`sec` is positioned so that the pixel (`xsec`, `ysec`) lies on top of the pixel in `ref` at (`xref`, `yref`). The overlap area is divided into three sections, 20 high-contrast points in band `bandno` of image `ref` are found in each, and each high-contrast point is searched for in `sec` using `hwindowsize` and `hsearchsize` (see `im_correl()`).

A linear model is fitted to the 60 tie-points, points a long way from the fit are discarded, and the model refitted until either too few points remain or the model reaches good agreement.

The detected displacement is used with `im_tbmerge()` to join the two images together.

`mwidth` limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: `im_tbmerge()`, `im_lrmosaic()`, `im_insert()`, `im_global_balance()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`bandno` : | band to search for features
`xref` : | position in reference image
`yref` : | position in reference image
`xsec` : | position in secondary image
`ysec` : | position in secondary image
`hwindowsize` : | half window size
`hsearchsize` : | half search size
`balancetype` : | no longer used
`mwidth` : | maximum blend width
Returns : | 0 on success, -1 on error

### im_lrmosaic1 ()

    int                 im_lrmosaic1                        (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);

This operation joins two images left-right (with `ref` on the left) given an approximate pair of tie-points. `sec` is scaled and rotated as necessary before the join.

Before performing the transformation, the tie-points are improved by searching band `bandno` in an area of `sec` of size `hsearchsize` for a match of size `hwindowsize` to `ref`.

`mwidth` limits the maximum width of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: `im_tbmosaic1()`, `im_lrmerge()`, `im_insert()`, `im_global_balance()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`bandno` : | band to search for features
`xr1` : | first reference tie-point
`yr1` : | first reference tie-point
`xs1` : | first secondary tie-point
`ys1` : | first secondary tie-point
`xr2` : | second reference tie-point
`yr2` : | second reference tie-point
`xs2` : | second secondary tie-point
`ys2` : | second secondary tie-point
`hwindowsize` : | half window size
`hsearchsize` : | half search size
`balancetype` : | no longer used
`mwidth` : | maximum blend width
Returns : | 0 on success, -1 on error

### im_tbmosaic1 ()

    int                 im_tbmosaic1                        (VipsImage *ref,
                                                             VipsImage *sec,
                                                             VipsImage *out,
                                                             int bandno,
                                                             int xr1,
                                                             int yr1,
                                                             int xs1,
                                                             int ys1,
                                                             int xr2,
                                                             int yr2,
                                                             int xs2,
                                                             int ys2,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             int balancetype,
                                                             int mwidth);

This operation joins two images top-bottom (with `ref` on the top) given an approximate pair of tie-points. `sec` is scaled and rotated as necessary before the join.

Before performing the transformation, the tie-points are improved by searching band `bandno` in an area of `sec` of size `hsearchsize` for a match of size `hwindowsize` to `ref`.

`mwidth` limits the maximum height of the blend area. A value of "-1" means "unlimited". The two images are blended with a raised cosine.

Pixels with all bands equal to zero are "transparent", that is, zero pixels in the overlap area do not contribute to the merge. This makes it possible to join non-rectangular images.

If the number of bands differs, one of the images must have one band. In this case, an n-band image is formed from the one-band image by joining n copies of the one-band image together, and then the two n-band images are operated upon.

The two input images are cast up to the smallest common type (see table Smallest common format in arithmetic).

See also: `im_lrmosaic1()`, `im_tbmerge()`, `im_insert()`, `im_global_balance()`.

`ref` : | reference image
`sec` : | secondary image
`out` : | output image
`bandno` : | band to search for features
`xr1` : | first reference tie-point
`yr1` : | first reference tie-point
`xs1` : | first secondary tie-point
`ys1` : | first secondary tie-point
`xr2` : | second reference tie-point
`yr2` : | second reference tie-point
`xs2` : | second secondary tie-point
`ys2` : | second secondary tie-point
`hwindowsize` : | half window size
`hsearchsize` : | half search size
`balancetype` : | no longer used
`mwidth` : | maximum blend width
Returns : | 0 on success, -1 on error

### im_global_balance ()

    int                 im_global_balance                   (VipsImage *in,
                                                             VipsImage *out,
                                                             double gamma);

`im_global_balance()` can be used to remove contrast differences in an assembled mosaic.

It reads the History field attached to `in` and builds a list of the source images that were used to make the mosaic and the position that each ended up at in the final image.

It opens each of the source images in turn and extracts all parts which overlap with any of the other images. It finds the average values in the overlap areas and uses least-mean-square to find a set of correction factors which will minimise overlap differences. It uses `gamma` to gamma-correct the source images before calculating the factors. A value of 1.0 will stop this.

Each of the source images is transformed with the appropriate correction factor, then the mosaic is reassembled. `out` always has the same BandFmt as `in`. Use `im_global_balancef()` to get float output and avoid clipping.

There are some conditions that must be met before this operation can work: the source images must all be present under the filenames recorded in the history on `in`, and the mosaic must have been built using only operations in this package.

See also: `im_global_balancef()`, `im_remosaic()`.

`in` : | mosaic to rebuild
`out` : | output image
`gamma` : | gamma of source images
Returns : | 0 on success, -1 on error

### im_global_balancef ()

    int                 im_global_balancef                  (VipsImage *in,
                                                             VipsImage *out,
                                                             double gamma);

Just as `im_global_balance()`, but the output image is always float. This stops overflow or underflow in the case of an extremely unbalanced image mosaic.

See also: `im_global_balance()`, `im_remosaic()`.

`in` : | mosaic to rebuild
`out` : | output image
`gamma` : | gamma of source images
Returns : | 0 on success, -1 on error

### im_correl ()

    int                 im_correl                           (VipsImage *ref,
                                                             VipsImage *sec,
                                                             int xref,
                                                             int yref,
                                                             int xsec,
                                                             int ysec,
                                                             int hwindowsize,
                                                             int hsearchsize,
                                                             double *correlation,
                                                             int *x,
                                                             int *y);

This operation finds the position of `sec` within `ref`.

The area around (`xsec`, `ysec`) is searched for the best match to the area around (`xref`, `yref`). It searches an area of size `hsearchsize` for a match of size `hwindowsize`. The position of the best match is returned, together with the correlation at that point.

Only the first band of each image is correlated. `ref` and `sec` may be very large --- the function extracts and generates just the parts needed. Correlation is done with `im_spcor()`; the position of the maximum is found with `im_maxpos()`.

See also: `im_match_linear()`, `im_match_linear_search()`, `im_lrmosaic()`.

`ref` : | reference image
`sec` : | secondary image
`xref` : | position in reference image
`yref` : | position in reference image
`xsec` : | position in secondary image
`ysec` : | position in secondary image
`hwindowsize` : | half window size
`hsearchsize` : | half search size
`correlation` : | return detected correlation
`x` : | return found position
`y` : | return found position
Returns : | 0 on success, -1 on error

### im_remosaic ()

    int                 im_remosaic                         (VipsImage *in,
                                                             VipsImage *out,
                                                             const char *old_str,
                                                             const char *new_str);

`im_remosaic()` works rather as `im_global_balance()`. It takes apart the mosaiced image in and rebuilds it, substituting images.

Unlike `im_global_balance()`, images are substituted based on their file‐ names. The rightmost occurence of the string `old_str` is swapped for `new_str`, that file is opened, and that image substituted for the old image.

It's convenient for multispectral images. You can mosaic one band, then use that mosaic as a template for mosaicing the others automatically.

See also: `im_lrmosaic()`, `im_global_balance()`.

`in` : | mosaic to rebuild
`out` : | output image
`old_str` : | gamma of source images
`new_str` : | gamma of source images
Returns : | 0 on success, -1 on error

### im_align_bands ()

    int                 im_align_bands                      (VipsImage *in,
                                                             VipsImage *out);

This operation uses `im_phasecor_fft()` to find an integer displacement to align all image bands band 0. It is very slow and not very accurate.

Use `im_estpar()` in preference: it's fast and accurate.

See also: `im_global_balancef()`, `im_remosaic()`.

`in` : | image to align
`out` : | output image
Returns : | 0 on success, -1 on error

### im_maxpos_subpel ()

    int                 im_maxpos_subpel                    (VipsImage *in,
                                                             double *x,
                                                             double *y);

This function implements:

"Extension of Phase Correlation to Subpixel Registration" by H. Foroosh, from IEEE trans. Im. Proc. 11(3), 2002.

If the best three matches in the correlation are aranged:

02 or 01 1 2

then we return a subpixel match using the ratio of correlations in the vertical and horizontal dimension.

( xs[0], ys[0] ) is the best integer alignment ( xs[ use_x ], ys[ use_x ] ) is equal in y and (+/-)1 off in x ( xs[ use_y ], ys[ use_y ] ) is equal in x and (+/-)1 off in y

Alternatively if the best four matches in the correlation are aranged in a square:

01 or 03 or 02 or 03 32 12 31 21

then we return a subpixel match weighting with the sum the two on each side over the sum of all four, but only if all four of them are very close to the best, and the fifth is nowhere near.

This alternative method is not described by Foroosh, but is often the case where the match is close to n-and-a-half pixels in both dimensions.

See also: `im_maxpos()`, `im_min()`, `im_stats()`.

`in` : | input image
`x` : | output position of maximum
`y` : | output position of maximum
Returns : | 0 on success, -1 on error