I created an application for a target hit evaluation using a webcam. The camera is never vertical to the target so there is a significant perspective distortion and I need a bird’s eye view. Fixing this with getPerspectiveTransform() and cv::warpPerspective() is easy but cv::warpPerspective() is very slow (at least for my purpose).

Faster perspective correction can be done using cv::remap() if you don’t mind that the maps creation takes a long time. Since the camera has a fixed focus then the maps can be created only once and then just run the remapping. The speedup of cv::remap() compared to cv::warpPerspective() is almost 5x.

Maps creation

The main problem is how to efficiently create those maps. My first try was to calculate the points one by one:

void perspective_to_maps(const cv::Mat &perspective_mat, const cv::Size &img_size,
                          cv::Mat &map1, cv::Mat &map2)
{
  // invert the matrix because the transformation maps must be
  // bird's view -> original
  cv::Mat inv_perspective(perspective_mat.inv());

  cv::Mat map_x(img_size, CV_32FC1);
  cv::Mat map_y(img_size, CV_32FC1);

  // bird's view point coordinates
  cv::Mat point = (cv::Mat_(3, 1)<double> 0, 0, 1);
  double *ppoint = point.ptr<double>(0);
  // original point coordinates
  cv::Mat orig_point(cv::Size(3, 1), CV_64FC1);
  double *porig_point = orig_point.ptr<double>(0);

  for (int y = 0; y < img_size.height; y++)
  {
    ppoint[1] = y;
    float *pmap_x = map_x.ptr<float>(y);
    float *pmap_y = map_y.ptr<float>(y);
    for (int x = 0; x < img_size.width; x++)
    {
      ppoint[0] = x;
      // perspective transformation by matrix multiplication
      orig_point = inv_perspective * point;
      pmap_x[x] = porig_point[0] / porig_point[2];
      pmap_y[x] = porig_point[1] / porig_point[2];
    }
  }

  // remap() with integer maps is faster
  cv::convertMaps(map_x, map_y, map1, map2, CV_16SC2);
}

Faster

Previous solution works well but takes a considerable time to calculate the maps even with maximum compiler optimizations. Lately I found function cv::perspectiveTransform() which can do the heavy lifting. Prepare simple XY matrix and apply transformation at once.

void perspective_to_maps(const cv::Mat &perspective_mat, const cv::Size &img_size,
                          cv::Mat &map1, cv::Mat &map2)
{
  // invert the matrix because the transformation maps must be
  // bird's view -> original
  cv::Mat inv_perspective(perspective_mat.inv());
  inv_perspective.convertTo(inv_perspective, CV_32FC1);

  // create XY 2D array
  // (((0, 0), (1, 0), (2, 0), ...),
  //  ((0, 1), (1, 1), (2, 1), ...),
  // ...)
  cv::Mat xy(img_size, CV_32FC2);
  float *pxy = (float*)xy.data;
  for (int y = 0; y < img_size.height; y++)
    for (int x = 0; x < img_size.width; x++)
    {
      *pxy++ = x;
      *pxy++ = y;
    }

  // perspective transformation of the points
  cv::Mat xy_transformed;
  cv::perspectiveTransform(xy, xy_transformed, inv_perspective);

  // split x/y to extra maps
  assert(xy_transformed.channels() == 2);
  cv::Mat maps[2]; // map_x, map_y
  cv::split(xy_transformed, maps);

  // remap() with integer maps is faster
  cv::convertMaps(maps[0], maps[1], map1, map2, CV_16SC2);
}

This generator is 5x-10x faster depending on architecture (ARM, x64, …).

Question: can be the XY matrix prepared faster? Any comments are welcome.