Learn AI Create the Future — with Pranay
Main site

3D Vision and Depth

Homographies and perspective warp

A homography is the 3x3 matrix that relates two views of one flat surface, which is why four corners are enough to flatten a photographed page.

Read these first

On this page 9
  1. The short answer
  2. The analogy you have already lived
  3. Why it exists
  4. How it works
  5. The one condition that matters
  6. The second case that works
  7. Where you have already seen it
  8. Remember this
  9. What to learn next

One lesson, three depths. Pick the one that fits you today — you can switch any time.

Beginner — No maths. Plain English.

The short answer

A homography is the recipe that turns a photo of a flat thing into a straight-on view of it.

The analogy you have already lived

Think of a shadow of a rectangular window on the floor. In the morning it is a long slanted shape. At noon it is nearly a rectangle again.

The window never changed. Straight edges stayed straight in every shadow. What changed was the stretching, and the stretching follows one fixed rule for the whole shape.

That rule, written as nine numbers, is a homography.

Why it exists

Every "scan document" button on a phone is doing this. You photograph a page at an angle and get back a flat rectangle, as if you had used a scanner.

The same recipe is behind panorama stitching. Also a sports broadcast painting a line onto the pitch, and a translation app pasting text over a signboard.

All of them share one condition: the thing being warped is flat.

How it works

You need four matched corners. Where each corner sits in the photo, and where you want it to end up.

   photographed page                 flat page
     (142, 88) ..... (455, 63)        (0,0) ....... (300,0)
        .                .              .              .
        .                .      ->      .              .
     (96,336) ....... (508,372)      (0,400) ..... (300,400)

Four corners give you eight numbers of information. A homography has eight things to work out. So four corners is exactly enough, with nothing spare.

Once you have it, every other pixel of the page follows along automatically.

The one condition that matters

A homography only works for a flat surface.

A page, a wall, a floor, a signboard, a whiteboard. Those are fine.

A face, a chair, a room, a person standing in front of a wall. Those are not. Different parts sit at different distances, and one recipe cannot stretch them all correctly.

The code below shows this failing precisely. Two points that land on exactly the same pixel in the first photo land 44 pixels apart in the second. One recipe cannot send one pixel to two places.

That 44-pixel gap is not an error. It is depth information, and it is what the rest of this section is built on.

The second case that works

There is one other situation where a homography is exact even for a non-flat scene: when the camera only rotates and never moves sideways.

Spin on the spot and the whole world behaves like a flat picture painted on the sky. That is why panorama mode asks you to pivot rather than walk.

Where you have already seen it

  • Phone document scanning that snaps a page flat.
  • Panorama mode, which needs you to rotate on the spot.
  • Broadcast graphics painted onto a cricket pitch.
  • Translation apps replacing signboard text in place.
  • Top-down "bird's eye" views made from a slanted CCTV camera.

Remember this

  • A homography maps one flat surface between two views, using nine numbers.
  • Four matched corners determine it exactly.
  • It is correct for flat things, and wrong for anything with depth.

What to learn next

Developer — Code and libraries.

Setup

bash
pip install opencv-python==4.10.0.84 numpy==1.26.4

We solve the homography by hand with SVD, check it against OpenCV, use it to flatten a photographed page, and then break it on purpose.

Solve it, use it, then break it

homography.py
import cv2
import numpy as np

# The four corners of a page as they appear in a photo taken at an angle,
# and where we want them: a clean 300 x 400 rectangle.
photo_corners = np.float32([[142, 88], [455, 63], [508, 372], [96, 336]])
flat_corners = np.float32([[0, 0], [300, 0], [300, 400], [0, 400]])


def dlt(src, dst):
    """Solve for H by hand. Each point pair gives two rows of A; H is the null space."""
    A = []
    for (x, y), (u, v) in zip(src, dst):
        A.append([-x, -y, -1, 0, 0, 0, u * x, u * y, u])
        A.append([0, 0, 0, -x, -y, -1, v * x, v * y, v])
    _, _, Vt = np.linalg.svd(np.array(A, float))
    H = Vt[-1].reshape(3, 3)
    return H / H[2, 2]                       # fix the arbitrary scale


H_mine = dlt(photo_corners, flat_corners)
H_cv = cv2.getPerspectiveTransform(photo_corners, flat_corners)
print("homography solved by hand:")
print(np.round(H_mine, 6))
print("\nthe same thing from cv2.getPerspectiveTransform:")
print(np.round(H_cv, 6))
print("\nlargest disagreement:", f"{np.abs(H_mine - H_cv).max():.2e}")


def apply(H, pts):
    p = np.hstack([pts, np.ones((len(pts), 1))])
    q = (H @ p.T).T
    return q[:, :2] / q[:, 2:3]              # the divide is what makes it perspective


print("\nthe four corners land exactly where we asked:")
for src, got, want in zip(photo_corners, apply(H_mine, photo_corners), flat_corners):
    print(f"   photo {src} -> {np.round(got, 3)}   wanted {want}")

print("\nnine degrees of freedom, eight of them free:")
print("   H has 9 entries but only its ratios matter, so 8 unknowns")
print("   each corner pair gives 2 equations, so 4 corners is exactly enough")

# Now warp a real picture. The 'page' is a grid of lines on a plane.
page = np.full((400, 300), 255, np.uint8)
for x in range(0, 301, 50):
    cv2.line(page, (x, 0), (x, 400), 0, 5)
for y in range(0, 401, 50):
    cv2.line(page, (0, y), (300, y), 0, 5)
cv2.circle(page, (150, 200), 40, 0, -1)

H_forward = np.linalg.inv(H_mine)            # flat page -> the angled photo
photo = cv2.warpPerspective(page, H_forward, (620, 460), borderValue=255)
recovered = cv2.warpPerspective(photo, H_mine, (300, 400), borderValue=255)


def show(img, cols=62, rows=20):
    """Shrink with area averaging so thin lines survive, then print as text."""
    small = cv2.resize(img, (cols, rows), interpolation=cv2.INTER_AREA)
    for row in small:
        print("   " + "".join("#" if v < 205 else "." for v in row))


print("\nthe page as photographed:")
show(photo)

print("\nthe same page after warping it flat:")
show(recovered)

print("\nsquares in the photo are trapeziums; after the warp they are squares again")

print("\nwhat a homography CANNOT do: anything off the plane")
K = np.array([[500.0, 0, 320], [0, 500.0, 240], [0, 0, 1]])
R, _ = cv2.Rodrigues(np.array([0.0, 0.20, 0.0]))          # camera 2 turns and steps right
t = np.array([[0.30], [0.0], [0.0]])

wall = np.array([[-0.6, -0.4, 3.0], [0.6, -0.4, 3.0], [0.6, 0.4, 3.0],
                 [-0.6, 0.4, 3.0], [0.0, 0.0, 3.0]])       # five points on one wall
lamp = np.array([[0.0, 0.0, 1.6]])                         # one point well in front of it


def view(pts, R=np.eye(3), t=np.zeros((3, 1))):
    cam = (R @ pts.T + t).T
    return (K @ (cam / cam[:, 2:3]).T).T[:, :2]


left, right = view(wall), view(wall, R, t)
H = cv2.getPerspectiveTransform(left[:4].astype(np.float32),
                                right[:4].astype(np.float32))

for name, pts in [("5th point, on the wall", wall[4:]), ("a lamp in front of it", lamp)]:
    a, b = view(pts), view(pts, R, t)
    p = np.append(a[0], 1.0)
    q = H @ p
    predicted = q[:2] / q[2]
    print(f"   {name:<24} H predicts {np.round(predicted, 1)}, "
          f"truth {np.round(b[0], 1)}, off by {np.linalg.norm(predicted - b[0]):7.1f} px")
print("   a homography is exact for one plane and wrong for everything else")
Output
homography solved by hand:
[[ 1.50657500e+00  2.79445000e-01 -2.38524842e+02]
 [ 2.14455000e-01  2.68498100e+00 -2.66730982e+02]
 [ 9.81000000e-04  1.62500000e-03  1.00000000e+00]]

the same thing from cv2.getPerspectiveTransform:
[[ 1.50657500e+00  2.79445000e-01 -2.38524842e+02]
 [ 2.14455000e-01  2.68498100e+00 -2.66730982e+02]
 [ 9.81000000e-04  1.62500000e-03  1.00000000e+00]]

largest disagreement: 1.05e-09

the four corners land exactly where we asked:
   photo [142.  88.] -> [-0.  0.]   wanted [0. 0.]
   photo [455.  63.] -> [300.   0.]   wanted [300.   0.]
   photo [508. 372.] -> [300. 400.]   wanted [300. 400.]
   photo [ 96. 336.] -> [  0. 400.]   wanted [  0. 400.]

nine degrees of freedom, eight of them free:
   H has 9 entries but only its ratios matter, so 8 unknowns
   each corner pair gives 2 equations, so 4 corners is exactly enough

the page as photographed:
   ..............................................................
   ..............................................................
   ..............................................................
   ..................#....#....#....##....#.....#................
   ..............#...#...##....#....##....##....#................
   .............#...##...##...##....##....##.#.#.#...............
   .............#...#....##....#....##....##....##...............
   ............##...#....##########.##.....#......#..............
   ............#...##..############################..............
   ...........##...##....#.########.##.....##.....#..............
   ...........#....#....##....##....################.............
   ...........#################################....#.............
   ..........#....##....#.....##.....#......##.....##............
   ..........#....################################..#............
   ..........#....#.....#.....##.....#......##......#............
   ...........................#......##.....##.......#...........
   ..............................................................
   ..............................................................
   ..............................................................
   ..............................................................

the same page after warping it flat:
   #........###.......###........##........###.......###........#
   #........##.........##........##........##.........##........#
   ##############################################################
   #........##.........##........##........##.........##........#
   #........###........##........##........###........##........#
   ##.###..####.###..#######..##.###..##..######..##.###.##..##.#
   #........##.........##........##........##.........##........#
   ##############################################################
   #........##.........##...#############..##.........##........#
   #........###........##.####################........##........#
   ##.......###..###########################################....#
   #........##.........##..##############..##.........##........#
   ##############################################################
   #........##.........##........##........##.........##........#
   #........###........##........##........###........##........#
   ##.####..###..##..####.###..#######.###.###.###.######.###.###
   #........##.........##........##........##.........##........#
   ##############################################################
   #........##.........##........##........##.........##........#
   #........###........##........##........###........##........#

squares in the photo are trapeziums; after the warp they are squares again

what a homography CANNOT do: anything off the plane
   5th point, on the wall   H predicts [472.4 240. ], truth [472.4 240. ], off by     0.0 px
   a lamp in front of it    H predicts [472.4 240. ], truth [517. 240.], off by    44.6 px
   a homography is exact for one plane and wrong for everything else

Reading the output carefully

The hand-written DLT and OpenCV agree to 1.05e-09. That is floating-point noise, not a difference in method. getPerspectiveTransform solves the same eight-equation system; writing it out once removes the black box.

Look at the structure of H. The top-left 2x2 block is a general linear map. The third column is a translation. The bottom row, [0.000981, 0.001625, 1], is the interesting part — it is what makes this a perspective transform rather than an affine one. Zero that row and you get a parallelogram map, which cannot represent a receding surface.

-0. is not a bug. The first corner maps to -0.0, which is negative zero from floating-point arithmetic. It equals zero.

The warped page has straight, evenly-spaced lines again. In the photograph, the grid squares are trapeziums of different sizes; after warping, they are uniform. Note the small aliasing on the horizontal lines in the text rendering — that is the ASCII downsampling, not the warp.

The failure case is the most important six lines here. Two 3D points, a spot on a wall at 3 m and a lamp at 1.6 m, both sit exactly on the optical axis. In the left view they land on the same pixel. In the right view they land 44.6 pixels apart.

A homography is a function from pixels to pixels. One input, one output. It cannot send one left-image pixel to two different right-image pixels, so it necessarily gets one of them wrong — and it got the off-plane point wrong by 44.6 pixels.

That 44.6 is what depth looks like. Every stereo and structure-from-motion technique in this section is a way of measuring that residual and turning it back into a distance.

Finding the homography from many noisy points

With more than four correspondences and real noise, use cv2.findHomography:

python
H, mask = cv2.findHomography(src_pts, dst_pts, cv2.USAC_MAGSAC, 3.0)

USAC_MAGSAC is a robust estimator that ignores wrong matches, covered in RANSAC for robust fitting. mask marks which correspondences were used. Passing 0 instead of a robust method gives plain least squares, and a single bad match then ruins the whole result.

Common mistakes

Not normalising points before DLT. The matrix A mixes terms of order 1 with terms of order u*x — around 100,000 for typical pixels. That conditioning is poor. Hartley's normalisation, translating points to zero mean and scaling so the average distance to the origin is the square root of two, is standard and is what findHomography does internally. Our tiny four-point case works without it; a 200-point fit would suffer.

Warping in the wrong direction. cv2.warpPerspective(src, H, size) uses H to map source pixels to destination. If your H goes the other way, pass flags=cv2.WARP_INVERSE_MAP or invert it. Getting a black image is the usual symptom.

Three collinear corners. Any three of the four points on a line makes the system rank-deficient, and the returned matrix is meaningless. Check for it.

Applying it to a scene with depth. The failure demonstrated above. If you are stitching a panorama and the camera translated at all, near objects will ghost — this is called parallax, and no homography removes it.

Forgetting the divide. H @ p gives three numbers, and you must divide the first two by the third. Skipping it produces an affine-looking result that is subtly wrong everywhere.

Try it yourself

Set H_mine[2, 0] and H_mine[2, 1] to zero and warp again. The result is a parallelogram: parallel lines stay parallel, and the page no longer looks like it recedes. Those two small numbers carry the entire sense of perspective.

What to learn next

Researcher — Mathematics and papers.

Definition

A homography (projective transformation, collineation) is an invertible linear map on homogeneous coordinates:

$$ \begin{bmatrix} u' \ v' \ w' \end{bmatrix} = H \begin{bmatrix} u \ v \ w \end{bmatrix}, \qquad H \in \mathbb{R}^{3\times3}, \ \det H \neq 0 $$

with image coordinates recovered as $(u'/w', v'/w')$. Because scaling $H$ scales all outputs equally, $H$ is defined up to scale and has 8 degrees of freedom.

Homographies form the group $PGL(3,\mathbb{R})$ acting on $\mathbb{P}^2$. They preserve incidence, collinearity, and cross-ratio; they do not preserve length, angle, area, or parallelism.

The two situations where it is exact

A plane in the scene. Let a plane be $\mathbf{n}^\top \mathbf{X} = d$ in the first camera's frame. For a second camera related by $(R, \mathbf{t})$:

$$ H = K_2 \left( R + \frac{\mathbf{t}\,\mathbf{n}^\top}{d} \right) K_1^{-1} $$

Every term is visible in this expression: the intrinsics of both cameras, the relative pose, and the plane. The dependence on $\mathbf{n}/d$ is why a homography is specific to one plane — a second plane at a different depth induces a different $H$.

Pure rotation. With $\mathbf{t} = \mathbf{0}$ the plane terms vanish and

$$ H = K_2 R K_1^{-1} $$

independent of scene structure entirely. This is the panorama case, and it is exact for arbitrary geometry.

Everything else is inexact, with residual parallax proportional to $\lVert\mathbf{t}\rVert \cdot (1/Z_1 - 1/Z_{\text{plane}})$. The 44.6-pixel error in the code is exactly this quantity for a 1.6 m point against a 3 m plane with a 0.3 m baseline.

Direct linear transform

From $\mathbf{x}' \times H\mathbf{x} = \mathbf{0}$ (the cross product removes the unknown scale), each correspondence gives three equations of which two are independent. Stacking $n \geq 4$ correspondences gives $A\mathbf{h} = \mathbf{0}$ with $A \in \mathbb{R}^{2n \times 9}$, and the solution is the right singular vector of $A$ for its smallest singular value — the null space when $n = 4$ and noise-free.

This minimises the algebraic error $\lVert A\mathbf{h}\rVert$ subject to $\lVert\mathbf{h}\rVert = 1$, which has no geometric meaning. Two refinements matter in practice:

Hartley normalisation (Hartley, 1997, In defense of the eight-point algorithm). Apply similarity transforms $T, T'$ so each point set has zero centroid and mean distance $\sqrt{2}$ from the origin, solve for $\tilde{H}$, then set $H = T'^{-1}\tilde{H}T$. Without it, the condition number of $A$ can exceed $10^8$ and the estimate degrades badly with noise. This is not optional for real data.

Geometric refinement. Minimise the symmetric transfer error

$$ \sum_i d\bigl(\mathbf{x}'_i, H\mathbf{x}_i\bigr)^2 + d\bigl(\mathbf{x}_i, H^{-1}\mathbf{x}'_i\bigr)^2 $$

by Levenberg-Marquardt, initialised from the normalised DLT solution. The statistically optimal alternative is the Sampson error, a first-order approximation to the true reprojection error that is cheaper than full bundle adjustment and nearly as accurate.

Decomposing a homography

Given $H$ and known $K$, cv2.decomposeHomographyMat returns up to four solutions $(R, \mathbf{t}/d, \mathbf{n})$. The four-fold ambiguity is intrinsic: two from the sign of the plane normal, two from a reflection. Disambiguation requires the cheirality constraint — reconstructed points must lie in front of both cameras — and usually still leaves two candidates unless a third view or a prior on $\mathbf{n}$ is available.

This decomposition is the basis of planar visual odometry and of AR marker tracking, where the scene is known to be a plane.

The hierarchy of 2D transformations

GroupDoFPreservesMatrix form
Euclidean3Lengths, angles, areas$[R\ \mathbf{t}; \mathbf{0}\ 1]$, $R \in SO(2)$
Similarity4Angles, ratios of lengths$[sR\ \mathbf{t}; \mathbf{0}\ 1]$
Affine6Parallelism, ratios along a line$[A\ \mathbf{t}; \mathbf{0}\ 1]$
Projective8Incidence, cross-ratiofull $3\times3$

The bottom row of $H$ is what separates affine from projective. Setting it to $[0, 0, 1]$ collapses a homography to an affine map, and the line at infinity is then preserved — which is why affine warps cannot represent a receding plane.

Where homographies still matter in 2026

Despite deep learning taking over most of vision, homography estimation remains classical in production, because it is exact, fast, and interpretable. It appears in AR plane tracking, document rectification, sports-field registration, image stitching, and as the initialisation step for planar SLAM.

Learned alternatives exist — deep homography estimation regresses the four-corner offsets directly and is more robust on low-texture pairs — but the analytic solution remains the default when features are available, and learned methods are typically used to supply better correspondences rather than to replace the solve.

What to learn next