Reprojection error

The reprojection error is a geometric error corresponding to the image distance between a projected point and a measured one. It is used to quantify how closely an estimate of a 3D point $\widehat{\mathbf{𝐗}}$ recreates the point's true projection $\mathbf{𝐱}$. More precisely, let $\mathbf{𝐏}$ be the projection matrix of a camera and $\widehat{\mathbf{𝐱}}$ be the image projection of $\widehat{\mathbf{𝐗}}$, i.e. $\widehat{\mathbf{𝐱}}=\mathbf{𝐏}\widehat{\mathbf{𝐗}}$. The reprojection error of $\widehat{\mathbf{𝐗}}$ is given by $d(\mathbf{𝐱},\widehat{\mathbf{𝐱}})$, where $d(\mathbf{𝐱},\widehat{\mathbf{𝐱}})$ denotes the Euclidean distance between the image points represented by vectors $\mathbf{𝐱}$ and $\widehat{\mathbf{𝐱}}$.

Minimizing the reprojection error can be used for estimating the error from point correspondences between two images. Suppose we are given 2D to 2D point imperfect correspondences $\{{\mathbf{x}}_{\mathbf{𝐢}}\leftrightarrow {{\mathbf{x}}_{\mathbf{𝐢}}}^{\prime }\}$. We wish to find a homography $\widehat{\mathbf{𝐇}}$ and pairs of perfectly matched points $\widehat{{\mathbf{x}}_{\mathbf{𝐢}}}$ and ${{\widehat{\mathbf{𝐱}}}_i}^{\prime }$, i.e. points that satisfy ${\widehat{{\mathbf{x}}_{\mathbf{𝐢}}}}^{\prime }=\hat{H}{\hat{\mathbf{x}}}_{\mathbf{𝐢}}$ that minimize the reprojection error function given by

$\sum _{i}d({\mathbf{x}}_{\mathbf{𝐢}},\widehat{{\mathbf{x}}_{\mathbf{𝐢}}}{)}^2+d({{\mathbf{x}}_{\mathbf{𝐢}}}^{\prime },{\widehat{{\mathbf{x}}_{\mathbf{𝐢}}}}^{\prime }{)}^2$

So the correspondences can be interpreted as imperfect images of a world point and the reprojection error quantifies their deviation from the true image projections $\widehat{{\mathbf{x}}_{\mathbf{𝐢}}},{\widehat{{\mathbf{x}}_{\mathbf{𝐢}}}}^{\prime }$