Definition
The Generalized Cross Validation (GCV) criterion is a very similar method to the PRESS method. GCV is a rotation-invariant form of the PRESS method. The deriviation of the GCV from PRESS is shown in .
GCV leads to choosing \(\lambda\) as the minimizer of the GCV function \(\mathcal{G}(\lambda)\), defined by
\[\begin{equation}
\mathcal{G}(\lambda)
\equiv
\frac{ \| (\mathbf{I} - \mathbf{A}_\lambda)\mathbf{B}\mathbf{b} \|^2 }
{\text{tr}(\mathbf{I} - \mathbf{A}_\lambda)^2},
\end{equation}\]
where \(\mathbf{A}_\lambda \equiv \mathbf{B}\mathbf{T}(\mathbf{T}^\mathsf{T}\mathbf{Q}\mathbf{T} + \lambda\mathbf{H})^{-1}\mathbf{T}^\mathsf{T}\mathbf{B}^\mathsf{T}\), \(\text{tr}(\cdot)\) is the trace of a matrix, and \(\mathbf{Q}=\mathbf{B}^\mathsf{T}\mathbf{B}\).
Using series-expansion form of the solution, \(\mathcal{G}(\lambda)\) can be written as
\[\begin{equation}
\mathcal{G}(\lambda)
=
\frac{\rho}
{\left(r - \sum_{i=1}^r f_{\lambda,i} \right)^2}.
\label{eq:gcv_series}
\end{equation}\]
Deriviation of \eqref{eq:gcv_series}
Recalling the Generalized Tikhonov regularized solution form and the series expansion, we obtain the following:
\[\begin{split}\begin{align*}
\mathbf{A}_\lambda \mathbf{B}\mathbf{b}
&=
\mathbf{B}\mathbf{T} \mathbf{x}_\lambda\\
&=
\mathbf{B}\mathbf{T}\tilde{\mathbf{V}}\mathbf{F}_\lambda\mathbf{S}^{-1}\mathbf{U}^\mathsf{T}\hat{\mathbf{b}}\\
&=
\mathbf{U}\mathbf{S}\mathbf{F}_\lambda\mathbf{S}^{-1}\mathbf{U}^\mathsf{T}\mathbf{B}\mathbf{b}\quad(\because \mathbf{B}\mathbf{T}\tilde{\mathbf{V}} = \mathbf{U}\mathbf{S})\\
&=
\mathbf{U}\mathbf{F}_\lambda\mathbf{U}^\mathsf{T}\mathbf{B}\mathbf{b}.\\
\therefore
\mathbf{A}_\lambda &= \mathbf{U}\mathbf{F}_\lambda\mathbf{U}^\mathsf{T}.
\end{align*}\end{split}\]
Then we have
\[\begin{split}\begin{align*}
\text{numerator of } \mathcal{G}(\lambda)
&=
\| (\mathbf{I} - \mathbf{A}_\lambda)\mathbf{B}\mathbf{b} \|^2 \\
&=
\| \mathbf{B}\mathbf{b} - \mathbf{B}\mathbf{T}\mathbf{x}_\lambda \|^2\\
&=
\| \mathbf{b} - \mathbf{T}\mathbf{x}_\lambda\|_\mathbf{Q}^2 = \rho.\\\\
\text{denominator of } \mathcal{G}(\lambda)
&=
\text{tr}(\mathbf{I} - \mathbf{A}_\lambda)^2\\
&=
\left(
\text{tr}(\mathbf{I}) - \text{tr}(\mathbf{A}_\lambda)
\right)^2\\
&=
\left(
\text{tr}(\mathbf{I}) - \text{tr}(\mathbf{U}\mathbf{F}_\lambda\mathbf{U}^\mathsf{T})
\right)^2\\
&=
\left(
\text{tr}(\mathbf{I}) - \text{tr}(\mathbf{F}_\lambda)
\right)^2
\quad\left(
\because \text{tr}(\mathbf{U}\mathbf{F}_\lambda\mathbf{U}^\mathsf{T}) = \text{tr}(\mathbf{U}^\mathsf{T}\mathbf{U}\mathbf{F}_\lambda) = \text{tr}(\mathbf{F}_\lambda)
\right)\\
&=
\left(
\sum_{i=1}^r 1 - \sum_{i=1}^r f_{\lambda,i}
\right)^2\\
&=
\left(
r - \sum_{i=1}^r f_{\lambda,i}
\right)^2.
\end{align*}\end{split}\]