Canonical Correlation and Whitening

Worked Example

Consider the population covariances for the "Separated" spectrum with rotation of 45 degrees. We have S_(xx) = diag(1², 1) since First X scale s = 1. S_(yy) = I. The spectrum is "Separated", so the canonical correlations are r_1 = 0.8 and r_2 = 0.2. At 45 degrees, the rotation matrix R(α) is:

[[cos(45), -sin(45)],
 [sin(45),  cos(45)]]

The raw cross-covariance is calculated as S_(xy) = diag(s, 1) R(α) diag(r_1, r_2). With s=1, cos(45^\circ) = sin(45^\circ) = √2/2 ≈ 0.7071067811865476.

1. Entry S_(xy)[0][0] = s · cos(45^\circ) · r_1 = 1 · (√2/2) · 0.8 = 0.4√2 ≈ 0.5656854249492381.
2. The whitened cross-covariance is K = S_(xx)^(-1/2) S_(xy) S_(yy)^(-1/2). Here, it's just R(α) diag(r_1, r_2). Its [0][0] entry is also 0.4√2 ≈ 0.5656854249492381.
3. The joint covariance determinant is det(S_(xx)) · det(I - K^T K) = 1 · (1 - 0.8²)(1 - 0.2²) = (1 - 0.64)(1 - 0.04) = 0.36 · 0.96 = 0.34559999999999985.

Experiment

You can verify how scaling and rotation affect the raw covariance but not the canonical correlations:

1. Select Separated for Spectrum.
2. Select 45 for Rotation degrees.
3. Observe the raw cross covariance zero zero is 0.5656854249492381.
4. Change First X scale to 2.
5. Note that the raw cross covariance zero zero doubles to 1.1313708498984762, but the first and second canonical correlations remain 0.8 and 0.2 respectively.

Try the interactive experiment: https://riano.app/module/canonical-correlation-whitening.

Limitations

The visualization assumes population covariances without sampling noise, so there are no inferential p-values or sample data points. When the spectrum is "Perfect" (r_1 = 1), the joint covariance matrix becomes singular (determinant 0) indicating perfect linear dependency, while both marginal whitenings remain valid. For "Equal" or "Zero" spectra, the canonical directions are nonunique and do not imply identifiable vectors.

References

The fundamental formulation of canonical correlation analysis connects strongly to the singular value decomposition of the whitened cross-covariance matrix. For a more detailed derivation, see the textbook reference:

• Penn State STAT 505 Lesson 13: Canonical Correlation Analysis