Whitening transformation（白化變換）

本文轉載自查看原文 2012-06-20 15:31 3573 (8) 概率論和統計學

(from: wikipedia)

The whitening transformation is a decorrelation method which transforms a set of random variables having the covariance matrix $Σ$ into a set of new random variables whose covariance is aI, where a is a constant and I is the identity matrix. The new random variables are uncorrelated and all have variance 1. The method is called "whitening" because it transforms the input matrix to the form of white noise, which by definition is uncorrelated and has uniform variance. It differs from decorrelation in that the variances are made to be equal, rather than merely making the covariances zero. That is, where decorrelation results in a diagonal covariance matrix, whitening produces a scalar multiple of the identity matrix.

Definition

Define $X$ to be a random vector with covariance matrix $Σ$ and mean 0. The matrix $Σ$ can be written as the outer product of $X$ and $X T$ :

$\Sigma = \operatorname{E}[XX^T]$

(注，我初一看以為寫錯了，按我的想法好像寫成 E[X_iX_j] 更靠譜，我又想了想，后來明白了，E[X_iX_j]這樣寫的話，括號里面成了兩個變量，而實際應該是一個變量（總體），前后可以是兩個變量樣本值。所以 E[XX^T]這樣寫是有道理的，只是注意不要誤解吧。還可以這樣寫 E[XY^T]，X和Y是同分布，但是這是表示兩個一般變量協方差的常用方法，顯然和這里X是向量的環境不符合。)

Define $Σ 1/2$ as

$\Sigma^{1/2}(\Sigma^{1/2})^T = \Sigma$

Define the new random vector $Y = Σ -1/2 X$ . The covariance of Y is

$\begin{align} \operatorname{Cov}(Y) &= \operatorname{E}[YY^T] \\ &= \operatorname{E}[(\Sigma^{-1/2}X)(\Sigma^{-1/2}X)^T] \\ &= \operatorname{E}[(\Sigma^{-1/2}X)(X^T\Sigma^{-1/2})] \\ &= \Sigma^{-1/2}\operatorname{E}[XX^T]\Sigma^{-1/2} \\ &= \Sigma^{-1/2}\Sigma\Sigma^{-1/2} \\ &= I \end{align}$

Thus, Y is a white random vector.

Based on the fact that the covariance matrix is always positive semi-definite, $Σ 1/2$ can be derived using eigenvalue decomposition:

$\Sigma\Phi = \Phi\Lambda$

$\Sigma = \Phi\Lambda\Phi^T$

$\Sigma^{1/2} = \Phi\Lambda^{1/2}$

Where the matrix $Λ 1/2$ is a diagonal matrix with each element being square root of the corresponding element in $Λ$ . To show that this equation follows from the prior one, multiply by the transpose:

$\begin{align} \Sigma^{1/2}(\Sigma^{1/2})^T &= \Phi\Lambda^{1/2}(\Phi\Lambda^{1/2})^T \\ &= \Phi\Lambda^{1/2}(\Lambda^{1/2})^T\Phi^T \\ &= \Phi\Lambda\Phi^T \\ &= \Sigma \\ \end{align}$

免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。

猜您在找 Deep Learning 學習筆記（9）：主成分分析( PCA )與白化( whitening ) 特征的非線性變換（Feature Non-linear Transformation）創建自定義地理（坐標）轉換工具Create Custom Geographic Transformation 的地理變換方法 Spark（四）Spark之Transformation和Action Deep learning：十(PCA和whitening) 【數學】矩陣白化原理及推導白化（預處理步驟）【轉】白化（預處理步驟）【轉】數據預處理：PCA，SVD，whitening，normalization PCA和白化練習之處理圖像