- training set： 用來訓練模型
- validation set : 用來做model selection
- test set : 用來評估所選出來的model的實際性能
我們知道，在做模型訓練之前，我們必須選擇所訓練的模型的形式：線性模型(y = wx+b)或者非線性模型(SVM,decision tree,neural network….)。選擇好模型之后，我們才會開始訓練，訓練的目標是確定模型的參數，訓練一般是通過設計損失函數，然后對損失函數進行優化來完成訓練。
而很多時候我們並不知道哪種模型適合，所以往往我們需要對多種模型進行訓練，訓練完之后就會得到多個模型的結果，我們希望從這些訓練好的模型中選擇最適合的模型。我們通過用validation set對所有模型進行測試，然后選出error rate最小的那個模型。
所以說valaidation set主要是用來選擇模型的。
The main trick here is to 'hold out' a portion of our data from training and use the models performance on that sub-set of the data as a proxy for the true risk.
This data is known as 'validation' data. It contrasts with test data, because it's values are known at the model design time. However, in contrast to test data we don't use it to fit our model.
This means that it doesn't exhibit the same bias that the empirical risk does when estimating the true risk.