Probability和Likelihood的區別

Bayes for Beginners: Probability and Likelihood 好好看，非常有用。

以前死活都不理解Probability和Likelihood的區別，為什么這兩個東西的條件反一下就相等。

定義：

Probability是指在固定參數的情況下，事件的概率，必須是0-1，事件互斥且和為1. 我們常見的泊松分布、二項分布、正態分布的概率密度圖描述的就是這個。

Likelihood是指固定的結果，我們的參數的概率，和不必為1，不必互斥，所以只有ratio是有意義的。

 

至於為什么L=P，這是因為定義就是這樣的，wiki解釋得非常清楚。

 

Likelihood function

Consider a simple statistical model of a coin flip, with a single parameter  that expresses the "fairness" of the coin. This parameter is the probability that a given coin lands heads up ("H") when tossed.  can take on any numeric value within the range 0.0 to 1.0. For a perfectly fair coin,  = 0.5.

Imagine flipping a coin twice, and observing the following data : two heads in two tosses ("HH"). Assuming that each successive coin flip is IID, then the probability of observing HH is

Hence: given the observed data HH, the likelihood that the model parameter  equals 0.5, is 0.25. Mathematically, this is written as

This is not the same as saying that the probability that , given the observation HH, is 0.25. (For that, we could apply Bayes' theorem, which implies that the posterior probability is proportional to the likelihood times the prior probability.)

Suppose that the coin is not a fair coin, but instead it has . Then the probability of getting two heads is

Hence

More generally, for each value of , we can calculate the corresponding likelihood. The result of such calculations is displayed in Figure 1.

In Figure 1, the integral of the likelihood over the interval [0, 1] is 1/3. That illustrates an important aspect of likelihoods: likelihoods do not have to integrate (or sum) to 1, unlike probabilities.