1、What is Maximum Likelihood?
極大似然是一種找到最可能解釋一組觀測數據的函數的方法。
Maximum Likelihood is a way to find the most likely function to explain a set of observed data.
在基本統計學中，通常給你一個模型來計算概率。例如，你可能被要求找出X大於2的概率，給定如下泊松分布:X ~ Poisson (2.4)。在這個例子中，已經給定了你泊松分布的參數 λ(2.4)，在現實生活中，您沒有這么奢侈，因為您沒有確定參數的模型:您必須將數據與模型相匹配。這就是最大可能性(MLE)的作用。在統計學中，最大似然估計(maximum likelihood estimation, MLE)是在給定觀測值的情況下估計統計模型參數的一種方法。MLE試圖在給定觀測值的情況下找到使似然函數最大化的參數值。得到的估計稱為最大似然估計，也縮寫為MLE。
In elementary statistics, you are usually given a model to find probabilities. For example, you might be asked to find the probability that X is greater than 2, given the following Poisson distribution: X ~ Poisson (2.4) In this example, you are given the parameter, λ, of 2.4 for the Possion distribution. In real life, you don’t have the luxury of having a model given to you: you’ll have to fit your data to a model. That’s where Maximum Likelihood (MLE) comes in.
In statistics, maximum likelihood estimation (MLE) is a method of estimating the parameters of a statistical model, given observations. MLE attempts to find the parameter values that maximize the likelihood function, given the observations. The resulting estimate is called a maximum likelihood estimate, which is also abbreviated as MLE.
MLE采用已知的概率分布模型(如正態分布)，並將數據集與這些分布進行比較，以便找到數據的合適匹配。一個分布模型對應的參數可以有無窮個。例如正態分布的均值可以是0，也可以是100億以上。最大似然估計是找到最可能生成待測樣本的總體參數的一種方法。數據與模型的匹配程度稱為“擬合優度”。
MLE takes known probability distributions (like the normal distribution) and compares data sets to those distributions in order to find a suitable match for the data. A Family of distributions can have an infinite amount of possible parameters. For example, the mean of the normal distribution could be equal to zero, or it could be equal to ten billion and beyond. Maximum Likelihood Estimation is one way to find the parameters of the population that is most likely to have generated the sample being tested. How well the data matches the model is known as “Goodness of Fit.”
例如，研究人員可能有興趣找出吃特定食物的老鼠的平均體重增加。研究人員無法測量每只老鼠的體重，所以只能取樣。大鼠體重增加呈正態分布;最大似然估計可用於求基於該樣本的總體增重的均值和方差
For example, a researcher might be interested in finding out the mean weight gain of rats eating a particular diet. The researcher is unable to weigh every rat in the population so instead takes a sample. Weight gains of rats tend to follow a normal distribution; Maximum Likelihood Estimation can be used to find the mean and variance of the weight gain in the general population based on this sample
MLE根據似然函數的最大值來選擇模型參數。
MLE chooses the model parameters based on the values that maximize the Likelihood Function.
2、The Likelihood Function（似然函數，是一種表示概率的方法；似然表示得到樣本的概率；最大似然表示的是得到樣本最大概率的參數）
給定一個特定的概率分布模型，樣本的似然是得到樣本的概率。似然函數是一種表示概率的方法:最大概率得到樣本的參數是最大似然估計。
一句話：似然表示概率；似然函數表示得到概率的方法；最大似然表示的得到最大概率的參數
The likelihood of a sample is the probability of getting that sample, given a specified probability distribution model. The likelihood function is a way to express that probability: the parameters that maximize the probability of getting that sample are the Maximum Likelihood Estimators.
假設你有一組從一個未知分布參數Θ的總體得到的隨機變量X1, X2…Xn。該分布的概率密度函數(PDF) f(Xi,Θ)模型,Xi是隨機變量的集合,Θ是未知參數。最大似然函數你想知道Θ最可能的值是什么,得到隨機變量Xi。本例的聯合概率密度函數為:
Let’s suppose you had a set of random variables X1, X2…Xn taken from an unknown population distribution with parameter Θ. This distribution has a probability density function (PDF) of f(Xi,Θ) where f is the model, Xi is the set of random variables and Θ is the unknown parameter. For the maximum likelihood function you want to know what the most likely value for Θ is, given the set of random variables Xi. The joint probability density function for this example is:
3、The Basic Idea
It seems reasonable that a good estimate of the unknown parameter θ would be the value of θ that maximizes the probability, errrr... that is, the likelihood... of getting the data we observed. (So, do you see from where the name "maximum likelihood" comes?) So, that is, in a nutshell, the idea behind the method of maximum likelihood estimation. But how would we implement the method in practice? Well, suppose we have a random sample X1, X2,..., Xn for which the probability density (or mass) function of each Xi is f(xi; θ). Then, the joint probability mass (or density) function of X1, X2,..., Xn, which we'll (not so arbitrarily) call L(θ) is:
The first equality is of course just the definition of the joint probability mass function. The second equality comes from that fact that we have a random sample, which implies by definition that the Xi are independent. And, the last equality just uses the shorthand mathematical notation of a product of indexed terms. Now, in light of the basic idea of maximum likelihood estimation, one reasonable way to proceed is to treat the "likelihood function" L(θ) as a function of θ, and find the value of θ that maximizes it.
4、example1
假設權重隨機選擇的美國女大學生與未知的正態分布均值μ和標准差σ。隨機抽取的10名美國女大學生的體重(以磅為單位)如下:
115 122 130 127 149 160 152 138 149 180
根據上面給出的定義,識別似然函數和μ的極大似然估計量,所有的美國女大學生的平均重量。使用給定的樣本,找到一個最大似然估計的μ。
Based on the definitions given above, identify the likelihood function and the maximum likelihood estimator of μ, the mean weight of all American female college students. Using the given sample, find a maximum likelihood estimate of μ as well.
5、example2
Suppose we have a random sample X1, X2,..., Xn where:
- Xi = 0 if a randomly selected student does not own a sports car, and
- Xi = 1 if a randomly selected student does own a sports car.
Assuming that the Xi are independent Bernoulli random variables with unknown parameter p, find the maximum likelihood estimator of p, the proportion of students who own a sports car.
6、文獻
https://newonlinecourses.science.psu.edu/stat414/node/191/（寫的很好，里面有很多的例子）
https://en.wikipedia.org/wiki/Maximum_likelihood_estimation
https://www.statisticshowto.datasciencecentral.com/maximum-likelihood-estimation/