增廣拉格朗日乘子法（Augmented Lagrange Method）

轉載自：增廣拉格朗日乘子法（Augmented Lagrange Method）

增廣拉格朗日乘子法的作用是用來解決等式約束下的優化問題，

 

假定需要求解的問題如下：

　　　　minimize　　 f(X)

　　　　s.t.:　　　　　h(X)=0

其中，f:Rⁿ->R; h:Rⁿ->R^m

 

朴素拉格朗日乘子法的解決方案是：

　　　　L(X,λ)=f(X)+μh(X);　　μ:R^m

　　　　此時，求解L對X和μ的偏導同時為零就可以得到最優解了。

 

增廣拉格朗日乘子法的解決方案是：

　　　　L_c(x,λ)=f(X)+μh(X)+^₁/₂c|h(X)|²

　　　　每次求出一個xi，然后按照梯度更新參數μ，c每次迭代逐漸增大（使用ALM方法好像還有一些假設條件）

　　　　整個流程只需要幾步就可以完成了，一直迭代就可得到最優解了。

　　　　

 

 

參考文獻：

　　[1]Multiplier and Gradient Methods,1969

　　[2]constrained optimization and lagrange multiplier methods(page 104),1982

 

wiki：https://en.wikipedia.org/wiki/Augmented_Lagrangian_method

Let us say we are solving the following constrained problem:

subject to

This problem can be solved as a series of unconstrained minimization problems. For reference, we first list the penalty method approach:

The penalty method solves this problem, then at the next iteration it re-solves the problem using a larger value of  (and using the old solution as the initial guess or "warm-start").

The augmented Lagrangian method uses the following unconstrained objective:

and after each iteration, in addition to updating, the variable  is also updated according to the rule

where  is the solution to the unconstrained problem at the kth step, i.e. 

The variable  is an estimate of the Lagrange multiplier, and the accuracy of this estimate improves at every step. The major advantage of the method is that unlike the penalty method, it is not necessary to take  in order to solve the original constrained problem. Instead, because of the presence of the Lagrange multiplier term,  can stay much smaller.

The method can be extended to handle inequality constraints. For a discussion of practical improvements, see.^[4]