什么是Condition Number（條件數）?

In the field of numerical analysis, the condition number of a function with respect to an argument measures how much the output value of the function can change for a small change in the input argument. This is used to measure how sensitive a function is to changes or errors in the input, and how much error in the output results from an error in the input. Very frequently, one is solving the inverse problem – given  one is solving for x, and thus the condition number of the (local) inverse must be used. In linear regression the condition number can be used as a diagnostic for multicollinearity.

在數值分析領域，一個函數關於一個參數的條件數（Condition Number）測量了函數的輸出值相對於輸入參數的變化強度。這用來測量一個函數相對於輸入變化或誤差有多敏感，以及輸出結果相對於輸入中的誤差的誤差變化。非常頻繁地，一個是解決逆問題f(x)=y，一個是解決x，因此必須使用逆的條件數。在線性擬合中，條件數可以用來做多重共線性的診斷。

The condition number is an application of the derivative, and is formally defined as the value of the asymptotic worst-case relative change in output for a relative change in input. The "function" is the solution of a problem and the "arguments" are the data in the problem. The condition number is frequently applied to questions in linear algebra, in which case the derivative is straightforward but the error could be in many different directions, and is thus computed from the geometry of the matrix. More generally, condition numbers can be defined for non-linear functions in several variables.

條件數是導數的一個應用，而且定義為相對於輸入值的改變而輸出值相對漸進最壞的改變。“函數”是一個問題的解決方法，而“參數”是問題的數據。條件數頻繁地被應用於線性代數問題，其中導數是前向的，但是誤差可以是多種不同方向的，因此通過矩陣的幾何而計算。更一般地，條件數可以定義在帶有多個變量的非線性函數中。

A problem with a low condition number is said to be well-conditioned, while a problem with a high condition number is said to be ill-conditioned. The condition number is a property of the problem. Paired with the problem are any number of algorithms that can be used to solve the problem, that is, to calculate the solution. Some algorithms have a property called backward stability. In general, a backward stable algorithm can be expected to accurately solve well-conditioned problems. Numerical analysis textbooks give formulas for the condition numbers of problems and identify known backward stable algorithms.

帶有低條件數的問題被稱為良好-條件的，而帶有高的條件數的問題則被稱為病態-條件的。條件數是一個問題的屬性。跟一個問題匹配的是任何可以用來解決該問題的算法數量，那就是，計算該解決方案。一些算法有一個屬性稱為反向穩定。一般來說，反向穩定的算法可以用來精確地解決良好-條件的問題。數值分析書給出了問題的條件數公式，並且命名為反向穩定算法。

As a rule of thumb, if the condition number , then you may lose up to  digits of accuracy on top of what would be lost to the numerical method due to loss of precision from arithmetic methods.^[3] However, the condition number does not give the exact value of the maximum inaccuracy that may occur in the algorithm. It generally just bounds it with an estimate (whose computed value depends on the choice of the norm to measure the inaccuracy).

憑經驗法估計，如果條件數，那么由於數學算法的精度缺失你可能失去在數值方法上k點的精度。但是，條件數並不給出算法中可能出現的精確的最大失真值。它通常只是給出一個估計值（它的計算值取決於測量失真的norm(范數)的選擇）。

Matrices矩陣

For example, the condition number associated with the linear equation Ax = b gives a bound on how inaccurate the solution x will be after approximation. Note that this is before the effects of round-off error are taken into account; conditioning is a property of the matrix, not the algorithm or floating point accuracy of the computer used to solve the corresponding system. In particular, one should think of the condition number as being (very roughly) the rate at which the solution, x, will change with respect to a change in b. Thus, if the condition number is large, even a small error in b may cause a large error in x. On the other hand, if the condition number is small then the error in x will not be much bigger than the error in b.

例如，與線性等式Ax=b相關的條件數給出解決方案x在近似之后的不精確的邊界。注意這在考慮四舍五入取整化零的影響之前；條件是該矩陣的屬性，不是算法或浮點數准確性。尤其是，應該把條件數想象成解決方案x將跟隨b的變化的變化率。因此，如果條件數大，那么哪怕是b中很小的誤差也可能會引起x中特別大的誤差。另一方面，如果條件數小，那么x重的誤差將不會比b中的誤差大多少。

The condition number is defined more precisely to be the maximum ratio of the relative error in x to the relative error in b.

條件數更精確地定義為x中的相對誤差與b中的相對誤差的最大比值。

Let e be the error in b. Assuming that A is a nonsingular matrix, the error in the solution A⁻¹b is A⁻¹e. The ratio of the relative error in the solution to the relative error in b is

當e表示b中的誤差。假設A是一個非奇異矩陣，解決方案A⁻¹b中的誤差為A⁻¹e。解決方案中的相對誤差相對於b中的相對誤差的比率為

This is easily transformed to

這很容易轉換成

The maximum value (for nonzero b and e) is then seen to be the product of the two operator norms as follows:

最大值（對於非零b和e）因此可以看作是這兩個操作范數的積如下所示：

The same definition is used for any consistent norm, i.e. one that satisfies

When the condition number is exactly one (which can only happen if A is a scalar multiple of a linear isometry), then a solution algorithm can find (in principle, meaning if the algorithm introduces no errors of its own) an approximation of the solution whose precision is no worse than that of the data.

However, it does not mean that the algorithm will converge rapidly to this solution, just that it won't diverge arbitrarily because of inaccuracy on the source data (backward error), provided that the forward error introduced by the algorithm does not diverge as well because of accumulating intermediate rounding errors.^{[clarification needed]}

The condition number may also be infinite, but this implies that the problem is ill-posed (does not possess a unique, well-defined solution for each choice of data -- that is, the matrix is not invertible), and no algorithm can be expected to reliably find a solution.

The definition of the condition number depends on the choice of norm, as can be illustrated by two examples.

If   is the norm (usually noted as ) defined in the square-summable sequence space ℓ² (which matches the usual distance in a standard Euclidean space), then

 

where  and  are maximal and minimal singular values of  respectively. Hence

•  If  is normal then

 

where  and  are maximal and minimal (by moduli) eigenvalues of  respectively.

• If  is unitary then

The condition number with respect to L² arises so often in numerical linear algebra that it is given a name, the condition number of a matrix.

If   is the norm (usually denoted by ) defined in the sequence space ℓ^∞ of all bounded sequences (which matches the maximum of distances measured on projections into the base subspaces), and  is lower triangular non-singular (i.e., ) then

The condition number computed with this norm is generally larger than the condition number computed with square-summable sequences, but it can be evaluated more easily (and this is often the only practicably computable condition number, when the problem to solve involves a non-linear algebra^{[clarification needed]}, for example when approximating irrational and transcendental functions or numbers with numerical methods).

If the condition number is not too much larger than one (but it can still be a multiple of one), the matrix is well conditioned which means its inverse can be computed with good accuracy. If the condition number is very large, then the matrix is said to be ill-conditioned. Practically, such a matrix is almost singular, and the computation of its inverse, or solution of a linear system of equations is prone to large numerical errors. A matrix that is not invertible has condition number equal to infinity.

 

 

>>補充知識：

矩陣 A 的條件數等於 A 的范數與 A 的逆的范數的乘積，即 cond(A)=‖A‖·‖A^(-1)‖，對應矩陣
的 3 種范數，相應地可以定義 3 種條件數。（矩陣的范數有哪幾種？）

函數 cond(A,1)、cond(A)戒 cond(A inf) 是判斷矩陣病態與否的一種度量，條件數越大矩陣越病態。條件數事實上表示了矩陣計算對於誤差的敏感性。對於線性方程組 Ax=b，如果 A 的條件數大，b 的微小改變就能引起解 x 較大的改變，數值穩定性差。如果 A 的條件數小，b 有微小的改變，x 的改變也很微小，數值穩定性好。它也可以表示 b 不變，而 A 有微小改變時，x 的變化情況。比如線性方程組。

的解是(x,y)=(0.0,0.1), 

而 

的解是(x,y)=(-0.17,0.22)，可見 b 很小的擾動就引起了 x 很大的變化，這就是 A 矩陣條件數大的表現。

一個極端的例子，當 A 奇異時，條件數為無窮，這時即使不改變 b，x 也可以改變。奇異的本質原因在於矩陣有 0 特征值，x 在對應特征向量的方向上運動不改變 Ax 的值。如果一個特征值比其它特征值在數量級上小很多，x 在對應特征向量方向上很大的移動才能產生 b 微小的變化，這就解釋了為什么這個矩陣為什么會有大的條件數，事實上，正則陣在二范數下的條件數就可以表示成 abs(最大特征值/最小特征值)。

 

參考：https://en.wikipedia.org/wiki/Condition_number

https://www.cnblogs.com/hxsyl/p/5071434.html

https://blog.csdn.net/lanchunhui/article/details/51372831