貝葉斯網絡——看來我要的是參數評估

Python的貝葉斯網絡學習庫pgmpy介紹和使用

 

文章目錄

• pgmpy
• • 代碼記錄
  • • 包
    • parameter Learning
    • structural Learning with Score
    • structural Learning with Constraint
    • structural Learning with Hybrid
    • mian
  • 運行展示
  • • 參數學習
    • 評分搜索
    • 約束
    • 混合

 

---

pgmpy

Parameter learning: Given a set of data samples and a DAG that captures the dependencies between the variables, estimate the (conditional) probability distributions of the individual variables.

Structure learning: Given a set of data samples, estimate a DAG that captures the dependencies between the variables.

pgmpy.org
github.com/pgmpy/pgmpy_notebook/blob/master/blob/master/notebooks

代碼記錄

包

""" 學習鏈接 ： http://pgmpy.org/ https://github.com/pgmpy/pgmpy_notebook/blob/master/notebooks/9.%20Learning%20Bayesian%20Networks%20from%20Data.ipynb """ # ====================BN模型========================= # 貝葉斯模型 from pgmpy.models import BayesianModel # ====================參數學習========================= # 參數估計 from pgmpy.estimators import ParameterEstimator # MLE參數估計 from pgmpy.estimators import MaximumLikelihoodEstimator # Bayesian參數估計 from pgmpy.estimators import BayesianEstimator # ====================結構學習========================= # ========評分搜索========================= # 評分 from pgmpy.estimators import BdeuScore, K2Score, BicScore # 窮舉搜索 from pgmpy.estimators import ExhaustiveSearch # 爬山搜索 from pgmpy.estimators import HillClimbSearch # ======== 約束 ========================= from pgmpy.estimators import ConstraintBasedEstimator # 獨立性 from pgmpy.independencies import Independencies # ======== 混合 ========================= from pgmpy.estimators import MmhcEstimator # ==================== 通用庫 ========================= import pandas as pd import numpy as np 

parameter Learning

def parameterLearning(): data = pd.DataFrame(data={'fruit': ["banana", "apple", "banana", "apple", "banana","apple", "banana", "apple", "apple", "apple", "banana", "banana", "apple", "banana",], 'tasty': ["yes", "no", "yes", "yes", "yes", "yes", "yes", "yes", "yes", "yes", "yes", "no", "no", "no"], 'size': ["large", "large", "large", "small", "large", "large", "large", "small", "large", "large", "large", "large", "small", "small"]}) model = BayesianModel([('fruit', 'tasty'), ('size', 'tasty')]) # fruit -> tasty <- size print("========================================================")  pe = ParameterEstimator(model, data) print("\n", pe.state_counts('fruit')) # unconditional print("\n", pe.state_counts('size')) # unconditional print("\n", pe.state_counts('tasty')) # conditional on fruit and size print("========================================================") mle = MaximumLikelihoodEstimator(model, data) print(mle.estimate_cpd('fruit')) # unconditional print(mle.estimate_cpd('tasty')) # conditional  print("========================================================") est = BayesianEstimator(model, data) print(est.estimate_cpd('tasty', prior_type='BDeu', equivalent_sample_size=10)) # Setting equivalent_sample_size to 10 means # that for each parent configuration, we add the equivalent of 10 uniform samples # (here: +5 small bananas that are tasty and +5 that aren't). print("========================================================") # Calibrate all CPDs of `model` using MLE: model.fit(data, estimator=MaximumLikelihoodEstimator) print("========================================================") # generate data data = pd.DataFrame(np.random.randint(low=0, high=2, size=(5000, 4)), columns=['A', 'B', 'C', 'D']) model = BayesianModel([('A', 'B'), ('A', 'C'), ('D', 'C'), ('B', 'D')]) model.fit(data, estimator=BayesianEstimator, prior_type="BDeu") # default equivalent_sample_size=5 for cpd in model.get_cpds(): print(cpd) 

從數據中學習貝葉斯網絡的CPD參數

正常情況下，我們手頭上有的只是數據，對一些CPD的參數值我們通常情況下無法獲取，或者獲取的代價比較大，那么怎么從數據中學習到貝葉斯網絡的參數以及結構呢？這里，我們首先講解一下參數的學習，即CPD的參數學習。通常采用的方式有：極大似然估計和貝葉斯估計，極大似然估計對樣本數量的要求比較高，特別是當數據分布不均勻時，容易發生過擬合線性，為了解決這一問題，通常是采用貝葉斯估計的方式進行參數學習…

首先，先造一些數據

import pandas as pd data = pd.DataFrame(data={'fruit': ["banana", "apple", "banana", "apple", "banana","apple", "banana", "apple", "apple", "apple", "banana", "banana", "apple", "banana",], 'tasty': ["yes", "no", "yes", "yes", "yes", "yes", "yes", "yes", "yes", "yes", "yes", "no", "no", "no"], 'size': ["large", "large", "large", "small", "large", "large", "large", "small", "large", "large", "large", "large", "small", "small"]}) data 

 

	fruit	size	tasty
0	banana	large	yes
1	apple	large	no
2	banana	large	yes
3	apple	small	yes
4	banana	large	yes
5	apple	large	yes
6	banana	large	yes
7	apple	small	yes
8	apple	large	yes
9	apple	large	yes
10	banana	large	yes
11	banana	large	no
12	apple	small	no
13	banana	small	no

在知道模型結構的情況下，構建貝葉斯網絡模型

from pgmpy.models import BayesianModel model = BayesianModel([('fruit', 'tasty'), ('size', 'tasty')]) 

參數學習是一項估計條件概率分布（CPD）值的任務，它涉及到水果、大小和美味等變量。

為了理解給定的數據，我們可以從計算變量的每個狀態發生的頻率開始。如果變量依賴於父級，則根據父級狀態有條件地進行計數，即對於每個父級配置分別進行計數：

from pgmpy.estimators import ParameterEstimator pe = ParameterEstimator(model, data) print("\n", pe.state_counts('fruit')) print("\n", pe.state_counts('tasty')) # 在fruit和size的條件下，tasty的頻數 

         fruit
apple       7
banana      7

 fruit apple       banana      
size  large small  large small
tasty                         
no      1.0   1.0    1.0   1.0
yes     3.0   2.0    5.0   0.0

極大似然估計

from pgmpy.estimators import MaximumLikelihoodEstimator mle = MaximumLikelihoodEstimator(model, data) print("\n", mle.estimate_cpd('fruit')) print("\n", mle.estimate_cpd('tasty')) # 在fruit和size的條件下，tasty的概率分布 mle.get_parameters() 

 +---------------+-----+
| fruit(apple)  | 0.5 |
+---------------+-----+
| fruit(banana) | 0.5 |
+---------------+-----+

 +------------+--------------+--------------------+---------------------+---------------+
| fruit      | fruit(apple) | fruit(apple)       | fruit(banana)       | fruit(banana) |
+------------+--------------+--------------------+---------------------+---------------+
| size       | size(large)  | size(small)        | size(large)         | size(small)   |
+------------+--------------+--------------------+---------------------+---------------+
| tasty(no)  | 0.25         | 0.3333333333333333 | 0.16666666666666666 | 1.0           |
+------------+--------------+--------------------+---------------------+---------------+
| tasty(yes) | 0.75         | 0.6666666666666666 | 0.8333333333333334  | 0.0           |
+------------+--------------+--------------------+---------------------+---------------+

[<TabularCPD representing P(fruit:2) at 0x24daed0e278>,
 <TabularCPD representing P(size:2) at 0x24daed0e400>,
 <TabularCPD representing P(tasty:2 | fruit:2, size:2) at 0x24daed0e518>]

model.fit(data, estimator=MaximumLikelihoodEstimator) 

print(model.get_cpds('fruit')) print(model.get_cpds('size')) print(model.get_cpds('tasty')) 

+---------------+-----+
| fruit(apple)  | 0.5 |
+---------------+-----+
| fruit(banana) | 0.5 |
+---------------+-----+
+-------------+----------+
| size(large) | 0.714286 |
+-------------+----------+
| size(small) | 0.285714 |
+-------------+----------+
+------------+--------------+--------------------+---------------------+---------------+
| fruit      | fruit(apple) | fruit(apple)       | fruit(banana)       | fruit(banana) |
+------------+--------------+--------------------+---------------------+---------------+
| size       | size(large)  | size(small)        | size(large)         | size(small)   |
+------------+--------------+--------------------+---------------------+---------------+
| tasty(no)  | 0.25         | 0.3333333333333333 | 0.16666666666666666 | 1.0           |
+------------+--------------+--------------------+---------------------+---------------+
| tasty(yes) | 0.75         | 0.6666666666666666 | 0.8333333333333334  | 0.0           |
+------------+--------------+--------------------+---------------------+---------------+

變量估計

from pgmpy.inference import VariableElimination infer = VariableElimination(model) for i in infer.query(['tasty', 'size', 'fruit']).values(): # 打印P(tasty), P(size), P(fruit) print(i) print('大，香蕉是美味的概率:\n', infer.query(['tasty'], evidence={'fruit': 1, 'size': 0})['tasty']) # 大，香蕉是否美味的概率 

+---------+--------------+
| tasty   |   phi(tasty) |
+=========+==============+
| tasty_0 |       0.3393 |
+---------+--------------+
| tasty_1 |       0.6607 |
+---------+--------------+
+---------+--------------+
| fruit   |   phi(fruit) |
+=========+==============+
| fruit_0 |       0.5000 |
+---------+--------------+
| fruit_1 |       0.5000 |
+---------+--------------+
+--------+-------------+
| size   |   phi(size) |
+========+=============+
| size_0 |      0.7143 |
+--------+-------------+
| size_1 |      0.2857 |
+--------+-------------+
大，香蕉是美味的概率:
 +---------+--------------+
| tasty   |   phi(tasty) |
+=========+==============+
| tasty_0 |       0.1667 |
+---------+--------------+
| tasty_1 |       0.8333 |
+---------+--------------+

雖然非常簡單，但ML估計量存在數據擬合過度的問題。在上述CPD中，大型香蕉美味的概率估計為0.833，因為6個觀察到的大型香蕉中有5個美味。但是請注意，一個小香蕉好吃的概率估計為0.0，因為我們只觀察到一個小香蕉，而它恰好不好吃。但這很難讓我們確定小香蕉不好吃！我們只是沒有足夠的觀測數據來依賴觀測到的頻率。如果觀察到的數據不能代表潛在分布，那么ML估計將非常遙遠。

在估計貝葉斯網絡參數時, 數據不足是一個常見的問題。即使總樣本量非常大, 則對於每個父配置, 狀態計數是有條件地完成的, 這將導致巨大的碎片。如果一個變量有3個父項, 每個父項可以采用10個狀態, 則 10^3 = 1000個父級配置將分別進行狀態計數。這使得 MLE 在學習貝葉斯網絡參數時非常脆弱和不穩定。一種緩解 MLE 超擬合的方法是貝葉斯參數估計。

貝葉斯參數估計

貝葉斯參數估計是從已有的CPD開始的，它表達了我們在觀察數據之前對變量的看法。然后，利用觀測數據中的狀態計數更新這些“先驗”。

我們可以認為先驗由偽狀態計數組成，這些偽狀態計數在標准化之前添加到實際計數中。除非要對變量分布的特定信念進行編碼，否則人們通常會選擇統一的先驗，即認為所有狀態都是可均等的先驗。

一個非常簡單的先驗是所謂的k2先驗，它只是在每個狀態的計數上加1。一個更明智的先驗選擇是bdeu（bayesian-dirichlet等價一致先驗）。對於bdeu，我們需要指定一個等效的樣本大小n，然后偽計數等於觀察到每個變量的n個均勻樣本（以及每個父配置）。

補充：Dirichlet分布

簡單的例子來說明。假設你手上有一枚六面骰子。你拋擲1000次，得到一個朝向的分布p1 = <H1, H2, H3, H4, H5, H6>。H1是指數字1朝上次數，H2是指數字2朝上次數， H3， H4， H5， H6依次類推。你再拋擲1000次，又會得到一個朝向的分布p2。重復N次之后，你就會得到N個分布：p1, p2, p3, ... , pn. 假如有這樣一個分布D，能夠描述拋這枚骰子1000次，得到p1的概率是多少，那么我們就可以簡單地把D理解為分布在pi之上的分布。而pi本身又是一個分布，所以D就是分布的分布。Dirichlet分布，可以理解為多項式分布的分布。它的一個樣本點是一個多項式分布。鏈接：https://www.zhihu.com/question/23749913/answer/135084553

通俗易懂的理解見：https://www.cnblogs.com/bonelee/p/14329635.html

from pgmpy.estimators import BayesianEstimator esy = BayesianEstimator(model, data) print(esy.estimate_cpd('tasty', prior_type='BDeu', equivalent_sample_size=10)) 

+------------+---------------------+--------------------+--------------------+---------------------+
| fruit      | fruit(apple)        | fruit(apple)       | fruit(banana)      | fruit(banana)       |
+------------+---------------------+--------------------+--------------------+---------------------+
| size       | size(large)         | size(small)        | size(large)        | size(small)         |
+------------+---------------------+--------------------+--------------------+---------------------+
| tasty(no)  | 0.34615384615384615 | 0.4090909090909091 | 0.2647058823529412 | 0.6428571428571429  |
+------------+---------------------+--------------------+--------------------+---------------------+
| tasty(yes) | 0.6538461538461539  | 0.5909090909090909 | 0.7352941176470589 | 0.35714285714285715 |
+------------+---------------------+--------------------+--------------------+---------------------+

CPD中的估計值現在更加保守。特別是，對一個不好吃的小香蕉的估計現在大約是0.64而不是1.0。將equivalent_sample_size設置為10意味着，對於每個父級配置，我們添加等效的10個均勻樣本（這里+5個美味的小香蕉和+5個不美味的小香蕉）。

model2 = BayesianModel([('fruit', 'tasty'), ('size', 'tasty')]) model2.fit(data, estimator=BayesianEstimator) for i in model2.get_cpds(): print(i) infer2 = VariableElimination(model2) print("大，香蕉是否美味的概率分布：\n", infer2.query(['tasty'], evidence={'fruit': 1, 'size': 0})['tasty']) 

+------------+---------------------+---------------------+---------------------+--------------------+
| fruit      | fruit(apple)        | fruit(apple)        | fruit(banana)       | fruit(banana)      |
+------------+---------------------+---------------------+---------------------+--------------------+
| size       | size(large)         | size(small)         | size(large)         | size(small)        |
+------------+---------------------+---------------------+---------------------+--------------------+
| tasty(no)  | 0.30952380952380953 | 0.38235294117647056 | 0.22413793103448276 | 0.7222222222222222 |
+------------+---------------------+---------------------+---------------------+--------------------+
| tasty(yes) | 0.6904761904761905  | 0.6176470588235294  | 0.7758620689655172  | 0.2777777777777778 |
+------------+---------------------+---------------------+---------------------+--------------------+
+---------------+-----+
| fruit(apple)  | 0.5 |
+---------------+-----+
| fruit(banana) | 0.5 |
+---------------+-----+
+-------------+----------+
| size(large) | 0.657895 |
+-------------+----------+
| size(small) | 0.342105 |
+-------------+----------+
大，香蕉是否美味的概率分布：
 +---------+--------------+
| tasty   |   phi(tasty) |
+=========+==============+
| tasty_0 |       0.2241 |
+---------+--------------+
| tasty_1 |       0.7759 |
+---------+--------------+

先到參數學習這里，下一節：structure learning

structural Learning with Score

def structuralLearning_Score(): """ score-based structure learning constraint-based structure learning The combination of both techniques allows further improvement: hybrid structure learning """ print("===================基於評分=================================") # create random data sample with 3 variables, where Z is dependent on X, Y: data = pd.DataFrame(np.random.randint(0, 4, size=(5000, 2)), columns=list('XY')) data['Z'] = data['X'] + data['Y'] bdeu = BdeuScore(data, equivalent_sample_size=5) k2 = K2Score(data) bic = BicScore(data) model1 = BayesianModel([('X', 'Z'), ('Y', 'Z')]) # X -> Z <- Y model2 = BayesianModel([('X', 'Z'), ('X', 'Y')]) # Y <- X -> Z print("==========基於評分===model1===============") print(bdeu.score(model1)) print(k2.score(model1)) print(bic.score(model1)) print("==========基於評分===model2===============") print(bdeu.score(model2)) print(k2.score(model2)) print(bic.score(model2)) print("==========基於評分===局部評分==============") print(bdeu.local_score('Z', parents=[])) print(bdeu.local_score('Z', parents=['X'])) print(bdeu.local_score('Z', parents=['X', 'Y'])) print("==========基於評分===窮舉搜索算法==============") # 窮舉搜索（計算困難），啟發式搜索 es = ExhaustiveSearch(data, scoring_method=bic) # 獲取分數最高的分數 best_model = es.estimate() print(best_model.edges()) print("\n 遍歷所有的分數:") for score, dag in reversed(es.all_scores()): print(score, dag.edges()) print("==========基於評分===爬山搜索算法==============") data = pd.DataFrame(np.random.randint(0, 3, size=(2500, 8)), columns=list('ABCDEFGH')) data['A'] += data['B'] + data['C'] data['H'] = data['G'] - data['A'] hc = HillClimbSearch(data, scoring_method=BicScore(data)) best_model = hc.estimate() print(best_model.edges()) 

structural Learning with Constraint

def structuralLearning_Constraint(): print("===================基於約束=================================") # Identify independencies in the data set using hypothesis tests # Construct DAG (pattern) according to identified independencies data = pd.DataFrame(np.random.randint(0, 3, size=(2500, 8)), columns=list('ABCDEFGH')) data['A'] += data['B'] + data['C'] data['H'] = data['G'] - data['A'] data['E'] *= data['F'] # Independencies in the data can be identified # using chi2 conditional independence tests est = ConstraintBasedEstimator(data) print("==========基於約束===條件獨立測試===============") # test_conditional_independence(X, Y, Zs) # 判斷X，Y在Zs的條件下是否條件獨立 # check if X is independent from Y given a set of variables Zs: print(est.test_conditional_independence('B', 'H')) # dependent False print(est.test_conditional_independence('B', 'E')) # independent True print(est.test_conditional_independence('B', 'H', ['A'])) # independent True print(est.test_conditional_independence('A', 'G')) # independent True print(est.test_conditional_independence('A', 'G', ['H'])) # dependent False print("==========基於約束===DAG構建=================") """ 1. 構造一個無向骨架——estimate_skeleton() 2. 利用強迫邊進行定向，得到部分有向無環圖(PDAG;- skeleton_to_pdag() 3. 通過以某種方式保守地定向剩余的邊，將DAG模式擴展到DAG—pdag_to_dag() Step 1.&2. form the so-called PC algorithm. PDAGs are DirectedGraphs, that may contain both-way edges, to indicate that the orientation for the edge is not determined. """ skel, seperating_sets = est.estimate_skeleton(significance_level=0.01) print("Undirected edges: ", skel.edges()) pdag = est.skeleton_to_pdag(skel, seperating_sets) print("PDAG edges: ", pdag.edges()) model = est.pdag_to_dag(pdag) print("DAG edges: ", model.edges()) print("==========基於約束===DAG構建===estimate方法=================") # he estimate()-method provides a shorthand for the three steps above # and directly returns a BayesianModel # 三步並作一步，直接返回一個網絡結構 print(est.estimate(significance_level=0.01).edges()) print("==========基於約束===DAG構建===從independencies中學習======") ind = Independencies(['B', 'C'], ['A', ['B', 'C'], 'D']) ind = ind.closure() # required (!) for faithfulness model = ConstraintBasedEstimator.estimate_from_independencies("ABCD", ind) print(model.edges()) 

structural Learning with Hybrid

def structuralLearning_Hybrid(): """ MMHC算法[3]結合了基於約束和基於分數的方法。它有兩部分: 1. 使用基於約束的構造過程MMPC學習無向圖骨架 2. 基於分數的優化(BDeu分數+修改爬山) """ print("===================混合方法=================================") # 實驗數據生成 data = pd.DataFrame(np.random.randint(0, 3, size=(2500, 8)), columns=list('ABCDEFGH')) data['A'] += data['B'] + data['C'] data['H'] = data['G'] - data['A'] data['E'] *= data['F'] # 構建無向圖骨架 mmhc = MmhcEstimator(data) skeleton = mmhc.mmpc() print("Part 1) Skeleton: ", skeleton.edges()) # 基於分數優化 # use hill climb search to orient the edges: hc = HillClimbSearch(data, scoring_method=BdeuScore(data)) model = hc.estimate(tabu_length=10, white_list=skeleton.to_directed().edges()) print("Part 2) Model: ", model.edges()) print("===================兩步划為一步=================================") # MmhcEstimator.estimate(self, scoring_method=None, tabu_length=10, # significance_level=0.01) # mmhc.estimate(scoring_method=BdeuScore(data),tabu_length=10) 

mian

if __name__ == "__main__": parameterLearning() structuralLearning_Score() structuralLearning_Constraint() structuralLearning_Hybrid() 

---

運行展示

參數學習

1 ============參數估計=================================================
1 ========state_counts 統計信息=======

         fruit
apple       7
banana      7

        size
large    10
small     4

 fruit apple       banana      
size  large small  large small
tasty                         
no      1.0   1.0    1.0   1.0
yes     3.0   2.0    5.0   0.0
1 ========MLE估計CPD==按變量===========
+---------------+-----+
| fruit(apple)  | 0.5 |
+---------------+-----+
| fruit(banana) | 0.5 |
+---------------+-----+
+------------+--------------+--------------------+---------------------+---------------+
| fruit      | fruit(apple) | fruit(apple)       | fruit(banana)       | fruit(banana) |
+------------+--------------+--------------------+---------------------+---------------+
| size       | size(large)  | size(small)        | size(large)         | size(small)   |
+------------+--------------+--------------------+---------------------+---------------+
| tasty(no)  | 0.25         | 0.3333333333333333 | 0.16666666666666666 | 1.0           |
+------------+--------------+--------------------+---------------------+---------------+
| tasty(yes) | 0.75         | 0.6666666666666666 | 0.8333333333333334  | 0.0           |
+------------+--------------+--------------------+---------------------+---------------+
1 ========貝葉斯估計CPD==按變量========
+------------+---------------------+--------------------+--------------------+---------------------+
| fruit      | fruit(apple)        | fruit(apple)       | fruit(banana)      | fruit(banana)       |
+------------+---------------------+--------------------+--------------------+---------------------+
| size       | size(large)         | size(small)        | size(large)        | size(small)         |
+------------+---------------------+--------------------+--------------------+---------------------+
| tasty(no)  | 0.34615384615384615 | 0.4090909090909091 | 0.2647058823529412 | 0.6428571428571429  |
+------------+---------------------+--------------------+--------------------+---------------------+
| tasty(yes) | 0.6538461538461539  | 0.5909090909090909 | 0.7352941176470589 | 0.35714285714285715 |
+------------+---------------------+--------------------+--------------------+---------------------+
1 ========fit函數估計===所有變量=======
1 ===================================
+------+----------+
| A(0) | 0.506593 |
+------+----------+
| A(1) | 0.493407 |
+------+----------+
+------+--------------------+---------------------+
| A    | A(0)               | A(1)                |
+------+--------------------+---------------------+
| B(0) | 0.5183395779925064 | 0.48076533711277586 |
+------+--------------------+---------------------+
| B(1) | 0.4816604220074936 | 0.5192346628872241  |
+------+--------------------+---------------------+
+------+--------------------+--------------------+---------------------+--------------------+
| A    | A(0)               | A(0)               | A(1)                | A(1)               |
+------+--------------------+--------------------+---------------------+--------------------+
| D    | D(0)               | D(1)               | D(0)                | D(1)               |
+------+--------------------+--------------------+---------------------+--------------------+
| C(0) | 0.5160626836434867 | 0.5142942227516378 | 0.49917576756645377 | 0.4964179104477612 |
+------+--------------------+--------------------+---------------------+--------------------+
| C(1) | 0.4839373163565132 | 0.4857057772483621 | 0.5008242324335462  | 0.5035820895522388 |
+------+--------------------+--------------------+---------------------+--------------------+
+------+--------------------+--------------------+
| B    | B(0)               | B(1)               |
+------+--------------------+--------------------+
| D(0) | 0.5029982010793523 | 0.4918114639504693 |
+------+--------------------+--------------------+
| D(1) | 0.4970017989206476 | 0.5081885360495306 |
+------+--------------------+--------------------+

評分搜索

2 ===================基於評分=================================
2 ==========基於評分===model1===============
-13939.038934816337
-14329.822136429982
-14295.079563299281
2 ==========基於評分===model2===============
-20900.389985754824
-20927.22925737244
-20944.436530518695
2 ==========基於評分===局部評分==============
-9232.535088735991
-6990.879293129073
-57.11895038935745
2 ==========基於評分===窮舉搜索算法==============
[('X', 'Z'), ('Y', 'Z')]

	 遍歷所有的分數:
-14295.079563299281 [('X', 'Z'), ('Y', 'Z')]
-14326.77068416731 [('X', 'Y'), ('Z', 'X'), ('Z', 'Y')]
-14326.770684167312 [('Y', 'X'), ('Z', 'X'), ('Z', 'Y')]
-14326.770684167312 [('Y', 'Z'), ('Y', 'X'), ('Z', 'X')]
-14326.770684167312 [('X', 'Z'), ('Y', 'Z'), ('Y', 'X')]
-14326.770684167312 [('X', 'Y'), ('X', 'Z'), ('Z', 'Y')]
-14326.770684167312 [('X', 'Y'), ('X', 'Z'), ('Y', 'Z')]
-16536.707465219723 [('X', 'Y'), ('Z', 'Y')]
-16537.846854154086 [('Y', 'X'), ('Z', 'X')]
-18701.669239663883 [('Z', 'X'), ('Z', 'Y')]
-18701.669239663883 [('Y', 'Z'), ('Z', 'X')]
-18701.669239663886 [('X', 'Z'), ('Z', 'Y')]
-20911.606020716295 [('Z', 'Y')]
-20911.606020716295 [('Y', 'Z')]
-20912.745409650663 [('Z', 'X')]
-20912.745409650663 [('X', 'Z')]
-20943.297141584328 [('Y', 'X'), ('Z', 'Y')]
-20943.297141584328 [('Y', 'Z'), ('Y', 'X')]
-20943.297141584328 [('X', 'Y'), ('Y', 'Z')]
-20944.436530518695 [('X', 'Z'), ('Y', 'X')]
-20944.436530518695 [('X', 'Y'), ('Z', 'X')]
-20944.436530518695 [('X', 'Y'), ('X', 'Z')]
-23122.682190703075 []
-23154.373311571104 [('Y', 'X')]
-23154.373311571104 [('X', 'Y')]
2 ==========基於評分===爬山搜索算法==============
[('A', 'H'), ('A', 'C'), ('A', 'B'), ('C', 'B'), ('G', 'H')]

約束

3 ===================基於約束=================================
3 ==========基於約束===條件獨立測試===============
False
True
True
True
False
3 ==========基於約束===DAG構建=================
Undirected edges:  [('A', 'B'), ('A', 'C'), ('A', 'H'), ('E', 'F'), ('G', 'H')]
PDAG edges:        [('A', 'H'), ('B', 'A'), ('C', 'A'), ('E', 'F'), ('F', 'E'), ('G', 'H')]
DAG edges:         [('A', 'H'), ('B', 'A'), ('C', 'A'), ('F', 'E'), ('G', 'H')]
3 ==========基於約束===DAG構建===estimate方法=================
[('A', 'H'), ('B', 'A'), ('C', 'A'), ('F', 'E'), ('G', 'H')]
3 ==========基於約束===DAG構建===從independencies中學習======
[('A', 'D'), ('B', 'D'), ('C', 'D')]

混合

4 ===================混合方法=================================
Part 1) Skeleton:  [('A', 'H'), ('A', 'C'), ('E', 'F'), ('G', 'H')]
Part 2) Model:     [('A', 'C'), ('E', 'F'), ('H', 'A'), ('H', 'G')]
4 ===================兩步划為一步=================================
。。。