用處
基於SVD實現模型壓縮以適配低功耗平台
根據nnet3bin/nnet3-copy，nnet3-copy或nnet3-am-copy的"--edits-config"參數中，新支持了以下選項：
apply-svd name=<name-pattern> bottleneck-dim=<dim>
查找所有名字與<name-pattern>匹配的組件，類型需要是AffineComponent或其子類。如果<dim>小於組件的輸入或輸出維數，則對組件參數進行奇異值分解，只保留最大<dim>奇異值，將這些組件替換為兩個組件：LinearComponent和NaturalGradientAffineComponent（的序列）。又見'reduce-rank'。
示例cd
dir=`mktemp -d`
nnet3-am-copy --edits='apply-svd name=*.affine bottleneck-dim=64' $dir/final.mdl $dir/final_svd.mdl
vimdiff <(nnet3-info --print-args=false $dir/final.raw 2>&1|sort) <(nnet3-info --print-args=false $dir/final.raw 2>&1|sort)
component-node name=tdnn1.affine component=tdnn1.affine input=lda input-dim=195 output-dim=1024
component name=tdnn1.affine type=NaturalGradientAffineComponent, input-dim=195, output-dim=1024, learning-rate=0.00136, max-change=0.75, linear-params-rms=0.4864, linear-params-row-norms=[percentiles(0,1,2,5 10,20,50,80,90 95,98,99,100)=(0.92,1.0,1.1,1.2 1.3,1.6,6.4,8.8,10 11,13,14,17), mean=5.79, stddev=3.56], linear-params-col-norms=[percentiles(0,1,2,5 10,20,50,80,90 95,98,99,100)=(5.0,5.5,8.3,12 13,14,15,17,18 19,20,20,24), mean=15.4, stddev=2.46], bias-{mean,stddev}=-0.06099,0.2027, rank-in=20, rank-out=80, num-samples-history=2000, update-period=4, alpha=4
component-node name=tdnn1.affine_a component=tdnn1.affine_a input=lda input-dim=195 output-dim=64
component-node name=tdnn1.affine_b component=tdnn1.affine_b input=tdnn1.affine_a input-dim=64 output-dim=1024
component name=tdnn1.affine_a type=LinearComponent, input-dim=195, output-dim=64, learning-rate=0.00136, max-change=0.75, params-rms=0.3461, params-row-norms=[percentiles(0,1,2,5 10,20,50,80,90 95,98,99,100)=(4.0,4.0,4.0,4.1 4.2,4.3,4.8,5.2,5.5 5.6,5.8,5.8,6.0), mean=4.81, stddev=0.496], params-col-norms=[percentiles(0,1,2,5 10,20,50,80,90 95,98,99,100)=(0.36,0.45,0.75,1.9 2.0,2.3,2.7,3.2,3.4 3.6,3.7,3.9,4.7), mean=2.7, stddev=0.618], use-natural-gradient=true, rank-in=40, rank-out=80, num-samples-history=2000, update-period=4, alpha=4
component name=tdnn1.affine_b type=NaturalGradientAffineComponent, input-dim=64, output-dim=1024, learning-rate=0.00136, max-change=0.75, linear-params-rms=0.151, linear-params-row-norms=[percentiles(0,1,2,5 10,20,50,80,90 95,98,99,100)=(0.12,0.13,0.14,0.16 0.19,0.23,1.1,1.6,1.8 2.1,2.5,2.6,3.8), mean=1, stddev=0.676], linear-params-col-norms=[percentiles(0,1,2,5 10,20,50,80,90 95,98,99,100)=(4.0,4.0,4.0,4.1 4.2,4.3,4.8,5.2,5.5 5.6,5.8,5.8,6.0), mean=4.81, stddev=0.496], bias-{mean,stddev}=-0.06099,0.2027, rank-in=20, rank-out=80, num-samples-history=2000, update-period=4, alpha=4
結果
經過解碼測試，SVD后的模型識別率極差，完全無法使用。
需要再對模型進行retrain。
使用SVD實現模型壓縮后，再進行幾輪迭代
在已有訓練樣本的情況在，假設總iteration=2000，，將final.mdl進行SVD得到final_svd.mdl，再鏈接為0.mdl，運行一個epoch：local/chain/run_tdnn.sh --stage 16 --num_epochs 1
在之前的epoch的基礎上，再訓幾個epochs
local/chain/run_tdnn.sh --stage 16 --num_epochs 2 --train_stage 1155