元學習——從MAML到MAML++

作者：凱魯嘎吉 - 博客園 http://www.cnblogs.com/kailugaji/

    Few-shot learning領域最近有了實質性的進展。這些進步大多來自於將few-shot learning作為元學習問題。Model-Agnostic Meta-Learning (MAML)是目前利用元學習進行few-shot learning的最佳方法之一。MAML簡單，優雅，功能強大，但是它有很多問題，比如對神經網絡結構非常敏感，經常導致訓練時不穩定，需要費力的超參數搜索來穩定訓練和實現高泛化，並且在訓練和推理時間上都非常昂貴的計算。在文"How to train your MAML"中，對MAML進行了各種改進，不僅穩定了系統，而且大幅度提高了MAML的泛化性能、收斂速度和計算開銷。所提方法稱之為MAML++。本博文首先介紹什么是元學習，經典的Model-Agnostic Meta-Learning的定義與執行過程，進而說明MAML面臨的缺點與挑戰，針對這些問題，進行相應改進，從而得到MAML++。

1. Meta Learning (Learn to Learn)

2. Black-Box Adaption vs Optimization-Based Approach

3. MAML

MAML Computation Graph

4. MAML Problems

5. MAML++

MAML with Multi-Step Loss Computation Graph

6. 參考文獻

[1] Finn, C., Abbeel, P. & Levine, S. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. ICML 2017. Code: https://github.com/cbfinn/maml, https://github.com/dragen1860/MAML-Pytorch

Finn個人主頁：https://ai.stanford.edu/~cbfinn/

[2] Antoniou, A.,  Edwards, H., &  Storkey, A. How to train your MAML. ICLR 2019. Code: https://github.com/AntreasAntoniou/HowToTrainYourMAMLPytorch

[3] How to train your MAML: A step by step approach · BayesWatch https://www.bayeswatch.com/2018/11/30/HTYM/

[4] CS 330: Deep Multi-Task and Meta Learning http://web.stanford.edu/class/cs330/

[5] Meta-Learning: Learning to Learn Fast https://lilianweng.github.io/lil-log/2018/11/30/meta-learning.html