论文笔记-Sequence to Sequence Learning with Neural Networks

本文转载自查看原文 2017-12-23 16:37 1479 deep learning/ 论文笔记

大体思想和RNN encoder-decoder是一样的，只是用来LSTM来实现。

paper提到三个important point：

1）encoder和decoder的LSTM是两个不同的模型

2）deep LSTM表现比shallow好，选用了4层的LSTM

3）实践中发现将输入句子reverse后再进行训练效果更好。So for example, instead of mapping the sentence a,b,c to the sentence α,β,γ, the LSTM is asked to map c,b,a to α,β,γ, where α, β, γ is the translation of a, b, c. This way, a is in close proximity to α, b is fairly close to β, and so on, a fact that makes it easy for SGD to “establish communication” between the input and the output.

免责声明！

本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系本站邮箱yoyou2525@163.com删除。

猜您在找 论文笔记《Notes on convolutional neural networks》论文笔记——NEURAL ARCHITECTURE SEARCH WITH REINFORCEMENT LEARNING 论文笔记：Learning Dynamic Memory Networks for Object Tracking 论文笔记之：Semi-Supervised Learning with Generative Adversarial Networks 《Learning Convolutional Neural Networks for Graphs》论文阅读【论文笔记】Progressive Neural Networks 渐进式神经网络论文笔记：Diffusion-Convolutional Neural Networks （传播-卷积神经网络）《C-RNN-GAN: Continuous recurrent neural networks with adversarial training》论文笔记论文笔记：Learning regression and verification networks for long-term visual tracking