WARNING: THIS SITE IS A MIRROR OF GITHUB.COM / IT CANNOT LOGIN OR REGISTER ACCOUNTS / THE CONTENTS ARE PROVIDED AS-IS / THIS SITE ASSUMES NO RESPONSIBILITY FOR ANY DISPLAYED CONTENT OR LINKS / IF YOU FOUND SOMETHING MAY NOT GOOD FOR EVERYONE, CONTACT ADMIN AT ilovescratch@foxmail.com
Skip to content

fix: hidden state#0 number #58

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
conglanjun wants to merge 2 commits into
base: main
Choose a base branch
from
Open

fix: hidden state#0 number #58

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b679e39
fix: hidden state#0 number
Apr 23, 2025
f5d3b2d
fix: hidden state#0 number
Apr 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/篇章2-Transformer相关原理/2.1-图解attention.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,9 @@ for Statistical Machine Translation](http://emnlp2014.org/papers/pdf/EMNLP201417

让我们来进一步可视化一下基于RNN的seq2seq模型中的编码器在第1个时间步是如何工作：

![rnn](./pictures/1-6-rnn.gif) 动态图：如图所示，RNN在第2个时间步，采用第1个时间步得到hidden state#10（隐藏层状态）和第2个时间步的输入向量input#1，来得到新的输出hidden state#1。
![rnn](./pictures/1-6-rnn.gif) 动态图：如图所示，RNN在第2个时间步，采用第1个时间步得到hidden state#0（隐藏层状态）和第2个时间步的输入向量input#1，来得到新的输出hidden state#1。

看下面的动态图，让我们详细观察一下编码器如何在每个时间步得到hidden sate，并将最终的hidden state传输给解码器，解码器根据编码器所给予的最后一个hidden state信息解码处输出序列。注意，最后一个 hidden state实际上是我们上文提到的context向量。
看下面的动态图，让我们详细观察一下编码器如何在每个时间步得到hidden sate，并将最终的hidden state传输给解码器，解码器根据编码器所给予的最后一个hidden state信息解码输出序列。注意，最后一个 hidden state实际上是我们上文提到的context向量。
![](./pictures/1-6-seq2seq.gif) 动态图：编码器逐步得到hidden state并传输最后一个hidden state给解码器。

接着，结合编码器处理输入序列，一起来看下解码器如何一步步得到输出序列的l。与编码器类似，解码器在每个时间步也会得到 hidden state（隐藏层状态），而且也需要把 hidden state（隐藏层状态）从一个时间步传递到下一个时间步。
Expand Down