Building a Minimal Transformer From Scratch in PyTorch — and Comparing It With an RNN
This is a transformer from scratch in PyTorch build, compared directly against an RNN so the difference isn’t just theoretical. In the previous article, we explored the idea behind Transformers: instead of processing a sequence strictly one element at a time, a Transformer uses self-attention to learn which parts of that sequence matter to one…