Poster
in
Workshop: Blog Track Poster Session
Thinking Like Transformers
Alexander M Rush · Gail Weiss
MH1-2-3-4 #167
Abstract:
Thinking like Transformers proposes a computational framework for Transformer-like calculations. The framework uses discrete computation to simulate Transformer computations. The resulting language RASP is a programming language where every program compiles down to a specific Transformer. In this blog post, we reimplement a variant of RASP in Python (RASPy). The language is roughly compatible with the original version, but with some syntactic changes for simplicity. With this language, we consider a challenging set of puzzles to walk through and understand how it works.
Chat is not available.