Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Blog Track Poster Session

Thinking Like Transformers

Alexander M Rush · Gail Weiss

MH1-2-3-4 #167

Abstract:

Thinking like Transformers proposes a computational framework for Transformer-like calculations. The framework uses discrete computation to simulate Transformer computations. The resulting language RASP is a programming language where every program compiles down to a specific Transformer. In this blog post, we reimplement a variant of RASP in Python (RASPy). The language is roughly compatible with the original version, but with some syntactic changes for simplicity. With this language, we consider a challenging set of puzzles to walk through and understand how it works.

Chat is not available.