Public Writing
Attention is not all you need: on LLMs as Creative Machines
What changes when a technical system is described with a word we also use for human experience?
This essay questions the use of “attention” to describe how language models work. It asks what the term brings with it from older ideas of mind, will, and concentration.
Rather than treating the model as a mind choosing what to focus on, it explores distraction and atmosphere as other ways of thinking about our work with these systems.
- Status
- Unpublished essay.