Repeated theme: use lighter "augmentation" of attention (e.g. clustering, dotprod, lsh lookup, etc) to vet only highly relevant tokens for expensive attention.
gashon
/ long-transformers.md
Last active
October 11, 2025 08:51
WIP: Long Context Transformers notes