Study shows Transformers can generalize to varying task lengths.
problem Understanding when and how Transformers can generalize to different input lengths.
method Proposed a unifying framework and introduced the RASP-Generalization Conjecture.
result Transformers tend to length generalize on tasks if solvable by short RASP programs.