This paper examines how skip connections prevent rank collapse in sequence models.
problem Rank collapse in sequence models, leading to reduced expressivity and training instabilities.
method Analytical and ablation studies of lambda-skip connections in SSMs.
result A sufficient condition to prevent rank collapse across various architectures.