Compact Line Transformers