This is SParTA: A modular, efficient and scalable transformer architecture for jet-tagging

Overview & Synopsis

We present the Sparse Particle-interaction Transformer Architecture (SParTA), a modular, scalable, and efficient transformer for jet tagging at high-energy colliders. Exploiting the steeply falling transverse-momentum ($p_T$) spectrum of jet constituents, SParTA factorizes constituent processing into two topological regimes. Ordered by descending $p_T$, only the $N$ hardest constituents (the hard core) participate in quadratic $O(N^2)$ differential interaction attention driven by pairwise physical and learned relational features. The remaining soft constituents are integrated via linear $O(lN)$ cross-attention to the hard core, preserving soft-radiation and color-flow information while avoiding the $O(l^2)$ memory bottleneck of full self-attention over all $l$ constituents. Pairwise interaction features are computed dynamically in-network from constituent four-momenta and conditioned on global jet-level substructure observables. On the quark-gluon tagging benchmark, a 300K-parameter SParTA model achieves an AUC of 92.02% and accuracy of 85.06%, matching the 2.14M-parameter Particle Transformer (ParT) at one-seventh the parameter count and outperforming IAFormer by 21% in background rejection at 30% signal efficiency. On the boosted top-tagging benchmark, SParTA outperforms ParT and ParticleNet with 511K parameters, while we analyze the architectural trade-offs relative to fully dense multi-layer interaction networks.