Sustainable Processing-in-DRAM Integration and Replacement for Transformer Inference Servers

Overview & Synopsis

Processing-in-DRAM (PID) accelerators are emerging as a sustainable alternative to GPUs for transformer model inference. They bring compute to data, mitigate the memory bandwidth limitation, and reduce the overall energy footprint. Yet, carbon analysis of PID accelerators is covered for only a limited set of hardware and workloads. In this work, we present a carbon modeling framework for PID-enabled inference servers that captures embodied carbon along with idle and active operational carbon. Our evaluation spans three GPU baselines, six PID architectures, and eight transformer workloads. Additionally, we propose an aging-aware model that finds the carbon-optimal replacement period for PID accelerators over a 10-year horizon. Across evaluated configurations, our analysis shows that including PIDs in AI inference servers increases the total embodied carbon by at most 2.5\%, while the total carbon is reduced by up to 1.14$\times$ for encoder and 1.04$\times$ for decoder transformer workloads. The results also depend on GPU selection for pairing, where introducing PID modules in high-end HGX-H100 servers incurs 5.4--5.9$\times$ higher total carbon compared to A100 and H100 servers, due to their larger idle and embodied carbon. Introducing PID modules in higher-end GPU servers, compared to the A100-only baseline, increases (decreases) the total carbon-delay product (CDP) for encoder workloads (decoder workloads).