IEEE

Design Considerations for LLM Inference in Data Centers: Chip and Interconnect

Abstract: This paper presents a cost-efficient chip prototype optimized for large language model (LLM) inference. We identify four key specifications – computational FLOPs (flops), memory bandwidth ...