Concept: Interleaved memory

CXL-SpecKV: A Disaggregated FPGA Speculative KV-Cache for Datacenter LLM Serving

Offloading memory to remote accelerators improves LLM inference speed and reduces costs