Inference System

2024

MoE-Lightning: High-Throughput MoE Inference on Memory-constrained GPUs

December 31, 2024

TwinPilots: A New Computing Paradigm for GPU-CPU Parallel LLM Inference

December 30, 2024

Hybrid Heterogeneous Clusters Can Lower the Energy Consumption of LLM Inference Workloads

December 29, 2024

HETEGAN: Heterogeneous Parallel Inference for Large Language Models on Resource-Constrained Devices

December 28, 2024

FIDDLER: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models

December 27, 2024

Accelerating Distributed MoE Training and Inference with Lina

June 6, 2024