Hongxiang Fan (@HongxiangF) profile picture

Research scientist, Samsung AI Cambridge and University of Cambridge

Joined June 2021
๐Ÿš€๐Ÿš€๐Ÿš€ We've open-sourced our recent work, a cost-effective approach to accelerate LLM! ๐Ÿ”‘ Key Features: - Memory Efficiency: Achieve up to 2.5x speedup with a minimal runtime memory overhead of just 0.0004%, making it ideal for edge and mobile environments. - Training Efficiency: Efficient training process, requiring only 16 hours on a single A100-40GB GPU. - Orthogonal Optimization: Orthogonal to speculative decoding, our approach provides the potential for synergistic integration. ๐Ÿ”— Explore our demo and code: github.com/hmarkc/parallel-pโ€ฆ
1
5
11
2,244