Pinned Tweet
๐๐๐ We've open-sourced our recent work, a cost-effective approach to accelerate LLM! ๐ Key Features:
- Memory Efficiency: Achieve up to 2.5x speedup with a minimal runtime memory overhead of just 0.0004%, making it ideal for edge and mobile environments.
- Training Efficiency: Efficient training process, requiring only 16 hours on a single A100-40GB GPU.
- Orthogonal Optimization: Orthogonal to speculative decoding, our approach provides the potential for synergistic integration.
๐ Explore our demo and code: github.com/hmarkc/parallel-pโฆ
Our MICRO'22 paper is online today at arxiv.org/abs/2209.09570! Our work received all three ACM Artifact Evaluation badges! Thanks to all the collaborators! @thomasccp @SIVenieris @kouris_a @niclane7 @mohsaied See you in Chicago!!!
