We've been in the kitchen cooking 🔥 Excited to release the first @AIatMeta LLama-3 8B with a context length of over 1M on @huggingface - coming off of the 160K context length model we released on Friday! A huge thank you to @CrusoeEnergy for sponsoring the compute. Let us know if you want to work with our team on custom models or automating business workflows: gradient.ai/development-lab 🔗 huggingface.co/gradientai/Ll…

We just released the first LLama-3 8B with a context length of over 160K onto Hugging Face! SOTA LLMs can learn to operate on long context with minimal training (< 200M tokens, powered by @CrusoeEnergy's compute) by appropriately adjusting RoPE theta. 🔗 huggingface.co/gradientai/Ll…