✨ bitsandbytes now has native ZeroGPU support since the new multi-backend refactor.
This shows how central @PyTorch has become in the AI landscape:
🔗 Extend torch the right way (custom ops, modes, etc.) and your software plugs seamlessly into a thriving ecosystem
Recently finished writing a new blogpost about @PyTorch compilation in ZeroGPU Spaces.
Worth reading if you're interested in learning about :
- PyTorch ahead-of-time compilation
- ZeroGPU internals
huggingface.co/blog/zerogpu-…
Charles retweeted
Wrote an FA3 attention processor for @Alibaba_Qwen Image using the 🤗 Kernels library. The process is so enjoyable!
Stuff cooking stuff coming 🥠
gist.github.com/sayakpaul/ff…
Charles retweeted
Felt frustrated when using `torch.compile` as it takes forever? 🤬
You SHOULD switch to regional compilation & see if it is just as beneficial as using full compilation.
Let the numbers (Flux.1-Dev) convince you 🫡
🚀 ZeroGPU v2 update
We just switch to Nvidia H200 last week
It means that @huggingface Spaces are now equipped with:
- 🧠 70GB vram
- ⚡ 2.5x more flops
🔓This will hopefully unlock unseen use cases
💰 It also makes Pro plan a seriously cheap CUDA compute option
Charles retweeted
ComfyUI → @huggingface Spaces → serverless ZeroGPU ✨😌
We wrote a tutorial on how to turn any ComfyUI workflow into an easy to use Gradio app and (optionally) host it for free with ZeroGPU 💥
huggingface.co/blog/run-comf…
⚡We've just rolled out a major update on ZeroGPU!
Major improvements:
- 2x faster GPU coldstarts
- More efficient CPU memory usage (meaning more slots for the community)
- ZeroGPU initialization now displays a progress bar
- Greatly improved PyTorch compatibility
Charles retweeted
GPU-Poor no more: super excited to officially release ZeroGPU in beta today. Congrats @victormustar & team for the release!
In the past few months, the open-source AI community has been thriving. Not only Meta but also Apple, NVIDIA, Bytedance, Snowflake, Databricks, Microsoft, Google, and more have released open models and datasets on Hugging Face, which now hosts over 1M models on the Hub which have been downloaded over a billion times. More than that, many are starting to be better than proprietary APIs.
This movement has been supported not only by big tech but also by a thriving open-source AI community that includes academic labs, startups, and independent hobbyists. For example, more than 35,000 variation models of Llama have been shared on Hugging Face since Meta’s first version a year ago—including more than 7,000 based on Llama-3—ranging from quantized and merged models to specialized models in biology and Mandarin, to name a few. More than 4 million AI builders are now using Hugging Face.
However, the open-source community doesn’t have the same resources available to train and demo these models that big tech have at their disposal, which is why ChatGPT remains the most used AI application today.
@huggingface is fighting this by launching ZeroGPU, a shared infrastructure for indie and academic AI builders to run AI demos on Spaces, giving them the freedom to pursue their work without the financial burden of compute costs. Spaces have been the most popular way to build AI demos, with over 300,000 AI demos created so far on CPU or paid GPU (and a thousand more every day). To foster the continued development of the AI ecosystem, Hugging Face is committing $10M of free GPUs with the launch today of ZeroGPU.
Technically speaking, ZeroGPU leverages Hugging Face's experience in hosting and serving more than 100 Petabytes monthly from the Hugging Face Hub. ZeroGPU allows Spaces to run on multiple GPUs by making Spaces efficiently hold and release GPUs as needed (as opposed to a classical GPU Space that holds exactly one GPU at any time). This architecture is also more energy-efficient since GPUs are shared rather than duplicated. ZeroGPU uses @nvidia A100 GPU devices under the hood.
You can learn more about ZeroGPU here: huggingface.co/zero-gpu-expl…
More than 1,300 ZeroGPU spaces have been built since we started giving early access to AI builders on May 1, 2024: huggingface.co/spaces/enzost…
You can explore some examples from @victormustar: teapaw.org/victormustar/sta…
You can find the article from @kyliebytes: theverge.com/2024/5/16/24156…
🤗🤗🤗
Charles retweeted
Replying to @huggingface
@huggingface GPU zero are now running on A100!
huggingface.co/zero-gpu-expl…
By adding a simple annotation, your Spaces with grants are able to run
- on multiple GPUs
- on demand GPUs, release as needed
Come join the org and start making awesome demos on many A100!
Model downloads from @huggingface Spaces should be way faster overall
This hf.space/cbensimon/llama-2-1… Llama-13B Space boots in 30s instead of 15min previously. That's a 30x speedup
It is running on a new Space hardware 🤫 but you can expect decent speed-ups on regular Spaces too
Hugging Face Spaces builds should now be faster overall
Major improvement is cache restore with an up to 4x speedup
On the screenshot, restoring 6GB of Docker cache takes 17s compared to a previous ~70s
There's still a 3x speedup left with further optimizations
Charles retweeted
Added a full LLM type system and context-free grammar to Python. Includes all typescript types, regexes, and arbitrary grammars. Makes coding programs with LLM calls woven in super easy.
Thanks @ggerganov for llama.cpp and @abetlen for llama-cpp-python that enable this.
Charles retweeted
Some people said that closed APIs were winning...
but we will never give up the fight for open source AI ⚔️⚔️
Today is a big day as we launch the first open source alternative to ChatGPT:
HuggingChat 💬
Powered by Open Assistant's latest model – the best open source chat model right now – and @huggingface Inference API.
Try it out now:
hf.co/chat
Charles retweeted
float i,e,R,s,x,y,z;vec3 a,q,p,d=.5-FC.rgb/r.y;for(q.yz--,a=q;i++<1e2;i>76.?e+=1e-4,d=a:d){e*=R;p=q+=d*e*.5;o+=log(++e+cos(z/6.+vec4(0,1,2,0))*.3)/1e2;p=vec3(log(R=length(p))-t*.5,e=-p.z/R,atan(p.x,p.y));for(z=s=1.;s<1e2;s+=s)e+=exp(x=sin(y=PI*(dot(cos(p*s),p/p)))-1.4)/s,z+=x/y;}



























