AI research scientist @axon_us. Previously @UW and @Tsinghua_Uni, co-worked with @uwnlp in AI/ML/NLP.
Bellevue, WA
Joined February 2020
- Tweets26
- Following264
- Followers68
- Likes161
Bowen Zhao retweeted
Excited to share some new work: we show how to efficiently train small vision-language models to use a zoom tool with GRPO. We identify key points for doing small-scale RL for tool use in VLMs generally, and share what works (and what doesn’t) under significant resource constraints. arXiv link below ⬇️
Bowen Zhao retweeted
We've open-sourced a MCP that allows big models to use huggingface computer vision models as tools.
This allows Claude to act as a "visual agent", using other task specific models to help it solve problems.
Below, is an example of Claude using an open vocab object detector to zoom in on small details to solve a hard problem that it could not solve natively.
Additionally, we've written a blog post discussing why outsourcing vision capabilities from large models is something you should consider.
MCP Repo: github.com/groundlight/mcp-v…
Blog post: groundlight.ai/blog/vision-a…
Bowen Zhao retweeted
We just released an open-source framework that makes it easy to build visual reasoning agents (with GRPO).
github.com/groundlight/r1_vl…
It’s my pleasure making some minor contribution to this great project! This attention rollout video is my favorite “visualization of the year” so far. Try out our demo if you are interested and stay tuned for our future research!
We trained a Visual LLM to reason using GRPO, and open sourced the code. Tiny 3B model beats all the big players (GPT, Claude, etc 0-shot) after RL training on this cryptogram task. Live demo and links: groundlight.ai/blog/visual-r…
Vision foundation models have kinda stagnated recently, but now that we have shown how to incorporate reason, I think we'll be able to make progress again.
Bowen Zhao retweeted
Just dropped ⚡ - our latest youtube tutorial, running Groundlight's computer vision on a Raspberry Pi! Watch it here: youtube.com/watch?v=YpNKHjuZ…
Tim Huff, our Robotics Engineer, walks you step-by-step how to create an application to get notified if someone steals your parking spot.
Unlike traditional computer vision:
☑ No need for a dataset
☑ Works on day one
☑ Adapts to your real environment
Experiments show that APT maintains 98% task performance when pruning 60% of the parameters in small models, and it preserves 86.4% of LLaMA's performance with 70% density. Furthermore, APT speeds up LMs’ finetuning by up to 8x and reduces LLMs’ training memory cost by up to 70%.
Based on these intuitions, we design APT adapters that learn the pruning masks via an outlier-aware salience scoring function to remove irrelevant LM parameter blocks and add more tuning parameters during fine-tuning according to tuning layer importance.
Our intuition is that LM parameters' importance to downstream tasks varies, so pruning insensitive weights earlier is a way. Also, we notice that task-specific skills of LMs live in a subset of its parameters, so adding more tuning parameters may help convergence.
As modern LLMs scale larger, training and serving them on existing hardware becomes challenging. However, methods like PEFT only improve training efficiency, and pruning only improves inference efficiency at the cost of extensive training. APT combines the benefits of both!
Bowen Zhao retweeted
#CVPR2024 down the street gets our science team out of the office!
10,000+ computer vision researchers in Seattle figuring out what's next in the field from multimodal foundation models to the most creative applications.
Bowen Zhao retweeted
🚀 Exciting News from Groundlight AI! 🚀 Introducing, the Groundlight Hub, your easy access to computer vision.
youtu.be/u3xPnOnvTcE
Bowen Zhao retweeted
Our Robotics Engineer, Tim Huff, gives a demo of Groundlight AI's Visual Inspections 🔍 and Anomaly Detection solution, compatible with @Universal_Robot 🤖 cobots:
Bowen Zhao retweeted
Importance of Demand Generation ✍️
🌟 The number of GPUs we own and the robustness of our L1 integration are crucial, but they're not the sole determinants of our long-term success. It's all about creating demand for our product.
🥅 Our ultimate goal is to stimulate demand for our GPU nodes through diverse integrations and utilities. We're committed to providing solutions that meet the needs of our users and drive adoption.
📈 In just the last 3 days, we've seen a significant uptick, accumulating nearly 400 additional rental hours for our $GPUs. This demonstrates the growing interest and demand for our platform.
🥇 Our primary focus remains clear: to establish #NodeAI as the premier infrastructure provider for high-performance AI.
Demand generation is key, and we're dedicated to making it happen.
#AI #DEPIN #GPU
