Principal Researcher at Microsoft
Joined August 2016
- Tweets61
- Following43
- Followers71
- Likes361
NorthCUA is now open source 🚀
We’re releasing Nano 9B, Mini 35B and Pro 397B, alongside our Harness framework and a 123-page technical report.
Our goal: agents that can take a task from user intent to completed work across real desktop and web applications.
Huge thanks to everyone who helped bring this project to life. Try it on your own tasks, share what works, and help us push CUAs further.
🌐 Project: beipo.top/blog/northcua/en/
🤗 Models: huggingface.co/beipo-ai/Nort…
💻 Code: github.com/beipo-ai/NorthCUA
📄 Report: arxiv.org/pdf/2611.02102v1
Getting there takes more than where to click. NorthCUA connects scalable task environments, verified interaction experience and staged training into a complete learning loop. RSI reusable skills and preserves successful milestones, helping agents build on earlier progress.
Try the N2.5 CUA capabilities and see how far they can go! 🚀
And we’re just getting started:
🧩 NexCUA, our dedicated CUA harness
🤖 NexCUA-specialized models
are coming soon.
Stay tuned. More action ahead. 👀
Meet Nex-N2.5 family, our latest open source agentic models.
Mini (35B) and Pro (397B) bring multimodal understanding and computer use. Max (1.6T) is a text-only MoE model built for complex reasoning, coding, and agent workflows.
Strong benchmark results in workflow automation and computer use:
⚡ Max scores 50.2 on AutomationBench v1.0.6—just 0.1 points behind Claude Opus 5
⚡ Pro scores 56.4 on OSWorld-2, ahead of Qwen3.8-Max at 46.7
With continuous action, visual feedback, and self-correction, NEX-N2.5 can work across Blender and CAD tools, handle expense reimbursements, and even play PC games—turning visual understanding into sustained, real-world execution.
🔗@huggingface
huggingface.co/nex-agi/Nex-N… huggingface.co/nex-agi/Nex-N… huggingface.co/nex-agi/Nex-N…
🔗@modelscope
modelscope.cn/models/nex-agi… modelscope.cn/models/nex-agi… modelscope.cn/models/nex-agi…
🔗 Github github.com/nex-agi/Nex-N2.5
🔗Website nex-agi.com/
We built a Guangzhou 3D map simulator with Nex N2.
From OpenStreetMap data to an interactive 3D geospatial environment:
- tilted 3D map view
- simulated sun, weather, traffic, and camera controls
- landmark annotations
- agent-ready urban navigation / spatial reasoning scenarios
This is the kind of lightweight “world simulator” we want Nex N2 to help build: not just answering questions, but turning real-world data into interactive environments for agents.
Try Nex N2 Pro:
OR: openrouter.ai/nex-agi/nex-n2…
HF: huggingface.co/nex-agi/Nex-N…
#NexN2 #NexAGI #LLMAgent
This is the kind of lightweight “world simulator” we want Nex N2 to help build: not just answering questions, but turning real-world data into interactive environments for agents.
Try Nex N2 Pro:
OR: openrouter.ai/nex-agi/nex-n2…
HF: huggingface.co/nex-agi/Nex-N…
#NexN2 #NexAGI #LLMAgents
Please do apply if you are interested!
🔊2026 Summer Internship @MSFTResearch Deep Learning Group🔊
We’re looking for a self-motivated intern with strong background on ⛑️building GUI agent environments and/or 🏗️reinforcement learning.
📩Interested? Send your CV + a short intro to [email protected]!
Grateful to AK for highlighting UFO³! 🛸
Proud of this collaboration — building an agent framework that scales across devices and unlocks next-gen automation.
📄 Paper: arxiv.org/abs/2511.11332
▶️ Demo: youtube.com/watch?v=NGrVWGcJ…
💻 Code: github.com/microsoft/UFO
🚀 UFO³ — Weaving the Digital Agent Galaxy 🛸
Every device you own, servers, laptops, mobiles, working together as a single AI workforce 💻📱.
The era of multi-device agents begins 🌌.
Paper: arxiv.org/abs/2511.11332
Code: github.com/microsoft/UFO
Demo: youtube.com/watch?v=NGrVWGcJ…
Proud to share our new work: GUI-Actor! 🎯
A coordinate-free GUI grounding approach inspired by how humans perceive & act.
Outperforms models 10x its size — and the verifier helps others too!
📄 arXiv: 2506.03143
🌐 microsoft.github.io/GUI-Acto…
🚀 Excited to share GUI-Actor—a new approach for GUI grounding! Big thanks to @_akhaliq for featuring our work!
🌐 Project page: microsoft.github.io/GUI-Acto…
📜 Paper: arxiv.org/pdf/2506.03143
🤔 What's limiting coordinate generation-based GUI grounding?
1️⃣ Weak spatial-semantic alignment
2️⃣ Ambiguous supervision signals
3️⃣ Vision–action granularity mismatch
👀 But think about it: humans don’t calculate precise screen coordinates—we perceive elements and then act directly.
💡 Meet GUI-Actor: a VLM with an attention-based action head that:
✅ Addresses above limitations
✅ Proposes multiple candidate regions in one pass, enabling flexible downstream strategies.
✅ Performs coordinate-free grounding that better mirrors human behavior
➕ We also introduce a grounding verifier to select the most plausible action region — and it can boost other grounding methods too.
🎯 Results? GUI-Actor achieves SOTA on several benchmarks, even GUI-Actor-7B outperforms UI-TARS-72B on ScreenSpot-Pro, all using the same Qwen2-VL backbone.
Proud to share our new work — SWE-bench-Live is now live!
A live-updating benchmark for real-world bug fixing, where even top agents like Claude 3.7 Sonnet + OpenHands stumble.
Try it out & follow us 👉 swe-bench-live.github.io
#LLM #SWEbenchLive #AIforCode
🤔 Have we really made great progress on software engineering tasks?
🚀 Introducing SWE-bench-Live, a live-updatable benchmark for real-world bug fixing.
😺 Even the best combo, OpenHands + Claude 3.7 Sonnet, sees a major performance drop!
👉 swe-bench-live.github.io/
🧵 1/4
Code repo: github.com/microsoft/SWE-ben…
Paper: arxiv.org/pdf/2505.23419
Try it out!
![GitHub - microsoft/SWE-bench-Live: [NeurIPS 2025 D&B] 🚀 SWE-bench Goes Live!](https://pbs.twimg.com/card_img/2117249631033937921/FdI5uwrr?format=jpg&name=800x419)
🚀 UFO2 – The Desktop AgentOS is here!
Multi‑agent automation inside Windows, featuring:
• Hybrid GUI + native API actions
• Picture‑in‑Picture virtual desktop
• +SOTA task‑success & ‑51 % LLM calls
Paper, Code & docs ➜ github.com/microsoft/UFO
#LLMAgents #DesktopAutomation
Chaoyun Zhang retweeted
1/ VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Model
The paper introduces VEM, an innovative environment-free reinforcement learning framework that trains GUI agents using a pretrained Value Environment Model to estimate state-action values from offline data, significantly improving task success rates and efficiency without requiring direct environmental interactions.
Jiani Zheng, Lu Wang, Fangkai Yang, Chaoyun Zhang(@vyokky ), Lingrui Mei, Wenjie Yin, Qingwei Lin, Dongmei Zhang, Saravan Rajmohan, Qi Zhang
arxiv.org/pdf/2502.18906
