Senior Researcher @MSFTResearch. Working on Machine Learning, and Generative Models.
China
Joined February 2011
- Tweets55
- Following453
- Followers114
- Likes490
Tianyu He retweeted
Introducing Code Video Model 🎬
Amazing work led by @raywzy1 and the team!
Agentic coding as the engine for controllable video generation.
Scene layout, motion, timing, camera trajectories, and physical dynamics, all in editable code.
A coding agent turns prompts into executable programs that render proxy videos. Our training-free early-step conditioning lets a pretrained video model follow the proxy’s structure while adopting a reference image’s appearance.
From physics-grounded motion and bullet time to 3D/4D reconstruction and robotics simulation, it all starts with code.
Watch the demo and explore more visual results 👇
Project page: code-video-model.github.io/
Paper: code-video-model.github.io/t…
Code: github.com/code-video-model/…
Data: huggingface.co/datasets/code…
Youtube: youtube.com/watch?v=dTtiVkSR…
Love seeing this direction take off. In 2024, we showed a video model pick up new tasks from seconds of demo. **NO** task-specific fine-tuning or curated dataset. Just autoregressive prediction, the behavior emerged.
💥Project: microsoft.com/en-us/research…
💥Paper: arxiv.org/abs/2407.07356
Tianyu He retweeted
Today, we are introducing XYZ-Aquila-mini (35B) and XYZ-Aquila-pro (397B)—two Deep Search agents developed through our bounded, verifiable AI4AI R&D system.
Across seven Deep Search benchmarks, XYZ-Aquila-mini leads the comparable-scale systems included in our evaluation. XYZ-Aquila-pro achieves the highest publicly reported scores in our comparison on BrowseComp-ZH, LiveBrowseComp, and WideSearch.
But the bigger story is how these systems were built.
In two months, a core team of just over ten people coordinated more than 300 agent workers and thousand-accelerator-equivalent compute across task construction, model training, runtime optimization, and evaluation.
Humans defined the goals, constraints, risk boundaries, and acceptance criteria. AI analyzed failures, proposed candidate solutions, modified code and data, ran experiments, and accumulated reusable experience.
In this article, we share the results, the system behind them, and what we learned about making AI-driven R&D verifiable, auditable, and cumulative.
Read the full story below.
We build the minds that build.
Homepage: xyz-lab.ai/
Github: github.com/XYZ-AI-Lab
Tech Report: xyz-lab.ai/blogs/ai4ai-at-sc…
Tianyu He retweeted
A message to Anthropic leadership: You're not special. Making sure AI goes well is a team effort not a "you effort."
Tianyu He retweeted
Standard VLA training comes with a heavy "Embodiment Tax" 💸 : fine-tuning VLMs on action data erodes their competence.
Can we fix this WITHOUT freezing weights or using auxiliary data?
Introducing UAM: A Dual-Stream Perspective on Forgetting 🔗 [cladernyjorn.github.io/Unifi…]
Excited to share that our paper Geometry Forcing will be presented at ICLR 2026!
📍 Poster Session
🗓 Sat, Apr 25, 2026
⏰ 3:15 PM – 5:45 PM (-03)
📌 Pavilion 4, P4-#3116
We’d love to chat—feel free to stop by!
Also presenting 'Quotient-Space Diffusion Models' as an Oral:
🗓 Sat, Apr 25, 2026
⏰ 3:27 PM – 3:37 PM (-03)
📌 Room 201 C
See you there!
Tianyu He retweeted
SFT curates responses. RL curates sampling.
RL improves by curating what the model experiences: condition; distribution; weighting of what gets learned from.
Better signal curation shifts the performance-compute curve upward.
Full write-up below 👇
hendrydong.github.io/blogs/p…
Tianyu He retweeted
PERSIST is a world model that ditches pixel-based histories for a 3D world state. Instead of searching through an ever-growing sequence of past pixel observations, PERSIST retrieves spatial information from a dynamically evolving 3D representation.
This change improves the spatial memory, 3D consistency, and long-horizon stability of the model, enabling interactive experiences within coherent and evolving 3D worlds.
#MachineLearning #WorldModels #GenerativeAI #3DComputerVision #ComputerVision #Genie3 #AI
Tianyu He retweeted
It’s extremely good that Anthropic has not backed down, and it’s siginficant that OpenAI has taken a similar stance.
In the future, there will be much more challenging situations of this nature, and it will be critical for the relevant leaders to rise up to the occasion, for fierce competitors to put their differences aside. Good to see that happen today.
[4/N] The results are impressive: by aligning intermediate representation with geometry features, our method substantially improves visual quality and 3D consistency over the baseline methods.
[3/N] We introduce two complementary alignment objectives: Angular Alignment and Scale Alignment that enforce directional consistency and scale-related information respectively.
[2/N] Our solution is quite simple (therefore scalable): guide the model’s intermediate representations toward geometry-aware structure by aligning them with features from a pretrained geometric foundation model, e.g., VGGT.
[1/N] To bridge the gap between video diffusion models and the underlying dynamic 3D structure of the physical world, we propose Geometry Forcing (GF) that encourages video diffusion models to internalize 3D representations during training.
🤩Exciting to see this advancement!!
'Borrowing from both LLMs and diffusion models, we applied an advanced distillation strategy to the diffusion component that ensures both speed and quality.'
🤩Looks like that the technical solution is very similar to ours: nextframed.github.io/
Tianyu He retweeted
🚨Our Workshop on MM Foundation Models for Spatial Intelligence at ACMMM 2025 still accepts submissions — only 2 weeks left!
🏆 With Oral presentations and Best Paper Award
📅 Deadline: 11th July, 2025
📍 Details: sites.google.com/view/mm25-s…
@JiajunDeng12 @deeptimhe #ACMMM25
