
Researcher @AlibabaGroup; Phd @ZJU_China; Before: Visiting Phd @UCLA; Intern @MSFTResearch; @AntGroup
Joined April 2022
- Tweets106
- Following202
- Followers127
- Likes448
Pinned Tweet
🚨 New Blog Drop!
🚀 "Reflection on Knowledge Editing: Charting the Next Steps" is live!
💡 Ever wondered why knowledge editing in LLMs still feels more like a lab experiment than a real-world solution? In this post, we dive deep into where the research is thriving — and where it's falling short. From foundational breakthroughs to the practical roadblocks no one’s talking about, we connect the dots and propose what’s needed to move forward. Join the conversation! #KnowledgeEditing #LLMs #AI #ModelEditing
📌 If you're working on LLMs, model updates, or mechanism interpretability, you don’t want to miss this.
👉 Read the full post: yyzcowtodd.cn/rethinkedit
Key insights from our analysis:
0⃣ Current evaluation metrics and benchmarks inadequately assess knowledge updates in LRMs, highlighting the need for more comprehensive evaluation frameworks.
1⃣ Scaling challenges persist, with significant memory and computational constraints limiting the practical application of editing methods for larger or quantized local models.
🎁 Resource Release: To support the research community, we release covariance matrices for Qwen2.5-32B & QwQ-32B models for the current locate-and-edit methods.
2⃣ We outline promising research directions for developing language models that can effectively learn, adapt, and evolve their knowledge base.
Huge thanks to the brilliant collaborators who made this deep dive into #ModelEditing possible! @uclanlp @CanyuChen3 @Jiachen_Gu @dsmall2apple1 @ManlingLi_ @VioletNPeng
Yunzhi Yao retweeted
Unfortunately, due to a scheduling conflict, I won’t be able to attend #EMNLP2026 in person this year. But you can still connect with our ZJUNLP team members at the conference. Hope you all have a great time!
Title: When Should Models Change Their Minds? Contextual Belief Management in Large Language Models
ArXiv: arxiv.org/abs/2605.30219
Code: github.com/zjunlp/CBM
Location: Hall H
Time: Mon 26 Oct 4:45 p.m. CET — 6:15 p.m. CET
Title: LongDS-Bench: On the Failure of Long-Horizon Agentic Data Analysis
ArXiv: arxiv.org/abs/2605.30434
Code: github.com/zjunlp/DataMind
Location: G13–14
Time: Tue 27 Oct 9:00 a.m. CET — 10:30 a.m. CET
Title: OceanGym: Evaluating Language-Grounded Embodied Agents in Underwater Environments
ArXiv: arxiv.org/pdf/2509.26536
Code: github.com/OceanGPT/OceanGym
Location: Hall H
Time: Tue 27 Oct 9:00 a.m. — 10:30 a.m. CET
Title: Exploring Autonomous Agentic Data Engineering for Model Specialization
ArXiv: arxiv.org/pdf/2605.30407
Code: github.com/zjunlp/DataAgent
Location: Hall H
Time: Sun 25 Oct 4:00 p.m. — 5:30 p.m. CET
Title: WorldMind: Aligning Agentic World Models via Knowledgeable Experience Learning
ArXiv: arxiv.org/pdf/2601.13247
Code: github.com/zjunlp/WorldMind
Location: Hall H
Time: Sun 25 Oct 11:00 a.m. — 12:30 p.m. CET
Title: EM²Mem: Event-Centric Multimodal Memory for Large Language Models
ArXiv: arxiv.org/pdf/2609.00551
Code: github.com/zjunlp/LightMem-E…
Location: Hall H
Time: Sun 25 Oct 11:00 a.m. — 12:30 p.m. CET
Title: TokenPilot: Cache-Efficient Context Management for LLM Agents
ArXiv: arxiv.org/pdf/2606.17016
Code: github.com/zjunlp/LightRSI
Location: Hall H
Time: Sun 25 Oct 4:00 p.m. — 5:30 p.m. CET
Yunzhi Yao retweeted
Fine-tuning can improve a target skill while weakening others. Can we improve a fine-tuned LLM by undoing part of its fine-tuning?
Introducing 🚀SPACE🚀: a post-hoc method that improves average performance both within and beyond the training domain.
No extra training. No data. Just the pre- and post-SFT checkpoints for calibration with one single step of code.
1/n
Yunzhi Yao retweeted
LLMs are often trained and evaluated alone, yet we find that leaderboards built to rank soloists miss what makes a good team player.
Solo RL can make an LLM better alone, but worse together 😬
Compositional RL: GRPO rewarded by its Shapley marginal to the multi-LLM system 🧵👇
Yunzhi Yao retweeted
🚀 Introducing OpenVLHarness:
Turn any MLLM into an evidence-based visual reasoner that thinks and codes in visual space.
📈 GPT-6 Astra gains +9.9 points on average over 22 datasets.
⚡ Beats Codex and other VL harnesses by 7.7–8.5 points, costing ≤37% as much.
Yunzhi Yao retweeted
Human intelligence grows by building on experience. We acquire knowledge, refine skills, revise beliefs, and carry what we learn into future situations.
🤔 Can LLM agents do the same?
🧠 Introducing our new work, “Cortex: Consolidating Context into Parametric Memory,” in collaboration with ByteDance Seed. #Cortex #KnowledgeEditing #LLM #Memory
Our inspiration comes from biological systems consolidation: the hippocampus rapidly encodes new experiences, and interactions with the neocortex help integrate them into long-term memory.
LLM agents already benefit from documents, demonstrations, and feedback in context. But those gains do not automatically persist once the context is removed. With Cortex, we study how to consolidate those benefits into model parameters, so they remain useful beyond the original context—while preserving capabilities outside the intended change.
External memory preserves information for later retrieval. We explore a complementary route: integrating the useful effects of experience into the model itself.
📄 Paper: huggingface.co/papers/2610.2…
💻 Code (will be released soon): github.com/zjunlp/cortex
💡 What can be consolidated?
Experience can call for different kinds of change:
• Addition: acquire new knowledge or a new skill.
• Revision: update an outdated fact or behavior.
• Suppression: learn not to produce specified information or responses.
We bring these context-specified changes in knowledge, skills, and behavior together under Memory Consolidation: recover the benefits of context after removing it, while preserving capabilities outside the intended change.
⚙️ How does Cortex work?
We organize the approach around three questions: what to learn, where to write it, and how to make it usable without the original context.
What: rehearse future uses.
Cortex builds training examples around situations where the context should help. In most tasks, it generates prospective questions and answers them with the context available.
Where: write into feed-forward networks.
We explore two mechanisms: low-rank weight updates (Synaptic) and additional nonlinear feature channels (Structural). Both can be folded into standard model layers.
How: let a model with context teach a model without it.
The pre-update model itself serves as the teacher. It receives the original context; the student learns to respond without it. We align both their output distributions and internal representations, training only the newly introduced parameters.
For example, the teacher can consult documentation for an unfamiliar API while producing a solution. The student learns from that supervision so it can use the API after the documentation is removed.
🔬 What counts as successful consolidation?
Learning the new material is only part of the challenge. We also need to measure what happens to the capabilities the model already had.
Our testbed covers five task families:
• Agent experience: reuse useful strategies from prior experience.
• Novel API learning: use unfamiliar APIs without the original documentation in context.
• Persona steering: retain a specified change in how the model behaves.
• Knowledge suppression: reduce responses containing specified information.
• Factual editing: revise facts supplied through context.
We measure two outcomes together:
• Consolidation Rate (CR): how much of the performance gain provided by context is recovered after that context is removed.
• Retention Rate (RR): how much of the original performance outside the intended change is preserved.
We test single, batch, and sequential updates. Sequential evaluation asks a further question: as new experiences arrive, do earlier gains survive?
📊 What did we find?
We evaluate Cortex on 4B–120B dense and mixture-of-experts models, including Qwen, Gemma 3, and GPT-OSS.
• After 50 sequential factual updates on Qwen2.5-14B, Cortex-Structural achieves 88.9% CR and 99.8% RR, adding 0.74M parameters per update.
• On GPT-OSS-120B, Cortex-Structural recovers the full context-provided gain on ALFWorld and slightly exceeds it (101.2% CR), while achieving 95.2% RR.
• On GPT-OSS-20B, it reaches 90.7% CR in single-context API learning.
• Across three Qwen backbones, the two Cortex variants achieve approximately 80–91% CR in batch factual editing.
All results are evaluated without the original source contexts.
This is the direction we want to explore: agents that turn useful experience into capabilities they can carry into future interactions.
Yunzhi Yao retweeted
As many of you know, over the past two years I’ve shifted my research from foundation models and NLP into AI-native chip design. What I’ve found: the problems feel 10x harder than software engineering. Writing functionally correct RTL is only the beginning.
As models saturate on benchmarks like CVDP and ChipBench, I keep coming back to one question: how far can AI go toward designing a complete chip on its own?
Today, we’re thrilled to introduce Build-A-Chip-Bench, the industry’s first AI benchmark for end-to-end chip design, built by ChipAgents in collaboration with @NVIDIA. It tests whether AI can take a design from specification all the way to verified GDS.
The top model scored just 16.6%.
Today’s models can produce functionally correct RTL in some cases, but reliably meeting physical requirements remains a major challenge. We still have a long way to go, and that’s what makes this research so exciting to me.
Proud of our research team and grateful to our industry partners for helping us give the community an honest measure of where we are and how much work lies ahead.
Yunzhi Yao retweeted
Super excited to share this work from our amazing interns and team at @Google, co-led with awesome @chunliang_tw!
Meet DuoDiscover: a full-stack code optimization agent. Its secret sauce is co-discovering the optimization target AND a cheap proxy evaluator (a "yardstick") at the same time.
A quick spoiler of the results:
🏆 #1 on NVIDIA's SOL-ExecBench
🐍 2× the human experts on SWE-fficiency
🚀 1.92× on vLLM; 1.44× on FreeToken
🔥 2.68× over production TPU kernels
🖥️ Beats the best human and AI-designed CPU prefetchers
Optimizing AI infra. Accelerating science. Dive into more details below 👇
AI is cracking math problems. What's holding it back in engineering?
The cost of evaluation. Agents can generate ideas in seconds, but testing them can take hours or days.
💡 Meet DuoDiscover: where evaluation is part of what agents discover 👇🧵
Some highlights:
🏆 #1 on NVIDIA's SOL-ExecBench
🐍 2× the experts on SWE-fficiency
🚀 1.92× on vLLM; 1.44× on FreeToken
🔥 2.68× over production TPU kernels
🖥️ Beats the best human and AI-designed CPU prefetchers
Yunzhi Yao retweeted
🪶 Long-horizon Agent sessions tend to get heavier over time: context grows, repeated computation accumulates, and useful history becomes harder to manage. #RSI #LightRSI
We are exploring LightRSI, a path toward lower-cost continual self-improvement for Agents.
⭐ GitHub: github.com/zjunlp/LightRSI
📄 Paper: arxiv.org/abs/2606.17016
🚀 We start with TokenPilot:
🔹 Reduce redundant context in rollouts
🔹 Stabilize reusable prompt prefixes
🔹 Trim noisy tool outputs
🔹 Manage history by task lifecycle
🔌 Integrations: OpenClaw, Claude Code, Codex, pi, DeepSeek Harness, and OpenCode. Feature availability varies by host.
📊 Visual snapshots make context optimization easier to inspect. Context Cleaner lets users review and explicitly select tasks to clean on supported hosts.
TokenPilot makes the execution side lighter first. Next, we are developing lighter-weight ways to use Agent trajectories for future self-improvement.
🎉 The TokenPilot paper has been accepted to EMNLP 2026!
Yunzhi Yao retweeted
🔥 LightMem-Ego just got a major upgrade — new system, faster interaction, and a new real-time demo!
What if your AI glasses could not only see and hear, but also remember your daily life?
LightMem-Ego continuously turns first-person visual/audio streams into current, short-term, and long-term memory.
⚡ Interactive memory QA
• 0.42s P50 TTFT on Rokid Glass
• 0.53s P50 TTFT on phone
• 4.7s P50 TTFT for long-term memory QA
🛠️ We’ve also substantially updated the open-source system:
streaming answers, multi-session support, online memory editing, improved Glass + Web integration, and easier self-hosted deployment with Docker.
Now LightMem-Ego can continuously help you:
🔍 Find misplaced objects
💬 Recall conversations
📅 Summarize past activities
🔁 Discover daily routines
👓 Get hands-free wearable assistance
Ask:
“Where did I leave my badge?”
“What did the doctor tell me?”
“What do I usually do after arriving at the office?”
See. Hear. Remember. Recall.
🎥 Check out the new demo below!
📄 Paper: arxiv.org/abs/2607.11487
💻 GitHub: github.com/zjunlp/LightMem-E…
Yunzhi Yao retweeted
Excited to introduce Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence.
Our goal: Understand AI, Improve AI.
Existing AI Scientists focus on training recipes for models or specific solutions to scientific tasks.
Our Mechanist studies how AI itself works internally (how do AI models acquire knowledge, reason, and act?), through an autonomous process: Hypothesis, Experiment, Verification, Iteration.
Two resources support it:
🕸️ Scientific Knowledge Graph: 13,000 interpretability papers + 43M papers across 26 disciplines
🧰 Method library: 32 methods for mechanism analysis and causal intervention
What you can do with Mechanist:
🔁 Reproduce interpretability paper: ~10%+ higher reliability than Claude Code
💡 Generate new research hypotheses: Draw inspiration from 26 disciplines
🔎 Discover unknown behaviors & risks: Mechanist find that unsafe traits can be transmitted through apparently safe training data in a multimodal scientific lab setting, increasing the unsafe response rate by 28.3%.
🧠 Build mechanism theories: Mechanist identifies separable personal-belief and attributed-belief heads governing how models use learned knowledge in complex contexts. Only ~0.05% of parameters carry these belief heads.
🎯 Intervene on internal mechanisms:Mechanist adaptively modulates belief heads at inference to resolve conflicting information and reason about others’ beliefs, improving response accuracy by up to 15.3% without retraining the base model.
🧬 Use mechanisms for scientific design:Mechanist shifts from generate-and-rerank to mechanism-guided generation, steering α-helix-associated features in Evo2 to increase α-helical content by 12.8%.
📄 Paper: arxiv.org/abs/2608.12036
💻 Code:github.com/zjunlp/Mechanist
🌐 Project Website:mechanist.openkg.cn/
Yunzhi Yao retweeted
Introducing EM²Mem: Event-Centric Multimodal Memory for Large Language Models 🎬🧠
Multimodal memory is becoming essential for LLMs to reason over long videos and extended experiences.
But today’s multimodal memory is still largely fragment-centric.
Relevant evidence may be scattered across:
- visual frames and captions,
- transcripts and dialogue,
- temporally distant events,
- structured facts and relations,
- or long-term patterns spanning the entire video.
Existing systems often retrieve these pieces independently — and leave the LLM to reconstruct the missing cross-modal and temporal alignment at inference time.
We call this the retrieve-then-align bottleneck.
EM²Mem changes this.
We introduce EM²Mem, an Event-Centric Multimodal Memory framework that organizes heterogeneous evidence around shared event anchors before retrieval — turning fragmented multimodal observations into coherent, grounded, and query-ready memory.
⚡ What EM²Mem enables:
🧩 Event-Centric Multimodal Memory
Each event becomes a unified memory cell that binds visual captions, transcripts, keyframes, structured metadata, temporal context, and provenance — instead of storing them as isolated fragments.
🔗 Align-then-Retrieve
Rather than performing cross-modal fusion after retrieval, EM²Mem aligns heterogeneous evidence during memory construction and directly retrieves grounded event-level evidence at inference time.
⏱️ Multi-Scale Temporal Context
Event memories are connected to context across multiple temporal scales, enabling reasoning from local moments to long-range activities and behaviors.
🕸️ Event-Linked Memory Graph
An episodic graph captures cross-event entities, objects, places, topics, and temporal transitions, while a semantic graph stores long-term facts, habits, preferences, and stable relations — all grounded back to supporting events.
🎯 Compact & Grounded Evidence Readout
EM²Mem retrieves relevant event cells, expands graph-linked evidence, and compiles a compact query-specific evidence view for answer generation.
📈 Results
Across EgoLifeQA, Ego-R1 Bench, and Video-MME (L), EM²Mem improves average accuracy over the strongest memory baseline by +2.0, +2.4, and +3.7 points, while:
⚡ reducing per-query latency by 4.67×
📉 reducing total inference tokens by 63.66%
🎯 improving strict event-level Top-5 evidence recall by +7.0 points
Our analysis further shows that construction-time multimodal alignment consistently outperforms retrieval-time fusion.
The key is not only what memory stores, but how evidence is organized before retrieval.
👓 We also bring multimodal memory into real-world personal AI with LightMem-Ego — our streaming memory system for smartphones and AI glasses, supporting everyday-life recall and personalized assistance.
📖 Paper:
arxiv.org/abs/2609.00551v1
⌨️ Code:
github.com/zjunlp/LightMem
👓 LightMem-Ego Demo:
github.com/zjunlp/LightMem-E…
We believe multimodal memory should not be a collection of searchable fragments.
It should be organized around the events that give those fragments meaning.
#EM2Mem #MultimodalMemory #LLM #Memory #Agent #Multimodal #LongVideo #NLP
Yunzhi Yao retweeted
Our latest paper "Towards principled knowledge editing methods for large language model reasoning" is now published in Nature Machine Intelligence. 🧠 @NatMachIntell
📄 nature.com/articles/s42256-0…
As AI moves toward reasoning, agents, parameter memory, and recursive self-improvement (RSI), a fundamental question is becoming increasingly important:
If an AI can improve itself, how should it revise what it already “knows”?
Our Perspective argues that knowledge in modern reasoning models is not a collection of isolated facts stored in independent parameter slots.
It is an entangled, belief-dependent, and context-sensitive system.
This calls for a shift:
From isolated facts → entangled knowledge
From updating knowledge → revising beliefs
From parameter updates → evolving parameter memory
We highlight three directions for next-generation knowledge editing:
→ Knowledge entanglement: update interconnected knowledge circuits (structures) rather than isolated facts.
→ Belief-aware editing: understand model confidence and intervene differently depending on what the model already believes.
→ Reasoning-augmented revision: use reasoning, evidence, and contextual scaffolding to make knowledge updates coherent.
The deeper question is not simply how to give AI more memory.
It is:
Can AI understand what it believes, why it believes it, and how to revise those beliefs coherently?
This may become a critical foundation for continual learning, controllable AI, parameter memory, and recursive self-improvement.
Would love to hear thoughts from the communities working on LLM reasoning, memory, model editing, continual learning, mechanistic interpretability, agents, and RSI.
#NatureMachineIntelligence #NatureMI #NLP #LLM #KnowledgeEditing #MechanisticInterpretability #KnowledgeCircuits #Circuits
🧠 Unlock the black box of LLMs at #KDD2026!
Join our Tutorial on Interpretability in the Era of Large Language Models.
📅 Aug 9 | 1:00–5:00 PM KST
📍 Room 401B, ICC Jeju, Korea
Come and chat with us! 🔍✨
#LLM #Interpretability #MachineLearning #AI #KDD2026
Just landed in Seoul for #ICML2026! 🛬🇰🇷
If you're into interpretability / explainable AI, I'd love to chat!Feel free to DM me or grab me at the venue!
1
100
Yunzhi Yao retweeted
Two students take the same exam. Both score 100 — one solved it himself, the other Googled every answer.
A semester later, the gap is huge.
That's the problem with today's AI agents. I write a detailed blog to share my recent thoughts on this, mainly based on Theory of Agents. I promise this is definitely worth 30 minutes of your time.
Blog: notion.so/Second-Half-of-Age…
Project: hrwise-nlp.github.io/assets/…
Yunzhi Yao retweeted
I have evaluated OPUS on models exceeding 1 trillion parameters, yielding encouraging results. However, due to practical constraints, we are unable to include these findings in the current version of the manuscript.
Additionally, OPUS has shown potential for identifying toxic samples. While toxicity detection falls outside the scope of this specific paper, it remains an important focus of my broader research into data quality. As LLM researchers, we "debug" and analysis data every day lol.
Finally, I have added a brief section exploring how offline OPUS can be used to identify toxic (low-utility samples within a training batch). The manuscript will briefly touch upon this application.
When it scales to 1T params, I found that the utility estimation provides insights that extend beyond standard heuristic data quality metrics.
Reintroducing our ICML Oral paper: OPUS: Towards Efficient and Principled Data Selection in LLM Pre-training in Every Iteration.
When I first started working on pre-training, I had a rude awakening: most algorithms that look elegant in academic papers—no matter what awards they won—simply do not work at trillion-token scale with 1T-parameter models. Scale is the ultimate referee; all fancy guarantees collapse back to reality.
So I forced myself back to first principles. What does "good data" actually mean? How do we evaluate pre-training when next-token prediction cannot be judged by a simple benchmark? Can a validation proxy like perplexity reliably map to downstream scores? It sounds like ML 101, but at massive scale, this is surprisingly unresolved.
During my time at the Qwen team (starting from what now feels like the "ancient" Qwen2.5 era before Qwen3), I spent enormous effort on data mixture engineering. We designed stages, tuned domain ratios—some empirically driven, some numerically justified. But the uncomfortable truth is: 90% of data engineering is still human heuristic. It is craft, not science. I fully accept that LLM research is experimental science, but I constantly wanted to pursue something less dependent on human taste—in some sense, the bitter lesson for data selection.
So in OPUS, we honestly tackle four questions that had been bothering me for a while:
1️⃣ How do we define a good validation set that can actually guide data selection?
2️⃣ How do we define data quality scientifically—not by human preference, but by what the model itself structurally needs?
3️⃣ How do we select data based on the model's live weight updates at each iteration, rather than freezing human-designed stages? I want this to be online and adapt in real time.
4️⃣ How do we make all of this efficient and scalable?
The core philosophy: stop treating pre-training data as a static mixture designed by humans, and start treating it as a dynamic optimization problem where the model itself tells itself what to learn next.
Check our paper: arxiv.org/abs/2602.05400
Code for GPT series: github.com/gszfwsb/OPUS
#Qwen3 #DataCentricAI #OPUS #ICML2026 #LLM #Pretraining #DataSelection #MachineLearning
Yunzhi Yao retweeted
Introducing MemTrace: Making LLM Memory Systems Finally Debuggable 🔍🧠
Memory is becoming a core component of AI agents. But today’s memory systems are still a "black box".
When a memory-augmented agent fails, the real error may have happened:
- dozens of turns earlier,
- inside a retrieval step,
- during memory consolidation,
- or from a corrupted update that silently propagates over time.
Existing logs cannot recover these long-range causal chains.
MemTrace changes this.
We introduce the automated tracing framework for LLM memory systems — turning opaque memory pipelines into transparent execution graphs that can be inspected, explored, and diagnosed step by step.
⚡ What MemTrace enables:
🧩 Plug-and-Play Instrumentation
Seamlessly integrates with diverse memory systems (RAG, Mem0, EverMemOS, etc.) without modifying the original architecture.
🧠 Transparent Memory Execution
Transforms opaque memory pipelines into structured execution graphs, making information flow, retrieval, updates, and propagation fully traceable.
🔍 Error Attribution
Pinpoint the exact operation responsible for failure across long-horizon memory execution.
🚨 Benchmark Auditing
While building MemTraceBench, we found that failure attribution in memory systems remains highly challenging — MemTrace still has substantial room for improvement.
We also discovered annotation errors in existing memory benchmarks, revealing broader reliability issues in current memory-agent evaluation.
🔄 Towards Self-Evolving Agents
MemTrace is not only a debugging tool.
Its fine-grained attribution signals can directly drive closed-loop optimization, enabling agents to automatically repair faulty behaviors and continuously evolve from failures.
📈 Using MemTrace-guided optimization, we improve downstream task performance by up to 7.62%.
📖 Paper: arxiv.org/abs/2605.28732
⌨️ Code (coming soon):
• MemTrace: github.com/zjunlp/MemTrace
• smartcomment: github.com/zjunlp/smartcomme…
• MemBase: github.com/zjunlp/MemBase
We believe memory systems need the same thing software engineering once needed:
not bigger models — but observability, tracing, and debugging infrastructure. #MemTrace #LLM #NLP #Agent #Tracing #Debugging
Yunzhi Yao retweeted
When should LLMs update, preserve, or ignore information?
Contextual Belief Management is what long-horizon reasoning was missing. We introduce BeliefTrack—and show that optimizing belief states cuts reasoning failures by over 70%.
Yunzhi Yao retweeted
🚀 Excited to share our latest progress on scientific idea evaluation!
As LLMs and AI Scientists generate research ideas at scale, a key bottleneck emerges: how do we know which ideas are truly novel, valid, feasible, and impactful?
Our ICML 2026 paper, InnoEval, frames idea evaluation as a knowledge-grounded, multi-perspective reasoning problem, going beyond simple LLM-as-a-Judge with:
🔎 heterogeneous deep knowledge search
👥 an innovation review board with diverse academic personas
📊 multi-dimensional evaluation across clarity, novelty, feasibility, validity, and significance
We further extend this direction with SciAtlas: a large-scale scientific knowledge graph covering 26 disciplines, 43M+ papers, 157M entities, and 3B triples.
🧭 More importantly, SciAtlas is packaged into Agent Skills, enabling research agents to directly call structured scientific knowledge for:
📚 paper search & literature review
🧩 idea grounding & similarity analysis
⚖️ evidence-grounded idea evaluation
💡 idea generation from retrieved papers
📈 research trend synthesis
👤 researcher profile analysis
Our goal: AI systems that do not only generate ideas, but can also locate, challenge, evaluate, and improve them with grounded scientific evidence. #SciAtlas #KG #AIScientist #Agent #AI4Research
📄 InnoEval: arxiv.org/abs/2602.14367
📄 SciAtlas: arxiv.org/abs/2605.22878
🔗 Repo: github.com/zjunlp/SciAtlas







































