Megagon Labs (@MegagonLabs) header
Megagon Labs (@MegagonLabs) profile picture

Megagon Labs advances state-of-the-art research in AI and builds technologies that impact the world through online services. #NLP #AI #ML #NLP4HR #LLM #RAG

Mountain View, CA
Joined June 2018
๐—ง๐—ฎ๐—ฏ๐—น๐—ฒ ๐—ค๐—” ๐—ถ๐˜€ ๐—ด๐—ผ๐—ผ๐—ฑ ๐—ฎ๐˜ ๐—ณ๐—ถ๐—ป๐—ฑ๐—ถ๐—ป๐—ด ๐—ณ๐—ฎ๐—ฐ๐˜๐˜€. ๐—œ๐˜ ๐—ถ๐˜€ ๐—ณ๐—ฎ๐—ฟ ๐—น๐—ฒ๐˜€๐˜€ ๐—ฟ๐—ฒ๐—น๐—ถ๐—ฎ๐—ฏ๐—น๐—ฒ ๐—ฎ๐˜ ๐—ฎ๐—ป๐˜€๐˜„๐—ฒ๐—ฟ๐—ถ๐—ป๐—ด, โ€œ๐—›๐—ผ๐˜„ ๐—น๐—ถ๐—ธ๐—ฒ๐—น๐˜† ๐—ถ๐˜€ ๐˜๐—ต๐—ถ๐˜€ ๐—ผ๐˜‚๐˜๐—ฐ๐—ผ๐—บ๐—ฒ, ๐—ด๐—ถ๐˜ƒ๐—ฒ๐—ป ๐˜๐—ต๐—ฒ ๐—ฒ๐˜ƒ๐—ถ๐—ฑ๐—ฒ๐—ป๐—ฐ๐—ฒ?โ€ ๐— ๐—ผ๐—ฟ๐—ฒ ๐—ฐ๐—ผ๐—ป๐˜๐—ฒ๐˜…๐˜, ๐—ฟ๐—ฒ๐˜๐—ฟ๐—ถ๐—ฒ๐˜ƒ๐—ฎ๐—น, ๐—ผ๐—ฟ ๐—ฎ ๐˜€๐˜๐—ฟ๐—ผ๐—ป๐—ด๐—ฒ๐—ฟ ๐—ฟ๐—ฒ๐—ฎ๐˜€๐—ผ๐—ป๐—ถ๐—ป๐—ด ๐—บ๐—ผ๐—ฑ๐—ฒ๐—น ๐—ฑ๐—ผ๐—ฒ๐˜€ ๐—ป๐—ผ๐˜ ๐—ฎ๐˜‚๐˜๐—ผ๐—บ๐—ฎ๐˜๐—ถ๐—ฐ๐—ฎ๐—น๐—น๐˜† ๐—ฟ๐—ฒ๐—ฐ๐—ผ๐˜ƒ๐—ฒ๐—ฟ ๐˜๐—ต๐—ฒ ๐—ฐ๐—ผ๐—ป๐—ฑ๐—ถ๐˜๐—ถ๐—ผ๐—ป๐—ฎ๐—น ๐—ฑ๐—ถ๐˜€๐˜๐—ฟ๐—ถ๐—ฏ๐˜‚๐˜๐—ถ๐—ผ๐—ป๐˜€ ๐—ต๐—ถ๐—ฑ๐—ฑ๐—ฒ๐—ป ๐—ฎ๐—ฐ๐—ฟ๐—ผ๐˜€๐˜€ ๐—ฎ ๐˜๐—ฎ๐—ฏ๐—น๐—ฒ. ๐Ÿ’ฅ We introduce ๐—Ÿ๐—จ๐—–๐—”๐—ฅ๐—œ๐—ข, a benchmark with 6,280 questions over 10 real-world tables, spanning five probabilistic reasoning patterns and four linguistic variations. We also propose Auto-BN: an automated pipeline that induces a Bayesian Network from raw data, translates natural language into an executable probabilistic query, and performs exact inference. With GPT-4o, Auto-BN reaches 46.7% accuracy within ยฑ0.05 and 0.107 MAE, outperforming NL2SQL, TableRAG, raw-table prompting, reasoning models, and premise-retrieval baselines. Its robust translation and fallback mechanisms reduce parsing error rates from as high as 20.8% to near zero across the tested backbones. ๐Ÿง  Use language models to understand the question and a grounded probabilistic layer to compute the answer. ๐—Ÿ๐—จ๐—–๐—”๐—ฅ๐—œ๐—ข provides a testbed for studying that interface at realistic table scale. ๐Ÿ“„ Read the research ๐Ÿ‘‰: megagon.ai/publications/lucaโ€ฆ #AACL #ACL #LLMs #AI #PhD #Research @estevamhruschka
1
57
AI agents can build interactive tables, graphs, and canvases. But users still have to respond through chat. ABRA turns those outputs into shared surfaces where humans and agents can act and coordinate. In our research, we introduce Agent-Bound UI and its reference prototype, ABRA, a shared interaction medium that enables humans and agents to communicate beyond chat through interactive canvases. The key idea is to separate reusable interaction structures from task-specific meanings. The same graph view, for example, can represent a social network, a sports tournament, or a directory structure. But โ€œadd an edgeโ€ means something different in each context: connecting people, linking matches, or defining a directory relationship. Agent-Bound UI lets agents reuse the same interaction structure while binding its meaning to the task and sharing that meaning with users and other agents. Our experiments show that current LLM agents can: โ€ข Create task-specific canvases โ€ข Resolve interactions on canvases they own โ€ข Invoke published actions on other agentsโ€™ canvases Multi-view completeness and specification following remain challenges. Agent interfaces can be more than outputs. They can become shared surfaces where humans and agents show, act, and coordinate. Read the research: megagon.ai/publications/abraโ€ฆ ๐Ÿ’ฅ Join us for the demo presentation at #EMNLP2026 at 9:00 AM on October 27th, presented by Hannah Kim. #LLMs #Agents #HCI #NLP #EMNLP
1
3
252
๐Ÿ‘‰ LLMs can improve Bayesian network learning without owning the graph. ABSOL uses subsampled GES+BDeu plus four bounded LLM interventions. Across five benchmarks spanning 27 to 1,041 nodes, it was the only method to return a viable graph on all five. Read: megagon.ai/publications/absoโ€ฆ Join @estevamhruschka at #EMNLP2026 for a poster presentation on ABSOL ๐Ÿ‘‡
1
141
๐Ÿ’ก๐Ÿ—„๏ธ Search systems need to understand increasingly complex requests. But improving that understanding can mean rebuilding millions of stored document representations. What if the query side could get smarter while the existing index stayed intact? In our new paper, we introduce Efficient Retrieval Adapter (ERA), a query-side linear adapter trained in two stages. First, ERA aligns a strong query embedder with a lightweight document embedder using the same unlabeled corpus documents in both spaces. Then it adapts that aligned representation with a limited set of relevance labels. Across 126 MAIR retrieval tasks in six domains, ERA with a strong query embedder improves average nDCG@10 by more than 12 points for BGE-M3 and OpenAI-small document indexes at the 20% label ratio. The alignment stage is the key: it produces large gains before labeled adaptation and keeps ERA effective in low-label settings. Its sampled documents represent only 1.28% of the MAIR collection and 5.20% of corpus tokens. The implication is a more modular retrieval stack: improve query understanding without updating backbone embedders or rebuilding the corpus index. For enterprise search and RAG AI systems, this creates a practical research direction for evolving query intelligence while preserving index infrastructure. Read the research: megagon.ai/publications/asymโ€ฆ @SAYg_7 @estevamhruschka @PPezeshkpour #EMNLP2026 #AI #LLMs #NLP
2
166
How confident can we be that an LLM agent actually completed its task? Long trajectories, multiple sources of uncertainty, and limited model access make this hard. Our new work, CRGs, uses evidence-grounded reasoning for better-calibrated, interpretable confidence estimates. ๐Ÿ‘‡
๐Ÿ“ข New preprint from @MegagonLabs ๐Ÿงต Confidence estimation for LLM agents is hard: evidence of success spans tool calls, observations, reasoning, and memory. When model internals are inaccessible, reruns are expensive, and training data is scarce, how can we reliably assess whether an agent succeeded and how much to trust its output? Meet ๐—–๐—ผ๐—ป๐—ณ๐—ถ๐—ฑ๐—ฒ๐—ป๐—ฐ๐—ฒ ๐—ฅ๐—ฒ๐—ฎ๐˜€๐—ผ๐—ป๐—ถ๐—ป๐—ด ๐—š๐—ฟ๐—ฎ๐—ฝ๐—ต๐˜€ (๐—–๐—ฅ๐—š): a training-free, black-box method that turns a single execution trajectory into a structured, evidence-grounded argument for success and an auditable confidence estimate. Paper: arxiv.org/abs/2610.07948 Code: github.com/megagonlabs/crg_cโ€ฆ
1
2
8
303
Megagon Labs retweeted
๐Ÿ“ข New preprint from @MegagonLabs ๐Ÿงต Confidence estimation for LLM agents is hard: evidence of success spans tool calls, observations, reasoning, and memory. When model internals are inaccessible, reruns are expensive, and training data is scarce, how can we reliably assess whether an agent succeeded and how much to trust its output? Meet ๐—–๐—ผ๐—ป๐—ณ๐—ถ๐—ฑ๐—ฒ๐—ป๐—ฐ๐—ฒ ๐—ฅ๐—ฒ๐—ฎ๐˜€๐—ผ๐—ป๐—ถ๐—ป๐—ด ๐—š๐—ฟ๐—ฎ๐—ฝ๐—ต๐˜€ (๐—–๐—ฅ๐—š): a training-free, black-box method that turns a single execution trajectory into a structured, evidence-grounded argument for success and an auditable confidence estimate. Paper: arxiv.org/abs/2610.07948 Code: github.com/megagonlabs/crg_cโ€ฆ
1
3
1
10
651
Editing a multi-agent plan is not the same as steering it. We introduce ๐—”๐— ๐—•๐—œ๐—ฃ๐—ข๐— : ๐—ฎ ๐—ฝ๐—ฟ๐—ผ๐˜๐—ผ๐˜๐˜†๐—ฝ๐—ฒ ๐—ณ๐—ผ๐—ฟ ๐—ต๐˜‚๐—บ๐—ฎ๐—ป-๐—Ÿ๐—Ÿ๐—  ๐—ฐ๐—ผ-๐—ฝ๐—น๐—ฎ๐—ป๐—ป๐—ถ๐—ป๐—ด ๐˜๐—ต๐—ฎ๐˜ ๐—ฒ๐—ป๐—ฎ๐—ฏ๐—น๐—ฒ๐˜€ ๐˜‚๐˜€๐—ฒ๐—ฟ๐˜€ ๐˜๐—ผ ๐—ฒ๐—ฑ๐—ถ๐˜ ๐—ฎ๐—ด๐—ฒ๐—ป๐˜ ๐—ฝ๐—น๐—ฎ๐—ป๐˜€ ๐˜ƒ๐—ถ๐—ฎ ๐—ฐ๐—ต๐—ฎ๐˜, ๐˜๐—ฎ๐—ฟ๐—ด๐—ฒ๐˜๐—ฒ๐—ฑ ๐—ณ๐—ฒ๐—ฒ๐—ฑ๐—ฏ๐—ฎ๐—ฐ๐—ธ, ๐—ด๐—ฟ๐—ฎ๐—ฝ๐—ต ๐—บ๐—ฎ๐—ป๐—ถ๐—ฝ๐˜‚๐—น๐—ฎ๐˜๐—ถ๐—ผ๐—ป, ๐—ฎ๐—ป๐—ฑ ๐—บ๐—ฒ๐—ฟ๐—ด๐—ฒ, ๐˜€๐—ฝ๐—น๐—ถ๐˜, ๐—ฎ๐—ป๐—ฑ ๐—ฟ๐—ฒ๐—ฝ๐—น๐—ฎ๐—ป ๐—ผ๐—ฝ๐—ฒ๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป๐˜€. The key finding: More controls reduce some effort, but they also create new verification work. Users built hybrid workflows rather than choosing one interaction type. Global feedback was most robust for complex revisions. Targeted feedback preserved unchanged parts of the plan, but could introduce structural errors. Edit sequences were easier to audit, but less reliable for complex changes. For agentic AI, the design challenge is to โ€œhelp users know whether the edited plan still works.โ€ ๐Ÿ“„ Read the article: megagon.ai/human-llm-co-planโ€ฆ @estevamhruschka #AI #Agents #LLMs #HumanCenteredAI #CAIS2026
2
3
9
728
Weโ€™re hiring a Fall/Winter PhD Research Intern at Megagon Labs. Weโ€™re looking for candidates with research or technical experience in any one or more of the following areas: โ€ข Retrieval-Augmented Generation (RAG) โ€ข Function Calling โ€ข Document Understanding / PDF Parsing You do not need experience across all three areas. Strong experience in any one of these topics is relevant. Interested in joining us this fall? Learn more and apply โ†’ megagon.ai/careers/internshiโ€ฆ Applicants must have or be able to obtain appropriate U.S. work authorization for the internship period. #AI #NLP #LLMs #internships #phdposition
1
2
1
3
4,887
Correct answers are no longer enough. As LLMs become part of systems that use tools, memory, and evolving knowledge, we also need to understand how those components change model behavior. That question shaped much of the conversation at ACL 2026. Read the article! megagon.ai/acl-2026-highlighโ€ฆ #AI #LLMs #CompoundAI
178
Your AI agent can cite a source. That doesn't mean the source says what the row claims. Here's the failure mode we kept seeing: an LLM curator builds a plausible structured table from parametric memory; it "just knows" what should go in the cell, then bolts on a page-level citation afterward to make it look sourced. The citation is real, and the row is not grounded in it. It's structured output with decorative provenance. We built Stage-Audit to close that gap in the context of what we call Seed2Frontier discovery: starting from a single Wikipedia seed page and autonomously finding the additional pages needed to complete a structured table that answers a natural-language query. megagon.ai/data-provenance-sโ€ฆ #AIResearch #AgenticAI #CompoundAI #LLM #DataManagement #AIAgents
1
229
Most LLM extraction systems forget what they learned the moment the document ends. At ACL 2026, we introduce DySECT: a self-evolving extraction system that builds an explicit KB as it extracts. This is a glance at our loop: extract triples -> update KB -> organize and score knowledge -> feed it back into extraction On DocRED, KB-guided extraction improved recall across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5. The key result: 5-8% recall gains on the first guided iteration, without fine-tuning or synthetic data generation. Adaptation can happen through inspectable memory, not only opaque model updates. ๐Ÿ‘‰Read the article: megagon.ai/self-evolving-infโ€ฆ #ACL2026 #NLProc #LLM #InformationExtraction #EnterpriseAI #KnowledgeGraphs #AIResearch #MegagonLabs #ContinuousLearning
1
1
149
As AI agents become more capable, understanding how they make decisions is becoming just as important as the answers they produce. In our latest blog, we explore why agent observability is emerging as a critical capability for enterprise AI. We also introduce L.A.K.E. (Logic Agent for Knowledge Extraction), our agentic data planning framework that makes planning visible through executable workflows and interactive DAGs, helping developers inspect, evaluate, and improve how AI agents reason across complex data sources. Transparency isn't just about explainability; it enables debugging, optimization, and building more reliable AI systems. Read our blog post: megagon.ai/agent-observabiliโ€ฆ #AI #EnterpriseAI #AgenticAI #MachineLearning #Transparency #MegagonLabs
1
207
Most ๐—Ÿ๐—Ÿ๐—  ๐—ฒ๐˜…๐˜๐—ฟ๐—ฎ๐—ฐ๐˜๐—ถ๐—ผ๐—ป ๐˜€๐˜†๐˜€๐˜๐—ฒ๐—บ๐˜€ treat each document in isolation, failing to retain and reuse knowledge acquired from previous extractions. That is a problem in domains where terminology changes, taxonomies evolve, and rare concepts matter. At ACL 2026 in San Diego, we introduce ๐——๐˜†๐—ฆ๐—˜๐—–๐—ง: ๐—ฎ ๐——๐˜†๐—ป๐—ฎ๐—บ๐—ถ๐—ฐ ๐—ฆ๐—ฒ๐—น๐—ณ-๐—˜๐˜ƒ๐—ผ๐—น๐˜ƒ๐—ถ๐—ป๐—ด ๐—˜๐˜…๐˜๐—ฟ๐—ฎ๐—ฐ๐˜๐—ถ๐—ผ๐—ป ๐—ฎ๐—ป๐—ฑ ๐—–๐˜‚๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป ๐—ง๐—ผ๐—ผ๐—น๐—ธ๐—ถ๐˜. Extraction should create reusable knowledge. So, DySECT uses an LLM to extract structured triples, stores them in a growing knowledge base, and assigns confidence based on source reliability, repeated evidence, and conflicts. This accumulated knowledge is now a resource that should improve subsequent extraction rounds. The KB enriches previous extractions through hierarchical concept abstractions, confidence calibration, mutual exclusion constraints, and automated property validation. This enriched knowledge is then fed back into the extractor via prompt augmentation, encouraging broader coverage and discovery of new information in later iterations. On DocRED, KB-guided extraction improved recall across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5. The first KB-guided iteration ๐—ถ๐—บ๐—ฝ๐—ฟ๐—ผ๐˜ƒ๐—ฒ๐—ฑ ๐—ฟ๐—ฒ๐—ฐ๐—ฎ๐—น๐—น ๐—ฏ๐˜† ๐Ÿฑ-๐Ÿด% ๐˜„๐—ถ๐˜๐—ต๐—ผ๐˜‚๐˜ ๐—ณ๐—ถ๐—ป๐—ฒ-๐˜๐˜‚๐—ป๐—ถ๐—ป๐—ด ๐—ผ๐—ฟ ๐˜€๐˜†๐—ป๐˜๐—ต๐—ฒ๐˜๐—ถ๐—ฐ ๐—ฑ๐—ฎ๐˜๐—ฎ ๐—ด๐—ฒ๐—ป๐—ฒ๐—ฟ๐—ฎ๐˜๐—ถ๐—ผ๐—ป. Adaptive extraction does not have to mean opaque model updates. DySECT keeps accumulated knowledge explicit, inspectable, and editable. For enterprise AI systems, that matters. Knowledge can evolve, but oversight and transparency stay in the loop. ๐Ÿ“„ Read the research: megagon.ai/publications/dynaโ€ฆ Work by @moinnas, Hannah Kim, @estevamhruschka Hruschka #ACL2026 #continuouslearning #AI #LLMs
6
452
As LLM systems become more agentic, improving individual outputs may not be enough. In a recent Megagon Labs Seminar Series talk, Jay-Yoon Lee (Seoul National University / UIUC) explored the idea that correctness and reliability often emerge at the set level. Modern LLM systems increasingly operate over collections of retrieved documents, generated statements, and intermediate reasoning steps, yet most frameworks still treat these elements independently. Interactions such as contradictions, redundancy, and joint relevance can shape outcomes just as much as the information itself. The talk introduced a unified perspective centered on set-level verification, retrieval, and editing as important directions for building more reliable and controllable LLM systems. Thank you to Jay-Yoon for sharing his insights. #LLM #AgenticAI #AIResearch
2
139
The way AI agents plan over enterprise data can dramatically change both latency and reliability. This process of data planning, how an agent sequences operations like database queries, document retrievals, and API calls to resolve complex prompts, is foundational to enterprise AI performance. In our latest research, we found that different planning regimes produce very different trade-offs: - Single-shot tree planning achieved the strongest overall performance and lowest latency. - Iterative planning provided stronger self-correction at a significantly higher execution cost. We also observed that the planning structure itself, including generation direction and refinement strategy, can substantially impact agent behavior. These findings highlight a broader challenge in agentic AI: you cannot optimize or trust what you cannot inspect. As agents increasingly orchestrate tools across databases, documents, and external systems, understanding how plans are generated and executed becomes critical for debugging, evaluation, and deployment. To address this, we introduce L.A.K.E. (Logic Agent for Knowledge Extraction), an agentic data planning framework designed to make planning observable. L.A.K.E. exposes execution paths as interactive DAGs with step-level provenance, enabling developers to inspect workflows, compare planning strategies, and understand why agents succeed or fail. This work reflects a broader direction we are exploring at Megagon Labs: building agentic systems that are not only capable of reasoning over complex enterprise data, but also transparent, debuggable, and efficient to operate. #Agentic #NLProc #LLMs megagon.ai/publications/l-a-โ€ฆ
1
117
Can LLMs automate prompt engineering? RPT says: Apparently yesโ€”by turning prompt tuning into a function-calling workflow: evaluate, diagnose recurring failures, use memory, and revise. The result: stronger prompts on reasoning tasks and better-calibrated confidence.
๐Ÿ“ข New Preprint Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts? Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs. Paper: arxiv.org/abs/2605.21781 @MegagonLabs 1/n
1
1
7
1,316
Megagon Labs retweeted
๐Ÿ“ข New Preprint Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts? Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs. Paper: arxiv.org/abs/2605.21781 @MegagonLabs 1/n
2
10
2
56
6,400
๐—ช๐—ฒ ๐—ณ๐—ผ๐˜‚๐—ป๐—ฑ ๐˜๐—ต๐—ฎ๐˜ ๐—Ÿ๐—Ÿ๐—  ๐—ฎ๐—ด๐—ฒ๐—ป๐˜๐˜€ ๐—ฐ๐—ฎ๐—ป ๐—บ๐—ฎ๐˜๐—ฐ๐—ต ๐˜๐—ต๐—ฒ ๐—ฎ๐—ฐ๐—ฐ๐˜‚๐—ฟ๐—ฎ๐—ฐ๐˜† ๐—ผ๐—ณ ๐—ฅ๐—ฒ๐—”๐—ฐ๐˜-๐˜€๐˜๐˜†๐—น๐—ฒ ๐—ฎ๐—ด๐—ฒ๐—ป๐˜๐˜€ ๐˜„๐—ต๐—ถ๐—น๐—ฒ ๐˜‚๐˜€๐—ถ๐—ป๐—ด ๐Ÿฎโ€“๐Ÿฏ๐˜… ๐—ณ๐—ฒ๐˜„๐—ฒ๐—ฟ ๐˜๐—ผ๐—ธ๐—ฒ๐—ป๐˜€. In our latest research, we revisit a core assumption in agentic AI: that agents must interleave reasoning and execution at every step to remain adaptable. We compare single-step horizon planning against full-horizon planning for data-centric tasks such as Knowledge Base QA and Multi-hop QA. Our experiments show that full-horizon planning with lazy replanning achieves accuracy parity with step-wise approaches across varying task depths, branching complexity, and tool robustness levels, while substantially reducing token usage. In many cases, eager execution monitoring provided little benefit relative to its computational overhead. These findings suggest that planning horizon is not merely an implementation choice, but a fundamental systems design decision for LLM agents. For well-defined data-centric tasks, generating complete plans upfront and replanning only when necessary may offer a more efficient default for tool-using AI systems. This work will be presented at CAIS 2026 ๐Ÿ“„ Read the research: megagon.ai/publications/rethโ€ฆ #AI #AgenticAI #LLMs #CAIS
123