๐ง๐ฎ๐ฏ๐น๐ฒ ๐ค๐ ๐ถ๐ ๐ด๐ผ๐ผ๐ฑ ๐ฎ๐ ๐ณ๐ถ๐ป๐ฑ๐ถ๐ป๐ด ๐ณ๐ฎ๐ฐ๐๐. ๐๐ ๐ถ๐ ๐ณ๐ฎ๐ฟ ๐น๐ฒ๐๐ ๐ฟ๐ฒ๐น๐ถ๐ฎ๐ฏ๐น๐ฒ ๐ฎ๐ ๐ฎ๐ป๐๐๐ฒ๐ฟ๐ถ๐ป๐ด, โ๐๐ผ๐ ๐น๐ถ๐ธ๐ฒ๐น๐ ๐ถ๐ ๐๐ต๐ถ๐ ๐ผ๐๐๐ฐ๐ผ๐บ๐ฒ, ๐ด๐ถ๐๐ฒ๐ป ๐๐ต๐ฒ ๐ฒ๐๐ถ๐ฑ๐ฒ๐ป๐ฐ๐ฒ?โ ๐ ๐ผ๐ฟ๐ฒ ๐ฐ๐ผ๐ป๐๐ฒ๐
๐, ๐ฟ๐ฒ๐๐ฟ๐ถ๐ฒ๐๐ฎ๐น, ๐ผ๐ฟ ๐ฎ ๐๐๐ฟ๐ผ๐ป๐ด๐ฒ๐ฟ ๐ฟ๐ฒ๐ฎ๐๐ผ๐ป๐ถ๐ป๐ด ๐บ๐ผ๐ฑ๐ฒ๐น ๐ฑ๐ผ๐ฒ๐ ๐ป๐ผ๐ ๐ฎ๐๐๐ผ๐บ๐ฎ๐๐ถ๐ฐ๐ฎ๐น๐น๐ ๐ฟ๐ฒ๐ฐ๐ผ๐๐ฒ๐ฟ ๐๐ต๐ฒ ๐ฐ๐ผ๐ป๐ฑ๐ถ๐๐ถ๐ผ๐ป๐ฎ๐น ๐ฑ๐ถ๐๐๐ฟ๐ถ๐ฏ๐๐๐ถ๐ผ๐ป๐ ๐ต๐ถ๐ฑ๐ฑ๐ฒ๐ป ๐ฎ๐ฐ๐ฟ๐ผ๐๐ ๐ฎ ๐๐ฎ๐ฏ๐น๐ฒ.
๐ฅ We introduce ๐๐จ๐๐๐ฅ๐๐ข, a benchmark with 6,280 questions over 10 real-world tables, spanning five probabilistic reasoning patterns and four linguistic variations. We also propose Auto-BN: an automated pipeline that induces a Bayesian Network from raw data, translates natural language into an executable probabilistic query, and performs exact inference.
With GPT-4o, Auto-BN reaches 46.7% accuracy within ยฑ0.05 and 0.107 MAE, outperforming NL2SQL, TableRAG, raw-table prompting, reasoning models, and premise-retrieval baselines. Its robust translation and fallback mechanisms reduce parsing error rates from as high as 20.8% to near zero across the tested backbones.
๐ง Use language models to understand the question and a grounded probabilistic layer to compute the answer. ๐๐จ๐๐๐ฅ๐๐ข provides a testbed for studying that interface at realistic table scale.
๐ Read the research ๐: megagon.ai/publications/lucaโฆ
#AACL #ACL #LLMs #AI #PhD #Research
@estevamhruschka
Can LLMs help scale Bayesian network learning?
Meet ABSOL: a hybrid approach combining parallelizable statistical learning with bounded LLM guidance.
Explore the architecture, benchmarks, and results in our latest article.
megagon.ai/bayesian-network-โฆ
#LLMs #EMNLP #NLP #AI
AI agents can build interactive tables, graphs, and canvases.
But users still have to respond through chat.
ABRA turns those outputs into shared surfaces where humans and agents can act and coordinate.
In our research, we introduce Agent-Bound UI and its reference prototype, ABRA, a shared interaction medium that enables humans and agents to communicate beyond chat through interactive canvases.
The key idea is to separate reusable interaction structures from task-specific meanings.
The same graph view, for example, can represent a social network, a sports tournament, or a directory structure. But โadd an edgeโ means something different in each context: connecting people, linking matches, or defining a directory relationship.
Agent-Bound UI lets agents reuse the same interaction structure while binding its meaning to the task and sharing that meaning with users and other agents.
Our experiments show that current LLM agents can:
โข Create task-specific canvases
โข Resolve interactions on canvases they own
โข Invoke published actions on other agentsโ canvases
Multi-view completeness and specification following remain challenges.
Agent interfaces can be more than outputs. They can become shared surfaces where humans and agents show, act, and coordinate.
Read the research: megagon.ai/publications/abraโฆ
๐ฅ Join us for the demo presentation at #EMNLP2026 at 9:00 AM on October 27th, presented by Hannah Kim.
#LLMs #Agents #HCI #NLP #EMNLP
๐ LLMs can improve Bayesian network learning without owning the graph.
ABSOL uses subsampled GES+BDeu plus four bounded LLM interventions. Across five benchmarks spanning 27 to 1,041 nodes, it was the only method to return a viable graph on all five.
Read: megagon.ai/publications/absoโฆ
Join @estevamhruschka at #EMNLP2026 for a poster presentation on ABSOL ๐
๐ก๐๏ธ Search systems need to understand increasingly complex requests. But improving that understanding can mean rebuilding millions of stored document representations.
What if the query side could get smarter while the existing index stayed intact?
In our new paper, we introduce Efficient Retrieval Adapter (ERA), a query-side linear adapter trained in two stages. First, ERA aligns a strong query embedder with a lightweight document embedder using the same unlabeled corpus documents in both spaces. Then it adapts that aligned representation with a limited set of relevance labels.
Across 126 MAIR retrieval tasks in six domains, ERA with a strong query embedder improves average nDCG@10 by more than 12 points for BGE-M3 and OpenAI-small document indexes at the 20% label ratio.
The alignment stage is the key: it produces large gains before labeled adaptation and keeps ERA effective in low-label settings. Its sampled documents represent only 1.28% of the MAIR collection and 5.20% of corpus tokens.
The implication is a more modular retrieval stack: improve query understanding without updating backbone embedders or rebuilding the corpus index.
For enterprise search and RAG AI systems, this creates a practical research direction for evolving query intelligence while preserving index infrastructure.
Read the research: megagon.ai/publications/asymโฆ
@SAYg_7 @estevamhruschka @PPezeshkpour
#EMNLP2026 #AI #LLMs #NLP
Megagon Labs retweeted
How confident can we be that an LLM agent actually completed its task?
Long trajectories, multiple sources of uncertainty, and limited model access make this hard.
Our new work, CRGs, uses evidence-grounded reasoning for better-calibrated, interpretable confidence estimates.
๐
๐ข New preprint from @MegagonLabs ๐งต
Confidence estimation for LLM agents is hard: evidence of success spans tool calls, observations, reasoning, and memory. When model internals are inaccessible, reruns are expensive, and training data is scarce, how can we reliably assess whether an agent succeeded and how much to trust its output?
Meet ๐๐ผ๐ป๐ณ๐ถ๐ฑ๐ฒ๐ป๐ฐ๐ฒ ๐ฅ๐ฒ๐ฎ๐๐ผ๐ป๐ถ๐ป๐ด ๐๐ฟ๐ฎ๐ฝ๐ต๐ (๐๐ฅ๐): a training-free, black-box method that turns a single execution trajectory into a structured, evidence-grounded argument for success and an auditable confidence estimate.
Paper: arxiv.org/abs/2610.07948
Code: github.com/megagonlabs/crg_cโฆ
Megagon Labs retweeted
๐ข New preprint from @MegagonLabs ๐งต
Confidence estimation for LLM agents is hard: evidence of success spans tool calls, observations, reasoning, and memory. When model internals are inaccessible, reruns are expensive, and training data is scarce, how can we reliably assess whether an agent succeeded and how much to trust its output?
Meet ๐๐ผ๐ป๐ณ๐ถ๐ฑ๐ฒ๐ป๐ฐ๐ฒ ๐ฅ๐ฒ๐ฎ๐๐ผ๐ป๐ถ๐ป๐ด ๐๐ฟ๐ฎ๐ฝ๐ต๐ (๐๐ฅ๐): a training-free, black-box method that turns a single execution trajectory into a structured, evidence-grounded argument for success and an auditable confidence estimate.
Paper: arxiv.org/abs/2610.07948
Code: github.com/megagonlabs/crg_cโฆ
Editing a multi-agent plan is not the same as steering it.
We introduce ๐๐ ๐๐๐ฃ๐ข๐ : ๐ฎ ๐ฝ๐ฟ๐ผ๐๐ผ๐๐๐ฝ๐ฒ ๐ณ๐ผ๐ฟ ๐ต๐๐บ๐ฎ๐ป-๐๐๐ ๐ฐ๐ผ-๐ฝ๐น๐ฎ๐ป๐ป๐ถ๐ป๐ด ๐๐ต๐ฎ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐ ๐๐๐ฒ๐ฟ๐ ๐๐ผ ๐ฒ๐ฑ๐ถ๐ ๐ฎ๐ด๐ฒ๐ป๐ ๐ฝ๐น๐ฎ๐ป๐ ๐๐ถ๐ฎ ๐ฐ๐ต๐ฎ๐, ๐๐ฎ๐ฟ๐ด๐ฒ๐๐ฒ๐ฑ ๐ณ๐ฒ๐ฒ๐ฑ๐ฏ๐ฎ๐ฐ๐ธ, ๐ด๐ฟ๐ฎ๐ฝ๐ต ๐บ๐ฎ๐ป๐ถ๐ฝ๐๐น๐ฎ๐๐ถ๐ผ๐ป, ๐ฎ๐ป๐ฑ ๐บ๐ฒ๐ฟ๐ด๐ฒ, ๐๐ฝ๐น๐ถ๐, ๐ฎ๐ป๐ฑ ๐ฟ๐ฒ๐ฝ๐น๐ฎ๐ป ๐ผ๐ฝ๐ฒ๐ฟ๐ฎ๐๐ถ๐ผ๐ป๐.
The key finding:
More controls reduce some effort, but they also create new verification work.
Users built hybrid workflows rather than choosing one interaction type.
Global feedback was most robust for complex revisions. Targeted feedback preserved unchanged parts of the plan, but could introduce structural errors. Edit sequences were easier to audit, but less reliable for complex changes.
For agentic AI, the design challenge is to โhelp users know whether the edited plan still works.โ
๐ Read the article: megagon.ai/human-llm-co-planโฆ
@estevamhruschka
#AI #Agents #LLMs #HumanCenteredAI #CAIS2026
Weโre hiring a Fall/Winter PhD Research Intern at Megagon Labs.
Weโre looking for candidates with research or technical experience in any one or more of the following areas:
โข Retrieval-Augmented Generation (RAG)
โข Function Calling
โข Document Understanding / PDF Parsing
You do not need experience across all three areas. Strong experience in any one of these topics is relevant.
Interested in joining us this fall? Learn more and apply โ megagon.ai/careers/internshiโฆ
Applicants must have or be able to obtain appropriate U.S. work authorization for the internship period.
#AI #NLP #LLMs #internships #phdposition
Correct answers are no longer enough. As LLMs become part of systems that use tools, memory, and evolving knowledge, we also need to understand how those components change model behavior.
That question shaped much of the conversation at ACL 2026.
Read the article!
megagon.ai/acl-2026-highlighโฆ
#AI #LLMs #CompoundAI
Your AI agent can cite a source. That doesn't mean the source says what the row claims.
Here's the failure mode we kept seeing: an LLM curator builds a plausible structured table from parametric memory; it "just knows" what should go in the cell, then bolts on a page-level citation afterward to make it look sourced. The citation is real, and the row is not grounded in it. It's structured output with decorative provenance.
We built Stage-Audit to close that gap in the context of what we call Seed2Frontier discovery: starting from a single Wikipedia seed page and autonomously finding the additional pages needed to complete a structured table that answers a natural-language query.
megagon.ai/data-provenance-sโฆ
#AIResearch #AgenticAI #CompoundAI #LLM #DataManagement #AIAgents
Most LLM extraction systems forget what they learned the moment the document ends. At ACL 2026, we introduce DySECT: a self-evolving extraction system that builds an explicit KB as it extracts.
This is a glance at our loop:
extract triples -> update KB -> organize and score knowledge -> feed it back into extraction
On DocRED, KB-guided extraction improved recall across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5.
The key result: 5-8% recall gains on the first guided iteration, without fine-tuning or synthetic data generation.
Adaptation can happen through inspectable memory, not only opaque model updates.
๐Read the article: megagon.ai/self-evolving-infโฆ
#ACL2026 #NLProc #LLM #InformationExtraction #EnterpriseAI #KnowledgeGraphs #AIResearch #MegagonLabs #ContinuousLearning
As AI agents become more capable, understanding how they make decisions is becoming just as important as the answers they produce.
In our latest blog, we explore why agent observability is emerging as a critical capability for enterprise AI. We also introduce L.A.K.E. (Logic Agent for Knowledge Extraction), our agentic data planning framework that makes planning visible through executable workflows and interactive DAGs, helping developers inspect, evaluate, and improve how AI agents reason across complex data sources.
Transparency isn't just about explainability; it enables debugging, optimization, and building more reliable AI systems.
Read our blog post: megagon.ai/agent-observabiliโฆ
#AI #EnterpriseAI #AgenticAI #MachineLearning #Transparency #MegagonLabs
Most ๐๐๐ ๐ฒ๐
๐๐ฟ๐ฎ๐ฐ๐๐ถ๐ผ๐ป ๐๐๐๐๐ฒ๐บ๐ treat each document in isolation, failing to retain and reuse knowledge acquired from previous extractions. That is a problem in domains where terminology changes, taxonomies evolve, and rare concepts matter.
At ACL 2026 in San Diego, we introduce ๐๐๐ฆ๐๐๐ง: ๐ฎ ๐๐๐ป๐ฎ๐บ๐ถ๐ฐ ๐ฆ๐ฒ๐น๐ณ-๐๐๐ผ๐น๐๐ถ๐ป๐ด ๐๐
๐๐ฟ๐ฎ๐ฐ๐๐ถ๐ผ๐ป ๐ฎ๐ป๐ฑ ๐๐๐ฟ๐ฎ๐๐ถ๐ผ๐ป ๐ง๐ผ๐ผ๐น๐ธ๐ถ๐.
Extraction should create reusable knowledge. So, DySECT uses an LLM to extract structured triples, stores them in a growing knowledge base, and assigns confidence based on source reliability, repeated evidence, and conflicts.
This accumulated knowledge is now a resource that should improve subsequent extraction rounds. The KB enriches previous extractions through hierarchical concept abstractions, confidence calibration, mutual exclusion constraints, and automated property validation. This enriched knowledge is then fed back into the extractor via prompt augmentation, encouraging broader coverage and discovery of new information in later iterations.
On DocRED, KB-guided extraction improved recall across GPT-4.1, GPT-4.1-mini, LLaMA-3.3 70B, and Kimi K2.5. The first KB-guided iteration ๐ถ๐บ๐ฝ๐ฟ๐ผ๐๐ฒ๐ฑ ๐ฟ๐ฒ๐ฐ๐ฎ๐น๐น ๐ฏ๐ ๐ฑ-๐ด% ๐๐ถ๐๐ต๐ผ๐๐ ๐ณ๐ถ๐ป๐ฒ-๐๐๐ป๐ถ๐ป๐ด ๐ผ๐ฟ ๐๐๐ป๐๐ต๐ฒ๐๐ถ๐ฐ ๐ฑ๐ฎ๐๐ฎ ๐ด๐ฒ๐ป๐ฒ๐ฟ๐ฎ๐๐ถ๐ผ๐ป.
Adaptive extraction does not have to mean opaque model updates. DySECT keeps accumulated knowledge explicit, inspectable, and editable.
For enterprise AI systems, that matters. Knowledge can evolve, but oversight and transparency stay in the loop.
๐ Read the research: megagon.ai/publications/dynaโฆ
Work by @moinnas, Hannah Kim, @estevamhruschka Hruschka
#ACL2026 #continuouslearning #AI #LLMs
As LLM systems become more agentic, improving individual outputs may not be enough.
In a recent Megagon Labs Seminar Series talk, Jay-Yoon Lee (Seoul National University / UIUC) explored the idea that correctness and reliability often emerge at the set level. Modern LLM systems increasingly operate over collections of retrieved documents, generated statements, and intermediate reasoning steps, yet most frameworks still treat these elements independently.
Interactions such as contradictions, redundancy, and joint relevance can shape outcomes just as much as the information itself. The talk introduced a unified perspective centered on set-level verification, retrieval, and editing as important directions for building more reliable and controllable LLM systems.
Thank you to Jay-Yoon for sharing his insights.
#LLM #AgenticAI #AIResearch
The way AI agents plan over enterprise data can dramatically change both latency and reliability.
This process of data planning, how an agent sequences operations like database queries, document retrievals, and API calls to resolve complex prompts, is foundational to enterprise AI performance.
In our latest research, we found that different planning regimes produce very different trade-offs:
- Single-shot tree planning achieved the strongest overall performance and lowest latency.
- Iterative planning provided stronger self-correction at a significantly higher execution cost.
We also observed that the planning structure itself, including generation direction and refinement strategy, can substantially impact agent behavior.
These findings highlight a broader challenge in agentic AI: you cannot optimize or trust what you cannot inspect.
As agents increasingly orchestrate tools across databases, documents, and external systems, understanding how plans are generated and executed becomes critical for debugging, evaluation, and deployment.
To address this, we introduce L.A.K.E. (Logic Agent for Knowledge Extraction), an agentic data planning framework designed to make planning observable. L.A.K.E. exposes execution paths as interactive DAGs with step-level provenance, enabling developers to inspect workflows, compare planning strategies, and understand why agents succeed or fail.
This work reflects a broader direction we are exploring at Megagon Labs: building agentic systems that are not only capable of reasoning over complex enterprise data, but also transparent, debuggable, and efficient to operate.
#Agentic #NLProc #LLMs
megagon.ai/publications/l-a-โฆ
Megagon Labs retweeted
Can LLMs automate prompt engineering?
RPT says: Apparently yesโby turning prompt tuning into a function-calling workflow: evaluate, diagnose recurring failures, use memory, and revise.
The result: stronger prompts on reasoning tasks and better-calibrated confidence.
๐ข New Preprint
Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts?
Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs.
Paper: arxiv.org/abs/2605.21781
@MegagonLabs
1/n
Megagon Labs retweeted
๐ข New Preprint
Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts?
Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs.
Paper: arxiv.org/abs/2605.21781
@MegagonLabs
1/n
๐ช๐ฒ ๐ณ๐ผ๐๐ป๐ฑ ๐๐ต๐ฎ๐ ๐๐๐ ๐ฎ๐ด๐ฒ๐ป๐๐ ๐ฐ๐ฎ๐ป ๐บ๐ฎ๐๐ฐ๐ต ๐๐ต๐ฒ ๐ฎ๐ฐ๐ฐ๐๐ฟ๐ฎ๐ฐ๐ ๐ผ๐ณ ๐ฅ๐ฒ๐๐ฐ๐-๐๐๐๐น๐ฒ ๐ฎ๐ด๐ฒ๐ป๐๐ ๐๐ต๐ถ๐น๐ฒ ๐๐๐ถ๐ป๐ด ๐ฎโ๐ฏ๐
๐ณ๐ฒ๐๐ฒ๐ฟ ๐๐ผ๐ธ๐ฒ๐ป๐.
In our latest research, we revisit a core assumption in agentic AI: that agents must interleave reasoning and execution at every step to remain adaptable.
We compare single-step horizon planning against full-horizon planning for data-centric tasks such as Knowledge Base QA and Multi-hop QA.
Our experiments show that full-horizon planning with lazy replanning achieves accuracy parity with step-wise approaches across varying task depths, branching complexity, and tool robustness levels, while substantially reducing token usage.
In many cases, eager execution monitoring provided little benefit relative to its computational overhead.
These findings suggest that planning horizon is not merely an implementation choice, but a fundamental systems design decision for LLM agents.
For well-defined data-centric tasks, generating complete plans upfront and replanning only when necessary may offer a more efficient default for tool-using AI systems.
This work will be presented at CAIS 2026
๐ Read the research: megagon.ai/publications/rethโฆ
#AI #AgenticAI #LLMs #CAIS









