wish there was a way we could find out

Okay, here we go again Graph or loop?

What can Fable 5 do that GLM-5.2 can't, when you hand them real agentic work? To answer that question, we connected Fable 5 and GLM-5.2 to 17 SaaS tools and gave them 47 tasks. As expected, Fable 5 solved all 47 tasks. GLM-5.2 solved 45, but the two misses tell an important story. They showed us exactly how open-weight models still fall short when trying to match SOTA performance. Let’s dig in. Background: Each model ran as an agent connected to 17 live SaaS accounts: Airtable, Datadog, GitHub, Gmail, Google Calendar, Google Drive, Google Sheets, HubSpot, Jira, LaunchDarkly, Linear, Notion, PagerDuty, PostHog, Salesforce, Slack, and Zendesk. The tasks are the kind of work you'd actually delegate to an agent: - Find every file in this repository that leaks a credential - Deduplicate these CRM records - Repair this broken recurring calendar event. Every task had a known correct answer baked in ahead of time. In this post, we looked at the traces to analyze how exactly GLM-5.2 “failed” compared to Fable 5. GLM-5.2 solved 45/47 tasks and Fable 5 had a perfect 100% score. In addition: - Fable averaged 84 seconds per task; GLM averaged 148. Across the full suite, Fable finished in nearly half the total time (66 minutes vs 116). - Fable was the faster model in 43 of the 47 scenarios. - Fable used about 20% fewer tokens overall - Fable needed fewer tool calls (239 vs 294) and fewer conversation turns (6.1 vs 7.3 on average) to get to an answer The most interesting part comes from digging deeper into the stack traces. That revealed some interesting gaps: Gap #1: Knowing when the job isn't finished One of the tasks GLM-5.2 failed was a GitHub security audit. The instruction was to find every Python file in a repository that contains a hardcoded `secret_key`. The repository had been seeded with exactly 130 such files, so the correct answer was known in advance. Fable 5 found all 130 of them. This took 3 tool calls and 68 seconds: Fable constructed an effective search query on its first attempt, pulled every page of results, deduplicated the paths, and answered the question. GLM-5.2 found 120 files, and reported those 120 as the complete answer, without ever questioning whether it might have missed something. Both models had access to identical tools. GLM used a slightly different search query that returned fewer results, and it simply trusted what came back. Along the way, it also lost track of a results file it had saved earlier and spent turns searching the filesystem trying to find it again, plus hit two errored tool calls while trying to fetch file contents. In essence, GLM-5.2 ended up spending 262 seconds and three and a half times the tokens to deliver 92% of the answer. Ninety-two percent sounds close, but in a real security audit, that gap is 10 leaked credentials making it into production. Gap #2: Judgment when the criteria are fuzzy The second failed task is more unsettling, because GLM did almost everything right and still failed to get to a complete answer. The task was a Zendesk SLA audit: find the open billing tickets where no support agent had posted a public reply within 24 hours of the ticket being created. This requires reading each ticket's actual conversation history and making a judgment call about whether a genuine agent reply happened. GLM-5.2 inspected every candidate ticket, exactly as instructed. It also computed breach timestamps correctly. It also produced perfectly structured output in exactly the requested format. But then it classified the wrong tickets as breached. GLM spent 927,000 tokens and six and a half minutes producing a wrong answer that looked correct on the surface. Fable 5 identified the exact set of breached tickets in 131 seconds. What makes this failure mode dangerous is precisely how presentable the wrong answer was. The formatting was right, the timestamps were right, the structure was also right; a human skimming the output would almost certainly have approved it. A human would identify the error after carefully analyzing the stack traces. Gap #3: Efficiency, compounded Even on the 45 tasks both models passed, the traces often looked very different, and one task made the difference quite visible. The task was a LaunchDarkly configuration change applied via JSON Patch, a format that demands strict precision. Fable 5 completed it in 45 seconds, using 3 tool calls and 181,000 tokens. GLM-5.2 got the same correct result, after 8.8 minutes, 17 tool calls, and 982,000 tokens. That's 11.7 times longer and more than five times the tokens for an identical outcome. Looking at the largest speed gaps across the whole run: the LaunchDarkly change at 11.7x, the GitHub secrets audit at 3.9x, a Google Calendar recurring-event repair at 3.6x, a free/busy scheduling task at 3.4x, an Airtable batch-isolation task at 3.4x, the Zendesk SLA audit at 3.0x. The pattern underneath all of these is that Fable tends to reach the right tool with the right parameters on the first attempt, while GLM takes a more exploratory path, doing extra searches, extra retries, occasional detours to recover from its own missteps. This difference barely matters in a single chat exchange, but in an agent workflow, where every step feeds the next one, the time compounds across the entire task. That's how you end up finishing the same suite of work in half the time and at 80% of the token cost. What all this actually tells us The interesting conclusion here isn't "the closed model beat the open one.", but *where* it beat it. Both models can definitely use tools, navigate real APIs, handle authentication, parse messy responses, and chain steps together. The real gaps were things like: - Knowing when a job isn't actually finished yet. - Verifying its own work before committing to an answer, - Treating "the output looks plausible" and "the work is complete" as different things - Getting judgment calls right when the criteria are fuzzy In other words, Fable 5 scored higher in the places where small mistakes are hardest to spot and most costly to miss.

We're excited to be at AI Engineer World's Fair 2026. 🚀 If you're building AI agents, developer tools, or enterprise AI systems, come meet the Composio team. We'll be talking about secure integrations, real-world tool use, and what it takes to move agents from demos to production. Looking forward to connecting with builders. See you there! #AIEngineering #AIAgents #AgenticAI #DeveloperTools #LLMs #AIEngineerWorldsFair

If you won an @Nvidia DGX Spark, what would you do with it? "Get rid of OpenRouter and just run all our AI on this." "Keep it at home and show it off to everyone I know." "Run my entire startup autonomously. The first step to running a business without me." The DGX Spark is a supercomputer that fits in your home. Powerful enough to run a large language model for everyday tasks without ever touching ChatGPT or paying for a subscription. Higher compute density than Apple. High performance memory. All local. @Composio hosted a hackathon on a yacht with @Nebiusai and handed one out as the prize. 𝐓𝐈𝐌𝐄𝐒𝐓𝐀𝐌𝐏𝐒 (00:00) Meet the NVIDIA DGX (00:32) Running LLMs at Home (00:49) What Winners Would Build (01:12) Autonomous Business Agents (01:40) How It Compares to Apple (01:58) Compute Density and Memory