
RS @Meta. Frmr PhD Student @umdcs advised by @SoheilFeizi, intern @ MSR + FAIR. Interpretability / Reliability / Fairness. https://xitter.cf/t.co/Vj3mDk9uZ3
New York, NY
Joined March 2022
- Tweets239
- Following276
- Followers1.3K
- Likes1.4K
Pinned Tweet
What if you could turn your vision-only model into a VLM, by only training a linear layer using a modest amount of unlabeled images?
In Text2Concept (ICML exhibit hall 1 tmrw at 11⏰), we show how to do exactly that, unlocking new powers for countless models at minimal cost. 🧵
Mazda Moayeri retweeted
Meta has released Muse Spark 1.3, their fourth Muse Spark model release in five months. Muse Spark 1.3 (max), which is in limited preview for Meta’s partners, scores 62 on the Artificial Analysis Intelligence Index, behind only Claude Fable 5.1 and Claude Opus 5. The variant available now, Muse Spark 1.3 (xhigh), scores 61 and ties with GPT-5.6 Sol (max) and Grok 4.6 (high). Both variants’ gains come primarily from improvements in agentic work and scientific capabilities
Muse Spark 1.3 (xhigh) enters the Artificial Analysis Intelligence Index at 61, up 4 points from Muse Spark 1.2 (57, August) and 8 points from Muse Spark 1.1 (53, July). It enters tied with GPT-5.6 Sol (max), Grok 4.6 (high), and Claude Opus 5 (high), and behind Claude Fable 5.1 (max, 66), Claude Opus 5 (max, 63), and Claude Fable 5 (max, 62)
Muse Spark 1.3 (max), which is in a limited preview stage, lands at 62. This higher index score is enabled by gains vs. Muse Spark 1.3 (xhigh) in Tau3-Bench Banking (52% vs. 47%) and GDPval-AA v2 (1,754 Elo vs. 1,709). Muse Spark 1.3 (max) is second only to Claude’s Fable and Opus variants in total score
Congratulations to @AIatMeta, @finkd, and @alexandr_wang on the release!
Key Takeaways:
➤ Continued improvement on agentic knowledge work tasks. At the launch of Muse Spark 1.2, we noted its significant gains in agentic knowledge work performance vs. Muse Spark 1.1. The latest iteration continues this trend, with Muse Spark 1.3 (xhigh) demonstrating a notable 12-point gain vs. Muse Spark 1.2 in Tau3-Bench Banking (35% to 47%), a 5-point gain in Terminal-Bench 2.1 (80% to 85%), and a new GDPval-AA v2 Elo of 1709 against its predecessor’s 1615. Muse Spark 1.3 (max) improves further on Tau3-Bench Banking (52%) and GDPval-AA v2 (1,754 Elo). This Tau3-Bench Banking score is #1 among all models. Muse Spark 1.3 (max) achieves these higher agentic work scores by using more turns and total reasoning tokens, reasoning 62% more on GDPval-AA v2 and 28% more on Tau3-Bench Banking compared to Muse Spark 1.3 (xhigh)
➤ The lowest cost per task for any model at 59+ on the Artificial Analysis Intelligence Index. Muse Spark 1.3 (xhigh) costs $0.55 per Intelligence Index task at Meta's unchanged $1.25/$4.25 per 1M token pricing ($0.15 for cached input), with its peers GPT-5.6 Sol (max) and Grok 4.6 (high) costing $0.95 and $0.94 respectively, a 70%+ premium. This places Muse Spark 1.3 (xhigh) on the Pareto frontier for Intelligence vs. Cost per Task. Its cost per task is higher than Muse Spark 1.2 ($0.40 per task), driven by ~57% more input tokens per task on agentic evaluations, with output tokens up only ~8%. Pricing for Muse Spark 1.3 (max) is not yet publicly available
➤ Scientific Reasoning results rose across the board, led by CritPt. CritPt was the standout non-agentic score gain vs. Muse Spark 1.2, with a material +8 points for the xhigh variant (18% to 26%), and GPQA Diamond achieved +4 points (90% to 94%), while Humanity’s Last Exam and SciCode each gained a more modest 2-3 points (45% to 47% and 56% to 59%, respectively). Muse Spark 1.3 (max) achieved roughly similar scores to the xhigh variant, gaining 2 points in Humanity’s Last Exam, tying on GPQA Diamond, and losing a point on CritPt vs. Muse Spark 1.3 (xhigh)
➤ Minor regressions in only two evaluations. Both Muse Spark 1.3 (xhigh) and Muse Spark 1.3 (max) dropped 4 points in AA-LCR (83% to 79%) when compared to Muse Spark 1.2, and AA-Omniscience (Accuracy) fell 3 points for xhigh and 1 point for max. The drops in AA-Omniscience (Accuracy) are due to a higher abstention rate (not answering questions when unsure), which also lowered the hallucination rate for Muse Spark 1.3 (xhigh)
Other model details (xhigh variant):
➤ Context window: 1M tokens, unchanged from Muse Spark 1.2
➤ Pricing: unchanged from Muse Spark 1.2: $1.25/$4.25 per 1M input/output tokens, with cache hits discounted to $0.15 per 1M
➤ Input modalities: text, image, video
➤ Availability: Meta's first-party API and Muse Code
Mazda Moayeri retweeted
Our work on Block-Sparse Featurizer is out 🧊 :)
We revive an old idea from the structured sparsity literature and use it to carve activation space into meaningful regions.
It's a first concrete answer to the question our concept manifolds work left open ! :)
Secretary @marcorubio
Iranian immigrants have long contributed to the U.S. economy, especially in STEM, medicine and AI. Most strongly oppose the Islamic regime and support the goal of a strong and secure USA under President Trump and your leadership.
Over the past few months, many have faced serious hardship due to the USCIS hold. They are here legally and often hold advanced degrees. Returning them to Iran would place them in danger and risk sending highly trained experts in strategic fields back to the Iranian regime.
We would greatly appreciate it if you could revisit this decision.
Thank you for your consideration.
Mazda Moayeri retweeted
🧐 Can language models explore hidden environments? (1/n)
We introduce interactive tasks where models must probe for hidden info to find good solutions—but SOTA models often under-explore and end up sub-optimal.
Demo: explore-exploit-bench.github…
Paper: arxiv.org/abs/2601.22345
Mazda Moayeri retweeted
🚀 RELAI is live — a platform for building reliable AI agents
🔁 We complete the learning loop for agents: simulate → evaluate → optimize
- Simulate with LLM personas, mocked MCP servers/tools and grounded synthetic data
- Evaluate with code + LLM evaluators; turn human reviews into optimization-ready benchmarks
- Optimize with Maestro; tune prompts, configs and even agent graph for improved quality, cost and latency
Works with OpenAI Agents SDK, Google ADK, LangGraph, and all other agent frameworks
🌐 Get started (free): relai.ai
⭐ Open-source SDK: github.com/relai-ai/relai-sd…
I'll be speaking at this #ICCV2025 WS today at 4:30pm, room 308B.
Fav part of prepping was talking to my friends *outside* of AI, who (turns out) don't trust these systems nor those who build them.
Come by to hear how my perspective on trust has transformed over the past year.
Can we ever truly trust foundation models—and if so, how?
Our ICCV TrustFM workshop (t2fm-ws.github.io/T2FM-ICCV2…) is now accepting submissions (deadline: 8/1, attending: 10/19-10/23, Hawai'i)
Submit, attend, and learn from everyone around the world who is making FMs more trustworthy
Co-organized with Zeliang Zhang, @ChenliangXu, @huang_chao_cs, @MuCai7, @DrogoKhal4, @jd92wang, @hendrycks, @jure, etc.
@trustworthy_ml @StanfordAILab
Mazda Moayeri retweeted
🚨 70 million US workers are about to face their biggest workplace transmission due to AI agents. But nobody asks them what they want.
While AI races to automate everything, we took a different approach: auditing what workers want vs. what AI can do across the US workforce.🧵
Mazda Moayeri retweeted
We then analyze what the hierarchical concepts look like, finding an intriguing set of **shared** concepts across modality-specific backbones to enable hierarchical structures. This stands in contrast to the “usual understanding” of disparate representations spaces in VLMs!
8/11
Nice clean application of certified robustness to detect test set contamination with provably low false positive rates. Way to go @chengez1114 @Wenxiao__Wang !
🚨 New paper alert: DyePack – a provably robust way to flag LLMs that train on benchmark test sets. No model loss or logits needed, and false positive rates are theoretically bounded and exactly computable. Intrigued? Check out our paper at arxiv.org/abs/2505.23001
Thread below 👇
presenting this work on finer-grained foundation model evaluation at poster 623 this afternoon :) #ICLR25
Mazda Moayeri retweeted
In Singapore! We have an exciting set of talks and papers at #ICLR2025.
@besanushi , Vibhav and I will be around to chat about our recent inference scaling evaluation report. You can also talk to @MLMazda and @alesstolfo about their internship projects on model understanding.
Remember the Ghibli memes? Never direct replicas, but always unmistakably in that style?
Would you call that copying? How can you tell, and crucially, in a way that art + legal folks would understand + adopt?
We grapple with these qs in ArtSavant🎨 #iclr poster 535 this PM🧵
3. Are styles being copied by generative models? Under just simple prompting (i.e. ‘{prompt} in the style of Miyazaki’), yes for about 20% of artists.
Less than I expected! But then again, these were with last year's models...
If you've got any feelings or feedback about any of these questions, come by our poster (535) this afternoon!
This was a fun project (interpretability x art x law x society), and I look fwd to hearing ppl's thoughts today.
Special s/o to @b_shrir @BasuSamyadeep @FeiziSoheil
Mazda Moayeri retweeted
🚀 Introducing Data Agents— generate accurate, reasoning-based AI benchmarks from your own data in minutes!
⚡ With Data Agents, we’ve created 100+ benchmarks with 100K+ samples using docs from tools like React, PyTorch, Kubernetes, LangChain, and more.
📂 All benchmarks are now live on RELAI.ai and HuggingFace: huggingface.co/relai-ai
📊 Coming next: Our LLM leaderboard — see how popular models perform across these benchmarks.
📅 Want your own custom AI benchmarks? Book a demo here: calendly.com/d/crx2-k7b-pcm
More Details 👇



























