Software Engineer at Google Research. I work on Gemini factuality. Opinions my own. He/him.
Israel
Joined March 2010
- Tweets8K
- Following1K
- Followers3.5K
- Likes125
Factuality isn't just a text problem, it's a visual one too. Latest image models can generate stunning infographics that are, unfortunately, riddled with factual errors.
We can’t let aesthetic quality mask data "hallucinations." (1/3)
For developers, the fix isn't just "better pixels." We need:
1. Rigorous factuality evals for multimodal output.
2. Robust RAG pipelines that feed image models the correct, grounded data to work with.
Precision must lead the generation process. (2/3)
The current default for most GenAI workflows—from slide deck generation to automated coding—is to omit attribution. In my view, this is an extremely dangerous default for the industry to maintain. (1/3)
When models output information without citations, they remove the user's ability to verify "ground truth."
In professional environments, attribution isn't a courtesy; it's a critical safety mechanism. We shouldn't be asking users to trust a black box. (2/3)
At @GoogleResearch, we believe transparency must be a first-class citizen in the UI.
Moving toward a "citation-by-default" model is the only way to ensure AI acts as a reliable partner rather than just a convincing generator. (3/3)
Standard prompt engineering dictates forcing direct answers for simple trivia to reduce latency and save tokens.
But our latest @GoogleResearch paper shows this "optimization" actively degrades model accuracy on factual recall. (1/4) 🧵
Through controlled experiments, we identified two core driving mechanisms: 1️⃣ Computational Buffer Effect: Model uses reasoning tokens to execute latent computations. 2️⃣ Factual Priming: Intermediate related statements shift target facts closer to activation. (3/4)
The penalty? Hallucinating intermediate steps dramatically corrupts the final answer.
To fix this, pipelines must use trajectory sampling—generating multiple paths and filtering out traces with unverified factual claims.
Paper: arxiv.org/abs/2603.09906 (4/4)
In LLM reliability, "helpfulness" can be a double-edged sword. Most models are optimized to answer every query, but we also need to measure their capacity for adversarial pushback.
The Bullshit Benchmark is an interesting evaluation designed to test exactly that. (1/4)
From a factuality perspective, this tracks a model's ability to identify logical fallacies and push back when a user is "completely off."
It’s about moving beyond sycophancy toward agents that prioritize reality, especially in professional-grade workflows. (3/4)
Tracking these "refusal" metrics is a vital part of our work at @GoogleResearch as we build more trustworthy partners.
Explore the Bullshit Benchmark here: petergpt.github.io/bullshit-… (4/4)
Intelligence is nothing without reliability. The latest @ArtificialAnlyls results for Gemini 3.1 Pro show we're making massive gains in the "hallucination gap." (1/3)
Gemini 3.0 Pro already led in accuracy on the AA Omniscience benchmark. But 3.1 Pro significantly reduces the hallucination rate, now leading the combined index.
It’s not just smarter; it’s more discerning. (2/3)
With a 57 Intelligence Index score and 1M token context, this is a major win for high-stakes RAG and enterprise workflows.
Reducing hallucinations while scaling reasoning is the critical frontier for my work at @Google Research.
Analysis: artificialanalysis.ai/models… (3/3)
The prevailing evaluation paradigm for research agents is flawed. Most benchmarks rely on single-answer verification, like finding a "needle in a haystack”, while ignoring the comprehensiveness gap.
We need to measure if agents actually perform a deep investigation. (1/4)
From a systems engineering perspective, this focuses on the trade-off between attribution precision and information density.
Current SOTA models often fail at producing comprehensive lists or pushing back against stale, non-authoritative inputs. (3/4)
We’ve open-sourced the DeepSearchQA suite on Kaggle to help the community move toward agents with the exhaustive reliability required for professional-grade research.
Read our full paper on arXiv: arxiv.org/abs/2601.20975 (4/4)