Pinned Tweet
Finishing two weeks of tabular DL announcements with the python package for TabPack 🏎️
👂 👀
Go see and talk to Dima about their latest work!
This Monday, Nov 2, Dmitry Baranchuk (@DmitryBaranchuk) will present his paper:
Representation-Space MMD for Diffusion Language Models
Project also co-led by Ilya Drobyshevskiy (@free001style) and Ilia Sudakov (@sudakov_il)!
One more tabular ML release from our researchers, and a third one coming soon!
Stay tuned 🥠
I've long argued for more diverse evaluation setups in tabular ML research. This will continue. It was a key motivation behind our new paper
A thread on data scaling in MLPs, GBDTs and tabular foundation models, and what it means for generalization.
arxiv.org/abs/2610.26794
Find more details in the paper, and try the alternatives to MSE in your next tabular regression problem, we would be glad to know how it works out!
paper: arxiv.org/abs/2610.13909
code: github.com/yandex-research/t…
(8/N=8)
We propose one way to address the stopping-time mismatch at the training objective level we introduce Adaptive Quantile Regression
On 28 datasets with 1,000+ training samples, it improves on quantile loss on several backbones, without changing the inference procedure
(7/N)
On the 7 largest real datasets in our benchmark, choosing separate checkpoints for groups with different estimated data uncertainty can improve predictions! Random groups of the same size do not show the same benefit.
(6/N)
We observe that highly noisy regions in tabular datasets start overfitting while low-noise regions are still learning.
One global early-stopping checkpoint has to compromise between them.
(5/N)
Next, motivated by our earlier work on the role of data uncertainty in supervised tabular DL arxiv.org/abs/2509.04430
We look at early stopping and how modifying a Quantile loss can lead to a better compromise in it
(4/N)
Quantile loss [Koenker & Bassett, 1978] (also used by recent TFMs like Kumo Tabular and TabICLv2) is our main practical recommendation.
It most consistently improves upon MSE, across various MLP-based architectures.
(3/N)
Alternatives to MSE have received a lot of attention
We looked at many of them on tabular supervised regression
Among 18 losses we find only binning-based losses (popularized in RL not? so long ago arxiv.org/abs/2403.03950) and Quantile loss manage to improve upon the MSE
(2/N)
Training neural networks for tabular regression? Our latest work may be of interest
And it's not only about which of the 18! regression objectives is best, but touches on the role of noise and global early stopping in tabular DL in general
arxiv.org/abs/2610.13909
Thread (1/N)
If you are more of a visual learner, see the video @iNikitaGushchin and @sudakov_il cooked up below:
teapaw.org/sudakov_il/status/2107…
MMD (Maximum Mean Discrepancy) has shown promise in image generation. Can it improve diffusion language models too?
We post-train DLMs to match generated and reference distributions in a frozen DLM’s feature space. It works for both discrete and continuous models, and scales to 16B! 🚀
The results are strong across text generation, math reasoning, and code! Here’s how it works 👇
Our diffusion team is branching into text!
arxiv.org/abs/2610.06648
We propose a simple post-training method for diffusion LMs: swap the usual training objective for kernel-based distribution matching in feature space.
Promising gains from small-scale experiments to 16B models!
go see Roma at COLM and read reasoning shift!
In SF for COLM this week! Our paper Reasoning Shift was accepted as a spotlight at COLM’s Efficient Reasoning Workshop🙊
📚arxiv.org/abs/2604.01161
Super excited to meet Bay Area people, open for coffee and research chats, especially RL, reasoning & AI safety. Hit me up! ☕️🌉
We are excited to explore what agent architectures become possible when observation, reasoning and action don’t share one sequential loop, and when agents are allowed to arrange their cache and program at the inference engine level!
Read the paper and talk to us!
(5/5)
The engine efficiently batches inference across coroutines. Built on Mini-SGLang, supporting MHA, Gated DeltaNet and multimodal MRoPE
github.com/dvmazur/async_llm
(4/5)
An agent is a set of Python async/await coroutines. Each writes to a cache block and chooses which blocks to attend to through cache views. It can coordinate with asyncio events, locks and queues
(3/5)
Now, we introduce an abstraction around these ideas, inspired by asyncio, moving concurrent execution from token-level API calls into the inference engine layer
(2/5)
We argued in a recent blog post, that KV cache can serve as an agent runtime, if we rearranging cache blocks cleverly at inference time
teapaw.org/YandexResearch/status/…
(1/5)
New blogpost from our researchers 🧊
We discuss our recent work on how sharing and scheduling KV-cache state lets pretrained LLMs observe, reason, and act concurrently without additional training
research.yandex.com/blog/the…