Igor Filippov (@igorfeelippov) profile picture

pre-training at @MireloAI, ex. @moonvalley, @Snapchat, startups

Amsterdam
Joined January 2022
Photo by Igor Filippov (@igorfeelippov)Photo by Igor Filippov (@igorfeelippov)Photo by Igor Filippov (@igorfeelippov)
Igor Filippov retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
2,367
7,254
5,755
55,906
10,094,771
Igor Filippov retweeted
Most real2sim demos you see now are actually real2mesh demos. In order to extract physical properties from objects you need 4d scene understanding. We created a pipeline which reconstructs ego+exo footage into photorealistic temporal representation and then tunes simulation using our SDK until it follows reality 1:1. This unlocks benchmarks for both real2sim and sim2real, while simultaneously gives a source of ultra precise hand-objects interaction data.
13
27
2
290
18,603
How to cook a new model 🧑‍🍳 (Mirelo SFX 2.0 edition): 1. Data. Curation, annotation, filtering. Repeat. and again :) 2. Model. Remove informational bottlenecks. Simplify. 3. Training. Log everything. Make it stable and scale. In the end the ultimate test: does it actually sound like what's happening on screen?? No? → go to step 1.
1
12
257
Igor Filippov retweeted
ok it's official... AI-written unit tests and integration tests are empirically proven to be unhelpful. you should tell your agents to stop writing tests by themselves on the deepswe eval set, banning sonnet 5.5 high from writing any tests actually resulted in slightly higher success rate (non stat-sig), with less time and token spent (stat-sig). this is pretty hard empirical evidence across the tests that were written by the "tests allowed" baseline arm, 65% of them were unit tests, 35% were integration tests, neither bucket resulted in any improvement compared to not writing any tests at all i also picked a random sample of 44 tasks subset where i completely disabled executing even existing tests - it also did not affect success rate at all i spot checked many tests written in the baseline arm, and my intuition is that most tests are simply a repetition of the implementation agent-written tests do not add any value because both the implementation and the tests were simply the agent's interpretation of our intent. the tests aren't any more accurate than the implementation itself i suspect we can still extract some value from unit tests and integration tests if we describe them ourselves when we believe we can articulate our intent better through test cases than through requirements. but i have not proven this yet also worth noting, during deepswe eval the agent wrote almost no e2e tests (only 17, compared to 3000+ unit/integration tests written). so this analysis does not prove nor disprove the value of e2e tests - i will do another eval specifically for that
464
208
187
2,808
480,205
Igor Filippov retweeted
$6,000 in used 3090s runs a 122b model at 93 tok/s single stream with 198k context. at home. I spent six months measuring everything this box can do and put it all in one article. what to buy. how to split 4 cards, and why the popular way throws away 3.4x. which quants are a trap on ampere. what broke. all numbers from actual runs including the stuff that failed lol
6
7
75
3,001
Igor Filippov retweeted
Today, together with @kyutai_labs, we’re introducing our new Audio-to-MIDI model. It takes a finished recording, identifies the instruments playing, and returns separate MIDI tracks for each — voice, drums, bass, keys, and more. Unlike most existing solutions, our model works directly from the full mix rather than requiring separate stems. It also detects chords, key, and tempo, giving producers broader musical context. We’ve written more about the model, the problem, and how it works here: mirelo.ai/blog/turning-audio…
102
677
234
4,867
922,816
Igor Filippov retweeted
Six offline RL distillation losses. One base model. The exact same math rollouts. Do they actually learn the same thing? Turns out most "new" losses — RFT, DFT, offline GRPO — write nearly the same direction in weight space as plain SFT. Only DPO learns something genuinely different: near-orthogonal, its own loss basin, rewires what the network computes. Reward-weighting changes the step size. DPO changes the direction. Accepted at @icmlconf (MechInterp workshop) paper + interactive companion 👇 huggingface.co/spaces/AlexWo…
3
29
1
187
17,609
Igor Filippov retweeted
if fable 5 agrees to do your work instead of downgrading you to opus, you’re in the permanent underclass
42
155
22
5,023
271,723
Igor Filippov retweeted
Tired of getting rejects and reading reviewer comments that clearly missed the point of your paper? I built an peer review skill for Claude Code that actually gives you useful feedback before you submit. 👇
5
9
41
1,523
Igor Filippov retweeted
We just released our first models at @poolsideai – including Laguna XS.2 (open weights) that competes with Qwen3.6-35B. I worked across pretraining — happy to answer questions! We now have great understanding of exactly all components that went into training through principled ablations, and now we're confident to scale. This year would be 🔥 for us! More coming very soon
19
26
5
241
25,568
Igor Filippov retweeted
If we already have AGI — why can't it solve a sudoku puzzle? According to any reasonable definition, a "reasoning" AI should be able to arrange a few numbers in a 9x9 grid without much trouble. But frontier LLMs are really bad at this. By "this" we mean Constraint Satisfaction Problems — stuff like chip design, energy grid optimization, HFT. This is because token-by-token generation tends to commit to bad choices very early on, and can't revise them later. However, there is a whole other approach — Energy-Based Models, and researchers like @YannLeCun have been pushing for it since the 1980s. Now it finally seems to be feasible, and, among other things, can give us cheap and efficient formal software verification — which is very much needed, given just how messy ALL of humanity's software turned out to be. EBMs minimize an energy function in latent space. High energy means high constraint violation (something's wrong); low — you're close to the truth. Unlike LLMs, they optimize the entire trace at once — and can refine it iteratively. This allows for much more precision with much less compute 🧵
2
4
1
30
777
Igor Filippov retweeted
The first world-class clean AI video model is here. Marey by Moonvalley—built for filmmakers, trained exclusively on licensed data. Brought to you by the leading minds in AI and film. The future starts now. Join the waitlist: bit.ly/MV_waitlist
16
41
20
179
73,545
I landed my first ever open-source PR right into the core of the @huggingface 🤗 @diffuserslib 🧨! I made a new Video2Video Pipeline with ControlNet support. github.com/huggingface/diffu… Docs: huggingface.co/docs/diffuser… A big shout-out to @aryanvs_ for his support and guidance.
2
3
24
712