joyship // eigenvalue enjoyer
42
Joined May 2025
- Tweets772
- Following594
- Followers571
- Likes20.9K
Rishta naya rabba dil chu raha hai
Kheeche mujhe koi dore, teri ore
Teri ore, teri ore, teri ore, hai rabba
Teri ore, teri ore, teri ore
small work update:
- trained nanogpt mixture-of-expert without load balancer
- matched the same performance with nanogpt dense layer on public benchmark
- beats the performance of moe with sota load balancer
it was a nice saturday :D
in some random 3pm, i am in search of more natural way to cluster than k-means to prevent expert collapse in moe
having breakdown back to back since last 4 days due to pile up of both work and personal project. i am so bored of building agent harness now and more i work on it, more i realise the anti-thesis of it.
lately discussing with opus/fable on ideas that i want to experiment with.
despite me knowing that the approach can have a possible exploration, these ANT mfs would start pushing me back to a different direction i don't really care about.
pacing frontier my ass.
there are sometimes certain kind of work that changes your life completely - after finishing them you are not the same person anymore.
it ends up making you excited about what's next despite the inability to grasp the formulation or prior academic background.
for me it would be random matrix theory, back to determinants and matrices baby. rethinking what i had been doing for past 4 years but this time with a different kind of math entirely.
same excitement as when i used to read resnick halliday in class 9