
security specialist @houseofhades
latent space
Joined June 2015
- Tweets3.1K
- Following113
- Followers194
- Likes4.9K
aalquwayfili retweeted
AIとの対話を実験的に公開します。
note.com/hirasawa_susumu
aalquwayfili retweeted
قبل شهر، نشرنا «قافية» على قتهب، وبفضل الله -سبحانه- ثم دعمكم وصلنا إلى:
- ١٠٠ نجمة
- ٩ مطورين: @0xSm0ky @fhad_________ @forbiddenkeygen @MohannadNaj @bitbyte_log @AlkhazrajyMB @_0xFa @khengari77 @ivbme
- ٤ داعمين: @ysralot @forbiddenkeygen @bitbyte_log
فالحمد والشكر لله ❤️
إذا كنت مبرمج، أو تعرف مبرمج يبحث عن مشروع يساهم فيه ويتعلم منه، أو كانت عندك فكرة لميزة ودك تضيفها وتساعد بها الناس...
فموقع قافية صار مفتوح المصدر!
مشروع عام، غير ربحي، من الجميع وللجميع، في خدمة العربية وأهلها
مستودع قتهب: github.com/raaqimorg/qafiyah
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
aalquwayfili retweeted
We just released Polars 2.0.
It removed many of our legacy decisions makes the streaming engine our default and promotes SQL to a first class citizen within Polars.
It comes with initial out-of-core (spill to disk) support, a new Map data type and a lot of performance improvements. In fact, we think Polars is now one of the fastest analytical SQL engines on a single node. See benchmarks in the post: pola.rs/posts/release-polars…
aalquwayfili retweeted
New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s!
This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with incredibly clever engineering and ML judgement. If a flop is low value on a particular step, skip it.
Specifically:
-(~8s) Sampled softmax. If a token doesn’t appear in a batch, skip its lm_head fwd/bwd some fraction of the time.
-Sparse values. Only run an optimizer step for ngram embeddings that occurred in the batch. Set beta1 to zero to enable this. Beta2 is applied retroactively when the row is later used.
-Sparse updates. Only update ngram and value embeddings once every 4 steps instead of once every 2.
-Sparse communication. Shard the n-gram table across GPUs, and only pass the rows receiving updates on each step.
-Sparse optimizer states. For the n-gram table, reduce from 2 floats in Adam optimizer per param, to 1 float per 768 params.
-Hand-rolled flash attention for 64 dim heads.
There are several additions that add accuracy too:
-(~4s) EMA during last 300 steps, combined with lifting final_lr to 0.3 instead of 0.15.
-(~1s) A new optimizer, Anvil2, which expands muon via a second tracked momentum buffer, improves the ortho coefficients, and modifies the cautious weight decay application.
-A couple additional dynamic skip connections in the network.
The most striking consequence of the ‘flop aware paradigm’ is you can grow parameters arbitrarily large,
only limited by the available memory, since you can selectively choose how to expend flops on those parameters on each step. NanoGPT has kept active parameters below 124M, but total is unbounded, and has grown to 640M through embedding sparsity over the last year. This PR takes that to its logical conclusion on the 8xH100, scaling up to 65B sparse embedding parameters, which accounts for 25% of the PR’s gains. At frontier scale, where one is not bounded by an 8xH100, one could imagine where this paradigm could lead.
github.com/KellerJordan/modd…
As this was a very notable PR, I spoke with Deven for an hour to learn how he did it. Here’s his story on the changes: hyperstition.cc/training-nan…
aalquwayfili retweeted
نشكر @forbiddenkeygen على رابع مساهمة: اكتشاف وإصلاح حيلة تتيح للمستخدم تجاوز حد معدل الطلبات (rate-limit bypass)
github.com/raaqimorg/qafiyah
When several people talk at once, a transcript can get messy fast.
Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
aalquwayfili retweeted
ندور ناس معضلة!
قاعدين نكبر في sarj.ai بشكل كبير وندور ناس رهيبة تشاركنا الرحلة، ندور عن رهيبين بالمنتج والهندسة والمبيعات والعمليات وووو
راح اشارك معكم الايام الجاية اشياء اكثر عننا!!
jobs.ashbyhq.com/sarjai
Announcing Discovery Loop!
I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
♾
Learn more at: discoveryloop.com
aalquwayfili retweeted
eval for video models: generate an entire new Lord of the Rings movie from scratch
call that next-Tolkien prediction






















