aalquwayfili (@forbiddenkeygen) header
aalquwayfili (@forbiddenkeygen) profile picture

security specialist @houseofhades

latent space
Joined June 2015
Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)Photo by aalquwayfili (@forbiddenkeygen)
Lecture on AI at Kyoto University 😊
24
48
10
1,067
57,702
aalquwayfili retweeted
قبل شهر، نشرنا «قافية» على قتهب، وبفضل الله -سبحانه- ثم دعمكم وصلنا إلى: - ١٠٠ نجمة - ٩ مطورين: @0xSm0ky @fhad_________ @forbiddenkeygen @MohannadNaj @bitbyte_log @AlkhazrajyMB @_0xFa @khengari77 @ivbme - ٤ داعمين: @ysralot @forbiddenkeygen @bitbyte_log فالحمد والشكر لله ❤️
إذا كنت مبرمج، أو تعرف مبرمج يبحث عن مشروع يساهم فيه ويتعلم منه، أو كانت عندك فكرة لميزة ودك تضيفها وتساعد بها الناس... فموقع قافية صار مفتوح المصدر! مشروع عام، غير ربحي، من الجميع وللجميع، في خدمة العربية وأهلها مستودع قتهب: github.com/raaqimorg/qafiyah
1
9
44
3,617
aalquwayfili retweeted
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel. Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year. We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
87
106
114
1,265
347,847
aalquwayfili retweeted
We just released Polars 2.0. It removed many of our legacy decisions makes the streaming engine our default and promotes SQL to a first class citizen within Polars. It comes with initial out-of-core (spill to disk) support, a new Map data type and a lot of performance improvements. In fact, we think Polars is now one of the fastest analytical SQL engines on a single node. See benchmarks in the post: pola.rs/posts/release-polars…
16
157
23
1,266
91,411
aalquwayfili retweeted
New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s! This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with incredibly clever engineering and ML judgement. If a flop is low value on a particular step, skip it. Specifically: -(~8s) Sampled softmax. If a token doesn’t appear in a batch, skip its lm_head fwd/bwd some fraction of the time. -Sparse values. Only run an optimizer step for ngram embeddings that occurred in the batch. Set beta1 to zero to enable this. Beta2 is applied retroactively when the row is later used. -Sparse updates. Only update ngram and value embeddings once every 4 steps instead of once every 2. -Sparse communication. Shard the n-gram table across GPUs, and only pass the rows receiving updates on each step. -Sparse optimizer states. For the n-gram table, reduce from 2 floats in Adam optimizer per param, to 1 float per 768 params. -Hand-rolled flash attention for 64 dim heads. There are several additions that add accuracy too: -(~4s) EMA during last 300 steps, combined with lifting final_lr to 0.3 instead of 0.15. -(~1s) A new optimizer, Anvil2, which expands muon via a second tracked momentum buffer, improves the ortho coefficients, and modifies the cautious weight decay application. -A couple additional dynamic skip connections in the network. The most striking consequence of the ‘flop aware paradigm’ is you can grow parameters arbitrarily large, only limited by the available memory, since you can selectively choose how to expend flops on those parameters on each step. NanoGPT has kept active parameters below 124M, but total is unbounded, and has grown to 640M through embedding sparsity over the last year. This PR takes that to its logical conclusion on the 8xH100, scaling up to 65B sparse embedding parameters, which accounts for 25% of the PR’s gains. At frontier scale, where one is not bounded by an 8xH100, one could imagine where this paradigm could lead. github.com/KellerJordan/modd… As this was a very notable PR, I spoke with Deven for an hour to learn how he did it. Here’s his story on the changes: hyperstition.cc/training-nan…
37
156
67
1,474
429,143
aalquwayfili retweeted
نشكر @forbiddenkeygen على رابع مساهمة: اكتشاف وإصلاح حيلة تتيح للمستخدم تجاوز حد معدل الطلبات (rate-limit bypass) github.com/raaqimorg/qafiyah
2
10
1,769
aalquwayfili retweeted
When several people talk at once, a transcript can get messy fast. Our new Nemotron 3 Diarization model tracks who spoke when, even when voices overlap. It handles up to eight speakers, has 100M parameters, and is now available on @huggingface 🤗
204
855
241
9,669
1,070,522
ندور ناس معضلة! قاعدين نكبر في sarj.ai بشكل كبير وندور ناس رهيبة تشاركنا الرحلة، ندور عن رهيبين بالمنتج والهندسة والمبيعات والعمليات وووو راح اشارك معكم الايام الجاية اشياء اكثر عننا!! jobs.ashbyhq.com/sarjai
11
30
2
442
59,602
aalquwayfili retweeted
Announcing Discovery Loop! I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor. ♾ Learn more at: discoveryloop.com
898
2,153
969
21,745
6,774,294
aalquwayfili retweeted
Introducing Claude Fable 5: a Mythos-class model that we’ve made safe for general use. Its capabilities exceed those of any model we’ve ever made generally available.
4,906
14,054
9,953
103,200
57,746,606
eval for video models: generate an entire new Lord of the Rings movie from scratch call that next-Tolkien prediction
10
4
109
7,172
aalquwayfili retweeted
Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&D. I remain deeply passionate about education and plan to resume my work on it in time.
7,849
10,876
7,550
148,685
28,115,839