Yoray Herzberg retweeted
The core operation in most Mixture-of-Experts (MoE) workloads is applying each expert to all of the tokens it gets assigned. The naive way to implement this would be as a series of matrix multiplications: compute the output of expert 1, then expert 2, and so on. For this purpose, most implementations use a specialized *Grouped GEMM* kernel, which fuses all of these multiplications together, computing the outputs of all experts in parallel, with a single launch.
We’ve just finished optimizing our own implementation. The resulting kernel is faster than all other implementations we tested, and in some cases has a 15% speedup over the best-performing baseline we found (quack), mostly in small decode shapes.
Link to full blog in the comments.
We developed the fastest FP8 Grouped GEMM on B200! Check it out: pearlresearch.ai/research/bl…. Source code coming soon...
A fun weekend project: implementing Schnorr's Protocol in Rust!
How can we create a ZK proof for knowledge of a solution to the discrete log problem? And can we make such a proof completely non-interactive?
vaktibabat.github.io/posts/s…
Ever wondered how cardinalities of multi-billion element datasets can be estimated with only a negligible constant amount of memory?
Merkle Trees let a Prover prove the inclusion of some item in a dataset with logarithmic verification time. Moreover, they only require the verifier to have access to a constant amount of bytes!
Check out my post implementing them in Go here: vaktibabat.github.io/posts/m…
In my new post, we use Rust to implement a garbled circuits-based algorithm that solves the millionaires' problem from scratch! This algorithm allows two parties to evaluate a multivariate function privately. Check it out here!
vaktibabat.github.io/posts/s…
#cryptography #rust
In my new post, we use CUDA and Rust to write a GPU-accelerated MD5 hash cracker, implementing all of the algorithms from scratch. Check it out here!
vaktibabat.github.io/posts/c…
#cryptography #cuda #rust #md5 #gpu
Ant-colony optimization is a family of optimization algorithms designed to approximate solutions to NP-hard problems. In this post, we implement the Ant System (AS) algorithm in #golang, and use it to approximate solutions to the TSP
vaktibabat.github.io/posts/a…
During the past couple of weeks, I've participated in the #flareon11 reversing CTF by Mandiant (part of Google Cloud)! Here are my writeups for the 8/10 challenges I solved:
vaktibabat.github.io/posts/f…
#reversing #security #cryptography #ctf
I've uploaded a new post about implementing an Elliptic Curve-based Secure Chat, using Rust and Protobufs!
vaktibabat.github.io/posts/e…
#cryptography #ecc #rust #cybersecurity #protobuf #crypto
In my new post, we deploy a digit-recognition Neural Network (trained from scratch in Rust in a previous post of mine: vaktibabat.github.io/posts/R…) with Rust and WebAssembly! vaktibabat.github.io/posts/r…
#rust #wasm #programming #ai
During the past couple of days, I've participated in idekCTF 2024, which had many interesting challenges! Here are my writeups for the tasks I solved (2 web, 1 rev, and 1 crypto) :)
vaktibabat.github.io/posts/i…
#ctf #security #programming
In my latest post, we implement a Deep-Q Network from scratch (in Rust), and train it for 10,000 games on Blackjack!
We then run some interesting experiments on the resulting model, such as comparing it to common Blackjack strategies. Check it out here! vaktibabat.github.io/posts/D…