Pinned Tweet
Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking
arxiv.org/abs/2403.09629
It trains language models to generate rationales at each token to explain future text, to improve their predictions.
🧵👇
Daily AI Papers retweeted
As an ML engineer, implementation >>>>everything.
Knowing is theory. Implementation is understanding.
Few outstanding topics it has:
1. Reinforcement Learning - ppo, dqn
2. Transformer - classical to Retro, switch, gpt models
3. Diffusion models - stable, DDPM, DDIM, UNET
4. GANs - cycle, wasserstein, stylegan & few more
5. Graph neural networks - GAT, GATv2
Skip the tutorial hell & learn about various models
Learn implementations in this GitHub repo.
I’ll share more resources later. Link in comments 👇
Daily AI Papers retweeted
We've open-sourced our internal AI coding IDE.
We built this IDE to help with coding and to experiment with custom AI workflows.
It's based on a flexible extension system, making it easy to develop, test, and tweak new ideas quickly. Each extension is a Python script that runs locally.
(Links in replies)
🧶👇
Daily AI Papers retweeted
GEPA appears to be an effective method for enhancing LLM performance, requiring significantly fewer rollouts than reinforcement learning (RL).
It maintains a pool of system prompts. It uses an the LLM to improve them by reflecting on the generated answers and the scores/feedback for a minibatch of problems.
GEPA keeps the Pareto frontier of system prompts. That is, if system prompt A performs worse on every validation problem compared to system prompt B, A is filtered out from the pool.
👇
Daily AI Papers retweeted
Added the JAX transformer model to annotated paper implementations project.
xitter.cf/vpj/status/14325860709…
Link 👇
Coded a transformer model in JAX from scratch. This was my first time with JAX so it might have mistakes.
lit.labml.ai/github/vpj/jax_…
This doesn't using any high-level frameworks such as Flax.
🧵👇
Daily AI Papers retweeted
Wrote an annotated Triton implementation of Flash Attention 2. (Links in reply)
This is based on the flash attention implementation by the Triton team. Changed it to support GQA and cleaned up a little bit.
Check it out to read the code for forward and backward passes along with the math and derivations. Hope this helps understand transformer attention and flash attention better.
There's about 60 more annotated deep learning paper implementations on this website.
ALT The annotated code. Click on math symbols or identifiers to highlight them.
Daily AI Papers retweeted
The new training also improved GPQA from 64.2% to 67.3% and MMLU Pro from 64.2% to 67.3%.
This model was also trained with the same reasoning datasets we used to train the v1.0 model. We mixed more general instruction data with answers sampled from the Mistral-Small-24B-Instruct-2501 model during the SFT to reduce the degradation of IFEval, which seems to have resulted in generalization of reasoning to non math and coding problems.
The datasets and the models are available on @huggingface.
Follow @notbadai for updates.
Daily AI Papers retweeted
Uploaded the dataset of 270k math reasoning samples that we used to finetune Notbad v1.0 Mistral 24B (MATH-500=77.52% GSM8k Platinum=97.55%) to @huggingface (link in reply)
Follow @notbadai for updates
Daily AI Papers retweeted
We're open-sourcing a math reasoning dataset with 270k samples, generated by our RL-based self-improved Mistral 24B 2501 model and used to train Notbad v1.0 Mistral 24B.
Available on Hugging Face: huggingface.co/datasets/notb…
Daily AI Papers retweeted
You can now download the Notbad v1.0 Mistral 24B model from @huggingface
huggingface.co/notbadai/notb…
Try it on chat.labml.ai
Daily AI Papers retweeted
📢 We are excited to announce Notbad v1.0 Mistral 24B, a new reasoning model trained in math and Python coding. This model is built upon the @MistralAI Small 24B 2501 and has been further trained with reinforcement learning on math and coding.
Daily AI Papers retweeted
Annotated @PyTorch implementation of of LoRA (Low Lank Adaptation of LLMs)
📝 Code + Notes: nn.labml.ai/lora/index.html
📎 Paper: arxiv.org/abs/2106.09685
LoRA freezes the pre-trained model and trains smaller injected weights, enabling faster and memory efficient fine-tuning.
👇
Daily AI Papers retweeted
I first found plotting the distribution useful when I was trying RL algorithms on Atari around 2018/19. I used Tensorboard back then. It was quite useful to look at the score distribution of the rollouts. It showed how the policy was behaving clearer than looking at the mean score.
We later implemented it on top of @vega_vis Altair library because it gave a lot more features (compare multiple distributions, zoom) than Tensorboard.
Daily AI Papers retweeted
We wrote up some of the best practices we feel are useful for ML projects
github.com/labmlai/labml/blo…
Here's a summary 🧵👇
