
pursuing the adjacent possible for the next era of human-aligned ai. cofounder @latentspaceai 👩🏻💻🇨🇦 @uoftcompsci
San Francisco, CA
Joined June 2009
- Tweets2.1K
- Following439
- Followers1.2K
- Likes829
Pinned Tweet
We’ll be presenting our paper “Low Distortion Block-Resampling with Spatially Stochastic Networks” today at 9pm PST🧩🏰🎨
Joint work w/ @martinarjovsky @darrylbarnhart @iantbd @LatentSpaceAI!
➡️ paper: arxiv.org/abs/2006.05394
➡️ poster: neurips.gather.town/app/pl8s…
#NeurIPS2020
sarah hong ✨ retweeted
Thousands of startups and small businesses are at risk due to the SVB failure. We ask for depositors to be made whole, and for regulation to prevent this catastrophe.
650 founders who employ more than 22,000 people, have already signed.
Please spread the word.
ycombinator.com/blog/urgent-…
sarah hong ✨ retweeted
THRILLED to share that ChatGPT, our new model optimized for dialog, is now public, free, and accessible to everyone. chat.openai.com/
Try talking with ChatGPT, our new AI system which is optimized for dialogue. Your feedback will help us improve it. openai.com/blog/chatgpt/
sarah hong ✨ retweeted
So excited to finally open up access to these models! Couldn't have asked for a better team to do this with: @stephenroller, @NamanGoyal21 (@myleott + @sam_shleifer at the start too)!
Today Meta AI is sharing OPT-175B, the first 175-billion-parameter language model to be made available to the broader AI research community. OPT-175B can generate creative text on a vast range of topics. Learn more & request access: ai.facebook.com/blog/democra…
sarah hong ✨ retweeted
Human's won't be obsoleted by AGI, we just need better tooling to stay on top.
super excited for this 🎉
watch @DarrylBarnhart's presentation @NVIDIAGTC if you're interested in learning about our approach for scaling LLM with retrieval and appreciate a good library of babel reference 📚🐝🌎
Catch @DarrylBarnhart speak @NVIDIAGTC today about our exciting work @LatentSpaceAI on scaling LLM with retrieval! 📚
There’ll be an open QA in 30 mins: reg.rainfocus.com/flow/nvidi…
sarah hong ✨ retweeted
As we kick off GR13, we’re especially proud to announce:
A 🇺🇦Support for Ukraine🇺🇦 round
We’ve raised $700K to go towards humanitarian aid for Ukrainians
To learn more and get involved, read this 🧵
Me: The greatest love story: Alice & Bob… across time, for all time
Him: Not for all time. It’s until the second law of thermodynamics defeats them — the last lovers — and dissipates them and all their messages into the uniform distribution — the maximum entropy state.
Me: 🙃
sarah hong ✨ retweeted
Here’s another memetic isomorphism:
Crypto is more like physics, and AI is more like biology – not in “biologically inspired” sense, but in the nature of possible solutions.
So using the wordcel / rotator basis, Darwin is a wordcel and Newton is the ultimate rotator. (1/4)
Replying to @VitalikButerin
16. It's fascinating how the two axes seem to imply each other:
Pure tinkering is fast and effective, but it is unprincipled, so lacks large-scale legitimacy, so can only cooperate through centralization
Theorycel thinking is slower, but well-suited for decentralization
sarah hong ✨ retweeted
New blog post: diffusion models are autoencoders!
benanne.github.io/2022/01/31…
I take a closer look at this connection, inspired by the work of @YSongStanford, @StefanoErmon, @kjgeras, @RandomlyWalking, @gyomalin_ML and @jaschasd, among others!
a response from CPT author re eval, emphasizing that CPT is one model vs many, fine-tuned specific ones (similar to GPT3 vs task-specific models):
teapaw.org/arvind_io/status/14871…
I do wish there were better ways to evaluate real-world settings! any public datasets / benchmarks? 🤔
4/n
everything @alfcnz does w @PyTorch, esp visualizations as a tool for deeper understanding is mesmerizing -- they're like works of art 🎨👀
watch his vids and you'll see what I mean.
start here ➡️ @NYUDataScience Deep Learning course with @ylecun!
youtube.com/watch?v=mTtDfKgL…
3/n
the forums should be your first go-to place when you run into bugs.
it is also where @ptrblck_de lives, the man who is single-handedly solving all your @pytorch woes.
but srsly, he puts a lot of care into helping the community out. thank you! 🥰
teapaw.org/ptrblck_de/status/1221…
4/n
I just posted my 10,000th reply in the @PyTorch discuss forum! Thanks everyone for creating such a great community, for the guidance and mentorship I received, and @soumithchintala for starting this journey.
TaskMoE is an exciting direction for MoEs! They route data to experts based on task (compared to tokens).
This enables the ability to extract task-specific subnetworks such that ✨inference << training✨ params; it’s like distillation without actually distilling.
MoE models can be a future for large language models, but having to distill into small models can be painful. Our work, task-level MoE for extracting subnetworks, is a simple idea to bypass distillation. And we're still at the very beginning :)
Blog: ai.googleblog.com/2022/01/le…
ERNIE-ViLG: @BaiduResearch image ↔ text model
• params: 10B
• data: 145M (Chinese) image-text pairs
• training: PaddlePaddle platform (see sec 3.2.2)
• result: achieves a zero-shot FID of 14.7 (for ref, GLIDE from @OpenAI is 12.89; ⬇️ is better)
teapaw.org/BaiduResearch/status/1…
Introducing ERNIE-ViLG, a pre-training framework for bidirectional image-text generation. We trained a 10-billion parameter ERNIE-ViLG model and obtained SOTA on both text-image and image-text synthesis tasks. Read the paper for more: arxiv.org/pdf/2112.15283
1
2
This vid series from @AnthropicAI focused on interpretability of transformers, "reverse-engineering" them so to speak, is superb.
Appreciate seeing @catherineols, @nelhage, and @ch402's evolving thoughts (and their positivity is very contagious)! 😻
youtube.com/playlist?list=PL…
6






















