rockafeller

The software engineering behind the X "algorithm" is epic. It has to distill 500 million daily tweets down to a personalized handful it shows each person. This algorithm runs ~57,000 times a second 🤯 ... and each execution takes 220 seconds CPU time, but runs in 1.5 seconds real time on average, which means compute is distributed across 12+ million CPU cores working continuously. Each step of recommendation algorithm pipeline is fascinating: maintaining real-time interaction graph, embedding spaces for clustering interests, ranking via 48M param neural net, etc. I love that it's mostly been made open source, and hope more companies do the same!

"An eye for an eye makes the whole world blind." - Gandhi