🇨🇦 Backend Brat. Distributed Diva. Relentless Learner.

Canada
Joined June 2009
It’s important to fact check loud voices cuz there’s a lot of followers in this world. People who idolize you and your advice. If someone’s promoting a way of working just to push a narrative that doesn’t actually work and people apply this advice today they can lose their jobs
8
19
3
250
12,561
Kelly Sommers retweeted
Thank you for making TLA+ a useful skill for a broad community @poteto ! It's great to see the impact of the skills. Post more! TLA+ has a deep stack. Consider using TLC, Apalache, and TLAPS. For example, we can formally prove (or disprove) the correctness of TLA+ specs lifted from code; agents make it incredibly automatic. We can also check model-code conformance (to make sure the model faithfully describes code). The #Specula project has a set of skills you might find useful, github.com/specula-org/Specu…
there's something really cool about asking your agent to draw a statechart (hypothetical example in this image), then using the /create-formal-verification-skill you made with pstack to find bugs through something like TLA+. you can both understand your code better and how the user flow should work, and fix issues users would otherwise see
2
15
2
147
8,934
Kelly Sommers retweeted
It's great to see more and more folks starting to use TLA+. @lemmster
we've been trying this approach of skill based generating TLA+ at @datadoghq for quite some time. wrote a few blogs on it. some of my team affectionately made a sesh-mode skill in one of our oss repos. Added all links in comment. We now have something more advanced, will share soon. a meta-skill that generates the whole suite is a great idea. antithesis skills do something similar for DST, research properties > workload > triage
1
3
8
1,539
Kelly Sommers retweeted
Before blaming S3 for slow writes, benchmark your WAL. We used LAION to benchmark storing multimodal data with a WAL. Lance writes blobs directly using Lance Blob v2, so SSTable flushes contain only metadata. At concurrency 32, Lance delivered 11.2% more write throughput than SlateDB while persisting 48.2% fewer bytes on S3. na2.hubs.ly/H08wr9H0
1
2
25
2,562
Kelly Sommers retweeted
Datadog are doing very cool stuff in this space, worth a follow!
we've been trying this approach of skill based generating TLA+ at @datadoghq for quite some time. wrote a few blogs on it. some of my team affectionately made a sesh-mode skill in one of our oss repos. Added all links in comment. We now have something more advanced, will share soon. a meta-skill that generates the whole suite is a great idea. antithesis skills do something similar for DST, research properties > workload > triage
4
56
3,936
Kelly Sommers retweeted
No repo left behind. We’ve replicated 99.99999% of our Git repositories on Azure. The last 0.00001% are our busiest. This new blog post explores how changing the way our quorum serializes writes opens the door to replicas that keep no persistent state of their own. github.blog/engineering/arch…
2
15
1
118
13,463
Wow ChatGPT has started confusing itself between projects. I asked it to do a merge conflict and it reached outside the current project path and did the merge in a totally different checkout the project wasn’t based on.
8
13
2,207
Kelly Sommers retweeted
the compression time is actually significant portion of the request/response time. this implies that the coding harnesses should be compressing up to the previous message in the background/caching and then completing it when the new message comes in.
openai uses zstd which might be a little better
2
3
18
2,311
Kelly Sommers retweeted
I've uploaded a video of my "Vacuuming Large Tables" talk to youtube.com/watch?v=ylqJ5CY2…. I did that talk a bunch of times but I think this was the only recording. If you're coming to my REPACK talk next week at PGConf EU, bonus points if you watch this one first. #postgres
1
4
1,190
Kelly Sommers retweeted
Our CEO, Max, officially announced that TiDB X will soon be open source.
🤖 Made with AI
3
6
29
2,345
Kelly Sommers retweeted
"In scale testing, one cluster ran over 6,700 pods across more than 100 node pools, with the busiest nodes holding 15 blocks (240 IPs) each." planetscale.com/blog/planets…
4
2
65
5,512
[new blog post] Atlas Infinite: MongoDB's Disaggregated Architecture muratbuffalo.blogspot.com/20…
16
74
6,816
Kelly Sommers retweeted
Today we're open-sourcing pg_vectorsearch, a new vector search extension for PostgreSQL. The performance improvements over HNSW are pretty remarkable. At 100 million vectors: ◆︎ Faster queries, 4× less RAM: 6 ms vs. 8 ms at 95% recall, using 128 GB vs. 512 GB RAM. HNSW couldn't even build on 128 GB. ◆︎ 7× smaller indexes: 25 GB vs. 180 GB at 100M vectors. ◆︎ 50× faster index builds: 49 minutes vs. 42 hours on identical hardware. And at 1 million vectors, PRISM delivers 15 ms queries at 99% recall on just 2 GB RAM, versus HNSW's 392 ms on 4 GB. What leads to these improvements? Rather than traversing an HNSW graph with random memory accesses, our new PRISM index partitions vectors based on distance, compresses them using RaBitQ, and scans them efficiently using SIMD vectorization techniques. This takes better advantage of modern CPUs and memory locality. This is actually our second open-source vector index. We learned a lot from building DiskANN-based pgvectorscale, including some important limitations of graph-based approaches. Those lessons convinced us there was a better architectural foundation, so we started over. pg_vectorsearch is still in alpha, with more engineering ahead. But we believe fast, scalable vector search should live alongside your relational data, without requiring a separate vector database. Released under the permissive PostgreSQL License, just like our pg_textsearch extension for BM25 keyword search. Together, they provide the building blocks for hybrid vector + keyword search in PostgreSQL. Feedback and contributions welcome! Code and blog post below. ⚡️⚡️⚡️
38
99
12
859
39,474
🔥 I’ve seen a lot of teams struggle with expensive and slow MSSQL offerings on Azure so this will be fire.
4
2
82
11,026
Accord is pretty cool!
Fedor Ryabinin getting into the depths of the Accord protocol, the first leaderless protocol deployed in a widely used production systems.
1
16
3,370
Kelly Sommers retweeted
AWS's own docs say gp3 volumes can drop below 90% of provisioned IOPS 1% of the time. That's 86 hours a year, per volume. Spread a database across 768 of them and there's a 99.65% chance one is hurting production right now. planetscale.com/blog/the-rea…
7
8
2
155
21,903
Sprinkle some perf slang in your chats and watch the toaster cook
2
1
46
4,728
Posting throughput only benchmarks is amateur hour.
12
5
1
106
5,823
In case someone’s curious where I left off with my fork of the original Rails code I fixed some sloppy cache code that hits the DB on every cache hit (yes really) Pretty confident most of the issues are just app code and you could get it down even more.
11
9
10
250
28,019
This is how real experience speaks. Not pushing a narrative even though there’s an opportunity to make yourself look great with a hot take. A measured and responsible answer.
my truthful answer is that i would never presume what the problems are. i have not seen the inside of github's architecture since 2019. i know they have good folks working there. i know that type of scale coming so quickly is very hard to react to when you run at the scale GiHub runs at. nothing is really that elastic, and orders of magnitudes more traffic showing up very quickly is hard to get in front of. I've worked at extreme scale before and seen companies spend three months just preparing for events that will increase their traffic by 10% or 20% percent, which is nothing compared to what GitHub's facing now.
5
12
373
15,502