swe@PingCAP - The company behind TiDB. Oracle/MySQL/InnoDB team lead in a past life
California, USA
Joined April 2012
- Tweets4.8K
- Following261
- Followers5.4K
- Likes4.1K
Sunny Bains @TiDB retweeted
On October 11th, NebuSec responded to a security incident affecting PingCAP’s internal systems. An unauthorized party used a compromised GitHub account and exposed cloud credentials to access PingCAP internal systems.
Throughout our review of PingCAP's internal system, NebuSec found the intrusion was confined to internal source repositories and internal analytics, storage, and operational tools.
NebuSec found no evidence of attacker command execution or persistence, and no evidence that data stored in customer TiDB databases was accessed
Read our report here: nebusec.ai/research/pingcap-…
Sunny Bains @TiDB retweeted
To our customers and partners: I know you have questions about PingCAP’s security incident. I’ve shared what we know, what the independent investigation found, and steps your team can take now. We’re here to help. Read my update:
pingcap.com/trust-hub/securi…
If the B-Link tree uses the LSM as a storage layer then it should be able to use compaction to purge the expired/deleted keys from its leaf pages. The SMO can be deferred to the B-Link tree layer. This should at least get rid of a lot of latching and lookup for full deletes.
Probably try this next weekend. I love these LLMs, so easy to just FAFO.
Yes, and TiDB has been doing this for several years now.
Over the weekend I tried my hand at the B-link tree over LSM. The LLM got it to work, single threaded performance acceptable for inserts, point selects not too shabby.
However, doesn’t scale to two threads, layered unnecessary complexity at the B-Link tree level around MVCC, the design was laid out quite clearly on how to leverage the LSM engine MCCC. Despite clear instructions couldn’t remove the cruft later either. Seems like it will have to be done by hand.
Sunny Bains @TiDB retweeted
メルカリさんのTiDB以降の話で、Auroraとか色んなDBも試したけど最終的にTiDBにした理由とか、今の課題とか色々お話が聞けて面白かった！
飲み屋で聞きたいDB話って感じだったｗ
DBRE必見！ メルカリのTiDB移行裏話 youtu.be/aORW_AlCV-M?si=H7sX… @YouTubeより
I was considering an experiment over the weekend, implement a B-link tree over the LSM engine. Use the same transaction and MVCC infrastructure and avoid latching altogether. Use savepoints for retries. Read only queries work over snapshots over the LSM. The pages can have finer granularity and write only the sub-pages to the LSM that were updated.
Have yet to flesh it out fully but it seems worth trying. If it works and is performant then writing an R-Tree for GiS functionality becomes easier.
Using flags in programming conversations has more or less disappeared, except when describing legacy system interfaces. Now the same bits usually sit behind an enum, a typestate, or a named option, and the design gets described in terms of those states and transitions.
I think that is progress.
Sunny Bains @TiDB retweeted
Tomorrow: live TiDB lab on MySQL fleet consolidation.
Resource Groups, cost modeling, and Plaid's 234-database story.
Sep 24 | 10am PT → pingcap.com/event/tidb-use-c…
This is what the full insert test run graph looks like. I want it to be stable after peak. the drop at 8k connections isn't too bad but it bothers me :-). it's 1.7M/s.
I think a requirement to use specialized malloc libraries is a sign of poorly written software.
Memory management and its performance implications should not be an afterthought.
Weekend hacking.
Replacing Tokio with a custom runtime for a storage engine use case, the db engine can now do 2.1M row insert/s with 10 rows per insert. Stated in a less glamorous way, 210K inserts/s 🙂at 512 concurrent threads.
Very pleased.
Some people like to take a single node database, build their own distributed transaction layer on top. So far so good. Then compare latencies of their single node underlying database with another database that provides strong consistency through distributed transactions and horizontal scalability out of the box.
Sunny Bains @TiDB retweeted
I am giving a keynote next Tuesday Sept 22 at the "Rows & Columns Summit" on the torrid history of hybrid transaction/analytical processing (HTAP) database management systems: Broken Dreams, Broken Promises, Broken Marriages.
rowsandcolumnssummit.com/
This is a very interesting paper: BTRLoG: Low-Latency Logging for Cloud Database Systems
arxiv.org/pdf/2606.27051
This is a great article. I think it’s more general than just research teams.
Every research team runs down some long dark alleys of the soul. Here's Serdar Benderli, our Research team manager, talking about some of the failure modes he's/we've seen -- hopefully this blog post saves yours from a few.
antithesis.com/blog/2026/way…
