End-to-end private inference and confidential GPU VMs
San Francisco
Joined September 2026
- Tweets21
- Following0
- Followers60
- Likes26
1/ Breakthrough: First production KV cache offloading implementation under confidential computing (CC).
Same gain as without CC.
NVIDIA B300s, CC on:
throughput: +141%
time to first token: 39 s → 1.5 s
GPU to host copies: 4 GB/s → 50 GB/s
4/ Running in production on MiniMax M3.
3.15x the output once GPU memory fills.
95% of prompt tokens served from cache, against 39% without offloading.
5/ Confidential AI no longer means giving up performance.
Full write-up by @b0xtch:
confidential.ai/blog/kv-offl…
Confidential Kubernetes (C8s) just reached 100 GitHub stars.
Next target: 1000 stars
github.com/confidential-dot-…
Confidential AI retweeted
We wrote up how c8s secures the network inside a confidential Kubernetes cluster. Short version: a pod only gets a certificate after hardware attestation, only attested peers can connect, and certificates expire within hours.
confidential.ai/blog/c8s-net…
Confidential Kubernetes (c8s): v0.12 to v0.33 in three weeks. Some highlights:
- env and mounts pinned at admission
- GPUs fail closed unless in CC mode
- post-quantum attested channels
- one CVM per k8s node
- sealed nodes, no shell into running workloads
github.com/confidential-dot-…
We open-sourced confidential-inference.
The code behind api.confidential.ai: gateway, serving images, allowlist.
Every production deploy ships as a signed release bundle. Read the code, check the digests, verify the cluster.
github.com/confidential-dot-…
We're now listed on confidentialinference.net. Thanks @aaaljaz.
Nice detail: it shows whether each provider is post-quantum secure (which we are).
Open-weight models hit 78.4% of token volume on Vercel's AI Gateway last week, up from about 30% 3 months ago.
They are flying.
They are also very complementary to confidential inference architectures, because they simplify the trust assumptions.
In short, anything that is public and open is much easier to verify than stuff which is proprietary and closed.
Looks like today may be a record day for token volume % of open models on Vercel AI Gateway:
🟦 Open 78.4% 🟨 Closed 21.6%
While spend 💲 usually tells a different story, #3 and #4 today are Moonshot AI & DeepSeek. Adding Z.ai, their combined spend surpasses OpenAI (#2).
(Do note that's the spend for inference of the model across providers (mostly in the US), not revenue going directly to the open weight labs.)
Confidential AI retweeted
We're hiring at Confidential AI:
🏗️ AI Infrastructure Engineer
⚡ AI Inference Engineer
🔐 Low-Level Systems & Security Engineer
We're building end-to-end privacy for inference.
confidential.ai/careers
The report is generated for your session. report_data binds the attestation to a nonce your browser just sent, so what you see was produced now, for you. Not cached, not replayed.
Same check every c8s client runs before it sends a sensitive byte.
confidential.ai/docs/c8s/con…
You can now verify a live Confidential Kubernetes (c8s) cluster from your browser.
Every check runs in your browser tab.
Attestation chains to Intel's TDX root. Firmware, kernel and rootfs match the audited image. A pinned CA proves it is the expected cluster. Nvidia GPU attestation included too.
confidential-inference-clust…
confOS v0.5.0 is out.
- The minimal, hardened Linux distro for private, verifiable workloads.
- Linux 6.18 LTS
- Read-only runtime filesystem, verified by dm-verity
- CPU and GPU attestation fails closed
- Open source under AGPLv3.
github.com/confidential-dot-…
Confidential AI retweeted
🎉 confidential OS v0.5.0 just released github.com/confidential-dot-…. confOS is the the minimal, hardened linux distro for private, verifiable workloads published by confidential.ai.
- upgraded to linux 6.18 LTS
- hardened runtime filesystem, read-only via dm-verity
- improved cpu and gpu attestation behavior, fail closed
confOS is open source under AGPLv3