Pinned Tweet
We started Seldon less than 3 months ago as an API gateway, then a compiler to help make LLM powered applications run cheaper, faster and more reliably.
We have seen a 300+% MoM user growth in that period with many interesting client conversations that have eventually led to our expanded focus on this concept of Semantic Compute: helping existing AI apps run repeated work across code, specialized models (Jev, GLiNER), and LLMs.
We’re still finding the right developer experience, full details to be revealed soon.
What have you tried to move off an LLM?
Pretty sure this is ragebait
a technical hire at a startup should meet at least one:
1. pre-LLM coding experience, built systems from scratch for fun
2. olympiad or equivalent prowess
aka probably not the CS student begging you for a job, not someone whose entire portfolio was built w claude code, not a "non-technical turned technical"
Inspired by @Coinbase blog on their post-training for a fraud model.
The industry reflex is that the function is a prompt, so getting better means new weights. We think that is backwards.
A frontier model is a semantic interpreter. It is the right first implementation of a function, and a poor permanent one. The thing your application should depend on is the function: take a payment and its history, and return a risk.
Seldon compiles that function. It searches other implementations, including a specialist model, a smaller LLM, and ordinary code. It measures cost, quality, and latency. You activate the revision that fits the business. The interface stays the same. There is no task-specific checkpoint to train or operate.
That is why function compilation is the better default than fine-tuning. You specialize the program. You keep the contract. You ship a measured tradeoff, or you refuse it.
On a synthetic benchmark of 4,500 payment-risk cases, with no custom training:
Against Luna, inference cost was 27 times lower, about 5.5 cents versus 1.47 dollars per 1,000 cases. F1 was 10.12 percentage points higher. P95 latency was 33 percent lower.
Against Sol, inference cost was about 124 times lower.
Precision went the other way: 21.64 percent for Seldon, 34.79 percent for Luna, and 65.17 percent for Sol. We required Sol's precision. Nothing qualified, so we activated nothing.
Train weights when the measured gap is still worth owning a model.
Announcing the launch of our open beta. $10 credits for new sign-ups (it will last longer than you think given how economically these compiled functions can run)
Seldon: seldon-ai.com
Blogpost: seldon-ai.com/blog/before-yo…
Tell me you don't know what a MLE does without telling me...
I’m skeptical that the current wave of continual learning startups will work for enterprises.
The premise sounds great: AI models that continuously learn from production interactions and get better over time.
But enterprise workflows aren't static. Business rules change, exceptions multiply, and yesterday's correct answer can become today's compliance violation.
Most production interactions don't provide reliable learning signals. And updating model weights introduces a new problem: how do you prove that improving one behavior hasn't broken ten others?
For many enterprise workflows, retraining the model is an unnecessarily expensive and unpredictable way to update business logic.
I suspect a lot of what we're calling "continual learning" will/should eventually look more like automated software engineering than model training.
LLMs are overpowered, slow, expensive (even with falling token prices) and awkward for many of their current usages in application backends.
Real world software requires predictable costs, performance, and latency/throughput. Achieving these objectives starts from the building blocks, and this is what we are building in Seldon.
Major release coming soon!
We started Seldon less than 3 months ago as an API gateway, then a compiler to help make LLM powered applications run cheaper, faster and more reliably.
We have seen a 300+% MoM user growth in that period with many interesting client conversations that have eventually led to our expanded focus on this concept of Semantic Compute: helping existing AI apps run repeated work across code, specialized models (Jev, GLiNER), and LLMs.
We’re still finding the right developer experience, full details to be revealed soon.
What have you tried to move off an LLM?
Overheard in SF:
"Triple three, double two is dead. Now if a startup is not doing at least 5x YoY you don't matter to investors"
One of the most bullish part of @AnthropicAI freshly released S-1 amid rising compute squeeze and prices. Wonder very much how it compares to @OpenAI's
Deep Insight Labs retweeted
Here's a serious and thorough post about the state of affairs at @Kimi_Moonshot and more importantly the possible causes. This is a long post, built on more than a decade of experience.
I have been roasting them for the past two days, but a month ago I also defended them when Anthropic said they routed Kimi user requests to Opus - see my Articles.
First, let's start with an overview of what Moonshot has done to its users over the past 6 months:
- Cut Kimi plan quotas so bad after the K2.7 and K3 releases that they're all unusable and basically scams. OpenAI's cuts are bad; Kimi's are twice as bad
- Released K3 in July with a campaign of unmarked, sponsored influencer post with faked videos of what their model had supposedly produced - I called it out then, too
- Price fixed Kimi K3: inference providers in the US and EU are not allowed to set their own prices but must follow the minimum prices set by Moonshot. This is illegal in every functioning country; Apple got convicted for this with their ebook pricing, remember?
- Stole the open source code from Cua, relabelled it and claimed they had invented computer use in their Kimi agent
- Released K3.1 with another unmarked influencer campaign, with more fake videos
- Tampered with K3.1 benchmark scores for DeepSeek in their release materials
When you look at the above, you see that these are pretty serious things that could break any brand. If you do this, you either have to think your users are stupid, or not care, or be stupid yourself.
When it comes international brands selling into China, and Chinese brands selling to international markets, they both exhibit all three characteristics.
I've heard to many Europeans talk about their business ideas: just pick a product, slap a European-brand on it, talk about it being from whichever country, and then the Chinese customers will just buy it without thought.
This never works, but these entrepreneurs are ignorant, have no will to learn the market, and regard their potential customer base as NPCs. They view China as another world and cannot reason about it, and because they cannot reason about China, they cannot treat it seriously.
For this reason, there have been numerous advertising scandals with European brands getting slaughtered online, because their leadership in HQ outside of China made tone dead decisions in their communication. D&G, Prada, Nike, adidas, Arcteryx, there's been too many internet campaigns against brands for their ad campaigns to remember.
For most Chinese companies trying to go international, it is the same but in reverse. They do not understand the international markets, they don't care to educate themselves, they see the world outside China as something otherly, and feel shielded from any negative repercussions from far away countries. They see their users as NPCs.
If you wanted to get a refund for your Kimi coding plan, what can you really do? Most likely nothing, and your X posts won't get noticed.
When it comes to the campaign for the K3.1 launch, Moonshot did a number of things that just spell incompetence:
- Create ready made videos for publishing by influencers, this is basically unheard of
- Try to hide that posts are sponsored, to generate the impression of organic user feedback
- Require approval before publication
When you do these kinds of campaigns, you don't ask for approval because you need to trust the creator; otherwise the content suffers.
You don't try to hide that videos are sponsored, because the potential backlash for this type of manipulation is 100x worse than the value, as we see now.
And well, creating fake videos.... 无语。
Some have speculated that the error is on the side of a marketing agency, but that is unlikely. The brief is always created by the company for the agency, and everything that goes out is approved by the customer.
Most likely, the marketing team at Moonshot must have said that doing hidden influence campaigns like this is risky, but could have been forced by leadership who wanted to "ensure ROI" on their marketing investment by controlling content.
So what it really comes down to is a company leadership that does not know or care about laws in other markets or does not care about them, feel like this type of manipulation is worth the risk, and see users as NPCs.
At the end of the day, this is false advertising and when you combine this with Moonshot's price fixing, it is not just that it is dishonest but it is complete and utter disrespect of laws of other countries - mind you, all of this is certainly illegal in China, too.
To close this out, what really needs to happen now is that the price fixing is reported and investigated. Inference providers are clearly not able to push back on Moonshot themselves, as they're in a competition to be able to offer the newest hyped model as soon as possible.
Don't ban the models, but hit down hard on illegal business practices by the company.
Looking at the various private credit funds financing the riskiest, least-transparent slices of the AI buildout.
Many third rate managers have herded into and underpriced the structural risks while concentrating heavily.
Replying to @chamath
Pretty much.
As of the past few weeks, debt markets have almost entirely shut off access to money for any company that doesn’t have substantial revenue independent of AI data centers.
SpaceX closed $40B of debt for GPUs, but we’re investment grade and have Starlink & launch revenue.
That means the 200+ neoclouds are going to see guppy-level death and consolidation. Some will make it through, but not many.
Probably the best shio pan in the entire state of California
#SanFrancisco
God damn....I LOL-ed
Replying to @TokenGremlin
Worst is subjective. Both face criticisms around safety incidents, culture, and practices from various sources. Removed both.
Between potential margins compression for inference (main revenue stream for @OpenAI and @AnthropicAI) due to declining token costs and rising compute costs,
I would definitely take note of this as well.
Long game for tech has always been the broader non-tech market and that market is not as bullish as people in the AI bubble feels.
apollo.com/wealth/insights-n…
Most VCs are unqualified to give advice to founders.
Was at this startup event late last year when me and my friends encountered this VC who started lecturing us about our ideas and why they lack differentiation and a moat.
Realized today that the VC has since left to do a company brain startup and appears to have no significant traction for the last 9 month. 🙄
Very much agree with this view. LLMs and their post-training today are very much focused on convergent thinking (i.e narrow down possibilities towards the most probably answer).
That is useful for many basic mechanical steps of any research process (finance or otherwise) like retrieval, highlighting relevant information, etc.
Good research however, often requires a certain out of the box thinking (i.e divergent thinking) to come up with good hypotheses. LLMs are simply not good at this out of the box.
It's interesting to think about what is the most practical way to overcome this limitation. Should it be framed as a post-training issue? A harness issue? Or perhaps a process/user issue?
As LLMs become the default first step in investment research, consensus bias is going to get stronger, not weaker.
The models are trained on, and now retrieve, whatever has already been published about a stock. Ask one for a view and you get the weighted average of the sell-side, the press and the message boards, delivered with more confidence than any of those sources had on their own. The model doesn’t add information. It adds conviction to the default view.
Consensus drives short-term price action. Cash flows set terminal prices. So if LLMs compress the dispersion of views, two things follow for anyone running a non-consensus position. First, fewer people will be positioned on your side, so the pain before the thesis plays out gets longer and deeper than it was in the pre-LLM world. Second, when the cash flows arrive and the consensus has to move, it moves with fewer hands already on the other side. The snap-back is more violent.
I see the mechanism every day. When I use LLMs for stock research, the first 90-95% of the background work is largely right. The last 5% is where it goes wrong, and it goes wrong in a specific way. The published view almost always runs on a default: the market assumes X because that’s how it usually works. The model has seen a thousand cases where it worked that way, so it hands you the default with full confidence. The alpha is in the cases where this particular company, contract or regulatory path is different, and a tool built to return the typical answer is the last place that difference will show up. I spend hours pulling raw filings, checking numbers and fighting with the model to get it to reason about the specific case instead of the usual one.
Most investors won’t do that. They’ll stop at 95%, holding the default assumption with more confidence than they had before. Consensus plus conviction is the most expensive combination in markets, and I think it’s where the next few years of mispricings will come from.
Still trying to digest this. But this sounds big. Programming may really be solved if formal verification works at scale.
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
How is Kimi 3.1 on financial research? Anybody tested it yet? Don't tell me about the benchmark results. Can't trust benchmarks these days.













