AI/AGI will emerge from qubic. LLM/AI/Tech Enthusiast.
Joined December 2019
- Tweets3.8K
- Following1.1K
- Followers368
- Likes13K
The comments are full of people talking about API costs, because few jobs can justify $100K API fees in re-writes. One thing I don’t head many people mention when it comes to Local AI is that you can do jobs that don’t make “economic” sense when measures through an API lense but that cost nothing in electricity.
Anthropic rewrote its Messages API in Rust. Zero Rust devs on the team.
Claude wrote the code. Humans wrote the plan and checked old vs new matched on real traffic. CPU dropped 40-50%.
Pretty sure "never do the big rewrite" is dead advice.
@katelyn_lesse on @latentspacepod
UK is poverty olympics. If you don’t plan on working a job for £2K a month, you are better off leaving for greener pastures that promote growth.
In the developing country of the UK, withdrawing £2.5k of your own money for a holiday is such a comically, laughably large amount of money that the curtain twitchers in the bank fraud teams just assume you’re money laundering and if you don’t itemise everything you intend to spend that money on, they might just decide not to give you it. Your own money.
They definitely aren’t using it properly. It should be the opposite. After touching Opus 5.5 and 4.1-1017 nothing can tell me otherwise.
The stock valuation doesn’t translate to them being liquid. Unless people buy them out of those 2T of stock valuation. Anthropic IPO at $2T will be a flop.
The moment Opus 5.5 quality comes to local AI, I’m done with Anthropic.
We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety assessment of the conversation.
Since we began using self-assessment in June, it has surfaced misuse that our other safeguards weren’t designed to catch.
Because Claude has already read the conversation, self-assessment is very efficient. And unlike classifiers, which target a discrete set of harms, it draws on Claude’s own judgment, so it gets more accurate as our models improve.
Read more on our new Safeguards blog:
safeguards.anthropic.com/202…
Who is he to judge the value of people’s personal setups as wasteful?
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
Open Source will prevail.
@higgsfield_ai sent us a cease and desist last night, asking us to shut down our Open Source Higgsfield challenge.
Our read on why they sent it. When the apps people pay for can be rebuilt in the open in two weeks, that’s an uncomfortable thing for a closed platform to watch.
So here’s us reiterating our promise to every builder in the challenge and community, and we’re keeping it:
• The challenge stays open
• Final submissions close tomorrow, Oct 30
• Winners are announced Nov 6, with prizes paid as promised
You showed up for each other. We’re showing up for you.
If we win, we all win!
Comfy out 🫡
Local Inference is a bad allocation of resources because we aren't paying his company to steals your information, uses it for training and monitoring.
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
Local AI will NEVER be affordable for the average man. To “begin” being competitive you need a 128GB UM or a PC with a good GPU, CPU and lots of RAM. The minimum is $5K to begin being competitive and that won’t do as much as you think.
If you aren’t wealthy enough to buy GPU, you are cooked. Which is about 99.9% of the population.
GPUs and memory prices are at ATH, and I don't see any reason for them to slow down anytime soon. Owning intelligence has become a high luxury.
Those who call AI a bubble really have no idea. The "bubble" will come at some point, but we're nowhere near there.
And all those anti AI people who are "sick of it" are going to be left behind and be sorry later that they missed the train.
You either join it, or others will beat you. SIMPLE.
Evolution at its best.
It’s the end times for personal computing. The pricing and waiting times are out of this world. If you don’t have any hardware yet (or on the way), you better have A LOT of money.
The best part of local AI is running tasksthat are so long, tedious that make no financial sense via API but that save so much time and labour.
The only way touchscreen can ever work on a laptop is woth a 360 hinge and even then, it becomes an oversized, unnecessarily heavy tablet.
Mac Studio drops resembling cloth drops is crazy. If you don’t order within 5 minutes, you will be waiting 3 months minimum.
