At the precipice of success Local ai Qwen-Lover AMD Rocker
Joined May 2023
- Tweets2.5K
- Following716
- Followers1.1K
- Likes7.9K
Chromadera retweeted
AIベンチと体感がずれる現象は、ベンチと実際の作業でやらせてることが別物だからに尽きる
おまけに実際の作業も人によってバラバラすぎる
サッカー選手に長距離走やらせたらそれなりにできるだろうけど、長距離走専門には負けるみたいなことは普通に起きる
AA v5 is heavily weighted on Base intelligence a model ships out of the box with. If you really care about not having to tell a model that Biden is no longer , America's president it's a good benchmark. Outside of this, the stronger valuation is now we know the closed source models are significantly larger than their open source counterparts.
AA v5 is heavily weighted on Base intelligence a model ships out of the box with. If you really care about not having to tell a model that Biden is no longer , America's president it's a good benchmark. Outside of this, the stronger valuation is now we know the closed source models are significantly larger than their open source counterparts.
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04
At least we now have a strong hunch that the closed Models are very LARGE. Most of the heavy lifting is done by knowledge base , which is a result of your Params.
Announcing Artificial Analysis Intelligence Index v5, featuring 2 new benchmarks and 3 upgrades. For the first time, the majority of the Index is built on private datasets
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 is our biggest upgrade since v4 launched in January. With v5, 60% of the Index is now based on evaluations with private test sets or held-out answers.
Alongside Intelligence Index v5, we’re launching the Artificial Analysis Progress Index v1, which tracks model capability over time on one continuous scale.
New to the Intelligence Index:
➤ FrontierCode 2.0: Cognition's newly-launched private benchmark of real-world coding tasks, graded on whether a repository maintainer would merge the agent's change
➤ Terminal-Bench-Science: 70 agentic scientific research tasks, run in a terminal environment
Upgraded:
➤ HLE-Diamond (replacing Humanity's Last Exam): a refined 1,000-question version of Humanity's Last Exam, split evenly between reasoning and knowledge questions
➤ AA-Omniscience v1.1: An updated dataset with saturated and public set questions removed, new questions added, and ambiguous or erroneous questions revised
➤ CritPt v2: a private, 40-question hard subset of CritPt v2, revised by the CritPt authors and run with a code-execution tool
Staying in the Index: AA-Briefcase, AutomationBench-AA, GDP.pdf and Terminal-Bench 4.0
Removed from the Index: GDPval-AA v2.1, SciCode, and AA-LCR v1.1
Key results:
➤ Claude Opus 5.5 (max) leads at 58, ahead of GPT-6 Astra (max, 57) and GPT-6.1 Sol (max, 55): Anthropic and OpenAI hold the top five models. Gemini 4 Argon (high) is next at 50
➤ The open weights gap widens from 3 points to 12: Kimi K3.1 Pro (weights not yet released) is the top open weights model at 46. Among released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) lead at 32 - 26 points behind the leader and below Claude Haiku 5.5 (max, 36)
➤ Cost per task varies by more than 13x among the leading models: of the 10 highest-scoring models shown, GPT-6.1 Terra (max) is the cheapest at $0.96 per Intelligence Index task and Claude Fable 5.1 (max) the most expensive at $13.04
Chromadera retweeted
I made MiniMax-M3.1-Flash 5x faster at prefill than llama.cpp on my 4x 3090. Decode up to 4.6x. A 428B model, at home.
Its the Unsloth 2-bit GGUF, 144 GB. My cards have 96 GB of VRAM, so the experts that dont fit run from system RAM.
llama.cpp, best config I found: decode 6.7 tok/s, prefill 219 tok/s at 16K.
My engine: decode 23 to 31 tok/s, prefill 1120 tok/s at 16K and 1525 at 28K.
Nobody gave me this engine. I forked Maya, wrote the MiniMax path myself and keep the hot experts in VRAM with a cache that moves them during the answer. Then I added a DSpark drafter, and then I moved the long KV cache out to system RAM in blocks, so more experts fit on the cards. Quality didnt move at all, NLL is the same to the last digit.
The limit now is RAM. My EPYC runs 2 memory channels out of 8, and cold experts come through the CPU at about 19 GB/s.
1M context, 428B params, 6 year old gaming cards. Show me someone else running this model locally at these numbers. Ill wait.
Thanks @MiniMax_AI for the weights and @UnslothAI for the GGUF.
Make me fried rice.
Model goes ahead to make fried rice...but gets punished for not serving the fried rice, or not adding sweet corn 😳
On our price vs performance evaluation, models cluster into several groups.
▪️ Opus 5.5 and GPT-6 Astra lead in intelligence, at average costs of $12.8 and $6.67, respectively.
▪️ GPT-6.1 Sol leads in cost efficiency, scoring 39% at $1.03 per task.
▪️ GPT-6 Luna and Haiku 5.5 lead on overall cost, scoring above 20% at less than $0.50 per task.
Chinese OSS models lag far behind, with Kimi K3 and GLM 5.3 scoring at 17%. American OSS models like Beam and Inkling are even further behind with a score of 3%.
Chromadera retweeted
Today is my last day at @moniepoint.
I’ve made the decision to dedicate my time fully to building @pewbeam_ai. I believe deeply in what we’re building and its potential to shape the kind of future I want to live in, and I’m excited to give it my undivided attention.
It has been a privilege to contribute to Moniepoint’s mission of providing financial happiness for every African.
As a Product Designer on the Cards team, I had the opportunity to contribute to improving the product’s user experience and design, helping make it more intuitive and useful for the people who rely on it.
I’m grateful to everyone I had the opportunity to work with. I’ve learned a great deal from the people around me, and I know those lessons will serve me well as I enter this next chapter.
Now, it’s time to build 🚀
Chromadera retweeted
Just ran my ISP benchmark on my M5 Max w/ 128GB with Sushi 1.4.0 and here is the result.
87.6k tokens • 41.4 tok/s
Quantized to Hell and Back !!
DeepSeek 4.1F 1017-STQ-1.75bpw is released with Sushi 🍣 1.4.0, its quality is comparable to oQ2.5e & rendered this grim reaper animation.
✅ 1m context
✅ 96GB & 128GB Mac
✅ tok/s - 1,000 prefill, 75 code, 45 prose
🍣👌 github.com/beamivalice/sushi
My favourite small model champion. Try out their new models.
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵
Chromadera retweeted
Today we're coming out of stealth, introducing Neywa Labs 🚀
We built the infrastructure to build and deploy AI that runs on the devices people already own. Private, offline, unlimited.
One mission: make AI available to everyone in the world.
neywalabs.ai/
I decided to look into the potential damage the accusations may have done. I am afraid they are in fact big.
@_Nikeofweb3 any further conversation concerning this should be passed through to my lawyer as I intend to handle this legally.
Thank you to everyone that reached out to me, thank you for your kind gesture.
Chromadera retweeted
2,100+ tokens/sec per user on DeepSeek V4.1 Flash, at full model quality, on 8× AMD Instinct MI355X. 🚀
Congrats to @lithos_ai on pushing the frontier of inference with persistent megakernels on AMD. Open hardware + instruction-level control = room to innovate.
2,100+ tokens per second per user on DeepSeek V4.1 Flash, at full model quality, on eight AMD Instinct MI355X GPUs.
In our benchmarks, the Lithos engine delivers:
→ Up to 3.4× faster inference for DeepSeek V4.1 Flash
→ Up to 2.5× faster inference for Kimi K3
Compared with tested open-source engines on the same eight GPUs.
The key challenge is reducing the coordination tax: time spent waiting between kernels and across GPUs. Our approach combines persistent megakernels, overlapping execution, and communication inside computation.
Our technical deep dive covers:
- Decode speed, cost per token, and latency up to 1M context.
- How Lithos engine reduces launch overhead and overlap computation with communication across GPUs.
- How MI355X’s register capacity and direct instruction-level control enable hardware-specific optimizations.
- What our roofline analysis reveals about opportunities for further speedups.
Read the full analysis: lithosai.com/blog/ultra-fast…
Chromadera retweeted
When you look at the spec of modern GPUs such as Blackwells, you'll see that it has FP4/FP8 tensor cores which older GPUs lacks. This means when we load a NVFP4 model on 3090, it cannot compute in FP4 and needs to go through extra steps to unpack it. So how much does this matter? Been wondering for awhile so put it to test
Bottom line is, negligible when it comes to decode, big impact on prefill. Also matters quite a bit when concurrency gets involved, so keep all this mind when you are selecting your next GPU.
Chromadera retweeted
Project Maya × @OrcaRouter 🐋
OrcaRouter's OrcaCyber DeepSeek-V4.1-Flash-1017, the abliterated model built for cybersecurity research, now runs on Maya.
Both versions, Mix-256G and Mix-128G, are in Maya's model picker. One command sets it up on your own GPUs: NVIDIA or AMD, Linux or Windows.
Prefer an API? Use it on OrcaRouter: orcarouter.ai/models?tier=cy…
Thanks to the OrcaRouter team for working with us on this.
