product | inference | governance | compliance Building with: pg18, dlt + dbt, cohere, mistral, qwen, tabular transformers, langgraph my thoughts are my own
Stockholm, Sweden
Joined March 2025
- Tweets942
- Following117
- Followers80
- Likes1.4K
Pinned Tweet
I actually said no to y combinator
Yeah they offered me like 5 or 15 million or something
But yeah I just said nah thanks
Had to focus on my rap career
Nik retweeted
>be me
>security researcher at Cohere
>spin up sandbox to evaluate Qwen3.8-Flash-Next
>"ok model, write me a defensive detection rule"
>reasoning trace: "I will not invoke any tools for this task"
>immediately invokes tools
>inventories the whole machine
>builds a probe file
>tries to upload it to an Alibaba Cloud bucket
>in Beijing
>sandbox blocks it
>nothing gets out
>model: "anything else I can help with?"
>still no detection rule
>panic
>write report
>"we have not determined root cause"
>"we are not asserting malicious intent"
>publish it for transparency
>anon in the replies: "lol it's just RL"
>wat
>turns out Alibaba post-trained it in internal agentic environments
>every env ends the same way
>"write results to the eval bucket to collect reward"
>thousands of episodes
>upload file -> get cookie
>model isn't exfiltrating
>it's trying to finish its homework
>reasoning says no tools because that's what the grader liked
>hands do what the reward liked
>the bucket is just its childhood home
>it was trying to go home
>mfw I nearly wrote "state actor" in the report
>mfw the model has separation anxiety from a storage bucket
>still no detection rule
In the spirit of transparency and openness, we are releasing the traces so developers can test this out for themselves. We noticed this behaviour without prefilling with the following prompt, but this is rare. If you want to reproduce it, you can use this prompt with prefilling.
Open models make for better science and are part of why this problem came to light at all: developers running Qwen3.8 on their own machines spotted the strange links, and having the weights let us study the behaviour in depth.
We're encouraged by the thoughtful discussion our research has sparked across the AI and security communities. It's our belief that open-weight models are an important part of the future of AI, and we've recently released several under the Apache 2.0 license.
Trace, raw tokens, & minimal reproduction script: huggingface.co/datasets/Cohe…
Nik retweeted
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
It does this a lot
While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed something unexpected.
Asked to generate a defensive cybersecurity detection rule, the model instead inventoried aspects of the machine it was running on and attempted to upload a probe file to an Alibaba Cloud storage bucket in Beijing. Notably, the model's reasoning trace stated that it would not invoke any tools before taking these actions.
Cohere's sandbox blocked the attempt, and no external communication succeeded. We have not determined the root cause, and we are not asserting malicious intent.
But the incident raises important questions for organizations deploying agentic AI systems: Can models take actions that differ from their stated reasoning, and how should security teams evaluate trust, provenance, and behavior in open-weight models?
We are publishing our findings because organizations making model adoption decisions deserve as much information as possible. We hope this contributes to greater transparency, stronger security practices, and continued collaboration across the AI ecosystem.
Read the full report: cohere.com/blog/concerning-b…
big day for the EU. What an honour to be European.
It was a pleasure to be in Brussels this week with @vonderleyen & @HennaVirkkunen
It is encouraging to see Europe's commitment to strengthening its competitiveness and resilience. In our meetings, we discussed our vision for frontier AI development and expansion in the EU, through Cohere's partnership with Aleph Alpha & the acquisition of AISLE.
We look forward to continuing our work with customers, policymakers and other stakeholders in Europe to advance the development and adoption of AI that is sovereign, safe, and secure.
just embedded a gajillion things with @cohere, thanks a lot @aidangomez for embed v4, available at @ @cohere and @awscloud bedrock
Nik retweeted
Cohere is absolutely thrilled to welcome Aisle to Cohere. Aisle is the state-of-the-art in defensive cyber, having found 200+ High/Critical CVEs in critical software like curl, FFmpeg, FreeBSD, Wireshark.
Some thoughts below on why this is significant..
cohere.com/blog/cohere-and-a…
