Securing global platforms and raising the next generation of hackers.
Joined September 2010
- Tweets1.6K
- Following507
- Followers2.1K
- Likes2.5K
Max Moroz retweeted
Over the last few months at @river_ai_inc we’ve developed what we believe is the best system in the world for building your own internal custom models, driven by 10 years of experience training top models at DeepMind, OpenAI and xAI.
In order to best support our customers, we’ve decided to do something radical: We also deploy our API on-prem on your compute cluster, and if you don’t have compute we will help you find it.
Again and again we’ve seen that custom models designed for your own internal workflows outperform top frontier models on performance, at a fraction of the cost (1 percent or less).
We’re just at the beginning of the custom model era, but its already clear that in the future more than 99% of LLM tokens will be generated on efficient custom models. The economics of it are just too powerful, and it’s far easier than people think once you have the right tools in place.
The reason why small models will dominate is due to the physics of deep learning: A general model that can do everything needs more capacity and training it on the whole distribution of all tasks is more challenging, so the model will be larger and more expensive, and wont perform perfectly on every task. A small model trained on a targeted domain usually outperforms it.
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
Max Moroz retweeted
15 years on, slowhttptest has a rewrite.
slowhttptest can do HTTP/2 now. Took a rewrite in C++17 to get there. Beta's up, installs alongside, nothing renamed.
Curious what it finds in real world, try i out github.com/shekyan/slowhttpt…
Max Moroz retweeted
Very sad for Android. @DaveKSecure
We have a partial port of GrapheneOS to the Pixel 11 series after a week of work on it. We're unable to complete the port due to lack of support for ARM hardware memory tagging in software, firmware and near certainly hardware. It appears Google cut an important security feature to save money.
ARM hardware memory tagging (MTE) is used by GrapheneOS across the entire base OS including the kernel and every standard base OS process. It's only temporarily disabled for a few device-specific processes. It greatly improves protection against nearly all remote exploits and many local exploits.
Pixel 8 launched with hardware MTE support in October 2023. We integrated it into our hardened_malloc project and began using it across the OS later that month. Android and the Pixel OS never started using it by default. Android Advanced Protection Mode in Android 16 enables it for a few processes.
Apple's Memory Integrity Enforcement (MIE) is an always enabled feature on the iPhone 17. It's simply a high quality implementation of MTE using the latest standard extensions. It uses MTE in the most secure mode in the kernel and a large portion of userbase. They did a very good job integrating it.
Apple's MIE and Android 16+ AAPM don't use MTE for user installed apps unless those explicitly opt in. GrapheneOS enables it for more apps automatically and has a toggle for users to opt-in for every user installed app. There's a per-app toggle to opt-out for incompatible apps which is uncommon.
Neither iOS or Android encourage app developers to opt into MTE and other more aggressive security features used in the base OS. Apple's docs warn developers of performance and stability issues. Even Signal doesn't opt-in. Our approach enables forcing using MTE in the standard allocators regardless.
Pixel 11 does have security improvements including moving to post-quantum secure verified boot (ML-DSA) and replacing Samsung Shannon IMS with AOSP IMS. Titan M3 should significantly improve protection against data extraction in Before First Unlock state. It's too bad they ruined it by cutting MTE.
Pixel 11 series is a lot more expensive for an incremental improvement to the CPU, the same underpowered GPU and reduced RAM for the Pro base models. They finally caught up to the last generation of Qualcomm cellular radio. It's overpriced, the upgrades aren't impressive and losing MTE is appalling.
Compared to the Pixel 11, a Snapdragon 8 Elite Gen 5 has ~40% higher single threaded CPU performance, ~80% higher multi threaded performance, over 100% higher GPU performance and a far better cellular radio. It also finally has MTE. The next gen is what will be in the first Motorola with GrapheneOS.
Pixel 9a and earlier (including Nexus devices) were the Android Open Source Project reference devices. Pixel support was removed from AOSP with Android 16. It's now harder to support Pixels than many other devices and massive progress towards open source firmware and driver libraries was discarded.
Compared to the stock Pixel OS, GrapheneOS ships AOSP patches months earlier and Linux kernel patches many months earlier. However, we rely on them for firmware and most driver updates. We also want to move to new kernel branches earlier. These things can be improved with our Motorola partnership.
We strongly recommend against buying Pixel 11 devices. Pixel 8, 9 and 10 have much better overall security for GrapheneOS. Pixel 10 is cheaper with similar hardware and MTE. Pixel 11's Titan M3 should improve BFU security for users without a strong passphrase, but losing MTE craters AFU security.
We haven't determined what to do about this situation. It may be best for us to skip the Pixel 11 series devices. We can shift our focus entirely to the upcoming Motorola devices instead. Pixel 10a was really a 9th gen Pixel, so hopefully the Pixel 11a does the same with 10th gen and includes MTE.
Max Moroz retweeted
My team has a new position: performance modeling for our ML chip(s).
This position requires deep understanding of uArch and strong software engineering (our stack is bazel, C++, protos).
Please apply here:
lnkd.in/gH_MwmuT
Max Moroz retweeted
OpenSSH 10.5 release notes are a great example of an OSS project leaning into the changes AI driven code analysis is bringing. I worry about how the many understaffed but important projects out there will handle the volume of reports.
openssh.org/releasenotes.htm…
Max Moroz retweeted
1000+ fixes later (including one teenage bug!), here's how @googlechrome is using AI across the software bug lifecycle to make the web safer for billions: blog.google/security/chrome-… 🔎🪰✨
Max Moroz retweeted
1. Releasing TokenFuzz v1.0 - github.com/tokenfuzz/tokenfu… - a different perspective on auditing and benchmarking live code using LLMs. It compares model capabilities on new security findings and reproducible crashes rather than past-known vulns, and provides qualitative data in a table to augment research. An example below -
Max Moroz retweeted
🇺🇸 God Bless America 🇺🇸
NOW is the most exciting time for us as a civilization.
We stand at the absolute dawn of a new era. We are about to land the most consequential technological and scientific breakthroughs in human history, empowering us to conquer the very challenges that have plagued humanity since the beginning of time.
On this Fourth of July, I am reminded of our Founding Fathers’ core conviction: that liberty and self-governance require a virtuous, vigilant, and fiercely independent populace. They built a republic on unalienable rights, knowing that a great nation survives only through active courage and personal morality. Today, we have a monumental opportunity to unlock the truest essence of our species, which is our innate, unstoppable desire to dream, build, and innovate in the physical world. There is no better place to lead this renaissance than the United States of America.
After six years living abroad in Britain, leading security at the absolute dawn of generative AI at @GoogleDeepMind, I knew it was time to return home. There is no better moment than now to pursue greater, bolder challenges at the helm of my country in this thrilling era of American Dynamism and technological dominance.
@demishassabis is a true visionary. He taught me directly that artificial intelligence is the ultimate tool for scientific discovery. "If you solve intelligence, you solve everything else," he told me during my final interview with him in the summer of 2019.
Now, it is time to take that vision and forge it into reality.
In ancient myth, the Titan God Prometheus defied the heavens, stole fire from the gods, and gifted it to humanity, igniting the very dawn of human civilization, technology, and progress. Today, I am proud to share that I have joined @JeffBezos and @vik_bajaj at @Prometheus, where we are capturing that fire to power the next frontier of AI for the physical economy.
The vision they laid out in our very first meetings is formidable enough to make the Gods themselves take notice. We are merging the infinite potential of intelligence with the raw power of the physical economy.
It has been an extraordinary ride already, and we are just getting started. As an American, I am deeply inspired by the mission we have in front of us, not just as an organization, but as a country. One nation, under God, indivisible, with liberty and technological triumph for all.
If you are an exceptional engineer, want to work on the absolute frontier of what is possible and solve the hardest problems facing existence, my DMs are open.
LFG! 🔥🦅 🇺🇸
Max Moroz retweeted
A few weeks ago we announced the docking partner for our Otter Pup 2 mission, @GilmourSpace's ElaraSat. Otter Pup has now made it through Long-Range Rendezvous (LRR), the first phase of our docking mission sequence.
During LRR, Otter Pup used Starfish's autonomous guidance and navigation software to close the distance from hundreds of kilometers away to hold at 10 km from ElaraSat.
Throughout the approach, the team continued testing, collecting data, and refining software performance in orbit.
Now that we’ve completed the first phase of our docking mission sequence, we’re preparing to transition to relative navigation and take pictures of ElaraSat. Stay tuned!
Max Moroz retweeted
Arbitrary code execution in objdump -g
We have a thing for finding bugs in bug finding tools. IDA Pro, Ghidra, Binja Sidekick, or radare2. You name it we hacked it. Our friends were saying we should try objdump. So here we go.
Blog post: blog.calif.io/p/oobdump-relo…
AI-generated PoC and writeup: github.com/califio/publicati…
How has the analysis of the reasonable expectation of privacy doctrine evolved in the second half of the 20th century?
Katz v. U.S. (1967) established the famous two-part test, highlighting a subjective expectation of privacy and a societal recognition of that expectation as reasonable. Later, Smith v. Maryland (1979) emphasized the third-party doctrine and assumption of risk, presuming that individuals accept the risk of disclosure when sharing data. California v. Greenwood (1988) extended this, denying protection to discarded trash, while Harris v. U.S. (1968) solidified the "plain view" exception.
However, the Court began to draw finer lines. In Oliver v. U.S. (1984), the Court distinguished between "open fields" and the home's "curtilage," offering less protection to the former. California v. Ciraolo (1986) and Florida v. Riley (1989) involved aerial surveillance, limiting protection for what is visible from public airspace. Similarly, Dow Chemical v. U.S. (1986) permitted aerial photography of industrial facilities, distinguishing commercial privacy from the stricter sanctity of the home.
Kyllo v. U.S. (2001) marked a turning point. The use of thermal imaging to see inside a home was recognized as a search requiring a warrant. The Court emphasized the intimacy of the home and held that employing technology not in "general public use" to obtain information was unconstitutional. This decision echoed Warren and Brandeis's call for adapting the law to new technologies, ensuring the Fourth Amendment protects the home even when the intrusion is invisible.
While the core of the Katz test remains, its application has shifted. The analysis has become more nuanced, balancing privacy rights with law enforcement needs while reflecting an evolving understanding of privacy in the face of technological advancements.
The "Right to Be Let Alone" (1890 vs. Today)
In the Law Review article “The Right to Privacy” (1890), Samuel Warren and Louis D. Brandeis made an argument for the legal recognition of an individual’s right “to be let alone.” One of their central arguments, which is especially relevant today, was that the law should adapt to protect individuals from new forms of privacy invasion created by modern technological developments.
Warren and Brandeis analyzed existing legal doctrines and precedents to highlight that the state of the law was not sufficient to address the challenges posed by innovations such as instantaneous photography and widespread newspaper enterprises. The authors asserted that the law must evolve to keep pace with changes in society and technology.
They also analyzed remedies offered by the existing body of law, such as compensation for injuries due to defamation, and noted that these were not directly applicable to privacy intrusions. The main focus of protecting privacy, they argued, is the individual's right to an "inviolate personality," regardless of whether material damages resulted from the violation.
This argument is extremely relevant today, as we live in an era of ubiquitous data collection, monitoring, and tracking due to the popularity and pervasiveness of digital devices. The core principle articulated by Warren and Brandeis—that the law must not remain stagnant in the face of technological and business progress—serves as a powerful reminder that we must continuously evaluate and advance our legal frameworks to safeguard our right to privacy.
Max Moroz retweeted
Looking for security researcher with great public profile. Remote. API / AI exploits focus on novel techniques. No XSSers please ;) reply here or DM. Please repost
United States (FBI) vs. Apple Inc. (2016): whether the government would have prevailed in compelling Apple to unlock an iPhone used by a terrorist remains a significant open question.
The Fourth Amendment protects individuals from unreasonable searches and seizures. A reasonable search generally requires a warrant issued upon probable cause. Additionally, the All Writs Act allows courts to issue orders to help law enforcement carry out its duties.
The key argument in the government’s motion was that the All Writs Act order is lawful and binding. In fact, the court had issued similar orders in the past, and Apple had complied with those orders without much objection.
The government also claimed that Apple didn’t deny having the technical capability to comply with the order. This is factually correct, but Apple argued the issue was not about ability, but about compulsion. They contended that forcing them to write new code to create a "backdoor" constituted compelled speech in violation of the First Amendment. Unlike previous cases, Apple did not have the tool ready; they would have had to build it from scratch.
Another argument the government made was that the tool developed by Apple for unlocking the iPhone could remain in Apple's full control. It is reasonable to expect that such a system would likely be needed for future cases. Therefore, the government argued it would be unreasonable for Apple to claim an undue burden just because they might have to use the system again.
To strengthen the support of its motion, the government noted that no other statute limits the enforcement of the All Writs Act in this case. They specifically noted that the Communications Assistance for Law Enforcement Act ("CALEA") is not applicable and cited precedents of court authority to issue All Writs Act orders in support of lawful search warrants.
Despite Apple's strong First Amendment defense, the government likely would have prevailed in compelling Apple to unlock the iPhone under the All Writs Act.
P.S.
This post in my “legal series” is really special, because the class where we studied this topic was taught by the very person who represented the government in the case–a United States Attorney Eileen M. Decker. What a privilege to have been able to learn about all this from the professor’s first-hand experience!
Must law enforcement obtain a warrant to gain access to information collected by Unmanned Aerial Systems (UAS)?
The Fourth Amendment protects people from unreasonable searches and seizures. Generally, a warrant based on probable cause is required for a search to be considered reasonable. However, there are exceptions. The "plain view" doctrine allows the seizing of evidence if the officer is lawfully in a position to observe it (Harris v. United States, 1968) and it is visible to the naked eye (California v. Ciraolo, 1986). Additionally, in Florida v. Riley (1989), the Court held that aerial surveillance from a helicopter in public airspace did not constitute a search requiring a warrant.
UAS (drones) are increasingly used by law enforcement. This technology offers notable benefits for police work, providing greater mobility and officer safety. On the other hand, it intensifies privacy concerns.
In Katz v. United States (1967), the Court emphasized that the Fourth Amendment protects "people, not places." What a person "seeks to preserve as private, even in an area accessible to the public, may be constitutionally protected." While a single image taken by a drone from a public vantage point might not violate a reasonable expectation of privacy, continuous surveillance is a different story.
Half a century after Katz, in Carpenter v. United States (2018), the Court held that obtaining cell-site location information required a warrant because individuals have a reasonable expectation of privacy in the "whole of their physical movements." A similar argument was used in Leaders of a Beautiful Struggle v. Baltimore Police Dept. (2021), where the Fourth Circuit ruled that a persistent aerial surveillance program was unconstitutional without a warrant
Law enforcement generally must obtain a warrant to access UAS data, unless a specific exception such as the plain view doctrine or exigent circumstances applies.
Max Moroz retweeted
Lots of discussion about AI browsers these days! It's exciting, _and_ it comes with new security challenges that require proactive, layered defenses.
Our latest @googlechrome security blog shares some of our approach to agentic security: goo.gle/3XKSufM 🔐🕸️🤖✨
Is unlocking your phone with your face or fingerprint a violation of your Fifth Amendment rights?
The Fifth Amendment protects a person from being compelled to be a "witness against himself." This protection applies to testimonial evidence (revealing the contents of one’s mind). It generally does not apply to physical evidence, like a key to a safe or fingerprints used for identification.
Courts have analyzed whether a biometric unlock is a "physical" or "testimonial" act, and the results are split.
- The Physical View: Some courts argue it is analogous to providing a fingerprint for ID or standing in a lineup, which is not protected (State v. Diamond, Commonwealth v. Baust).
- The Testimonial View: Others point out it is the functional equivalent of entering a passcode, which is testimonial and protected (In re Search of a Residence in Oakland).
The distinction often comes down to the "Foregone Conclusion" doctrine.
If the government already knows that a specific device belongs to a specific suspect and can prove they can unlock it, compelling the unlock reveals no new information--it is a non-testimonial "foregone conclusion" (Commonwealth v. Gelfgatt).
However, if ownership is not established, the act of successfully unlocking the device testifies to your control over it. That act provides new information, arguably triggering Fifth Amendment protection.
Whether compelling a fingerprint violates the Constitution depends on the specific facts. The key question is: does the act of opening the device simply grant access, or does it force the suspect to reveal facts the government doesn't already know?
Max Moroz retweeted
Excited to finally release the report @openrouter and I have been working on.
The past year marked a decisive shift in how we build and use AI. Reasoning moved from the edges of research into the center of real-world production, driven by breakthroughs in model capability, open-source momentum, and the rise of AI-native applications. Today, we are releasing a large empirical study of this transition, based on more than 100 trillion tokens of real-world usage. OpenRouter now serves more than five million developers and routes traffic across more than 300 models from over 60 providers, with daily volume crossing one trillion tokens just last week.
Happy anniversary, o1 🍓
We collaborated with @a16z to publish the **State of AI** - an empirical report on how LLMs have been used on OpenRouter.
After analyzing more than 100 trillion tokens across hundreds of models and 3+ million users (excluding 3rd party) from the last year, we have a lot of insights to share.
As AI chatbots become increasingly popular and powerful, a critical legal issue arises: is a warrant required to access personal conversations with these agents?
The Fourth Amendment generally requires a warrant for private communications, but the "third party doctrine" (Smith v. Maryland) creates an exception for information voluntarily shared with service providers.
However, AI chatbot conversations are unique. They often involve personal thoughts and feelings, making them arguably similar to talking to a therapist or writing in a private diary. Unlike regular search queries, these responses carry a significant degree of individualization.
Technically, the third party doctrine applies because users voluntarily disclose this information to the provider. The traditional view is that once you share data with a third party, you lose your expectation of privacy.
But we must look at Carpenter v. U.S. (2018). In that case, the Court looked beyond just the volume of data and focused on the nature of the information. They ruled that data revealing the "privacies of life" requires protection. A comprehensive log of AI conversations is arguably far more revealing and sensitive than the cell-site location records discussed in Carpenter.
Practically, I believe conversations with AI chatbots are similar to emails. They should be treated as "content" rather than "metadata," qualifying as protected electronic communications under the Stored Communications Act (SCA).
A warrant should be required to access personal conversations with AI chatbots. We must ensure our laws recognize them for what they are: private dialogues and not just data points.
