The AI paradox: security, sovereignty, and the new centralization
The RBAC (Role-Based Access Control) model is fundamentally buckling under the weight of modern AI integration. As people begin running AI agents in user space, granting them full access to personal data for the purpose of grepping and summing up information, we are entering a profound paradox. We demand that AI be as helpful as possible, but in doing so, we effectively leak that very data as part of the prompt context.
Our current Identity Model is lagging. The issue isn’t that RBAC is inherently broken; rather, it is our habit of granting all-or-nothing permissions to mimic human behavior. We are building agents as if they were humans simply to circumvent the lack of programmatic access, creating both a blessing and a curse at the same time that introduces severe security considerations.
Moving toward agent-centric identity
The solution lies in a shift from User-Centric to Agent-Centric identity. Instead of agents inheriting the broad permissions of a user, we should implement scoped tokens or capability-based security (similar to Linux capabilities).
If an agent only needs to “read my calendar,” it should be granted a specific, short-lived capability token rather than full “User Mode” access.
By moving toward this model, we can preserve the principle of least privilege while still enabling the utility of autonomous agents.
The “lethal trifecta” and the shift to endpoint security
Around a year ago, Simon Willison coined the term “lethal trifecta” in regards to the security implications of AI agents. The trifecta consists of three aspects of treating with AI:
- Access to private data.
- Ability to externally communicate (hence the risk of exfiltration).
- Exposure to untrusted content (context poisoning, prompt injection to bypass safety checks, etc).
This dynamic also mirrors the evolution of encryption. We spent decades perfecting network-layer encryption (HTTPS) to prevent third parties from eavesdropping us over the wire. While that successfully rendered man-in-the-middle attacks practically useless for transit, the rise of agents accessing everything on the client side for context means our focus must shift. As pointed out by Meredith Whittaker, president of the Signal foundation (you know, the messaging app) on the subject of AI agents embedded in operating systems:
“Instead of having to break [Signal’s] gold standard encryption algorithm, which have been tested and mathematically proven to be secure, you just have to leverage the type of access these pervasive agents are being given into your application, into your intimate data [by the operating system itself] in ways that are from a security architecture perspective very insecure.”
This highlights a critical shift: we need to focus our security efforts on the endpoint itself.
Privacy is facing a similar erosion. As AI adoption expands its scope, the years of awareness-raising regarding data privacy are being overshadowed by the convenience of the tool. For the general public, centralized AI will have a hard time coexisting with these privacy standards, though for the companies collecting the data, this transition is a godsend.
Lessons from the evolution of the internet
We are currently re-living the start of the internet, yet we are choosing to ignore the lessons of the past decades. The internet began as a decentralized haven created by scientists and optimists, only to be centralized by corporations who promised to mutualize means of production and reduce costs through mass-scale economics.
However, the real turning point wasn’t just the price of the service; it was the realization that user data was a pile of gold. When companies turned into data brokers, the focus shifted from “who is selling my data” to “when will they realize they can sell my data and how much data they can collect”.
The past decade of internet was driven by the monetization of data, and we are now seeing the same pattern emerging with centralized AI models.
This cycle is repeating with AI at an even higher scale:
- Investors are buying every bit of compute and energy source available to sell tokens-as-a-service.
- The barrier to entry is now unfairly high, making it even more centralized than the early internet.
- The public is closing their eyes to the invisible invasion of privacy because the tangible results of AI frontier models are so compelling.
The geopolitical aspect
We are already seeing a trench being dug between the USA and Europe regarding AI regulation. While the EU has pioneered the GDPR, the Digital Markets Act and the upcoming AI Act are driving up the cost of owning and protecting massive amounts of user data for companies.
We see this play out in the real world: companies like Apple have already made certain AI features unavailable in Europe as a protest against these costs. Yet, the consumer opinion remains complex, end users often side with the availability of features over the nuances of regulation.
Conclusion
Just when we finally start to realize how much data we have surrendered, we are doing it again at a scale never before seen. For the first time, AI feels truly general, available to the IT professional, the chemist, and the gardener alike. It is no longer a tool for a select few; its complexity is no longer a barrier.
But this generality comes with a cost. When users blindly trust their AI models on everything, the power held by the companies providing those models becomes absolute. We are trading sovereignty for convenience, and the price of that trade is being written in the data we provide today.
However, the rise of local LLMs offers a potential path out of this paradox. By moving inference to our own hardware, we can finally achieve a form of private model that offers the context-rich utility of AI without the inherent leak of cloud-based prompt contexts. It represents a way to reclaim sovereignty, ensuring that our data stays within our own walls while still reaping the rewards of the AI revolution.
References
[Simon Willison - The Lethal Trifecta]: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
[Is AI a Threat to Privacy? - Interview of Meredith Whittaker]: https://www.youtube.com/watch?v=5Wi6hse46l0
