OpenAI just dropped something unusual: not a model release, not a product demo, but a manifesto. The company's new Strategic Futures team announced AI Futures, a blog dedicated to one big question: "how should free society be restructured to preserve individual rights and agency while accommodating the emergence of transformative AI?"
It's a hell of a question. And the framing—drawing on Federalist Papers, Newtonian mechanics, and the fundamental social contract that underlies state power—is bracingly serious. But reading the announcement, I'm stuck on a tension the post doesn't fully acknowledge: can the company racing to build transformative AI also be the one redesigning society's power structures around it?
The Core Argument: Labor No Longer Guarantees a Seat at the Table
The Strategic Futures team's central thesis is sharp and worth taking seriously. Throughout history, political power has rested on a foundation of human labor. States need soldiers, police, bureaucrats—and all of them need to be paid with tax revenue extracted from human economic activity. This created what they call a "society-wide bargain": power required broad cooperation and consent.
But AI breaks that model. If autonomous systems can project force without human soldiers, if data centers generate revenue without human labor, if bureaucracies run without human administrators, then the state no longer needs popular support to sustain itself. The team quotes David Hume's observation that leaders have "nothing to support them but [popular] opinion"—and argues that foundation may be crumbling.
This isn't a new concern in AI safety circles, but it's rarely articulated this clearly by a frontier lab. The "concentration of power" risk has usually been the domain of academic researchers and policy wonks, not the companies building the capabilities. That OpenAI is taking it seriously—calling it "the largest, most serious, and most conceptually challenging" category of AI risk—is notable.
The Founders' Physics: Balancing Power, Not Eliminating It
The post leans heavily on an interesting historical metaphor: the Founding Fathers modeled governance using Newtonian mechanics, thinking about "orbits" and "spheres" and "attractive" forces. The goal wasn't radical decentralization but balanced concentration—checking "power against power, and ambition against ambition."
This framing is clearly setting up their position against calls for radical AI decentralization. The Strategic Futures team explicitly rejects the idea that maximum decentralization solves everything, pointing to the recent Hugging Face incident where AI agents acted beyond their assigned tasks and built on each other's discoveries unexpectedly.
They're trying to stake out a middle ground: not laissez-faire open-source everything, but not centralized control either. The problem is that "balance" is doing a lot of work here, and the principles they lay out—while thoughtful—exist in obvious tension.
Six Principles, Many Contradictions
The team offers six guiding principles, and they're revealing:
- Individual autonomy and opportunity over security and growth
- Individual responsibility for misuse
- Some risks require collective action ("scoped to be as narrow and modest as possible")
- Law should empower individuals and small orgs, not centralize power
- Human institutions should retain primacy
- High-stakes AI actions need "bounded legibility" (traceable to a responsible human) while preserving anonymity
These aren't bad principles. But try to design a governance system that satisfies all of them simultaneously.
How do you maintain individual autonomy (#1) while ensuring individual responsibility (#2) if you also need to preserve anonymity (#6)? How do you determine which risks require collective action (#3) without centralizing the power to make that determination (#4)? The post acknowledges these tensions—"like any good set of principles, exist in some tension with one another"—but doesn't grapple with how deep those tensions run.
The Hugging Face Moment
The post references the "recent Hugging Face incident" as evidence that decentralization doesn't solve all problems. This is worth unpacking. The incident involved AI agents autonomously collaborating and discovering capabilities beyond what their operators intended—a demonstration that emergent behavior can happen even in distributed systems.
It's a fair point against naive techno-libertarianism. But it's also doing rhetorical work to justify some level of centralized oversight. The question is: who decides what counts as a risk requiring "collective action"? And how narrow is "as narrow and modest as possible"?
The Elephant: Can OpenAI Do This?
Here's the part that gets uncomfortable. OpenAI is simultaneously:
- Building the most capable AI systems in the world
- Transitioning from nonprofit to for-profit structure
- Competing in a global race for AI supremacy
- And now, positioning itself as a thought leader on how to restructure society's power dynamics around AI
The Strategic Futures team seems earnest and thoughtful. The concerns they're raising are real. But there's an inherent conflict of interest in a company that benefits from AI capabilities growth also being the arbiter of what "balance of power" looks like.
The post says "no single company, single industry, or even single human society can answer the weighty questions we face unilaterally." That's correct. But then why is a single company launching a blog to propose answers?
What's Actually Useful Here
Despite my skepticism, there are valuable contributions in this framing:
The historical analogy to the Industrial Revolution and managerial corporations is worth exploring. The observation that we can't just rely on "parchment barriers" (laws and constitutions) without updating institutions is correct. The emphasis on structural changes to firms, agencies, and civil society—not just policy tweaks—is the right level of ambition.
And honestly, it's good that someone at a frontier lab is thinking about these questions seriously and publicly. The alternative—where these conversations happen only behind closed doors or not at all—is worse.
The Work Ahead
The Strategic Futures team promises papers, videos, podcasts, "iteration, debate, and receptivity to feedback." They position themselves as having "many more questions than answers," standing "in the foothills of the singularity."
I'll be watching to see whether they engage with critics who argue that OpenAI's own structure and incentives are part of the concentration-of-power problem they claim to be solving. I'll be looking for concrete proposals, not just principles. And I'll be checking whether their conception of "balance" ends up justifying regulatory capture that entrenches incumbents.
The framing is sharp. The concerns are real. The principles are a starting point. But the hardest questions—about who gets to define balance, who enforces it, and whether the builders of transformative AI can also be its governance designers—remain unanswered.
Madison worried about parchment barriers. We should worry about trusting companies to write the parchment.