Powers expand not only because of bad actors, but because infrastructure, budgets, vendors, secrecy, political incentives, and useful data all create pressure for a narrow tool to find broader uses.
Editorial research synthesisSafety and institutional powerWorldwide principles with selected jurisdictional examplesUpdated 2026-09-02
Direct answer
The expansion ratchet is the tendency of emergency or exceptional powers to broaden after deployment because institutions have sunk costs, incentives to preserve capability, vague mandates, weak sunsets, and valuable retained data. AI safety systems are especially vulnerable when they combine identity verification, query logs, behavioral scoring, and remote policy updates. Resistance requires hard sunsets, purpose limitation, automatic deletion, independent authorization, public statistics, and technical separation.
Key points
Mission creep can occur without conspiracy; ordinary institutional incentives are enough.
Data retention is the fuel of secondary use.
Broad statutory or policy terms let targets change while infrastructure remains constant.
A soft promise not to repurpose a system is weaker than architecture that makes repurposing difficult.
Sunsets must terminate authority and data, not merely schedule a future discussion.
Ten forces that drive expansion
Sunk costsOnce a system exists, institutions seek more uses for it.
Bureaucratic growthBudgets and relevance increase with mission scope.
SecrecyHidden rules reduce public and adversarial review.
Broad definitionsElastic terms permit new targets without new authority.
CapabilityAvailable tools create pressure to use them.
Weak sunsetsRenewal becomes routine and politically asymmetric.
Political fearOfficials are blamed for missed threats more than diffuse privacy loss.
NormalizationRepeated exposure lowers public resistance.
Vendor incentivesSuppliers lobby for continuing and expanded markets.
Data utilityRecords collected for one purpose prove attractive for another.
Historical lessons
Financial reporting, telecommunications surveillance, border searches, contact tracing, protest policing, and platform-liability regimes show different versions of the same problem. A narrow predicate justifies infrastructure. The infrastructure later expands through lower thresholds, new agencies, new offenses, or new data uses.
The lesson is not that every expansion is unjustified. It is that the initial authorization should assume future pressure and build resistance into the system. History repeatedly shows that a promise of narrow use is not a control mechanism.
How the ratchet could operate in AI
From exceptional AI safety to ordinary social control
Initial justification
Possible expansion
Verify identity for access to extreme compute
Require identity for ordinary model use and private inquiry
Log attempts to obtain catastrophic capabilities
Retain all prompts for routine policing or employment screening
Score imminent mass-harm risk
Create generalized political, financial, or social risk scores
Refuse narrow weaponization assistance
Add broad ideological or controversial-topic prohibitions
Disable a malicious high-risk account
Use computational access as a general sanction mechanism
The critical design distinction is whether the policy can change targets without rebuilding the technical system. A universal identity-and-logging layer is easy to repurpose. A request-local, non-retained capability check is much harder to turn into generalized surveillance.
Six anti-ratchet safeguards
Hard sunsets: authority and collection cease automatically unless reenacted through public legislation.
Purpose limitation: permitted use is exhaustively defined and unrelated use is prohibited, not merely discouraged.
Automatic deletion: data expires at the hardware or system level unless preserved under targeted legal process.
Independent authorization: the operating agency cannot approve its own exceptional access.
Public statistics and audit logs: society can see who uses the system, for what purpose, and with what results.
Technical separation: exceptional-risk data is isolated from general law-enforcement, advertising, employment, and recommendation systems.
These safeguards impose friction deliberately. In this domain, friction is not inefficiency; it is a constitutional feature that prevents extraordinary power from becoming routine.
