research-input · RPT-FE0B5E6C2B09
The Expansion Ratchet: Why Powers Created for Exceptional Cases Rarely Stay Exceptional
Source identity
- Record
RPT-FE0B5E6C2B09- Original file
- Expansion Ratchet In AI Safety.md
- Original SHA-256
83664c187ffda9112a07cd75a3a42233aa54dad0cb754f6b9596cf35835c76b0- Source family
- v035
- Public copy SHA-256
83664c187ffda9112a07cd75a3a42233aa54dad0cb754f6b9596cf35835c76b0- Original custody
- Exact bytes retained in the private project package under
docs/_evidence/; not a public download.
Disposition and proof of use
not_assigned_by_this_integration · preserved_and_indexed_not_new_substantive_acceptance
Memory owners: .uai/file-handoff.uai, .uai/report-synthesis.uai, .uai/research-review.uai. These are private repository records, not public web routes.
Jump to a report section
- The Expansion Ratchet: Why Powers Created for Exceptional Cases Rarely Stay Exceptional
- Part I: The Structural Anatomy of the Expansion Ratchet
- Part II: Historical Manifestations of Institutional Ratcheting
- Financial Monitoring: The Bank Secrecy Act and Geographic Targeting
- Counterterrorism and Telecommunications: CALEA, Section 215, and Section 702
- Public Health and Emergency Powers: COVID-19 Digital Contact Tracing
- Border Security: The 100-Mile Zone and Device Searches
- Surveillance and Protest Policing: UK Section 44 and Smart Infrastructure
- Censorship and Online-Platform Regulation: FOSTA-SESTA and NetzDG
- Part III: Applying the Ratchet Effect to AI Safety Infrastructure
- Part IV: Institutional Designs to Resist Expansion
- 1. Hard Sunset Clauses
- 2. Purpose Limitation and Explicit Prohibitions on Secondary Use
- 3. Automatic Deletion
- 4. Independent Authorization and Judicial Review
- 5. Public Statistics and Audit Logs
- 6. Technical Separation
- Conclusion
- Works cited
Report body
The Expansion Ratchet: Why Powers Created for Exceptional Cases Rarely Stay Exceptional
The architectural framework of contemporary mass surveillance, behavioral monitoring, and legal restriction frequently originates not in a vacuum of authoritarian ambition, but rather in a democratically sanctioned response to a perceived existential crisis. When a society is confronted with a severe, localized, or catastrophic threat—such as a global pandemic, a coordinated terrorist attack, transnational human trafficking, or organized financial crime—the public and legislative bodies are highly motivated to grant extraordinary powers to the state or to corporate entities. However, historical analyses demonstrate a pervasive and nearly inevitable phenomenon: powers introduced for narrow, exceptional circumstances almost never remain confined to their original scope. Instead, they undergo an institutional ratchet effect, commonly referred to as function creep or mission creep.
Function creep is formally defined in science and technology studies as an imperceptibly transformative and contestable change in a data-processing system’s or legal authority's proper activity1. It operates at the nexus of reverse adaptation, policy spillover, and the self-augmentation of technology1. Once the legal justifications are established, the physical infrastructure is constructed, and the bureaucratic workflows are normalized, the newly created mechanisms operate like a mechanical ratchet. They move easily in the direction of greater surveillance, broader enforcement, and deeper societal integration, while powerfully resisting any movement toward reduction or dismantling. This report investigates the historical trajectory of the expansion ratchet across multiple policy domains, including financial monitoring, counterterrorism, telecommunications, public health, border security, surveillance, censorship, and online-platform regulation. By distilling the structural causes of this phenomenon, the analysis then projects these historical lessons onto the emerging domain of artificial intelligence (AI) safety infrastructure. Finally, the report develops a framework of institutional designs intended to render function creep technically and legally unfeasible.
Part I: The Structural Anatomy of the Expansion Ratchet
The transformation of exceptional powers into normalized tools of routine state and corporate control is rarely the product of a centralized conspiracy. Rather, it is the emergent result of predictable structural, economic, and psychological incentives operating within complex bureaucracies. The literature identifies ten recurring structural causes that facilitate the expansion ratchet, transforming temporary measures into permanent fixtures of governance.
| Structural Cause | Mechanism of Expansion | Societal Impact |
|---|---|---|
| Sunk Infrastructure Costs | Once immense financial resources are expended to build a system (e.g., massive data centers, smart streetlights, nationwide sensor networks), institutions are highly incentivized to maximize the system's utility by applying it to broader use cases3. | Technologies procured for benign municipal purposes (like energy savings) are secretly retrofitted for mass behavioral policing and surveillance. |
| Bureaucratic Incentives | Government agencies and corporate compliance departments inherently seek to expand their budgets, personnel, and institutional relevance. Adopting new missions utilizing existing tools is a reliable strategy for self-preservation and growth5. | Agencies duplicate efforts and endlessly widen their mandates, leading to costly, overlapping bureaucratic empires that resist downsizing6. |
| Secrecy | When systems operate under classified or confidential regimes, public and legislative oversight is bypassed. Secret interpretations of laws allow agencies to stretch definitions without democratic pushback7. | The public remains entirely unaware that their data is being harvested or their communications intercepted until a whistleblower exposes the practice. |
| Broad Statutory Definitions | Legislation drafted during emergencies often utilizes vague, open-ended language (e.g., "national security," "particular importance," "relevant to an investigation")7. | Courts and aggressive prosecutors interpret these ambiguities expansively, stretching the law to cover activities Congress never intended to regulate. |
| Technological Capability | Technology is inherently dynamic. As hardware and software develop new features (e.g., higher resolution lenses, advanced predictive policing algorithms), the sheer capability to monitor behaviors invites the actualization of that control3. | Runaway feedback loops occur where predictive algorithms justify increased policing, which generates more arrest data, further confirming the algorithm4. |
| Weak Sunset Provisions | Expiration dates on emergency powers are often either omitted entirely, drafted vaguely, or routinely renewed via legislative rubber-stamping with minimal public debate. | Temporary crisis measures seamlessly calcify into permanent features of the domestic legal landscape. |
| Political Fear | Lawmakers face immense political pressure to appear decisive against crime, terrorism, or pandemics. Rolling back a surveillance power carries the catastrophic political risk of being blamed for subsequent crises9. | A one-way legislative ratchet is created where politicians will only vote to expand security powers, never to restrict them, fearing electoral retaliation. |
| Public Normalization | As the public becomes accustomed to a new technology or security measure over years or decades, baseline privacy expectations erode. | Intrusions once viewed as dystopian or unconstitutional become accepted, unquestioned parts of daily civilian life4. |
| Vendor Lobbying | Private corporations that manufacture surveillance, filtering, or security technologies actively lobby legislative bodies for the expansion of their use to secure ongoing government contracts and open new commercial markets3. | Profit motives deeply entangle with state power, creating a military-industrial or surveillance-industrial complex that requires perpetual threat narratives. |
| Data Utility | Data collected for a specific, narrow reason (e.g., public health tracking) inevitably proves highly valuable for secondary purposes (e.g., criminal investigations)3. | Fundamental privacy promises are broken as the temptation to utilize pristine datasets for law enforcement overrides ethical and legal boundaries. |
These ten structural factors rarely operate in isolation; they are highly synergistic. For instance, broad statutory definitions combine with secrecy to prevent judicial review, while sunk infrastructure costs combine with vendor lobbying to push military-grade technologies into local municipal police departments. The outcome is a governance environment where boundaries dissolve almost automatically.
Part II: Historical Manifestations of Institutional Ratcheting
To fully comprehend the mechanics and inevitability of the expansion ratchet, an exhaustive review of historical case studies is required. Across vastly different sectors—from the financial system to telecommunications, and from public health to border security—a consistent pattern emerges. In every instance, authorities appealed to an exceptional, terrifying threat to secure power, only to deploy that exact power against the general population for routine administrative or law enforcement purposes.
Financial Monitoring: The Bank Secrecy Act and Geographic Targeting
The Bank Secrecy Act (BSA) of 1970 represents a foundational case study in how static thresholds and bureaucratic mission creep transform targeted law enforcement tools into mechanisms of mass civilian surveillance.
| Rhetorical Claim (The Exceptional Case) | Documented Expansion (The Normalized Reality) |
|---|---|
| The BSA was introduced explicitly to target high-level, sophisticated criminality. Proponents argued it was a necessary tool to dismantle wealthy drug cartels, organized crime syndicates, and major tax evaders engaging in "abnormally large transactions" that no ordinary citizen would undertake12. | The reporting threshold for a Currency Transaction Report (CTR) was set at $10,000 in 1970 and has never been adjusted for inflation13. Today, $10,000 captures routine commerce—used car purchases, small business deposits, and contractor payments—effectively turning bank tellers into surveillance agents monitoring ordinary Americans15. |
The erosion of the threshold’s value through inflation is only one vector of the BSA's expansion. The bureaucratic incentive to gather more granular financial intelligence led the Financial Crimes Enforcement Network (FinCEN) to utilize Geographic Targeting Orders (GTOs). These orders allowed the government to bypass the established $10,000 threshold entirely in specific regions. For example, in zip codes along the U.S.-Mexico border, FinCEN ordered non-bank financial services (such as check-cashing businesses used by unbanked or lower-income populations) to file detailed reports on all transactions exceeding a mere $20016. This mandated the collection of birthdates, Social Security numbers, and home addresses for practically every customer interaction.
The rationale for this expansion rested on the technological capability to process vast amounts of data and the bureaucratic belief that collecting the "whole haystack" of financial data prevents crime. However, the societal cost is the total normalization of financial surveillance. A law justified by the necessity of catching millionaire drug lords is now utilized to bury small family businesses in hours of daily compliance paperwork, threatening them with ruinous fines if they fail to act as unpaid surveillance outposts for the federal government12.
Counterterrorism and Telecommunications: CALEA, Section 215, and Section 702
The domain of telecommunications surveillance provides the most well-documented examples of the expansion ratchet, largely due to the exposure of classified programs by whistleblowers. The intersection of advanced digital networking, broad statutory definitions, and total secrecy catalyzed a historic expansion of state power following the September 11, 2001, terrorist attacks.
| Rhetorical Claim (The Exceptional Case) | Documented Expansion (The Normalized Reality) |
|---|---|
| The USA PATRIOT Act’s Section 215 and FISA Section 702 were sold to the public as vital, narrowly tailored tools strictly for tracking foreign terrorists. Intelligence agencies insisted these tools were essential to prevent another catastrophic attack on American soil and would not target the domestic population9. | The Foreign Intelligence Surveillance Court (FISC) secretly redefined the word "relevant" in Section 215 to permit the bulk collection of the telephony metadata of nearly every American citizen, effectively abolishing the requirement for individualized suspicion9. |
The Communications Assistance for Law Enforcement Act (CALEA) of 1994 laid the technological groundwork by mandating that telecommunications carriers design their networks to facilitate state wiretapping, embedding surveillance capabilities directly into the internet's architecture19. Once this sunk infrastructure was in place, the legislative authorizations of the 2000s allowed intelligence agencies to exploit it fully. Under Section 215, the National Security Agency (NSA) engaged in bulk metadata collection, analyzing the communication patterns, relationships, and locations of millions of innocent citizens9. The rhetoric of finding a "needle in a haystack" was operationalized by collecting the entire haystack and querying it at will17.
Similarly, Section 702 of the Foreign Intelligence Surveillance Act—which authorizes PRISM and Upstream data collection—was statutorily restricted to targeting non-U.S. persons located abroad17. However, the data collected under this authority became overwhelmingly useful for secondary purposes. Domestic law enforcement agencies, including the FBI, routinely engage in "backdoor searches." They query the massive Section 702 databases using the names and identifiers of U.S. citizens to find evidence of ordinary, domestic crimes entirely unrelated to national security or foreign intelligence17. A power granted under the terrifying specter of international terrorism has completely metastasized into a normalized tool for domestic policing, operating without the traditional constitutional requirement of a probable-cause warrant.
Public Health and Emergency Powers: COVID-19 Digital Contact Tracing
The COVID-19 pandemic provided a globally synchronized, real-time demonstration of the expansion ratchet. To manage the public health crisis, governments deployed digital contact tracing applications, movement tracking, and quarantine enforcement technologies at an unprecedented scale22.
| Rhetorical Claim (The Exceptional Case) | Documented Expansion (The Normalized Reality) |
|---|---|
| Health ministries worldwide assured citizens that digital contact tracing data would be highly restricted, anonymous, and utilized exclusively for identifying viral exposure. Absolute promises were made that health data would be firewalled from law enforcement and state security apparatuses22. | In Singapore, the government explicitly walked back its privacy promises, allowing the police to access TraceTogether proximity data for criminal investigations under the broad powers of the Criminal Procedure Code, fundamentally violating public trust11. |
In Singapore, the TraceTogether system utilized Bluetooth signals to log proximity data between citizens. Because widespread adoption was mandatory for the country to reopen its economy, citizens were coerced into participation under the strict, explicit promise that the data was exclusively for epidemiological tracing24. However, the temptation of the secondary utility of this data proved insurmountable for law enforcement. The Singapore Police Force accessed TraceTogether data to investigate a murder, and the government subsequently admitted that the data could be utilized for any criminal investigation11. While the government later passed legislation attempting to limit this access to "serious" crimes like terrorism and rape, the core conceptual firewall between civil health infrastructure and criminal surveillance was permanently destroyed11.
The ratchet effect in public health surveillance was not limited to Singapore. In Israel, the government bypassed standard legislative procedures by invoking emergency powers to utilize the Shin Bet—the nation's domestic intelligence and counterterrorism agency—to conduct cellular contact tracing10. Retooling a classified counterterrorism surveillance infrastructure to monitor the movements of the civilian population during a viral outbreak exemplifies how existing technological capabilities are rapidly repurposed during emergencies. The normalization of these measures leaves citizens vulnerable to permanent tracking, as the physical and legal architectures of surveillance long outlast the biological threat10.
Border Security: The 100-Mile Zone and Device Searches
The sovereignty of a nation necessitates the ability to control its borders, but the legal doctrines established to protect physical borders have been subject to aggressive, bureaucratic function creep, deeply eroding constitutional protections for millions of citizens.
| Rhetorical Claim (The Exceptional Case) | Documented Expansion (The Normalized Reality) |
|---|---|
| The border search exception to the Fourth Amendment was originally justified as a necessary, routine inspection mechanism at physical ports of entry to intercept smuggled goods, contraband, and agricultural pests crossing the international boundary27. | The legal definition of the "border" was expanded by federal regulation to include a 100-mile zone extending inward from any external boundary or coastline, subjecting nearly two-thirds of the U.S. population to checkpoints, roving patrols, and warrantless searches far from any actual border28. |
The U.S. border search exception historically permitted customs agents to search physical luggage and cargo without a warrant or probable cause27. However, broad statutory definitions within the Immigration and Nationality Act permitted federal agencies to define a "reasonable distance" from the border as 100 air miles28. Within this massive geographic area, Customs and Border Protection (CBP) operates internal checkpoints where motorists are routinely detained and questioned without individualized suspicion29. The legal exception created to intercept physical contraband has ratcheted inward, subjecting vast swaths of the domestic population to a quasi-constitutional zone where Fourth Amendment rights are severely degraded28.
Furthermore, the technological capability to store immense amounts of data on personal electronic devices has resulted in a radical expansion of what constitutes a "routine" search. CBP agents now claim the authority to conduct warrantless forensic searches of smartphones and laptops at border crossings27. A legal doctrine established in 1789 to inspect physical ledgers and barrels of rum is now weaponized to download and analyze the entire digital lives, private correspondence, financial records, and intimate associations of travelers. The failure to adapt legal restrictions to new technological realities has allowed border security powers to metastasize into generalized digital surveillance27.
Surveillance and Protest Policing: UK Section 44 and Smart Infrastructure
The creeping normalization of surveillance is starkly visible in the policing of public spaces, where anti-terrorism legislation and smart-city infrastructure are routinely redirected to monitor civic dissent and ordinary criminality.
| Rhetorical Claim (The Exceptional Case) | Documented Expansion (The Normalized Reality) |
|---|---|
| Section 44 of the UK's Terrorism Act 2000 granted police the extraordinary power to stop and search individuals without any reasonable suspicion, strictly framed as a necessary deterrent against bombings and terrorist plots32. | The geographic authorizations for Section 44 were drawn so broadly that the entirety of Greater London was designated a stop-and-search zone. Police routinely abused the power to harass peaceful protesters, environmental activists, and photographers documenting public events32. |
The application of Section 44 demonstrates the political fear of appearing insufficiently protective; once the anti-terrorism law was passed, bureaucratic incentives drove police to utilize it as a general tool for social control. It was deployed not against terrorists, but as a mechanism of intimidation against lawful civic behavior. The abuse became so systemic that the European Court of Human Rights eventually ruled it a violation of the right to privacy, forcing its curtailment33.
Similarly, local municipalities frequently fall victim to vendor lobbying, purchasing "smart streetlights" equipped with sensors and cameras under the rhetorical guise of reducing energy bills and monitoring air quality3. However, the sunk infrastructure costs and the technological capability of these devices invariably lead to function creep. Within years of installation, police departments tap into the smart streetlight networks to mine video surveillance footage for ordinary crimes, vandalism, and the monitoring of political demonstrations, such as Black Lives Matter protests3. The public is subjected to mass surveillance through infrastructure originally sold as a green-energy initiative, proving that once data is collected, it will be used for policing.
Censorship and Online-Platform Regulation: FOSTA-SESTA and NetzDG
Internet regulation frequently relies on the moral imperative of protecting highly vulnerable populations from severe harm. This emotional resonance is leveraged to establish vast infrastructures of censorship and platform liability, which subsequently ratchet into generalized suppression of speech.
| Rhetorical Claim (The Exceptional Case) | Documented Expansion (The Normalized Reality) |
|---|---|
| In the U.S., FOSTA-SESTA was championed with the singular, emotionally unassailable goal of fighting child sex trafficking. In Germany, the NetzDG law was passed to combat severe, illegal hate speech and incitement to violence on social media networks36. | FOSTA-SESTA resulted in massive collateral censorship, driving consensual adult sex workers off safe platforms and pushing them into highly dangerous, unmonitored shadow economies37. NetzDG’s threat of massive fines forced platforms to deploy automated filters that routinely over-block legitimate political discourse, satire, and journalism38. |
FOSTA-SESTA (Allow States and Victims to Fight Online Sex Trafficking Act) fundamentally altered the internet landscape by rolling back Section 230 safe harbor protections for specific content, making platforms directly liable for user actions. The threat of crushing civil and criminal liability created a massive chilling effect. Rather than surgically targeting traffickers, platforms preemptively banned adult content, marginalized LGBTQIA+ communities, and shut down vital harm-reduction forums where vulnerable individuals shared safety information and screened clients40. The law effectively erased the digital footprint of sex workers, increasing their exposure to physical violence and police harassment37. The infrastructure of censorship built to protect the vulnerable became the primary vector of their endangerment.
Similarly, Germany’s Network Enforcement Act (NetzDG) mandated that social media platforms delete "obviously unlawful" content within 24 hours or face fines of up to 50 million euros38. This created an overwhelming bureaucratic incentive for tech companies to over-block speech. When faced with ambiguous content and a ticking 24-hour clock, platforms default to deletion. Consequently, the rapid-censorship infrastructure designed for extreme hate speech ratcheted into a generalized behavioral policing mechanism, suppressing controversial but entirely legal political debates.
Part III: Applying the Ratchet Effect to AI Safety Infrastructure
As artificial intelligence capabilities scale toward artificial general intelligence (AGI), a consensus is rapidly forming among researchers and policymakers regarding the necessity of "AI safety infrastructure." The stated goal is to prevent catastrophic, existential risks—such as the autonomous generation of synthetic bioweapons, highly persuasive automated phishing campaigns, the proliferation of radiological weapons, or the disruption of critical national infrastructure. To mitigate these extreme cases, governments are proposing sweeping regulations. For example, U.S. Executive Order 14110 introduces compute thresholds and stringent Know Your Customer (KYC) reporting requirements for cloud infrastructure providers to track exactly who is training frontier AI models43.
However, applying the historical lessons of the expansion ratchet reveals a profound and immediate risk. The infrastructures currently being proposed for extreme AI risks possess all the structural characteristics—sunk costs, technological capability, vendor lobbying, and the potential for deep secrecy—that inevitably lead to normalized, generalized control over civilian populations. If these extreme-case mechanisms are successfully embedded into the foundational architecture of the next generation of computing, they will almost certainly be repurposed.
The following analysis details how mechanisms originally developed for catastrophic AI safety will ratchet into normal mechanisms of societal management.
| AI Safety Mechanism (Extreme Case Justification) | The Ratchet Effect (Normalized Future Use) |
|---|---|
| Identity-Linked Querying: Mandating KYC verification to ensure anonymous terrorists cannot prompt frontier models to design advanced biological or chemical weapons43. | The End of Anonymous Inquiry: Once the capability to link a verified biological identity to a prompt stream exists, the data becomes immensely useful for secondary purposes. Law enforcement will subpoena prompt histories to solve local crimes. Eventually, users will realize their intellectual, medical, and political explorations are permanently tied to their legal identity, chilling free speech and intellectual curiosity. |
| Behavioral Scoring: Using AI to continuously monitor user interactions to detect dangerous mindsets, catastrophic intent, or plotting of mass violence. | Social Credit and Risk Profiling: The same behavioral scoring algorithms can be effortlessly adjusted to score users for "misinformation consumption," "political radicalization," or "financial unreliability." A system built to flag bioterrorists will seamlessly evolve into a social credit system utilized by insurance companies, banks, or domestic security agencies to penalize non-conformist civilian behavior4. |
| General Content Filtering: Training guardrails into LLMs to strictly refuse to output bomb-making instructions or child sexual abuse material (CSAM). | Political Moderation and Over-blocking: Because the technical infrastructure for refusal is already sunk into the model's architecture, there is zero marginal cost to adding new categories of prohibited speech. General content filtering will ratchet down to include the policing of controversial scientific debates, the suppression of legal but disfavored speech, and the enforcement of shifting political orthodoxies, mirroring the over-blocking seen under NetzDG38. |
| Automated Surveillance: Building models with the capacity to automatically flag and report users who attempt to bypass safety filters (jailbreaking) for malicious physical intent. | Forward-Deployed State Spying: This automated surveillance apparatus can easily be co-opted by authoritarian or highly polarized governments. A state could mandate that the AI report users who prompt the model for information about protest organizing, unionizing, or political opposition. The model itself becomes a forward-deployed surveillance agent, continuously monitoring the private thoughts and drafts of the population. |
| Access Control: The ability to physically or cryptographically lock a user out of computational resources if they are deemed an imminent threat to national security. | Digital Excommunication: Originally envisioned as a "kill switch" for rogue actors training dangerous models, this extreme mechanism could be normalized. Access to basic, everyday computational utilities could be revoked for individuals who have unpaid fines, who have expressed dissenting views online, or who are deemed "high risk" by opaque bureaucratic algorithms. The existential kill switch becomes a tool for everyday social enforcement. |
Furthermore, the data generated by AI safety monitoring will inevitably bleed into the private sector. If employers can access an API that provides an "AI Safety Risk Score," they will eagerly utilize it for employment screening, preemptively rejecting applicants who exhibit non-standard thought patterns or historical prompt anomalies. In the realm of education, AI monitoring tools meant to prevent students from generating dangerous materials will shift into general education monitoring. These systems will track students' attention spans, political leanings, and emotional volatility, creating a permanent, inescapable developmental record that follows them into adulthood. The architecture of safety is functionally indistinguishable from the architecture of totalitarian control; the only differentiator is policy, and policy is deeply malleable to the forces of the expansion ratchet.
Part IV: Institutional Designs to Resist Expansion
Because the incentives for mission creep are deeply embedded in the structure of government and corporate bureaucracies, resisting the expansion ratchet requires institutional designs that make expansion technically, legally, and bureaucratically exhausting. History proves that rhetorical promises—such as "we will only use this for contact tracing" or "this is only for foreign terrorists"—are worthless against the gravity of institutional incentives17. To prevent AI safety infrastructure and future emergency powers from being weaponized against the general public, the following mechanisms must be hard-coded into both jurisprudence and software architecture.
1. Hard Sunset Clauses
Legislation authorizing exceptional powers must include hard sunset clauses that automatically and irrevocably terminate the authority on a specific date. Unlike soft sunsets, which allow for presumptive renewal or continuation pending review, hard sunsets require the agency to completely re-litigate the necessity of the power in a public legislative forum. They must prove, from a blank slate, that the exceptional circumstance still exists. If the legislature fails to act, the power and its associated infrastructure must legally cease to operate, forcing a systemic flush of the bureaucracy.
2. Purpose Limitation and Explicit Prohibitions on Secondary Use
Statutes must be drafted with mathematically precise, exhaustive definitions of permissible use, paired directly with explicit, enumerated prohibitions on secondary use. For instance, if an AI identity-verification system is authorized exclusively for the counter-proliferation of weapons of mass destruction, the law must explicitly state that the data cannot be subpoenaed for domestic law enforcement, civil litigation, immigration enforcement, or tax audits. This eliminates the "broad statutory definition" loophole9 and establishes that secondary use is not a matter of prosecutorial discretion, but a federal crime.
3. Automatic Deletion
To combat the structural risk of data becoming useful for secondary purposes, systems must employ cryptographic and automated deletion protocols. Data retention is the absolute lifeblood of function creep44. If data is not retained, it cannot be queried later by a sprawling intelligence apparatus. AI safety logs, proximity data, or metadata must be designed at the hardware level to auto-delete on a rolling basis (e.g., after 24 or 48 hours) unless specifically preserved under a targeted, individualized warrant.
4. Independent Authorization and Judicial Review
Exceptional powers must never be self-authorized by the agencies that execute them. The transition of the Section 215 program and Section 702 backdoor searches occurred largely because intelligence agencies operated with minimal adversarial friction within secret courts9. Any use of exceptional infrastructure must require independent authorization from an Article III judge in a truly adversarial proceeding—where a cleared public advocate actively argues against the government's request. Ex parte, secret courts systematically fail to prevent mission creep because they operate in an echo chamber of executive branch reasoning.
5. Public Statistics and Audit Logs
Secrecy breeds expansion. To counter this, surveillance and safety technologies must feature immutable, publicly verifiable audit logs. While the specific names of targets in an ongoing investigation may remain confidential to protect operational security, the meta-statistics of the system's use must be published quarterly. The public must know exactly how many times a database was queried, what percentage of queries resulted in physical arrests, and which specific agencies accessed the data. If a tool designed to catch bioterrorists is shown via public statistics to be utilized 99% of the time for domestic drug enforcement, the public and legislature are immediately alerted to the function creep, allowing for course correction.
6. Technical Separation
Legal boundaries are easily and frequently crossed if the technical architecture permits seamless integration. Therefore, institutional design must mandate technical separation. Databases holding exceptional-risk monitoring data must be physically air-gapped or cryptographically isolated from general law enforcement databases. Access should require multiparty computation or dual-key cryptography, where multiple independent institutions (e.g., a technical oversight board and a judicial panel) must physically and digitally consent to unlock the data. By baking severe friction into the hardware and software layers, the casual, everyday abuse of exceptional powers becomes technically impossible.
Conclusion
The historical record is unambiguous and highly consistent: powers created for exceptional cases rarely stay exceptional. Whether analyzing the ossification of the Bank Secrecy Act's thresholds13, the creeping jurisdiction of border search exceptions into our digital lives27, the weaponization of counterterrorism metadata against domestic citizens9, or the betrayal of public health privacy promises in the wake of COVID-1911, the expansion ratchet is a persistent, structural feature of institutional behavior. The mechanisms of sunk costs, bureaucratic incentives, and the overwhelming temptation of secondary data utility ensure that, absent immense institutional friction, targeted tools will invariably become generalized weapons of mass surveillance and behavioral control.
As society stands on the precipice of developing massive regulatory and technical infrastructures to ensure AI safety, policymakers and technologists must recognize that good intentions and rhetorical promises are entirely insufficient safeguards. If the capability to monitor, score, filter, and control human interaction with computational systems is built to prevent existential threats, it will inevitably be utilized to enforce political orthodoxy, track employees, and manage domestic populations. The only defense against the expansion ratchet is the proactive implementation of aggressive institutional designs—hard sunsets, unyielding purpose limitations, automatic cryptographic deletion, adversarial judicial review, and rigorous technical separation. Failing to mandate these friction-inducing designs will guarantee that the mechanisms built to save civilization will ultimately be used to cage it.
Works cited
- Full article: The concept of function creep, https://www.tandfonline.com/doi/full/10.1080/17579961.2021.1898299
- The concept of function creep - ResearchGate, https://www.researchgate.net/publication/350113451_The_concept_of_function_creep
- Function creep: turning everyday tech into spyware - Context News, https://www.context.news/newsletter?id=c875f7381fea5c11685c4b69ad4bcc70
- AI Surveillance: Privacy Nightmare or Utopia? - Remindax, https://www.remindax.com/blog/ai-surveillance-privacy-nightmare-or-utopia/
- #Reviewing Mission Creep - The Strategy Bridge, https://thestrategybridge.org/the-bridge/2016/6/15/reviewing-mission-creep
- "Mission Creep" Leads to Costly Duplication and Ineffectiveness, https://www.ntu.org/foundation/detail/mission-creep-leads-to-costly-duplication-and-ineffectiveness
- Function creep, altered affordances, and safeguard rollbacks, https://verfassungsblog.de/os6-function-creep/
- Secret Law - Digital Commons @ University at Buffalo School of Law, https://digitalcommons.law.buffalo.edu/cgi/viewcontent.cgi?article=1910&context=journal_articles
- Competing Visions of the NSA's 215 Program, https://scholar.law.colorado.edu/cgi/viewcontent.cgi?article=1216&context=ctlj
- Technosurveillance mission creep in Israel's COVID-19 response, https://www.brookings.edu/articles/technosurveillance-mission-creep-in-israels-covid-19-response/
- The Big Read: What's the big deal with data privacy? Thorny ... - CNA, https://www.channelnewsasia.com/today/big-read/big-read-whats-big-deal-data-privacy-thorny-complex-issues-confront-citizens-and-govts-5887456
- The Right to Financial Privacy - Cato Institute, https://www.cato.org/policy-analysis/right-financial-privacy
- Bank Secrecy Act Reporting Thresholds 2026 - Zyphe, https://www.zyphe.com/resources/blog/bank-secrecy-act-bsa-explained
- Form 8300 and reporting cash payments of over $10,000 - IRS, https://www.irs.gov/businesses/small-businesses-self-employed/form-8300-and-reporting-cash-payments-of-over-10000
- The USA has a $10k threshold for transactions that need reported, https://www.reddit.com/r/askanything/comments/1qv42ne/the_usa_has_a_10k_threshold_for_transactions_that/
- FinCEN Border Cash Surveillance - The Institute for Justice, https://ij.org/case/fincen-border-cash-surveillance/
- USA Liberty Act Won't Fix What's Most Broken with NSA Internet, https://www.eff.org/deeplinks/2017/10/usa-liberty-act-wont-fix-whats-most-broken-nsa-internet-surveillance
- IPB0108 - Evidence on Draft Investigatory Powers Bill, https://committees.parliament.uk/writtenevidence/62581/html/
- Read "Protecting Individual Privacy in the Struggle Against Terrorists, https://www.nationalacademies.org/read/12452/chapter/13
- Decoding Encryption Policy: Balancing Privacy, Security, and Law, https://wise-intern.org/wp-content/uploads/2019/04/Zeyi_Lin_-FinalPaper.pdf
- How Congress Should Evaluate Section 702's Security Value When, https://www.lawfaremedia.org/article/how-congress-should-evaluate-section-702s-security-value-when-debating-its-reauthorization
- Surveillance and the 'New Normal' of Covid-19: Public Health, Data, https://covid19research.ssrc.org/public-health-surveillance-and-human-rights-network/report/
- Ethics of digital contact tracing wearables - Journal of Medical Ethics, https://jme.bmj.com/content/48/9/611
- TraceTogether app - Contact-tracing system in Singapore, https://lkyspp.nus.edu.sg/gia/article/tracetogether-questions-of-data-and-privacy
- TraceTogether Privacy Policy ninja edit.. what is next? : r/singapore, https://www.reddit.com/r/singapore/comments/ks5l0p/tracetogether_privacy_policy_ninja_edit_what_is/
- COVID-19 (Temporary Measures)(Amendment) Bill on Investigatory, https://www.wp.sg/parliament/covid-19-temporary-measures-amendment-bill-on-investigatory-exceptions-to-tracetogether--speech-by-speech-by-lo-pritam-singh
- Border search exception - Wikipedia, https://en.wikipedia.org/wiki/Border_search_exception
- Understanding Your Constitutional Rights in the '100-Mile Border, https://www.criminallegalnews.org/news/2025/jul/1/understanding-your-constitutional-rights-100-mile-border-zone-primer-non-citizens-united-states-when-confronted-law-enforcement/
- The '100-mile border zone' and the Surveillance of Frontline, https://scholarlycommons.law.case.edu/cgi/viewcontent.cgi?article=1032&context=sjlcr
- 100 Mile Border Zone | American Civil Liberties Union, https://www.aclu.org/know-your-rights/border-zone
- Explainer: U.S. Border Patrol Authorities and the 100-Mile Border Zone, https://forumtogether.org/article/explainer-u-s-border-patrol-authorities-and-the-100-mile-border-zone/
- Criminal Justice Bill: 29 Oct 2003: House of Lords debates, https://www.theyworkforyou.com/lords/?id=2003-10-29a.283.2
- Covert Derogations and Judicial Deference: Redefi… – McGill Law, https://www.erudit.org/en/journals/mlj/2011-v56-n4-mlj1817631/1005848ar/
- Photography and the law - Wikipedia, https://en.wikipedia.org/wiki/Photography_and_the_law
- Negotiating British-Muslim Identity: Hybridity, Exclusion and, https://livrepository.liverpool.ac.uk/3001177/1/200636935_Aug%202015.pdf
- regulation of internet intermediary service provider liability and, https://scholarlypublications.universiteitleiden.nl/access/item%3A3463679/view
- (PDF) Markets for Rules: The Promise and Peril of Blockchain, https://www.researchgate.net/publication/327121561_Markets_for_Rules_The_Promise_and_Peril_of_Blockchain_Distributed_Governance
- an analysis of the first NetzDG reports | Internet Policy Review, https://policyreview.info/articles/analysis/reading-between-lines-and-numbers-analysis-first-netzdg-reports
- Regulating Speech Online: Free Speech Values in Constitutional, https://wustllawreview.org/2022/01/22/regulating-speech-online-free-speech-values-in-constitutional-frames/
- (PDF) LGBTQIA+ sex work in the time of Covid 19 - ResearchGate, https://www.researchgate.net/publication/362176068_LGBTQIA_sex_work_in_the_time_of_Covid_19
- Policing sex work online - SFU Library Thesis Template, https://summit.sfu.ca/_flysystem/fedora/2025-02/etd23627.pdf
- White paper on feminist internet research, https://firn.genderit.org/sites/default/files/2022-10/firn-whitepaper-2022.pdf
- (PDF) Training Compute Thresholds: Features and Functions in AI, https://www.researchgate.net/publication/380720022_Training_Compute_Thresholds_Features_and_Functions_in_AI_Governance
- Toward a U.S. Data-Retention Standard for ISPs | EDUCAUSE Review, https://er.educause.edu/articles/2006/12/toward-a-us-dataretention-standard-for-isps
- The Foreign Intelligence Surveillance Court in the Wake of the War, https://scholarship.law.umn.edu/cgi/viewcontent.cgi?article=1575&context=faculty_articles