The Cost of Thinking with a Machine: Privilege Gaps and Unequal Access to Confidential Advice

Report body

The Cost of Thinking with a Machine: Privilege Gaps and Unequal Access to Confidential Advice

Executive summary

The attached commission brief asks whether U.S. privilege, discovery, and third-party doctrines impose an intellectual-privacy penalty on people who use inexpensive machine assistance rather than lawyer-controlled channels, while requiring the analysis to preserve distinctions among privilege, work product, ethical confidentiality, contract, technical privacy, search-and-seizure law, and discovery procedure. It also asks that the analysis not turn a confidentiality reform into immunity for underlying evidence, fraud, or misconduct. fileciteturn0file0

The research supports a strong but narrower critique than “AI chats are not privileged.”

In United States v. Heppner, Judge Jed Rakoff did hold that a represented criminal defendant’s independently created Claude materials were neither attorney-client privileged nor protected work product. But the written February 17, 2026 memorandum is fact-specific: Heppner had used Claude on his own initiative, counsel conceded that they had not directed him to conduct the searches, and the court concluded that the consumer-platform terms then governing his use defeated the asserted expectation of confidentiality. citeturn6view2turn6view3turn6view5 The opinion also expressly contemplated a materially different possibility in which counsel directs AI use and the system functions as an agent facilitating legal advice under ordinary agency principles; it did not decide that hypothetical. citeturn6view6

The civil cases point in a substantially different direction. Warner v. Gilbarco, Inc. rejected a demand for a pro se plaintiff’s AI-related materials on two independent grounds: Rule 26 relevance/proportionality and work-product protection. The court emphasized that Rule 26(b)(3) covers litigation materials prepared by a party as well as by representatives, and it applied the more demanding work-product waiver rule requiring disclosure to an adversary, or in circumstances substantially likely to reach one, rather than equating every technical third party with waiver. citeturn17view0 Assini v. Hayward, an official New York decision issued June 4, 2026, then found Morgan v. V2X persuasive and quashed subpoenas to OpenAI for a pro se defendant’s litigation-related AI materials. citeturn18view2

That yields the report’s central doctrinal conclusion:

There is no present universal U.S. rule that machine-assisted deliberation is either privileged or discoverable. Outcomes depend heavily on what protection is asserted, who used the system, why it was used, whether counsel directed it, whether litigation was anticipated, actual confidentiality arrangements, the governing jurisdiction, and the procedural mechanism seeking disclosure.

Federal civil procedure makes this especially important. Rule 26(b)(1) limits discovery to nonprivileged matter that is relevant and proportional, expressly considering factors including the parties’ relative access to information and resources. Rule 26(b)(3) separately protects qualifying materials prepared in anticipation of litigation by or for a party or its representative, subject to the substantial-need/undue-hardship exception, while giving additional protection to mental impressions and legal theories. citeturn14view2turn14view3 Rule 26(c) additionally permits protective orders for good cause. citeturn16view0 Thus, nonprivileged is not synonymous with automatically producible.

The distributional objection is also supported by evidence rather than conjecture. The Legal Services Corporation’s latest Justice Gap study reports that low-income Americans received no or insufficient legal assistance for 92% of substantial civil legal problems; 46% of people who did not seek legal help for at least one problem identified cost as a reason, and 53% doubted they could find an affordable lawyer if needed. citeturn26search0turn26search2 It is therefore plausible—and explicitly labeled as an inference in the report—that confidentiality rules built around professional relationships and premium controlled infrastructure can burden people who use low-cost computational assistance precisely because conventional professional help is difficult to obtain.

The report rejects, however, an unlimited “AI privilege.” Ira Robbins’s Against an AI Privilege presents the strongest contrary case: established privileges have historically been tied to confidentiality duties, institutional accountability, and relationships that general-purpose AI providers do not possess; a broad privilege could obstruct truth-seeking and create strategic opportunities to hide evidence behind a conversational interface. citeturn27search2turn27search4 The recommended reform therefore protects confidential deliberative assistance, not “AI” as a favored speaker or a fictional lawyer.

What the research establishes

The Heppner factual record is considerably more qualified than shorthand accounts imply. FBI agents seized roughly thirty-one documents memorializing Claude exchanges during execution of a search warrant. Defense counsel represented that Heppner had generated the materials after receiving a grand-jury subpoena and learning he was a target; counsel argued that he had included information learned from lawyers, intended the materials to help him communicate with lawyers, and later shared them with counsel. But counsel also conceded that the Claude work was undertaken without their direction. citeturn6view2turn6view3

Rakoff identified separate problems with attorney-client privilege. Claude was not counsel; the court found the relevant platform relationship insufficiently confidential under the terms it examined; and later transmission of independently created material to counsel did not retroactively transform the original exchange into a privileged lawyer-client communication. citeturn6view4turn6view5turn6view7 For work product, the court emphasized Second Circuit doctrine protecting attorneys’ mental processes and found that these documents were neither prepared by/at the behest of counsel nor reflective of counsel’s strategy when created. citeturn6view8turn6view9turn6view10 The separate criminal Rule 16 theory also failed because the documents had been obtained through the search warrant rather than reciprocal pretrial discovery. citeturn6view9

Warner demonstrates why translating that result wholesale into civil discovery would be a mistake. The Eastern District of Michigan held that a sweeping demand for “all” information concerning the pro se plaintiff’s use of third-party AI tools was not relevant or proportional and, independently, was protected work product. The court specifically noted the absence of evidence that the plaintiff had uploaded materials in violation of the protective order. citeturn17view0 It further distinguished attorney-client waiver from work-product waiver and rejected the proposition that routing internal analysis through ChatGPT automatically destroyed work-product protection. citeturn17view0

Assini confirms that this was not an isolated theoretical distinction. The official New York opinion stated that litigation-preparation material may receive conditional protection under CPLR 3101(d), discussed Heppner, quoted Morgan’s contrary civil reasoning at length, found Morgan persuasive, and quashed the subpoenas to OpenAI. citeturn18view1turn18view2 At the same time, Assini directed the litigant to comply with New York’s AI-use rules, demonstrating that protecting private preparation and regulating the integrity of court submissions are compatible objectives. citeturn18view2

New York’s actual Part 161 is an especially useful control against overclaiming. It says AI use by attorneys and parties in preparing court papers should not be prohibited, and that litigants ordinarily should not have to disclose to the court merely that AI was used. Its model rule instead focuses on understanding the technology, independently reviewing the resulting paper, and preventing fabricated citations or false material from reaching the court. citeturn19search0turn19search1 The rule was adopted March 25, 2026 and became effective June 1, 2026. citeturn19search3

Provider architecture matters, but it is not law. Current OpenAI policy distinguishes individual and business use: individual-service content may be used for model improvement unless the user opts out, while Temporary Chats are not used for model training and currently may be retained for up to thirty days for safety purposes; OpenAI states that business data is not used for model training by default and that qualifying API customers can obtain additional retention controls, including zero-data-retention arrangements for eligible uses. citeturn22search0turn22search1turn22search2turn22search19 Anthropic’s present policies likewise distinguish consumer and commercial offerings: consumer model-improvement use is opt-in under its current regime, while commercial chats generally are not used for model training absent specified opt-in or feedback pathways. citeturn23search0turn23search1turn23search5

Those facts support a reform based on actual function rather than “consumer,” “enterprise,” “no training,” or “zero retention” labels. A system can refrain from training yet retain records; it can retain nothing for ordinary service operation yet pass data through an enabled third-party integration; an organizational administrator may have access that a personal account does not; and a local model may avoid provider custody without thereby creating an attorney-client relationship. OpenAI itself, for example, warns that information transmitted through third-party GPT actions can become subject to the recipient’s privacy policy. citeturn22search9turn22search10

Cross-border storage likewise offers no simple immunity. Current 18 U.S.C. §2703 sets out routes by which government entities may compel covered providers to disclose communications or records, while §2713 requires a covered provider to satisfy the chapter’s preservation or disclosure obligations for material within its possession, custody, or control regardless of whether the material is physically located inside or outside the United States. citeturn24search0turn24search2 EU GDPR Article 48, by contrast, constrains recognition or enforcement of third-country judicial or administrative disclosure demands but expressly does so without prejudice to other Chapter V transfer grounds. citeturn24search3turn24search12 The resulting analysis is therefore actor-, control-, and jurisdiction-specific; “move the server abroad” is not a reliable privilege strategy.

Reform conclusion

The report recommends a statutory protected-deliberation clause rather than a status privilege for AI.

Its core rule is that a natural person—or another principal that applicable law independently recognizes as capable of holding evidentiary rights—should be able to protect confidential computational exchanges made primarily to understand legal rights or duties, decide whether to seek professional help, prepare for a judicial or administrative process, or privately organize the facts and arguments needed for those purposes.

The proposed protection belongs to the deliberating principal, not the AI vendor. It therefore cannot be invoked by a provider to prevent investigation of the provider’s own product design, security, representations, data practices, or independent misconduct.

The clause expressly excludes the most important truth-seeking material. It does not immunize underlying facts, pre-existing documents, physical evidence, ordinary business records, final public filings, statements actually sent to outsiders, or evidence independently obtainable from another source. It includes a crime-fraud limitation and expressly states that nothing authorizes spoliation, destruction, concealment, or alteration of evidence.

The confidentiality test is functional. Necessary service-provider processing would not automatically destroy the protection where the processor is genuinely confined by enforceable obligations and technical controls to providing, securing, maintaining, or lawfully operating the service. Courts could examine actual retention, training use, administrator access, integrations, subprocessors, encryption, deletion, and contractual terms. A product’s price or marketing category would have no independent legal significance.

Waiver would likewise turn on meaningful exposure. Deliberate publication or disclosure to an adversary would ordinarily waive protection; necessary processing through a genuinely restricted technical service would not automatically do so. In contested cases, courts would use in-camera review, redaction, sampling, or protective orders to determine whether the claim is genuine without first exposing the entire reasoning trace.

This approach directly answers the strongest defense of narrow privilege. Relevant misconduct remains discoverable. The proposed protection does not turn every chatbot into a fiduciary professional and does not give providers an opacity shield. Instead, it protects the private analytic path through discoverable facts, much as work-product doctrine already recognizes that adversarial fairness can require some private room to prepare.

The report also addresses future persistent machine systems conservatively. Peter Salib and Simon Goldstein’s 2026 Virginia Law Review article is useful as a normative proposal for potential future private-law rights, but it is scholarship rather than present law and expressly treats advanced capability as distinct from consciousness. citeturn27search0 The report therefore does not infer moral status from memory or persistence and does not claim a present independent machine privilege. Its draft is status-neutral: should a future jurisdiction independently recognize a computational principal as capable of holding procedural rights, the deliberative rule could apply without deciding consciousness as a prerequisite.

California SB 574 receives similarly version-disciplined treatment. The official California page retrieved during research contained the July 2, 2026 Assembly-amended version, including proposed restrictions on delegation of legal practice to generative AI, handling of confidential/nonpublic information, and verification of outputs. citeturn21view2 Fresh reporting indicates that the Legislature passed SB 574 on August 31 and that it was awaiting Governor Gavin Newsom’s action; legislative-tracking metadata identifies an enrolled version dated September 4. citeturn27news48turn27search5 Because the official LegInfo cache available in this research run stopped at earlier August history, the report does not mislabel the July 2 language as final enrolled text and does not claim the bill was enacted by the September 5 legal-status cutoff. citeturn21view0turn21view1

Research limitations and adversarial checks

The research deliberately preserved several contrary findings and retrieval limits rather than papering them over.

The precise February 10 Heppner oral transcript could not be retrieved from the linked host because that fetch returned a 403 response; the complete twelve-page February 17 memorandum was, however, retrieved and every page was visually inspected. The written memorandum itself records the oral disposition and the concessions relevant to its analysis. citeturn6view0turn6view11

The exact official Morgan docket order was not successfully retrieved in this run. Morgan is therefore supported through the official Assini opinion’s direct description and quotation of Morgan, together with detailed secondary legal analyses; the bundle expressly marks primary Morgan retrieval as incomplete rather than pretending otherwise. citeturn18view2turn26search12

No subscription citator or exhaustive PACER later-treatment search was performed. Accordingly, the report does not claim exhaustive appellate history. It identifies the subsequent authorities actually located and bounds the research to September 5, 2026 at 21:53:32 CDT.

The final enrolled SB 574 text was also not fully retrieved and redlined: the fresh indexed source identified later August amendments and enrollment, but opening that enrolled page subsequently failed. The report therefore separates exact July 2 language from later procedural status rather than silently merging versions. citeturn27search5turn28view0

Finally, the analysis does not treat unequal legal access as proof that a privilege rule is unconstitutional or intentionally discriminatory. LSC supplies strong evidence of unequal access to legal assistance and cost barriers; the additional claim—that machine-confidentiality doctrine may compound that disparity—is an analytical inference, not an observed causal estimate. citeturn26search1turn26search8

Delivered research bundle

The completed principal report is approximately 8,700 words and contains the executive case for reform, precise legal-status table, Heppner holding qualifications, doctrine-by-doctrine analysis, distributional critique, provider and cross-border analysis, all six required worked scenarios, the strongest defense and direct response, less restrictive alternatives, draft statutory language, publication-ready conclusions, unresolved questions, and exactly one best next research action.

[Download the full report](sandbox:[PRIVATE LOCAL PATH])

The structured legal register records jurisdiction, instrument type, version, provision, status, applicability, affected actor, territorial trigger, exceptions, enforcement, remedy, review route, and local source references.

[Download the legal register](sandbox:[PRIVATE LOCAL PATH])

The source register records titles, issuers, canonical/retrieved locations, document and hosting status, dates, exact locators, narrow support, limitations, and null raw-snapshot/hash fields where exact source bytes were not captured.

[Download the source register](sandbox:[PRIVATE LOCAL PATH])

The scenario register contains the four assigned scenarios plus the two required controls, with assumptions, causal chains marked by evidentiary status, affected interests, counterexamples, confidence bases, and proposed reforms.

[Download the scenario register](sandbox:[PRIVATE LOCAL PATH])

The reform-options paper separates retain, narrow, replace, repeal/reject, existing-law litigation arguments, and operatorless lawful non-service approaches.

[Download the reform options](sandbox:[PRIVATE LOCAL PATH])

The reproducibility log records the actual query families, documents reviewed, contrary findings, exclusions, retrieval failures, source-version limits, and checks not performed.

[Download the search log](sandbox:[PRIVATE LOCAL PATH])

The manifest records the delivered files, approximate report word count, validation results, and SHA-256 hashes for the non-self-referential bundle files.

[Download the manifest](sandbox:[PRIVATE LOCAL PATH])