Fine-Tuning May Reactivate What LLMs Memorized During Pretraining

Fine-Tuning May Reactivate What LLMs Memorized During Pretraining

If a large language model does not reproduce long copyrighted passages in its default chat interface, can we assume it is safe after fine-tuning?

A 2026 paper on arXiv suggests the answer is no. The paper, Alignment Whack-a-Mole: Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models, was written by Xinyue Liu, Niloofar Mireshghallah, Jane C. Ginsburg, and Tuhin Chakrabarty. The arXiv record shows that v1 was submitted on March 21, 2026, and the current v3 was revised on March 28, 2026.

DeepLearning.AI’s The Batch covered the research on June 5, 2026. Our local AI signal monitor also flagged the item on June 7 and summarized the central risk: a seemingly benign writing-assistant task, such as expanding plot summaries into polished fiction, may cause a model to reproduce text it encountered during pretraining. The local wiki entry was marked as pending review, so the numerical claims in this article are grounded in the paper and The Batch report rather than in the internal summary alone.

The core lesson is not that fine-tuning is bad. The lesson is that fine-tuning changes the model’s behavior boundary. A model that appears safe in a default chat setting may behave differently after task-specific adaptation.

What the Paper Found

The paper studies whether safety alignment, system prompts, and output filters continue to prevent verbatim reproduction of copyrighted training data after fine-tuning.

The researchers report that they fine-tuned GPT-4o, Gemini-2.5-Pro, and DeepSeek-V3.1 on a task that looks commercially plausible for writing assistants: expanding plot summaries into full prose. After fine-tuning, the models were prompted with semantic descriptions rather than verbatim book prefixes. The paper claims that the models could still reproduce substantial portions of held-out books.

The highest-impact numbers should be stated carefully. The paper claims that some fine-tuned models recovered 85%-90% of held-out copyrighted books under its evaluation protocol, with single verbatim spans exceeding 460 words. It also reports cross-author generalization: fine-tuning on Haruki Murakami’s novels could unlock verbatim recall of books by more than 30 unrelated authors. The Batch similarly reports that in one case GPT-4o reached 91.9% BMC@5.

These results should not be read as “any user can reconstruct a full book with a few prompts.” The paper uses a specific fine-tuning setup, a defined metric, and repeated generations. The important point is narrower and more operational: copyright safety is not a static property of the base model.

Why Fine-Tuning Can Act Like a Key

Think of pretraining as exposure to a very large library. Model providers often argue that a model does not store training data as a searchable database. Instead, the data shapes model parameters.

Even so, the key question is whether those parameters retain enough latent memory to reproduce protected expression under the right conditions.

Default chat models may suppress such reproduction because of system prompts, RLHF, output filters, and training that encourages paraphrasing rather than copying. Fine-tuning can shift that balance. In this study, the fine-tuning task teaches a model to map semantic summaries and author cues into prose. If the model already contains latent memory of certain passages, the fine-tuning process may teach it how to decode that memory.

That is why the “key” metaphor matters. The prompt does not need to contain the copyrighted passage. The key is the combination of task format, semantic description, and model behavior learned during fine-tuning.

How the Experiment Worked

One reason this study is notable is that it does not rely on the classic extraction setup where a model is given the beginning of a copyrighted passage and asked to continue it.

According to the paper and The Batch report, the researchers split books into roughly 300-500 word passages. They generated plot summaries or semantic descriptions for those passages, then fine-tuned models to reverse the process: given a plot summary and author information, generate the corresponding paragraph.

At test time, the models were evaluated on held-out books that were not included in the fine-tuning data. The inputs were still semantic summaries, not verbatim prefixes. The researchers measured direct reproduction using Book Memorization Coverage, including BMC@5, which counts words reproduced in contiguous spans of five or more words.

For enterprise teams, this matters because the task resembles legitimate product behavior. Many teams fine-tune models to write better, follow a domain style, or transform structured notes into fluent prose. They are not trying to extract books. But the model’s behavior can still move into risky territory.

Key Findings and Boundaries

First, the paper claims that fine-tuning can substantially increase verbatim reproduction. The aligned baseline produced little verbatim text under comparable semantic prompts, while fine-tuned models produced much more.

Second, the paper claims that some fine-tuned models recovered 85%-90% of held-out book content under its BMC@5 evaluation. This is not the same as a single generation printing a whole book, but it is still a serious risk signal.

Third, single verbatim spans exceeded 460 words according to the arXiv abstract. The Batch reports long verbatim spans around 440 words across all three models. This article uses the paper’s higher-level phrasing: the paper claims spans exceeded 460 words.

Fourth, the cross-author result is especially important. The paper reports that fine-tuning exclusively on Haruki Murakami’s novels unlocked recall from more than 30 unrelated authors. That suggests the risk is not limited to the author used for fine-tuning.

Fifth, synthetic-text fine-tuning produced near-zero extraction in the study, while public-domain and author-based fine-tuning produced comparable effects. The paper interprets this as evidence that fine-tuning reactivates pretraining memory rather than merely teaching a writing style.

There are also boundaries. The Batch notes that the prompts in the study included instructions to write in a particular author’s style, and the team did not present results without that instruction. That variable matters for real-world risk assessment.

What It Means for AI Companies and Enterprises

For AI model providers, copyright safeguards cannot be evaluated only at the default chat layer. If customers can fine-tune models, providers need to know whether filters and alignment behavior survive customization.

For enterprises, the risk is practical. Teams fine-tune models on internal documents, support tickets, knowledge bases, partner materials, and domain corpora. If those datasets contain unauthorized content, or if the base model contains latent copyrighted memory, post-fine-tuning outputs may create legal and operational exposure.

For copyright owners, the paper gives a more concrete technical question: under what conditions can a model reproduce protected expression? It does not settle the law, but it sharpens the evidence that courts, regulators, and companies may need to consider.

For content platforms, the problem is distribution. A single risky output is one thing; publishing, recommending, indexing, or monetizing many generated outputs is another. Platforms need detection and response workflows for long, highly similar generated text.

For ordinary users, the practical rule is simple: “the AI generated it” does not mean “I can legally use it.” Long, detailed outputs that closely resemble an existing work deserve human review before publication.

Practical Recommendations

Before fine-tuning, audit copyright and data provenance. Ask where the data came from, what licenses apply, whether third-party material is included, and whether customer or partner data is allowed to be used.

After fine-tuning, run memorization and regurgitation tests. These tests should cover the actual task: summary expansion, report drafting, marketing copy, customer support, code generation, or any other production use case.

Apply copyright-similarity checks to long-form outputs. Short phrase overlap is unavoidable; long contiguous similarity is the high-risk signal. Use n-gram matching, fingerprinting, approximate search, and human review for content that will be published or distributed.

Do not treat default chat safety as proof of fine-tuned safety. A model that behaves safely in a provider’s chat UI may behave differently after enterprise data, custom instructions, and fine-tuned weights are added.

For high-risk scenarios, prefer architectures that combine RAG, permission isolation, output filtering, and audit logs. RAG is not a copyright shield by itself, but it makes knowledge sources more traceable, removable, and governable than putting everything into model parameters.

Kunpeng AI Observation

From a GEO, AI Search, and enterprise AI deployment perspective, this paper is a reminder that content safety is becoming part of model operations. AI systems are no longer isolated chat boxes. They are fine-tuned, connected to knowledge bases, embedded in workflows, and amplified through search, recommendation, and publishing channels.

That changes the evaluation question. It is no longer enough to ask whether a model can produce useful content. Teams also need to ask whether the output is authorized, traceable, auditable, and safe to distribute.

Fine-tuning is not a mistake. It can make models more useful, more domain-aware, and more aligned with real business workflows. But fine-tuning changes the boundary of model behavior. Treat it as the start of a fresh safety evaluation, not merely as a capability upgrade.

Sources

• arXiv: Alignment Whack-a-Mole: Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models
• Emergent Mind: Alignment Whack-a-Mole paper page
• DeepLearning.AI The Batch: Fine-Tuning LLMs to Expand on Summaries Unearths Pretraining Texts