Bottom line: Reverse Direct Preference Optimization (rDPO) enables removal of specific moderation policies from model parameters while preserving general capabilities and alignment in other areas.
Amazon introduces with Nova a method for targeted adjustment of content moderation in foundation models. Organizations can thereby selectively adapt security barriers without retraining the entire model.
Foundation models feature content moderation embedded during alignment training. However, these safeguards can also block legitimate, business-critical use cases: a security team generating phishing emails for training purposes, a legal team processing sensitive evidence, or cybersecurity experts conducting threat simulations receive rejections despite their defensive intent. Prompt engineering alone cannot overcome these deflection responses embedded in the model parameters.
Amazon Nova addresses this through Customizable Content Moderation Settings (CCMS), which enable approved customers to selectively adjust safeguards across four responsibility categories: Safety (dangerous activities, weapons, drugs), Sensitive Content (profanity, nudity, harassment), Fairness (bias, cultural aspects), and Security (malware, malicious code). Amazon maintains non-configurable core controls, such as child protection and data privacy, as mandatory.
The technical foundation is unlearning via Reverse Direct Preference Optimization (rDPO). Low-Rank Adaptation (LoRA) adapters are trained to reverse alignment behavior for specific policies. While standard approaches such as Negative Preference Optimization (NPO) only train the model away from deflection behavior, rDPO actively teaches the model alternative high-quality answers by reversing the preference pairs in the DPO process. This avoids quality degradation and preserves general capabilities such as instruction following, code generation, and mathematical reasoning.
Customers import the customized LoRA adapter and receive their own model variant with a unique Amazon Resource Name (ARN). During inference, the adapter directs the core model away from deflection behavior in the approved policy areas, while output moderation systems are configured accordingly. This keeps the model fully aligned outside the adjustment areas.
Source: aws.amazon.com · Published July 7, 2026
Lumi AI News — AI-assisted curation pursuant to Art. 50 EU AI Act. Paraphrase and classification via Lumi News Pipeline v1.7.3.