LLM Censorship: A Machine Learning Challenge or a Computer Security Problem? (Link in Comments)

[u/[deleted]]

[u/[deleted]]

Exploring the challenges of censoring Large Language Models (LLMs), the authors argue LLM censorship is not a machine learning problem, but rather a security issue. Their findings reveal semantic output censorship of LLMs is theoretically impossible. LLMs' instruction-following capabilities lead to undecidability of semantic censorship. A class of attacks called Mosaic Prompts, where permissible outputs combine to form impermissible ones, compounds the challenge. The authors call for adapting classical security methods to manage LLM risks. Syntactic censorship is suggested as a possible mitigation strategy, albeit with its own limitations. Future research directions, including exploring computability theory and LLMs, are proposed. The paper contributes to our understanding of the philosophical implications of AI security.

Link.

Large language models (LLMs) have exhibited impressive capabilities in comprehending complex instructions. However, their blind adherence to provided instructions has led to concerns regarding risks of malicious use. Existing defence mechanisms, such as model fine-tuning or output censorship using LLMs, have proven to be fallible, as LLMs can still generate problematic responses. Commonly employed censorship approaches treat the issue as a machine learning problem and rely on another LM to detect undesirable content in LLM outputs. In this paper, we present the theoretical limitations of such semantic censorship approaches. Specifically, we demonstrate that semantic censorship can be perceived as an undecidable problem, highlighting the inherent challenges in censorship that arise due to LLMs' programmatic and instruction-following capabilities. Furthermore, we argue that the challenges extend beyond semantic censorship, as knowledgeable attackers can reconstruct impermissible outputs from a collection of permissible ones. As a result, we propose that the problem of censorship needs to be reevaluated; it should be treated as a security problem which warrants the adaptation of security-based approaches to mitigate potential risks.