Anthropic's latest 'capability switch' design is quite interesting: modular control of double-edged sword knowledge, more resistant to fine-tuning attacks than post-hoc forgetting, adding another layer of insurance to AI safety.

CoinWorld news: Anthropic and AE Studio released new research, attempting to install a switch for AI's "dual-use knowledge." Dual-use knowledge refers to capabilities that can be used for legitimate research but also may be misused, such as virology, cybersecurity, and nuclear physics. The research team tested four types of dual-use domains in real-data experiments: virology, cybersecurity, nuclear physics, and niche programming languages. The four modules can be combined into 16 capability configurations. Compared to training a separate model for each configuration, this method requires only one training. Results show that Gram, while retaining general capabilities, can relatively well remove target capabilities, and its resistance to small-scale malicious fine-tuning is stronger than the post-training "forgetting" method.