Share research related to AI & machine learning security

Research

"Recent studies developed jailbreaking attacks, which construct jailbreaking prompts to ``fool'' LLMs into responding to harmful questions. Early-stage jailbreaking attacks require access to model internals or significant human efforts. More advanced attacks utilize genetic algorithms for automatic and black-box attacks. However, the random nature of genetic algorithms significantly limits the effectiveness of these attacks. In this paper, we propose RLbreaker, a black-box jailbreaking attack driven by deep reinforcement learning (DRL). We model jailbreaking as a search problem and design an RL agent to guide the search, which is more effective and has less randomness than stochastic search, such as genetic algorithms. Specifically, we design a customized DRL system for the jailbreaking problem, including a novel reward function and a customized proximal policy optimization (PPO) algorithm. Through extensive experiments, we demonstrate that RLbreaker is much more effective than existing jailbreaking attacks against six state-of-the-art (SOTA) LLMs. We also show that RLbreaker is robust against three SOTA defenses and its trained agents can transfer across different LLMs. We further validate the key design choices of RLbreaker via a comprehensive ablation study."

When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search by Xuan Chen, Yuzhou Nie, Wenbo Guo, Xiangyu Zhang

Home

Forum

Events

Content

People

Contributors

Albums

Clubs

Chat

Channels

Resources

Help

Welcome to the MLSecOps Community, the premier hub for educational information about building security into AI and machine learning life cycles end-to-end. We look forward to meeting you!

MLSecOps Community

Job Title

Login to the MLSecOps Community community to connect with others, attend community events, and more!

Login or Sign Up for even more

To unlock all parts of the community and get the best experience, complete your profile.

Hey, complete your profile

Check out the additional resources below for more educational content; including videos, blogs, playbooks, and more!