Behind the Scenes of AI Security: Red Teaming Strategies and Innovations

Name: Behind%20the%20Scenes%20of%20AI%20Security:%20Red%20Teaming%20Strategies%20and%20Innovations
Uploaded: 2025-01-14

Posted Jan 14, 2025 | Views 131

speaker

Ruchir Patwa

VP of Engineering @ Protect AI

Ruchir Patwa is a seasoned security leader with deep expertise in AI security, privacy, and product engineering. He spent six years at Google in the US and India, spearheading critical security initiatives before serving as the CISO for MPL, India’s largest gaming platform. Driven by a passion for enabling secure adoption of AI in enterprises, he co-founded SydeLabs, which was acquired by Protect AI in 2024. He now leads engineering and product efforts for Recon, Protect AI’s automated AI red teaming platform.

+ Read More

SUMMARY

Learn about the bleeding-edge of Generative AI security in this live stream featuring Ruchir Patwa, former co-founder and CEO of SydeLabs and now VP of Engineering at Protect AI. Learn about the evolving practice of red teaming for AI—exploring innovative strategies, practical insights, and the intersection of tools, processes, and people in securing AI/ML systems.

+ Read More

TRANSCRIPT

This is a recording of the session we held on January 8, 2025 with Ruchir Patwa.

Ruchir Patwa is a seasoned security leader with deep expertise in AI security, privacy, and product engineering. He spent six years at Google in the US and India, spearheading critical security initiatives before serving as the CISO for MPL, India’s largest gaming platform. Driven by a passion for enabling secure adoption of AI in enterprises, he co-founded SydeLabs, which was acquired by Protect AI in 2024. He now leads engineering and product efforts for Protect AI Recon - Automated Red Teaming for GenAI.

Session references & resources (including time stamp from video mention):

(04:44) Foundation model - "What is a Foundation Model? An Explainer for Non-Experts"

(05:38) MITRE ATLAS (TM) - (Adversarial Threat Landscape for Artificial-Intelligence Systems)

(07:13) Learn more about threat modeling for AI systems in Chapter 2 of the free "MLSecOps Foundations" certification course - "Applying MLSecOps to Secure the AI Lifecycle."

(07:55) Real world scenarios:

"Air Canada ordered to pay customer who was misled by airline’s chatbot" - The Guardian

"Man who exploded Cybertruck in Las Vegas used ChatGPT in planning, police say" - NPR

(09:01) Retrieval Augmented Generation "Rag Security 101" (blog)

(09:18) AI Agents "AI Agents: Chapter 3 - Practical Approaches to AI Agents Security" (blog)

(10:15) Automated red teaming & human-led red teaming for AI "Why Automated Red Teaming is Essential for GenAI Security" (blog)

(16:46) Jump to this minute mark to hear Ruchir talk about Recon and its ability to evaluate the safety and security of GenAI systems. Comprehensive scanning of risks and vulnerabilities to ensure the safety and security of GenAI systems, at every step of development. Recon uses both an attack library and an LLM agent based solution for red teaming.

(23:00) Jump to this minute mark to learn about top risks that AI architecture should address in GenAI use case implementations.

(25:25) Garbage in, garbage out? "AI hallucinations: what business leaders should know" - PwC

(33:15) Jump to this minute mark to learn more about AI Security vs. AI Safety.

(36:10) LearnPrompting.org Podcast episode with CEO of Learn Prompting, Sander Schulhoff: "Generative AI Prompt Hacking and Its Impact on AI Security & Safety"

+ Read More