Breaking AI: Inside the Art of LLM Pen Testing
- Date:
- July 16, 2025
- Time:
- 2:00pm EST | 7:00pm BST

Large Language Models (LLMs) are reshaping enterprise technology and redefining what it means to secure software. But here’s the problem: most penetration testers are using the wrong tools for the job. Traditional techniques focus on exploits and payloads, assuming the AI is just another application. But it’s not.
In this session, Brian D., Security Consultant III at Bishop Fox, makes the case that effective LLM security testing is more about persuasion than payloads. Drawing on hands-on research and real-world client engagements, Brian reveals a new model for AI pen testing – one grounded in social engineering, behavioral manipulation, and even therapeutic dialogue.
You’ll explore Adversarial Prompt Exploitation (APE), a methodology that targets trust boundaries and decision pathways using psychological levers like emotional preloading, narrative control, and language nesting. This is not Prompt Injection 101 — it’s adversarial cognition at scale – using real-world case studies to demonstrate success.
This virtual sessionalso tackles key operational challenges: the limitations of static payloads and automation, the complexity of reproducibility, and how to communicate findings to executive and technical leadership.
Key Takeaways:
- Why conventional penetration testing methodologies fail on LLMs
- How attackers exploit psychological and linguistic patterns, not code
- Practical adversarial techniques: emotional preloading, narrative leading, and more
- Frameworks for simulating real-world threats to LLM-based systems
- How to think like a social engineer to secure AI
Who Should Watch:
This session is ideal for professionals involved in securing, testing, or developing AI systems, particularly those using large language models (LLMs). Penetration testers and red teamers will find it valuable as it introduces a new adversarial framework that goes beyond traditional payload-based approaches, focusing instead on behavioral manipulation and social engineering. AI/ML security practitioners and researchers will gain insight into emerging psychological attack techniques—such as emotional preloading and narrative control—that exploit how LLMs process language, not code. The virtual session also offers practical strategies and case studies, making it useful for developers seeking to better understand how attackers interact with their models. Additionally, CISOs and technical managers will benefit from discussions on the operational challenges of LLM security testing, such as reproducibility and how to communicate complex findings to leadership. Overall, this session provides a critical perspective for anyone working on the front lines of AI security.