Topics

AI Safety & Alignment

What does it take for AI systems to behave as intended, even under pressure? These episodes cover jailbreaks and misuse, sycophancy and alignment failures, and the research on controlling model behaviour in the real world.

5 episodes · 9 conference interviews

Is Your AI Just Flattering You? Sycophancy, AI Policies, and More, with Dr. Malihe Alikhani

Episode 23

Is Your AI Just Flattering You? Sycophancy, AI Policies, and More, with Dr. Malihe Alikhani

July 22, 2026 · 1 h 8 min · WiAIR Podcast • Season 2

Only 19% of Americans say AI has actually improved their productivity - so why the gap between the hype and reality? In this episode of Women in AI Research, Dr. Malihe Alikhani (Northeastern University, Contextual AI Lab) unpacks the hidden failures in how we build and deploy AI: why sycophancy is really a collapse of alignment, why bigger models aren't better aligned, and why "thin" alignment breaks down in the real world. Chapters 00:00 "It's scary how much these models just want to keep us happy" 02:01 Dr. Malihe Alikhani - From advising Congress to the UN to leading her own lab 05:51 Inside the Contextual AI Lab: meaning is a function of context 09:35 Interactive inference: teaching models to act under uncertainty 11:42 What makes a paper stand out: the path to the Best Paper Award 16:21 Why building better systems is never enough 22:18 "Breaking the AI Mirror": sycophancy as a collapse of alignment 27:03 Sycophancy as a productivity problem — and cutting it by 60% 31:00 Bayesian rationality, AI literacy & who's responsible 34:51 How to actually write your paper's ethics section (from an ACL ethics chair) 41:52 "Hype & Harm": thin vs. thick alignment 45:03 Why alignment is NOT an emergent property of scale 47:08 Fairness as adaptive focus & AI policy across borders 52:28 "How Are Americans Using AI?": adoption without benefit 54:12 The 19% productivity gap vs. the industry hype 57:27 Who actually benefits: fit vs. access & how workers really feel 1:04:05 Bridging the technical and policy worlds Resources & Links: • Breaking the AI Mirror: Sycophancy, productivity, and the future of collaboration (brookings.edu/articles/breaking-the-ai-mirror/) • Hype and harm: Why we must ask harder questions about AI and its alignment with human values (brookings.edu/articles/hype-and-harm-why-we-must-ask-harder-questions-about-ai-and-its-alignment-with-human-values/) • How are Americans using AI? Evidence from a nationwide survey (brookings.edu/articles/how-are-americans-using-ai-evidence-from-a-nationwide-survey/) Connect with Dr. Malihe Alikhani: • x.com/malihealikhani 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow WiAIR at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast

100% Jailbreak Success? The Hard Truth About AI Safety, with Dr. Saadia Gabriel (Part 2)

Episode 21

100% Jailbreak Success? The Hard Truth About AI Safety, with Dr. Saadia Gabriel (Part 2)

April 17, 2026 · 34 min · WiAIR Podcast • Season 2

What actually happens when AI systems fail in the real world? In this final part of our conversation with Saadia Gabriel (UCLA), we unpack one of the most urgent challenges in modern AI: why even the most advanced models remain vulnerable to manipulation - and what that means for safety, fairness, and society. From multi-turn jailbreaking attacks with near 100% success rates to misinformation shaping human beliefs, this conversation goes beyond surface-level concerns and dives into how harms actually emerge in deployed systems. We explore: • Why current guardrails are not enough • How realistic attack scenarios differ from academic benchmarks • The connection between model vulnerabilities and societal harm • What AI can (and cannot) do about misinformation and persuasion • The open research problems that still don’t have solutions Timestamps: 00:00 — Introduction of Saadia Gabriel and episode overview 00:45 — From Toy Attacks to Real-World AI Weaponization 02:20 — Misinformation in the Age of Generative AI 03:00 — How AI Can Model Human Beliefs & Manipulation 06:50 — Why Facts Don’t Work: The Need for Empathetic AI 09:40 — Can AI Persuade Us Toward Critical Thinking? 11:30 — Toxicity Detection Is Broken (And Why It Matters) 14:40 — Hidden Bias: In-Group vs Out-Group Perspectives 16:10 — Fixing Data: Representation & Community-Centered AI 20:50 — Are We Measuring AI Wrong? Benchmarks vs Reality 23:30 — Multilingual AI: The Evaluation Crisis Nobody Talks About 26:50 — What Translation Misses: Culture, Safety & Blind Spots 29:10 — The Future: Multilingual Safety & Adversarial Robustness Resources & Links: • Generative AI in the Era of 'Alternative Facts' (aclanthology.org/2024.emnlp-main.487/) • ModelCitizens: Representing Community Voices in Online Safety (aclanthology.org/2025.emnlp-main.1571/) • Translation as a Scalable Proxy for Multilingual Evaluation (arxiv.org/abs/2601.11778) Connect with Dr. Saadia Gabriel: • x.com/GabrielSaadia • bsky.app/profile/skgabrie.bsky.social

From Hate Speech to Best Paper: Building Safer AI Systems, with Dr. Saadia Gabriel (Part 1)

Episode 20

From Hate Speech to Best Paper: Building Safer AI Systems, with Dr. Saadia Gabriel (Part 1)

April 15, 2026 · 29 min · WiAIR Podcast • Season 2

What does it mean to build AI systems we can actually trust? In this first part of our conversation with Saadia Gabriel (UCLA), we explore the deeply personal and technical journey behind her work on AI safety, misuse, and responsible NLP. From experiencing targeted hate speech firsthand to receiving a best paper nomination, Saadia shares how her lived experience shaped her research — and why language models must be designed with both capability and risk in mind. 🧠 In this episode, we cover: • How personal experiences influence AI research directions • The intersection of NLP, security, and privacy • Why LLMs can be both powerful and dangerous • What it means to build trustworthy AI systems • Lessons from working across multiple research paradigms • How to pursue high-impact research as a PhD or early-career scientist Timestamps: 00:00 - Introduction of Saadia Gabriel and episode overview 00:54 - Influence of dual mentorship on research approach 03:42 - The purpose and vision of the MARS lab 06:23 - Women and underrepresented groups' focus on harm and toxicity issues 11:02 - AI safety vs. AI ethics: interconnected but distinct realms 14:33 - How the concept of AI safety is evolving with adversarial testing 15:56 - The origins of multi-turn jailbreaking research 19:21 - Why multi-turn attacks outperform single-turn methods 22:28 - Role and impact of optimization in attack depth and breadth 24:40 - Strategies for mitigating conversational and multi-turn system vulnerabilities Resources & Links: • X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents (arxiv.org/abs/2504.13203) Connect with Dr. Saadia Gabriel: • x.com/GabrielSaadia • bsky.app/profile/skgabrie.bsky.social

Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen

Episode 19

Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen

March 4, 2026 · 60 min · WiAIR Podcast • Season 2

In this conversation, Dr. Hila Gonen (Assistant Professor at the University of British Columbia) joins us to explore the deep insights into how large language models (LLMs) leak semantic information, behave across languages, and how researchers can uncover their root causes. Dr. Gonen shares her journey in interpreting AI systems, addressing biases, and controlling model outputs for safer, fairer applications. In this episode: • The influence of prompt elements, like colour, on model predictions • How semantic leakage impacts model outputs unintentionally • The role of multilinguality and modality in model safety and behaviour • Interventional vs. observational approaches to understanding models • Challenges in controlling and aligning AI behavior across languages and domains • Future directions in model interpretability, safety, and causal analysis Key Topics: • Color and semantic influence on language model completions • The concept of semantic leakage and examples from real prompts • Differences between bias, hallucination, and leakage failures • Unintended behaviours discovered through experimentation • The importance of model interpretability and transparency • Roots of behaviour: training data and internal representations • Interventional analysis as a causal tool in NLP research • Cross-lingual and cross-modal alignment in safety detection • Challenges in evaluating safety across languages and modalities • Strategies for building robust controls against unseen attack types • The future of AI research: combining performance with reliability and safety • Ethical considerations: avoiding directions that hinder societal benefit Timestamps: 00:00 Introduction to Hila and Her Background 02:24 The Intersection of Academia and Industry 05:14 Multilingualism and Its Influence on Research 07:34 Focusing on Complex Problems in AI 12:06 Understanding Language Models and Their Behavior 12:48 The Challenge of Model Transparency 14:39 Defining Semantic Leakage 18:25 Evaluating Model Behavior and Bias 24:29 Interventional Analysis in AI Research 38:48 Cross-lingual and multi-modality safety evaluations and their importance 43:14 Controlling LLM generation 45:10 How low-resource languages and modalities pose safety challenges 50:49 The persistent safety gaps across languages and modalities 54:47 Building future-proof control mechanisms against unknown attack vectors 56:26 The evolving focus of AI safety: performance, factuality, fairness, and reliability 57:49 Should AI researchers continue working on LLMs or not? 59:01 Dr. Gonen’s outlook on applying language models in healthcare Resources & Links: • Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (arxiv.org/pdf/1903.03862) • Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models (aclanthology.org/2025.naacl-long.35) • Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior(arxiv.org/abs/2510.14261) • OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Modalities and Languages (arxiv.org/abs/2505.23856) Connect with Dr. Hila Gonen: • linkedin.com/in/hila-gonen-076ab7105/ • x.com/hila_gonen Note: This episode emphasizes practical and theoretical challenges in model interpretability, safety, bias detection, and causality—providing a comprehensive view suitable for researchers, practitioners, and AI enthusiasts interested in responsible AI development.

AI Safety Beyond Benchmarks --  Dr. Swabha Swayamdipta on Evaluation, Personalization, and Control

Episode 17

AI Safety Beyond Benchmarks -- Dr. Swabha Swayamdipta on Evaluation, Personalization, and Control

January 21, 2026 · 1 h 1 min · WiAIR Podcast • Season 2

As language models become more capable, the hardest questions are no longer just about performance, but about safety, interpretation, and control. In this episode of Women in AI Research, we speak with Swabha Swayamdipta, Assistant Professor of Computer Science at the University of Southern California and co-Associate Director of the USC Center for AI and Society. Swabha’s research examines how the design and deployment of language models intersect with real-world risks — from how models behave in unexpected ways to how seemingly technical choices can have broader societal consequences. We talk about AI safety from multiple angles: what it means when hidden inputs to models can sometimes be inferred from their outputs, why personalization introduces new trade-offs around privacy and user agency, and how assumptions about model behavior can quietly shape downstream harms. Rather than focusing only on accuracy or benchmarks, the conversation asks what kinds of evidence we actually need to trust these systems in practice. CHAPTERS 00:00 Swabha's Journey into NLP Research 04:25 Navigating Career Challenges and Building Networks 08:58 The Importance of AI Safety and Reliability 10:49 Addressing Security and Privacy Concerns in Language Models 13:41 Innovations in Language Model Inversion 20:33 Balancing Personalization and Privacy in AI 27:08 Incorporating Psychological Scaffolds in Language Models 30:19 The Duality of AI: Enhancing Human Decision-Making 32:17 AI in Social Issues: Addressing Homelessness 35:18 OATH-Frames: Analyzing Public Sentiment on Homelessness 46:40 Suicide Prevention: AI's Role in Critical Interventions 56:12 The Responsibility of AI Researchers: Balancing Capability and Safety REFERENCES 13:52 Better Language Model Inversion by Compactly Representing Next-Token Distributions (arxiv.org/abs/2506.17090) 27:18 Improving Language Model Personas via Rationalization with Psychological Scaffolds (ui.adsabs.harvard.edu/abs/2025arXiv250417993J/abstract) 35:21 OATH-Frames: Characterizing Online Attitudes Towards Homelessness with LLM Assistants (arxiv.org/abs/2406.14883) 46:52 Uncovering Intervention Opportunities for Suicide Prevention with Language Model Assistants (arxiv.org/abs/2508.18541) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AISafety #LanguageModels #AIResearch #ResponsibleAI #WomenInAI #NLP #MachineLearning #AIAlignment #wiair #wiairpodcast

Researchers on this topic

From conferences and workshops

Interviews recorded on site

All topics