Topics

Interpretability & Explainability

What is happening inside a model, and can it explain its own decisions? These conversations cover explainable AI, the faithfulness of model explanations, and methods for uncovering the root causes of model behaviour.

4 episodes · 4 conference interviews

Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen

Episode 19

Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen

March 4, 2026 · 60 min · WiAIR Podcast • Season 2

In this conversation, Dr. Hila Gonen (Assistant Professor at the University of British Columbia) joins us to explore the deep insights into how large language models (LLMs) leak semantic information, behave across languages, and how researchers can uncover their root causes. Dr. Gonen shares her journey in interpreting AI systems, addressing biases, and controlling model outputs for safer, fairer applications. In this episode: • The influence of prompt elements, like colour, on model predictions • How semantic leakage impacts model outputs unintentionally • The role of multilinguality and modality in model safety and behaviour • Interventional vs. observational approaches to understanding models • Challenges in controlling and aligning AI behavior across languages and domains • Future directions in model interpretability, safety, and causal analysis Key Topics: • Color and semantic influence on language model completions • The concept of semantic leakage and examples from real prompts • Differences between bias, hallucination, and leakage failures • Unintended behaviours discovered through experimentation • The importance of model interpretability and transparency • Roots of behaviour: training data and internal representations • Interventional analysis as a causal tool in NLP research • Cross-lingual and cross-modal alignment in safety detection • Challenges in evaluating safety across languages and modalities • Strategies for building robust controls against unseen attack types • The future of AI research: combining performance with reliability and safety • Ethical considerations: avoiding directions that hinder societal benefit Timestamps: 00:00 Introduction to Hila and Her Background 02:24 The Intersection of Academia and Industry 05:14 Multilingualism and Its Influence on Research 07:34 Focusing on Complex Problems in AI 12:06 Understanding Language Models and Their Behavior 12:48 The Challenge of Model Transparency 14:39 Defining Semantic Leakage 18:25 Evaluating Model Behavior and Bias 24:29 Interventional Analysis in AI Research 38:48 Cross-lingual and multi-modality safety evaluations and their importance 43:14 Controlling LLM generation 45:10 How low-resource languages and modalities pose safety challenges 50:49 The persistent safety gaps across languages and modalities 54:47 Building future-proof control mechanisms against unknown attack vectors 56:26 The evolving focus of AI safety: performance, factuality, fairness, and reliability 57:49 Should AI researchers continue working on LLMs or not? 59:01 Dr. Gonen’s outlook on applying language models in healthcare Resources & Links: • Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (arxiv.org/pdf/1903.03862) • Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models (aclanthology.org/2025.naacl-long.35) • Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior(arxiv.org/abs/2510.14261) • OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Modalities and Languages (arxiv.org/abs/2505.23856) Connect with Dr. Hila Gonen: • linkedin.com/in/hila-gonen-076ab7105/ • x.com/hila_gonen Note: This episode emphasizes practical and theoretical challenges in model interpretability, safety, bias detection, and causality—providing a comprehensive view suitable for researchers, practitioners, and AI enthusiasts interested in responsible AI development.

Can We Trust AI Explanations? Dr. Ana Marasović on AI Trustworthiness, Explainability & Faithfulness

Episode 11

Can We Trust AI Explanations? Dr. Ana Marasović on AI Trustworthiness, Explainability & Faithfulness

October 8, 2025 · 1 h 23 min · WiAIR Podcast • Season 1

In this conversation, Jekaterina Novikova and Malikeh Ehgaghi interview Ana Marasović anamarasovic.com, an expert in AI trustworthiness, focusing on the complexities of explainability, the realities of academic research, and the dynamics of human-AI collaboration. We discuss the importance of intrinsic and extrinsic trust in AI systems, the challenges of evaluating AI performance, and the implications of synthetic evaluations. Don't have time for the full episode? Watch it in parts: Part 1 - The Messy Journey of Research: youtu.be/KszW2czg7x8?si=UXKicrI-m8N4VhjO Part 2 - The Role of Explainability and Reasoning Traces in AI Trust: youtu.be/yQN6PuI0DB0 Part 3 - Robustness and Trustworthy AI: youtu.be/kh7N0GtgfpE CHAPTERS 00:00 Introduction - Ana Marasović 02:15 The Messiness of Research 04:33 The Realities of Academic Life 08:21 Balancing Research and Funding 12:13 Understanding Trust in AI Systems 15:23 Exploring Explainability and Human Decision-Making 17:52 Application-Grounded Evaluations 22:09 The Gap Between Research and Real-World Applications 23:41 Counterintuitive Findings in Human-AI Collaboration 29:16 Balancing Human-AI Collaboration 31:01 Exploring Human-AI Teams 35:53 Understanding Reasoning Traces in AI 39:40 The Relationship Between Model Size and Reasoning 46:54 Evaluating Reasoning Language Models 50:54 Robustness in AI Systems 57:52 The State of AI Evaluation 01:01:31 Synthetic Evaluation: Pros and Cons 01:06:39 Human vs. LLM Annotations 01:08:41 Balancing Efficiency and Robustness in Benchmarking 01:14:55 Trust in AI: Intrinsic vs. Extrinsic 01:19:05 Future Directions in AI Research REFERENCES: 17:56 On Evaluating Explanation Utility for Human-AI Decision Making in NLP (aclanthology.org/2024.findings-emnlp.439/) 31:02 Effective Human-AI Teams via Learned Natural Language Rules and Onboarding (arxiv.org/abs/2311.01007) 41:10 Chain-of-Thought Unfaithfulness as Disguised Accuracy (arxiv.org/abs/2402.14897) 41:52 On Measuring Faithfulness or Self-consistency of Natural Language Explanations (arxiv.org/abs/2311.07466) 01:01:51 What Has Been Lost with Synthetic Evaluation? (arxiv.org/abs/2505.22830) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AI #airesearch #aiexplainability #aitrust #aifaithfulness #wiair #wiairpodcast

Unlocking LLM Reasoning, with Simeng Sophia Han

Episode 9

Unlocking LLM Reasoning, with Simeng Sophia Han

August 27, 2025 · 54 min · WiAIR Podcast • Season 1

How can we go beyond accuracy to truly understand large language models? In this episode of the Women in AI Research podcast, hosts Jekaterina Novikova and Malikeh Ehghaghi sit down with Simeng Sophia Han (PhD candidate at @yale, Research Scientist Intern at @meta , ex @googledeepmind, ex @amazon aws) to explore the future of 𝐋𝐋𝐌 𝐫𝐞𝐚𝐬𝐨𝐧𝐢𝐧𝐠, 𝐞𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧, 𝐚𝐧𝐝 𝐞𝐱𝐩𝐥𝐚𝐢𝐧𝐚𝐛𝐥𝐞 𝐀𝐈. 🌟 What you’ll learn in this episode: • Why evaluating reasoning goes beyond correctness • How brain teasers uncover hidden strengths and weaknesses of LLMs • The importance of symbolic reasoning for complex problem solving • The role of mentorship and early research experiences in shaping careers • Why consistency in AI outputs is essential for building trust • How humans combine brute force and intuition — and what this means for AI ToC: 00:00 Introduction to LLM Reasoning and Evaluation 02:36 Simeng Sophia Han's Research Journey 06:26 Reflections on Early Research Experiences 11:25 Understanding LLM Reasoning Beyond Accuracy Metrics 16:16 Exploring Brain Teasers in LLM Reasoning 22:25 Example of Mathematical Problem Solving with Constraints 24:16 Cognitive Science Insights for Language Models 29:13 Challenges in Human-Written Reasoning Chains 32:16 Explaining the Black Box of LLMs 37:28 Consistency and Trustworthiness in AI Models 39:17 Symbolic Reasoning in LLMs 42:00 Neuro-Symbolic Reasoning Approaches 46:33 Future Directions in AI Research 49:11 Advice for Women in AI Research REFERENCES: 01:26 Simeng Sophia Han - Google Scholar profile (scholar.google.ca/citations?hl=en&user=D0dpploAAAAJ) 11:40 Creativity or Brute Force? Using Brainteasers as a Window into the Problem-Solving Abilities of Large Language Models (arxiv.org/abs/2505.10844) 25:35 HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning (arxiv.org/abs/2409.19381) 29:25 P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (arxiv.org/abs/2410.09207) 39:05 Folio: Natural Language Reasoning with First-Order Logic (arxiv.org/abs/2209.00840) 41:54 HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning (arxiv.org/abs/2409.19381) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #LLM #AIResearch #ExplainableAI #Reasoning #aireasoning #MachineLearning #CognitiveScience #SymbolicReasoning #WiAIRPodcast #WiAIR

Interpretable AI, with Dr. Faiza Khan Khattak

Episode 5

Interpretable AI, with Dr. Faiza Khan Khattak

June 4, 2025 · 58 min · WiAIR Podcast • Season 1

How can we build AI systems that are fair, explainable, and truly responsible? In this episode of the #WiAIR podcast, we sit down with Dr. Faiza Khan Khattak, the CTO of an innovative AI startup, with a rich background in both academia and industry. From fairness in machine learning to the realities of ML deployment in healthcare, this conversation is packed with insights, real-world challenges, and powerful reflections. ToC: [00:00] Intro [01:37] Faiza’s Journey in AI [07:50] Exploring Fairness in AI [11:17] The Role of Mentorship [13:27] Transitioning Between Academia, Industry, and Entrepreneurship [17:02] Challenges Faced by Women in AI [22:01] MLHOps in Healthcare [24:30] Operationalizing ML Models in Healthcare [27:25] Exploring Explainability in AI Models [31:31] Risks of AI in Healthcare [33:23] Addressing Social Bias in AI [38:40] The Role of Reasoning in Interpretability [41:04] Generalizability of Reasoning Across Models [46:25] Machine Unlearning and Bias Mitigation [52:57] Adversarial Attacks and Bias Induction REFERENCES: [21:40] MLHOps: Machine Learning Health Operations (ieeexplore.ieee.org/abstract/document/10811924) [33:33] Using Chain-of-Thought Prompting for Interpretable Recognition of Social Bias (openreview.net/forum?id=QyRganPqPz) [35:09] Dialectic Preference Bias in Large Language Models (ojs.aaai.org/index.php/AAAI-SS/article/view/35613) [41:42] The Impact of Unstated Norms in Bias Analysis of Language Models (arxiv.org/abs/2404.03471) [46:26] Can Machine Unlearning Reduce Social Bias in Language Models? (aclanthology.org/2024.emnlp-industry.71/) [52:58] BiasKG: Adversarial Knowledge Graphs to Induce Bias in Large Language Models (arxiv.org/abs/2405.04756v1) 👉 Whether you're an AI researcher, a developer working on LLMs, or someone passionate about Responsible AI, this episode is for you. 📌 Subscribe to hear more inspiring stories and cutting-edge ideas from women leading the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #WomenInAI #WiAIR #WiAIRpodcast #ResponsibleAI #FairnessInAI #AIHealthcare #ExplainableAI #LLMs #AIethics #BiasMitigation #MachineUnlearning #InterpretableAI #AIstartup #AIforGood

Researchers on this topic

From conferences and workshops

Interviews recorded on site

All topics