Episode 19
Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen
March 4, 2026 · 60 min · WiAIR Podcast • Season 2
In this conversation, Dr. Hila Gonen (Assistant Professor at the University of British Columbia) joins us to explore the deep insights into how large language models (LLMs) leak semantic information, behave across languages, and how researchers can uncover their root causes. Dr. Gonen shares her journey in interpreting AI systems, addressing biases, and controlling model outputs for safer, fairer applications. In this episode: • The influence of prompt elements, like colour, on model predictions • How semantic leakage impacts model outputs unintentionally • The role of multilinguality and modality in model safety and behaviour • Interventional vs. observational approaches to understanding models • Challenges in controlling and aligning AI behavior across languages and domains • Future directions in model interpretability, safety, and causal analysis Key Topics: • Color and semantic influence on language model completions • The concept of semantic leakage and examples from real prompts • Differences between bias, hallucination, and leakage failures • Unintended behaviours discovered through experimentation • The importance of model interpretability and transparency • Roots of behaviour: training data and internal representations • Interventional analysis as a causal tool in NLP research • Cross-lingual and cross-modal alignment in safety detection • Challenges in evaluating safety across languages and modalities • Strategies for building robust controls against unseen attack types • The future of AI research: combining performance with reliability and safety • Ethical considerations: avoiding directions that hinder societal benefit Timestamps: 00:00 Introduction to Hila and Her Background 02:24 The Intersection of Academia and Industry 05:14 Multilingualism and Its Influence on Research 07:34 Focusing on Complex Problems in AI 12:06 Understanding Language Models and Their Behavior 12:48 The Challenge of Model Transparency 14:39 Defining Semantic Leakage 18:25 Evaluating Model Behavior and Bias 24:29 Interventional Analysis in AI Research 38:48 Cross-lingual and multi-modality safety evaluations and their importance 43:14 Controlling LLM generation 45:10 How low-resource languages and modalities pose safety challenges 50:49 The persistent safety gaps across languages and modalities 54:47 Building future-proof control mechanisms against unknown attack vectors 56:26 The evolving focus of AI safety: performance, factuality, fairness, and reliability 57:49 Should AI researchers continue working on LLMs or not? 59:01 Dr. Gonen’s outlook on applying language models in healthcare Resources & Links: • Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (arxiv.org/pdf/1903.03862) • Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models (aclanthology.org/2025.naacl-long.35) • Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior(arxiv.org/abs/2510.14261) • OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Modalities and Languages (arxiv.org/abs/2505.23856) Connect with Dr. Hila Gonen: • linkedin.com/in/hila-gonen-076ab7105/ • x.com/hila_gonen Note: This episode emphasizes practical and theoretical challenges in model interpretability, safety, bias detection, and causality—providing a comprehensive view suitable for researchers, practitioners, and AI enthusiasts interested in responsible AI development.










