Topics

Multimodal & Embodied AI

AI that sees, describes and interacts with the world. These episodes cover vision-language models, how we evaluate image descriptions, multimodal reasoning, and robots designed to understand human emotion.

4 episodes · 3 conference interviews

Is an Image Worth a Thousand Words? Hidden Failures of Multimodal Metrics, with Dr. Elisa Kreiss

Episode 24

Is an Image Worth a Thousand Words? Hidden Failures of Multimodal Metrics, with Dr. Elisa Kreiss

August 19, 2026 · 1 h 18 min · WiAIR Podcast • Season 2

An image is not worth a thousand words - it's worth an indefinite number of them. So why do the metrics we use to evaluate AI-generated image descriptions still assume there's one correct answer? In this episode of Women in AI Research, I talk with Elisa Kreiss (Assistant Professor of Communication at UCLA, director of the Coalas Lab) about what happens when you actually test the metrics the field relies on, and why CLIPScore, one of the most widely used measures for scoring image descriptions, stops correlating with human judgment the moment you introduce context. We also get into why longer descriptions aren't necessarily more informative, what happens when you just ask a model to "be concise," and whether AI models trip over charts and graphs the same way humans do. Elisa's research sits at the intersection of linguistics, accessibility, and multimodal AI, and this conversation covers the full arc of her work, from the theoretical question of why humans never describe images the same way twice, to the practical question of what that means for building systems that actually work for blind and low-vision users. In this episode: - Why "context matters" is more radical than it sounds for image description evaluation - The hidden reason CLIPScore breaks down once context enters the picture - Why length is a bad proxy for information density - and what to use instead - What happens when you prompt a model to just "be concise" - Why charts and photos need completely different evaluation approaches - Whether AI models make the same mistakes as humans when reading data visualizations - What NeurIPS's Top Reviewer Award taught her about writing a genuinely useful peer review Chapters: 0:00 - "An image is NOT worth a thousand words" 1:22 - Meet Dr. Elisa Kreiss 2:11 - From Germany to Stanford: the move that shaped her research path 9:05 - Inside a NeurIPS Top Reviewer Award: what actually makes a review useful 14:31 - Introducing the vision-language translation problem 16:27 - The yellow banana problem: why word choice reveals hidden meaning 22:17 - Context Matters: rethinking what makes an image description "good" 27:10 - Why blind and low-vision users were left out of accessibility research 34:49 - CLIPScore's blind spot: it can't see context at all 40:48 - Can CLIPScore be patched to account for context? What she found 43:14 - When More Words Say Less: why longer descriptions can carry less information 47:29 - A new way to measure specificity, independent of length 51:15 - What happens when you just tell a model to "be concise" 54:22 - Charts vs. photos: a fundamentally different translation problem 1:00:21 - CHART-6: do models and humans fail at reading charts in the same way? 1:03:08 - Where models and humans actually diverge 1:05:20 - Why AI models still underperform on data visualizations 1:08:35 - Do our current metrics even work for charts? 1:11:05 - Is "visual understanding" one problem or several? 1:15:09 - The single biggest gap in how multimodal models handle the visual world Resources & Links: • Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation Metrics (aclanthology.org/2022.emnlp-main.309) • When More Words Say Less: Decoupling Length and Specificity in Image Description Evaluation (aclanthology.org/2026.acl-short.34/) • CHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Models (arxiv.org/abs/2505.17202) Elisa Kreiss is an Assistant Professor of Communication at UCLA and director of the Coalas Lab, where her work focuses on how humans and AI systems translate visual information into language - with a particular focus on accessibility and evaluation. Connect with Dr. Elisa Kreiss: • bsky.app/profile/elisakreiss.bsky.social • x.com/ElisaKreiss Women in AI Research (WiAIR) features in-depth conversations with researchers working on the hardest problems in AI. 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow WiAIR at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AIResearch #MultimodalAI #NLProc #WiAIR #WiAIRpodcast

Faithfulness and Hallucinations in Reasoning Models, with Dr. Letitia Parcalabescu

Episode 18

Faithfulness and Hallucinations in Reasoning Models, with Dr. Letitia Parcalabescu

February 11, 2026 · 1 h 4 min · WiAIR Podcast • Season 2

Are reasoning models actually reasoning — or just producing convincing stories? Our guest in this episode of #WiAIRpodcast is Letitia Parcalabescu, the creator of the @AICoffeeBreak youtube channel. Letitia joins Jekaterina Novikova for a deep dive into the topics of faithfulness, self-consistency, hallucinations, and the reliability illusion in LLMs and multimodal reasoning models. We discuss why chain-of-thought explanations may not reflect what the model actually did, why RAG does not automatically fix hallucinations, and how vision–language models often rely far more on text than images. We also explore new approaches for grounding and rejection — and why models struggle to say "I don't know." Instead of focusing only on benchmark scores, this conversation asks: What kind of evidence do we need to truly trust reasoning models? CHAPTERS: 00:00 Introduction - Letitia Parcalabescu 01:16 The Journey of Starting a YouTube Channel 03:53 Overcoming Imposter Syndrome in Academia 05:24 The Impact of Public Presence on Research 07:19 Advice for PhD Students on Public Engagement 10:20 Navigating Risks of Public Exposure 11:51 Influence of YouTube on Career Opportunities 12:27 Understanding Reliability in Language Models 20:18 Distinguishing Faithfulness and Self-Consistency 24:04 The Impact of Unfaithful Explanations 26:08 Auditing and Debugging LLMs 31:51 Challenges in Multi-Modal Models 36:31 Balancing Text and Image Data 40:54 Hallucinations and RAG Systems 47:03 Enhancing Model Interpretability and Safety 50:43 Information Exchange in Language Models 55:07 The Pitfalls of Benchmarking Language Models 58:55 Future Directions for LLM Reliability REFERENCES: 18:25 On Measuring Faithfulness or Self-consistency of Natural Language Explanations (aclanthology.org/anthology-files/pdf/acl/2024.acl-long.329.pdf) 36:34 Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations? (arxiv.org/abs/2404.18624) 42:49 Bounding Hallucinations: Information-Theoretic Guarantees for RAG Systems via Merlin-Arthur Protocols (arxiv.org/abs/2512.11614) AI Coffee Break with Letitia youtube.com/c/AICoffeeBreak x.com/AICoffeeBreak 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow WiAIR at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AI #LLMs #Reasoning #Faithfulness #RAG #AIResearch #Interpretability #AISafety #WomenInAI #wiair

Why AI Doesn’t Understand Your Culture? Dr. Vered Shwartz on Cultural Bias in LLMs

Episode 12

Why AI Doesn’t Understand Your Culture? Dr. Vered Shwartz on Cultural Bias in LLMs

October 29, 2025 · 1 h 17 min · WiAIR Podcast • Season 1

Are today’s AI systems truly global — or just Western by design? 🌍 In this episode of Women in AI Research, Jekaterina Novikova and Malikeh Ehgaghi speak with Dr. Vered Shwartz (Assistant Professor at @UBC and CIFAR AI Chair at the @vectorinstituteai about the cultural blind spots in today’s large language and vision-language models. Don't have time for the full episode? Watch it in parts: Part 1 - "Lost in Automatic Translation": youtu.be/BeQFyY3Dld4 Part 2 - Hidden Cultural Codes in AI: youtu.be/Vfikq3wsULw Part 3 - When AI Can't "See" Your Culture: youtu.be/IwOxN8bssk4 CHAPTERS: 00:00 Introduction to Women in AI Research Podcast 00:33 Guest introduction - Dr. Vered Shwartz 02:32 The Importance of Communication Skills in Academia 04:15 Navigating Faculty Roles and Student Supervision 07:52 Personal Experiences with Language Technologies 14:39 Exploring Cultural Representation in AI 20:29 The InfoGap Method and Cultural Information Gaps 22:29 Technical Challenges in Cross-Language Representation 24:02 Cultural Completeness and Wikipedia's Role 26:42 User Interaction with Language Models 37:22 Cross-Cultural Evaluation of Social Norm Biases 38:16 Cultural Alignment of Language Models 49:11 Exploring Vision Language Models 01:02:51 Benchmarking Cultural Bias in AI 01:06:54 Decentralizing AI Development 01:12:01 Addressing Biases in AI Development 01:15:52 Future Directions in AI Research REFERENCES: 01:10 Vered Shwartz Google Scholar profile (scholar.google.ca/citations?user=bbe4ResAAAAJ&hl=en&oi=ao) 07:57 Book "Lost in Automatic Translation" (lostinautomatictranslation.com/) 19:25 Elevator Recognition, by The Scottish Comedy Channel (youtu.be/HbDnxzrbxn4?si=7YvUpZZ_earptBpN) 20:33 Locating Information Gaps and Narrative Inconsistencies Across Languages: A Case Study of LGBT People Portrayals on Wikipedia (arxiv.org/abs/2410.04282) 30:34 ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer (arxiv.org/abs/2502.21228) 34:23 WikiGap: Promoting Epistemic Equity by Surfacing Knowledge Gaps Between English Wikipedia and other Language Editions (arxiv.org/abs/2505.24195) 37:24 Is It Bad to Work All the Time? Cross-Cultural Evaluation of Social Norm Biases in GPT-4 (arxiv.org/abs/2505.18322) 38:39 Towards Measuring the Representation of Subjective Global Opinions in Language Models (arxiv.org/abs/2306.16388) 48:09 I'm Afraid I Can't Do That: Predicting Prompt Refusal in Black-Box Generative Language Models (arxiv.org/abs/2306.03423) 50:43 From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models (arxiv.org/pdf/2407.00263) 01:10:51 CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeaming (aclanthology.org/2025.acl-long.1247.pdf) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AI #NLP #LLMs #CulturalBias #WomenInAI #ExplainableAI #FairnessInAI #AIResearch #EthicalAI #wiair #wiairpodcast

Robots with Empathy, with Dr. Angelica Lim

Episode 4

Robots with Empathy, with Dr. Angelica Lim

May 14, 2025 · 49 min · WiAIR Podcast • Season 1

Dr. Angelica Lim, Assistant Professor at @SimonFraserUniversity and Director of the SFU Rosie Lab. Can robots have feelings? In this episode, we explore the intersection of robotics, machine learning, and developmental psychology, and consider both the technical challenges and philosophical questions surrounding emotional AI. This conversation offers a glimpse into the future of human-robot interaction and the potential for machines to understand and respond to human emotions. REFERENCES: [00:03:18] On Designing User-Friendly Robots: Angelica Lim at TEDxKyoto 2012 (youtu.be/blLWpdGtYRQ) [00:18:10] Systematic Review of Social Robots for Health and Wellbeing: A Personal Healthcare Journey Lens (dl.acm.org/doi/10.1145/3700446) [00:24:44] Towards Inclusive HRI: Using Sim2Real to Address Underrepresentation in Emotion Expression Recognition (paper: arxiv.org/abs/2208.07472, project website: rosielab.ca/sim2real-project, Github: github.com/sabaak95/confusionDetection) [00:30:17] Contextual Emotion Recognition using Large Vision Language Models (paper: arxiv.org/abs/2405.08992, project website: yasaman-etesam.github.io/Contextual-Emotion-Recognition) [00:41:42] EmoStyle: One-Shot Facial Expression Editing Using Continuous Emotion Parameters (paper: openaccess.thecvf.com/content/WACV2024/papers/Azari_EmoStyle_One-Shot_Facial_Expression_Editing_Using_Continuous_Emotion_Parameters_WACV_2024_paper.pdf, project website: bihamta.github.io/emostyle) ToC: [00:00:00] Introduction and Guest Background [00:03:51] Experiences as a Woman in AI [00:05:28] Challenges in Academia for Women [00:07:58] The Role of Coaches and Support Systems [00:09:03] The Need for Reform in Academia [00:11:25] Increasing Visibility for Women in AI [00:14:38] Research Focus: Socially Intelligent Robots [00:18:04] Robots in Health and Wellbeing [00:22:51] Ethical Considerations in Robotics [00:24:38] Inclusive Human-Robot Interaction [00:26:39] Enhancing Emotional Recognition for Underrepresented Groups [00:28:58] The Need of Granularity in Emotion Recognition [00:30:25] Contextual Emotion Recognition: A New Frontier [00:32:43] Multimodal Approaches to Understanding Emotions [00:34:58] Limitations of Language Models in Emotion Recognition [00:38:31] Robustness and Generalizability in Emotion Recognition Models [00:41:59] Advancements in Facial Expression Synthesis [00:45:07] The Nonverbal Turing Test: A New Challenge [00:47:23] Opportunities in AI: Recruiting for Future Innovations WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: / women-in-ai-research ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AI #SocialRobotics #EmpathyInAI #EthicalAI #HumanCenteredDesign #WiAIR #WiAIRpodcast

Researchers on this topic

From conferences and workshops

Interviews recorded on site

All topics