Topics

Multilingual AI & Tokenization

Most AI is built for a handful of languages. These episodes look at multilingual and low-resource language models, how models behave across languages, and how tokenization shapes what a model can learn.

4 episodes · 6 conference interviews

Can AI Learn Languages Humans Can't? Impossible Languages &  Tokenization, with Julie Kallini

Episode 26

Can AI Learn Languages Humans Can't? Impossible Languages & Tokenization, with Julie Kallini

November 4, 2026 · 1 h 16 min · WiAIR Podcast • Season 2

Can a language model learn a language no human ever could? Stanford PhD researcher Julie Kallini put Noam Chomsky's claim to the test, and won the ACL 2024 Best Paper Award for it. In this episode of Women in AI Research, Julie takes us from "impossible languages" to the hidden costs of tokenization, and shows how byte-level models could make LLMs faster, fairer across languages, and smarter about where they spend compute. We also talk about choosing a PhD advisor, leaving a software engineering job at Meta for a PhD, and what doing research at FAIR is really like. In this episode: → Why it would be unethical to raise a child on an invented language, but not a language model → Why linguists still can't agree on what makes a language "impossible" → Why ChatGPT couldn't count the R's in "strawberry", and what tokenization has to do with it → How tokenization can make the same request cost more in some languages than others → MrT5: teaching a byte-level model to decide which bytes it can safely drop → Fast BLT: speeding up byte-level generation with diffusion and self-speculation → Diffusion vs. autoregressive generation: practical advice for practitioners → What's next: adaptive compression, or why spend dozens of tokens on the MIT license? 📄 Papers discussed • Mission: Impossible Language Models (ACL 2024, Best Paper): arxiv.org/abs/2401.06416 • MrT5: Dynamic Token Merging for Efficient Byte-level Language Models (ICLR 2025): arxiv.org/abs/2410.20771 • Fast Byte Latent Transformer: arxiv.org/abs/2605.08044 • Byte Latent Transformer: Patches Scale Better Than Tokens (Meta FAIR): arxiv.org/abs/2412.09871 • Information Locality as an Inductive Bias for Neural Language Models (Someya et al., ACL 2025): arxiv.org/abs/2506.05136 👩‍🔬 About Julie Kallini Julie is a fourth-year Computer Science PhD student at Stanford, advised by Chris Potts and Dan Jurafsky. Her research spans computational linguistics, tokenization, efficient language model architectures, and byte-level modeling. ⏱️ Chapters 00:00 Finding the right PhD advisor 05:41 Painting as a creative outlet 07:40 From Meta software engineer to Stanford PhD 13:32 Can language models teach us how humans learn language? 18:57 Mission: Impossible Language Models (ACL 2024 Best Paper) 24:16 What an "impossible language" looks like 27:44 Why linguists can't define "impossible" 30:25 Could you build impossible languages from scratch? 33:30 From linguistics to tokenization 36:40 The hidden costs of subword tokenization 41:29 MrT5: dynamic token merging for byte-level models 46:26 How MrT5 learns what to delete 50:10 How close are byte-level models to everyday use? 52:37 Fast Byte Latent Transformer: diffusion & self-speculation 58:41 Research scientist at FAIR vs. academia 1:02:12 Diffusion vs. autoregressive: advice for practitioners 1:05:13 What people overlook about tokenization 1:08:48 What's next: adaptive compression 🎙️ About WiAIR Women in AI Research (WiAIR) features in-depth conversations with women shaping AI research: their work, their ideas, and the paths that led them there. 🔔 Subscribe for new episodes #AIResearch #Tokenization #LLM

Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen

Episode 19

Does Liking Yellow Make You a School Bus Driver? Hidden Failures in LLMs, with Dr. Hila Gonen

March 4, 2026 · 60 min · WiAIR Podcast • Season 2

In this conversation, Dr. Hila Gonen (Assistant Professor at the University of British Columbia) joins us to explore the deep insights into how large language models (LLMs) leak semantic information, behave across languages, and how researchers can uncover their root causes. Dr. Gonen shares her journey in interpreting AI systems, addressing biases, and controlling model outputs for safer, fairer applications. In this episode: • The influence of prompt elements, like colour, on model predictions • How semantic leakage impacts model outputs unintentionally • The role of multilinguality and modality in model safety and behaviour • Interventional vs. observational approaches to understanding models • Challenges in controlling and aligning AI behavior across languages and domains • Future directions in model interpretability, safety, and causal analysis Key Topics: • Color and semantic influence on language model completions • The concept of semantic leakage and examples from real prompts • Differences between bias, hallucination, and leakage failures • Unintended behaviours discovered through experimentation • The importance of model interpretability and transparency • Roots of behaviour: training data and internal representations • Interventional analysis as a causal tool in NLP research • Cross-lingual and cross-modal alignment in safety detection • Challenges in evaluating safety across languages and modalities • Strategies for building robust controls against unseen attack types • The future of AI research: combining performance with reliability and safety • Ethical considerations: avoiding directions that hinder societal benefit Timestamps: 00:00 Introduction to Hila and Her Background 02:24 The Intersection of Academia and Industry 05:14 Multilingualism and Its Influence on Research 07:34 Focusing on Complex Problems in AI 12:06 Understanding Language Models and Their Behavior 12:48 The Challenge of Model Transparency 14:39 Defining Semantic Leakage 18:25 Evaluating Model Behavior and Bias 24:29 Interventional Analysis in AI Research 38:48 Cross-lingual and multi-modality safety evaluations and their importance 43:14 Controlling LLM generation 45:10 How low-resource languages and modalities pose safety challenges 50:49 The persistent safety gaps across languages and modalities 54:47 Building future-proof control mechanisms against unknown attack vectors 56:26 The evolving focus of AI safety: performance, factuality, fairness, and reliability 57:49 Should AI researchers continue working on LLMs or not? 59:01 Dr. Gonen’s outlook on applying language models in healthcare Resources & Links: • Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them (arxiv.org/pdf/1903.03862) • Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models (aclanthology.org/2025.naacl-long.35) • Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior(arxiv.org/abs/2510.14261) • OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Modalities and Languages (arxiv.org/abs/2505.23856) Connect with Dr. Hila Gonen: • linkedin.com/in/hila-gonen-076ab7105/ • x.com/hila_gonen Note: This episode emphasizes practical and theoretical challenges in model interpretability, safety, bias detection, and causality—providing a comprehensive view suitable for researchers, practitioners, and AI enthusiasts interested in responsible AI development.

Multilingual AI, with Dr. Annie En-Shiun Lee

Episode 13

Multilingual AI, with Dr. Annie En-Shiun Lee

November 19, 2025 · 1 h 20 min · WiAIR Podcast • Season 1

Is English just one of the languages you speak? If so, the AI tools you use might miss things that makes your voice multilingual. In this episode of Women in AI Research, Jekaterina Novikova speaks with Dr. Annie En-Shiun Lee about her work on multilingual and multicultural AI — from the widening language gap and the lack of benchmarks for underrepresented languages, to why domain-specific data matters more than just scaling up models. We talk about the limits of cross-lingual transfer, the risks of English-centric reasoning in AI, and the technical, ethical, and cultural challenges of building models that truly serve global communities. Don't have time for the full episode? Watch it in parts: Part 1 - youtu.be/hkKF3fKr6zo Part 2 - youtu.be/XXeOAt2DO1I Part 3 - youtu.be/NJafotDn-J0 CHAPTERS: 00:00 Introduction to Women in AI Research 00:30 Guest introduction - Dr. Annie En-Shiun Lee 03:22 Annie's Journey in Multilingual AI 05:43 Balancing Parenthood and Academia 10:35 Challenges of Motherhood in Academia 19:28 Evaluating Multilingual Language Models 22:46 SIB-200 Dataset and Its Impact 33:17 Introducing a Multilingual Reward Model 38:48 The Impact of Model Size on Performance 42:29 Definition of Low-Resourse Languages 46:05 Focus on Non-English Languages and Their Specifics 48:16 The Pitfalls of English as a Pivot Language 58:42 Cross-Lingual Transfer in NLP: Current State and Challenges 01:08:13 Understanding Languages with Non-Latins Scripts 01:12:01 Advice for Researchers on Underrepresented Languages REFERENCES: 23:03 SIB-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects (aclanthology.org/2024.eacl-long.14.pdf) 27:13 URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Base (aclanthology.org/2025.coling-main.463.pdf) 33:23 mR3: Multilingual Rubric-Agnostic Reward Reasoning Models (arxiv.org/pdf/2510.01146) 40:36 ProxyLM: Predicting language model performance on multilingual tasks via proxy models (aclanthology.org/2025.findings-naacl.106.pdf) 46:29 ATAIGI: An AI-Powered Multimodal Learning App Leveraging Generative Models for Low-Resource Taiwanese Hokkien (aclanthology.org/2025.naacl-demo.2.pdf) 46:35 Enhancing Taiwanese Hokkien Dual Translation by Exploring and Standardizing of Four Writing Systems (aclanthology.org/2024.lrec-main.538.pdf) 01:02:43 AlignFreeze: Navigating the Impact of Realignment on the Layers of Multilingual Models Across Diverse Languages (aclanthology.org/2025.naacl-short.48.pdf) 01:08:51 Irokobench: A new benchmark for african languages in the age of large language models (aclanthology.org/2025.naacl-long.139.pdf) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #AI #NLP #LLMs #Multilingual #WomenInAI #FairnessInAI #AIResearch #EthicalAI #wiair #wiairpodcast

Decentralized AI, with Wanru Zhao

Episode 6

Decentralized AI, with Wanru Zhao

June 25, 2025 · 52 min · WiAIR Podcast • Season 1

🔍 𝐂𝐚𝐧 𝐭𝐡𝐞 𝐟𝐮𝐭𝐮𝐫𝐞 𝐨𝐟 𝐀𝐈 𝐛𝐞 𝐝𝐞𝐜𝐞𝐧𝐭𝐫𝐚𝐥𝐢𝐳𝐞𝐝? 𝐇𝐨𝐰 𝐝𝐨 𝐰𝐞 𝐬𝐜𝐚𝐥𝐞 𝐛𝐞𝐲𝐨𝐧𝐝 𝐬𝐜𝐚𝐥𝐢𝐧𝐠 𝐥𝐚𝐰𝐬? 𝐀𝐧𝐝 𝐰𝐡𝐚𝐭 𝐝𝐨𝐞𝐬 𝐢𝐭 𝐫𝐞𝐚𝐥𝐥𝐲 𝐭𝐚𝐤𝐞 𝐭𝐨 𝐛𝐮𝐢𝐥𝐝 𝐢𝐧𝐜𝐥𝐮𝐬𝐢𝐯𝐞, 𝐦𝐮𝐥𝐭𝐢𝐥𝐢𝐧𝐠𝐮𝐚𝐥 𝐋𝐋𝐌𝐬? In this episode of the #WiAIRpodcast, Wanru Zhao discusses decentralized and collaborative AI methods, the limitations of scaling laws, fine-tuning strategies, data attribution challenges in LLMs, and multilingual learning in federated settings—all while reflecting on her experiences across UK and Canadian research ecosystems. She also shares her academic journey, vision for the future of AI, and how she navigates research roadblocks with creativity and curiosity. 🧠 Whether you're building #llms, exploring #FederatedLearning, or just passionate about more inclusive and sustainable AI research—this episode is packed with insights, encouragement, and visionary thinking. 👉 Watch now and be part of the future of AI that’s collaborative, global, and radically inclusive. ToC: [00:00] Introduction to Wanru Zhao and Her Journey [02:56] Academic Pathway: From MPhil to PhD [06:29] Research Ecosystems: UK vs Canada [09:56] Collaboration Between Industry and Academia [12:46] Navigating Challenges in Research [16:13] Experiences of Being an Outsider in AI [18:37] Breaking Unspoken Rules in Academia [21:33] Scaling Laws in Language Models [25:57] Challenges in Data Attribution for LLMs [27:17] Leveraging High-Quality Public Data [30:37] Model Merging Techniques for Fine-Tuning [32:42] The big picture behind CLUES [35:17] Future Directions in Collaborative LLM Development [36:39] Incentivizing Contributions in Collaborative Learning [38:02] Challenges in Multilingual Fine-Tuning [45:30] Scaling Collaborative Learning Paradigms [46:36] Recent advances - Cascade inference [49:08] Advice for Young Researchers in AI REFERENCES: [00:27] Wanru Zhao - Google Scholar profile (scholar.google.com/citations?user=XrkIQQoAAAAJ&hl=en) [25:59] CLUES: Collaborative Private-domain High Quality Data Selection for LLMs via Training Dynamics (proceedings.neurips.cc/paper_files/paper/2024/file/ff80e644be415af4bcd7e4b4efb2152f-Paper-Conference.pdf) [38:07] Breaking Physical and Linguistic Borders: Multilingual Federated Prompt Tuning for Low-Resource Languages (openreview.net/pdf?id=HyRwexERAo) [45:19] Include: Evaluating Multilingual Language Understanding With Regional Knowledge (arxiv.org/abs/2411.19799) , and Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation (arxiv.org/abs/2504.07072) [46:42] CASCADIA: A Cascade Serving System for Large Language Models (arxiv.org/abs/2506.04203) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #FederatedLearning #AIResearch #LLM #WomenInAI #ScalingLaws #DecentralizedAI #MultilingualAI #NeurIPS2024 #ICLR2024 #DataCentricAI #ModelMerging #WiAIR #WiAIRpodcast #TechForInclusion #RepresentationMatters #MachineLearning #NLP #WomenInSTEM #OpenScience #VectorInstitute #UniversityOfCambridge #AWS #MicrosoftResearch

Researchers on this topic

From conferences and workshops

Interviews recorded on site

All topics