Topics

Language, Cognition & the Brain

What can neuroscience and linguistics tell us about language models, and what can language models tell us about the mind? These conversations bridge cognitive science, the brain's language network and modern AI.

4 episodes · 2 conference interviews

Can AI Learn Languages Humans Can't? Impossible Languages &  Tokenization, with Julie Kallini

Episode 26

Can AI Learn Languages Humans Can't? Impossible Languages & Tokenization, with Julie Kallini

November 4, 2026 · 1 h 16 min · WiAIR Podcast • Season 2

Can a language model learn a language no human ever could? Stanford PhD researcher Julie Kallini put Noam Chomsky's claim to the test, and won the ACL 2024 Best Paper Award for it. In this episode of Women in AI Research, Julie takes us from "impossible languages" to the hidden costs of tokenization, and shows how byte-level models could make LLMs faster, fairer across languages, and smarter about where they spend compute. We also talk about choosing a PhD advisor, leaving a software engineering job at Meta for a PhD, and what doing research at FAIR is really like. In this episode: → Why it would be unethical to raise a child on an invented language, but not a language model → Why linguists still can't agree on what makes a language "impossible" → Why ChatGPT couldn't count the R's in "strawberry", and what tokenization has to do with it → How tokenization can make the same request cost more in some languages than others → MrT5: teaching a byte-level model to decide which bytes it can safely drop → Fast BLT: speeding up byte-level generation with diffusion and self-speculation → Diffusion vs. autoregressive generation: practical advice for practitioners → What's next: adaptive compression, or why spend dozens of tokens on the MIT license? 📄 Papers discussed • Mission: Impossible Language Models (ACL 2024, Best Paper): arxiv.org/abs/2401.06416 • MrT5: Dynamic Token Merging for Efficient Byte-level Language Models (ICLR 2025): arxiv.org/abs/2410.20771 • Fast Byte Latent Transformer: arxiv.org/abs/2605.08044 • Byte Latent Transformer: Patches Scale Better Than Tokens (Meta FAIR): arxiv.org/abs/2412.09871 • Information Locality as an Inductive Bias for Neural Language Models (Someya et al., ACL 2025): arxiv.org/abs/2506.05136 👩‍🔬 About Julie Kallini Julie is a fourth-year Computer Science PhD student at Stanford, advised by Chris Potts and Dan Jurafsky. Her research spans computational linguistics, tokenization, efficient language model architectures, and byte-level modeling. ⏱️ Chapters 00:00 Finding the right PhD advisor 05:41 Painting as a creative outlet 07:40 From Meta software engineer to Stanford PhD 13:32 Can language models teach us how humans learn language? 18:57 Mission: Impossible Language Models (ACL 2024 Best Paper) 24:16 What an "impossible language" looks like 27:44 Why linguists can't define "impossible" 30:25 Could you build impossible languages from scratch? 33:30 From linguistics to tokenization 36:40 The hidden costs of subword tokenization 41:29 MrT5: dynamic token merging for byte-level models 46:26 How MrT5 learns what to delete 50:10 How close are byte-level models to everyday use? 52:37 Fast Byte Latent Transformer: diffusion & self-speculation 58:41 Research scientist at FAIR vs. academia 1:02:12 Diffusion vs. autoregressive: advice for practitioners 1:05:13 What people overlook about tokenization 1:08:48 What's next: adaptive compression 🎙️ About WiAIR Women in AI Research (WiAIR) features in-depth conversations with women shaping AI research: their work, their ideas, and the paths that led them there. 🔔 Subscribe for new episodes #AIResearch #Tokenization #LLM

What Makes a Sentence Memorable? Inside Language, Memory and LLMs, with Dr. Greta Tuckute

Episode 25

What Makes a Sentence Memorable? Inside Language, Memory and LLMs, with Dr. Greta Tuckute

September 16, 2026 · 1 h 17 min · WiAIR Podcast • Season 2

Why don't bigger LLMs look more like the human brain? In the brain's language network, today's large models explain only slightly more variance than GPT-2 XL - and the reason says a lot about what those brain regions actually do. Dr. Greta Tuckute (Research Fellow at the Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard) joins Jekaterina Novikova on Women in AI Research to unpack what brain-LLM alignment does and does not tell us. Greta works where neuroscience, cognitive science and AI meet - and when asked to keep only one of those three labels, AI is the first one she drops. The conversation covers how brain alignment develops over training, how sparse autoencoders can turn the "you are just comparing one black box to another" critique into something testable, and what memory experiments reveal about how meaning is stored - from why "pineapple" sticks in memory and "light" does not, to which sentence embeddings predict what people remember. In this episode: - Brain alignment tracks formal linguistic competence, not reasoning - and it emerges after not much more than a developmentally plausible ~100M tokens, not 300B - Why better next-word prediction stops meaning "more brain-like" once a model has mastered language - Sparse autoencoder features plus surprisal: for one frontal brain region, surprisal alone does almost as well as 32,000 SAE features, while some voxels are captured by just six features - Brains and LLMs share the main, high-variance features of language - not the idiosyncratic ones - Why learning from BPE tokens puts brain-LLM comparisons on "pretty shaky ground", and what should come next - A distinctive meaning makes words and sentences memorable - and SBERT predicts human sentence memory better than the other embedding models tested - From running a photography business at 14 to a PhD at MIT, and why trying the other path first was worth it CHAPTERS 00:00 Preview 01:30 Greta Tuckute. Growing up between languages and countries 04:53 Neuroscience, AI or cognitive science: why AI is the label she'd drop 11:10 From a photography business at 14 to a research career 16:54 The unicorn demo that pulled her into language neuroscience 19:05 Why AI researchers should care how brains process language 21:32 From Language to Cognition (EMNLP 2025): alignment across training 24:40 You don't need 300B tokens to get a brain-like LLM 25:57 How LLMs outgrow the human language network 27:39 What "brain alignment" actually measures 31:16 Why better next-word prediction stops meaning more brain-like 33:39 Bigger models, barely better brain predictions 36:27 Isn't this just comparing two black boxes? 38:57 One number vs 32,000 SAE features: the surprisal result 41:35 Surprisal, and why fMRI can't resolve single words 44:58 Do brains and LLMs share the big features or the quirks? 50:23 What makes a representation "good" - and the BPE token problem 54:24 Memory as a window into meaning 57:14 Pineapple vs light: why some words stick 59:59 Misremembering her own paper title (which proves the point) 1:01:09 "Olive oil works well for tanning": what makes a sentence memorable 1:03:33 Which embeddings predict what people remember? 1:06:29 Can unusual syntax make a sentence memorable? 1:09:22 Open question: how do individual brains differ? 1:11:55 From static snapshots to dynamic transformations 1:12:55 What the answers would change, and where AI still struggles 1:15:04 Wrap-up PAPERS DISCUSSED • From Language to Cognition: How LLMs Outgrow the Human Language Network - aclanthology.org/2025.emnlp-main.1237/ • Interpreting Brain Responses to Language with Sparse Features from Language Models - arxiv.org/abs/2606.06857 • Driving and suppressing the human language network using large language models - nature.com/articles/s41562-023-01783-7 • Intrinsically memorable words have unique associations with their meanings - doi.org/10.1037/xge0001742 • A distinctive meaning makes a sentence memorable - doi.org/10.1016/j.jml.2025.104700 MENTIONED IN THIS EPISODE • Anna Ivanova on formal vs functional linguistic competence (WiAIR): youtu.be/e36ryy0Dsdo GRETA TUCKUTE Website: tuckute.com Bluesky: bsky.app/profile/gretatuckute.bsky.social X: x.com/GretaTuckute Women in AI Research (WiAIR) is a podcast and YouTube channel where Jekaterina Novikova talks with women doing AI research about their work, the questions driving it, and the paths that brought them there. #LLM #Neuroscience #AIResearch #WiAIR #WiAIRpodcast

Can Language Alone Create Intelligence? Insights from Neuroscience and AI, with Dr. Anna Ivanova

Episode 22

Can Language Alone Create Intelligence? Insights from Neuroscience and AI, with Dr. Anna Ivanova

June 17, 2026 · 1 h 12 min · WiAIR Podcast • Season 2

Do large language models truly understand language—or are they sophisticated pattern matchers? In this conversation, Dr. Anna Ivanova (Asst. Prof. at Georgia Tech) explores one of the important questions in AI: the relationship between language, thought, and intelligence. Drawing from neuroscience, cognitive science, and AI research, Anna explains why language understanding is harder to define than most people realize, why reasoning and language are not the same thing, and what today's LLMs can and cannot tell us about human cognition. Key Topics: • Do LLMs understand language or merely generate convincing text? • The difference between formal and functional linguistic competence • What LLMs can learn from language alone—and what they cannot • Why human cognition and AI cognition may be fundamentally different • Theory of mind, reasoning, and common misconceptions about AI capabilities • How cognitive scientists evaluate the "thinking" abilities of LLMs • What neuroscience can teach AI researchers about interpretability • Why understanding AI requires studying both behavior and internal representations • The future of multimodal models and AI cognition Chapters 00:00 Introduction. The Interdisciplinary Journey of Anna Ivanova 08:22 Navigating the Lab: Surprises and Lessons Learned 12:48 Mentorship in Academia: A Two-Way Street 17:03 The Challenges of Interdisciplinary Research 21:34 Understanding Language: The Role of LLMs 26:27 Dissociating Language and Thought in LLMs 39:00 Understanding the Gap: Humans vs. LLMs 41:07 The Role of Data in Learning 45:27 Beyond Next-Word Prediction: New Training Methods 50:01 Evaluating Cognitive Abilities of LLMs 56:05 LLMs as Models of Human Cognition 01:02:02 Exploring Internal Representations in LLMs 01:09:49 Interdisciplinary Insights: AI and Human Cognition Resources & Links: • What does it mean to understand language? (arxiv.org/abs/2511.19757) • Dissociating language and thought in large language models (static1.squarespace.com/static/64c800a2f333f04f50bf2020/t/66f46f57900cdf4b7060e3ec/1727295320856/Mahowald_Ivanova_et_al_2024_TiCS.pdf) • How to evaluate the cognitive abilities of LLMs (drive.google.com/file/d/1OPCTpkluqs8xY3Bs6_--18eXU9u06oEN/view) • How Do LLMs Use Their Depth? (arxiv.org/abs/2510.18871) • True Lens (github.com/AlignmentResearch/tuned-lens) Connect with Dr. Anna Ivanova: • bsky.app/profile/neuranna.bsky.social • x.com/neuranna 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow WiAIR at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast

Do LLMs Understand Meaning? Neuroscience, Evaluation, and the Future of AI, with Dr. Maria Ryskina

Episode 15

Do LLMs Understand Meaning? Neuroscience, Evaluation, and the Future of AI, with Dr. Maria Ryskina

December 31, 2025 · 1 h 4 min · WiAIR Podcast • Season 1

Do large language models actually understand meaning — or are we over-interpreting impressive behavior? In this episode, we speak with Dr. Maria Ryskina, CIFAR AI Safety Postdoctoral Fellow at the Vector Institute for AI, whose research bridges neuroscience, cognitive science, and artificial intelligence. Together, we unpack what the brain can (and cannot) teach us about modern AI systems — and why current evaluation paradigms may be missing something fundamental. We explore how language models can predict brain activity in regions linked to visual processing, what this reveals about cross-modal knowledge, and why scale alone may not resolve deeper conceptual gaps in AI. The conversation also tackles the growing importance of interpretability, especially as AI systems become more embedded in high-stakes, real-world contexts. Beyond technical questions, Maria shares why community matters in AI research, particularly for underrepresented groups — and how diversity directly shapes the kinds of scientific questions we ask and the systems we ultimately build. CHAPTERS 00:00 Introduction - Dr. Maria Ryskina 02:42 Community Building in AI 10:40 Representation, Lived Experience, and AI Research 14:17 AI Meets Cognitive Neuroscience 23:04 Brain Alignment and Conceptual Meaning in Language Models 30:38 World Knowledge and the EWoK Framework 42:37 Evaluating What Models Know 51:15 Biological Inspiration, Alignment, and the Future of AI REFERENCES 11:41 Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification (proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf) 12:36 Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Models (aclanthology.org/2023.findings-acl.502) 17:38 Language models align with brain regions that represent concepts across modalities (arxiv.org/abs/2508.11536) 30:53 Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models (Paper - direct.mit.edu/tacl/article/doi/10.1162/TACL.a.38/133506, project - ewok-core.github.io) 49:13 Prompting is not a substitute for probability measurements in large language models (aclanthology.org/2023.emnlp-main.306/) 50:04 Auxiliary task demands mask the capabilities of smaller language models (openreview.net/forum?id=U5BUzSn4tD) 🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI. WiAIR website: ♾️ women-in-ai-research.github.io Follow us at: ♾️ LinkedIn: linkedin.com/company/women-in-ai-research/ ♾️ Bluesky: bsky.app/profile/wiair.bsky.social ♾️ X (Twitter): x.com/WiAIR_podcast #ArtificialIntelligence #LanguageModels #AIResearch #Neuroscience #Interpretability #AIEvaluation #CognitiveScience #LLMs #AISafety #ResponsibleAI #WomenInAI #AIResearchers #ResearchPodcast #wiair #wiairpodcast

Researchers on this topic

From conferences and workshops

Interviews recorded on site

All topics