Welcome to
Women in AI Research

An image is not worth a thousand words - it's worth an indefinite number of them. So why do the metrics we use to evaluate AI-generated image descriptions still assume there's one correct answer?In this episode of Women in AI Research, I talk with Elisa Kreiss (Assistant Professor of Communication at UCLA, director of the Coalas Lab) about what happens when you actually test the metrics the field relies on, and why CLIPScore, one of the most widely used measures for scoring image descriptions, stops correlating with human judgment the moment you introduce context. We also get into why longer descriptions aren't necessarily more informative, what happens when you just ask a model to "be concise," and whether AI models trip over charts and graphs the same way humans do.Elisa's research sits at the intersection of linguistics, accessibility, and multimodal AI, and this conversation covers the full arc of her work, from the theoretical question of why humans never describe images the same way twice, to the practical question of what that means for building systems that actually work for blind and low-vision users.In this episode:Why "context matters" is more radical than it sounds for image description evaluationThe hidden reason CLIPScore breaks down once context enters the pictureWhy length is a bad proxy for information density - and what to use insteadWhat happens when you prompt a model to just "be concise"Why charts and photos need completely different evaluation approachesWhether AI models make the same mistakes as humans when reading data visualizationsWhat NeurIPS's Top Reviewer Award taught her about writing a genuinely useful peer reviewResources & Links:Context Matters for Image Descriptions for Accessibility: Challenges for Referenceless Evaluation MetricsWhen More Words Say Less: Decoupling Length and Specificity in Image Description EvaluationCHART-6: Human-Centered Evaluation of Data Visualization Understanding in Vision-Language Models🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI.Follow WiAIR at:⁠⁠LinkedIn⁠⁠⁠⁠Bluesky⁠⁠⁠⁠X (Twitter)⁠⁠⁠⁠⁠WiAIR website⁠

Only 19% of Americans say AI has actually improved their productivity - so why the gap between the hype and reality? In this episode of Women in AI Research, Dr. Malihe Alikhani (Northeastern University, Contextual AI Lab) unpacks the hidden failures in how we build and deploy AI: why sycophancy is really a collapse of alignment, why bigger models aren't better aligned, and why "thin" alignment breaks down in the real world.Key topicsThe impact of moving across different AI contexts on system designThe role of language as performative and active in shaping realityInteractive inference and uncertainty in AI systemsThe importance of context in meaning and system designAI policy, transparency, and societal impactSycophantic behavior in large language modelsMeasuring AI alignment: thin vs. thickAI adoption across sectors and demographic groupsThe role of policy in AI development and safetyEthical considerations in AI research and deploymentResources & Links:Breaking the AI Mirror: Sycophancy, productivity, and the future of collaborationHype and harm: Why we must ask harder questions about AI and its alignment with human valuesHow are Americans using AI? Evidence from a nationwide surveyConnect with Dr. Malihe Alikhani:https://x.com/malihealikhani🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI.Follow WiAIR at:⁠LinkedIn⁠⁠Bluesky⁠⁠X (Twitter)⁠⁠⁠WiAIR website⁠

Do large language models truly understand language—or are they sophisticated pattern matchers?In this conversation, Dr. Anna Ivanova (Asst. Prof. at Georgia Tech) explores one of the important questions in AI: the relationship between language, thought, and intelligence. Drawing from neuroscience, cognitive science, and AI research, Anna explains why language understanding is harder to define than most people realize, why reasoning and language are not the same thing, and what today's LLMs can and cannot tell us about human cognition.Key Topics:Do LLMs understand language or merely generate convincing text?The difference between formal and functional linguistic competenceWhat LLMs can learn from language alone—and what they cannotWhy human cognition and AI cognition may be fundamentally differentTheory of mind, reasoning, and common misconceptions about AI capabilitiesHow cognitive scientists evaluate the "thinking" abilities of LLMsWhat neuroscience can teach AI researchers about interpretabilityWhy understanding AI requires studying both behavior and internal representationsThe future of multimodal models and AI cognitionResources & Links:What does it mean to understand language?Dissociating language and thought in large language modelsHow to evaluate the cognitive abilities of LLMsHow Do LLMs Use Their Depth?True LensConnect with Dr. Anna Ivanova:https://bsky.app/profile/neuranna.bsky.socialhttps://x.com/neuranna🎧 Subscribe to stay updated on new episodes spotlighting brilliant women shaping the future of AI.Follow WiAIR at:LinkedInBlueskyX (Twitter)⁠WiAIR website⁠