LLMs Infer Cultural Context but Fail to Apply It When Responding

Sun, 25 Oct 2026 00:00:00 +0000

Yisong Miao Presents His Vector Institute Research Internship Work (with UBC NLP) at the NUS SRI Event

Thu, 02 Jul 2026 00:00:00 +0000

On the afternoon of 2 July 2026, NUS SoC held the 2026 edition of its Summer Research Internship (SRI) Research Gathering and Networking Event. SRI is our school’s program for attracting talent from around the globe to conduct research at our school over the summer. Our group is delighted to host Mr. Thura Aung (KMITL) and Mr. Ha Minh Hieu (HUST) this summer. Dr. Anand Bhojan delivered the opening remarks, and Prof. Chan Mun Choon gave a keynote about SoC’s PhD program.

Yisong Miao, a PhD student from our group, was invited to give a 20-minute presentation at the gathering. He shared his experience as a Vector Institute research intern in 2025, under the supervision of Prof. Vered Shwartz and Prof. Jian Zhu (both from the NLP Group at the University of British Columbia). This is the most interdisciplinary research Yisong has done, connecting pragmatic speaker models (closely related to his PhD topic of discourse) with LLMs’ cultural competence, language grounding, and vision language models. The project examines whether LLMs can act as pragmatic speakers by adapting their responses to users’ cultural backgrounds in conversations grounded in images. The team built Cultural and Pragmatic Response Inference (CAPRI), a new dataset in which users provide cultural cues and ask questions requiring culturally appropriate answers. For example, “What is the temperature?” should be answered in °C or °F depending on the user’s cultural background. Extensive evaluations show that state-of-the-art models can largely infer the user’s cultural background, but often fail to apply this knowledge in their responses. Explicit reasoning about the user’s background, step by step, improves their responses, but can be costly in real-time conversation. The work will appear at EMNLP 2026.

Yisong felt especially honored to give the première présentation of this work at this event, as SRI brings together junior students from around the globe, including Brazil, France, and the USA (which are also represented in the CAPRI dataset!). The seminar concluded with a discussion about future work on extending the CAPRI dataset to more cultures in Southeast Asia (where many seminar audiences come from) and elsewhere around the world.

Prof. Vered Shwartz said: “I’m really excited about this benchmark. Most of the existing work in cultural competence focuses on testing LLMs on their cultural knowledge. Our benchmark allows us to test whether LLMs can also recognize cultural context and apply this knowledge in context. I hope our work will inform future model design. At a personal level, it was a pleasure working with Yisong and Jian on this project. Yisong was always so methodological, organized, and energetic, even though our virtual meetings were happening at 1 am Singapore time. I hope to meet him in person soon!”

Links: Earlier news featuring Yisong as a Vector Institute research intern; The project indexed at our website.

Below is a gallery of the seminar photo

Yisong with the 2026 cohort of NUS SRI interns after the seminar:

Pragmatic Speaker Model | Web IR NLP Group @ NUS

LLMs Infer Cultural Context but Fail to Apply It When Responding

Yisong Miao Presents His Vector Institute Research Internship Work (with UBC NLP) at the NUS SRI Event