Abstract

A long-term goal of AI research is to build intelligent agents that can effectively communicate with humans, perceive their multimodal environment, and execute a diverse range of real-world tasks, from everyday household chores to complex, mission-critical tasks such as battlefield reconnaissance. These embodied agents are envisioned to operate both autonomously and collaboratively via human interaction.

This talk focuses on addressing fundamental challenges in human-agent interaction and collaboration.
First, we introduce our recent efforts towards building generalizable embodied agents that can better adapt to novel tasks and environments through non-spurious decision making. Then, we underscore the significance of streamlined human-agent communication in fostering efficient collaboration, with our proposed new benchmarks. Lastly, we address the "last mile problem" of embodied agents and present VLMbench, a new compositional benchmark for vision-and-language robotic manipulation.
This talk concludes with a discussion of future research plans.

Biography

Xin (Eric) Wang is an Assistant Professor of Computer Science and Engineering at UC Santa Cruz. His research interests include Natural Language Processing, Computer Vision, and Machine Learning, with a focus on Multimodal and Embodied AI. Before joining UCSC, he obtained his Ph.D. degree from UC Santa Barbara in 2020 and Bachelor's degree from Zhejiang University in 2015. He worked at Google Research, Facebook AI Research, Microsoft Research, and Adobe Research.

Xin has served as Area Chair for conferences such as ACL, NAACL, EMNLP, ICLR, and NeurIPS, as well as Senior Program Committee for AAAI and IJCAI. He has organized numerous workshops and tutorials at conferences such as ACL, NAACL, CVPR, and ICCV. He has received several awards and recognitions for his work, including a CVPR Best Student Paper Award (2019), a Google Research Faculty Award (2022), and three Amazon Alexa Prize Awards (2022-2023).