My ongoing research is situated at the intersection of AI and multimodal communication, focusing on two main areas: Human-AI Intelligence and Multimodal Representational Learning. Theoretically, my dissertation, titled "From Language Models to Multimodal Intelligence," explores these models within the framework of symbolic versus embodied cognition, leveraging various deep neural network (DNN) simulations and human-AI interaction studies.

In layman's terms:

Human-AI Interaction: This research primarily involves human-LLM interaction and human-language vision model interaction design to examine how human preferences align or deviate from those of generative AI models (e.g., image caption alignment). It also explores how our communication with large generative models is affected when pairing users with language-only and language-vision conditions. I work closely with my committee members to format my study design, including contributions from Dr. Elisa Kreiss's Computation and Language for Society lab and Dr. Hongjing Lu's Computational Vision and Learning Lab.

Additionally, I collaborate with complex systems scholars to use multi-agent LLMs to simulate how agent personalities impact problem-solving in negotiation games, and with social neuroscientists to examine individual differences in the biosignals associated with "feeling connected" in human-GPT-4o interactions.

Multimodal Representational Learning: This research can be broadly understood as Applied Deep Learning/Machine Learning with a Computational Cognitive Science focus. I utilize various combinations of deep learning models (CNN, RNN-LSTM, Transformer, Seq2Seq, etc.) for diverse goals, from improving classification performance for multimodal datasets, to facilitating downstream statistical analysis on de-spatialized/de-temporalized embeddings from raw signal forms, and conducting large-scale simulations on signal patterns of embeddings across models, modalities, and datasets. I work closely with my primary advisor, Dr. Rick Dale, and seek guidance from Dr. Hongjing Lu.

At Dr. Dale's Communicative-Mind (Co-Mind) Lab, I actively collaborate with social neuroscientists as DNN modelers to streamline the integration of neurosignals (fNIRS) with other behavioral signals (facial expressions, body movements), bridging the analysis of these raw signals with high-level social constructs (shared reality, connectedness, etc.).