Biography

☀️ I’m currently a first-year PhD student at the Department of Computing, The Hong Kong Polytechnic University (PolyU), advised by Prof. Qing Li and Prof. Wenqi Fan. I got my Master’s degree from Harbin Institute of Technology, Shenzhen (HITSZ) in March 2024, under the supervision of Prof. Zheng Zhang. Before that, I received my Bachelor’s degree from Harbin Institute of Technology at Weihai in June 2021.

News

🔥 [Dec, 2024]   One paper “Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model” has been accepted by AAAI 2025. 🎉 🎉 🎉

🔥 [Mar, 2024]   One paper has been accepted by IEEE TIP! 🎉 🎉 🎉

Research Interest

🚀 [Multimodal Large Language Models]: Retrieval-Augmented Generation, Instruction Tuning

🚀 [Multimedia Learning]: Deep Hashing Retrieval, Multimodal Retrieval

🚀 [Trustworthy Maching Learning]: Adversarial Examples, Backdoor Learning

Education

• PhD Candidate in Computer Science, 2024.09-2027.09 (expected)
  The Hong Kong Polytechnic University, advised by Prof. Qing Li and Prof. Wenqi Fan

• M.E. in Computer Science and Technology, 2021.09 - 2024.03
  Harbin Institute of Technology, Shenzhen, advised by Prof. Zheng Zhang

• B.E. in Computer Science and Technology, 2017.09 - 2021.06
  Harbin Institute of Technology, Weihai

Experiences

• March. 2024 - August. 2024: Research Intern
  • TAO Technology, Alibaba Group, Beijing
  • Working on vision and language understanding

Publications

(*) beside authors’ names indicates equal contributions.

HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning

Zhuohang Jiang^*, Pangjing Wu^*, Ziran Liang^*, Peter Q. Chen^*, Xu Yuan^*, Ye Jia^*, Tu Jiancheng^*, Chen Li, Peter H. F. Ng, Li Qing

preprint, 2025.

[paper] [code]

Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model

Xu Yuan^*, Li Zhou^*, Zenghui Sun, Zikun Zhou, and Jinsong Lan

The 39th Association for the Advancement of Artificial Intelligence (AAAI), 2025. [CCF-A]

This work is done during an internship at Alibaba Group.

[paper] [code] [project]

BadCM: Invisible Backdoor against Cross-Modal Learning

Zheng Zhang^*, Xu Yuan^*, Lei Zhu, Jingkuan Song, and Liqiang Nie

IEEE Transactions on Image Processing (TIP), 33: 2558-2571, 2024. [CCF-A]

[paper] [code]

Semantic-Aware Adversarial Training for Reliable Deep Hashing

Xu Yuan, Zheng Zhang, Xunguang Wang, and Lin Wu

IEEE Transactions on Information Forensics and Security (TIFS), 18: 4681-4694, 2023. [CCF-A]

[paper] [code]

Academic Service

• Serving as a conference reviewer of ACMMM 2025, KDD 2025 and ICDE 2025.
• Serving as a journal reviewer of TMM, TIFS, TKDD, and Information Fusion.

Honors and Awards

• PolyU Presidential PhD Fellowship, 2024
• Outstanding Master Thesis Award, Harbin Institute of Technology, 2024
• Binxing Fang Scholarship, 2023
• Outstanding Graduate, Harbin Institute of Technology, 2021.
• Outstanding Bachelor Thesis Award, Harbin Institute of Technology, 2021.
• First price, the 19th National Undergraduate Robotics Contest (ROBOCON), China, 2020.