Zhongkai Yu

I am currently a second-year PhD student in the CSE department of UC San Diego (since 2024), working with Prof.Yufei Ding. Before coming to UCSD, I obtained my master’s degree from Institute of Computing Technology, Chinese Academy of Sciences (ICT, CAS), under the supervision of Prof. Yunji Chen and Prof. Xing Hu. Prior to that, I received my bachelor’s degree from Shanghai Jiao Tong University (SJTU).

My research centers on the co-evolution of LLM and hardware systems: I explore novel architectures and systems to accelerate LLM workloads, while harnessing LLMs to advance chip design. My work lies at the intersection of Computer Architecture, ML Systems, and AI4Chip.

Also, shout-out to my roommate Zaifeng, who is an expert in kernels, systems, and “cooking”. Check out his homepage and research.

Email: zhy055@ucsd.edu

Education

University of California, San Diego (UCSD), 2024 - Present
Ph.D. student in Computer Science & Engineering
University of Chinese Academy of Sciences (UCAS), 2021 - 2024
M.E. in Computer Technology
Shanghai Jiao Tong University (SJTU), 2017 - 2021
B.S. in Physics (Zhiyuan Honors Program)

Experience

NVIDIA Research, 2026.6 - 2026.9.
Research Intern, Architecture Group
Manager: Po-An Tsai, Jenny Huang, and Christos Kozyrakis
Samsung Semiconductor, 2025.6 - 2025.9.
Research Intern, AGI Lab
Manager: Shuyi Pei and Yangwook Kang
Cambricon Technologies, 2022.3 - 2024.3.
Digital IC Design Intern, Architecture Group
Manager: Dong Han and Shaoli Liu

Selected Publications

ArXiv

ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation

Zhongkai Yu^*, Yichen Lin^*, Chenyang Zhou, Yuwei Zhang, Kun Zhou, Junxia Cui, Haotian Ye, Zhengding Hu, Zaifeng Pan, Ruiyi Wang, Yujie Zhao, Hejia Zhang, Jingbo Shang, Jishen Zhao, and Yufei Ding

arXiv preprint arXiv:2605.12857, 2026

Bib PDF Code

@article{yu2026chipmate,
  title = {ChipMATE: Multi-Agent Training via Reinforcement Learning for Enhanced RTL Generation},
  author = {Yu, Zhongkai and Lin, Yichen and Zhou, Chenyang and Zhang, Yuwei and Zhou, Kun and Cui, Junxia and Ye, Haotian and Hu, Zhengding and Pan, Zaifeng and Wang, Ruiyi and Zhao, Yujie and Zhang, Hejia and Shang, Jingbo and Zhao, Jishen and Ding, Yufei},
  journal = {arXiv preprint arXiv:2605.12857},
  year = {2026},
}

ArXiv

ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design

Zhongkai Yu^*, Chenyang Zhou^*, Yichen Lin, Hejia Zhang, Haotian Ye, Junxia Cui, Zaifeng Pan, Jishen Zhao, and Yufei Ding

arXiv preprint arXiv:2601.21448, 2026

Bib PDF Code

@article{yu2026chipbenchnextstepbenchmarkevaluating,
  title = {ChipBench: A Next-Step Benchmark for Evaluating LLM Performance in AI-Aided Chip Design},
  author = {Yu, Zhongkai and Zhou, Chenyang and Lin, Yichen and Zhang, Hejia and Ye, Haotian and Cui, Junxia and Pan, Zaifeng and Zhao, Jishen and Ding, Yufei},
  journal = {arXiv preprint arXiv:2601.21448},
  year = {2026},
}

MICRO’26

AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving

Zhongkai Yu, Haotian Ye, Chenyang Zhou, Ohm Rishabh Venkatachalam, Zaifeng Pan, Zhengding Hu, Junsung Kim, Won Woo Ro, Po-An Tsai, Shuyi Pei, Yangwook Kang, and Yufei Ding

In Proceedings of the 59th IEEE/ACM International Symposium on Microarchitecture, 2026

Bib PDF

@inproceedings{yu2026ammamultichipletmemorycentricarchitecture,
  title = {AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving},
  author = {Yu, Zhongkai and Ye, Haotian and Zhou, Chenyang and Venkatachalam, Ohm Rishabh and Pan, Zaifeng and Hu, Zhengding and Kim, Junsung and Ro, Won Woo and Tsai, Po-An and Pei, Shuyi and Kang, Yangwook and Ding, Yufei},
  booktitle = {Proceedings of the 59th IEEE/ACM International Symposium on Microarchitecture},
  year = {2026},
  organization = {IEEE},
}

ISCA’26

Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference

Zhongkai Yu, Yue Guan, Zihao Yu, Chenyang Zhou, Zhengding Hu, Shuyi Pei, Yangwook Kang, Yufei Ding, and Po-An Tsai

arXiv preprint arXiv:2510.05497, 2026

Best Paper Award (2/845)

Bib PDF Code

@article{yu2026patternschaosforecastingdata,
  title = {Patterns behind Chaos: Forecasting Data Movement for Efficient Large-Scale MoE LLM Inference},
  author = {Yu, Zhongkai and Guan, Yue and Yu, Zihao and Zhou, Chenyang and Hu, Zhengding and Pei, Shuyi and Kang, Yangwook and Ding, Yufei and Tsai, Po-An},
  journal = {arXiv preprint arXiv:2510.05497},
  year = {2026},
}

MICRO’24

Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM

Zhongkai Yu^*, Shengwen Liang^*, Tianyun Ma, Yunke Cai, Ziyuan Nan, Di Huang, Xinkai Song, Yifan Hao, Jie Zhang, Tian Zhi, Yongwei Zhao, Zidong Du, Xing Hu, Qi Guo, and Tianshi Chen

In Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture, 2024

Bib PDF

@inproceedings{yu2024cambricon,
  title = {Cambricon-LLM: A Chiplet-Based Hybrid Architecture for On-Device Inference of 70B LLM},
  author = {Yu, Zhongkai and Liang, Shengwen and Ma, Tianyun and Cai, Yunke and Nan, Ziyuan and Huang, Di and Song, Xinkai and Hao, Yifan and Zhang, Jie and Zhi, Tian and Zhao, Yongwei and Du, Zidong and Hu, Xing and Guo, Qi and Chen, Tianshi},
  booktitle = {Proceedings of the 57th IEEE/ACM International Symposium on Microarchitecture},
  pages = {1474--1488},
  year = {2024},
  organization = {IEEE},
}

DAC’22

E2sr: an end-to-end video codec assisted system for super resolution acceleration

Zhuoran Song, Zhongkai Yu, Naifeng Jing, and Xiaoyao Liang

In Proceedings of the 59th ACM/IEEE Design Automation Conference, 2022

Bib PDF

@inproceedings{song2022e2sr,
  title = {E2sr: an end-to-end video codec assisted system for super resolution acceleration},
  author = {Song, Zhuoran and Yu, Zhongkai and Jing, Naifeng and Liang, Xiaoyao},
  pages = {229--234},
  booktitle = {Proceedings of the 59th ACM/IEEE Design Automation Conference},
  year = {2022},
}

Honors & Awards

2026 ISCA Best Paper Award
2026 Qualcomm Innovation Fellowship Finalist
2024 Lenovo Scholarship