Yutong (Kelly) He 何雨桐

Contact: yutonghe [at] cs [dot] cmu [dot] edu

kelly2024.JPG

PhD Student @ MLD, SCS, CMU

yutonghe [at] cs [dot] cmu [dot] edu

I am a forth-year PhD student in the Machine Learning Department, School of Computer Science at Carnegie Mellon University, advised by Prof. Zico Kolter and Prof. Ruslan Salakhutdinov. During my PhD, I have interned at Sony AI working with Naoki Murata and Yuki Mitsufuji, and Meta FAIR working with Ricky Chen.

Before coming to CMU, I was a master’s student at Stanford Computer Science with distinction in research. I was advised by Prof. Stefano Ermon, and closely worked with Prof. Christopher Manning, Prof. David Lobell and Prof. Marshall Burke. I completed my B.S in Mathematics and B.S. in Data Science with highest distinction at University of Rochester, where I worked with Prof. Henry Kautz and Prof. Jiebo Luo.

I was selected as a Siebel Scholar and a Xerox Engineering Research Fellow. I was also awarded an Outstanding Paper Award at ICLR 2022 and Doris Ermine Smith Award for Achievement in Mathematics.

My research interests include generative models, more specifically diffusion models and flow matching related topics, and how to properly use them in broader contexts.

News

09/2025 Two of our papers Accelerating Diffusion Models in Offline RL via Reward-Aware Consistency Trajectory Distillation and Blind Inverse Problem Solving Made Easy by Text-to-Image Latent Diffusion are accepted by NeurIPS 2025 SPIGM & ALERT workshop.
07/2025 Our papers Towards Reporting Bias in Visual-Language Datasets: Bi-modal Data Augmentation by Decoupling Object-Attribute Association is accepted by ICCV 2025 MRR workshop.
05/2025 I started my internship as a student research scientist at Meta FAIR.

Publications

  1. TMLR
    Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation
    Yutong He, Alexander Robey, Naoki Murata, Yiding Jiang, Joshua Nathaniel Williams, George J. Pappas, Hamed Hassani, Yuki Mitsufuji, Ruslan Salakhutdinov, and J Zico Kolter
    Transactions on Machine Learning Research (TMLR) 2025
  2. NeurIPS Workshop
    Accelerating Diffusion Models in Offline RL via Reward-Aware Consistency Trajectory Distillation
    Xintong Duan*, Yutong He*, Fahim Tajwar, Ruslan Salakhutdinov, J Zico Kolter, and Jeff Schneider
    In SPIGM & ALERT @ NeurIPS 2025
  3. NeurIPS Workshop
    Blind Inverse Problem Solving Made Easy by Text-to-Image Latent Diffusion
    Michail Dontas*, Yutong He*, Naoki Murata, Yuki Mitsufuji, J Zico Kolter, and Ruslan Salakhutdinov
    In SPIGM @ NeurIPS 2025
  4. ICCV Workshop
    Towards Reporting Bias in Visual-language Datasets: Bi-modal Data Augmentation by Decoupling Object-attribute Association
    Qiyu Wu, Mengjie Zhao, Yutong He, Lang Huang, Junya Ono, Hiromi Wakaki, and Yuki Mitsufuji
    In MRR @ ICCV 2025
  5. arXiv
    State combinatorial generalization in decision making with conditional diffusion models
    Xintong Duan, Yutong He, Fahim Tajwar, Wen-Tse Chen, Ruslan Salakhutdinov, and Jeff Schneider
    arXiv preprint 2025
  6. ICLR
    Manifold Preserving Guided Diffusion
    Yutong He*, Naoki Murata*, Chieh-Hsin Lai, Yuhta Takida, Toshimitsu Uesaka, Dongjun Kim, Wei-Hsiang Liao, Yuki Mitsufuji, J. Zico Kolter, Ruslan Salakhutdinov, and Stefano Ermon
    In International Conference on Learning Representations (ICLR) 2024
  7. ICLR
    Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion
    Dongjun Kim*, Chieh-Hsin Lai*, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon
    In International Conference on Learning Representations (ICLR) 2024
  8. arXiv
    Weather prediction with diffusion guided by realistic forecast processes
    Zhanxiang Hua*, Yutong He*, Chengqian Ma, and Alexandra Anderson-Frey
    arXiv preprint 2024
  9. arXiv
    Localized Text-to-Image Generation for Free via Cross Attention Control
    Yutong He, Ruslan Salakhutdinov, and J. Zico Kolter
    arXiv preprint 2023
  10. ISPRS
    Sphere2Vec: A general-purpose location representation learning over a spherical surface for large-scale geospatial predictions
    Gengchen Mai, Yao Xuan, Wenyun Zuo, Yutong He, Jiaming Song, Stefano Ermon, Krzysztof Janowicz, and Ni Lao
    ISPRS Journal of Photogrammetry and Remote Sensing 2023
  11. ICML
    CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations
    Gengchen Mai, Ni Lao, Yutong He, Jiaming Song, and Stefano Ermon
    In International Conference on Machine Learning (ICML) 2023
  12. NeurIPS
    SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery
    Yezhen Cong*, Samar Khanna*, Chenlin Meng, Patrick Liu, Erik Rozi, Yutong He, Marshall Burke, David B. Lobell, and Stefano Ermon
    In Neural Information Processing Systems (NeurIPS) 2022
  13. SIGSPATIAL
    Understanding Economic Development in Rural Africa using Satellite Imagery, Building footprints and Deep Models
    Amna Elmustafa, Erik Rozi, Yutong He, Gengchen Mai, Stefano Ermon, Marshall Burke, and David Lobell
    In International Conference on Advances in Geographic Information Systems (ACM SIGSPATIAL) 2022
  14. SIGDial
    Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent
    Ethan A. Chi, Caleb Chiam, Trenton Chang, Swee Kiat Lim, Chetanya Rastogi, Alexander Iyabor, Yutong He, Hari Sowrirajan, Avanika Narayan, Jillian Tang, Haojun Li, Ashwin Paranjape, and Christopher D. Manning
    In The 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDial) 2022
  15. ICLR Oral Award
    Comparing Distributions by Measuring Differences that Affect Decision Making
    Shengjia Zhao*, Abhishek Sinha*, Yutong He*, Aidan Perreault, Jiaming Song, and Stefano Ermon
    In International Conference on Learning Representations (ICLR) 2022
    Outstanding Paper Award [Top 0.15%]
    Oral Presentation [Top 1.6%]
  16. ICLR
    SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon
    In International Conference on Learning Representations (ICLR) 2022
  17. NeurIPS
    Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis
    Yutong He, Dingjie Wang, Nicholas Lai, William Zhang, Chenlin Meng, Marshall Burke, David B. Lobell, and Stefano Ermon
    In Neural Information Processing Systems (NeurIPS) 2021
  18. NeurIPS Workshop
    Tracking Urbanization in Developing Regions with Remote Sensing Spatial-Temporal Super-Resolution
    Yutong He*, William Zhang*, Chenlin Meng, Marshall Burke, David B. Lobell, and Stefano Ermon
    In Neural Information Processing Systems (NeurIPS) workshop on Machine Learning for the Developing World (ML4D) 2021
  19. Alexa Prize
    Neural Neural Everywhere: Controlled Generation Meets Scaffolded and Structured Dialogue
    Ethan A. Chi, Caleb Chiam, Trenton Chang, Swee Kiat Lim, Chetanya Rastogi, Alexander Iya-bor, Yutong He, Hari Sowrirajan, Avanika Narayan, Jillian Tang, Haojun Li, Ashwin Paranjape, and Christopher D. Manning
    In Alexa Prize Proceedings 2021
  20. CVPR
    Fine-grained Image-to-Image Transformation towards Visual Recognition
    Wei Xiong, Yutong He, Yixuan Zhang, Wenhan Luo, Lin Ma, and  Jiebo Luo
    In International Conference on Computer Vision and Pattern Recognition (CVPR) 2020
  21. arXiv
    Motion-Based Handwriting Recognition and Word Reconstruction
    Junshen Kevin Chen*, Wanze Xie*, and Yutong He*
    arXiv preprint 2020