Li Ding | 丁立

liding256@gmail.com

I'm currently at Google DeepMind ( Mountain View), working on AI control and agent safety & security research.

I received my PhD from UMass Amherst CICS under the supervision of Lee Spector. Before PhD, I was a researcher at MIT with Lex Fridman and Bryan Reimer. I did my master's at Univ. of Rochester with Chenliang Xu.

I also worked closely with Scott Niekum (UMass), Joel Lehman (formerly OpenAI), and Jeff Clune (UBC, DeepMind). I spent some time with wonderful people at Google Research and Meta.

Research

My research focuses on the control and safety of AI agents, ensuring that autonomous systems, particularly coding agents, remain aligned with human intent and operate within safe boundaries. I develop methods combining reinforcement learning from human feedback (RLHF), open-ended search, and oversight to improve the reliability and security of LLM-based agents.

Previously, I worked on pretraining, architecture, and runtime optimization for the Gemma and Gemini Nano models. My PhD research focused on alignment for safety and creativity using diversity-driven optimization in open-ended environments. My work has spanned reinforcement learning, generative models, robotics, computer vision, and program synthesis.

Highlights

Full publication list on Google Scholar.

Securing the Future of AI Agents
Google DeepMind Blog, 2026
blog

DeepMind's roadmap for AI control. We analyzed over a million agent trajectories to detect and categorize misbehavior of AI agents, and used the insights to build live monitoring for Gemini agents.

Pareto-Optimal Learning from Preferences with Hidden Context
Ryan Boldi, Li Ding, Lee Spector, Scott Niekum
RLC 2025
arXiv

POPL learns Pareto-optimal policies or reward functions in RLHF, addressing hidden contexts such as diverse group preferences without needing group labels, offering safe and fair alignment of agents and LLMs.

Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization
Li Ding, Jenny Zhang , Jeff Clune , Lee Spector , Joel Lehman
ICML 2024
NeurIPS 2023: ALOE Workshop (Spotlight)
project page arXiv demo talk code tutorial

QDHF learns diversity metrics from human feedback and uses those to optimize for the discovery of novel solutions, enhancing the task-solving capabilities of RL agents and the creativity of generative models.

Probabilistic Lexicase Selection
Li Ding, Edward Pantridge, Lee Spector
GECCO 2023
paper arXiv code

Optimizing Neural Networks with Gradient Lexicase Selection
Li Ding, Lee Spector
ICLR 2022
paper video poster code

We propose an optimization framework that improves the generalization of deep networks by learning more diverse representations.

Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Li Ding, Chenliang Xu
CVPR 2018
paper arXiv poster code

Misc.

Teaching:
TA for MIT 6.S094: Deep Learning for Self-Driving Cars (Winter 2018-19).
TA for MIT 6.S099: Artificial General Intelligence (Winter 2019).

Reviewer:
ICLR, NeurIPS, ICML, JMLR, CVPR, ICCV, ECCV, etc.

Open source projects:
google-research/ev3: Meta-learning optimization in JAX.
facebookresearch/d2go: Efficient model training and deployment on mobile platforms.
pyribs: An open-source library for quality diversity optimization.
mit-deep-learning: Tutorials and coding assignments for MIT Deep Learning courses (9k+ stars).

Side projects:

MIT AI Podcast
Helped prepare interview questions, search for guest speakers, etc. for a podcast hosted by Lex Fridman about technology, science, and the human condition.
(Ranked #1 on Apple Podcasts in the technology category, 1M views on YouTube.)
(My personal favorite episode is Tomaso Poggio, highly recommended!)

MIT Robocar Workshop
Instructor for a summer/winter workshop at MIT with Tom Bertalan to college and high school students on building and programming autonomous robocars.