CacheRL: Multi-Turn Tool-Calling Agents via Cached Rollouts and Hybrid Reward
Preprint arXiv
Huancheng Chen
Applied Scientist II, Microsoft AI
I am currently an Applied Scientist II at Microsoft AI, where I focus on training foundation models for search and agentic tasks in Bing.
My recent research spans efficient LLMs, agentic RL, skill evolution, and memory management. I previously worked on continual learning and trustworthy machine learning.
I received my Ph.D. in Electrical and Computer Engineering from UT Austin , advised by Haris Vikalo.
News
- September, 2026One paper about LLM quantization accepted as a NeurIPS2026 Spotlight (top 1%).
- June, 2026One paper about efficient agentic RL in arXiv.
- June, 2026One paper about continual learning accepted in TMLR.
- May, 2026One paper about foundation-preserving adaptation in arXiv.
Earlier updates
- March, 2026I joined Microsoft AI as Applied Scientist II.
- March, 2026Two papers accepted in CVPR2026 workshops.
- March, 2026One paper about long-horizon LLM agents in arXiv.
- February, 2026One paper about LLM quantization in arXiv.
- January, 2026One paper was accepted in ICLR2026.
- September, 2025One paper about continual learning in arXiv.
- June, 2025I joined Accenture as Senior AI Research Scientist.
- June, 2025Passed my PhD Defense!.
- November, 2024One paper about layout-to-image based on diffusion models in arXiv.
- September, 2024One paper about continual learning on foundation models in arXiv.
- November, 2024Pass my Ph.D progress review.
- September, 2024One paper accepted in NeurIPS2024.
- May, 2024One paper accepted in ICML2024.
- February, 2024One paper accepted in CVPR2024.
- February, 2024Joining PPML team in SonyAI as research intern.
- November, 2023One paper about mixed-precision quantization preprinted in arXiv.
- September, 2023One paper about client selection preprinted in arXiv.
- March, 2023One paper accepted in CVPR2023 workshop.
- Jan, 2023One paper accepted in ICLR2023.
Experience & Education
Full CVProfessional Experience
-
March 2026
Applied Scientist II
Microsoft AI
-
June 2025 – March 2026
Senior AI Research Scientist
Accenture
Advanced AI Center
Internship
-
Spring 2024
Research Internship
SonyAI
PPML team
-
Summer 2022
Research Internship
Toyota InfoTech Labs
AI/ML Infrastructure & Data Lab
-
Spring 2022
Research Internship
Nokia Bell Lab
Mathematics & Algorithms Research Group
Education
-
2020 – 2025
Ph.D. in Electrical and Computer Engineering
University of Texas at Austin
Advisor: Haris Vikalo
-
2015 - 2019
B.S. in Electrical Engineering
South China University of Technology
Publications
Google Scholar2026
Foundation-Preserving Adaptation via Generalized Rayleigh-Quotient Optimization
Preprint arXiv
Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memory
Preprint arXiv
CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization
NeurIPS'26 Conference on Neural Information Processing Systems (Spotlight, 0.91% acceptance rate)
Quantized Gradient Projection for Memory-Efficient Continual Learning
ICLR'26 International Conference on Learning Representation (poster)
Task-Agnostic Federated Continual Learning via Replay-Free Gradient Projection
TMLR'26 Transactions on Machine Learning Research
Training-Free Layout-to-Image Generation with Marginal Attention Constraints
CVPR'26 Efficient and On-Device Generation Workshop
Replay-Free Continual Low-Rank Adaptation with Dynamic Memory
CVPR'26 Domain Generalization Workshop
2024
Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learning
NeurIPS'24 Conference on Neural Information Processing Systems (poster)
Recovering Labels from Local Updates in Federated Learning
ICML'24 International Conference on Machine Learning (poster)
Mixed-Precision Quantization for Federated Learning on Resource-Constrained Heterogeneous Devices
CVPR'24 Conference on Computer Vision and Pattern Recognition (poster)
2023
Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Data
CVPR'23 Conference on Computer Vision and Pattern Recognition FedVision Workshop (oral)
The Best of Both Worlds Accurate Global and Personalized Models through Federated Learning with Data-Free Hyper-Knowledge Distillation
ICLR'23 International Conference on Learning Representation (poster)
2021
Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs
ICCV'21 International Conference on Computer Vision Workshop
Teaching
- EE351MDigital Signal Processing
- CS395TConvex Optimization
- EE380LData Mining
- EE422CSoftware Design and Implementation II (Java)
- EE381KStatistical Machine Learning
- CS395TFoundation of Predictive Machine Learning
Services
Conference Reviewer
- International Conference on Machine Learning (ICML), since 2022
- Conference on Neural Information Processing Systems (NeurIPS), since 2022
- International Conference on Learning Representations (ICLR), since 2024
- International Joint Conference on Artificial Intelligence (IJCAI), since 2024
- AAAI Conference on Artificial Intelligence (AAAI), since 2025
- IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), since 2025
- ACL Rolling Review, May 2026
Journal Reviewer
- IEEE Transactions on Mobile Computing (TMC)