Kaiwen Zhou

Hi there! I am a research scientist at Huawei Noah's Ark Lab.

Prior to joining Huawei, I completed my MPhil and PhD in computer science and engineering at the Chinese University of Hong Kong in 2019 and 2022, respectively. I obtained my BSc degree in computer science and technology at Fudan University in 2017.

I am currently interested in research topics related to Optimization for Machine Learning and Out-of-Distribution Generalization.

Email: kwzhou [at] cse (dot) cuhk (dot) edu (dot) hk
Office: Room 116, SHB, CUHK, Shatin, N.T., Hong Kong SAR.

scholar   dblp

Publications

  1. Pareto Invariant Risk Minimization: Towards Mitigating The Optimization Dilemma in Out-of-Distribution Generalization [paper] [github]
    Yongqiang Chen, Kaiwen Zhou, Yatao Bian, Binghui Xie and others.
    In International Conference on Learning Representations (ICLR), 2023.
    Also appeared in ICML Workshop on Principles of Distribution Shift, 2022.
  2. A Novel Extrapolation Technique to Accelerate WMMSE
    Kaiwen Zhou, Zhilin Chen, Guochen Liu, Zhitang Chen.
    In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2023.
  3. On the Finite-Time Complexity and Practical Computation of Approximate Stationarity Concepts of Lipschitz Functions [paper]
    Lai Tian, Kaiwen Zhou, Anthony Man-Cho So.
    In International Conference on Machine Learning (ICML), 2022.
  4. Fast and Reliable Evaluation of Adversarial Robustness with Minimum-Margin Attack [paper]
    Ruize Gao, Jiongxiao Wang, Kaiwen Zhou, Feng Liu, Binghui Xie, Gang Niu, Bo Han, James Cheng.
    In International Conference on Machine Learning (ICML), 2022.
  5. Accelerating Perturbed Stochastic Iterates in Asynchronous Lock-Free Optimization [paper]
    Kaiwen Zhou, Anthony Man-Cho So, James Cheng.
    In NeurIPS Workshop on Optimization for Machine Learning (NeurIPS OPT), 2022.
  6. Practical Schemes for Finding Near-Stationary Points of Convex Finite-Sums [paper]
    Kaiwen Zhou, Lai Tian, Anthony Man-Cho So, James Cheng.
    In International Conference on Artificial Intelligence and Statistics (AISTATS), 2022.
  7. Boosting First-Order Methods by Shifting Objective: New Schemes with Faster Worst-Case Rates [paper]
    Kaiwen Zhou, Anthony Man-Cho So, James Cheng.
    In Advances in Neural Information Processing Systems (NeurIPS), 2020.
  8. Amortized Nesterov's Momentum: A Robust Momentum and Its Application to Deep Learning [paper] [code]
    Kaiwen Zhou, Yanghua Jin, Qinghua Ding, James Cheng.
    In Conference on Uncertainty in Artificial Intelligence (UAI), 2020.
  9. Tight Convergence Rate of Gradient Descent for Eigenvalue Computation [paper]
    Qinghua Ding, Kaiwen Zhou, James Cheng.
    In International Joint Conference on Artificial Intelligence (IJCAI), 2020.
  10. Convolutional Embedding for Edit Distance [paper] [github]
    Xinyan Dai, Xiao Yan, Kaiwen Zhou, Yuxuan Wang, Han Yang, James Cheng.
    In International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2020.
  11. Direct Acceleration of SAGA using Sampled Negative Momentum [paper]
    Kaiwen Zhou, Qinghua Ding, Fanhua Shang, James Cheng, Danli Li, Zhi-Quan Luo.
    In International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.
  12. A Simple Stochastic Variance Reduced Algorithm with Fast Convergence Rates [paper]
    Kaiwen Zhou, Fanhua Shang, James Cheng.
    In International Conference on Machine Learning (ICML), 2018.
  13. Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization [paper]
    Fanhua Shang, Yuanyuan Liu, Kaiwen Zhou, James Cheng, Kelvin Kai Wing Ng, Yuichi Yoshida.
    In International Conference on Artificial Intelligence and Statistics (AISTATS), 2018.
  14. VR-SGD: A Simple Stochastic Variance Reduction Method for Machine Learning [paper]
    Fanhua Shang, Kaiwen Zhou, Hongying Liu, James Cheng, Ivor Tsang, Lijun Zhang, Dacheng Tao, Licheng Jiao.
    In IEEE Transactions on Knowledge and Data Engineering (TKDE), 2018.
  15. ASVRG: Accelerated Proximal SVRG [paper]
    Fanhua Shang, Licheng Jiao, Kaiwen Zhou, James Cheng, Yan Ren, Yufei Jin.
    In Asian Conference on Machine Learning (ACML), 2018.

Preprints

 (*indicates equal contribution)

  1. Efficient Private SCO for Heavy-Tailed Data via Clipping [paper]
    Chenhan Jin, Kaiwen Zhou, Bo Han, Ming-Chang Yang, James Cheng.
  2. An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms [paper]
    Binghui Xie*, Chenhan Jin*, Kaiwen Zhou, James Cheng, Wei Meng.
  3. Local Reweighting for Adversarial Training [paper]
    Ruize Gao*, Feng Liu*, Kaiwen Zhou, Gang Niu, Bo Han, James Cheng.
  4. Hyper-Sphere Quantization: Communication-Efficient SGD for Federated Learning [paper] [github]
    Xinyan Dai, Xiao Yan, Kaiwen Zhou, Han Yang, Kelvin Kai Wing Ng, James Cheng, Yu Fan.
  5. Norm-Range Partition: A Universal Catalyst for LSH based Maximum Inner Product Search (MIPS) [paper]
    Xiao Yan, Xinyan Dai, Jie Liu, Kaiwen Zhou, James Cheng.

Theses

  • Fast, Practical and Scalable First-Order Methods for Modern Machine Learning Problems [paper]
    The Chinese University of Hong Kong, PhD, 2022.
  • Accelerating Finite-sum Convex Optimization and Highly-smooth Convex Optimization [paper]
    Outstanding Thesis Award of Faculty of Engineering (Sole Winner)
    The Chinese University of Hong Kong, MPhil, 2019.

Academic Service

  • Conference refereeing: ICML 2021/22/23, NeurIPS 2021/22, AISTATS 2022/23, ICLR 2021/22/23.
  • Journal refereeing: Mathematical Programming, SIAM Journal on Optimization, Optimization Methods and Software, Mathematical Programming Computation, Transactions on Machine Learning Research.

Last update: Feb 23, 2023.