On Convergence of ADAM with Data Dependent Stepsize

We propose a data and dynamics driven constant stepsize for Adam that removes the need for costly tuning and avoids the pitfalls of decaying schedulers. Our estimate matches the best stepsizes found by exhaustive search, performs on par with popular schedulers, and scales to ImageNet and LLM fine-tuning. We further provide theory showing that Adam with our stepsize converges to critical points for smooth non-convex objectives

Alokendu Mazumder, RIshab Sabharwal, Bhartendu Kumar, Manan Tayal, Arnab Roy, Chirag Garg, Punit Rathore

In IEEE Transactions of Artificial Intelligence (IF: 6.7), 2026

Paper / Project Page / Code (Github)

Fractional Gradient Descent with Matrix Stepsizes for Non-Convex Optimisation

We establish convergence guarantees for fractional gradient descent on matrix-smooth non-convex functions and show how matrix smoothness accelerates convergence. We introduce Compressed Fractional Gradient Descent (CFGD) with matrix-valued stepsizes, proving faster convergence than scalar stepsizes in both single-node and distributed settings. This is the first work to study fractional gradient descent in federated/distributed optimisation.

Alokendu Mazumder, Keshav Vyas, Punit Rathore

In IEEE Transactions of Neural Networks and learning Systems (IF: 8.9), 2025

Paper / Project Page / Code (Github)

Learning Low Rank Latent Spaces With Simple Deterministic Autoencoders: Theoretical and Empirical Insights

Low-Rank Autoencoder (LoRAE) extends autoencoders with a low-rank regularizer to learn compact and implicit low-dimensional latent spaces. It achieves tighter theoretical error bounds and strong empirical performance in image generation and downstream classification. It even beats highly optimized GANs !

Alokendu Mazumder, Tirthajit Baruah, Bhartendu Kumar, Rishab Sharma, Vishwajeet Pattanaik, Punit Rathore,

In the Winter Conference on Applications of Computer Vision (WACV), 2024

Paper / Project Page / Code (Github)

Perceptual Quality Assessment of DIBR Synthesized Views Using Saliency Based Deep features

We propose an efficient no-training perceptual quality metric for DIBR synthesized views that exploits saliency-guided deep features extracted from a pretrained VGG-16 network. By focusing on perceptually important regions and fusing feature maps using cosine similarity, the method effectively captures geometric distortions and outperforms existing state-of-the-art QA metrics on standard benchmarks.

Shubham Chaudhary, Alokendu Mazumder, Deebha Mumtaz, Vinit Jakhetiya, Badri Subudhi

In International Comference on Image Processing (ICIP), 2021

Paper