In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness

Conference on Neural Information Processing Systems (NeurIPS) 2024 (🏆 Spotlight)

Advait Parulekar^*, Liam Collins^*, Aryan Mokhtari, Sujay Sanghavi, Sanjay Shakkottai (* = co-first authors)

On the role of soft-max activation in pretraining for in-context learning.

InfoNCE Loss Provably Learns Cluster-Preserving Representations

Conference on Learning Theory (COLT) 2023

Advait Parulekar, Liam Collins, Karthikeyan Shanmugam, Aryan Mokhtari, Sanjay Shakkottai

On how InfoNCE can learn meaningful representations by inheriting inductive biases.

A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models

Litu Rout, Advait Parulekar, Constantine Caramanis, Sanjay Shakkottai

On whether diffusion based inpainting can recover samples.

PAC Generalization via Invariant Representations

International Conference on Machine Learning (ICML) 2023

Advait Parulekar, Karthikeyan Shanmugam, Sanjay Shakkottai

On the number of random interventions needed for PAC generalization of invariant representations.

Improved Algorithms for Misspecified Linear Markov Decision Processes

International Conference on Artificial Intelligence and Statistics (AISTATS) 2022

Daniel Vial, Advait Parulekar, Sanjay Shakkottai, R. Srikant

On algorithms for linear Bandits and MDPs where the reward function isnt exactly linear.

Regret Bounds for Stochastic Shortest Path Problems with Linear Function Approximation

International Conference on Machine Learning (ICML) 2022

Daniel Vial, Advait Parulekar, Sanjay Shakkottai, R. Srikant

L1 Regression with Lewis Weights Subsampling

Approximation, Randomization, and Combinatorial Optimization. Algorithms and Techniques (RANDOM) 2021

Aditya Parulekar, Advait Parulekar, Eric Price

On the label complexity of finding approximately correct solutions to LAD regression.

Stochastic Linear Bandits with Protected Subspace

Advait Parulekar, Soumya Basu, Aditya Gopalan, Karthikeyan Shanmugam, Sanjay Shakkottai

On an algorithm for linear Bandits when there is an unknown subspace that provides no reward.

Locating conical degeneracies in the spectra of parametric self-adjoint matrices

SIAM Journal on Matrix Analysis and Applications, Volume 42

Gregory Berkolaiko, Advait Parulekar

On convergence of the Newton-Raphson algorithm to find matrices that have two identical eigenvalues from among a parameterized family of self-adjoint matrices.