arXiv research

Introduces new gradient-based methods for machine learning problems.

TrIM improves gradient-based dimension reduction and regression.

We formulate a general framework for competitive gradient-based learning that encompasses a wide breadth of multi-agent learning algorithms, and analyze the limiting behavior of competitive gradient-based learning algorithms using dynamical systems theory. For both general-sum and potential games, we characterize a non…

We review recent works on analyzing the dynamics of gradient-based algorithms in a prototypical statistical inference problem. Using methods and insights from the physics of glassy systems, these works showed how to understand quantitatively and qualitatively the performance of gradient-based algorithms. Here we review…

This paper surveys gradient-based multi-objective deep learning methods.

New bounds show faster convergence for learning algorithms.

New method creates universal perturbations to fool neural network interpretations.

Gradient-based explanations correlate with Android malware classifier robustness.

New method improves parameter estimation in complex stochastic models.

Reduces learning periodic neural networks to lattice problems, proving hardness under cryptographic assumptions.

We consider an online learning process to forecast a sequence of outcomes for nonconvex models. A typical measure to evaluate online learning algorithms is regret but such standard definition of regret is intractable for nonconvex models even in offline settings. Hence, gradient based definition of regrets are common f…

Novel BSG method for efficient stochastic optimization.

In recent years, Deep Learning has become the go-to solution for a broad range of applications, often outperforming state-of-the-art. However, it is important, for both theoreticians and practitioners, to gain a deeper understanding of the difficulties and limitations associated with common approaches and algorithms. W…

Develops shuffling gradient-based methods for nonconvex-concave minimax optimization.

Fully automating machine learning pipelines is one of the key challenges of current artificial intelligence research, since practical machine learning often requires costly and time-consuming human-powered processes such as model design, algorithm development, and hyperparameter tuning. In this paper, we verify that au…

New method combines QQA and gradient-based sampling for combinatorial optimization.

Gradient-based meta-learners such as Model-Agnostic Meta-Learning (MAML) have shown strong few-shot performance in supervised and reinforcement learning settings. However, specifically in the case of meta-reinforcement learning (meta-RL), we can show that gradient-based meta-learners are sensitive to task distributions…

This work bridges competitive learning with gradient-based learning for faster feature extraction.

Paper extends KPCA using dualization for faster, more robust algorithms.

Gradient-based training and pruning for radial basis function networks in materials physics.

Gradient-based algorithm improves model performance with triage.

Improves meta-learning efficiency with mixed-mode differentiation.

Paper analyzes dataset distillation for efficient encoding of task-relevant information.

This work investigates how gradient-based learning performs with structured data, revealing issues and improvements.

A novel decentralized deep learning algorithm using gradient-based optimization.

A novel gradient-based method optimizes decision trees for complex tasks.

Optimizes shortfall risk using gradient-based methods.

While on some natural distributions, neural-networks are trained efficiently using gradient-based algorithms, it is known that learning them is computationally hard in the worst-case. To separate hard from easy to learn distributions, we observe the property of local correlation: correlation between local patterns of t…

New method improves combinatorial optimization by overcoming inefficient sampling.

Proposes a gradient-based variable selection method for binary classification in RKHS.

FDS tackles long horizon hyperparameter optimization issues.

The paper proposes a gradient-based method for multi-penalty Ridge regression.

The paper tackles learning from imperfect human feedback, especially in dueling bandit problems.

Gradient-based methods are often used for policy optimization in deep reinforcement learning, despite being vulnerable to local optima and saddle points. Although gradient-free methods (e.g., genetic algorithms or evolution strategies) help mitigate these issues, poor initialization and local optima are still concerns …

Optimal selection of a subset of items from a given set is a hard problem that requires combinatorial optimization. In this paper, we propose a subset selection algorithm that is trainable with gradient-based methods yet achieves near-optimal performance via submodular optimization. We focus on the task of identifying …

Enhances gradient-based discrete samplers with parallel tempering for multimodal distributions.

Combines gradient-based and competitive learning for unsupervised feature extraction.

Proposes efficient calibration method for LIBOR Market Model with stochastic volatility.

We study the problem of meta-learning through the lens of online convex optimization, developing a meta-algorithm bridging the gap between popular gradient-based meta-learning and classical regularization-based multi-task transfer methods. Our method is the first to simultaneously satisfy good sample efficiency guarant…

In this paper we introduce Feature Gradients, a gradient-based search algorithm for feature selection. Our approach extends a recent result on the estimation of learnability in the sublinear data regime by showing that the calculation can be performed iteratively (i.e., in mini-batches) and in linear time and space wit…

Paper proposes a new method to measure model sensitivity using final model only.

DenseHMM improves HMMs by learning dense representations that enable gradient-based optimization.

Understanding the power of depth in feed-forward neural networks is an ongoing challenge in the field of deep learning theory. While current works account for the importance of depth for the expressive power of neural-networks, it remains an open question whether these benefits are exploited during a gradient-based opt…

Neural networks can achieve optimal sample complexity for learning single-index models.

We analyze the convergence of gradient-based optimization algorithms that base their updates on delayed stochastic gradient information. The main application of our results is to the development of gradient-based distributed optimization algorithms where a master node performs parameter updates while worker nodes compu…

New algorithm improves gradient-based ERM for smooth convex losses.

In this paper, we study the problem of monotone (weakly) DR-submodular continuous maximization. While previous methods require the gradient information of the objective function, we propose a derivative-free algorithm LDGM for the first time. We define $β$ and $α$ to characterize how close a function is to continuous D…

Improved DP algorithms for non-convex optimization with tighter generalization bounds.