arXiv research

CEFOL uses deep learning for dynamic programming with recursive utility.

Efficient algorithm for contextual bandits with first-order guarantees.

First-order ANIL learns shared representations even with overparametrization.

Efficient RNN algorithm guarantees convergence in online learning.

A new method learns node embeddings for signed directed networks by capturing both first-order and high-order topologies.

New algorithm for safer machine learning with different testing and training distributions.

SGD's performance improves with critical batch size, minimizing SFO complexity.

A key feature of inductive logic programming (ILP) is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher…

OptEx accelerates first-order optimization with parallelized iterations.

A framework for decentralized optimization using first-order methods.

This paper improves online learning algorithms for LP problems, achieving better regret bounds.

This paper achieves first-order regret bounds in reinforcement learning with large state spaces.

New methods boost first-order optimization with faster rates.

Optimal first-order methods are shown to be fundamental limits in functional estimation.

First order methods can take extremely long to find global minima of non-convex functions.

This work interprets GELU and related activations via a first-order loss function.

The paper analyzes reinforcement learning methods for estimating weights and quality functions with fast convergence rates.

New framework tackles bi-level optimization without LLS condition.

A new method speeds up quantum state estimation.

Deep networks learn features suddenly, akin to a phase transition.

New approach learns graph representations by contrasting first-order neighbors and graph diffusion views.

Standard gradient descent methods are susceptible to a range of issues that can impede training, such as high correlations and different scaling in parameter space.These difficulties can be addressed by second-order approaches that apply a pre-conditioning matrix to the gradient to improve convergence. Unfortunately, s…

OSGM uses online learning to adapt stepsize for faster convergence.

Adaptive learning rate algorithms such as RMSProp are widely used for training deep neural networks. RMSProp offers efficient training since it uses first order gradients to approximate Hessian-based preconditioning. However, since the first order gradients include noise caused by stochastic optimization, the approxima…

Examines how first-order differential operators can be equivalently transformed.

New algorithms optimize constrained problems faster, avoiding full set optimization.

New vector fields integrate first-order ODEs.

Graph Convolution Network (GCN) has been recognized as one of the most effective graph models for semi-supervised learning, but it extracts merely the first-order or few-order neighborhood information through information propagation, which suffers performance drop-off for deeper structure. Existing approaches that deal…

Study first-order locally convex Lie algebroids in Bastiani calculus.

We present a predictor-corrector framework, called PicCoLO, that can transform a first-order model-free reinforcement or imitation learning algorithm into a new hybrid method that leverages predictive models to accelerate policy learning. The new "PicCoLOed" algorithm optimizes a policy by recursively repeating two ste…

New federated learning methods improve model performance on non-IID data.

A new method solves complex constrained minimax problems.

Second-order guarantees for federated learning algorithms.

A guide for solving first-order elliptic boundary value problems.

The filtering-clustering models, including trend filtering and convex clustering, have become an important source of ideas and modeling tools in machine learning and related fields. The statistical guarantee of optimal solutions in these models has been extensively studied yet the investigations on the computational as…

First-order ODEs linked to flat surfaces, leading to integrability.

We simplify deep learning convergence analysis using basic math.

FOCOPS optimizes agent's behavior while adhering to constraints.

In this paper, we study optimization methods consisting of iteratively minimizing surrogates of an objective function. By proposing several algorithmic variants and simple convergence analyses, we make two main contributions. First, we provide a unified viewpoint for several first-order optimization techniques such as …

Extends a theorem for first-order elliptic operators on manifolds.

First-order stochastic methods are the state-of-the-art in large-scale machine learning optimization owing to efficient per-iteration complexity. Second-order methods, while able to provide faster convergence, have been much less explored due to the high cost of computing the second-order information. In this paper we …

Derives FACT, an alternative to NFA for neural networks, explaining feature learning.

Two classes of methods have been proposed for escaping from saddle points with one using the second-order information carried by the Hessian and the other adding the noise into the first-order information. The existing analysis for algorithms using noise in the first-order information is quite involved and hides the es…

For the implicit systems of first order ordinary differential equations on the plane there is presented the complete local classification of generic singularities of family of its phase curves up to smooth orbital equivalence. Besides the well known singularities of generic vector fields on the plane and the singularit…

We consider first order expansions of convex penalized estimators in high-dimensional regression problems with random designs. Our setting includes linear regression and logistic regression as special cases. For a given penalty function $h$ and the corresponding penalized estimator $\hatβ$, we construct a quantity $η$,…

A wide class of regularization problems in machine learning and statistics employ a regularization term which is obtained by composing a simple convex function ωwith a linear transformation. This setting includes Group Lasso methods, the Fused Lasso and other total variation methods, multi-task learning methods and man…

Paper develops fast method for computing optimal transport.

Markov logic networks (MLNs) reconcile two opposing schools in machine learning and artificial intelligence: causal networks, which account for uncertainty extremely well, and first-order logic, which allows for formal deduction. An MLN is essentially a first-order logic template to generate Markov networks. Inference …