arXiv research

First-order ANIL learns shared representations even with overparametrization.

ANIL adapts only a subset of parameters, reducing computational cost.

A new optimization method reduces memory and compute requirements for deep learning.

An important research direction in machine learning has centered around developing meta-learning algorithms to tackle few-shot learning. An especially successful algorithm has been Model Agnostic Meta-Learning (MAML), a method that consists of two optimization loops, with the outer loop finding a meta-initialization, f…

This thesis analyzes and improves convergence rates of bilevel optimization algorithms in machine learning.

Lipschitz constraints under L2 norm on deep neural networks are useful for provable adversarial robustness bounds, stable training, and Wasserstein distance estimation. While heuristic approaches such as the gradient penalty have seen much practical success, it is challenging to achieve similar practical performance wh…

Meta-learning improves GNN initializations for low-resource drug discovery.

Examines how first-order differential operators can be equivalently transformed.

AudioPaLM combines text and speech models to improve speech processing and translation.

New vector fields integrate first-order ODEs.

Study first-order locally convex Lie algebroids in Bastiani calculus.

A new method solves complex constrained minimax problems.

A guide for solving first-order elliptic boundary value problems.

First-order ODEs linked to flat surfaces, leading to integrability.

Extends a theorem for first-order elliptic operators on manifolds.

Two classes of methods have been proposed for escaping from saddle points with one using the second-order information carried by the Hessian and the other adding the noise into the first-order information. The existing analysis for algorithms using noise in the first-order information is quite involved and hides the es…

For the implicit systems of first order ordinary differential equations on the plane there is presented the complete local classification of generic singularities of family of its phase curves up to smooth orbital equivalence. Besides the well known singularities of generic vector fields on the plane and the singularit…

We consider first order expansions of convex penalized estimators in high-dimensional regression problems with random designs. Our setting includes linear regression and logistic regression as special cases. For a given penalty function $h$ and the corresponding penalized estimator $\hatβ$, we construct a quantity $η$,…

Efficient algorithm for contextual bandits with first-order guarantees.

Extends first-order flexes of surfaces to second-order flexes.

Optimal first-order methods are shown to be fundamental limits in functional estimation.

First order discretizations of Langevin diffusion can achieve better generalization error with additional smoothness assumptions.

The aim of this paper is fourfold. Firstly, we introduce and study the f-ultra-harmonic maps. Secondly, we recall the geometric dynamics generated by a first order normal PDE system and we give original results regarding the geometric dynamics generated by other first order PDE systems. Thirdly, we determine the Gauss …

CEFOL uses deep learning for dynamic programming with recursive utility.

The properties of the Riemann extensions of nonriemannian spaces defined by the first order systems of differential equations are considered.

The paper explores how different network architectures learn logical functions under GOTU, finding that a min-degree-interpolator is learned.

New algorithm reduces online decision-making regret with efficient LP re-solving and parallel first-order method.

A geometric multisymplectic formulation of the classical BRST symmetry of constrained first-order classical field theories is described. To effect this we introduce graded analogues of the bundles and manifolds of the multisymplectic formulation of first-order field theories. The Lagrange-d'Alembert formalism is also d…

New lower bounds for bilevel optimization with first-order oracles.

We analyse the structure of the first order operators in bimodules introduced by A. Connes. We apply this analysis to the theory of connections on bimodules generalizing thereby several proposals.

New methods boost first-order optimization with faster rates.

Unified bounds for iterative algorithms with Gaussian data matrices.

This paper presents a new methodology to compute first-order Greeks for barrier options under the framework of path-dependent payoff functions with European, Lookback, or Asian type and with time-dependent trigger levels. In particular, we develop chain rules for Wiener path integrals between two curves that arise in t…

A new method solves a complex optimization problem efficiently.

We study first order local invariants of Vassiliev type for Lagrangian immersions with generic planar caustics. For this we produce some examples of 2-parameter families of Lagrangian maps and study their bifurcation diagrams.

We give different proofs and prove new results on the non complete solvability of some systems of complex first order p.d.e.'s, especially related to the analysis on CR manifolds.

We construct a lagrangian geometric formulation for first-order field theories using the canonical structures of first-order jet bundles, which are taken as the phase spaces of the systems in consideration. First of all, we construct all the geometric structures associated with a first-order jet bundle and, using them,…

A new first-order sampler improves diffusion probabilistic model sampling quality.

We establish that first-order methods avoid saddle points for almost all initializations. Our results apply to a wide variety of first-order methods, including gradient descent, block coordinate descent, mirror descent and variants thereof. The connecting thread is that such algorithms can be studied from a dynamical s…

Study stabilizes second-order systems to first-order dynamics.

Derives a new first order differential equation for smooth surfaces.

Decides undecidability of equations and first-order theory for Seifert 3-manifold groups.

First order methods can take extremely long to find global minima of non-convex functions.

Generalizes Hamiltonian theory for variational problems, applied to first order gravity.

A framework for decentralized optimization using first-order methods.

For a surface in the 3-dimensional real projective space, we define a Gauss map, which is a quadric in $\mathbb R^{4}$ and called the first-order Gauss map. It will be shown that the surface is a Demoulin surface if and only if the first-order Gauss map is conformal, and the surface is a projective minimal coincidence …

New methods solve optimization problems with heavy-tailed noise, improving upon existing complexity bounds.

A new method speeds up quantum state estimation.