arXiv research

Deep models generate geometric objects with global properties.

The study examines when MAML's objective has a benign landscape.

MAML optimizes shared priors for subtasks in a nonconvex meta-objective.

In this paper, we consider the problem of unsupervised video object segmentation via background subtraction. Specifically, we pose the nonsemantic extraction of a video's moving objects as a nonconvex optimization problem via a sum of sparse and low-rank matrices. The resulting formulation, a nonnegative variant of rob…

We propose a general theory for studying the \xl{landscape} of nonconvex \xl{optimization} with underlying symmetric structures \tz{for a class of machine learning problems (e.g., low-rank matrix factorization, phase retrieval, and deep linear neural networks)}. In specific, we characterize the locations of stationary …

In this paper we provide a \emph{global} investigation of the geometry of parallelizable manifolds (or absolute parallelism geometry) frequently used for application. We discuss the different linear connections and curvature tensors from a global point of view. We give an existence and uniqueness theorem for a remarkab…

We present a new model for prediction markets, in which we use risk measures to model agents and introduce a market maker to describe the trading process. This specific choice on modelling tools brings us mathematical convenience. The analysis shows that the whole market effectively approaches a global objective, despi…

Learning new representations of input observations in machine learning is often tackled using a factorization of the data. For many such problems, including sparse coding and matrix completion, learning these factorizations can be difficult, in terms of efficiency and to guarantee that the solution is a global minimum.…

SogCLR uses small batch sizes for global contrastive learning, achieving similar performance to SimCLR.

A new method for clearing liability networks using sheaves on directed hypergraphs.

AGGLIO optimizes non-convex functions with local convexity guarantees.

Paper tackles federated learning with personalised bandit algorithms.

Optimizers find approximate global minima in non-convex problems.

Algorithm optimizes collaborative learning among distributed clients using kernel-based bandits.

The paper proves convergence to global optima for a class of distributed algorithms for nonconvex optimization in network-based multi-agent settings. Agents are permitted to communicate over a time-varying undirected graph. Each agent is assumed to possess a local objective function (assumed to be smooth, but possibly …

We prove the local convergence to minima and estimates on the rate of convergence for the stochastic gradient descent method in the case of not necessarily globally convex nor contracting objective functions. In particular, the results are applicable to simple objective functions arising in machine learning.

Paper develops a new objective for hierarchical clustering in Euclidean space.

Derivative-free method solves stochastic optimization problems with noisy objectives and constraints.

Many problems in machine learning involve calculating correspondences between sets of objects, such as point clouds or images. Discrete optimal transport provides a natural and successful approach to such tasks whenever the two sets of objects can be represented in the same space, or at least distances between them can…

Theoretical analysis of vision transformers' performance with MAE and CL objectives.

Novel method for high-dimensional BO using CMA to define local regions.

New method certifies global robustness of neural networks efficiently.

MORBO improves multi-objective BO for high-dimensional problems.

Global optimization problems whose objective function is expensive to evaluate can be solved effectively by recursively fitting a surrogate function to function samples and minimizing an acquisition function to generate new samples. The acquisition step trades off between seeking for a new optimization vector where the…

This paper finds sparsest ReLU networks for interpolating data.

New method tackles rugged optimization landscapes in contact-rich scenarios.

New perspective on federated learning as posterior inference, improving optimization.

New algorithm finds global maxima in multi-modal functions.

New method guarantees global convergence in variational inference.

The aim of the present paper is to establish a global theory of conformal changes in Finsler geometry. Under this change, we obtain the relationships between the most important geometric objects associated to $(M,L)$ and the corresponding objects associated to $(M,\tilde{L})$, $\widetilde{L}=e^{σ(x)} L$ being the Finsl…

We develop parallel predictive entropy search (PPES), a novel algorithm for Bayesian optimization of expensive black-box objective functions. At each iteration, PPES aims to select a batch of points which will maximize the information gain about the global maximizer of the objective. Well known strategies exist for sug…

Paper proposes a new algorithm combining gradient descent and Langevin dynamics.

ECD algorithm speeds up non-convex optimization, offering quantum and stochastic enhancements.

A new federated algorithm reduces regret in X-armed bandit problems.

Gradients help find global optima in complex functions.

Contrastive learning properties studied, including feature suppression and hierarchical learning.

The standard reinforcement learning (RL) formulation considers the expectation of the (discounted) cumulative reward. This is limiting in applications where we are concerned with not only the expected performance, but also the distribution of the performance. In this paper, we introduce micro-objective reinforcement le…

Global optimization finds applications in a wide range of real world problems. The multi-start methods are a popular class of global optimization techniques, which are based on the ideas of conducting local searches at multiple starting points. In this work we propose a new multi-start algorithm where the starting poin…

New method for mixed-variable GSA improves material design efficiency.

Paper proposes quantum methods for optimizing machine learning functions.

In this paper, the problem of safe global maximization (it should not be confused with robust optimization) of expensive noisy black-box functions satisfying the Lipschitz condition is considered. The notion "safe" means that the objective function $f(x)$ during optimization should not violate a "safety" threshold, for…

We present a global optimization algorithm for clustering data given the ratio of likelihoods that each pair of data points is in the same cluster or in different clusters. To define a clustering solution in terms of pairwise relationships, a necessary and sufficient condition is that belonging to the same cluster sati…

This work discovers latent field effects governing interacting dynamical systems.

The Global Vectors for word representation (GloVe), introduced by Jeffrey Pennington et al. is reported to be an efficient and effective method for learning vector representations of words. State-of-the-art performance is also provided by skip-gram with negative-sampling (SGNS) implemented in the word2vec tool. In this…

A new method learns object hierarchies from images to reason about physical interactions.

Kernel models learn low-dimensional predictive subspaces from input data.

A new approach corrects bias in federated learning due to varying communication links.

We consider the problem of analyzing the heterogeneity of clustering distributions for multiple groups of observed data, each of which is indexed by a covariate value, and inferring global clusters arising from observations aggregated over the covariate domain. We propose a novel Bayesian nonparametric method reposing …