arXiv research

New AI-block models for clustering high-dimensional variables based on maxima of random processes.

In this paper we consider the problem of finding stable maxima of expensive (to evaluate) functions. We are motivated by the optimisation of physical and industrial processes where, for some input ranges, small and unavoidable variations in inputs lead to unacceptably large variation in outputs. Our approach uses multi…

Dimensionality reduction is one of the key issues in the design of effective machine learning methods for automatic induction. In this work, we introduce recursive maxima hunting (RMH) for variable selection in classification problems with functional data. In this context, variable selection techniques are especially a…

We provide two fundamental results on the population (infinite-sample) likelihood function of Gaussian mixture models with $M \geq 3$ components. Our first main result shows that the population likelihood function has bad local maxima even in the special case of equally-weighted mixtures of well-separated and spherical…

Develops a machine learning method for parameter estimation in branching processes models.

Three types of Einstein metrics are disqualified as potential local maxima.

Proposes efficient FOBO algorithms for global maxima of expensive functions.

Develops FSC for maxima nominated samples, improving classification in rare-event data.

A large number of problems in optimization, machine learning, signal processing can be effectively addressed by suitable semidefinite programming (SDP) relaxations. Unfortunately, generic SDP solvers hardly scale beyond instances with a few hundreds variables (in the underlying combinatorial problem). On the other hand…

The function on the Teichmueller space of complete, orientable, finite-area hyperbolic surfaces of a fixed topological type that assigns to a hyperbolic surface its maximal injectivity radius has no local maxima that are not global maxima.

We construct infinite families of closed hyperbolic surfaces that are local maxima for the systole function on their respective moduli spaces. The systole takes values along a linearly divergent sequence $(L_n)_{n\geq 1}$ at these local maxima. The only surface corresponding to $L_1\approx 3.057$ is the Bolza surface i…

The study finds multiple maxima for eigenfunctions on positively curved spheres.

SGD with large learning rates can converge to local maxima.

Random utility theory models an agent's preferences on alternatives by drawing a real-valued score on each alternative (typically independently) from a parameterized distribution, and then ranking the alternatives according to scores. A special case that has received significant attention is the Plackett-Luce model, fo…

Non-convex optimization with local search heuristics has been widely used in machine learning, achieving many state-of-art results. It becomes increasingly important to understand why they can work for these NP-hard problems on typical data. The landscape of many objective functions in learning has been conjectured to …

We study optimal investment in an asset subject to risk of default for investors that rely on different levels of information. The price dynamics can include noises both from a Wiener process and a Poisson random measure with infinite activity. The default events are modelled via a counting process in line with large p…

New method models precipitation extremes and spatial dependence.

New method resolves nonidentifiability in mixture models.

In this note, we show that some F-harmonic maps into spheres are global maxima of the variations of their energy functional on the conformal group of the sphere. Our result extends partially those obtained in [15] and [17] for harmonic and p-harmonic maps.

We consider the signed density of the extremal points of (two-dimensional) scalar fields with a Gaussian distribution. We assign a positive unit charge to the maxima and minima of the function and a negative one to its saddles. At first, we compute the average density for a field in half-space with Dirichlet boundary c…

For a translation surface, we define the systole to be the length of the shortest saddle connection. We give a characterization of the maxima of the systole function on a stratum, and give a family of examples providing local but nonglobal maxima on each stratum of genus at least three. We further study the relation be…

Nested sampling is an efficient algorithm for the calculation of the Bayesian evidence and posterior parameter probability distributions. It is based on the step-by-step exploration of the parameter space by Monte Carlo sampling with a series of values sets called live points that evolve towards the region of interest,…

This paper describes a new form of unsupervised learning, whose input is a set of unlabeled points that are assumed to be local maxima of an unknown value function v in an unknown subset of the vector space. Two functions are learned: (i) a set indicator c, which is a binary classifier, and (ii) a comparator function h…

Eigenfunction maxima inside high-d nodal domains.

In this paper, we investigate the geometry of a general class of gradient flows with multiple local maxima. we decompose the underlying space into disjoint regions of attraction and establish the adjacency criterion. The criterion states a necessary and sufficient condition for two regions of attraction of stable equil…

This paper optimizes Bayesian acquisition functions in Gaussian Processes for better optimization.

Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Prob…

Researchers solve the realization of Jordan-Kronecker invariants in Lie algebras.

In this paper, we show that Generative Adversarial Networks (GANs) suffer from catastrophic forgetting even when they are trained to approximate a single target distribution. We show that GAN training is a continual learning problem in which the sequence of changing model distributions is the sequence of tasks to the d…

Study on complexity of random polynomials with deterministic spikes, identifying phase transitions.

We present a first procedure that can estimate -- with statistical consistency guarantees -- any local-maxima of a density, under benign distributional conditions. The procedure estimates all such local maxima, or $\textit{modal-sets}$, of any bounded shape or dimension, including usual point-modes. In practice, modal-…

Two new invariants that are closely related to Milnor's curvature-torsion invariant are introduced. The first, the spiral index of a knot, captures the minimum number of maxima among all knot projections that are free of inflection points. This invariant is closely related to both the bridge and braid index of the knot…

We present an efficient block-diagonal ap- proximation to the Gauss-Newton matrix for feedforward neural networks. Our result- ing algorithm is competitive against state- of-the-art first order optimisation methods, with sometimes significant improvement in optimisation performance. Unlike first-order methods, for whic…

New test detects sparse alternatives in Gaussian random fields.

Study improves traffic prediction intervals for minor roads.

We consider the problem of estimating a large rank-one tensor ${\boldsymbol u}^{\otimes k}\in({\mathbb R}^{n})^{\otimes k}$, $k\ge 3$ in Gaussian noise. Earlier work characterized a critical signal-to-noise ratio $λ_{Bayes}= O(1)$ above which an ideal estimator achieves strictly positive correlation with the unknown ve…

In the modal approach to clustering, clusters are defined as the local maxima of the underlying probability density function, where the latter can be estimated either non-parametrically or using finite mixture models. Thus, clusters are closely related to certain regions around the density modes, and every cluster corr…

Gibbs sampling, as a model learning method, is known to produce the most accurate results available in a variety of domains, and is a de facto standard in these domains. Yet, it is also well known that Gibbs random walks usually have bottlenecks, sometimes termed "local maxima", and thus samplers often return suboptima…

We study extreme values of group-indexed stable random fields for discrete groups $G$ acting geometrically on spaces $X$ in the following cases: 1) $G$ acts freely, properly discontinuously by isometries on a CAT(-1) space $X$, 2) $G$ is a lattice in a higher rank Lie group, acting on a symmetric space $X$, 3) $G$ is t…

It is natural to ask whether solvsolitons are global maxima for the Ricci pinching functional F:=scal^2/|Ric|^2 on the set of all left-invariant metrics on a given solvable Lie group S, as it is to ask whether they are the only global maxima. A positive answer to both questions was given in a recent paper by the same a…

This paper concerns thin presentations of knots K in closed 3-manifolds M^3 which produce S^3 by Dehn surgery, for some slope gamma. If M does not have a lens space as a connected summand, we first prove that all such thin presentations, with respect to any spine of M have only local maxima. If M is a lens space and K …

Gaussian processes (GPs) are Bayesian nonparametric generative models that provide interpretability of hyperparameters, admit closed-form expressions for training and inference, and are able to accurately represent uncertainty. To model general non-Gaussian data with complex correlation structure, GPs can be paired wit…

Deep neural networks can solve optimal stopping problems without dimensionality issues.

Long-range correlation and fluctuation in the gold market time series of world's two leading gold consuming countries, namely China and India, are studied. For both the market series during the period 1985-2013 we observe a long-range persistence of memory in the sequences of maxima (minima) of returns in successive ti…

Study non-orientable link cobordisms using Floer homologies to prove inequalities.

Drawdowns measuring the decline in value from the historical running maxima over a given period of time, are considered as extremal events from the standpoint of risk management. To date, research on the topic has mainly focus on the side of severity by studying the first drawdown over certain pre-specified size. In th…

Frequency estimation is a fundamental problem in signal processing, with applications in radar imaging, underwater acoustics, seismic imaging, and spectroscopy. The goal is to estimate the frequency of each component in a multisinusoidal signal from a finite number of noisy samples. A recent machine-learning approach u…

Software finds ideal polyhedra with rational dihedral angles and volume maxima.