arXiv research

Method analyzes complexity of empirical risk landscapes for generalized linear models.

We extend Kac-Rice formula to compute expected intersections of random submanifolds.

Study finds the number of modes in Gaussian kernel density estimators scales with sqrt(β log β).

Formula for critical points of chi fields on manifolds.

We establish a few formulas that compute the volume of the zero-set (or nodal set) of a function on a compact Riemannian manifold as integrals of functionals of the function and its derivatives.

The study of topological properties of random smooth maps, focusing on Kac-Rice formula and Betti numbers.

Develops calculus for random submanifolds using zonoids.

The paper analyzes local minima in high-dimensional empirical risk minimization.

We analyze the landscape of empirical risk minimization for high-dimensional models, predicting phase transitions and critical point properties.

Non-convex optimization with local search heuristics has been widely used in machine learning, achieving many state-of-art results. It becomes increasingly important to understand why they can work for these NP-hard problems on typical data. The landscape of many objective functions in learning has been conjectured to …

We study rough high-dimensional landscapes in which an increasingly stronger preference for a given configuration emerges. Such energy landscapes arise in glass physics and inference. In particular we focus on random Gaussian functions, and on the spiked-tensor model and generalizations. We thoroughly analyze the stati…

Gradient-based algorithms are effective for many machine learning tasks, but despite ample recent effort and some progress, it often remains unclear why they work in practice in optimising high-dimensional non-convex functions and why they find good minima instead of being trapped in spurious ones. Here we present a qu…

In this work we analyse quantitatively the interplay between the loss landscape and performance of descent algorithms in a prototypical inference problem, the spiked matrix-tensor model. We study a loss function that is the negative log-likelihood of the model. We analyse the number of local minima at a fixed distance …

Study on complexity of random polynomials with deterministic spikes, identifying phase transitions.

Study on variance of Laplace eigenfunctions on manifolds.

We establish formulas that give the intrinsic volumes, or curvature measures, of sublevel sets of functions defined on Riemannian manifolds as integrals of functionals of the function and its derivatives. For instance, in the Euclidean case, if $f \in \mathcal{C}^3(\mathbb{R}^n, \mathbb{R})$ and 0 is a regular value of…

Study analyzes landscape complexity of empirical loss functions with correlated data.

Consider a $d\times d$ matrix $M$ whose rows are independent centered non-degenerate Gaussian vectors $ξ_1,...,ξ_d$ with covariance matrices $Σ_1,...,Σ_d$. Denote by $\mathcal{E}_i$ the location-dispersion ellipsoid of $ξ_i:\mathcal{E}_i={\mathbf{x}\in\mathbb{R}^d : \mathbf{x}^\topΣ_i^{-1} \mathbf{x}\leqslant1}$. We sh…

Non-asymptotic tail bounds for Kostlan-Shub-Smale field on sphere

New test detects sparse alternatives in Gaussian random fields.

Improved iterative methods for risk parity portfolio weights.

We describe a novel optimization method for finite sums (such as empirical risk minimization problems) building on the recently introduced SAGA method. Our method achieves an accelerated convergence rate on strongly convex smooth problems. Our method has only one parameter (a step size), and is radically simpler than o…

A new method combines Laplace and Variational Bayes for scalable inference.

Unified framework for model explanation methods based on feature removal.

This work reviews and evaluates methods for predicting prediction intervals in regression problems.

Derives kernel PCA with Nyström method for scalability.

In this paper, the author considers the numerical computation of CVA for large systems by Mote Carlo methods. He introduces two types of stochastic mesh methods for the computations of CVA. In the first method, stochastic mesh method is used to obtain the future value of the derivative contracts. In the second method, …

New method combines spectral and sparse methods for Gaussian processes.

Simple stochastic Newton and cubic Newton methods with fast convergence.

A comprehensive benchmark of 15 scRNA-seq imputation methods across various datasets and analyses.

New methods using natural gradient for structured optimization.

Recently, {\it stochastic momentum} methods have been widely adopted in training deep neural networks. However, their convergence analysis is still underexplored at the moment, in particular for non-convex optimization. This paper fills the gap between practice and theory by developing a basic convergence analysis of t…

We investigate methods for pricing American options under the variance gamma model. The variance gamma process is a pure jump process which is constructed by replacing the calendar time by the gamma time in a Brownian motion with drift, which makes it a time-changed Brownian motion. In general, the finite difference me…

A new method speeds up deep neural network training.

We propose a new stochastic dual coordinate ascent technique that can be applied to a wide range of regularized learning problems. Our method is based on Alternating Direction Multiplier Method (ADMM) to deal with complex regularization functions such as structured regularizations. Although the original ADMM is a batch…

NCG methods improve shape optimization efficiency.

Geometric methods study 3-manifold splittings.

We propose two localized Radial Basis Function (RBF) methods, the Radial Basis Function Partition of Unity method (RBF-PUM) and the Radial Basis Function generated Finite Differences method (RBF-FD), for solving financial derivative pricing problems arising from market models with multiple stochastic factors. We demons…

Proposes UTC method for stock price prediction with uncertainty quantification.

Various approaches to gene selection for cancer classification based on microarray data can be found in the literature and they may be grouped into two categories: univariate methods and multivariate methods. Univariate methods look at each gene in the data in isolation from others. They measure the contribution of a p…

Survey of spectral, probabilistic, and deep metric learning methods.

A novel weighted feature selection method using fuzzy sets improves classification accuracy and stability.

New method improves accuracy in computing implied volatility.

The paper examines Wiener process for LID estimation methods.

We evaluate how modern outlier detection methods perform in identifying outliers in e-commerce conversion rate data. Based on the limitations identified, we then present a novel method to detect outliers in e-commerce conversion rate. This unsupervised method is made more business relevant by letting it automatically a…

Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models. Despite their theoretical appeal, the applicability of these methods to real data is still limited due to a lack of robustness to model misspecification. In this paper we present a hierarchical approach to methods…

We propose an optimization method for minimizing the finite sums of smooth convex functions. Our method incorporates an accelerated gradient descent (AGD) and a stochastic variance reduction gradient (SVRG) in a mini-batch setting. Unlike SVRG, our method can be directly applied to non-strongly and strongly convex prob…

We discuss the relevance of the recent Machine Learning (ML) literature for economics and econometrics. First we discuss the differences in goals, methods and settings between the ML literature and the traditional econometrics and statistics literatures. Then we discuss some specific methods from the machine learning l…