arXiv research

kTULA improves sampling from distributions with super-linear log-gradients.

A novel AIRLS algorithm for multiaffine variable relations in high-dimensional problems.

Paper analyzes EM algorithm's trajectory in 2MLR, revealing cycloid behavior.

AM converges super-linearly for solving mixed linear regression problems.

A new optimization algorithm improves convergence in unconstrained problems.

KBB algorithm reduces sample complexity for policy evaluation in general state spaces.

In this paper, we introduce DICOD, a convolutional sparse coding algorithm which builds shift invariant representations for long signals. This algorithm is designed to run in a distributed setting, with local message passing, making it communication efficient. It is based on coordinate descent and uses locally greedy u…

We analyze the convergence behaviour of a recently proposed algorithm for regularized estimation called Dual Augmented Lagrangian (DAL). Our analysis is based on a new interpretation of DAL as a proximal minimization algorithm. We theoretically show under some conditions that DAL converges super-linearly in a non-asymp…

We propose a randomized second-order method for optimization known as the Newton Sketch: it is based on performing an approximate Newton step using a randomly projected or sub-sampled Hessian. For self-concordant functions, we prove that the algorithm has super-linear convergence with exponentially high probability, wi…

New algorithms improve sampling from complex distributions.

Paper develops RGN method for estimating low-rank tensors from noisy measurements.

In many learning tasks, structural models usually lead to better interpretability and higher generalization performance. In recent years, however, the simple structural models such as lasso are frequently proved to be insufficient. Accordingly, there has been a lot of work on "superposition-structured" models where mul…

TUSLA algorithm solves non-convex optimization problems with ReLU activations.

We consider the problem of solving mixed random linear equations with $k$ components. This is the noiseless setting of mixed linear regression. The goal is to estimate multiple linear models from mixed samples in the case where the labels (which sample corresponds to which model) are not observed. We give a tractable a…

Sharp convergence analysis for nonconvex regression models.

New method speeds up kernel-based machine learning for force field reconstruction.

New algorithms improve robust covariance estimation efficiency.

The paper establishes curvature estimates for solitons in higher dimensions.

We consider a stochastic volatility model which captures relevant stylized facts of financial series, including the multi-scaling of moments. The volatility evolves according to a generalized Ornstein-Uhlenbeck processes with super-linear mean reversion. Using large deviations techniques, we determine the asymptotic sh…

Study of deep Stable neural networks with various activation functions.

We develop the theory of linear algebra over a (Z_2)^n-commutative algebra (n in N), which includes the well-known super linear algebra as a special case (n=1). Examples of such graded-commutative algebras are the Clifford algebras, in particular the quaternion algebra H. Following a cohomological approach, we introduc…

We consider the problem of performing linear regression over a stream of $d$-dimensional examples, and show that any algorithm that uses a subquadratic amount of memory exhibits a slower rate of convergence than can be achieved without memory constraints. Specifically, consider a sequence of labeled examples $(a_1,b_1)…

Starting from an exact relationship between news, threshold and price return distributions in the stationary state, I discuss the ability of the Ghoulmie-Cont-Nadal model of traders to produce fat-tailed price returns. Under normal conditions, this model is not able to transform Gaussian news into fat-tailed price retu…

Spectral methods improve signal recovery in mixed GLMs with precise asymptotics.

New tensor recovery method improves efficiency under strict complementarity.

Wide adoption of complex RNN based models is hindered by their inference performance, cost and memory requirements. To address this issue, we develop AntMan, combining structured sparsity with low-rank decomposition synergistically, to reduce model computation, size and execution time of RNNs while attaining desired ac…

We present a novel active learning algorithm for community detection on networks. Our proposed algorithm uses a Maximal Expected Model Change (MEMC) criterion for querying network nodes label assignments. MEMC detects nodes that maximally change the community assignment likelihood model following a query. Our method is…

Study on smoothness of solutions to nonlinear equations on Riemannian manifolds.

Localized SVMs maintain SVM's consistency properties for large datasets.

Newly available data on the spatial distribution of retail activities in cities makes it possible to build models formalized at the level of the single retailer. Current models tackle consumer location choices at an aggregate level and the opportunity new data offers for modeling at the retail unit level lacks a theore…

We develop efficient algorithms for robust PCA that handle outliers.

We study the problem of identity testing of markov chains. In this setting, we are given access to a single trajectory from a markov chain with unknown transition matrix $Q$ and the goal is to determine whether $Q = P$ for some known matrix $P$ or $\text{Dist}(P, Q) \geq ε$ where $\text{Dist}$ is suitably defined. In r…

Tensor rank and low-rank tensor decompositions have many applications in learning and complexity theory. Most known algorithms use unfoldings of tensors and can only handle rank up to $n^{\lfloor p/2 \rfloor}$ for a $p$-th order tensor in $\mathbb{R}^{n^p}$. Previously no efficient algorithm can decompose 3rd order ten…

Equity auctions show linear price impact up to a large volume, then non-linear.

The paper improves SVM and localized SVM stability under triple perturbations.

UMFI improves feature importance methods by reducing runtime and enhancing performance.

We present the Parallel, Forward-Backward with Pruning (PFBP) algorithm for feature selection (FS) in Big Data settings (high dimensionality and/or sample size). To tackle the challenges of Big Data FS PFBP partitions the data matrix both in terms of rows (samples, training examples) as well as columns (features). By e…

SVM and linear regression models coincide in high dimensions.

We consider the problem of reconstructing a rank-$k$ $n \times n$ matrix $M$ from a sampling of its entries. Under a certain incoherence assumption on $M$ and for the case when both the rank and the condition number of $M$ are bounded, it was shown in \cite{CandesRecht2009, CandesTao2010, keshavan2010, Recht2011, Jain2…

Large deep learning models offer significant accuracy gains, but training billions to trillions of parameters is challenging. Existing solutions such as data and model parallelisms exhibit fundamental limitations to fit these models into limited device memory, while obtaining computation, communication and development …

One of the limiting factors of using support vector machines (SVMs) in large scale applications are their super-linear computational requirements in terms of the number of training samples. To address this issue, several approaches that train SVMs on many small chunks of large data sets separately have been proposed in…

One of the earliest conjectures in computational learning theory-the Sample Compression conjecture-asserts that concept classes (equivalently set systems) admit compression schemes of size linear in their VC dimension. To-date this statement is known to be true for maximum classes---those that possess maximum cardinali…

We present a plausible micro-founded model for the previously postulated power law finite time singular form of the crash hazard rate in the Johansen-Ledoit-Sornette model of rational expectation bubbles. The model is based on a percolation picture of the network of traders and the concept that clusters of connected tr…

New method for initializing low-rank neural networks improves performance.

New schemes improve error estimates for sampling from non-log-concave distributions.

Optimal algorithm for selecting high-quality arms from infinite bandit arms.

New algorithms for differentially private optimization in convex and non-convex settings with near-optimal rates.

Given a matrix $A\in \mathbb{R}^{n\times d}$ and a vector $b\in \mathbb{R}^n$, we consider the regression problem with $\ell_\infty$ guarantees: finding a vector $x'\in \mathbb{R}^d$ such that $\|x'-x^*\|_\infty \leq \fracε{\sqrt{d}}\cdot \|Ax^*-b\|_2\cdot \|A^\dagger\|$ where $x^*=\arg\min_{x\in \mathbb{R}^d}\|Ax-b\|…