arXiv research

Improved sample complexity for Gaussian Mixture Models using Pair Correlation Factor.

Defines MER for Bayesian learning, a gap between achievable and optimal performance.

Many important optimization problems, such as the minimum spanning tree and minimum-cost flow, can be solved optimally by a greedy method. In this work, we study a learning variant of these problems, where the model of the problem is unknown and has to be learned by interacting repeatedly with the environment in the ba…

Local gap theorem for Ricci shrinkers ensures flatness if certain functionals are close to zero.

We define the concordance crosscap number of a knot as the minimum crosscap number among all the knots concordant to the knot. The four-dimensional crosscap number is the minimum first Betti number of non-orientable surfaces smoothly embedded in 4-dimensional ball, bounding the knot. Clearly the 4-dimensional crosscap …

Max-product Belief Propagation (BP) is a popular message-passing algorithm for computing a Maximum-A-Posteriori (MAP) assignment over a distribution represented by a Graphical Model (GM). It has been shown that BP can solve a number of combinatorial optimization problems including minimum weight matching, shortest path…

Verifying the robustness property of a general Rectified Linear Unit (ReLU) network is an NP-complete problem [Katz, Barrett, Dill, Julian and Kochenderfer CAV17]. Although finding the exact minimum adversarial distortion is hard, giving a certified lower bound of the minimum distortion is possible. Current available m…

Cake wavelets minimize orientation score uncertainty.

Recent research has used margin theory to analyze the generalization performance for deep neural networks (DNNs). The existed results are almost based on the spectrally-normalized minimum margin. However, optimizing the minimum margin ignores a mass of information about the entire margin distribution, which is crucial …

Paper bounds PAC RL sample complexity in deterministic MDPs.

The paper explores the information-theoretic nature of excess risk in machine learning.

Study on invariant Seifert surfaces for strongly invertible knots, showing large gaps in genus.

Logarithmic regret achieved in Q-learning with positive gap.

Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.

We consider a sparse high dimensional regression model where the goal is to recover a $k$-sparse unknown vector $β^*$ from $n$ noisy linear observations of the form $Y=Xβ^*+W \in \mathbb{R}^n$ where $X \in \mathbb{R}^{n \times p}$ has iid $N(0,1)$ entries and $W \in \mathbb{R}^n$ has iid $N(0,σ^2)$ entries. Under certa…

We analyze the regret of combinatorial Thompson sampling (CTS) for the combinatorial multi-armed bandit with probabilistically triggered arms under the semi-bandit feedback setting. We assume that the learner has access to an exact optimization oracle but does not know the expected base arm outcomes beforehand. When th…

New method improves MMD estimation without convexity assumptions.

Let $(M,g)$ be a compact Ricci-flat 4-manifold. For $p \in M$ let $K_{max}(p)$ (respectively $K_{min}(p)$) denote the maximum (respectively the minimum) of sectional curvatures at $p$. We prove that if $K_{max} (p) \le \ -c K_{min}(p)$ for all $p \in M$, for some constant $c$ with $0 \leq c < \frac{2+\sqrt 6}{4}$, th…

Understanding separation effects on parameter estimation in finite Gaussian mixtures

A new method for SSMF improves upon existing algorithms.

Study finds no statistically significant trading edge in MNQ futures signals from OHLCV data.

We tackle the problem of penalty selection of regularization on the basis of the minimum description length (MDL) principle. In particular, we consider that the design space of the penalty function is high-dimensional. In this situation, the luckiness-normalized-maximum-likelihood(LNML)-minimization approach is favorab…

The performance of spectral clustering can be considerably improved via regularization, as demonstrated empirically in Amini et. al (2012). Here, we provide an attempt at quantifying this improvement through theoretical analysis. Under the stochastic block model (SBM), and its extensions, previous results on spectral c…

Study on policy gradient for stochastic bandits using diffusion approximation.

While progress has been made in understanding the robustness of machine learning classifiers to test-time adversaries (evasion attacks), fundamental questions remain unresolved. In this paper, we use optimal transport to characterize the minimum possible loss in an adversarial classification scenario. In this setting, …

Study shows gap between uniform convergence and test error in random feature models.

In this paper, from a theoretical perspective, we study how powerful graph neural networks (GNNs) can be for learning approximation algorithms for combinatorial problems. To this end, we first establish a new class of GNNs that can solve a strictly wider variety of problems than existing GNNs. Then, we bridge the gap b…

LinMED is a new linear bandit algorithm with near-optimal regret bound.

Investigates optimal PPI strategies in jump-diffusion models to mitigate downside risk.

Our paper characterizes how ReLU affects GD's implicit bias in high-dimensional neural networks.

K-Medoids(KM) is a standard clustering method, used extensively on semi-metric data.Error analyses of KM have traditionally used an in-sample notion of error,which can be far from the true error and suffer from generalization gap. We formalize the true K-Medoid error based on the underlying data distribution.We decompo…

We study the fundamental tradeoffs between statistical accuracy and computational tractability in the analysis of high dimensional heterogeneous data. As examples, we study sparse Gaussian mixture model, mixture of sparse linear regressions, and sparse phase retrieval model. For these models, we exploit an oracle-based…

In this paper, we prove a conjecture published in 1989 and also partially address an open problem announced at the Conference on Learning Theory (COLT) 2015. With no unrealistic assumption, we first prove the following statements for the squared loss function of deep linear neural networks with any depth and any widths…

FedAvg converges linearly to global minimum in federated learning with partial participation.

The strong symmetric genus of a finite group is the minimum genus of a compact Riemann surface on which the group acts as a group of automorphisms preserving orientation. A characterization of the infinite number of groups with strong symmetric genus zero and one is well-known and the problem is finite for each strong …

New algorithms identify best policies in discounted linear MDPs efficiently.

Quantum algorithm speeds up Lasso regression by quadratically faster per iteration.

Transfer learning improves MNI's performance in high-dimensional linear regression.

Paper relaxes factor analysis for noisy data, improving robustness.

This paper explains why double descent sometimes occurs weakly or not at all from an optimization perspective.

Abstract reviews algorithms for multi-index models, focusing on polynomial-time methods and their limitations.

A directed graph $G$ is $\textit{intrinsically linked}$ if every embedding of that graph contains a non-split link $L$, where each component of $L$ is a consistently oriented cycle in $G$. A $\textit{tournament}$ is a directed graph where each pair of vertices is connected by exactly one directed edge. We consider intr…

New statistical model improves protein alignment accuracy.

In a recent work (Chattopadhyay, A. K. et al, Europhys. Lett. {\bf 91}, 58003, 2010) based on food consumption statistics, we showed how a stochastic agent based model could represent the time variation of the income distribution statistics in a developing economy, thereby defining an alternative \enquote{poverty index…

We propose a new framework for deriving screening rules for convex optimization problems. Our approach covers a large class of constrained and penalized optimization formulations, and works in two steps. First, given any approximate point, the structure of the objective function and the duality gap is used to gather in…

Generative adversarial networks have been very successful in generative modeling, however they remain relatively challenging to train compared to standard deep neural networks. In this paper, we propose new visualization techniques for the optimization landscapes of GANs that enable us to study the game vector field re…

Influence maximization, adaptive routing, and dynamic spectrum allocation all require choosing the right action from a large set of alternatives. Thanks to the advances in combinatorial optimization, these and many similar problems can be efficiently solved given an environment with known stochasticity. In this paper, …

This paper proves SGD converges to global minimum for over-parameterized ReLU networks.