arXiv research

The paper analyzes how good initial guesses affect the amount of data needed for low-rank matrix recovery.

Untrained neural networks can unfairly assign predictions to the same class.

Singular Value Decomposition (SVD) constitutes a bridge between the linear algebra concepts and multi-layer neural networks---it is their linear analogy. Besides of this insight, it can be used as a good initial guess for the network parameters, leading to substantially better optimization results.

New methods improve accuracy in detecting concentric objects.

We give convergence guarantees for estimating the coefficients of a symmetric mixture of two linear regressions by expectation maximization (EM). In particular, we show that the empirical EM iterates converge to the target parameter vector at the parametric rate, provided the algorithm is initialized in an unbounded co…

In this article we provide a formulation of empirical bayes described by Atchade (2011) to tune the hyperparameters of priors used in bayesian set up of collaborative filter. We implement the same in MovieLens small dataset. We see that it can be used to get a good initial choice for the parameters. It can also be used…

PINNs solve neuronal parameter and state estimation problems with limited data.

New proof links initial class bias to DNN trainability, challenging traditional understanding.

Next generation deep neural networks for classification hosted on embedded platforms will rely on fast, efficient, and accurate learning algorithms. Initialization of weights in learning networks has a great impact on the classification accuracy. In this paper we focus on deriving good initial weights by modeling the e…

Counterfactual data augmentations may not ensure OOD robustness if performed by a context-guessing machine.

In this article we select the unknown dimension of the feature by re- versible jump MCMC inside a simulated annealing in bayesian set up of collaborative filter. We implement the same in MovieLens small dataset. We also tune the hyper parameter by using a modified empirical bayes. It can also be used to guess an initia…

Randomly guessing weights helps analyze RL benchmarks objectively.

The Poisson equation is commonly encountered in engineering, for instance in computational fluid dynamics (CFD) where it is needed to compute corrections to the pressure field to ensure the incompressibility of the velocity field. In the present work, we propose a novel fully convolutional neural network (CNN) architec…

Forward gradients improve neural network training without backpropagation issues.

The paper introduces negative controls to evaluate causal discovery algorithms, improving their reliability.

Stochastic differential equations are an important modeling class in many disciplines. Consequently, there exist many methods relying on various discretization and numerical integration schemes. In this paper, we propose a novel, probabilistic model for estimating the drift and diffusion given noisy observations of the…

We consider the fundamental problem of solving quadratic systems of equations in $n$ variables, where $y_i = |\langle \boldsymbol{a}_i, \boldsymbol{x} \rangle|^2$, $i = 1, \ldots, m$ and $\boldsymbol{x} \in \mathbb{R}^n$ is unknown. We propose a novel method, which starting with an initial guess computed by means of a …

Understanding deep neural networks is a major research objective with notable experimental and theoretical attention in recent years. The practical success of excessively large networks underscores the need for better theoretical analyses and justifications. In this paper we focus on layer-wise functional structure and…

Improved bounds for Black-Scholes volatility lead to faster root-finding.

Anderson acceleration (or Anderson mixing) is an efficient acceleration method for fixed point iterations $x_{t+1}=G(x_t)$, e.g., gradient descent can be viewed as iteratively applying the operation $G(x) \triangleq x-α\nabla f(x)$. It is known that Anderson acceleration is quite efficient in practice and can be viewed…

Using a bondholder who seeks to determine when to sell his bond as our motivating example, we revisit one of Larry Shepp's classical theorems on optimal stopping. We offer a novel proof of Theorem 1 from from \cite{Shepp}. Our approach is that of guessing the optimal control function and proving its optimality with mar…

Gradient descent amplifies random features in neural networks to useful ones.

In this article we present a method by which we can reduce a time series into a single point in $\mathbb{R}^{13}$. We have chosen 13 dimensions so as to prevent too many points from being labeled as "noise." When using a Euclidean (or Mahalanobis) metric, a simple clustering algorithm will with near certainty label the…

Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one. It is a classic problem, and the natural and empirically most popular approach to its solution has been the E…

GUESS improves surrogate model accuracy with adaptive sampling.

We describe a novel family of models of multi- layer feedforward neural networks in which the activation functions are encoded via penalties in the training problem. Our approach is based on representing a non-decreasing activation function as the argmin of an appropriate convex optimiza- tion problem. The new framewor…

A new algorithm finds minimizers in dueling optimization with a monotone adversary.

Novel probabilistic solver speeds up solving related linear systems.

Improved iterative methods for risk parity portfolio weights.

State-of-the-art password guessing tools, such as HashCat and John the Ripper, enable users to check billions of passwords per second against password hashes. In addition to performing straightforward dictionary attacks, these tools can expand password dictionaries using password generation rules, such as concatenation…

Cold-start problems are long-standing challenges for practical recommendations. Most existing recommendation algorithms rely on extensive observed data and are brittle to recommendation scenarios with few interactions. This paper addresses such problems using few-shot learning and meta learning. Our approach is based o…

Unhinged loss minimization fails to improve classifier accuracy for simple data.

We study a spectral initialization method that serves a key role in recent work on estimating signals in nonconvex settings. Previous analysis of this method focuses on the phase retrieval problem and provides only performance bounds. In this paper, we consider arbitrary generalized linear sensing models and present a …

We study a policy gradient method with L2 regularization for MAB problems.

We study the problem of finding strain-minimising stream surfaces in a divergence-free vector field. These surfaces are generated by motions of seed curves that propagate through the field in a strain minimising manner, i.e., they move without stretching or shrinking, preserving the length of their arbitrary arc. In ge…

Paper defines when early exercise of American options is optimal under negative rates.

Machine-generated interpretations do not improve users' guessing accuracy in image classifiers.

Study finds users mostly use recent market and decision information to guess market direction.

A method to derive Lagrangians from field equations in metric-affine theories of gravity.

In machine learning, Feature Selection (FS) is a major part of efficient algorithm. It fuels the algorithm and is the starting block for our prediction. In this paper, we present a new method, called Optimal Coordinate Ascent (OCA) that allows us selecting features among block and individual features. OCA relies on coo…

Paper solves best approximation by exponential functions for economic data.

We collect a few guesses on possible implications of a lower bound on the scalar curvature of a Riemannian manifold on the size and shape of this manifold.

This work explores feature learning tradeoffs in neural networks.

Gradient descent benefits from tangent kernel advantages under specific conditions.

We investigate the use of attentional neural network layers in order to learn a `behavior characterization' which can be used to drive novelty search and curiosity-based policies. The space is structured towards answering a particular distribution of questions, which are used in a supervised way to train the attentiona…

Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning models efficiently. First-order methods such as gradient descent are usually the methods of choice for training machine learning models. However, these methods converge to saddle points for certain ch…

Clustering is a fundamental problem in statistics and machine learning. Lloyd's algorithm, proposed in 1957, is still possibly the most widely used clustering algorithm in practice due to its simplicity and empirical performance. However, there has been little theoretical investigation on the statistical and computatio…

This paper develops a new class of nonconvex regularizers for low-rank matrix recovery. Many regularizers are motivated as convex relaxations of the matrix rank function. Our new factor group-sparse regularizers are motivated as a relaxation of the number of nonzero columns in a factorization of the matrix. These nonco…