arXiv research

Researchers find optimal dictionaries for minimizing average squared coefficients in random vector representations.

Deep reinforcement learning (DRL) methods such as the Deep Q-Network (DQN) have achieved state-of-the-art results in a variety of challenging, high-dimensional domains. This success is mainly attributed to the power of deep neural networks to learn rich domain representations for approximating the value function or pol…

A fast cross-validation method for high-dimensional data.

Paper develops efficient estimator for Hawkes processes using representer theorem.

New method quantifies life insurance risk using Monte Carlo simulations.

CD converges linearly for MCP/SCAD penalized least squares.

Riesz regression connects to density ratio estimation for causal inference.

Study shows how mini-batch GD with random reshuffling affects least squares regression dynamics.

New algorithm extracts shared latent space for cortico-muscular interactions.

The derivation of statistical properties for Partial Least Squares regression can be a challenging task. The reason is that the construction of latent components from the predictor variables also depends on the response variable. While this typically leads to good performance and interpretable models in practice, it ma…

RFRBoost uses random features to boost deep residual neural networks, improving performance and computational efficiency.

New method corrects least-squares temporal difference for better lambda-return estimation.

Principal Component Analysis (PCA) is a very successful dimensionality reduction technique, widely used in predictive modeling. A key factor in its widespread use in this domain is the fact that the projection of a dataset onto its first $K$ principal components minimizes the sum of squared errors between the original …

Cross validation residuals extended to GLS models.

Illustrates interleaved learning with Kalman Filter for linear least squares.

We present a new link invariant which depends on a representation of the link group in SO(3). The computer calculations indicate that an abelian version of this invariant is expressed in terms of the Alexander polynomial of the link. On the other hand, if we use non abelian representation, we get the squared non abelia…

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…

BCRL learns a Bellman complete representation for offline RL policy evaluation.

We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that …

Simplified proof shows SGD optimality for least squares.

The paper improves Kaczmarz algorithm with momentum for linear least squares.

New algorithm improves online binary classification with constant time complexity.

Reduced-rank method improves least-squares regression under output regularity.

A new learning method for complex functions using kernel and range space.

Paper uses deep learning to solve PDEs without supervision.

The paper extends NUP representations to factor graphs for better estimation.

New deep learning solver for high-dimensional derivative pricing.

Proposes a partitioned least squares model for feature grouping.

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

Correcting bias in least squares regression with volume-rescaled sampling.

A new algorithm solves nonnegative least squares faster with nonnegative data.

The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…

A structured prediction method for ranking labels.

We consider the problem of predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the parameters of the linear combination. When the input distribution is known, there already exists an algorithm having an expected excess…

The paper identifies saddlepoints in unsupervised auto-encoding neural nets.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

Analytic network learning simplifies multilayer network training without gradients.

Dictionaries are collections of vectors used for representations of random vectors in Euclidean spaces. Recent research on optimal dictionaries is focused on constructing dictionaries that offer sparse representations, i.e., $\ell_0$-optimal representations. Here we consider the problem of finding optimal dictionaries …

Least Squares EM converges globally for log-concave mixtures.

Study proves NN matching is equivalent to Riesz regression for debiased machine learning.

We introduce a novel semi-supervised version of the least squares classifier. This implicitly constrained least squares (ICLS) classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-supervised method…

The paper analyzes the risk of a least squares estimator under a spike covariance model.

New method speeds up solving L0-regularized least-squares problems.

Least squares estimator fails to achieve optimal risk in bounded distributions, but non-linear predictors can.

Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…

Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.

New algorithm converts data into sub-gaussian designs efficiently.

We introduce the implicitly constrained least squares (ICLS) classifier, a novel semi-supervised version of the least squares classifier. This classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-s…