arXiv research

Estimates change point in high dimensional time series models.

We prove new fast learning rates for the one-vs-all multiclass plug-in classifiers trained either from exponentially strongly mixing data or from data generated by a converging drifting distribution. These are two typical scenarios where training data are not iid. The learning rates are obtained under a multiclass vers…

Centered plug-in estimators reduce bias in Wasserstein distance estimation.

The paper optimizes hyperplanes for binary classification in high-dimensional data with latent Gaussian mixtures.

Convolutional neural networks handle rotated image symmetries without dimensionality issues.

BoostTransformer uses boosting to improve transformer efficiency and accuracy.

Study bridges welfare maximization and CATE estimation in policy learning.

Study uses Wasserstein distance to identify causal orders and unmix sources.

ULFS-KDPE estimates parameters efficiently without influence functions.

CD converges linearly for MCP/SCAD penalized least squares.

We study the adaptive estimation of copula correlation matrix $Σ$ for the semi-parametric elliptical copula model. In this context, the correlations are connected to Kendall's tau through a sine function transformation. Hence, a natural estimate for $Σ$ is the plug-in estimator $\hatΣ$ with Kendall's tau statistic. We …

Illustrates interleaved learning with Kalman Filter for linear least squares.

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…

Cross validation residuals are well known for the ordinary least squares model. Here leave-M-out cross validation is extended to generalised least squares. The relationship between cross validation residuals and Cook's distance is demonstrated, in terms of an approximation to the difference in the generalised residual …

We consider the classic supervised learning problem, where a continuous non-negative random label $Y$ (i.e. a random duration) is to be predicted based upon observing a random vector $X$ valued in $\mathbb{R}^d$ with $d\geq 1$ by means of a regression rule with minimum least square error. In various applications, rangi…

We compare the risk of ridge regression to a simple variant of ordinary least squares, in which one simply projects the data onto a finite dimensional subspace (as specified by a Principal Component Analysis) and then performs an ordinary (un-regularized) least squares regression in this subspace. This note shows that …

We address the problem of estimating the parameters of a time-homogeneous Markov chain given only noisy, aggregate data. This arises when a population of individuals behave independently according to a Markov chain, but individual sample paths cannot be observed due to limitations of the observation process or the need…

The paper improves Kaczmarz algorithm with momentum for linear least squares.

We formalize AURC and develop estimators for SC systems.

New algorithm improves online binary classification with constant time complexity.

Reduced-rank method improves least-squares regression under output regularity.

Simplified plug-in loss approximates EDL for reliable uncertainty estimation.

Proposes a partitioned least squares model for feature grouping.

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

The paper proves statistical consistency and fairness guarantees for a plug-in algorithm.

The kernel least mean squares (KLMS) algorithm is a computationally efficient nonlinear adaptive filtering method that "kernelizes" the celebrated (linear) least mean squares algorithm. We demonstrate that the least mean squares algorithm is closely related to the Kalman filtering, and thus, the KLMS can be interpreted…

A new algorithm solves nonnegative least squares faster with nonnegative data.

Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.

The paper identifies saddlepoints in unsupervised auto-encoding neural nets.

The paper proposes a least squares method for binary compressive sampling with low intrinsic dimension signals.

Boosting CNNs with dynamic feature selection and boosting weights improves accuracy and efficiency.

This paper studies an unsupervised deep learning-based numerical approach for solving partial differential equations (PDEs). The approach makes use of the deep neural network to approximate solutions of PDEs through the compositional construction and employs least-squares functionals as loss functions to determine para…

We propose a new forward-backward stochastic differential equation solver for high-dimensional derivatives pricing problems by combining deep learning solver with least square regression technique widely used in the least square Monte Carlo method for the valuation of American options. Our numerical experiments demonst…

We introduce a novel semi-supervised version of the least squares classifier. This implicitly constrained least squares (ICLS) classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-supervised method…

New method speeds up solving L0-regularized least-squares problems.

Least squares estimator fails to achieve optimal risk in bounded distributions, but non-linear predictors can.

Sparse linear regression, which entails finding a sparse solution to an underdetermined system of linear equations, can formally be expressed as an $l_0$-constrained least-squares problem. The Orthogonal Least-Squares (OLS) algorithm sequentially selects the features (i.e., columns of the coefficient matrix) to greedil…

We introduce the implicitly constrained least squares (ICLS) classifier, a novel semi-supervised version of the least squares classifier. This classifier minimizes the squared loss on the labeled data among the set of parameters implied by all possible labelings of the unlabeled data. Unlike other discriminative semi-s…

We prove the statistical consistency of kernel Partial Least Squares Regression applied to a bounded regression learning problem on a reproducing kernel Hilbert space. Partial Least Squares stands out of well-known classical approaches as e.g. Ridge Regression or Principal Components Regression, as it is not defined as…

Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…

This book introduces linear models and their theories rigorously.

The ratio of two probability densities can be used for solving various machine learning tasks such as covariate shift adaptation (importance sampling), outlier detection (likelihood-ratio test), and feature selection (mutual information). Recently, several methods of directly estimating the density ratio have been deve…

The least-squares support vector machine is a frequently used kernel method for non-linear regression and classification tasks. Here we discuss several approximation algorithms for the least-squares support vector machine classifier. The proposed methods are based on randomized block kernel matrices, and we show that t…

Efficiently estimates private least squares with linear error growth.

Here, we provide a supplementary material for Takayuki Osogami, "Uncorrected least-squares temporal difference with lambda-return," which appears in {\it Proceedings of the 34th AAAI Conference on Artificial Intelligence} (AAAI-20).

We prove strong consistency and asymptotic normality of least squares estimators for the subcritical Heston model based on continuous time observations. We also present some numerical illustrations of our results.

We study three fundamental statistical-learning problems: distribution estimation, property estimation, and property testing. We establish the profile maximum likelihood (PML) estimator as the first unified sample-optimal approach to a wide range of learning tasks. In particular, for every alphabet size $k$ and desired…

Improved Least-Squares Monte Carlo with finite-difference ansatz.