arXiv research

Proposes variational autoencoder for efficient MMSE estimation.

Study on LMMSE estimation with model mismatch, quantifying MSE trade-offs.

Robust diffusion adaptive estimation algorithms based on the maximum correntropy criterion (MCC), including adaptation to combination MCC and combination to adaptation MCC, are developed to deal with the distributed estimation over network in impulsive (long-tailed) noise environments. The cost functions used in distri…

Despite the simplicity and intuitive interpretation of Minimum Mean Squared Error (MMSE) estimators, their effectiveness in certain scenarios is questionable. Indeed, minimizing squared errors on average does not provide any form of stability, as the volatility of the estimation error is left unconstrained. When this v…

We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace $U$ and its estimate $\hat{U}$ may not be adequate as the MSE is not the natural metric in the Gra…

The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…

We study risk of the minimum norm linear least squares estimator in when the number of parameters $d$ depends on $n$, and $\frac{d}{n} \rightarrow \infty$. We assume that data has an underlying low rank structure by restricting ourselves to spike covariance matrices, where a fixed finite number of eigenvalues grow with…

The paper examines prediction and estimation risks of ridgeless least squares under general error assumptions.

We propose a minimum distance estimation method for robust regression in sparse high-dimensional settings. The traditional likelihood-based estimators lack resilience against outliers, a critical issue when dealing with high-dimensional noisy data. Our method, Minimum Distance Lasso (MD-Lasso), combines minimum distanc…

Dual Bayesian Affine Estimators for Wiener-type state-space models

Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs bas…

This study explains gradient flow dynamics in neural networks for small initialisation.

Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. Th…

There is some theoretical evidence that deep neural networks with multiple hidden layers have a potential for more efficient representation of multidimensional mappings than shallow networks with a single hidden layer. The question is whether it is possible to exploit this theoretical advantage for finding such represe…

Paper uses DFL to optimize portfolio risk and outperforms conventional methods.

Key to the imposition of appropriate minimum capital requirements on a daily basis requires accurate volatility estimation. Here, measures are presented based on discrete estimation of aggregated high frequency UK futures realisations underpinned by a continuous time framework. Squared and absolute returns are incorpor…

Double descent refers to the phase transition that is exhibited by the generalization error of unregularized learning models when varying the ratio between the number of parameters and the number of training samples. The recent success of highly over-parameterized machine learning models such as deep neural networks ha…

This book introduces linear models and their theories rigorously.

Paper proves L2 regression can learn k-juntas without distributional assumptions.

In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatio-temporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. To allow a sm…

Comparing with traditional learning criteria, such as mean square error (MSE), the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyis entropy estimator, called information potential (IP), is a popular MEE cost i…

Paper proposes a new method for covariance estimation using M-estimators with eigenvalue shrinkage.

The paper identifies the minimum mean-variance spanning set and its importance in asset evaluation.

The paper analyzes the risk of CV-tuned regularized estimators and connects it to SURE.

We develop efficient algorithms to estimate the stability of Ordinary Least Squares regression results.

The Rasch model is widely used for item response analysis in applications ranging from recommender systems to psychology, education, and finance. While a number of estimators have been proposed for the Rasch model over the last decades, the available analytical performance guarantees are mostly asymptotic. This paper p…

Deep generative priors offer powerful models for complex-structured data, such as images, audio, and text. Using these priors in inverse problems typically requires estimating the input and/or hidden signals in a multi-layer deep neural network from observation of its output. While these approaches have been successful…

Study uses DNN to accurately estimate daily ET o in various climates.

In this letter, we consider two sets of observations defined as subspace signals embedded in noise and we wish to analyze the distance between these two subspaces. The latter entails evaluating the angles between the subspaces, an issue reminiscent of the well-known Procrustes problem. A Bayesian approach is investigat…

New insights into learning for blind inverse problems with theoretical guarantees.

This paper considers probabilistic estimation of a low-rank matrix from non-linear element-wise measurements of its elements. We derive the corresponding approximate message passing (AMP) algorithm and its state evolution. Relying on non-rigorous but standard assumptions motivated by statistical physics, we characteriz…

This study calculates the maximum error of a famous estimation method.

Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.

Traditional Kalman filter (KF) is derived under the well-known minimum mean square error (MMSE) criterion, which is optimal under Gaussian assumption. However, when the signals are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises, the performance of KF will deteriorate serious…

Develops a new framework for robust regression with EGM.

The paper estimates key metrics for linear models with Markov or hidden Markov sources.

Minimum sum-of-squares clustering (MSSC) is a widely used clustering model, of which the popular K-means algorithm constitutes a local minimizer. It is well known that the solutions of K-means can be arbitrarily distant from the true MSSC global optimum, and dozens of alternative heuristics have been proposed for this …

New findings on computational limits for estimating hidden structures.

Paper solves outlier robust mean estimation near breakdown point.

Paper extends Bayesian Cramér-Rao bound with geometric considerations.

New method optimizes tail dependence coefficient estimation.

In a noiseless linear estimation problem, one aims to reconstruct a vector x* from the knowledge of its linear projections y=Phi x*. There have been many theoretical works concentrating on the case where the matrix Phi is a random i.i.d. one, but a number of heuristic evidence suggests that many of these results are un…

Proposes Deep LTMLE for estimating dynamic treatment effects in longitudinal studies.

In this paper we are dealing with two classes of mean curvature type problems that generalize the translating soliton problem. A first result proves that the solutions to these problems have unique interior critical points. Using this uniqueness result, we next derive a priori $C^0$ and $C^1$ estimates for the solution…

In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares (S-NNLS) problem. We introduce a family of probability densities referred to as the Rectified Gaussian Scale Mixture (R- GSM) to model the sparsity enforcing prior distribution for the solution. The R-GSM…

Efficient adjustment sets found for cost-minimized causal estimations.

In this paper, we continue our previous work on the Dirichlet mixture model (DMM)-based VQ to derive the performance bound of the LSF VQ. The LSF parameters are transformed into the $Δ$LSF domain and the underlying distribution of the $Δ$LSF parameters are modelled by a DMM with finite number of mixture components. The…

In this paper, we present a deep learning (DL) algorithm for channel estimation in communication systems. We consider the time-frequency response of a fast fading communication channel as a two-dimensional image. The aim is to find the unknown values of the channel response using some known values at the pilot location…