This study explains gradient flow dynamics in neural networks for small initialisation.
problem Understanding the training dynamics of neural networks for small initialisation.
method Analysis of gradient flow dynamics for one-hidden layer ReLU networks with orthogonal inputs.
result Gradient flow converges to zero loss and characterizes implicit bias towards minimum variation norm.
Study on LMMSE estimation with model mismatch, quantifying MSE trade-offs.
problem Model mismatch in LMMSE estimation with undermodeling.
method Analyzing the average MSE of LMMSE estimation with random regressors.
result Performance improvement depends on sufficient samples and model complexity.
Robust diffusion adaptive estimation algorithms based on the maximum correntropy criterion (MCC), including adaptation to combination MCC and combination to adaptation MCC, are developed to deal with the distributed estimation over network in impulsive (long-tailed) noise environments. The cost functions used in distri…
Proposes variational autoencoder for efficient MMSE estimation.
problem Efficient parameterized MMSE estimation for noisy observations.
method Variational autoencoder models data distribution, approximates MMSE.
result Proposed estimator performs well compared to state-of-the-art.
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…
Exact expressions for double descent and implicit regularization in over-parameterized models.
problem Understanding the generalization error of over-parameterized models like deep neural networks.
method Surrogate random design to replace standard i.i.d. design, leading to exact expressions for mean squared error and implicit regularization.
result Exact non-asymptotic expressions for double descent and implicit regularization in over-parameterized models.
The paper examines prediction and estimation risks of ridgeless least squares under general error assumptions.
problem Prediction and estimation risks of ridgeless least squares under realistic error structures.
method Analysis of prediction and estimation risks under general regression error assumptions, including clustered or serial dependence.
result The benefits of overparameterization extend to time series, panel, and grouped data.
The paper studies quadratic neural networks, proving existence of spurious minima and saddle points.
problem Understanding the loss landscape of neural networks with quadratic activations.
method Theoretical analysis of mean squared error loss for neural networks with quadratic activations.
result Proves existence of spurious local minima and saddle points in the training landscape of deep overparameterized quadratic neural networks.
Risk-aware MMSE improves stability in volatile scenarios.
problem In MMSE estimators, volatility of error is unconstrained, leading to significant performance differences.
method Introduces risk-aware MMSE by constraining expected predictive variance.
result Risk-aware MMSE provides better performance, especially in skewed, heavy-tailed distributions.
Comparing with traditional learning criteria, such as mean square error (MSE), the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyis entropy estimator, called information potential (IP), is a popular MEE cost i…
We consider the minimum error entropy (MEE) criterion and an empirical risk minimization learning algorithm in a regression setting. A learning theory approach is presented for this MEE algorithm and explicit error bounds are provided in terms of the approximation ability and capacity of the involved hypothesis space w…
This paper introduces minimum-risk recalibration for probabilistic classifiers, improving their reliability and accuracy.
problem Improving the reliability and accuracy of probabilistic classifiers.
method Minimum-risk recalibration within the MSE decomposition framework, analyzing UMB method and label shift adaptation.
result The optimal number of bins for UMB scales with n1/3, resulting in a risk bound of approximately O(n−2/3). This book introduces linear models and their theories rigorously.
problem Understanding linear models and their theories.
method Explains linear models from three perspectives, introduces maximum likelihood estimation, and proves least squares is the best unbiased linear model.
result Least squares is the best unbiased linear model in terms of mean squared error.
Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. Th…
Estimates generalization error for two-layer ReLU NNs through minimum norm solutions.
problem Estimating generalization error for two-layer ReLU NNs trained by mean squared error.
method Uses minimum norm solutions and Neural Tangent Kernel (NTK) regime to derive generalization error bounds.
result Derives an a priori generalization error bound for two-layer ReLU NNs without requiring exponentially large number of neurons.
Double Q-learning has the same mean-squared error as Q-learning under certain conditions.
problem Comparing the mean-squared error of Double Q-learning and Q-learning.
method Theoretical analysis based on Lyapunov equations for both tabular and linear function approximation settings.
result The asymptotic mean-squared error of Double Q-learning is exactly equal to that of Q-learning under specific conditions.
The paper provides mean-square error bounds for stochastic approximation algorithms.
problem Error bounds for recursive equations with Markovian disturbances.
method Analysis of mean-square error for stochastic approximation algorithms.
result Mean-square error achieves the optimal rate of O(1/n) under certain conditions. Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace U and its estimate U^ may not be adequate as the MSE is not the natural metric in the Gra…
Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs bas…
Dynamic linear models improve travel time prediction for congested freeways.
problem Accurate travel time prediction for congested freeways.
method Dynamic linear models (DLMs) with time-varying parameters.
result Significant improvements in travel time prediction accuracy, especially for short-term predictions.
Paper uses DFL to optimize portfolio risk and outperforms conventional methods.
problem Optimizing portfolio risk and return under uncertainty.
method Decision-focused learning (DFL) to derive global minimum variance portfolio (GMVP).
result DFL-based methods consistently deliver superior decision performance in portfolio optimization.
The Rasch model is widely used for item response analysis in applications ranging from recommender systems to psychology, education, and finance. While a number of estimators have been proposed for the Rasch model over the last decades, the available analytical performance guarantees are mostly asymptotic. This paper p…
This study calculates the maximum error of a famous estimation method.
problem Estimating rare items not seen in a sample.
method Characterizes the maximal mean-squared error of the Good-Turing estimator.
result Characterizes the maximal mean-squared error of the Good-Turing estimator.
Bayesian method recovers causal structure in SEMs with equal error variances.
problem Recovering causal structure in SEMs with equal error variances.
method Bayesian DAG selection method using g-priors and the key property of minimum expected squared errors.
result The method consistently recovers the true graph without additional distributional assumptions.
This paper studies MMV problem and its performance, proving a decoupling property for MMV algorithms.
problem Joint estimation of multiple signal realizations with common sparse support.
method Proved a decoupling property for ℓ2,1-LS algorithm, decomposing it into coupled and decoupled phases. result Performance of ℓ2,1-LS and MMV algorithms are affected by signal correlations and dictionary mismatch. We propose a minimum distance estimation method for robust regression in sparse high-dimensional settings. The traditional likelihood-based estimators lack resilience against outliers, a critical issue when dealing with high-dimensional noisy data. Our method, Minimum Distance Lasso (MD-Lasso), combines minimum distanc…
New method finds best arm minimizing mean-squared error in correlated bandits.
problem Finding an arm that best captures information about other arms.
method Formulated correlated bandit problem, derived MSE estimator, proposed algorithm.
result Proposed algorithm identifies best arm with error probability bounds.
Paper solves outlier robust mean estimation near breakdown point.
problem Estimating mean in presence of adversarial outliers.
method Sum-of-Squares approach to optimize error rate efficiently.
result Achieves optimal error rate for all ε ∈ [0, 1/2).
Paper extends Bayesian Cramér-Rao bound with geometric considerations.
problem Estimation of covariance matrices with geometric structures.
method Intrinsic Bayesian Cramér-Rao bound with Riemannian geometry.
result Performance bounds for covariance matrix estimation.
New proof shows how to identify DAGs with weakly increasing errors.
problem Identifying the true DAG in models with weakly increasing error variances.
method Minimum-trace DAG method and hill climbing algorithm with R2R neighborhood.
result Hill climbing algorithm without strict local optima under weakly increasing error variances.
New method optimizes tail dependence coefficient estimation.
problem Estimating tail dependence in nonparametric data.
method Optimal threshold selection combining mean squared error and copula estimation.
result Improved accuracy in tail dependence coefficient estimation.
This paper presents a ML-based receiver for SDR that outperforms conventional methods.
problem Complexity and performance issues in multiuser detection.
method Supervised learning for direct symbol detection without parameter estimation.
result The ML-based receiver achieves similar or better performance than SIC and MMSE receivers.
Paper bounds prediction error for misspecified Gaussian process models.
problem Guaranteeing model confidence for nonparametric Gaussian process regression.
method Derives an upper bound for mean square prediction error using pseudo-concave optimization.
result Upper bound for mean square prediction error of misspecified models.
In this paper, we continue our previous work on the Dirichlet mixture model (DMM)-based VQ to derive the performance bound of the LSF VQ. The LSF parameters are transformed into the ΔLSF domain and the underlying distribution of the ΔLSF parameters are modelled by a DMM with finite number of mixture components. The…
Paper proves L2 regression can learn k-juntas without distributional assumptions.
problem Learning k-juntas using L2 regression without distributional restrictions.
method L2 polynomial regression and minimum mean square estimation (MMSE).
result Agnostic PAC learning of k-juntas using L2 polynomial regression.
New findings on computational limits for estimating hidden structures.
problem Estimating hidden structures in noisy data.
method Use of low-degree polynomials as a restricted model of computation.
result Established low-degree hardness of recovery problems for easy detection problems.
Deep networks don't improve on shallow ones for finding minima.
problem Improving representation of multidimensional mappings with deep neural networks.
method Numerical training methods to find minima in deep and shallow networks.
result Minima found with deep networks are worse than those found with shallow networks.
New privacy mechanism reduces error in query results.
problem Achieving privacy while minimizing noise in query results.
method Extended sufficient and necessary condition for (ε,δ)-differential privacy for symmetric and log-concave noise densities. result Significantly lower mean squared errors than Laplace and Gaussian mechanisms.
Cryptocurrency prices predicted using LSTM, SVM, and polynomial regression.
problem Uncertainty in crypto coin values.
method Long Short Term Memory, Support Vector Machine, Polynomial Regression models.
result Support Vector Machine with linear kernel had the smallest mean square error.
Over the last decade, both the neural network and kernel adaptive filter have successfully been used for nonlinear signal processing. However, they suffer from high computational cost caused by their complex/growing network structures. In this paper, we propose two random Euler filters for complex-valued nonlinear filt…
New insights into learning for blind inverse problems with theoretical guarantees.
problem Learning in blind inverse problems where both signal and operator are unknown.
method Data-driven approaches using Linear Minimum Mean Square Estimators (LMMSEs) with theoretical analysis.
result Established equivalences with Tikhonov-regularized formulations and derived finite-sample error bounds.
Improved multi-task averaging reduces mean squared error in high-dimensional data.
problem Joint estimation of multiple distributions using independent data sets.
method Exploits similarities between tasks by shrinking naive estimators towards local averages.
result The method provides a significant reduction in mean squared error, especially in high-dimensional spaces.
New method addresses error bounds for PnP-ULA under mismatched models.
problem Error bounds for PnP-ULA under mismatched measurement and prior models.
method Posterior-L2 pseudometric to quantify error bounds.
result Explicit error bound for PnP-ULA under mismatched posterior distribution.
Study uses DNN to accurately estimate daily ET o in various climates.
problem Precise estimation of reference evapotranspiration (ET o ) for irrigation and water management.
method Investigated artificial neural network (ANN) and deep neural network (DNN) models using six meteorological inputs.
result DNN models, especially P-DNN-SeLU, achieve high accuracy in daily ET o estimation.
The paper introduces V(I) to guide algorithm choice and parameter tuning in financial forecasting.
problem Selecting optimal algorithms and tuning parameters for financial time-series forecasting.
method Estimating Shannon's mutual information and using it to define performance bounds.
result Illustrates the value of information for mean-square error minimization in cryptocurrency forecasts.
This letter presents an improved version of diffusion least mean ppower (LMP) algorithm for distributed estimation. Instead of sum of mean square errors, a weighted sum of mean square error is defined as the cost function for global and local cost functions of a network of sensors. The weight coefficients are updated b…
Signal processing tasks as fundamental as sampling, reconstruction, minimum mean-square error interpolation and prediction can be viewed under the prism of reproducing kernel Hilbert spaces. Endowing this vantage point with contemporary advances in sparsity-aware modeling and processing, promotes the nonparametric basi…