Paper introduces a new estimator for Rasch model with exact error analysis.
problem Estimating parameters of the Rasch model with performance guarantees.
method Develops a novel L-MMSE estimator for the Rasch model with nonasymptotic analysis.
result The L-MMSE estimator provides exact error analysis and performs similarly to state-of-the-art estimators.
Study on LMMSE estimation with model mismatch, quantifying MSE trade-offs.
problem Model mismatch in LMMSE estimation with undermodeling.
method Analyzing the average MSE of LMMSE estimation with random regressors.
result Performance improvement depends on sufficient samples and model complexity.
This study explains gradient flow dynamics in neural networks for small initialisation.
problem Understanding the training dynamics of neural networks for small initialisation.
method Analysis of gradient flow dynamics for one-hidden layer ReLU networks with orthogonal inputs.
result Gradient flow converges to zero loss and characterizes implicit bias towards minimum variation norm.
Proposes variational autoencoder for efficient MMSE estimation.
problem Efficient parameterized MMSE estimation for noisy observations.
method Variational autoencoder models data distribution, approximates MMSE.
result Proposed estimator performs well compared to state-of-the-art.
The most important aspect of any classifier is its error rate, because this quantifies its predictive capacity. Thus, the accuracy of error estimation is critical. Error estimation is problematic in small-sample classifier design because the error must be estimated using the same data from which the classifier has been…
Robust diffusion adaptive estimation algorithms based on the maximum correntropy criterion (MCC), including adaptation to combination MCC and combination to adaptation MCC, are developed to deal with the distributed estimation over network in impulsive (long-tailed) noise environments. The cost functions used in distri…
The paper studies quadratic neural networks, proving existence of spurious minima and saddle points.
problem Understanding the loss landscape of neural networks with quadratic activations.
method Theoretical analysis of mean squared error loss for neural networks with quadratic activations.
result Proves existence of spurious local minima and saddle points in the training landscape of deep overparameterized quadratic neural networks.
Exact expressions for double descent and implicit regularization in over-parameterized models.
problem Understanding the generalization error of over-parameterized models like deep neural networks.
method Surrogate random design to replace standard i.i.d. design, leading to exact expressions for mean squared error and implicit regularization.
result Exact non-asymptotic expressions for double descent and implicit regularization in over-parameterized models.
Comparing with traditional learning criteria, such as mean square error (MSE), the minimum error entropy (MEE) criterion is superior in nonlinear and non-Gaussian signal processing and machine learning. The argument of the logarithm in Renyis entropy estimator, called information potential (IP), is a popular MEE cost i…
Paper predicts VQ performance for LSF using DMM in the ΔLSF domain.
problem Predicting VQ performance for LSF parameters.
method Transform LSF parameters to ΔLSF domain, model with DMM, calculate MSE, estimate bit rate.
result Estimates minimum bit rate for transparent coding of LSF.
This paper introduces minimum-risk recalibration for probabilistic classifiers, improving their reliability and accuracy.
problem Improving the reliability and accuracy of probabilistic classifiers.
method Minimum-risk recalibration within the MSE decomposition framework, analyzing UMB method and label shift adaptation.
result The optimal number of bins for UMB scales with n1/3, resulting in a risk bound of approximately O(n−2/3). Estimates generalization error for two-layer ReLU NNs through minimum norm solutions.
problem Estimating generalization error for two-layer ReLU NNs trained by mean squared error.
method Uses minimum norm solutions and Neural Tangent Kernel (NTK) regime to derive generalization error bounds.
result Derives an a priori generalization error bound for two-layer ReLU NNs without requiring exponentially large number of neurons.
Approximations of loopy belief propagation, including expectation propagation and approximate message passing, have attracted considerable attention for probabilistic inference problems. This paper proposes and analyzes a generalization of Opper and Winther's expectation consistent (EC) approximate inference method. Th…
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
Paper uses DFL to optimize portfolio risk and outperforms conventional methods.
problem Optimizing portfolio risk and return under uncertainty.
method Decision-focused learning (DFL) to derive global minimum variance portfolio (GMVP).
result DFL-based methods consistently deliver superior decision performance in portfolio optimization.
Recently, a framework for application-oriented optimal experiment design has been introduced. In this context, the distance of the estimated system from the true one is measured in terms of a particular end-performance metric. This treatment leads to superior unknown system estimates to classical experiment designs bas…
Double Q-learning has the same mean-squared error as Q-learning under certain conditions.
problem Comparing the mean-squared error of Double Q-learning and Q-learning.
method Theoretical analysis based on Lyapunov equations for both tabular and linear function approximation settings.
result The asymptotic mean-squared error of Double Q-learning is exactly equal to that of Q-learning under specific conditions.
The paper provides mean-square error bounds for stochastic approximation algorithms.
problem Error bounds for recursive equations with Markovian disturbances.
method Analysis of mean-square error for stochastic approximation algorithms.
result Mean-square error achieves the optimal rate of O(1/n) under certain conditions. Dynamic linear models improve travel time prediction for congested freeways.
problem Accurate travel time prediction for congested freeways.
method Dynamic linear models (DLMs) with time-varying parameters.
result Significant improvements in travel time prediction accuracy, especially for short-term predictions.
This paper presents a ML-based receiver for SDR that outperforms conventional methods.
problem Complexity and performance issues in multiuser detection.
method Supervised learning for direct symbol detection without parameter estimation.
result The ML-based receiver achieves similar or better performance than SIC and MMSE receivers.
Paper extends Bayesian Cramér-Rao bound with geometric considerations.
problem Estimation of covariance matrices with geometric structures.
method Intrinsic Bayesian Cramér-Rao bound with Riemannian geometry.
result Performance bounds for covariance matrix estimation.
We consider the problem of subspace estimation in a Bayesian setting. Since we are operating in the Grassmann manifold, the usual approach which consists of minimizing the mean square error (MSE) between the true subspace U and its estimate U^ may not be adequate as the MSE is not the natural metric in the Gra…
Risk-aware MMSE improves stability in volatile scenarios.
problem In MMSE estimators, volatility of error is unconstrained, leading to significant performance differences.
method Introduces risk-aware MMSE by constraining expected predictive variance.
result Risk-aware MMSE provides better performance, especially in skewed, heavy-tailed distributions.
This study calculates the maximum error of a famous estimation method.
problem Estimating rare items not seen in a sample.
method Characterizes the maximal mean-squared error of the Good-Turing estimator.
result Characterizes the maximal mean-squared error of the Good-Turing estimator.
New privacy mechanism reduces error in query results.
problem Achieving privacy while minimizing noise in query results.
method Extended sufficient and necessary condition for (ε,δ)-differential privacy for symmetric and log-concave noise densities. result Significantly lower mean squared errors than Laplace and Gaussian mechanisms.
New method addresses error bounds for PnP-ULA under mismatched models.
problem Error bounds for PnP-ULA under mismatched measurement and prior models.
method Posterior-L2 pseudometric to quantify error bounds.
result Explicit error bound for PnP-ULA under mismatched posterior distribution.
New findings on computational limits for estimating hidden structures.
problem Estimating hidden structures in noisy data.
method Use of low-degree polynomials as a restricted model of computation.
result Established low-degree hardness of recovery problems for easy detection problems.
Over the last decade, both the neural network and kernel adaptive filter have successfully been used for nonlinear signal processing. However, they suffer from high computational cost caused by their complex/growing network structures. In this paper, we propose two random Euler filters for complex-valued nonlinear filt…
New method optimizes tail dependence coefficient estimation.
problem Estimating tail dependence in nonparametric data.
method Optimal threshold selection combining mean squared error and copula estimation.
result Improved accuracy in tail dependence coefficient estimation.
New insights into learning for blind inverse problems with theoretical guarantees.
problem Learning in blind inverse problems where both signal and operator are unknown.
method Data-driven approaches using Linear Minimum Mean Square Estimators (LMMSEs) with theoretical analysis.
result Established equivalences with Tikhonov-regularized formulations and derived finite-sample error bounds.
Paper bounds prediction error for misspecified Gaussian process models.
problem Guaranteeing model confidence for nonparametric Gaussian process regression.
method Derives an upper bound for mean square prediction error using pseudo-concave optimization.
result Upper bound for mean square prediction error of misspecified models.
Deep networks don't improve on shallow ones for finding minima.
problem Improving representation of multidimensional mappings with deep neural networks.
method Numerical training methods to find minima in deep and shallow networks.
result Minima found with deep networks are worse than those found with shallow networks.
This paper studies MMV problem and its performance, proving a decoupling property for MMV algorithms.
problem Joint estimation of multiple signal realizations with common sparse support.
method Proved a decoupling property for ℓ2,1-LS algorithm, decomposing it into coupled and decoupled phases. result Performance of ℓ2,1-LS and MMV algorithms are affected by signal correlations and dictionary mismatch. Signal processing tasks as fundamental as sampling, reconstruction, minimum mean-square error interpolation and prediction can be viewed under the prism of reproducing kernel Hilbert spaces. Endowing this vantage point with contemporary advances in sparsity-aware modeling and processing, promotes the nonparametric basi…
New method finds best arm minimizing mean-squared error in correlated bandits.
problem Finding an arm that best captures information about other arms.
method Formulated correlated bandit problem, derived MSE estimator, proposed algorithm.
result Proposed algorithm identifies best arm with error probability bounds.
The paper introduces V(I) to guide algorithm choice and parameter tuning in financial forecasting.
problem Selecting optimal algorithms and tuning parameters for financial time-series forecasting.
method Estimating Shannon's mutual information and using it to define performance bounds.
result Illustrates the value of information for mean-square error minimization in cryptocurrency forecasts.
Cryptocurrency prices predicted using LSTM, SVM, and polynomial regression.
problem Uncertainty in crypto coin values.
method Long Short Term Memory, Support Vector Machine, Polynomial Regression models.
result Support Vector Machine with linear kernel had the smallest mean square error.
In this paper we consider the use of the space vs. time Kronecker product decomposition in the estimation of covariance matrices for spatio-temporal data. This decomposition imposes lower dimensional structure on the estimated covariance matrix, thus reducing the number of samples required for estimation. To allow a sm…
Study minimax off-policy evaluation in multi-armed bandits with known and unknown behavior policies.
problem Evaluate policies in multi-armed bandits with unknown behavior policies.
method Develop minimax rate-optimal procedures for known and unknown behavior policies, including the Switch estimator and Chebyshev polynomial-based estimator.
result Plug-in estimator achieves optimal competitive ratio up to a logarithmic factor when behavior policy is unknown.
This paper considers probabilistic estimation of a low-rank matrix from non-linear element-wise measurements of its elements. We derive the corresponding approximate message passing (AMP) algorithm and its state evolution. Relying on non-rigorous but standard assumptions motivated by statistical physics, we characteriz…
This book introduces linear models and their theories rigorously.
problem Understanding linear models and their theories.
method Explains linear models from three perspectives, introduces maximum likelihood estimation, and proves least squares is the best unbiased linear model.
result Least squares is the best unbiased linear model in terms of mean squared error.
Optimization algorithms help overparameterized neural networks achieve high performance.
problem Understanding the convergence of optimization algorithms on overparameterized neural networks.
method Analyzing a broad class of optimization algorithms using dynamical systems and finite over-parameterized neural networks with ReLU activation.
result The Heavy Ball method converges to global minimum at a linear rate, while NAG converges sublinearly.
Most of the existing studies on voice conversion (VC) are conducted in acoustically matched conditions between source and target signal. However, the robustness of VC methods in presence of mismatch remains unknown. In this paper, we report a comparative analysis of different VC techniques under mismatched conditions. …
Enhances spoken speech quality using EEG signals.
problem Improves speech clarity in noisy environments.
method Generative adversarial network (GAN), gated recurrent unit (GRU), temporal convolutional network (TCN) regression models.
result Significant improvement in speech enhancement quality compared to traditional methods.
Paper proves L2 regression can learn k-juntas without distributional assumptions.
problem Learning k-juntas using L2 regression without distributional restrictions.
method L2 polynomial regression and minimum mean square estimation (MMSE).
result Agnostic PAC learning of k-juntas using L2 polynomial regression.
The study sets lower bounds on MMSE for inferring sensitive features from noisy data.
problem Estimating sensitive features from noisy observations of correlated features.
method Adversarial evaluation framework based on MMSE estimation with theoretical lower bounds.
result Derives closed-form bounds for linear models, showing optimality in noise variance.
Paper introduces a new method for error estimation in classification tasks with limited data.
problem Challenges in designing accurate classifiers and evaluating their performance with limited training data.
method Introduces a novel Bayesian MMSE estimator for optimal Bayesian transfer learning (OBTL) using Monte Carlo importance sampling.
result Proposed OBTL error estimation scheme outperforms standard methods, especially in small-sample settings.
Deep learning improves channel estimation in communication systems.
problem Improving channel estimation accuracy in communication systems.
method Proposed a deep learning pipeline using image processing techniques to estimate channel response.
result The proposed algorithm shows comparable performance to MMSE with full channel knowledge and better than ALMMSE.