Extends covariance estimation with multiple targets for better performance.
problem Improving covariance estimation for multiple targets.
method Combines multiple constant matrices with sample covariance matrix, derives estimators and proves convergence.
result The multi-target linear shrinkage estimator outperforms other estimators in various situations.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
Paper proposes a bias-constrained deep learning approach to non-linear estimation.
problem Designing unbiased estimators for non-linear models.
method Bias Constrained Estimator (BCE) using deep learning with bias constraints.
result Asymptotic MVUEs with Cramer Rao bound performance.
The paper examines conditions for linearity in a conditional mean estimator under vector Poisson noise.
problem Conditions for linearity of the conditional mean estimator in vector Poisson noise.
method Analyzes prior distributions and their impact on the conditional mean estimator's linearity.
result The only prior distribution that induces linearity is a product gamma distribution, and non-zero dark current parameter prevents linearity.
Linear Transformer Block combines MLP and linear attention for near-optimal ICL in linear regression.
problem Achieving near-optimal in-context learning (ICL) risk for linear regression with a Gaussian prior.
method Combines linear attention and MLP components in a Linear Transformer Block (LTB). Establishes correspondence with one-step gradient descent estimators ( G D e x t − β \mathsf{GD} ext{-}\mathbfβ GD e x t − β ). result LTB achieves nearly Bayes optimal ICL risk for linear regression with a Gaussian prior.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
New RL algorithm achieves nearly optimal performance for linear MDPs.
problem Optimal reinforcement learning for episodic linear MDPs.
method Weighted linear regression with variance estimator and rare-switching policy.
result Achieves nearly minimax optimal regret i l d e O ( d H 3 K ) ilde O(d\sqrt{H^3K}) i l d e O ( d H 3 K ) . Unified approach to linear regression using covariance fitting for optimal weights.
problem Finding optimal weights for linear regression models when weights are unknown.
method Covariance fitting SPICE-methodology to obtain data-adaptive weights.
result Tuned versions of known regularized estimators are unified under a common approach.
Paper introduces structured sparsity estimators for Generalized Linear Models.
problem Estimating structured sparsity in GLMs with debiased estimators.
method Extends Stucky and van de Geer's results to GLMs with structured sparsity.
result Proves oracle inequalities for structured sparsity estimators in GLMs.
Efficiently estimates sparse linear regression with heavy-tailed data and outliers.
problem Sparse estimation of linear regression coefficients with heavy-tailed covariates and noises, including outliers.
method Efficient computation of robust estimator with nearly optimal error bound.
result Nearly optimal error bound for robust sparse estimation.
Estimates time-varying parameters from two OLS estimates.
problem Time-varying linear regression with hidden dynamics.
method Combines two OLS estimates for stable linear dynamics.
result Finite sample guarantee on estimation error.
Estimates GLMs robustly against label corruptions.
problem Learning GLMs under adversarial label corruptions.
method Iterative trimmed maximum likelihood estimator.
result Achieves minimax near-optimal risk.
New method for robust linear regression in nearly linear time.
problem High-dimensional robust linear regression with adversarial corruption.
method Proposes estimators for two settings with near linear time complexity.
result Achieves optimal sample complexities and recovery guarantees.
This work shows that Gaussian is the only prior for optimal linear estimation in L 1 L^1 L 1 loss.
problem Optimal linear estimation of a random variable from noisy observations under L 1 L^1 L 1 fidelity criterion. method Analyzes the conditions under which the conditional median is a linear estimator and identifies the Gaussian distribution as the only prior that induces linearity.
result Gaussian is the only prior distribution that induces linearity in the conditional median for L 1 L^1 L 1 loss. Paper presents a machine learning method to improve significance tests for misspecified linear models.
problem Misspecification of linear assumptions in social science models leads to inaccurate significance levels.
method Apply machine learning to fit ground truth function, calculate linear approximation, and adjust the estimator.
result The method significantly outperforms linear regression for non-linear ground truth functions.
Robust method estimates state, input, and parameters of linear systems online.
problem Joint estimation of state, input, and parameters in noisy or outlier-prone measurements.
method Combines recursive, alternating, and iteratively-reweighted least squares into a single algorithm.
result Good performance in presence of outliers and compared to state-of-the-art methods.
Develops fast approximations for conditional Shapley values in linear and polynomial models.
problem Estimating conditional Shapley values using regression models is computationally expensive.
method A new approximative estimation method for conditional Shapley values using linear and polynomial regression models.
result Our method significantly reduces computation time compared to existing methods.
Paper proposes a debiased estimator for adaptive linear regression.
problem Non-normal asymptotic behavior of OLS estimator in adaptive linear regression.
method Adaptive linear estimating equations to construct debiased estimator.
result Established asymptotic normality of the debiased estimator.
Develops inequalities for high-dimensional linear processes with dependent innovations.
problem Estimating high-dimensional VAR(p) systems and HAC covariance estimation.
method Concentration inequalities for l ∞ l_\infty l ∞ norm of vector linear processes with sub-Weibull, mixingale innovations. result Obtained concentration bounds for the maximum entrywise norm of lag- h h h autocovariance matrices. Estimates causal effects in Gaussian Linear SCMs with finite data.
problem Estimating causal effects from observational data with latent confounders.
method Centralized Gaussian Linear SCMs (CGL-SCMs) and EM-based estimation algorithm.
result Learned CGL-SCM parameters accurately recover causal distributions from finite observational samples.
Estimates MLDS using tensor decomposition, improving upon existing methods.
problem Learning mixtures of linear dynamical systems from input-output data.
method Proposes a moment-based estimator using tensor decomposition.
result Improves sample complexity bounds for estimating MLDS.
The study estimates the expressiveness of GCNs with bounds on the number of linear regions.
problem Characterizing the expressiveness of graph convolutional networks (GCNs).
method Estimates the number of linear regions for one-layer and multi-layer GCNs.
result GCNs with multiple layers have exponentially more expressivity per parameter than one-layer GCNs.
A new method for estimating large-scale linear models with improved precision.
problem Estimating large-scale linear statistical models efficiently.
method Sequential Least-Squares Estimators with Fast Randomized Sketching (SLSE-FRS), integrating Sketch-and-Solve and Iterative-Sketching methods.
result SLSE-FRS produces high-precision estimators, outperforming state-of-the-art methods.
New Riemannian optimization improves variance estimation in mixed models.
problem Challenges in estimating variance parameters in linear mixed models due to constraints.
method Formulated as an optimization problem on a Riemannian manifold, using Riemannian gradient and Hessian.
result Yields higher quality variance parameter estimates compared to existing methods.
A new algorithm improves stochastic linear bandit performance using residual bootstrap.
problem Improving performance in stochastic linear bandit problems.
method Residual bootstrap exploration to estimate mean reward and pull the arm with the highest estimate.
result Proposed algorithm exttt{LinReBoot} achieves high-probability sub-linear regret under mild conditions.
Gradient estimate for linearized translator equation in R^4.
problem Analyzing singularity models of mean curvature flow in R^4.
method Proving a gradient estimate for the variation field W in the tip region.
result Sharp bound for the derivative of the variation field W in the tip region.
Estimating a constrained relation is a fundamental problem in machine learning. Special cases are classification (the problem of estimating a map from a set of to-be-classified elements to a set of labels), clustering (the problem of estimating an equivalence relation on a set) and ranking (the problem of estimating a …
Near-optimal algorithms for mean estimation and linear regression with Gaussian covariates and Huber contamination.
problem Gaussian mean estimation and linear regression with Gaussian covariates in the presence of Huber contamination.
method Near-optimal algorithms with optimal error guarantees, achieving sample complexity n = i l d e O ( d / ε 2 ) n = ilde{O}(d/ε^2) n = i l d e O ( d / ε 2 ) and almost linear runtime. result First sample near-optimal and almost linear-time algorithms with optimal error guarantees for both problems.
Proves energy estimates for tensorial wave equations, decoupling components for stability proof.
problem Proving stability of ( 1 + 3 ) (1+3) ( 1 + 3 ) -Minkowski space-time with various non-linearities. method Decouples energy estimates for tensorial wave equations, exploiting tensorial structure and Lie derivatives.
result Decoupled energy estimates for tensorial solutions, allowing new stability proofs.
Improved KernelSHAP via linear regression for ML model interpretation.
problem Efficiently estimating Shapley values in model-agnostic settings.
method Revisiting KernelSHAP via linear regression, developing techniques for convergence and uncertainty.
result Original KernelSHAP incurs negligible bias for significant variance reduction.
Efficiently estimates sparse linear regression with heavy-tailed and outlier-contaminated data.
problem Estimating sparse linear regression coefficients with heavy-tailed and outlier-contaminated data.
method Efficient computation of estimators with sharp error bounds.
result Sharp error bounds for efficient estimators.
The paper establishes boundary estimates for solutions to elliptic equations on Hermitian manifolds.
problem Boundary estimates for solutions to fully non-linear elliptic equations on Hermitian manifolds.
method Unified approach using quantitative boundary estimates, gradient estimates, and existence results.
result Established gradient estimates and unified approach to Dirichlet problem solutions.
Paper models non-linear dynamics from time series data.
problem Modeling non-linear dynamical systems from time series data.
method Introduces latent state modeling and a novel alternating minimization algorithm.
result LaNoLem achieves competitive performance in dynamics estimation and prediction.
Study improves robustness and sparsity in linear regression with adversarial outliers and heavy-tailed noise.
problem Outliers and heavy-tailed noise in linear regression coefficients.
method Sharp concentration inequalities and generic chaining.
result Sharper error bounds under weaker assumptions.
Unified strategy for efficient data compression and model estimation.
problem Limited interactive exploration and data interaction in linear model development and deployment.
method Conditionally sufficient statistics for optimal data compression and estimation of linear models.
result Linear models can be estimated from compressed data without loss of parameters or covariances.
Estimates causal effects using neural autoregressive density estimators.
problem Estimating causal effects in non-linear systems.
method Neural autoregressive density estimators within Pearl's do-calculus framework.
result Retrieves causal effects from non-linear systems without explicit modeling.
Two EM algorithms estimate prior distributions in mixture of linear regressions.
problem Estimating prior distributions in mixture of linear regressions.
method Two EM algorithms: one for continuous priors, one for discrete priors.
result Both algorithms accurately estimate prior distributions and the number of clusters.
This article investigates the quality of the estimator of the linear Monge mapping between distributions. We provide the first concentration result on the linear mapping operator and prove a sample complexity of n − 1 / 2 n^{-1/2} n − 1/2 when using empirical estimates of first and second order moments. This result is then used to der…
The growing size of modern data brings many new challenges to existing statistical inference methodologies and theories, and calls for the development of distributed inferential approaches. This paper studies distributed inference for linear support vector machine (SVM) for the binary classification task. Despite a vas…
The OLS estimator optimally identifies stable linear systems with a finite number of samples.
problem Identifying stable linear systems with a finite number of samples.
method Finite-time analysis of the Ordinary Least Squares (OLS) estimator for stable linear systems.
result The OLS estimator achieves optimal sample complexity for stable systems, matching existing lower bounds up to universal factors.
Extended Gauss-Markov theorem for linear estimation with bounded bias.
problem Linear estimation with bounded bias operator.
method Derive optimal estimator formulas for Nuclear and Spectral norms, analyze generalization error.
result Cross-validated Nuclear and Spectral regressors outperform Ridge regression in simulations.
This work extends Ledoit-Wolf shrinkage to unknown mean covariance estimation.
problem Large dimensional covariance matrix estimation with unknown mean under Kolmogorov asymptotics.
method Extending Ledoit-Wolf linear shrinkage to translation-invariant estimators, proving their convergence properties.
result A new estimator outperforms other standard estimators empirically.
Max-linear regression problem solved with convex programming.
problem Estimating parameters in max-linear regression models.
method Formulated and analyzed a scalable convex program called anchored regression (AR).
result AR provides high probability recovery of parameters with a sample complexity of k 4 p k^4p k 4 p . LSBI approximates likelihood with linear functions for cosmological parameter estimation.
problem Estimating cosmological parameters from complex data.
method Sequential Linear Simulation-based Inference (LSBI) using Gaussian approximations.
result LSBI achieves convergence after 4-5 rounds of simulations, comparable to neural methods.
Proposes a partially linear structure to capture nonlinear relationships in mixture of experts models.
problem Suboptimal estimates due to linearity assumption in mixture of experts models.
method Introduces a partially linear structure that incorporates unspecified functions to capture nonlinear relationships.
result Establishes the identifiability of the proposed model under mild conditions and introduces a practical estimation algorithm.
Paper establishes estimates for solutions on compact manifolds.
problem Solving fully non-linear equations on compact almost Hermitian manifolds.
method Establishes a priori estimates for solutions.
result Solves complex Hessian and Monge-Ampère equations.
This paper tackles efficient and scalable estimation of a complex model involving stochastic linear combinations of non-linear regressions.
problem Estimating a model involving stochastic linear combinations of non-linear regressions efficiently and scalably.
method The paper provides algorithms for estimating the model under specific assumptions about the variate vector and sample size, using techniques like zero-bias transformation and sub-sampling.
result The paper provides theoretical guarantees for the estimation of the model, showing that the estimation errors are of the order O ( p n ) O(\sqrt{\frac{p}{n}}) O ( n p ) and O ( 1 p + p n ) O(\frac{1}{\sqrt{p}}+\sqrt{\frac{p}{n}}) O ( p 1 + n p ) with high probability. Paper improves confidence intervals for LSA with multiplier bootstrap.
problem Improving confidence intervals for parameter estimation in LSA.
method Berry-Esseen bound for multivariate normal approximation and multiplier bootstrap.
result Valid confidence intervals for parameter estimation in LSA.