Paper introduces semi-supervised linear extremile regression for high-dimensional data.
problem Challenges in high-dimensional extremile regression due to data sparsity and overfitting.
method Proposes semi-supervised learning for linear extremile regression, achieving n \sqrt{n} n -consistency. result Demonstrates improved estimation efficiency and performance in high-dimensional settings.
We simplify complex regression coefficients using linearization and feature comparison.
problem Interpreting high-dimensional regression coefficients from nonlinear responses.
method Developed a linearization method to derive feature coefficients and compare them with regression coefficients.
result Shows how regression coefficients relate to linearized feature coefficients and how they change under regularization.
Robust learning mixtures of linear regressions improve robustness.
problem Improving robustness in learning mixtures of linear regressions.
method Connecting mixtures of linear regressions and mixtures of Gaussians with thresholding for a quasi-polynomial time algorithm.
result The algorithm has significantly better robustness than previous results.
This paper studies robust regression in the settings of Huber's ε ε ε -contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of ε ε ε -contamination models for various regression problems including nonpa…
Locally adaptive interpretable regression improves linear regression's predictability.
problem Linear regression's predictability is limited; it lacks adaptability.
method Locally adaptive interpretable regression (LoAIR) uses neural networks to predict percentile of a Gaussian distribution for regression coefficients.
result LoAIR achieves comparable or better predictive performance than state-of-the-art baselines.
The paper analyzes the generalizability of linear autoencoders and multivariate linear regression.
problem Limited theoretical understanding of linear autoencoders' performance.
method Proposes a PAC-Bayes bound for multivariate linear regression and shows LAEs as constrained models.
result The proposed PAC-Bayes bound is tight and correlates with practical metrics.
Study on thermodynamic costs of simple linear regression.
problem Understanding thermodynamic costs in machine learning models.
method Approximated thermodynamic lower bounds for exact and stochastic linear regression.
result Derived scaling laws for optimal dataset size based on generalization error.
PARC uses piecewise linear predictors for regression and classification.
problem Multivariate regression and classification problems.
method Alternates between ridge and softmax regression, and cluster assignment based on accuracy and separability.
result Converges to a local minimum in a finite number of steps.
Proposes a method for differentially private linear regression and synthetic data generation.
problem Lack of valid inference and synthetic data generation methods for small-scale datasets in privacy-aware settings.
method Gaussian differentially private linear regression with bias-corrected estimator and SDG procedure.
result Improves accuracy and provides valid confidence intervals for downstream tasks.
Develops fast approximations for conditional Shapley values in linear and polynomial models.
problem Estimating conditional Shapley values using regression models is computationally expensive.
method A new approximative estimation method for conditional Shapley values using linear and polynomial regression models.
result Our method significantly reduces computation time compared to existing methods.
Analyzes neural networks using linear models to understand their behavior.
problem Understanding multi-layer neural networks through linear models.
method Recalls and reviews four models: linear regression with concentrated features, kernel ridge regression, random feature model, and neural tangent model.
result Highlights limitations of linear theory and discusses approaches to overcome them.
Unified derivation of high-dimensional linear models using stochastic gradient descent.
problem Performance analysis of high-dimensional linear models trained with stochastic gradient descent.
method Derivation of a deterministic equivalence for the two-point function of a random matrix resolvent.
result Unified understanding of model performance including previously known and novel results.
We analyzed optimism in linear and kernel regression models.
problem Understanding predictive complexity in regression models.
method Derived closed-form asymptotic optimism for linear and kernel regression models.
result Scaled optimism is a useful measure for model complexity.
This paper studies the addition of linear constraints to the Support Vector Regression (SVR) when the kernel is linear. Adding those constraints into the problem allows to add prior knowledge on the estimator obtained, such as finding probability vector or monotone data. We propose a generalization of the Sequential Mi…
Proposes FARM model combining latent factor and sparse regression.
problem Testing adequacy of latent factor and sparse regression models.
method Factor Augmented sparse linear Regression Model (FARM) with FabTest and ANOVA type tests.
result Model robustness and effectiveness validated through experiments.
Lecture notes on advanced linear regression methods.
problem Understanding the properties of linear regression estimators in high dimensions.
method Proposition-proof exploration of least squares, ridgeless, ridge, and lasso estimators.
result Detailed analysis of the existence, uniqueness, relations, computation, and non-asymptotic properties of these estimators.
Algorithm optimizes quantized isotonic regression with log-linear time updates.
problem Optimizing quantized isotonic regression estimations.
method Modified PAVA algorithm for sequential optimization.
result Log-linear time updates for optimal quantized mapping.
Develops estimators for near-optimal linear regression under distribution shift.
problem Linear regression under distribution shift with scarce target domain data.
method Minimax linear risk estimators covering various transfer learning settings.
result Achieves near-optimal risk for linear regression problems under distribution shift.
Probit regression was first proposed by Bliss in 1934 to study mortality rates of insects. Since then, an extensive body of work has analyzed and used probit or related binary regression methods (such as logistic regression) in numerous applications and fields. This paper provides a fresh angle to such well-established…
Proof of Gaussian ML estimator consistency in linear auto-regressive models.
problem Consistency of Gaussian maximum likelihood estimator in linear auto-regressive models.
method Information-theoretic proof without stability assumptions.
result Nearly optimal non-asymptotic rates for parameter recovery.
Improved SGD for robust linear and ReLU regression with adversarial corruptions.
problem Robust regression with adversarial corruptions in streaming data.
method Stochastic gradient descent (SGD-exp) with exponentially decaying step size.
result Nearly linear convergence to true parameter with up to 50% Massart corruption rate.
AM converges super-linearly for solving mixed linear regression problems.
problem Learning linear regressors from unlabeled observations in multiple linear regression models.
method Alternating Minimization (AM) algorithm, which alternates between label estimation and regression solving.
result AM converges super-linearly in certain parameter regimes, requiring only O(log log(1/ε)) iterations to achieve an error of ε.
This paper compares linear regression and neural networks for pricing swing options.
problem Pricing swing options using approximation methods.
method Linear regression and neural networks for approximating the continuation value and swing price.
result The approximation methods converge to the actual swing price as the number of functions or Monte Carlo samples increases.
Introduces a new model for mapping matrices to matrices, subsuming linear regression.
problem Learning matrix-to-matrix mappings from data.
method Partial trace regression model, leveraging quantum information theory.
result Relevance demonstrated in matrix-to-matrix regression and positive semidefinite matrix completion.
Improved linear regression with privacy and robustness guarantees.
problem Private and robust linear regression with adversarial corruption.
method Differentially private stochastic gradient descent with full-batch gradient descent and adaptive clipping.
result Near optimal sample complexity for both private and robust linear regression.
Neural networks can be simplified to linear regression for easier understanding by statisticians.
problem Introducing neural networks to statisticians who are not familiar with them.
method Describing neural networks that approximate linear regression and discussing customizations.
result Statisticians can now understand neural networks by focusing on linear regression.
Private sketches protect linear regression data privacy.
problem Protecting sensitive information in linear regression.
method Release private sketches of datasets, compute approximate solutions.
result Private sketches maintain good approximation guarantees to the original problem.
Paper analyzes agnostic learning of mixed linear regression without generative models.
problem Learning mixed linear regression without assuming stochastic generation.
method Expectation Maximization (EM) and Alternating Minimization (AM) algorithms.
result AM and EM algorithms converge to population loss minimizers under standard conditions.
New methods combine low and high-fidelity data for accurate surrogate modeling.
problem Challenges in surrogate modeling for high-dimensional outputs with limited training data.
method Projection-based multifidelity linear regression methods integrating low-fidelity and high-fidelity data.
result Multifidelity methods achieve up to 12% improvement in median accuracy compared to single-fidelity methods.
Paper introduces data-dependent SSP for private linear and logistic regression.
problem Private linear and logistic regression with better performance.
method Data-dependent sufficient statistic perturbation (SSP) for linear and logistic regression.
result Data-dependent SSP outperforms state-of-the-art methods for linear and logistic regression.
Efficiently estimates sparse linear regression with heavy-tailed and outlier-contaminated data.
problem Estimating sparse linear regression coefficients with heavy-tailed and outlier-contaminated data.
method Efficient computation of estimators with sharp error bounds.
result Sharp error bounds for efficient estimators.
A novel Bayesian framework for private linear regression with MCMC.
problem Private linear regression in a distributed setting.
method Generative statistical model, MCMC algorithms, fast Bayesian estimation.
result The proposed methods provide well-rounded estimation and prediction.
Study reveals how manifold geometry impacts linear regression solutions.
problem Impact of manifold geometry on linear regression solutions.
method Linear regression applied to manifold-structured data, focusing on extrinsic geometry.
result Linear regression does not have a unique solution on flat manifolds.
Paper proposes a new method for Bayesian linear regression using spike-and-slab priors.
problem Identifying predictors with similar relationships in linear regression models.
method Hierarchical Bayesian models with spike-and-slab priors and a Gibbs sampler.
result The proposed method outperforms previous methods in simulations and real data analysis.
Develops privacy-preserving methods for longitudinal linear regression.
problem Protecting individual information in longitudinal data with privacy-preserving statistics.
method Proposes a user-level private regression estimator and a privatized covariance estimator for longitudinal linear regression under user-level differential privacy.
result Establishes theoretical guarantees for practical user-level differential privacy estimation and inference in longitudinal linear regression.
Enhances linear regression with Kalman filter for loss minimization.
problem Minimizing loss in linear regression models.
method Integrates Kalman filter and SGD for optimal weight updates.
result Develops optimal linear regression equation with minimum area under curve.
Study improves robustness and sparsity in linear regression with adversarial outliers and heavy-tailed noise.
problem Outliers and heavy-tailed noise in linear regression coefficients.
method Sharp concentration inequalities and generic chaining.
result Sharper error bounds under weaker assumptions.
Robustly estimates linear regression coefficients with adversarial and noisy data.
problem Estimating robust linear regression coefficients with adversarial and noisy data.
method Adversarial robust weighted Huber regression with polynomial computational complexity.
result Derives an estimation error bound that depends on the stable rank and condition number of the covariance matrix.
Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
Improved algorithm for conditional linear regression with heterogeneous covariances.
problem Identifying a linear predictor for a fraction of data with varying covariances.
method Polynomial time algorithm using Disjunctive Normal Form (DNF) to identify a condition and linear predictor.
result Removed requirement for similar covariances in each condition term, improving algorithm applicability.
GD outperforms ridge regression and SGD in linear regression problems.
problem Comparing the risks of GD, ridge regression, and SGD in linear regression problems.
method Instance-wise finite-sample risk analysis of GD, ridge regression, and SGD.
result GD outperforms ridge regression and is incomparable with SGD in some cases.
New AMP algorithm estimates signals and latent variables in mixed regression models.
problem Estimating signals and latent variables in mixed regression models.
method Approximate Message Passing (AMP) algorithm for matrix GLM.
result State evolution recursion and optimal denoising functions for precise error minimization.
Estimates CATEs using high-dimensional linear regression models.
problem Estimating individualized causal effects (CATEs) in two treatments.
method Proposes a Lasso regression method for consistently estimating CATEs under high-dimensional and non-sparse parameters, leveraging the assumption of implicit sparsity.
result The proposed method is consistent for estimating CATEs.
Machine learning improves joint default assessment by capturing non-linear dependencies.
problem Capturing non-linear dependencies among covariates for accurate joint default assessment.
method Application of machine learning techniques to credit card dataset, comparing with logistic regression.
result Machine learning outperforms logistic regression in assessing portfolio riskiness.
Study finds exact limits for sparse regression with fewer observations than usual.
problem Understanding sparse linear regression with sublinear sparsity.
method Adaptive interpolation method and modified AMP algorithm.
result Exact asymptotic expressions for mutual information and MMSE in sublinear sparsity.
Linear regression models are not as interpretable as commonly believed.
problem Interpretability of linear regression models is often overlooked.
method Analysis of common XAI metrics and challenges faced by linear regression models.
result Linear regression models are not inherently interpretable and require careful consideration.
Extends online linear regression to handle multivariate data.
problem Hierarchical forecasting with multivariate responses.
method Introduces MultiVAW, extending Vovk-Azoury-Warmuth algorithm to multivariate setting.
result Achieves logarithmic regret in time for multivariate online linear regression.
We present a novel algorithm for non-linear instrumental variable (IV) regression, DualIV, which simplifies traditional two-stage methods via a dual formulation. Inspired by problems in stochastic programming, we show that two-stage procedures for non-linear IV regression can be reformulated as a convex-concave saddle-…