Improves logistic regression performance with nonconvex programming.
problem Stochastic generalized linear regression with chance constraints.
method Nonconvex programming techniques, clustering, quantile estimation.
result Over 1 to 2 percent improvement in model performance.
This paper studies robust regression in the settings of Huber's ε-contamination models. We consider estimators that are maximizers of multivariate regression depth functions. These estimators are shown to achieve minimax rates in the settings of ε-contamination models for various regression problems including nonpa…
Study uniform consistency in nonparametric mixture models and mixed regression.
problem Uniform consistency in nonparametric mixture models and mixed regression models.
method Construct uniformly consistent estimators under general conditions, develop novel technical tools.
result Prove uniform consistency results for nonparametric mixtures and mixed regression models.
Brenier isotonic regression extends multi-output isotonic regression using optimal transport.
problem Enforcing cyclic monotonicity in multi-output regression.
method Leverage Kantorovich's optimal transport to find cyclically monotone couplings.
result Brenier isotonic regression outperforms baselines in probability calibration.
We analyzed optimism in linear and kernel regression models.
problem Understanding predictive complexity in regression models.
method Derived closed-form asymptotic optimism for linear and kernel regression models.
result Scaled optimism is a useful measure for model complexity.
Study improves H-consistency bounds for regression analysis.
problem Improving H-consistency bounds for regression analysis. method Generalized theorems and novel H-consistency bounds for various surrogate loss functions. result Derives principled surrogate losses for adversarial regression.
This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially moda…
The study establishes risk bounds for distributional regression estimators.
problem Estimating distributional regression models with nonparametric methods.
method Theoretical bounds for CRPS and MSE are derived for convex and non-convex constraints.
result Theoretical risk bounds are validated through experiments on simulated and real data.
Unified approach for nonparametric regression and conditional distribution learning.
problem Nonparametric regression and conditional distribution learning problems.
method Generative learning framework with deep neural networks to estimate a conditional generator.
result The approach estimates a regression function and a conditional generator simultaneously, providing good prediction intervals.
Proposes a method for differentially private linear regression and synthetic data generation.
problem Lack of valid inference and synthetic data generation methods for small-scale datasets in privacy-aware settings.
method Gaussian differentially private linear regression with bias-corrected estimator and SDG procedure.
result Improves accuracy and provides valid confidence intervals for downstream tasks.
Enhances tensor regression for interpretability and performance.
problem Interpreting and modeling multidimensional tensor data with structural heterogeneity.
method Generalized Nonnegative Structured Kruskal Tensor Regression (NS-KTR) with hybrid regularization and nonnegativity constraints.
result NS-KTR outperforms conventional methods in synthetic and real hyperspectral datasets.
Unified framework SVAM learns GLMs robustly to adversarial label corruption.
problem Learning GLMs under adversarial label corruption.
method SVAM framework based on variance reduction technique.
result Provable model recovery guarantees superior to state-of-the-art.
Kernel ridgeless regression with random features shows good generalization without explicit regularization.
problem Generalization of kernel ridgeless regression without explicit regularization.
method Investigation of ridgeless regression with random features and stochastic gradient descent, exploring the effect of random features error and spectral density optimization.
result Random features error exhibits the double-descent curve, leading to improved generalization.
The study examines generalization bounds for regression and classification tasks on adaptive input domains.
problem Understanding the generalization error in adaptive input domains for regression and classification.
method The analysis considers regression and classification separately, using Lipschitz continuity and 2-norm/0/1 loss for measurement. It also highlights the polynomial relationship between generalization bounds and network parameters.
result Generalization bounds for regression and classification are inversely proportional to a polynomial of the number of parameters, emphasizing the advantages of over-parameterized networks.
This paper characterizes the conditional distribution properties of the finite sample ridge regression estimator and uses that result to evaluate total regression and generalization errors that incorporate the inaccuracies committed at the time of parameter estimation. The paper provides explicit formulas for those err…
Proposes a new regression method using conditional GANs.
problem Traditional regression methods make strong assumptions about data distribution.
method Uses conditional GANs to learn prediction functions that match training data pairs.
result New method has better representation capabilities and performs well on real-world datasets.
Meta-theorems validate fair regression algorithms under demographic parity constraints.
problem Regression under demographic parity constraints.
method Meta-theorems and post-processing methods.
result Fair minimax optimal regression can be achieved through post-processing.
New algorithm broadens BART models applicability.
problem Limited applicability of Bayesian additive regression trees (BART) models due to conditional conjugacy.
method Introduces a reversible jump Markov chain Monte Carlo algorithm for generalized BART models.
result Extends BART models to arbitrary generalized BART models without conditional conjugacy.
SGD benefits from a directional bias in kernel regression models.
problem Improving generalization in kernel regression models.
method Generalized directional bias property of SGD in kernel regression.
result SGD converges along the eigenvector of the largest eigenvalue of the Gram matrix.
C-Mixup improves generalization in regression tasks by adjusting label similarity.
problem Improving generalization in regression tasks with limited data.
method C-Mixup adjusts the probability of mixing examples based on label similarity.
result C-Mixup achieves better generalization and robustness compared to vanilla mixup.
Meta-learning improves predictions with generalized ridge regression in high-dimensional settings.
problem Improving meta-learning performance in high-dimensional settings.
method Generalized ridge regression applied to high-dimensional multivariate random-effects linear models.
result Optimal predictive risk achieved when using the inverse of the covariance matrix of random coefficients.
New method for GLMs under DP provides private uncertainty quantification.
problem Private inference for GLMs with uncertainty quantification.
method Noise-aware DP Bayesian inference method for GLMs.
result Posterior uncertainty allows determination of statistically significant coefficients.
AI-generated variables bias regression estimates; methods correct for invalid inference.
problem Bias in regression estimates due to AI-generated variables.
method Two methods: bias correction and joint estimation.
result Valid inference restored through proposed methods.
The paper improves methods for generating prediction intervals in regression.
problem Uncertainty quantification in regression models.
method Formalizes prediction interval generation as an optimization problem, studying generalization and calibration.
result Empirical demonstration of improved testing performances compared to existing methods.
SGD implicitly regularizes linear regression problems better than ridge regression for many cases.
problem Understanding implicit regularization in linear regression problems.
method Comparing SGD and ridge regression on a broad class of least squares problems.
result SGD generalizes no worse than ridge regression for many problem instances, sometimes better.
The paper reformulates regression in infinite dimensions as an inverse problem, showing it's equivalent to compact inverse problems.
problem Learning a linear operator between Hilbert spaces from empirical observations.
method Reformulates regression as an inverse problem, proving equivalence to compact inverse problems under specific conditions.
result The inverse problem is equivalent to compact inverse problems in terms of spectral properties and regularisation theory.
Improved estimator reduces bias in statistical learning models.
problem Asymptotic bias in classic WDRO estimator.
method Adjusted Wasserstein distributionally robust estimator.
result Asymptotic unbiased estimator with smaller MSE.
Introduces a new model for mapping matrices to matrices, subsuming linear regression.
problem Learning matrix-to-matrix mappings from data.
method Partial trace regression model, leveraging quantum information theory.
result Relevance demonstrated in matrix-to-matrix regression and positive semidefinite matrix completion.
In this paper, we study regression problems over a separable Hilbert space with the square loss, covering non-parametric regression over a reproducing kernel Hilbert space. We investigate a class of spectral/regularized algorithms, including ridge regression, principal component regression, and gradient methods. We pro…
In this work, we generalize semi-supervised generative adversarial networks (GANs) from classification problems to regression problems. In the last few years, the importance of improving the training of neural networks using semi-supervised training has been demonstrated for classification problems. We present a novel …
Prototype-based generative replay framework for online continual regression.
problem Addressing the challenge of non-stationary data streams in regression tasks.
method Adaptive output-space discretization model for prototype-based generative replay.
result Reduces forgetting and provides more stable performance.
Paper tackles uncertainty prediction for deep sequential regression.
problem Challenges in generating accurate uncertainty estimates for deep recurrent networks.
method Flexible method that generates symmetric and asymmetric uncertainty estimates without stationarity assumptions.
result Outperforms competitive baselines on both drift and non-drift scenarios.
Paper extends nonparametric regression bounds for dependent β-mixing samples.
problem Analyzing error in nonparametric regression with dependent data.
method Extends uniform deviation inequalities from independent to dependent β-mixing samples. result Derives generalization bounds for nonparametric regression with dependent data.
New ensemble SVM model reduces prediction error without choosing best kernel.
problem Reducing prediction error in regression problems.
method Bagged-weighted support vector regression model with random machines.
result Regression Random Machines achieve lower generalization error.
The paper tackles domain generalization using functional regression.
problem Learning a model that generalizes well across different source distributions.
method Functional regression approach to learn a linear operator between marginal and conditional distributions.
result The proposed algorithm achieves finite sample error bounds for the idealized risk.
New method for efficient maximum likelihood estimation of p-generalized probit regression.
problem Efficient estimation of p-generalized probit regression models. method Combining sketching techniques with importance subsampling to obtain a coreset.
result Maximum likelihood estimator can be approximated efficiently up to a factor of (1+ε) on large data. Collider regression improves predictive performance in regression tasks.
problem Discarding prior causal knowledge in regression tasks.
method Collider regression framework incorporating probabilistic causal knowledge from collider structures.
result Proves positive generalization benefit and provides closed-form estimators.
This paper analyzes RMR under Markov-dependent samples, improving understanding of its generalization error.
problem Understanding the generalization error of RMR in Markov-dependent settings.
method Established the upper bound for RMR estimator under Markov-dependent samples, providing a learning rate.
result Markov dependence affects the generalization error, reducing it by a multiplicative factor of the spectral gap.
BARMPy offers a Python package for Bayesian Additive Regression Models.
problem Making complex Bayesian models accessible to machine learning practitioners.
method Object-oriented design compatible with SciKit-Learn, documentation and tutorial provided.
result Ease of use and compatibility with existing machine learning tools.
Paper analyzes agnostic learning of mixed linear regression without generative models.
problem Learning mixed linear regression without assuming stochastic generation.
method Expectation Maximization (EM) and Alternating Minimization (AM) algorithms.
result AM and EM algorithms converge to population loss minimizers under standard conditions.
Unified framework for fair regression under demographic parity.
problem Ensuring fairness in regression tasks subject to demographic parity constraints.
method Proposes a unified framework applicable to various regression tasks with a broad spectrum of loss functions, derived a novel characterization of the fair risk minimizer, and established theoretical consistency and convergence rates.
result Effective minimization of risk while satisfying fairness constraints across various regression settings.
This paper proposes robust matrix variate regression models with rank constraints and vector regularization.
problem High dimensional and noisy matrix-valued predictors in regression models.
method Rank constraint, vector regularization, alternating projected gradient descent algorithm.
result The proposed method achieves the minimax rate of estimation errors.
Study improves robustness and sparsity in linear regression with adversarial outliers and heavy-tailed noise.
problem Outliers and heavy-tailed noise in linear regression coefficients.
method Sharp concentration inequalities and generic chaining.
result Sharper error bounds under weaker assumptions.
Study shows sample complexity for logistic regression with normal covariates.
problem Estimating parameters of logistic regression with normal design.
method Analyzes sample complexity in terms of dimension and inverse temperature.
result Shows two change-points in sample complexity curve based on inverse temperature.
Synthetic data can amplify privacy in linear regression models.
problem Understanding how synthetic data can enhance privacy in linear regression models.
method Investigated through the linear regression framework, analyzing synthetic data generated from random inputs and controlled inputs.
result Releasing a limited number of synthetic data points amplifies privacy beyond the model's inherent guarantees when inputs are random, but not when inputs are controlled by an adversary.
Regression, unlike classification, has lacked a comprehensive and effective approach to deal with cost-sensitive problems by the reuse (and not a re-training) of general regression models. In this paper, a wide variety of cost-sensitive problems in regression (such as bids, asymmetric losses and rejection rules) can be…
The least squares Monte Carlo algorithm has become popular for solving portfolio optimization problems. A simple approach is to approximate the value functions on a discrete grid of portfolio weights, then use control regression to generalize the discrete estimates. However, the classical global control regression can …
This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we c…