arXiv research

Study improves $H$-consistency bounds for regression analysis.

Paper explores connections between loss functions and consistency in binary classification and regression.

Unified framework for fair regression under demographic parity.

We find a convex model for traditional nonlinear regression under L2 loss.

Paper introduces arctan pinball loss for XGBoost quantile regression.

Enhances linear regression with Kalman filter for loss minimization.

Optimal sketching bounds for sparse linear regression under various loss functions are established.

Classification and regression tasks in overparameterized models show different generalization properties.

Adversarial training makes logistic regression weight loss landscapes sharper.

Two models predict net loan losses using Bayesian and frequentist regression.

Develops a new framework for robust regression with EGM.

The paper introduces a new FOR framework using Huber and ε-insensitive losses.

Regression, unlike classification, has lacked a comprehensive and effective approach to deal with cost-sensitive problems by the reuse (and not a re-training) of general regression models. In this paper, a wide variety of cost-sensitive problems in regression (such as bids, asymmetric losses and rejection rules) can be…

This paper examines the Histogram Loss for regression, revealing its effectiveness without needing complex tuning.

LALR adapts learning rate for faster convergence in regression and neural nets.

Prevalidated ridge regression simplifies logistic regression for high-dimensional data.

In this paper, we introduce a novel combined reward cum penalty loss function to handle the regression problem. The proposed combined reward cum penalty loss function penalizes the data points which lie outside the $ε$-tube of the regressor and also assigns reward for the data points which lie inside of the $ε$-tube of…

Ridge regression CV loss may have multiple local optima.

This work improves online regression and contextual bandits using neural networks.

We present $α$-loss, $α\in [1,\infty]$, a tunable loss function for binary classification that bridges log-loss ($α=1$) and $0$-$1$ loss ($α= \infty$). We prove that $α$-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal y…

The paper explores MAE as a loss function for DNN vector-to-vector regression, proving its advantages over MSE.

We introduce a new principle for model selection in regression and classification. Many regression models are controlled by some smoothness or flexibility or complexity parameter c, e.g. the number of neighbors to be averaged over in k nearest neighbor (kNN) regression or the polynomial degree in regression with polyno…

This paper studies distributed estimation and support recovery for high-dimensional linear regression model with heavy-tailed noise. To deal with heavy-tailed noise whose variance can be infinite, we adopt the quantile regression loss function instead of the commonly used squared loss. However, the non-smooth quantile …

CcGAN tackles conditional image generation for continuous labels.

Paper proves L2 regression can learn k-juntas without distributional assumptions.

Paper analyzes statistical properties of log-cosh loss function.

Principal component regression (PCR) is a widely used two-stage procedure: principal component analysis (PCA), followed by regression in which the selected principal components are regarded as new explanatory variables in the model. Note that PCA is based only on the explanatory variables, so the principal components a…

Develops a privacy-preserving algorithm for sparse robust regression.

Adapting robust statistics to neural networks, researchers found neural networks can be more robust with certain loss functions.

There is growing evidence that converting targets to soft targets in supervised learning can provide considerable gains in performance. Much of this work has considered classification, converting hard zero-one values to soft labels---such as by adding label noise, incorporating label ambiguity or using distillation. In…

Least squares kernel based methods have been widely used in regression problems due to the simple implementation and good generalization performance. Among them, least squares support vector regression (LS-SVR) and extreme learning machine (ELM) are popular techniques. However, the noise sensitivity is a major bottlene…

A novel framework for regression with multiple experts, addressing challenges in infinite and continuous label spaces.

New model leads to optimal test loss in sparse linear regression.

We obtain the first positive results for bounded sample compression in the agnostic regression setting with the $\ell_p$ loss, where $p\in [1,\infty]$. We construct a generic approximate sample compression scheme for real-valued function classes exhibiting exponential size in the fat-shattering dimension but independen…

This paper aims at refined error analysis for binary classification using support vector machine (SVM) with Gaussian kernel and convex loss. Our first result shows that for some loss functions such as the truncated quadratic loss and quadratic loss, SVM with Gaussian kernel can reach the almost optimal learning rate, p…

Optimal convex loss function improves regression coefficient estimation.

Paper introduces robust kernel ridge regression using Cauchy loss for handling various noise types.

Paper introduces robust distribution regression using kernel methods.

Study binary choice with asymmetric loss, offering simple solutions.

This paper establishes minimax rates for online regression with arbitrary classes of functions and general losses. We show that below a certain threshold for the complexity of the function class, the minimax rates depend on both the curvature of the loss function and the sequential complexities of the class. Above this…

Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…

Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.

Guarantees uniform convergence for square-root Lipschitz losses.

Neural networks perform differently when regression is treated as classification.

Develops a fast algorithm for high-dimensional LASSO penalized quantile regression.

In this paper, we propose a novel asymmetric $ε$-insensitive pinball loss function for quantile estimation. There exists some pinball loss functions which attempt to incorporate the $ε$-insensitive zone approach in it but, they fail to extend the $ε$-insensitive approach for quantile estimation in true sense. The propo…

Study enhances robustness of In-CVaR based regression models under perturbation and contamination.

Introduces Soft-SVM for binary classification bridging logistic and SVM.