arXiv research

New loss function reduces outage probability in ML-assisted resource allocation.

Derives bounds for deterministic predictors using smooth loss functions.

New methods evaluate data representations by complexity of low-loss predictor learning.

This work improves structured prediction by learning the balance between signal and random noise.

The study finds a trade-off between model size, test loss, and training loss for linear predictors.

Optimizing proper loss yields calibrated models under specific conditions.

Loss minimization leads to multicalibration for neural networks.

Derandomizing PAC-Bayes bounds for smooth loss functions

Adversarial training achieves optimal test error for shallow networks.

New algorithms for multi-class classification with abstention.

We carefully study how well minimizing convex surrogate loss functions, corresponds to minimizing the misclassification error rate for the problem of binary classification with linear predictors. In particular, we show that amongst all convex surrogate losses, the hinge loss gives essentially the best possible bound, o…

Loss minimisation fails to capture epistemic uncertainty in second-order predictors.

Post-processing predictors reduces calibration errors for decision-making.

Study tail risk in high-frequency finance using $L_1$-regularized regression.

New machine learning paradigm ignores loss function until action time.

Study shows how to improve contextual bandits with loss predictors.

In this short note, we provide a sample complexity lower bound for learning linear predictors with respect to the squared loss. Our focus is on an agnostic setting, where no assumptions are made on the data distribution. This contrasts with standard results in the literature, which either make distributional assumption…

New bounds show linear predictors rarely overfit with certain optimization methods.

Sparse model selection by structural risk minimization leads to a set of a few predictors, ideally a subset of the true predictors. This selection clearly depends on the underlying loss function $\tilde L$. For linear regression with square loss, the particular (functional) Gradient Boosting variant $L_2-$Boosting exce…

New insights into when benign overfitting occurs in linear and classification tasks.

The problem of sequential probability forecasting is considered in the most general setting: a model set C is given, and it is required to predict as well as possible if any of the measures (environments) in C is chosen to generate the data. No assumptions whatsoever are made on the model class C, in particular, no ind…

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

This work analyzes fairness-accuracy trade-offs using causal methods.

We examine gradient descent on unregularized logistic regression problems, with homogeneous linear predictors on linearly separable datasets. We show the predictor converges to the direction of the max-margin (hard margin SVM) solution. The result also generalizes to other monotone decreasing loss functions with an inf…

Deep neural networks improve AFT model for non-linear predictors.

Flexible framework for bounding high-loss predictions using quantiles.

Empirically, the PAC-Bayesian analysis is known to produce tight risk bounds for practical machine learning algorithms. However, in its naive form, it can only deal with stochastic predictors while such predictors are rarely used and deterministic predictors often performs well in practice. To fill this gap, we develop…

This study addresses the issue of predicting the glaucomatous visual field loss from patient disease datasets. Our goal is to accurately predict the progress of the disease in individual patients. As very few measurements are available for each patient, it is difficult to produce good predictors for individuals. A rece…

Learning linear predictors with the logistic loss---both in stochastic and online settings---is a fundamental task in machine learning and statistics, with direct connections to classification and boosting. Existing "fast rates" for this setting exhibit exponential dependence on the predictor norm, and Hazan et al. (20…

Mack's estimator improves chain ladder prediction for large exposure insurance models.

We consider selection of random predictors for high-dimensional regression problem with binary response for a general loss function. Important special case is when the binary model is semiparametric and the response function is misspecified under parametric model fit. Selection for such a scenario aims at recovering th…

New test for SGD in binary classification reduces computation time.

Adaptive kernels from neural networks improve model performance.

Unified framework for portfolio optimization using multiple hypotheses.

This work uses PAC-Bayes for structured prediction with ILE, yielding insights and algorithms.

The paper improves PAC-Bayes bounds for data-dependent predictors.

Unified multitask learning framework for mixed-type outcomes.

Study improves flood loss risk models using historical data and rainfall data.

New algorithm reduces sample complexity for omnipredictors of SIMs.

The paper develops a deep neural network estimator for weakly dependent processes with various loss functions.

Unified framework for generalized Venn and Venn-Abers calibration for reliable prediction.

The study reduces a personality measurement instrument to 10 features with minimal loss of accuracy.

This paper proposes a new algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i.e., networks that provably satisfy some desired input-output properties. The key idea is to simultaneously train two networks: a predictor network that performs the task at hand,e.g., predicting…

The Lasso is a computationally efficient regression regularization procedure that can produce sparse estimators when the number of predictors (p) is large. Oracle inequalities provide probability loss bounds for the Lasso estimator at a deterministic choice of the regularization parameter. These bounds tend to zero if …

In this article, we derive concentration inequalities for the cross-validation estimate of the generalization error for stable predictors in the context of risk assessment. The notion of stability has been first introduced by \cite{DEWA79} and extended by \cite{KEA95}, \cite{BE01} and \cite{KUNIY02} to characterize cla…

Gradient methods avoid overfitting on separable data.

Transformer-based models overfit financial time series data, leading to increased prediction variance.

A framework for ranking with abstention, offering theoretical guarantees and practical effectiveness.