arXiv research

We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.

In this work we provide a theoretical framework for structured prediction that generalizes the existing theory of surrogate methods for binary and multiclass classification based on estimating conditional probabilities with smooth convex surrogates (e.g. logistic regression). The theory relies on a natural characteriza…

Surrogate-based analysis of interactions via local effect smooths

In statistical learning theory, convex surrogates of the 0-1 loss are highly preferred because of the computational and theoretical virtues that convexity brings in. This is of more importance if we consider smooth surrogates as witnessed by the fact that the smoothness is further beneficial both computationally- by at…

Linear-Core Surrogates combine fast optimization and statistical efficiency in classification and structured prediction.

Paper establishes a universal growth rate for smooth surrogate losses in classification.

Proposes a method for inference in high-dimensional classification with non-differentiable surrogate losses.

The paper proposes a method to model non-smooth functions using clustering, classification, and Gaussian process modeling.

Deep neural networks improve surrogate models for non-smooth quantities in uncertain geometries.

New method improves robustness of smoothed classifiers against adversarial attacks.

Study on $H$-consistency bounds for machine learning surrogates.

Optimizes hard-to-optimize metrics using adaptive surrogates.

New algorithm learns optimal resource allocation in wireless systems without models.

Bayesian optimization tackles non-smooth tuning problems.

PPOS improves PPO by smoothing the surrogate objective function.

Generative Bayesian Computation improves surrogates for expensive simulations.

The minimization of loss functions is the heart and soul of Machine Learning. In this paper, we propose an off-the-shelf optimization approach that can minimize virtually any non-differentiable and non-decomposable loss function (e.g. Miss-classification Rate, AUC, F1, Jaccard Index, Mathew Correlation Coefficient, etc…

Annealed Entropic Allocation improves ranking and selection by mitigating hard switching and improving finite-budget discrimination.

Paper shows SVM can achieve super fast convergence rates.

In this paper, we consider an $\ell_{0}$-norm penalized formulation of the generalized eigenvalue problem (GEP), aimed at extracting the leading sparse generalized eigenvector of a matrix pair. The formulation involves maximization of a discontinuous nonconcave objective function over a nonconvex constraint set, and is…

Survival trees exhibit end-cut preference, leading to biased splits.

New method smooths optimization for sparse regularization.

A new method for fast and robust sparsity learning over networks.

Unified approach for federated learning using MM optimization.

New model guarantees integer optimal solutions for combinatorial problems.

SBBO optimizes complex spaces using sampling-based models.

Signal estimation problems with smoothness and sparsity priors can be naturally modeled as quadratic optimization with $\ell_0$-"norm" constraints. Since such problems are non-convex and hard-to-solve, the standard approach is, instead, to tackle their convex surrogates based on $\ell_1$-norm relaxations. In this paper…

Optimizes expensive functions using adaptive RBF surrogate model.

New method improves solving combinatorial optimization problems with smoothed policies.

Proposes sigmoidF1 loss for multilabel classification, improving performance metrics.

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.

Unified framework for optimal transport on curved spaces using neural potentials.

We learn a compact surrogate model for optimization problems to reduce training and inference time.

Bayesian optimization on networks using Gaussian process models.

Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.

Comment on entropy learning for dynamic treatment regimes.

Gaptron algorithm reduces mistakes in online multiclass classification.

Study detects boundaries in unlabeled noisy images without labels.

New bounds show polyhedral surrogates are optimal for generalization.

New algorithm optimizes Hölder smooth functions in RKHS with tighter regret bounds.

The paper tackles fast rates in structured prediction problems.

Framework creates fast, interpretable surrogates for stochastic simulators with unbounded randomness.

New BdryMatérn GP model for reliable boundary integration on irregular domains.

Randomizing the Fourier-transform (FT) phases of temporal-spatial data generates surrogates that approximate examples from the data-generating distribution. We propose such FT surrogates as a novel tool to augment and analyze training of neural networks and explore the approach in the example of sleep-stage classificat…

Physics-informed neural networks improve surrogate modeling of turbulent Rayleigh-Bénard convection.

Stochastic Gradient Descent (SGD) has played a central role in machine learning. However, it requires a carefully hand-picked stepsize for fast convergence, which is notoriously tedious and time-consuming to tune. Over the last several years, a plethora of adaptive gradient-based algorithms have emerged to ameliorate t…

We formalize and study the natural approach of designing convex surrogate loss functions via embeddings, for problems such as classification, ranking, or structured prediction. In this approach, one embeds each of the finitely many predictions (e.g.\ rankings) as a point in $\mathbb{R}^d$, assigns the original loss val…

The paper proposes a scalable framework for uncertainty quantification and propagation in surrogate-based Bayesian inference.