Introduces MWLD to measure loss inequality across groups.
problem Machine learning's focus on average loss can lead to large group loss discrepancies.
method Defines MWLD, relates it to fairness and robustness, and provides estimation methods.
result MWLD can be estimated efficiently under certain weighting functions and reduces loss variance without significant accuracy loss.
A new method optimizes neural sequence models for better task performance.
problem Training neural sequence models with maximum likelihood estimation ignores task losses.
method Maximum likelihood guided parameter search (MGS) in the parameter space.
result MGS optimizes sequence-level losses, reducing repetition and non-termination.
We consider the problem of training probabilistic conditional random fields (CRFs) in the context of a task where performance is measured using a specific loss function. While maximum likelihood is the most common approach to training CRFs, it ignores the inherent structure of the task's loss function. We describe alte…
Gradient descent on logistic loss converges to the maximum-margin separator for separable data.
problem Understanding the convergence of gradient descent on separable datasets with specific loss functions.
method Analysis of gradient descent on linear models with super-polynomially tailed losses.
result For separable datasets, gradient descent converges to the maximum-margin separator for losses with super-polynomial tails, but not for heavier tails.
New method improves structured prediction models using random structured outputs.
problem Improving structured prediction models in natural language processing.
method Linear-time principled algorithms using maximum loss over random structured outputs under Gaussian perturbations.
result The method produces a tighter upper bound of the Gibbs decoder distortion.
The paper introduces a minimax approach to supervised learning problems.
problem Optimal decision rule minimizing worst-case expected loss over probability distributions.
method Generalization of maximum entropy principle applied to constrained distributions.
result Developed a new linear classifier called the maximum entropy machine.
EBMs trained with ML are shown to behave like GANs with a self-adversarial loss.
problem Training EBMs with ML is intractable due to intractable unnormalized distributions.
method Replaced MCMC with deterministic gradient descent ODE solutions to study density induced by dynamics.
result EBM training is effectively a self-adversarial procedure rather than ML estimation.
Score matching offers efficient estimation for certain distributions.
problem Estimating probability distributions with intractable constants.
method Score matching as an alternative to maximum likelihood.
result Score matching is computationally and statistically efficient for certain distributions.
Study optimal consumption for loss-averse agents considering past spending peaks.
problem Optimal consumption for loss-averse agents with reference to past spending maximum.
method Adopted S-shaped utility, concave envelope, HJB variational inequality, dual transform, and smooth-fit conditions.
result Obtained piecewise closed-form solutions for optimal consumption and investment control.
The paper analyzes the maximum margin algorithm's performance on noisy data.
problem Analyzing the performance of maximum margin algorithm on noisy data.
method Finite-sample analysis of maximum margin algorithm applied to noisy data.
result The maximum margin algorithm can achieve nearly optimal population risk with sufficient over-parameterization.
Quantum models face barren plateaus, but specific losses can be trainable.
problem Barren plateaus and loss concentration in quantum generative models.
method Investigated explicit and implicit losses, and their interplay.
result Explicit losses lead to new barren plateaus, while implicit losses can be trainable.
Here we present an application of two maxentropic procedures to determine the probability density distribution of compound sums of random variables, using only a finite number of empirically determined fractional moments. The two methods are the Standard method of Maximum Entropy (SME), and the method of Maximum Entrop…
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
Improves GAN training with a repulsive loss function.
problem Discourages learning of fine details in data.
method Proposes a repulsive loss function and a bounded Gaussian kernel.
result Significantly improves GAN performance without additional computational cost.
Novel ramp loss method improves weakly supervised machine translation and parsing.
problem Training neural models without gold labels in weak supervision scenarios.
method Adapted ramp loss objectives to promote positive outputs and discourage negative ones.
result Bipolar ramp loss objectives outperform other methods on weakly supervised tasks.
New method improves diversity in GAN-generated images.
problem Lack of diversity in GAN-generated images.
method Introducing moment reconstruction losses to replace the reconstruction loss in GAN training.
result Improved diversity in generated images without sacrificing visual fidelity.
Analyzes how BatchNorm flattens the loss landscape in neural networks.
problem Understanding BatchNorm's impact on neural network optimization.
method Mean-field theory applied to quantify BatchNorm's effect on loss landscape.
result BatchNorm flattens the loss landscape, allowing for larger learning rates.
Paper proposes a new loss function for training neural speech models.
problem Training high-performance neural speech waveform models.
method Uses short-time Fourier transform (STFT) spectra and assumes Gaussian and von Mises distributions for amplitude and phase spectra.
result Synthesized high-quality speech waveforms.
MELC uses entropy for multithreshold classification, showing consistency similar to SVM.
problem Consistency of multithreshold linear classifiers.
method Employed multithreshold maximum margin model based on information theory.
result Objective function upper bounds misclassified points, similar to hinge loss.
SEARNN improves RNN training by incorporating global-local losses.
problem RNNs trained with MLE fail to exploit structured losses and suffer from exposure bias.
method SEARNN introduces global-local losses through test-alike search space exploration.
result SEARNN outperforms MLE on OCR, spelling correction, and machine translation tasks.
Mirror flow optimizes separable data problems, converging to a maximum margin classifier.
problem Optimizing classification problems with separable data using mirror flow.
method Examine mirror flow on linearly separable classification problems, focusing on the horizon function of the mirror potential.
result Mirror flow converges to a maximum margin classifier for separable data under certain conditions.
New method for risk allocation under multimodality of loss distribution.
problem Risk assessment under multimodal conditional loss distribution.
method Maximum Likelihood Allocation (MLA) and multimodality adjustment.
result Multimodality adjustment improves soundness of risk allocations.
Score matching fails to train VAEs robustly, revealing autoencoding loss insights.
problem Catastrophic failure of variational score matching on VAE models.
method Analysis of existing variational score matching objectives and their equivalence to autoencoding losses.
result Score matching methods fail to produce robust VAE models, predicting poor performance.
New decision-theoretic calibration error metric improves prediction reliability.
problem Improving the reliability of predictions for decision-making.
method Proposed Calibration Decision Loss (CDL) and an efficient algorithm to achieve near-optimal CDL.
result Near-optimal CDL guarantees vanishing payoff loss from miscalibration.
This study found significant asymmetry between potential maximum gain and loss in asset returns, improving predictability and utility for investors.
problem Understanding the economic value of price extremes in asset returns.
method Decomposing asset returns into PMG and PML, analyzing relationships and asymmetry, and testing predictive power.
result Significant asymmetry between PMG and PML, improving asset return predictability and utility for investors.
This paper explains why distributional reinforcement learning is better than vanilla RL using small-loss bounds.
problem Understanding when and why distributional reinforcement learning (DistRL) is superior to vanilla reinforcement learning (RL).
method The paper uses small-loss bounds to explain the benefits of DistRL, proposing algorithms and proving bounds for different RL settings.
result Distributional reinforcement learning (DistRL) outperforms vanilla RL when optimal costs are small, as shown by small-loss bounds.
A fast method for training linear classifiers maximizes margins.
problem Training linear classifiers with maximum margins.
method Momentum-based gradient method derived from convex dual with Nesterov acceleration.
result Exponentially faster convergence rate compared to standard methods.
Quantum machine learning uses quantum cross entropy to minimize loss, but measurement loss affects this process.
problem Quantum machine learning's loss minimization through cross entropy is affected by measurement outcomes.
method Defined quantum cross entropy, proved its lower bounds, and investigated its relation to quantum fidelity and likelihood.
result Quantum cross entropy is lower-bounded by negative log-likelihood when derived from quantum data, but measurement outcomes can cause loss.
Study improves adversarial classification using distributionally robust models.
problem Improving robustness against adversarial attacks in classification models.
method Distributionally robust chance constraints with Wasserstein ambiguity, reformulated as a regularized ramp loss minimization problem.
result Standard descent methods can converge to the global minimizer for the distributionally robust adversarial classification model.
Study minimax rates for nonparametric density estimation with adversarial losses.
problem Estimating densities under various adversarial loss functions.
method General framework for analyzing minimax rates with different loss functions.
result Determines the minimax rate based on loss choice and density smoothness.
Bayesian analysis optimizes stop-loss thresholds based on drawdown distributions.
problem Arbitrary stop-loss levels in financial strategies.
method Bayesian analysis of drawdown distributions.
result Systematic selection of optimal stop-loss thresholds.
Simplified explanation of ML for mixtures and OT.
problem Maximum likelihood estimation for mixture models.
method Optimal transport with entropic regularization.
result EM algorithm is a block-coordinate descent on OT loss.
Optimizes risk measures given known marginal distributions of two unknown factors.
problem Determining an upper bound for spectral risk measures with unknown joint distribution.
method Introduces Maximum Spectral Measure (MSP) as a worst-case risk measure, formulated as an optimization problem with a more general objective function.
result Characterizes the continuity properties of the optimal value function and optimal solution set with respect to marginal distributions.
We improve GANs by enforcing gradient constraints for MMD critics.
problem Stabilizing and accelerating training of GANs.
method Propose a new gradient-based regularization method for MMD GANs.
result New loss function stabilizes and accelerates GAN training.
This manuscript shows that AdaBoost and its immediate variants can produce approximate maximum margin classifiers simply by scaling step size choices with a fixed small constant. In this way, when the unscaled step size is an optimal choice, these results provide guarantees for Friedman's empirically successful "shrink…
The paper examines the asymptotic normality of MLE in operational risk models.
problem The validity of asymptotic normality for MLE in operational risk models is questionable.
method The study evaluates the asymptotic normality of MLE for common severity distributions in operational risk models.
result The paper finds that asymptotic normality does not hold for operational risk models, leading to potential errors in confidence intervals.
New learning scheme outperforms ERM in individual data settings.
problem Learning from individual data samples.
method Information-theoretic approach using self-information loss.
result pNML scheme outperforms ERM in specific test challenges.
We propose a robust estimator to improve maximum likelihood in probabilistic models.
problem Overfitting and sensitivity to noise in maximum likelihood estimation.
method Distributionally robust maximum likelihood estimator that minimizes worst-case expected log-loss.
result The robust estimator is statistically consistent and performs well in regression and classification tasks.
Improved online convex optimization with delayed feedback using curvature.
problem Online convex optimization with curved losses and delayed feedback.
method Variant of follow-the-regularized-leader and Online Newton Step algorithm with adaptive learning rate.
result Regret bounds of order min { σ max ln T , d t o t } \min\{σ_{\max}\ln T, \sqrt{d_{\mathrm{tot}}}\} min { σ m a x ln T , d tot } for exp-concave losses. Improves reinforcement learning extrapolation in Gridworlds.
problem Generalizing to unseen states in reinforcement learning.
method Avoiding deterministic action choice, using ego-centric representation, incorporating symmetry, and adding an entropy term.
result Significant improvement in extrapolation performance in Gridworld environments.
Typically, operational risk losses are reported above a threshold. Fitting data reported above a constant threshold is a well known and studied problem. However, in practice, the losses are scaled for business and other factors before the fitting and thus the threshold is varying across the scaled data sample. A report…
We consider the game-theoretic scenario of testing the performance of Forecaster by Sceptic who gambles against the forecasts. Sceptic's current capital is interpreted as the amount of evidence he has found against Forecaster. Reporting the maximum of Sceptic's capital so far exaggerates the evidence. We characterize t…
Innovative game theory approach optimizes survival analysis metrics.
problem Survival analysis models trained with maximum likelihood do not directly optimize criteria like Brier score or Bernoulli log likelihood.
method Inverse-Weighted Survival Games: Construct objectives from re-weighted estimates featuring the other model, holding the latter fixed during training.
result Games optimize Brier score on simulations and real-world data.
Generative neural network simulates characteristic functions.
problem Simulating from characteristic functions inaccessible in closed form.
method Generative neural network with Maximum-Mean-Discrepancy loss.
result Universal algorithm independent of dimensionality and function properties.
In binary classification problems, mainly two approaches have been proposed; one is loss function approach and the other is uncertainty set approach. The loss function approach is applied to major learning algorithms such as support vector machine (SVM) and boosting methods. The loss function represents the penalty of …
New vector quantization method reduces relevance of parallel components in database points.
problem Scaling maximum inner product search to massive databases.
method Developed anisotropic vector quantization loss functions.
result Achieves state-of-the-art results on public benchmarks.
The paper explores how benign overfitting occurs in heavy-tailed input distributions.
problem Understanding overfitting in heavy-tailed input distributions.
method Analysis of maximum margin classifiers on unregularized logistic loss with gradient descent.
result Linear classifiers trained under certain conditions can asymptotically achieve the noise level as misclassification error.
Proposes a new loss function for reconciling hierarchical forecasts.
problem Hierarchical forecasting with reconciliation issues.
method Integrates a new loss function into maximum likelihood objectives for hierarchical data.
result Improves forecast accuracy over existing methods.