Study of loss functions for learning to defer, proving consistency.
problem Learning to defer in machine learning.
method Introduced a family of surrogate losses parameterized by Ψ and proved their consistency. result Proved realizable H-consistency and Bayes-consistency of specific surrogate losses. Paper connects risk consistency to L_p consistency for broader loss functions.
problem Establishing risk consistency for a wider class of loss functions.
method Analyzes the connection between risk consistency and L_p-consistency for various loss functions.
result Shifted loss functions do not reduce assumptions as much as other results.
Self-consistent models improve reinforcement learning by aligning predictions with future values.
problem Improving reinforcement learning by aligning model predictions with future values.
method Proposes multiple self-consistency updates to encourage a learned model and value function to be consistent with each other.
result Self-consistency helps both policy evaluation and control in both tabular and function approximation settings.
This paper tackles deferral learning with multiple experts, providing strong theoretical guarantees.
problem Optimizing input assignment to experts balancing accuracy and computational cost.
method Introducing new surrogate loss functions and efficient algorithms with strong theoretical learning guarantees.
result Realizable H-consistency, H-consistency bounds, and Bayes-consistency for deferral learning. In this paper we study the consistency of an empirical minimum error entropy (MEE) algorithm in a regression setting. We introduce two types of consistency. The error entropy consistency, which requires the error entropy of the learned function to approximate the minimum error entropy, is shown to be always true if the…
Study dynamic risk measures and performance indices using distortion functions.
problem Investigate time consistency of dynamic risk measures and performance indices generated by distortion functions.
method Analyze dynamic coherent risk measures (DCRMs) and dynamic weighted value at risk measures, proving their equivalence. Establish properties of families of DCRMs generated by distortion functions and define corresponding dynamic coherent acceptability indices (DCAIs). Examine time consistency of DCRMs and DCAIs.
result DCRM generated by distortion functions are sub-martingale time consistent but not super-martingale time consistent and not weakly acceptance time consistent.
Study uniform consistency in nonparametric mixture models and mixed regression.
problem Uniform consistency in nonparametric mixture models and mixed regression models.
method Construct uniformly consistent estimators under general conditions, develop novel technical tools.
result Prove uniform consistency results for nonparametric mixtures and mixed regression models.
New algorithms improve on consistency and robustness in convex function chasing with black-box advice.
problem Minimizing cost in normed vector space with black-box advice for convex function chasing.
method Two novel algorithms: INTERP and BDINTERP, exploiting convexity to achieve improved consistency and robustness.
result BDINTERP achieves near-optimal consistency-robustness trade-off for α-polyhedral cost functions.
Develops a framework for consistent loss functions with variable transformations.
problem Lack of theoretical understanding of variable transformations in consistent loss functions.
method Formal characterizations of consistency for transformed loss functions in two cases: realization and prediction variables.
result Establishes new identifiable and elicitable functionals for complex predictive tasks.
Study on top-k classification with new loss functions and algorithms.
problem Improving multi-class classification accuracy and cardinality trade-off.
method Introducing cardinality-aware loss functions and deriving their consistency bounds.
result New cardinality-aware algorithms for top-k classification. The paper proves neural networks' consistency and optimal convergence rates for various function classes.
problem Proving neural networks' consistency and optimal convergence rates for diverse function classes.
method Analyzes wide and deep ReLU neural networks trained on logistic loss and Kolmogorov-Donoho optimal function classes.
result Proves universal consistency and minimax optimal convergence rates for neural networks.
Improves risk and variability measures continuity and consistency.
problem Improving the continuity and consistency of risk and variability measures.
method Analyzes convex and order bounded above functionals on Frechet lattices and Orlicz spaces.
result Order-continuous, law-invariant functionals on Orlicz spaces are strongly consistent everywhere.
A consistency criterion for price impact functions in limit order markets is proposed that prohibits chain arbitrage exploitation. Both the bid-ask spread and the feedback of sequential market orders of the same kind onto both sides of the order book are essential to ensure consistency at the smallest time scale. All t…
AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …
Paper explores connections between loss functions and consistency in binary classification and regression.
problem Consistency in binary classification and regression applications.
method Characterization of conformable loss functions and derivation of a new Huber-type loss function.
result Margin-based loss functions are equivalent to loss functions of squared standardized logistic regression residuals.
In the practice of point prediction, it is desirable that forecasters receive a directive in the form of a statistical functional, such as the mean or a quantile of the predictive distribution. When evaluating and comparing competing forecasts, it is then critical that the scoring function used for these purposes be co…
Sublinearly structured DNNs achieve feature learning consistency for compositional functions.
problem Achieving feature-learning and prediction consistency in deep neural networks.
method Sublinearly structured DNNs
result Sublinearly structured DNNs match or surpass wide DNNs in prediction.
Unified surrogate loss framework for multi-label learning with strong consistency guarantees.
problem Improving consistency and accounting for label correlations in multi-label learning.
method Introducing multi-label logistic loss and extending it to comprehensive multi-label comp-sum losses, proving strong consistency guarantees for any multi-label loss.
result Unified surrogate loss framework benefiting from strong consistency guarantees for any multi-label loss.
The paper proves consistency of neural networks with regularization.
problem Overfitting in neural networks with large scale data.
method Theoretical framework of neural networks with regularization, sieves method, and minimal neural networks theory.
result The estimated neural network converges to the true underlying function as sample size increases.
Consistency regularization improves robustness to noisy labels.
problem Improving model robustness to noisy labels in machine learning.
method Empirical study of consistency regularization on noisy datasets.
result Consistency regularization improves model robustness to label noise.
A new approach to L2-consistent estimation of a general density functional using k-nearest neighbor distances is proposed, where the functional under consideration is in the form of the expectation of some function f of the densities at each point. The estimator is designed to be asymptotically unbiased, using t…
Proposes a new loss function for learning with noisy labels.
problem Improving model learnability with noisy labels.
method Uses generalized Jensen-Shannon divergence as a noise-robust loss function.
result Shows state-of-the-art results on noisy data.
A framework for sensitivity measures using scoring functions.
problem Constructing sensitivity measures for any elicitable functional.
method Score-based sensitivities constructed via consistent scoring functions.
result Demonstrated intuitive and desirable properties of score-based sensitivities.
We consider families of strongly consistent multivariate conditional risk measures. We show that under strong consistency these families admit a decomposition into a conditional aggregation function and a univariate conditional risk measure as introduced Hoffmann et al. (2016). Further, in analogy to the univariate cas…
Aggregation distorts causal discovery results but recovery is possible with partial linearity or prior.
problem Understanding how temporal aggregation affects causal discovery in aggregated data.
method Functional consistency and conditional independence consistency methods.
result Causal discovery results may be distorted by aggregation, but recovery is possible with certain conditions.
Observational cohort studies with oversampled exposed subjects are typically implemented to understand the causal effect of a rare exposure. Because the distribution of exposed subjects in the sample differs from the source population, estimation of a propensity score function (i.e., probability of exposure given basel…
Support vector machines (SVMs) are special kernel based methods and belong to the most successful learning methods since more than a decade. SVMs can informally be described as a kind of regularized M-estimators for functions and have demonstrated their usefulness in many complicated real-life problems. During the last…
We give a condition which ensures that the Paneitz operator of an embedded three-dimensional CR manifold is nonnegative and has kernel consisting only of the CR pluriharmonic functions. Our condition requires uniform positivity of the Webster scalar curvature and the stability of the CR pluriharmonic functions for a re…
Identification and scoring functions are statistical tools to assess the calibration and the relative performance of risk measure estimates, e.g., in backtesting. A risk measures is called identifiable (elicitable) it it admits a strict identification function (strictly consistent scoring function). We consider measure…
A central problem in ranking is to design a ranking measure for evaluation of ranking functions. In this paper we study, from a theoretical perspective, the widely used Normalized Discounted Cumulative Gain (NDCG)-type ranking measures. Although there are extensive empirical studies of NDCG, little is known about its t…
Constant price impact functions, much used in financial literature, are shown to give rise to paradoxical outcomes since they do not allow for proper predictability removal: for instance the exploitation of a single large trade whose size and time of execution are known in advance to some insider leaves the arbitrage o…
Proposes a method for multi-view clustering that integrates consistent and complementary graph regularizers.
problem Multi-view clustering where views have both consistent and complementary information.
method Consistent and complementary graph-regularized multi-view subspace clustering (GRMSC).
result The proposed method outperforms state-of-the-art methods on benchmark datasets.
Deep neural networks without regularization can achieve consistent estimates with good convergence rates.
problem The necessity of regularization in deep neural networks for consistent estimates.
method Gradient descent on an over-parametrized neural network without regularization, with specific initialization, step size, and number of steps.
result An estimate without regularization is universally consistent and achieves good convergence rates.
We provide a theoretical foundation for non-parametric estimation of functions of random variables using kernel mean embeddings. We show that for any continuous function f, consistent estimators of the mean embedding of a random variable X lead to consistent estimators of the mean embedding of f(X). For Matérn ke…
Study optimal stopping for group with diverse discount rates using an attitude function.
problem Optimal stopping for a group with diverse discount rates under an aggregation preference.
method Develop iterative approach using consistent planning for time-consistent equilibria.
result Characterize all time-consistent mild equilibria as fixed points of an operator.
This paper introduces new loss functions for balanced multi-class classification.
problem Balancing class imbalance in multi-class classification.
method Introduces two new surrogate loss families: GLA and GCA.
result GCA losses offer stronger theoretical guarantees in imbalanced settings.
Researchers analyze the relationship between ML cost functions and the C-index in survival analysis.
problem Understanding the relationship between ML cost functions and the C-index in survival analysis.
method Provided C-index Fisher-consistency results and excess risk bounds for various cost functions in survival analysis.
result Identified conditions under which ML cost functions are consistent with the C-index.
Equivalent characterizations of multiportfolio time consistency are deduced for closed convex and coherent set-valued risk measures on Lp(Ω,F,P;Rd) with image space in the power set of Lp(Ω,Ft,P;Rd). In the convex case, multiportfolio time consistency is equivalent to a cocycle condition on…
Graph-based semi-supervised learning is the problem of propagating labels from a small number of labelled data points to a larger set of unlabelled data. This paper is concerned with the consistency of optimization-based techniques for such problems, in the limit where the labels have small noise and the underlying unl…
The paper introduces Bellman-consistent pessimism to improve offline reinforcement learning without overly pessimistic bias.
problem Offline reinforcement learning's challenge of discovering good policies without exhaustive exploration.
method Introduces Bellman-consistent pessimism for function approximation, improving sample complexity and adaptability.
result Improves sample complexity by O(d) in the action space finite case, and automatically adapts to bias-variance tradeoff. The paper classifies Morse functions on 3-manifolds with specific level sets.
problem Characterizing 3-manifolds using Morse functions with certain level sets.
method Study of Morse functions with regular level sets consisting of spheres, tori, or Klein Bottles.
result Classification of Morse functions on specific 3-manifolds.
We formalize and study the natural approach of designing convex surrogate loss functions via embeddings, for problems such as classification, ranking, or structured prediction. In this approach, one embeds each of the finitely many predictions (e.g.\ rankings) as a point in Rd, assigns the original loss val…
Working in a continuous time setting, we extend to the general case of dynamic risk measures continuous from above the characterization of time consistency in terms of ``cocycle condition'' of the minimal penalty function. We prove also the supermartingale property for general time consistent dynamic risk measures. Whe…
Gradient span algorithms show consistent progress in high dimensions.
problem Understanding consistent training progress in large machine learning models.
method Proving deterministic behavior of gradient span algorithms on Gaussian random functions.
result Gradient span algorithms have asymptotically deterministic behavior in high dimensions.
We study time-consistency questions for processes of monetary risk measures that depend on bounded discrete-time processes describing the evolution of financial values. The time horizon can be finite or infinite. We call a process of monetary risk measures time-consistent if it assigns to a process of financial values …
Proposes a new multi-view graph learning framework to model consistency and inconsistency.
problem Graph learning methods often neglect inconsistency across multiple views, making them vulnerable to noisy datasets.
method Proposes a unified objective function to simultaneously model consistency and inconsistency, iteratively learning consistent and unified graphs.
result Demonstrates robustness and efficiency of the proposed approach on twelve multi-view datasets.
This paper establishes a theoretical foundation for consistency training in diffusion models.
problem Lack of a comprehensive theoretical understanding of consistency training in diffusion models.
method Demonstrates the necessity of a number of steps in consistency learning exceeding d5/2/ε for generating samples within ε proximity to the target distribution. result Establishes rigorous insights into the validity and efficacy of consistency models, offering theoretical underpinnings for their utility.
Dynamic submodular maximization with consistency constraints.
problem Maximizing submodular functions in a streaming environment with limited changes.
method Algorithms with trade-offs between consistency and approximation quality.
result Effective algorithms for real-world applications.