A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Paper explores connections between loss functions and consistency in binary classification and regression.
Gradient descent implicitly follows regularization for general losses.
AUC (area under ROC curve) is an important evaluation criterion, which has been popularly used in many learning tasks such as class-imbalance learning, cost-sensitive learning, learning to rank, etc. Many learning approaches try to optimize AUC, while owing to the non-convexity and discontinuousness of AUC, almost all …
A new method for exponentially weighted moving models using approximations.
Exponential Lasso improves Lasso's robustness to outliers and heavy-tailed noise.
New aggregation strategy handles unbounded losses with regret bounds.
We study online learning under logarithmic loss with regular parametric models. Hedayati and Bartlett (2012b) showed that a Bayesian prediction strategy with Jeffreys prior and sequential normalized maximum likelihood (SNML) coincide and are optimal if and only if the latter is exchangeable, and if and only if the opti…
We consider binary classification problems with positive definite kernels and square loss, and study the convergence rates of stochastic gradient methods. We show that while the excess testing loss (squared loss) converges slowly to zero as the number of observations (and thus iterations) goes to infinity, the testing …
The AdaBoost algorithm was designed to combine many "weak" hypotheses that perform slightly better than random guessing into a "strong" hypothesis that has very low error. We study the rate at which AdaBoost iteratively converges to the minimum of the "exponential loss." Unlike previous work, our proofs do not require …
Boosting with tempered exponential measures improves AdaBoost's convergence rate.
We consider on-line density estimation with a parameterized density from the exponential family. The on-line algorithm receives one example at a time and maintains a parameter that is essentially an average of the past examples. After receiving an example the algorithm incurs a loss which is the negative log-likelihood…
In this paper, we propose a novel {\em -exponentiated} transformation, , for loss functions. When the transformation is applied to a convex loss function, the transformed loss function become more robust to outliers. Using a novel generalization error bound, we have theoretically shown that the transforme…
Using Monte Carlo simulation to calculate the Value at Risk (VaR) as a possible risk measure requires adequate techniques. One of these techniques is the application of a compound distribution for the aggregates in a portfolio. In this paper, we consider the aggregated loss of Gamma distributed severities and estimate …
This work proposes the Bregman-Tweedie classification model and analyzes the domain structure of the extended exponential function, an extension of the classic generalized exponential function with additional scaling parameter, and related high-level mathematical structures, such as the Bregman-Tweedie loss function an…
We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…
The L1 loss landscape of neural nets near local minima behaves differently, revealing exponential decay and increased vertex density.
We consider the problem of rank loss minimization in the setting of multilabel classification, which is usually tackled by means of convex surrogate losses defined on pairs of labels. Very recently, this approach was put into question by a negative result showing that commonly used pairwise surrogate losses, such as ex…
Classification is the most important process in data analysis. However, due to the inherent non-convex and non-smooth structure of the zero-one loss function of the classification model, various convex surrogate loss functions such as hinge loss, squared hinge loss, logistic loss, and exponential loss are introduced. T…
We consider the problem of option hedging in a market with proportional transaction costs. Since super-replication is very costly in such markets, we replace perfect hedging with an expected loss constraint. Asymptotic analysis for small transactions is used to obtain a tractable model. A general expansion theory is de…
This work extends implicit bias analysis to multiclass classification using a new loss framework.
We analyze systems of agents sharing light-tailed risky claims issued by different financial objects. Assuming exponentially distributed claims, we obtain that both agents' and system's losses follow generalized exponential mixture distributions. We show that this leads to qualitatively different results on individual …
Novel loss functions improve decision tree learning from noisy data.
Researchers improve NCE by addressing its flat loss landscape issues.
A fast method for training linear classifiers maximizes margins.
New bound shows variational algorithms may struggle with barren plateaus.
AEW estimator achieves optimal risk in expectation for large enough temperatures.
We present a generalization of the adversarial linear bandits framework, where the underlying losses are kernel functions (with an associated reproducing kernel Hilbert space) rather than linear functions. We study a version of the exponential weights algorithm and bound its regret in this setting. Under conditions on …
It is known that Boosting can be interpreted as a gradient descent technique to minimize an underlying loss function. Specifically, the underlying loss being minimized by the traditional AdaBoost is the exponential loss, which is proved to be very sensitive to random noise/outliers. Therefore, several Boosting algorith…
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
Unified framework approximates gradient descent's implicit bias in high dimensions.
A standard introduction to online learning might place Online Gradient Descent at its center and then proceed to develop generalizations and extensions like Online Mirror Descent and second-order methods. Here we explore the alternative approach of putting Exponential Weights (EW) first. We show that many standard meth…
New algorithm reduces online logistic regression regret without exponential constant.
Fast classification for sparse models, even with correlated features.
Distributed learning of probabilistic models from multiple data repositories with minimum communication is increasingly important. We study a simple communication-efficient learning framework that first calculates the local maximum likelihood estimates (MLE) based on the data subsets, and then combines the local MLEs t…
Proposes an exponentially increasing step-size for faster parameter estimation in statistical models.
Paper shows SVM can achieve super fast convergence rates.
Proposes a new loss function for robust learning.
Catapult phase in neural nets shows exponential loss growth before quick decrease.
We provide a detailed study on the implicit bias of gradient descent when optimizing loss functions with strictly monotone tails, such as the logistic loss, over separable datasets. We look at two basic questions: (a) what are the conditions on the tail of the loss function under which gradient descent converges in the…
A framework connects VAEs to GLMs for better model initialization and performance.
A new method for robust training under label noise using weighted gradient descent.
Score matching offers efficient estimation for certain distributions.
Robust variable selection for high-dimensional data with missing and measurement errors.
We consider stochastic gradient descent and its averaging variant for binary classification problems in a reproducing kernel Hilbert space. In the traditional analysis using a consistency property of loss functions, it is known that the expected classification error converges more slowly than the expected risk even whe…
New optimization method improves generalization across various tasks.
New loss function improves convergence rate for neural networks.
Spectral risk measures are attractive risk measures as they allow the user to obtain risk measures that reflect their subjective risk-aversion. This paper examines spectral risk measures based on an exponential utility function, and finds that these risk measures have nice intuitive properties. It also discusses how th…
Study tackles non-stationary bandit convex optimization with new algorithms.