Enhances linear regression with Kalman filter for loss minimization.
problem Minimizing loss in linear regression models.
method Integrates Kalman filter and SGD for optimal weight updates.
result Develops optimal linear regression equation with minimum area under curve.
The paper presents a multi-power law for predicting loss curves across different learning rate schedules.
problem Understanding and optimizing the relationship between model performance and hyperparameters, especially learning rates.
method Proposes a multi-power law that combines power laws based on the sum of learning rates and additional laws for loss reduction due to decay.
result The multi-power law accurately predicts loss curves for unseen learning rate schedules and finds a schedule that outperforms cosine learning rate.
Comparison of decision curve analysis and cost curves for model evaluation.
problem Evaluating classification performance across different operating contexts.
method Comparison of Decision Curve Analysis (DCA) and Cost Curves.
result DCA and Cost Curves are closely related, with Brier curves being more generally applicable.
LoRA-Curve connects independent LoRA optima through continuous low-loss valleys, improving Bayesian model averaging.
problem Challenges in estimating epistemic uncertainty in LoRA-based Bayesian inference.
method Introduces LoRA-Curve, a segmented Bézier curve parameterization in the LoRA space, with free and anchored configurations.
result Empirically shows that connecting independent LoRA optima through continuous low-loss valleys improves mutual information of the predictive distribution.
The paper predicts loss scaling across different datasets and compute scales.
problem Predicting loss scaling across different datasets and compute scales.
method Derive shifted power law relationships between train and test losses.
result Shifted power law relationships hold for various datasets and tasks, improving prediction accuracy.
Efficient algorithm for evaluating hierarchical classification methods at multiple operating points.
problem Evaluating hierarchical classification methods at multiple operating points.
method Efficient algorithm to produce operating characteristic curves for any method that assigns scores to every class in the hierarchy.
result Top-down classifiers are dominated by a naive flat softmax classifier across the entire operating range.
WES improves neural network regression by stretching distribution error.
problem Improving prediction performance in neural-network-based regression.
method Proposed weighted empirical stretching (WES) loss function.
result WES outperforms existing loss functions, especially in extreme domains.
Study predicts SGD test loss for structured features.
problem Understanding test loss dynamics in SGD for structured data.
method Solveable model of SGD on mean square loss for arbitrary covariance structure.
result Simpler Gaussian model accurately predicts test loss of nonlinear models.
In the paper, we use and investigate copulas models to represent multivariate dependence in financial time series. We propose the algorithm of risk measure computation using copula models. Using the optimal mean-CVaR portfolio we compute portfolio's Profit and Loss series and corresponded risk measures curves. Value-…
Building upon recent advances in entropy-regularized optimal transport, and upon Fenchel duality between measures and continuous functions , we propose a generalization of the logistic loss that incorporates a metric or cost between classes. Unlike previous attempts to use optimal transport distances for learning, our …
New model predicts neural network performance from early training epochs, incorporating architecture impact.
problem Predicting neural network performance from early training epochs, neglecting architecture impact.
method Architecture-aware graph ordinary differential equation model.
result Model outperforms state-of-the-art methods for MLP and CNN learning curves.
We analyze impermanent loss in AMMs and show G3Ms are simplest.
problem Understanding impermanent loss in automated market makers.
method Developed a general framework and analyzed Geometric Mean Market Makers (G3Ms).
result G3Ms have the simplest impermanent loss characteristics.
New method saves computational budget by ranking and transferring learning curves.
problem Expensive automated machine learning methods for hyperparameter and neural architecture optimization.
method Tackles as a ranking and transfer learning problem, optimizing a pairwise ranking loss and leveraging learning curves from other datasets.
result Accelerates neural architecture search by a factor of up to 100 without significant performance degradation.
Calibrating classifiers reduces grouping loss using sufficiency criteria.
problem Grouping loss in probabilistic classifier calibration is often overlooked.
method Revisited Langford & Zadrozny's probing reduction approach and introduced Brier curves.
result The probing reduction approach reduces grouping loss and supports sufficient calibration.
Analyzes double descent in binary classification models with different losses.
problem Understanding the double descent phenomenon in binary classification models.
method Analytic study of gradient descent with logistic and square losses on binary linear classification models.
result The double descent phenomenon persists but with differences compared to logistic loss.
Investment horizon approach has been used to analyze indexes of Polish stock market.Optimal time horizon for each return value is evaluated by fitting appropriate function form of the distribution. Strong asymmetry of gain-loss curves is observed for WIG index, whereas gain and loss curves look similar for WIG20 and fo…
Paper tackles temporal overfitting in wind power curve modeling.
problem Temporal overfitting in wind power curve modeling.
method Proposes a Gaussian process-based method to partition and model time-invariant and time-varying components.
result Significant improvement in predicting responses for different time periods.
Proposes neuron alignment to optimize mode connectivity in neural networks.
problem Understanding and optimizing mode connectivity in deep neural networks.
method Introduces neuron alignment to approximate optimal weight permutations and improve mode connectivity.
result Neuron alignment significantly alleviates robust loss barriers and improves model robustness and accuracy.
Bayesian method corrects bias in imbalanced datasets.
problem Prevalence bias in machine learning datasets.
method Bayesian risk minimization framework, bias-corrected loss function.
result Corrected loss function improves model performance.
In most machine learning applications, classification accuracy is not the primary metric of interest. Binary classifiers which face class imbalance are often evaluated by the Fβ score, area under the precision-recall curve, Precision at K, and more. The maximization of many of these metrics can be expressed as a con…
New optimization method improves AUC for binary classification and changepoint detection.
problem Non-convex AUC and sub-optimal points in ROC curves.
method AUM (Area Under Min(FP, FN)) surrogate loss function based on sorting and summing ROC curve points.
result AUM minimization learning algorithm improves AUC and speeds up compared to previous methods.
Study proposes deep learning for VWAP execution in crypto markets, outperforming traditional methods.
problem Challenges in achieving VWAP due to dynamic volume and price factors.
method Direct optimization of VWAP execution using deep learning, bypassing volume curve prediction.
result Deep learning approach consistently achieves lower VWAP slippage in volatile markets.
New methods evaluate data representations by complexity of low-loss predictor learning.
problem Evaluating quality of data representations for downstream tasks.
method Surplus Description Length (SDL) and ε Sample Complexity (εSC) methods.
result Methods measure the information needed to approximate optimal predictor up to specified tolerance.
Adaptive market maker curves minimize arbitrage losses in DeFi.
problem Asset trading prices in AMMs trail behind centralized exchanges, causing LP losses.
method Adapts market maker bonding curves to trader behavior using a differential equation derived from the Glosten-Milgrom model.
result Optimal adaptive curves minimize arbitrage losses while remaining competitive.
Separable losses are inconsistent for structured prediction models.
problem Inconsistency of separable losses in structured prediction models.
method Analysis of separable negative log-likelihood losses for structured prediction.
result Separable losses are not Bayes consistent and may not predict the most probable structure.
Burq-Gérard-Tzvetkov and Hu established Lp estimates (2≤p≤∞) for the restriction of eigenfunctions to submanifolds. The estimates are sharp, except for the log loss at the endpoint L2 estimates for submanifolds of codimension 2. It has long been believed that the log loss at the endpoint can be remov…
Symmetric losses improve classifier robustness from corrupted labels.
problem Improving classifier performance from corrupted labels.
method Symmetric losses that satisfy a certain condition.
result Symmetric losses enhance robust classification from corrupted labels.
Traditionally, most of the existing attribute learning methods are trained based on the consensus of annotations aggregated from a limited number of annotators. However, the consensus might fail in settings, especially when a wide spectrum of annotators with different interests and comprehension about the attribute wor…
Benguria and Loss have conjectured that, amongst all smooth closed curves of length 2π in the plane, the lowest possible eigenvalue of the operator L=−Δ+κ2 was one. They observed that this value was achieved on a two-parameter family, O, of geometrically distinct ovals containing the round circle and c…
Convolutional neural networks predict the analytic rank of elliptic curves accurately.
problem Predicting the analytic rank of elliptic curves over Q.
method Applied one-dimensional convolutional neural networks to Frobenius traces.
result High accuracy predictions for analytic rank across various conductors.
Improved loss scaling for stochastic momentum algorithms in high dimensions.
problem Improving loss scaling for stochastic momentum algorithms in high dimensions.
method Dimension-adapted Nesterov acceleration (DANA) scales momentum hyperparameters based on model size and data complexity.
result DANA improves loss scaling exponents across various data and target complexities.
Transformers exhibit abrupt learning in matrix completion tasks.
problem Understanding abrupt learning in Transformers for matrix completion.
method Formulated matrix completion as MLM task, trained BERT model, analyzed model components.
result Sudden drop in loss despite no changes in training procedure or hyper-parameters.
To investigate whether training load monitoring data could be used to predict injuries in elite Australian football players, data were collected from elite athletes over 3 seasons at an Australian football club. Loads were quantified using GPS devices, accelerometers and player perceived exertion ratings. Absolute and …
Locus scores predictions for risk, reducing large-loss events.
problem Deployment cost from inaccurate predictions, especially large losses.
method Distribution-free loss-scale reliability score using any predictive distribution.
result Reduces large-loss frequency compared to standard heuristics.
The predictive quality of machine learning models is typically measured in terms of their (approximate) expected prediction accuracy or the so-called Area Under the Curve (AUC). Minimizing the reciprocals of these measures are the goals of supervised learning. However, when the models are constructed by the means of em…
New method improves fairness of facial recognition systems.
problem Facial recognition systems exhibit bias across different demographic groups.
method Optimizes centroid-based scores to reduce bias in pre-trained models.
result Demonstrates significant improvement in fairness with minimal loss in accuracy.
The paper proves learning-curve monotonicity for maximum likelihood estimators in various parametric settings.
problem Establishing monotonicity guarantees for maximum likelihood estimators.
method Variants of GPT-5.2 Pro were used to derive the results.
result The paper proves monotonicity for maximum likelihood estimators in Gaussian and Gamma variables.
Many real-world analytics problems involve two significant challenges: prediction and optimization. Due to the typically complex nature of each challenge, the standard paradigm is predict-then-optimize. By and large, machine learning tools are intended to minimize prediction error and do not account for how the predict…
A new framework learns differentiable structured losses from data.
problem Learning effective losses for complex structured prediction tasks.
method Contrastive learning to learn differentiable structured losses from output data.
result Achieves similar or better performance than kernel-based methods.
Flexible framework for bounding high-loss predictions using quantiles.
problem Need for rigorous guarantees in risk-sensitive applications.
method Order statistics of loss values, flexible quantile-based metrics.
result Ability to rigorously control loss quantiles on real-world datasets.
We unify f-divergences, Bregman divergences, surrogate loss bounds (regret bounds), proper scoring rules, matching losses, cost curves, ROC-curves and information. We do this by systematically studying integral and variational representations of these objects and in so doing identify their primitives which all are rela…
This paper improves risk bounds and calibration for smart predict-then-optimize method.
problem Improving risk bounds and calibration for smart predict-then-optimize method.
method Develops risk bounds and uniform calibration results for the SPO+ loss relative to the SPO loss.
result Empirical minimizer of the SPO+ loss achieves low excess true risk with high probability.
Decision trees improve decision-making by optimizing predictions of unknown parameters.
problem Optimizing decisions based on predicted unknown parameters.
method SPO Trees (SPOTs) for training decision trees under the SPO loss function.
result SPOTs provide higher quality decisions and significantly lower model complexity compared to other machine learning approaches.
Novel metrics improve machine learning models for ICU patient care.
problem Predicting vital sign trajectories for early detection of adverse events.
method Developed novel performance metrics aligned with clinical contexts, validated on simulated and real datasets, and optimized neural networks using these metrics.
result Neural networks trained with these metrics excel in predicting clinically significant events.
Deep networks can interpolate noisy data without losing generalization.
problem Characterizing the relationship between interpolation and generalization in overparameterized deep networks.
method Analyzing the loss landscape of neural network functions over volumes around training data points, varying model parameters and training epochs.
result Loss sharpness in the input space follows a double descent, with large models predicting noisy targets over larger volumes around training data points.
New ROC tools assess predictive abilities for any linearly ordered outcomes.
problem Fundamental restriction in ROC analysis for non-dichotomous outcomes.
method ROC movies and UROC curves for linearly ordered outcomes.
result CPA equals AUC for binary outcomes and relates to Spearman's coefficient for pairwise distinct outcomes.
Interpretable companion model for black-box classifiers.
problem Dilemma between interpretable and black-box models.
method Trains a companion model from data and black-box model predictions, optimizing a combination of accuracy and complexity.
result Companion model provides interpretable predictions with a slight accuracy loss for user choice.
Learning with non-modular losses is an important problem when sets of predictions are made simultaneously. The main tools for constructing convex surrogate loss functions for set prediction are margin rescaling and slack rescaling. In this work, we show that these strategies lead to tight convex surrogates iff the unde…