Paper proposes algorithms for active learning of reject option classifiers.
problem Active learning of reject option classifiers is unaddressed in machine learning.
method Developed novel algorithms using double ramp and double sigmoid loss functions.
result Proposed algorithms efficiently reduce the number of labeled examples required.
Novel ramp loss method improves weakly supervised machine translation and parsing.
problem Training neural models without gold labels in weak supervision scenarios.
method Adapted ramp loss objectives to promote positive outputs and discourage negative ones.
result Bipolar ramp loss objectives outperform other methods on weakly supervised tasks.
Seesaw optimizes training by balancing learning rate and batch size, accelerating model pretraining.
problem Optimizing training efficiency for large language models with adaptive optimizers.
method Develops a principled framework for batch-size scheduling, introducing Seesaw which multiplies learning rate by 1/√2 and doubles batch size.
result Empirically, Seesaw reduces wall-clock time by approximately 36% compared to cosine decay, matching theoretical limits.
Study improves adversarial classification using distributionally robust models.
problem Improving robustness against adversarial attacks in classification models.
method Distributionally robust chance constraints with Wasserstein ambiguity, reformulated as a regularized ramp loss minimization problem.
result Standard descent methods can converge to the global minimizer for the distributionally robust adversarial classification model.
This paper characterizes and designs loss functions for robust classification with abstention.
problem Ensuring robustness against adversarial attacks and knowing when to abstain from prediction.
method Proposes adversarial robust reject option loss and characterizes surrogates for calibration.
result Shifted Double Ramp Loss and Shifted Double Sigmoid Loss satisfy the calibration conditions.
Study develops a machine learning-based ramp metering model to improve freeway efficiency.
problem Improving ramp metering to maintain freeway efficiency under various traffic conditions.
method Machine learning approach using historical data to predict and manage traffic flow.
result The novel model outperforms a baseline traffic-responsive ramp metering algorithm.
Modeling solar ramping events with spatio-temporal point processes.
problem Predicting solar ramping events influenced by weather conditions.
method Novel spatio-temporal categorical point process model.
result Effective modeling of spatio-temporal correlations in solar ramping events.
We use surrogate losses to obtain several new regret bounds and new algorithms for contextual bandit learning. Using the ramp loss, we derive new margin-based regret bounds in terms of standard sequential complexity measures of a benchmark class of real-valued regression functions. Using the hinge loss, we derive an ef…
In this paper we show all possible ramps where an object can move with constant speed under the effect of gravity and friction. The planar ramp are very easy to describe, just rotate a curve with velocity vector (tanh(as),sech(as)). Recall that tanh(as)^2+sech^2(as) = 1. Therefore, the solution of the planar constant s…
Paper establishes generalization bounds for RNNs and improves existing results.
problem Theoretical understanding and generalization bounds for RNNs.
method New generalization error bound and Rademacher complexity calculation.
result Improved generalization bounds for RNNs, tighter than existing bounds.
The paper finds that circles and logarithmic spirals are the only constant-speed ramps for a specific force field.
problem Determining planar curves for constant-speed motion under specific force conditions.
method Analyzing the motion of a particle under friction and a central force field.
result Every solution to the constant-speed motion problem approaches either a circle or a logarithmic spiral.
Traditional plane-based clustering methods measure the cost of within-cluster and between-cluster by quadratic, linear or some other unbounded functions, which may amplify the impact of cost. This letter introduces a ramp cost function into the plane-based clustering to propose a new clustering method, called ramp-base…
Here we propose a general theoretical method for analyzing the risk bound in the presence of adversaries. Specifically, we try to fit the adversarial learning problem into the minimax framework. We first show that the original adversarial learning problem can be reduced to a minimax statistical learning problem by intr…
In this paper we propose a tractable quadratic programming formulation for calculating the equilibrium term structure of electricity prices. We rely on a theoretical model described in [21], but extend it so that it reflects actually traded electricity contracts, transaction costs and liquidity considerations. Our nume…
We analyze double descent in finite-width neural networks using influence functions.
problem Understanding double descent in finite-width neural networks.
method Using influence functions to derive population loss bounds and investigate loss function effects.
result Derived bounds exhibit double descent behavior at the interpolation threshold.
Automates detection of fast-ramped flexibility events for DSOs.
problem Monitoring and supervising flexibility activations in power systems.
method Unsupervised detection and open-set classification.
result Automatically identifies critical flexibility activations for early intervention.
Automates phased release strategy to balance risk and speed.
problem Balancing risk and speed in phased product releases.
method Formalizes as constrained batched bandit problem, uses adaptive Bayesian approach.
result Proposes algorithm that determines optimal release percentages.
Analyzes double descent in binary classification models with different losses.
problem Understanding the double descent phenomenon in binary classification models.
method Analytic study of gradient descent with logistic and square losses on binary linear classification models.
result The double descent phenomenon persists but with differences compared to logistic loss.
The support vector machine (SVM) is one of the most successful learning methods for solving classification problems. Despite its popularity, SVM has a serious drawback, that is sensitivity to outliers in training samples. The penalty on misclassification is defined by a convex loss called the hinge loss, and the unboun…
SAT improves adversarial training by smoothing the loss landscape through curriculum learning.
problem Adversarial training sacrifices clean accuracy for robustness and suffers from large generalization error.
method SAT uses curriculum learning to smooth the adversarial loss landscape, improving both clean and robust accuracy.
result SAT models improve clean and robust accuracy significantly compared to adversarial training and other baselines.
Structured learning is appropriate when predicting structured outputs such as trees, graphs, or sequences. Most prior work requires the training set to consist of complete trees, graphs or sequences. Specifying such detailed ground truth can be tedious or infeasible for large outputs. Our main contribution is a large m…
Study investigates overparametrization in survival models, revealing complex loss behavior.
problem Understanding overparametrization in survival models through interpolation.
method Defined interpolation and finite-norm interpolation, rigorously analyzed four survival models.
result Overparametrization can lead to improved performance in survival models, contrary to classical learning theory.
Generalizes double backpropagation for neural networks in Hilbert space.
problem Lack of a general description of derivatives in neural network loss functions.
method Develops a Hilbert space framework to cover various special cases of double backpropagation.
result Optimized backpropagation rules for real-world scenarios, reducing calculations by up to 33%.
Study predicts doubling of U.S. maize insurance claims due to climate change.
problem Climate change increases U.S. maize loss probability, impacting insurance claims.
method Neural Network Monte Carlo simulations to predict crop loss metrics.
result Doubling of annual probability of maize Yield Protection insurance claims by mid-century.
Deep networks can interpolate noisy data without losing generalization.
problem Characterizing the relationship between interpolation and generalization in overparameterized deep networks.
method Analyzing the loss landscape of neural network functions over volumes around training data points, varying model parameters and training epochs.
result Loss sharpness in the input space follows a double descent, with large models predicting noisy targets over larger volumes around training data points.
Project promoters, forecasters, and managers sometimes object to two things in measuring inaccuracy in travel demand forecasting: (1) using the forecast made at the time of making the decision to build as the basis for measuring inaccuracy and (2) using traffic during the first year of operations as the basis for measu…
Paper develops a duality approach for robust loss functions in infinite-dimensional RKHSs.
problem Robustness issues in infinite-dimensional RKHSs with operator-valued kernels.
method Develops a duality approach to solve OVK machines for various loss functions.
result Empirical improvements and theoretical stability analysis for robust structured data applications.
Double descent phenomenon explained in simple terms.
problem Understanding the surprising drop in test error in overparameterized models.
method Informal explanation using linear algebra and probability, visual intuition with polynomial regression, mathematical analysis with ordinary linear regression.
result Three factors create double descent: data undersampling, model size, and parameter count. Ablating any one of these factors prevents double descent.
Model shows loss curve with two distinct exponents due to sparse activations.
problem Sparse activations impact neural network scaling laws.
method Introduced a model for neural scaling laws under sparse activations, derived asymptotic population loss, and analyzed gradient-descent dynamics.
result Loss curve exhibits double-descent peak near interpolation threshold with two distinct scaling exponents.
It has been experimentally observed in recent years that multi-layer artificial neural networks have a surprising ability to generalize, even when trained with far more parameters than observations. Is there a theoretical basis for this? The best available bounds on their metric entropy and associated complexity measur…
Study on double descent behavior in two-layer neural networks for binary classification.
problem Understanding the double descent phenomenon in model test error.
method Two-layer neural network with ReLU activation for binary classification. Quantified model size by sample-to-dimension ratio. Empirical risk minimization using Convex Gaussian Min Max Theorem.
result Observed and investigated the double descent behavior of model test error.
Extends double linear policy with time-varying weights and proves robust positive expectation.
problem Ensuring robustness in policy optimization with time-varying parameters.
method Employed a novel elementary symmetric polynomials characterization approach to prove robust positive expectation (RPE). Derived explicit expressions for expected cumulative gain-loss and variance.
result Proved the robust positive expectation property holds for the extended double linear policy.
Efficient regularization mitigates catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting in single-step adversarial training.
method ELLE regularization term to enforce local linearity of the loss function.
result Our regularization term effectively mitigates catastrophic overfitting without the drawbacks of previous methods.
Let f⋆ be a function on Rd with an assumption of a spectral norm vf⋆. For various noise settings, we show that E∥f^−f⋆∥2≤(vf⋆4nlogd)1/3, where n is the sample size and f^ is either a penalized lea…
This paper explains why double descent sometimes occurs weakly or not at all from an optimization perspective.
problem Understanding the role of optimization in the phenomenon of double descent.
method Investigates model-wise double descent from an optimization perspective, proposing a unified explanation for its occurrence.
result Model-wise double descent is observed if and only if the optimizer can find a sufficiently low-loss minimum.
Improves statistical learning bounds with self-concordant losses.
problem Statistical prediction with nuisance components.
method Orthogonal statistical learning with self-concordant loss.
result Non-asymptotic bounds on excess risk improved by a dimension factor.
Two new algorithms improve Q* approximation in batch RL with linear error propagation.
problem Improving Q* approximation in batch reinforcement learning.
method Two novel algorithms that estimate Bellman error directly, without quadratic dependence.
result Linear-in-horizon error propagation for batch RL algorithms.
LMC loss barrier decreases to zero with large network width.
problem Understanding the LMC phenomenon in neural networks.
method Fine-grained analysis of LMC for two-layer ReLU networks.
result LMC loss barrier decreases to zero at a rate of O(m^-1/2) for large network width.
Study on RF regression with SGD shows double descent phenomenon.
problem Understanding generalization in RF models trained with SGD.
method Precise non-asymptotic error bounds derived for RF regression under constant and polynomial-decay step-size SGD.
result RF regression generalizes well for interpolation learning and exhibits double descent behavior.
This work reveals how label noise can cause a final ascent in neural network performance curves.
problem The impact of label noise on the performance of neural networks.
method Theoretical analysis and extensive experiments on various neural network architectures.
result Label noise can lead to a final ascent in the test loss curve, improving generalization at intermediate model widths.
More training data can hurt the generalization of adversarially robust models.
problem The challenge of balancing adversarial robustness and generalization in machine learning models.
method Investigation of three regimes based on adversary strength and empirical studies on various models.
result More training data can hurt the generalization of adversarially robust models in different regimes.
Double descent found in DRL, improving generalization with model capacity.
problem Generalization in over-parameterized DRL models.
method Actor-Critic framework, Policy Entropy metric.
result Policy Entropy significantly reduces as model capacity increases, indicating improved generalization.
The study analyzes a model for aggregate losses with dependent and overdispersed inter-losses times.
problem Analyzing aggregate loss models with dependent and overdispersed inter-losses times.
method The study uses a two-state Markovian arrival process (MAP2) and a Markov renewal process to model the inter-losses times. Severities are modeled using a heavy-tailed, double-Pareto Lognormal distribution. The model is estimated via direct maximization of the likelihood function.
result The model with dependence and overdispersion in inter-losses times leads to higher capital charges compared to a Poisson process.
We study dynamic hedging of counterparty risk for a portfolio of credit derivatives. Our empirically driven credit model consists of interacting default intensities which ramp up and then decay after the occurrence of credit events. Using the Galtchouk-Kunita-Watanabe decomposition of the counterparty risk price paymen…
Improves neural network performance with double regularization.
problem Challenges of mislabeled data in neural networks.
method Double regularization combining model complexity and observation reweighting.
result Stronger robustness and generalization against mislabeling.
Deep learning models can overfit noisy data without losing generalization.
problem Understanding the generalization of deep learning models in noisy data.
method Empirical investigation of epoch-wise double descent in fully connected neural networks trained on CIFAR-10 with 30% label noise.
result The model achieves strong re-generalization on test data after overfitting noisy training data, corresponding to a 'benign overfitting' state.
Motivated by the industry practice of pairs trading, we study the optimal timing strategies for trading a mean-reverting price spread. An optimal double stopping problem is formulated to analyze the timing to start and subsequently liquidate the position subject to transaction costs. Modeling the price spread by an Orn…
Multi-label classification is a type of supervised learning where an instance may belong to multiple labels simultaneously. Predicting each label independently has been criticized for not exploiting any correlation between labels. In this paper we propose a novel approach, Nearest Labelset using Double Distances (NLDD)…