Paper tackles catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting leads to sudden drop in robust accuracy.
method Proposes a method to prevent overfitting by using all adversarial examples.
result Demonstrates prevention of catastrophic overfitting and improves robustness.
The paper examines how spike strengths and alignments affect overfitting in linear regression models.
problem The impact of spike strengths and alignments on overfitting in linear regression models.
method Characterization of generalization error through exact expressions and analysis of spike strengths, aspect ratio, and target alignment.
result Increasing spike strength can lead to catastrophic overfitting before benign overfitting, especially in well-specified aligned problems.
New study finds many neural networks are not benignly overfitting.
problem Understanding the behavior of overfitting in neural networks.
method Exploring kernel ridge regression and deep neural networks to identify overfitting behaviors.
result Many interpolating methods, including neural networks, exhibit tempered overfitting rather than benign or catastrophic.
New method prevents adversarial training failure in deep networks.
problem Adversarial training failure in deep networks.
method GradAlign method to prevent catastrophic overfitting.
result GradAlign prevents adversarial training failure in deep networks.
Certified training improves robustness against adversarial attacks.
problem Certified training's gap with empirical robustness limits its practical utility.
method Combining adversarial attacks with network over-approximations.
result Certified training can prevent catastrophic overfitting and bridge the gap to multi-step baselines.
Efficient regularization mitigates catastrophic overfitting in single-step adversarial training.
problem Catastrophic overfitting in single-step adversarial training.
method ELLE regularization term to enforce local linearity of the loss function.
result Our regularization term effectively mitigates catastrophic overfitting without the drawbacks of previous methods.
A new method reduces adversarial training time without overfitting.
problem Catastrophic overfitting in single-step adversarial training.
method FGSMPR: FGSM with PGD Regularization.
result Reduces the gap to multi-step adversarial training.
Study the cost of overfitting in noisy KRR models.
problem Cost of overfitting in noisy kernel ridge regression.
method An agnostic view of overfitting cost as a function of sample size for any target function, using Gaussian universality ansatz and task eigenstructure.
result Characterization of benign, tempered, and catastrophic overfitting.
New bounds for KRR condition number reveal overfitting phenomena.
problem Characterizing overfitting in KRR with varying kernel spectral decay.
method Derived new bounds for kernel matrices, enhanced test error bounds, and identified feature independence role.
result Identified tempered and catastrophic overfitting phenomena.
Catastrophic forgetting continues to severely restrict the learnability of controllers suitable for multiple task environments. Efforts to combat catastrophic forgetting reported in the literature to date have focused on how control systems can be updated more rapidly, hastening their adjustment from good initial setti…
ARCADe detects anomalies in a sequence of tasks with limited data.
problem Learning a sequence of anomaly detection tasks with only normal class examples.
method Formulated as a meta-learning problem, ARCADe addresses catastrophic forgetting and overfitting.
result ARCADe outperforms baselines on three datasets.
We analyze MDL for binary classification, quantifying overfitting and underfitting.
problem Understanding the trade-off between underfitting and overfitting in MDL for binary classification.
method Complete characterization of the regularization curve for MDL, extending previous work to all λ. result Precise quantitative description of the worst case limiting error as a function of λ and noise level. New insights into Nadaraya-Watson interpolators show varied generalization behaviors.
problem Understanding generalization of interpolating predictors, especially in noisy data.
method Revisiting Nadaraya-Watson estimator with a single hyperparameter.
result Multiple overfitting behaviors exist, ranging from catastrophic to tempered.
Framework mitigates overfitting in quantitative trading strategies.
problem Overfitting during strategy transition from backtest to live trading.
method Three-stage protocol: IS, WFA, OOS; majority pass, purge gaps, cliff veto, etc.
result Demonstrates how to detect overfitting through performance decay and drawdown behavior.
Lifelong learning with deep neural networks is well-known to suffer from catastrophic forgetting: the performance on previous tasks drastically degrades when learning a new task. To alleviate this effect, we propose to leverage a large stream of unlabeled data easily obtainable in the wild. In particular, we design a n…
This paper tackles overfitting in CTR models by introducing Multi-Epoch learning with Data Augmentation.
problem Overfitting of the embedding layer in CTR models during multi-epoch training.
method Introduces Multi-Epoch learning with Data Augmentation (MEDA) framework to reduce overfitting and enhance performance.
result MEDA minimizes overfitting and achieves data augmentation through varied embedding spaces, improving performance without overfitting.
This research improves binary classification by balancing overfitting and generalization with a novel Bayesian approach.
problem Improving binary classification models to avoid overfitting and generalize well.
method Introduces a PAC-Bayes type learning rule with a balancing parameter λ to balance training error and KL divergence to a prior.
result A choice of λ ensures uniformly vanishing excess loss, even in the agnostic case, by under-regularizing or over-regularizing appropriately.
Despite all the success that deep neural networks have seen in classifying certain datasets, the challenge of finding optimal solutions that generalize still remains. In this paper, we propose the Boundary Optimizing Network (BON), a new approach to generalization for deep neural networks when used for supervised learn…
TKIL improves class-balanced performance in incremental learning.
problem Catastrophic forgetting in sequential learning tasks.
method Introduces Tangent Kernel for Incremental Learning (TKIL) based on Neural Tangent Kernel (NTK).
result TKIL achieves better overall accuracy and variance across classes.
High-frequency trading models fail due to overfitting and survivor bias.
problem Failure of hybrid DRL-EC trading systems in high-frequency environments.
method Deployed a population of 500 agents in a high-frequency cryptocurrency environment, analyzing failure modes through multi-disciplinary lens.
result Increasing model complexity without information asymmetry exacerbates systemic fragility.
A neural network approach to learn Cusp Catastrophe dynamics.
problem Complex behavior and non-convex parameter space in Cusp Catastrophe models.
method Training a deep neural network to learn dynamics without solving generating parameters.
result Demonstrated a neural network approach for the first time in Cusp Catastrophe models.
This paper describes some of the possibilities of artificial neural networks that open up after solving the problem of catastrophic forgetting. A simple model and reinforcement learning applications of existing methods are also proposed.
The paper values reinsurance contracts for dynamic catastrophe claims without arbitrage.
problem Valuation of reinsurance contracts for dynamic catastrophe claims without arbitrage.
method Compound dynamic contagion process, Esscher transform, Monte Carlo simulation.
result Arbitrage-free premiums for catastrophe stop-loss reinsurance contracts.
Adam optimizer leads to more forgetting in neural networks.
problem Understanding and quantifying catastrophic forgetting in neural networks.
method Comparative analysis of various optimization algorithms and metrics in different learning scenarios.
result Adam optimizer causes more forgetting compared to classical algorithms like SGD.
Bayesian online meta-learning framework tackles catastrophic forgetting in few-shot classification.
problem Catastrophic forgetting in few-shot classification problems.
method Bayesian online learning, meta-learning, Laplace approximation, variational inference.
result Framework effectively achieves goal of overcoming catastrophic forgetting in few-shot classification.
Unified Bayesian framework for CAT bond pricing.
problem Uncertainty in catastrophe occurrences and interest rates in CAT bond markets.
method Bayesian framework based on uncertainty quantification of catastrophes and interest rates.
result Unified asset pricing approach with informative expected risk premia.
Optimizes diversification in catastrophe risk pooling using asymptotic analysis.
problem Maximizing diversification benefit from catastrophic events in insurance pools.
method Asymptotic analysis to solve high-dimensional optimization problem.
result Derives an asymptotically optimal pool that approximates practical optimal pool.
Our method upweights easy samples to mitigate forgetting in fine-tuning.
problem Catastrophic forgetting in fine-tuning pre-trained models.
method Sample weighting based on pre-trained model's losses.
result Our method reduces forgetting by up to 0.8% on MetaMathQA while preserving more accuracy on pre-training datasets.
New methods improve insurance data quality for catastrophic events.
problem Improving precision and size of insurance data for catastrophic events.
method Bootstrap, bootknife, and GAN algorithms.
result Compared MSE and MAE of simulated outputs, direct algorithm for fuzzy expert opinion.
We introduce a random forest approach to enable spreads' prediction in the primary catastrophe bond market. We investigate whether all information provided to investors in the offering circular prior to a new issuance is equally important in predicting its spread. The whole population of non-life catastrophe bonds issu…
This paper develops a two-step estimation methodology, which allows us to apply catastrophe theory to stock market returns with time-varying volatility and model stock market crashes. Utilizing high frequency data, we estimate the daily realized volatility from the returns in the first step and use stochastic cusp cata…
Within the context of the banking-related literature on contingent convertible bonds, we comprehensively formalise the design and features of a relatively new type of insurance-linked security, called a contingent convertible catastrophe bond (CocoCat). We begin with a discussion of its design and compare its relative …
The study examines denoising and noisy-input regression under distribution shift, revealing double descent behavior and insights for data augmentation.
problem Understanding denoising in machine learning, especially under noisy inputs and distribution shift.
method Theoretical analysis of supervised denoising and noisy-input regression, considering low-rank data and proportional regime.
result The test error exhibits double descent under general distribution shift, indicating that overfitting the noise can be benign, tempered, or catastrophic.
Study analyzes catastrophic forgetting in continual learning using teacher-student networks.
problem Catastrophic forgetting in continuously learning systems.
method Teacher-student learning framework, similarity of input distributions and target functions.
result Network can avoid catastrophic forgetting with small input distribution similarity and large target function similarity.
Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining the contribution of individual parameters in an ANN to catastrophic…
Interpreting the behaviors of Deep Neural Networks (usually considered as a black box) is critical especially when they are now being widely adopted over diverse aspects of human life. Taking the advancements from Explainable Artificial Intelligent, this paper proposes a novel technique called Auto DeepVis to dissect c…
The paper introduces CoCoCat bonds for multi-region natural catastrophes, accounting for complex dependencies.
problem Valuation of multi-region contingent convertible bonds under complex dependencies.
method Developed a model accounting for inter-regional dependencies using change-of-measure techniques.
result Significant impact of inter-regional dependencies on CoCoCat bond pricing.
We consider an optimal control problem of a property insurance company with proportional reinsurance strategy. The insurance business brings in catastrophe risk, such as earthquake and flood. The catastrophe risk could be partly reduced by reinsurance. The management of the company controls the reinsurance rate and div…
We propose a model for an insurance loss index and the claims process of a single insurance company holding a fraction of the total number of contracts that captures both ordinary losses and losses due to catastrophes. In this model we price a catastrophe derivative by the method of utility indifference pricing. The as…
Paper tackles catastrophic forgetting in sequential learning.
problem Catastrophic forgetting in sequential learning.
method Regularizes training with sketches of Jacobian matrix of past data.
result Proves overcoming catastrophic forgetting for linear and wide neural networks.
The principal aim of this work is the evidence on empirical way that catastrophic bifurcation breakdowns or transitions, proceeded by flickering phenomenon, are present on notoriously significant and unpredictable financial markets. Overall, in this work we developed various metrics associated with catastrophic bifurca…
This work tackles catastrophic forgetting in neural networks by mimicking brain's metaplasticity.
problem Catastrophic forgetting in neural networks, where new tasks erase previously learned ones.
method Interpreting binarized neural networks as metaplastic systems, adjusting their training technique.
result Training technique reduces catastrophic forgetting without needing previously presented data.
This letter uses the Block Maxima Extreme Value approach to quantify catastrophic risk in international equity markets. Risk measures are generated from a set threshold of the distribution of returns that avoids the pitfall of using absolute returns for markets exhibiting diverging levels of risk. From an application t…
New analysis shows rational actors will deploy AGI despite negative social value due to catastrophic risk.
problem Rational actors will deploy AGI despite negative social value due to shared catastrophic risk.
method Continuous-time preemption game with shared catastrophic externalities, showing suicide region and welfare distortion.
result The suicide region widens as catastrophic risk grows, and two mechanisms can close it.
In this paper, we show that Generative Adversarial Networks (GANs) suffer from catastrophic forgetting even when they are trained to approximate a single target distribution. We show that GAN training is a continual learning problem in which the sequence of changing model distributions is the sequence of tasks to the d…
Synthetic data helps prevent forgetting when learning sequentially.
problem Catastrophic forgetting in neural networks.
method Generate synthetic data via two-step optimisation process using meta-gradients.
result Training on synthetic data prevents forgetting when learning sequentially.
Study optimal dividend strategies for insurers with natural catastrophe claims.
problem Maximizing dividends for a catastrophe insurer over its lifetime.
method Two-dimensional stochastic control problem, viscosity solutions, numerical approximation.
result Optimal dividend strategies identified for natural catastrophe insurers.
Catastrophic forgetting/interference is a critical problem for lifelong learning machines, which impedes the agents from maintaining their previously learned knowledge while learning new tasks. Neural networks, in particular, suffer plenty from the catastrophic forgetting phenomenon. Recently there has been several eff…