This paper introduces systolic geometry and a regularization technique.
problem Understanding the systolic volume of compact Riemannian manifolds.
method Presentation of systolic geometry and regularization technique.
result Linking systolic volume to homotopical invariants like Betti numbers and minimal entropy.
Temporal regularization improves stability in reinforcement learning.
problem High variance in reinforcement learning, especially in high-dimensional domains.
method Temporal regularization based on smoothness in value estimates over trajectories.
result Temporal regularization provides improvement even in high-dimensional Atari games.
Proves regularity for Brakke flow near stationary half-plane.
problem Regularity of Brakke flow near stationary half-planes.
method Viscosity techniques from Savin applied to Brakke's flow.
result Proves C1,α regularity for flows close to stationary half-planes. Regularization improves policy optimization in RL, especially on harder tasks.
problem Lack of conventional regularization in RL methods.
method Comprehensive study of regularization techniques on policy networks with multiple RL algorithms.
result Conventional regularization techniques significantly improve policy optimization, especially on harder tasks.
This paper analyzes Stochastic Depth regularization in ResNets.
problem Improving generalization in ResNets through regularization.
method Hybrid analysis combining perturbation and signal propagation.
result Principled guidelines for choosing survival rates in SD.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
We study the problem of supervised learning for both binary and multiclass classification from a unified geometric perspective. In particular, we propose a geometric regularization technique to find the submanifold corresponding to a robust estimator of the class probability P(y∣x). The regularization term meas…
Establish C^{1,2} regularity of American value functions in Heston model
problem Regularity of American put options in Heston model
method PDE techniques
result C^{1,2} regularity in exercise domain and smooth-fit principle
New method encodes function preferences into neural nets for better generalization.
problem Challenges in encoding explicit function preferences in neural network training.
method Function-space empirical Bayes (FSEB) regularization.
result FSEB leads to near-perfect semantic shift detection and improved generalization.
New regularization techniques improve stability of deep neural networks.
problem Improving stability of deep neural networks in high-dimensional data.
method Apply manifold regularization to develop new regularizers based on graph Laplacian sparsification.
result Empirically, networks achieve high stability in various perturbation models, including adversarial attacks.
Adapts RL regularization techniques to prevent overfitting.
problem Preventing reinforcement learning agents from overfitting to limited training environments.
method Selective Noise Injection (SNI) and Information Bottleneck (IB) techniques.
result Significantly improved generalization performance on Coinrun benchmark.
Exact spectral norm regularization improves neural network generalization.
problem Improving neural network generalization while protecting against noise.
method Exact spectral norm regularization of the Jacobian.
result Improved generalization performance compared to previous methods.
Proposes a new regularization technique for neural networks using elliptic operators.
problem Improving model behavior in underrepresented data regions.
method Modifies the empirical risk minimization objective to minimize an elliptic operator over the data domain.
result The proposed regularization technique anticipates error behavior outside the training set using existing elliptic operator theory.
New method tackles reinforcement learning of complex ω-regular objectives without models.
problem Learning ω-regular objectives in unknown MDPs.
method Constructive reduction to almost-sure reachability, compilation to limit-deterministic Buechi automata.
result Optimal strategies computed from MDP observations using reinforcement learning.
Regularization improves spectral embedding by focusing on the largest blocks.
problem Improving the quality of spectral embedding for graph data.
method Explained the impact of complete graph regularization on spectral embedding of a block model.
result Regularization forces spectral embedding to focus on the largest blocks, making it less sensitive to noise or outliers.
Study MinMax methods for optimization problems, including optimal transport.
problem Optimization problems, especially optimal transport.
method MinMax framework, regularization, neural networks, approximation theorems.
result Justification of neural networks for solving optimization problems.
A new training method for GANs improves image generation quality.
problem Training GANs is challenging and unstable.
method Consistency regularization to stabilize GAN training.
result CR-GAN achieves best FID scores for unconditional image generation.
Hybrid bootstrap improves model performance over dropout.
problem Improving predictive model performance through regularization.
method Resamples features from other training points instead of replacing them with zeros.
result Offers superior performance compared to dropout.
Proves and tests methods for learning time-series with breaks.
problem Learning time-series with structural breaks.
method Complete proofs and experimental validation of a regularized loss function.
result Experimental results support the validity of the techniques.
Unified techniques improve stability and replicability in changing data.
problem Concept drift in data generating distribution.
method Removing hidden confounding and causal regularization.
result Improves stability, replicability, and robustness in heterogeneous data.
Dropout technique is analyzed using information geometry.
problem Understanding the regularization performance of dropout in neural networks.
method Unified analysis from information geometry viewpoint.
result Dropout flattens the model manifold and its performance depends on curvature.
We extend the validity of the Penrose singularity theorem to spacetime metrics of regularity C1,1. The proof is based on regularisation techniques, combined with recent results in low regularity causality theory.
Proposes a new method to improve deep neural network performance.
problem Improving generalization in deep neural networks on unseen data.
method Post-synaptic potential regularization (PSP).
result PSP achieves comparable performance to sophisticated learning strategies and improves generalization over ℓ2 regularization. Mixout technique improves finetuning of large pretrained models on few instances.
problem Degenerate performance in finetuning large pretrained models with limited training data.
method Mixout technique, inspired by dropout, stochastically mixes model parameters.
result Stability and accuracy of finetuning improve significantly with mixout.
The paper studies how different entropic regularizations affect GAN solutions.
problem Improving numerical convergence and sparsity in GAN solutions.
method Entropic regularization of Wasserstein distance and Sinkhorn divergence.
result Entropy regularization promotes sparsity, while Sinkhorn divergence recovers unregularized solution.
New regularization techniques using mass transportation for better generalization.
problem Mitigating overfitting with scarce data.
method Distributionally robust optimization and worst-case expected loss.
result Generalization bounds and tractable learning problems.
New technique prevents Q-learning collapse by maximizing diversity among ensembles.
problem Value function collapse in ensemble Q-learning.
method Maximizing representation diversity through regularization.
result Regularized approach significantly outperforms existing methods.
Two regularization techniques improve GCNN explainability and preference from chemists.
problem Difficulty in rationalizing molecular graph neural network predictions.
method Batch Representation Orthonormalization (BRO) and Gini regularization applied during GCNN training.
result Regularization improves GCNN attribution methods and preference from chemists.
Re-initializing neural networks improves generalization but not as much as other techniques.
problem Understanding when and how re-initialization improves neural network performance.
method Empirical comparison of re-initialization with standard training and various regularization techniques.
result Re-initialization is beneficial for generalization but not as much as other techniques, especially when combined with careful tuning of hyperparameters.
Optimal model improves AUC, recall, and F1 score for class-imbalanced business risk.
problem Improving prediction of class-imbalanced business risk.
method Resampling, regularization, and model ensembling techniques.
result Boosting on DT with SMOTE oversampling achieves AUC, recall, and F1 score of 0.8633, 0.9260, and 0.8907, respectively.
New measure shows various training techniques control model complexity.
problem Understanding how to control model complexity in deep learning.
method Developed geometric complexity measure and demonstrated its effectiveness.
result Many training techniques control geometric complexity, providing a unified framework.
Paper optimizes Laplacian regularization for sparse network clustering.
problem Improving spectral clustering in sparse networks.
method Formally determines optimal Laplacian regularization.
result Proper regularization is closely tied to state-of-the-art techniques.
New technique improves deep network performance by selectively withholding data.
problem Ensuring deep networks generalize well to new inputs.
method Randomly withhold a part of the data in an arbitrary basis.
result Generalized Dropout yields notable performance gains.
Regularization and data augmentation can be class-dependent, leading to poor performance on some classes.
problem Class-dependent effects of regularization and data augmentation.
method Evaluation of regularization and data augmentation techniques on Imagenet and INaturalist datasets.
result Regularization and data augmentation can lead to significant performance drops on some classes.
We establish a regularity theorem for the Harmonic - Einstein Equation. As a byproduct of the local regularity, we also have a compactness theorem on Harmonic - Einstein equation. The method is mainly the Moser iteration technique which has been used and developed by \cite{BKN89}, \cite{Tian90}, \cite{TV05a} and others…
This paper introduces NPR, a technique to improve Bayesian inference for multi-modal, high-dimensional simulations.
problem Challenges in Bayesian inference for multi-modal, high-dimensional simulations.
method Introduces Neural Posterior Regularization (NPR) to enforce exploration of input parameter space.
result Empirically validated that NPR significantly improves performance on various simulation tasks.
Adaptive label regularization improves neural network performance.
problem Improving deep neural network performance through label regularization.
method Proposes an adaptive label regularization method that updates label representation online.
result Significant improvement in image recognition and text classification tasks.
New technique debiases distributed optimization, improving convergence rate.
problem Bias in local estimates limits effectiveness of distributed second order optimization.
method Surrogate sketching and scaled regularization to eliminate bias.
result The debiased local estimates lead to faster convergence in distributed optimization.
Heavy-tailed regularization improves deep neural network performance.
problem Improving generalization of deep neural networks.
method Introducing Heavy-Tailed Regularization, using differentiable penalty terms and Bayesian statistics.
result Heavy-tailed regularization outperforms conventional regularization techniques.
Bayesian CRM improves offline learning from logged bandit data.
problem Offline learning from logged bandit feedback.
method PAC-Bayesian analysis for a new generalization bound, novel regularization technique.
result New technique outperforms standard L2 regularization and is competitive with variance regularization. Novel regularization for Vision Transformers improves model generalization and sparsity.
problem Improving generalization and sparsity in Vision Transformers.
method Likelihood-guided variational Ising-based regularization.
result Improved generalization and sparsity in Vision Transformers.
Paper compares three regularization-based methods for HAR, highlighting their strengths and limitations.
problem Challenges in evolving ML models for dynamic health and well-being applications.
method Evaluation of three regularization-based continual learning approaches for Human Activity Recognition (HAR).
result No single technique outperformed all others in all scenarios considered.
Improved LDA using a nonlinear covariance estimator for better performance.
problem Inefficient LDA when data covariance is ill-conditioned.
method Regularized LDA with a positive semidefinite ridge-type estimator of the inverse covariance matrix.
result The proposed NL-RLDA classifier outperforms state-of-the-art methods across multiple datasets.
The paper examines how adversarial training and noise affect neural network performance.
problem Overfitting in adversarial training and data augmentation.
method Adversarial training and data augmentation with noise in the context of regularized regression in RKHS.
result Appropriate regularization can prevent overfitting and improve performance.
Hidden regular variation is a sub-model of multivariate regular variation and facilitates accurate estimation of joint tail probabilities. We generalize the model of hidden regular variation to what we call hidden domain of attraction. We exhibit examples that illustrate the need for a more general model and discuss de…
New findings suggest adversarial training does not flatten loss landscapes, challenging current intuition.
problem Understanding and improving generalization in deep learning.
method Loss surface visualization with filter normalization technique.
result Adversarial training does not result in flatter loss landscapes, challenging current intuition.
A novel approach to quantizing neural networks using periodic functions as regularizers.
problem Quantization of neural network parameters to reduce memory usage and computational cost.
method Using periodic functions (sine, cosine, hat) as regularizers during training to push weights into discrete points.
result Quantized models achieve the same accuracy as original models on CIFAR-10 and ImageNet datasets.
A new method sparsifies neural networks by reducing sensitive parameters to zero.
problem Challenges of memory-limited applications due to large number of neural network parameters.
method Quantifies output sensitivity, introduces a regularization term to gradually reduce sensitive parameters.
result Surpasses most recent techniques in sparsity and error rates, achieving twice the sparsity at equal error rates in some cases.