Improved Kriging model reduces prediction errors.
problem Improving prediction accuracy in Kriging models.
method Theta-regularized Kriging model with Lasso, Ridge, and Elastic-net penalties.
result The Theta-regularized Kriging model outperforms other penalized Kriging models in accuracy and stability.
Gradient descent implicitly regularizes neural networks by penalizing large loss gradients.
problem How to optimize deep neural networks without explicit regularization.
method Backward error analysis to calculate implicit gradient regularization and demonstrate its effectiveness empirically.
result Implicit gradient regularization biases gradient descent toward flat minima, improving model robustness and test errors.
The paper improves model robustness by regularizing posterior differences.
problem Improving model robustness in noisy input scenarios.
method Posterior differential regularization with f-divergence. result Regularizing with f-divergence improves model robustness. Noise regularization improves CDE models without overfitting.
problem Overfitting in neural network-based conditional density estimation.
method Noise regularization method that adds random perturbations to data.
result Noise regularization significantly outperforms other methods across various datasets and models.
Study reveals the regularization effect of variational distributions in VAEs.
problem Understanding the regularization role of variational distributions in VAEs.
method Analyzed the role of variational family in VAEs and studied the regularization effect on local geometry.
result Uncovered the implicit regularizer in the β-VAE objective and proposed a deterministic autoencoding objective. New measure shows various training techniques control model complexity.
problem Understanding how to control model complexity in deep learning.
method Developed geometric complexity measure and demonstrated its effectiveness.
result Many training techniques control geometric complexity, providing a unified framework.
Cross-regularization adapts model complexity during training.
problem Manual tuning of model complexity for overfitting prevention.
method Directly adapts regularization parameters through validation gradients during training.
result Organic emergence of architecture-specific regularization during training.
Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
LambdaOpt automatically optimizes regularization hyperparameters for better recommendation model performance.
problem Data sparsity and high-cardinality issues in recommendation models.
method LambdaOpt automatically updates regularization coefficients based on validation data, allowing fine-grained regularization.
result LambdaOpt leads to better generalized models with improved recommendation performance.
Establish C^{1,2} regularity of American value functions in Heston model
problem Regularity of American put options in Heston model
method PDE techniques
result C^{1,2} regularity in exercise domain and smooth-fit principle
ExpO regularizes models to improve their explainability.
problem Improving the interpretability of black-box models.
method ExpO is a hybridization of regularization and post-hoc explanation systems.
result Post-hoc explanations for ExpO-regularized models have better explanation quality.
Regularizes sparse Gaussian processes for better model performance.
problem Efficient learning of inducing inputs in latent variable models.
method Proposes a regularization approach to balance reconstruction and approximation performance.
result Improves both inference and prediction performance in latent variable models.
Regularization methods can overregulate, suppressing causal features.
problem Mitigating shortcuts in models exploiting spurious correlations.
method Analysis of regularization methods and their effects on causal features.
result Regularization can overregulate, suppressing causal features.
The paper studies how regularization parameters affect sparsity in deep neural networks.
problem Reducing the complexity of deep neural networks by promoting sparsity.
method Derives ℓ1-norm sparsity-promoting models, characterizes sparsity levels, and develops algorithms for selecting optimal regularization parameters. result Developed algorithms to select regularization parameters for desired sparsity levels in neural networks.
Proposes regularization for robust image models using Wasserstein geometry.
problem Robustness to in-class variations in input data.
method Wasserstein-2 geometry, Tikhonov-type regularizer, data augmentation.
result Improves generalization under adversarial perturbations and large variations.
Regularization improves spectral embedding by focusing on the largest blocks.
problem Improving the quality of spectral embedding for graph data.
method Explained the impact of complete graph regularization on spectral embedding of a block model.
result Regularization forces spectral embedding to focus on the largest blocks, making it less sensitive to noise or outliers.
Bayesian regularization improves policy performance in noisy MDPs.
problem Suboptimal policies from estimated model parameters.
method Bayesian regularization of MDP objective function with prior information.
result Regularized policies show better robustness against model noise.
A framework for analyzing regularizers to ensure trustworthy theory-driven model estimation.
problem Uncertain choice of regularizers can compromise the interpretability of deep grey-box models.
method Adapting neural net architecture and training objective to analyze regularizer behavior empirically.
result Empirical analysis of regularizers helps in making a justified choice for trustworthy theory-driven model estimation.
Novel regularization for Vision Transformers improves model generalization and sparsity.
problem Improving generalization and sparsity in Vision Transformers.
method Likelihood-guided variational Ising-based regularization.
result Improved generalization and sparsity in Vision Transformers.
Adaptive regularization improves deep learning model performance.
problem Improving generalization in deep learning models.
method Adaptive regularization via residual smoothing based on the heat equation.
result Our algorithm outperforms other optimization methods in generalization.
DMLreg uses expert knowledge to improve model performance in high-dimensional settings.
problem Improving model performance in high-dimensional prediction problems.
method Learning a Mahalanobis distance metric from expert comparisons and integrating it into a regularized linear model.
result DMLreg leads to improvements in model performance when expert knowledge is relevant.
AMP regularization improves deep learning models by favoring flat minima.
problem Improving deep learning model generalization and avoiding overfitting.
method AMP regularization uses adversarial model perturbation to minimize a norm-bounded perturbation of the empirical risk.
result AMP regularization leads to state-of-the-art performance across various deep architectures.
A simple regularization technique speeds up training of Neural ODEs.
problem Training Neural ODEs is computationally expensive.
method Randomly sampling the end time of the ODE during training.
result Significantly decreases training time and improves performance.
Regularizes black-box models to improve interpretability.
problem Improving interpretability of black-box models without sacrificing accuracy.
method Regularizes a black-box model at training time to connect model explainability, explanation system, and quality metrics.
result Substantial improvement in explanation fidelity and stability across various datasets and explanation systems, with slight accuracy trade-off.
In application domains such as healthcare, we want accurate predictive models that are also causally interpretable. In pursuit of such models, we propose a causal regularizer to steer predictive models towards causally-interpretable solutions and theoretically study its properties. In a large-scale analysis of Electron…
Dropout improves regularization in flexible models for rare features.
problem Understanding theoretical properties of dropout in generalized linear models.
method Theoretical analysis and application to adaptive smoothing with B-splines.
result Dropout prefers rare features in mean and dispersion parameters.
Local regularization fails in transductive learning for some multiclass problems.
problem Whether local regularization can learn all transductive multiclass problems.
method Provided a negative answer by exhibiting a specific multiclass problem.
result Local regularization cannot learn all transductive multiclass problems.
Inserts proximal mapping into deep networks for better regularization.
problem Effective regularization of deep learning models to handle adversarial perturbations and correlations between modalities.
method Proposes a new layer that directly produces regularized hidden layer outputs using proximal mapping.
result Outperforms state-of-the-art methods in robust temporal learning and multiview modeling.
Regularized MLE improves MoE models for high-dimensional data.
problem Modeling with high-dimensional predictors and feature selection.
method Gaussian gating network, ℓ1-regularized MLE, EM-Lasso algorithm, BIC-like criterion. result Regularized MLE outperforms standard MLE in clustering and regression tasks.
Propagation-regularization improves GNN performance by infusing extra graph information.
problem The effectiveness of graph Laplacian regularization in GNNs is questioned and improved upon.
method Introducing Propagation-regularization (P-reg) to enhance GNN performance.
result P-reg boosts GNN performance on various tasks across multiple datasets.
Double descent risk in L2-regularized models explained and mitigated.
problem Risk of overparameterized models in machine learning.
method Analysis of L2-regularized models, two-layer neural networks, and CNNs.
result Double descent risk in L2-regularized models can be explained and mitigated by adjusting regularization strengths.
New regularization scheme for FMs improves feature interaction selection.
problem Feature selection in FMs leads to loss of feature interactions.
method Proposes a new regularization scheme for FMs with upper bound of ℓ1 regularizer. result Improves feature interaction selection without restricting sparsity patterns.
Study iterative regularization for linear models with convex bias, improving robust sparse recovery.
problem Improving robust sparse recovery with iterative regularization for linear models.
method Primal-dual gradient approach, analyzing convergence in presence of noise, combining regularization and optimization.
result Theoretical results show state-of-the-art performances with computational speed-ups.
DANR improves network regularization for spatio-temporal data.
problem Inadequate edge weights compromise network regularization.
method Discrepancy-aware network regularization (DANR) with ADMM.
result DANR achieves improved performance on various tasks.
A simple regularization method improves model generalization.
problem Overfitting due to lack of labeled data in machine learning models.
method Density-fixing regularization method based on class prior distribution.
result Improves model generalization performance by approximating class prior distribution.
Unified framework for accurate coresets in latent variable models and regularized regression.
problem Efficiently training models on large datasets.
method Unified framework for constructing accurate coresets for latent variable models and ℓp-regularized regression. result Unified framework reduces coreset size for latent variable models and ℓp-regularized regression. We propose a tree regularization framework, which enables many tree models to perform feature selection efficiently. The key idea of the regularization framework is to penalize selecting a new feature for splitting when its gain (e.g. information gain) is similar to the features used in previous splits. The regularizat…
Optimal regularization can prevent the double descent phenomenon in learning models.
problem The double descent phenomenon in learning models, where test performance is non-monotonic in sample size and model size.
method Theoretical and empirical study of optimal ℓ2 regularization for linear regression models and neural networks. result Optimally-tuned ℓ2 regularization achieves monotonic test performance for certain models and mitigates the double descent phenomenon for more general models. The paper improves ALO for ℓ1-regularized models.
problem Estimating out-of-sample error for ℓ1-regularized models. method Developed a novel theory for ℓ1-regularized problems, bounding ALO error. result For ℓ1-regularized problems, ALO error goes to zero as p goes to infinity. Hidden regular variation is a sub-model of multivariate regular variation and facilitates accurate estimation of joint tail probabilities. We generalize the model of hidden regular variation to what we call hidden domain of attraction. We exhibit examples that illustrate the need for a more general model and discuss de…
Study compares dropout and l2 regularization in linear models.
problem Understanding the statistical behavior of dropout and l2 regularization in linear models.
method Derives non-asymptotic bounds for gradient descent iterates with dropout and compares them to l2 regularization.
result Indicates a more subtle relationship between dropout and l2 regularization, highlighting interactions between dynamics and randomness.
Study Lipschitz regularity for manifold-constrained ROF model on curved surfaces.
problem Lipschitz regularity for manifold-constrained ROF model on curved surfaces.
method Generalization of ROF model, existence and uniqueness of minimizers, regularity results on PDE system.
result Lipschitz regularity of minimizers without convexity requirements.
New insights into how deep models generalize, focusing on matrix factorization.
problem Understanding how deep models generalize and why they work well.
method Using Morse functions and dynamical systems to study implicit regularization.
result Solved a conjecture on implicit regularization in matrix factorization.
Paper proposes ASR framework to improve image generation by incorporating human knowledge.
problem Deep generative models struggle to capture rich structures and relations in multi-object images.
method Introduces amortized structural regularization (ASR) framework using posterior regularization (PR) to embed human knowledge.
result Empirical results show ASR significantly outperforms DGM baselines in inference accuracy and sample quality.
We analyze learning curves of RF models with convex regularization and derive precise asymptotic expressions.
problem Understanding the learning curves of RF models with general convex regularization.
method Novel multi-level application of the convex Gaussian min max theorem (CGMT) to compute precise asymptotic expressions.
result Precise asymptotic expressions for learning curves of RF models with separable strongly convex regularization or ℓ1 regularization. Regularizes trajectory optimization with denoising autoencoders.
problem Trajectory optimization models are prone to inaccuracies.
method Uses a denoising autoencoder trained on the same trajectories.
result Improves planning with gradient-based and gradient-free optimizers.
In this work we study input gradient regularization of deep neural networks, and demonstrate that such regularization leads to generalization proofs and improved adversarial robustness. The proof of generalization does not overcome the curse of dimensionality, but it is independent of the number of layers in the networ…
New method encodes function preferences into neural nets for better generalization.
problem Challenges in encoding explicit function preferences in neural network training.
method Function-space empirical Bayes (FSEB) regularization.
result FSEB leads to near-perfect semantic shift detection and improved generalization.