The paper explores parabolic regularity in geometric variational analysis.
problem Developing calculus rules and computation formulas for second-order generalized differential constructions.
method Introducing and applying the concept of parabolic regularity to geometric aspects of second-order variational analysis.
result Established new calculus rules and computation formulas for second-order generalized differential constructions.
New framework assesses regularization norms in ill-posed problems, revealing L2 instability and proposing adaptive fractional RKHS solutions.
problem Comparative analysis of regularization norms in ill-posed problems.
method Small noise analysis framework for Tikhonov and RKHS regularizations.
result Optimal convergence rates achieved with adaptive fractional RKHS, but hyper-parameters decay too fast.
This paper analyzes Stochastic Depth regularization in ResNets.
problem Improving generalization in ResNets through regularization.
method Hybrid analysis combining perturbation and signal propagation.
result Principled guidelines for choosing survival rates in SD.
Gradient descent implicitly regularizes neural networks by penalizing large loss gradients.
problem How to optimize deep neural networks without explicit regularization.
method Backward error analysis to calculate implicit gradient regularization and demonstrate its effectiveness empirically.
result Implicit gradient regularization biases gradient descent toward flat minima, improving model robustness and test errors.
Improved LDA method for better classification and dimensionality reduction.
problem Improving linear discriminant analysis for better classification performance.
method Integrates spectrally-corrected covariance matrix and regularized discriminant analysis.
result SRLDA has a linear classification global optimal solution under spiked model assumption.
The study analyzes large dimensional regularized discriminant analysis classifiers using Gaussian mixture models.
problem Understanding the performance of regularized discriminant analysis in large but finite dimensions.
method Large dimensional analysis using Gaussian mixture models and random matrix theory.
result The asymptotic classification error approaches a deterministic quantity dependent on class means and covariances.
The paper analyzes the complexity of manifold regularization methods.
problem Understanding the complexity of manifold regularization in semi-supervised learning.
method The paper derives sample complexity bounds and Rademacher bounds for semi-supervised methods.
result The semi-supervised method can only have a constant improvement, ignoring logarithmic terms.
Paper uses replica method to study overfitting in Cox model.
problem Overfitting in Cox model when p ~ N.
method Replica method from statistical physics.
result Established relationship between optimal regularization and p/N.
Study of regularized least squares in RKKS with indefinite kernels.
problem Asymptotic properties of regularized least squares with indefinite kernels in RKKS.
method Introducing a bounded hyper-sphere constraint, theoretical demonstration of globally optimal solution, modified error decomposition techniques, matrix perturbation theory.
result Derivation of learning rates in RKKS, same as RKHS under certain conditions.
Improved regression analysis using Padé approximants with new residuals and regularization.
problem Improving regression analysis with Padé approximants for accuracy and avoiding overfitting.
method New residuals in least squares method, system of linear equations for rational functions, Tikhonov regularization.
result Demonstrated efficiency in practical cases from physics and reliability theory.
Statistical analysis of regularization in continual learning tasks.
problem Understanding how regularization affects model performance in sequential learning.
method Derivation of convergence rates, iterative update formula, and optimal hyperparameters for generalized ℓ2-regularization.
result Optimal hyperparameters balance forward and backward knowledge transfer, improving model performance.
Paper analyzes bias-variance tradeoff in graph Laplacian regularization.
problem Understanding the optimal regularization parameter for graph Laplacian.
method Spectral graph properties and signal-to-noise ratio parameter used to determine optimal regularization.
result Selecting mediocre regularization is often suboptimal, suggesting near-optimal performance.
Sharp analysis improves RLHF sample complexity with KL-regularization.
problem Improving RLHF sample complexity with KL-regularization.
method Sharp analysis of KL-regularized contextual bandits and RLHF.
result Achieved an O(1/ε) sample complexity when ε is sufficiently small.
Proposes HGP to improve survival analysis models.
problem Improving survival analysis models by addressing density function changes.
method Imposes constraints on local data points by regularizing the hazard function gradient.
result HGP enhances survival analysis model performance.
Study iterative regularization for linear models with convex bias, improving robust sparse recovery.
problem Improving robust sparse recovery with iterative regularization for linear models.
method Primal-dual gradient approach, analyzing convergence in presence of noise, combining regularization and optimization.
result Theoretical results show state-of-the-art performances with computational speed-ups.
Proposes a causal regularizer for interpretable predictive models.
problem Accurate predictive models that are also causally interpretable in healthcare.
method Causal regularizer applied to neural network architecture for non-linear causality analysis.
result Causally-regularized model outperforms L1-regularized counterpart in causal accuracy and predictive performance.
New iterative regularization method tackles non-smooth, non-strongly convex functionals.
problem Tackles non-smooth, non-strongly convex functionals in regularization problems.
method Primal-dual algorithm with convergence and stability analysis.
result First iterative regularization procedure for non-smooth, non-strongly convex functionals.
New algorithms for latent class analysis using regularized spectral clustering.
problem Identifying latent classes within populations from categorical data.
method Developed two new algorithms using a regularized Laplacian matrix to estimate latent classes.
result Our algorithms provide consistent latent class analysis under mild conditions and can accurately infer the number of latent classes.
Introduces self-regularization for analyzing learning algorithms.
problem Analyzing and optimizing learning algorithms without explicit regularization.
method Develops a self-regularization framework for learning algorithms.
result Provides statistical analysis and minmax-optimal rates for self-regularized algorithms.
Paper analyzes and improves KL-regularized RL for LLMs with logarithmic regret.
problem Improving efficiency of RL fine-tuning for large language models.
method Optimism-based KL-regularized online contextual bandit algorithm with novel regret analysis.
result Achieves an O(ηlog(NRT)⋅dR) logarithmic regret bound. Physics-driven regularization improves DNNs for engineering tasks.
problem Training deep neural networks for engineering systems using incomplete data.
method Regularization using governing laws to penalize divergence from physical laws.
result Proposed method yields more interpretable and accurate DNN models.
New method synthesizes and analyzes probability measures using entropy-regularized optimal transport.
problem Synthesize and analyze probability measures with entropy-regularized optimal transport.
method Entropy-regularized Wasserstein-2 cost and Sinkhorn divergence for synthesis and analysis.
result Computed barycentric coefficients and their stability for classification of corrupted point cloud data.
New method learns semidefinite regularizers from data.
problem Learning suitable regularization functions from data without domain-specific expertise.
method Combines linear and semidefinite programming techniques for structured factorizations of data matrices.
result Algorithm identifies correct regularizer promoting data structure, converging locally linearly.
Proposes a filtering method for cluster analysis using ℓ0-norm regularization.
problem Improving cluster analysis by filtering data.
method Minimizes a least squares function with a weighted ℓ0-norm penalty, approximated by smooth non-convex functions. result The proposed method can enhance existing clustering techniques.
KL regularization helps RL algorithms by implicitly averaging q-values.
problem Understanding why KL regularization improves RL performance.
method An approximate value iteration scheme, studying KL and entropy regularization.
result Strong performance bound combining linear horizon dependency and averaging effect of estimation errors.
We study Tikhonov regularization for solving ill--posed operator equations where the solutions are functions defined on surfaces. One contribution of this paper is an error analysis of Tikhonov regularization which takes into account perturbations of the surfaces, in particular when the surfaces are approximated by spl…
Improved gradient descent for rectangular matrix completion without ℓ2,∞ regularization.
problem Nonconvex rectangular matrix completion without ℓ2,∞ regularization. method Gradient Descent without ℓ2,∞ regularization. result Improved sampling rate from O(poly(κ)μ3r3log3n/n) to O(μ2r2κ14logn/n). Simple analysis for fast rates in empirical minimization with concave losses and convex regularization.
problem Fast rates in empirical minimization with concave losses and convex regularization.
method Simple analysis using covering number and concentration inequality.
result First result of fast rates with high probability for exponential concave empirical risk minimization.
FA algorithm provides convergence guarantees for deep linear networks.
problem Training efficiency and convergence of deep neural networks.
method Theoretical analysis of Feedback Alignment (FA) algorithm for deep linear networks.
result Certain initializations lead to implicit anti-regularization, affecting learning effectiveness.
Proposes R2LDA for improved LDA classifier performance.
problem Poor performance of LDA classifiers in small to comparable data sizes.
method Doubly regularized LDA with automatic parameter selection.
result Consistent and effective performance in noisy test data.
Regularized MFPCA smooths multivariate functional data for clearer patterns.
problem Challenges in controlling roughness of multivariate functional PCs.
method ReMFPCA incorporates a roughness penalty in a penalized framework to smooth PCs.
result Smoothed multivariate functional PCs reveal clearer patterns.
New analysis reveals optimal regularization for ESNs, avoiding double descent.
problem Characterizing and optimizing Echo State Networks (ESNs) for precise bias-variance.
method Random matrix theory applied to ESNs in a teacher-student setting.
result ESNs achieve lower MSE with limited training samples and teacher memory.
Improves online learning algorithms for functional models with capacity assumptions.
problem Convergence rates of online stochastic gradient descent algorithms for functional linear models.
method Characterizations of slope function regularity, kernel space capacity, and sampling process covariance operator.
result Capacity assumptions can alleviate saturation of convergence rates as function regularity increases.
Quantum computing techniques improve graph analysis and community detection.
problem Analyzing large graphs efficiently and accurately.
method Used quantum annealing and quantum gate computers for community detection and regularity checking.
result Demonstrated the effectiveness of quantum computing in solving complex graph problems.
New regularization method reduces support of empirical risk minimization solutions.
problem Regularization in empirical risk minimization with relative entropy.
method Introduces Type-II regularization, characterizes solutions, analyzes properties of relative entropy.
result Type-II regularization collapses solution support into reference measure's support.
We establish a general theorem improving regularity of solutions of elliptic pseudodifferential equations. It allows to resolve in a unified way the regularity issue for a broad class of nonlinear elliptic equations and systems appearing in different areas of geometry and analysis.
Study Tikhonov regularization for non-linear inverse problems to improve image reconstruction accuracy.
problem Reconstructing quantities from noisy, non-linearly transformed observations.
method Tikhonov regularization using reproducing kernel Hilbert spaces.
result Developed optimal convergence rates for the estimator.
Proposes using knowledge graphs to improve probabilistic model estimation.
problem Lack of direct utilization of domain knowledge in regularization methods.
method Knowledge-based distant regularization framework using knowledge graph embeddings.
result Improvement in generalization capability of probabilistic models.
This paper explores how entropic regularization improves Wasserstein estimators' performance.
problem Improving the approximation and estimation properties of Wasserstein estimators.
method Entropic regularization of optimal transport costs to smooth Wasserstein estimators.
result Entropic regularization can achieve comparable statistical performance to un-regularized estimators at lower computational cost.
Proposes an L1-regularized functional SVM for binary classification with functional covariates.
problem Binary classification with multivariate functional covariates.
method L1-regularized functional support vector machine (SVM) with an accompanying algorithm.
result The proposed classifier performs well in prediction and feature selection.
New algorithm stabilizes bi-level hyperparameter optimization.
problem Stability issues in bi-level hyperparameter optimization.
method Uses Moreau-Yosida regularization to stabilize convergence.
result Significant improvement in loss values with fixed computation budget.
Paper analyzes NAC with neural networks for efficient policy optimization.
problem Improving sample and iteration complexity in policy optimization.
method Entropy regularization, averaging, neural network approximation, and optimization techniques.
result Entropy regularization and averaging ensure stability and sharp sample complexity bounds.
Deep learning method improves myelin water fraction estimation.
problem Estimating myelin water fraction in the brain using magnetic resonance relaxometry.
method Combines input layer regularization with automated regularization hyperparameter tuning.
result Proposed method outperforms classical methods and multi-layer perceptrons on in vivo brain data.
CRDA improves gene selection in microarray studies by reducing feature space.
problem Gene selection in high-dimensional microarray datasets.
method CRDA combines ℓq,1 norm minimization and hard thresholding for feature elimination. result CRDA outperforms competitors in misclassification and feature selection accuracy.
Proposes improved classification via transfer learning with regularized linear discriminant analysis.
problem High dimensionality and small sample sizes lead to poor classification performance.
method Regularized random-effects linear discriminant analysis, combining ridge estimates from target and source models.
result Explicit derivation of asymptotic weights and classification error rates in high-dimensional settings.
A method to prune 3D CNNs by assigning different regularization parameters to layers based on importance.
problem Massive computation and storage consumption in 3D CNNs.
method Regularization-based pruning method assigning different regularization parameters to different weight groups.
result Pruning leads to 2x speedup with minimal accuracy loss for 3DResNet18 and C3D.
New nonconvex regularizers improve low-rank matrix recovery efficiency and accuracy.
problem Efficiently recover low-rank matrices from incomplete data.
method Factor group-sparse regularization, related to Schatten-p norms.
result Improved generalization error bounds for Schatten-p norms as p decreases.
We provide theoretical analysis of the statistical and computational properties of penalized M-estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …