An unregularized score improves anomaly detection in images with varying complexity.
problem Inaccurate anomaly detection in images with complex shapes and small anomalies.
method Proposed an unregularized score for deep generative models to overcome the issue of lower likelihoods for complex objects.
result The unregularized score is robust to the complexity of samples and improves anomaly detection.
Study shows convergence of stochastic gradient method for unregularized Wasserstein optimization.
problem Wasserstein distributionally robust optimization under potential distribution shifts.
method Regularized approximation with stochastic gradient methods, convergence analysis.
result Stochastic gradient method converges to subgradients of unregularized objective as regularization vanishes.
In this paper, we consider unregularized online learning algorithms in a Reproducing Kernel Hilbert Spaces (RKHS). Firstly, we derive explicit convergence rates of the unregularized online learning algorithms for classification associated with a general gamma-activating loss (see Definition 1 in the paper). Our results…
Study risk-sensitive market making with entropy regularization for better quote control.
problem Risk-sensitive market making with exponential utility and penalties.
method Entropy-regularized certainty-equivalent Bellman policies for discrete-time market dynamics.
result Proves convergence and performance bounds for entropy-regularized policies.
This manuscript studies statistical properties of linear classifiers obtained through minimization of an unregularized convex risk over a finite sample. Although the results are explicitly finite-dimensional, inputs may be passed through feature maps; in this way, in addition to treating the consistency of logistic reg…
The article analyzes high-dimensional classification using empirical risk minimization with precise error predictions.
problem Classifying high-dimensional data with Gaussian mixture models.
method Theoretical analysis of ridge-regularized and unregularized empirical risk minimization for high-dimensional Gaussian mixture separation.
result The square loss is optimal for high-dimensional classification in both ridge-regularized and unregularized cases.
New bounds show empirical EOT adapts to simpler measure.
problem Statistical performance of empirical EOT estimators.
method Novel statistical bounds, empirical process theory, dual formulation.
result Empirical EOT and its unregularized version follow lower complexity adaptation.
Paper develops an online learning algorithm for functional data models.
problem Recovering slope functions or predictors in functional data models.
method Online regularized learning algorithm in reproducing kernel Hilbert spaces with polynomially decaying step-size.
result Established fast convergence rates for estimation error without capacity assumption.
gKRLS accelerates KRLS estimation for complex models.
problem Limited flexibility and high computation for KRLS.
method Re-formulate KRLS as a hierarchical model and implement random sketching.
result gKRLS can fit models on large datasets in minutes.
Mirror descent algorithm recovers low-rank matrices in matrix sensing.
problem Matrix sensing with low-rank matrices under certain conditions.
method Discrete-time mirror descent applied to empirical risk with Bregman divergence analysis.
result Mirror descent converges to a matrix minimizing a specific nuclear norm-related quantity.
Debiased Wasserstein barycenters improve on entropy regularization in OT.
problem Entropy regularization in OT introduces bias, leading to blurred barycenters.
method Propose debiased Wasserstein barycenters using Sinkhorn iterations.
result Debiased barycenters preserve fast Sinkhorn-like iterations without entropy smoothing bias.
Equivalence found between algorithmic regularization and convex penalization for convex losses.
problem Understanding the relationship between algorithmic regularization and convex penalization.
method Introducing a geometric condition and showing equivalence through optimization paths.
result Optimization paths of iterative algorithms on unregularized problems match those of corresponding penalized problems under certain conditions.
Exact expressions for double descent and implicit regularization in over-parameterized models.
problem Understanding the generalization error of over-parameterized models like deep neural networks.
method Surrogate random design to replace standard i.i.d. design, leading to exact expressions for mean squared error and implicit regularization.
result Exact non-asymptotic expressions for double descent and implicit regularization in over-parameterized models.
SGD implicitly regularizes linear regression problems better than ridge regression for many cases.
problem Understanding implicit regularization in linear regression problems.
method Comparing SGD and ridge regression on a broad class of least squares problems.
result SGD generalizes no worse than ridge regression for many problem instances, sometimes better.
Regularizers change the geometric properties of loss functions in neural networks.
problem Understanding how different regularizers affect the geometric properties of loss functions in neural networks.
method Examined several regularizers, including weight decay, to determine if the regularized loss function becomes Morse.
result For certain regularizers, the regularized loss function becomes Morse, indicating a change in geometric properties.
We analyze coresets for regularized regression problems and propose a modified lasso that yields smaller coresets.
problem Analyzing coresets for regularized regression problems.
method Examined coresets for ridge regression and proposed a modified lasso problem.
result No coreset for regularized regression can be smaller than the unregularized version when reqs. Paper develops a minimax optimal test for goodness-of-fit using kernel Stein discrepancy.
problem Developing a robust goodness-of-fit test for general domains.
method Kernel Stein Discrepancy (KSD) with spectral regularization and adaptive testing.
result Proposed regularized test achieves minimax optimality up to a logarithmic factor.
The Nonnegative Matrix Factorization (NMF) of the rating matrix has shown to be an effective method to tackle the recommendation problem. In this paper we propose new methods based on the NMF of the rating matrix and we compare them with some classical algorithms such as the SVD and the regularized and unregularized no…
Proposes ridge regression on Riemannian manifolds for time-series prediction.
problem Time-series prediction on Riemannian manifolds.
method Combines Riemannian least-squares fitting via Bézier curves, empirical covariance on manifolds, and Mahalanobis distance regularization.
result Significant error reduction in synthetic spherical experiments and hurricane forecasting.
Infinite width ReLU networks can approximate functions with bounded Euclidean norm.
problem Functions that can be approximated by ReLU networks with bounded Euclidean norm.
method Analyzing the minimal network norm required to approximate a given function.
result The minimal network norm for representing a function \( f \) is \( \max(\int |f''(x)| dx, |f'(-\infty) + f'(+\infty)|) \).
A new neural network approach for inverse problems.
problem Solving inverse problems in medical imaging and computer vision.
method Using a neural network as a regularization functional to learn from unsupervised data.
result The proposed framework can be applied even with unsupervised training data.
G-TRACER optimizes deep learning by promoting flat minima.
problem Promoting generalization in deep learning architectures.
method Geometric TRACE Ratio regularization, curvature-regularized optimizers.
result Converges to a neighborhood of local minima of unregularized objective.
Ensemble quantile classifier improves performance on high-dimensional data.
problem Discriminating high-dimensional data with heavy-tailed or skewed inputs.
method Regularized quantile classifier that assigns variable weights.
result Consistently estimates minimal population loss and is Bayes optimal.
Improved feature selection with regularization.
problem Feature selection in noisy data.
method Regularized greedy column subset selection.
result Significantly increased robustness and stability.
This is one in a series of papers devoted to the foundations of Symplectic Field Theory sketched in [Y Eliashberg, A Givental and H Hofer, Introduction to Symplectic Field Theory, Geom. Funct. Anal. Special Volume, Part II (2000) 560--673]. We prove compactness results for moduli spaces of holomorphic curves arising in…
In-network distributed estimation of sparse parameter vectors via diffusion LMS strategies has been studied and investigated in recent years. In all the existing works, some convex regularization approach has been used at each node of the network in order to achieve an overall network performance superior to that of th…
New method trains neural ODEs faster with fewer layers.
problem Training neural ODEs on large datasets is computationally expensive.
method Combines optimal transport and stability regularizations.
result Significant reductions in training time with no performance loss.
A new method for statistical inference using approximate Newton steps from stochastic gradients.
problem Efficient statistical inference for convex and non-convex learning problems.
method Approximate stochastic Newton steps based on finite differences.
result Efficient computation of statistical error covariance without exact second-order information.
Gradient descent on separable data converges to SVM's max-margin solution.
problem Gradient descent on logistic regression problems with linearly separable data.
method Examination of gradient descent on logistic regression with homogeneous predictors.
result Gradient descent converges to the max-margin solution of SVM.
Proposes a framework to improve deep neural networks' robustness and generalization.
problem Vulnerability to adversarial attacks and difficulty in generalizing to novel images.
method Disentangled deep autoencoding regularization framework.
result Significantly improves robustness against adversarial attacks and generalization to novel test data.
New AM regularization improves both accuracy and robustness.
problem Lack of robustness in deep neural networks.
method Average margin (AM) regularization for margin classifiers or deep neural networks.
result AM regularization can improve both accuracy and robustness to adversarial attacks.
Recurrent Neural Networks (RNNs) are rich models for the processing of sequential data. Recent work on advancing the state of the art has been focused on the optimization or modelling of RNNs, mostly motivated by adressing the problems of the vanishing and exploding gradients. The control of overfitting has seen consid…
The paper provides generalization bounds for metric learning using neural network embeddings.
problem Generalization guarantees for metric learning with neural network embeddings.
method Uniform generalization bounds for two regimes: sparse and bounded amplification.
result Dimension-free generalization bounds can be achieved even without sparsity in solutions.
Generative models' evaluation scores can be misleading, leading to inflated grades.
problem Misleading evaluation scores for generative models.
method Analyzed and compared various scores for evaluating synthetic vs. ground-truth data.
result The Eden score avoids grade inflation and better aligns with human perception.
A new method for efficient inference and model selection in SBMs using OT.
problem Efficient inference and model selection in stochastic block models.
method Interpreting MLVI as srGW with entropic regularization, then unregularizing for sparse solutions, and adding a sparsity-promoting regularizer.
result The method consistently recovers SBM parameters and selects the number of clusters in finite samples.
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
Gaussian surrogates improve Poisson imaging performance at low doses.
problem Improving Poisson imaging performance at low doses.
method Analysis of Poisson and Gaussian surrogate reconstruction objectives under Poisson noise.
result Gaussian surrogates can achieve MSE comparable to Poisson MAP at low doses.
The paper analyzes generalization properties of scalable kernel methods.
problem Understanding the generalization of doubly stochastic learning algorithms.
method Theoretical analysis of different variants of doubly stochastic learning algorithms in nonparametric regression.
result Derivation of generalization error convergence results for the algorithms.
This work improves likelihood of score-based diffusion ODEs using high-order denoising score matching.
problem The gap between maximum likelihood and score matching objectives for score-based diffusion ODEs.
method High-order denoising score matching to maximize likelihood.
result Score-based diffusion ODEs achieve better likelihood on synthetic and CIFAR-10 data.
Study compares multivariate scoring rules for distribution forecasts.
problem Evaluating the discrimination ability of multivariate scoring rules.
method Simulation study comparing energy and variogram scores using historical data.
result Variogram score with p=0.5 outperforms other scores.
Paper establishes NE existence and efficient algorithms for weakly monotone GMFGs.
problem Existence and efficient learning of Nash Equilibrium in λ-regularized GMFGs. method Establishes existence of NE for any λ-regularized GMFGs. Proposes efficient algorithms for weakly monotone GMFGs. result Efficient algorithms for weakly monotone GMFGs with provable convergence.
A new method improves data generation quality by correcting score mismatches.
problem Score mismatch issue in conditional score-based data generation methods.
method Denoising Likelihood Score Matching (DLSM) loss for classifier training.
result The proposed method outperforms previous methods on Cifar-10 and Cifar-100 benchmarks.
New scoring rules improve probabilistic classification model evaluation.
problem Traditional scoring rules misalign with the preference for correct classifications.
method Introduces Penalized Brier Score (PBS) and Penalized Logarithmic Loss (PLL) to modify proper scoring rules.
result PBS and PLL better identify optimal checkpoints and early stopping points, leading to superior F1 scores.
Extends denoising and score estimation to energy models via Tweedie's formula.
problem Linking denoising and score estimation for a wider range of distributions.
method Derives a fundamental identity connecting energy score derivatives and scores.
result Establishes a new identity for energy scores analogous to Tweedie's formula.
Mixed-SCORE+ improves community detection in weak signal networks.
problem Detecting communities in weak signal networks.
method Proposes Mixed-SCORE+ combining properties of Mixed-SCORE and SCORE+.
result Significantly improves detection error rates on Polblogs and weak signal networks.
The paper introduces regularized OT to create sparse transportation plans.
problem Lack of sparsity in entropic regularization of OT.
method Regularizing primal and dual OT formulations with strongly convex terms, leading to sparse transportation plans.
result Regularized OT can lead to sparser transportation plans than entropic regularization.
Examining ESG scoring method for reliability.
problem Reliability of ESG scoring methodology.
method Analyzing Refinitiv's ESG scoring process.
result Methodology needs improvement for trustworthiness.
New scoring rules for multivariate distributions and level sets.
problem Evaluating forecast accuracy for multivariate distributions and level sets.
method Theoretical framework for scoring rules, decomposition of multivariate scoring functions, numerical algorithm for computation.
result New scoring functions for multivariate distributions and level sets, including density and cumulative distribution level sets.