Paper proposes a new framework to improve stability-based bounds in deep learning.
problem Explaining generalization in overparameterized neural networks.
method Decomposes excess risk dynamics into signal and noise components, applying stability-based bounds only to the noise.
result The decomposition framework improves stability-based bounds and explains generalization in neural networks.
reval package selects best clustering solutions via stability-based validation.
problem Challenges in determining best clustering solutions due to lack of validation methods.
method Stability-based relative clustering validation methods.
result Determines best clustering solutions that generalize to unseen data.
A stability-based method selects the most desirable conformal prediction set.
problem Selecting the most desirable conformal prediction set from multiple valid sets invalidates coverage guarantees.
method A stability-based approach that ensures coverage for the selected prediction set.
result The stability-based approach maintains coverage guarantees for the selected prediction set.
Paper improves generalization bounds for noisy stochastic algorithms.
problem Improving generalization bounds for noisy stochastic algorithms.
method Introduces Exponential Family Langevin Dynamics (EFLD) and establishes data-dependent expected stability based generalization bounds.
result Sharp generalization bounds with O(1/n) sample dependence and gradient discrepancy.
Efficiently estimates sparse mean from heavy-tailed data.
problem Robustly estimating sparse mean from heavy-tailed distributions.
method Stability-based approach adapted for heavy-tailed data.
result Optimal sample complexity with logarithmic dependence on dimension.
This paper shows how to learn variational inequalities fast with strong monotonicity.
problem Learning variational inequalities efficiently.
method Extending convex optimization techniques to variational inequalities with strong monotonicity.
result Fast generalization rates of Θ(1/ε) for learning variational inequalities. We introduce a strengthening of K-stability, based on filtrations of the homogeneous coordinate ring. This allows for considering certain limits of families of test-configurations, which arise naturally in several settings. We prove that if a manifold with no automorphisms admits a cscK metric, then it satisfies this s…
PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our …
The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.
problem The tension between uniform stability and L2-stability in generalization bounds. method Establishes in-expectation first moment generalization error bounds for L2-stable randomized learning algorithms and uses subbagging to achieve near-tight exponential bounds. result Improves generalization bounds for convex and non-convex optimization problems with SGD.
A principal pair consists of a holomorphic principal G-bundle together with a holomorphic section of an associated Kaehler fibration. Such objects support natural gauge theoretic equations coming from a moment map condition, and also admit a notion of stability based on Geometric Invariant Theory. The Hitchin--Kobaya…
Paper addresses LSTM stability for thermal systems using infinity-norm.
problem Stability of LSTM networks in thermal systems.
method Derived ISS∞ condition for LSTM, developed training strategy. result ISS∞-promoted LSTM outperforms other models in thermal system case study. New theory for nonsmooth systems helps optimize and control complex functions.
problem Optimizing and controlling systems with nonsmooth functions.
method Higher-order averaging theory with nonsmooth near-identity transformation and lexicographic differentiation.
result Closed formula for nonsmooth first and second-order averaging.
A challenging problem in estimating high-dimensional graphical models is to choose the regularization parameter in a data-dependent way. The standard techniques include K-fold cross-validation (K-CV), Akaike information criterion (AIC), and Bayesian information criterion (BIC). Though these methods work well for lo…
Framework purifies approximate differential privacy to pure differential privacy.
problem Achieving pure differential privacy from approximate differential privacy.
method Randomized post-processing with calibrated noise to eliminate δ parameter.
result First statistically and computationally efficient reduction from approximate DP to pure DP.
SGD-trained deep nets have bounds on their generalization error.
problem Bounding generalization error for deep neural networks trained by SGD.
method Combining dynamical control of parameter norms and Rademacher complexity estimates.
result Explicit bounds depend on loss trajectory, work for various architectures.
The study identifies conditions under which algorithmic stability explains generalization in interpolating learning systems.
problem Understanding when algorithmic stability explains generalization in interpolating learning systems.
method Modeling training as a function-space trajectory and measuring sensitivity to single-sample perturbations.
result There exist interpolating regimes with small risk where contractive sensitivity cannot hold, showing that stability is not a universal explanation.
SAM improves generalization by operating near the edge of stability.
problem Improving generalization in neural networks.
method Sharpness-Aware Minimization (SAM) approach to training neural networks.
result SAM operates near the 'edge of stability' identified by the analysis.
New stability framework relaxes boundedness assumptions for generalization bounds.
problem Overly restrictive assumptions for modern learning settings with heavy-tailed or unbounded losses.
method Develops a stability-based framework requiring only finite Lp moment conditions. result Sharp generalization bounds derived for various learning paradigms.
New method selects best HTE estimator without ground-truth treatment effects.
problem Selecting best HTE estimator from multiple candidates.
method Cross-fitted, exponentially weighted test statistic with two-way sample splitting.
result Empirically, reliable error control and reduced false selections.
The paper derives uniform stability-based coverage bounds for conformal prediction methods.
problem Establishing theoretical guarantees for conformal prediction methods.
method Uniform stability perspective applied to full-conformal, jackknife+, and CV+ prediction regions.
result Coverage bounds for finite-dimensional models derived using a concentration argument.
Generative Adversarial Networks (GANs) are known to be difficult to train, despite considerable research effort. Several regularization techniques for stabilizing training have been proposed, but they introduce non-trivial computational overheads and interact poorly with existing techniques like spectral normalization.…
Develops a fast method to learn graph structures from large datasets.
problem Learning graph structures from huge datasets with computational intractability and high complexity.
method Minipatch Graph (MPGraph) estimator: breaks up the problem into minipatches, uses hard thresholding, and integrates hyperparameter tuning.
result Proves graph selection consistency and empirically shows superior accuracy and speed compared to state-of-the-art methods.
BAWS adapts window size for financial risk forecasting.
problem Adaptive selection of look-back window for financial risk modeling.
method Data-driven online learning method using bootstrap-based adaptive window selection (BAWS).
result BAWS improves risk forecasting, especially in data with structural changes.
Nonnegative low-rank matrix recovery can have spurious local minima.
problem Nonnegative low-rank matrix recovery problems can have spurious local minima.
method Investigated projected gradient methods for nonnegative low-rank recovery problems.
result Benign nonconvexity holds in the fully-observed case with RIP constant δ=0 but fails in the partially-observed case and higher-rank ground truths.
Sparse Convex Biclustering improves accuracy and robustness in high-dimensional datasets.
problem Challenges in clustering rows and columns of large-scale datasets due to noise and computational complexity.
method Sparse Convex Biclustering (SpaCoBi) using convex optimization and stability-based tuning.
result Significantly outperforms state-of-the-art methods in accuracy for high-dimensional datasets.
Algorithmic stability is a classical approach to understanding and analysis of the generalization error of learning algorithms. A notable weakness of most stability-based generalization bounds is that they hold only in expectation. Generalization with high probability has been established in a landmark paper of Bousque…
The paper analyzes the trade-off between smoothness and sparsity in GCN using lp-regularized learning.
problem Quantifying the trade-off between smoothness and sparsity in GCN.
method Proposes a novel SGD proximal algorithm for GCNs with an inexact operator to analyze the stability of the ℓp-regularized stochastic learning. result Establishes an explicit theoretical understanding of GCN with ℓp-regularized stochastic learning. Blind Source Separation (BSS) has proven to be a powerful tool for the analysis of composite patterns in engineering and science. We introduce Convex Analysis of Mixtures (CAM) for separating non-negative well-grounded sources, which learns the mixing matrix by identifying the lateral edges of the convex data scatter p…
A neural network method determines the latent dimensionality of NMF.
problem Determining the correct number of hidden features (latent dimensionality) in NMF.
method Combining NMFk with an MLP classifier trained on a dataset of matrices with known latent features.
result The MLP classifier in conjunction with NMFk achieves a greater than 95% success rate in determining the correct number of latent features.
New criterion selects optimal number of clusters based on stability.
problem Challenges in selecting optimal number of clusters in non-parametric clustering.
method Proposes a stability-based validation criterion combining between-cluster and within-cluster stability.
result Empirically demonstrates effectiveness in selecting optimal number of clusters.
DFFL tackles federated learning with heterogeneous objectives and constraints.
problem Federated learning with clients having different objectives and feasible regions.
method Derived heterogeneity bounds for cost-vector distances and support-function/shape-distance terms. Lifted pointwise bounds to local-versus-federated excess-risk comparison.
result Federation is beneficial when the statistical advantage of pooling exceeds a client-specific heterogeneity penalty.
Enhanced Lion optimizer CLion improves generalization with lower error.
problem Lion optimizer's generalization analysis is lacking.
method Algorithmic stability analysis and cautious sign function use.
result CLion has a lower generalization error of O(N1). Algorithm-dependent generalization error bounds are central to statistical learning theory. A learning algorithm may use a large hypothesis space, but the limited number of iterations controls its model capacity and generalization error. The impacts of stochastic gradient methods on generalization error for non-convex …
New algorithm optimizes robust estimation under mixed local and global corruptions.
problem Combining local and global corruptions in robust statistics.
method Information-theoretic approach using sliced-Wasserstein metric.
result Optimal error achieved in polynomial time for stronger local perturbations.
New bounds for KANs trained with DP-SGD, addressing correlated noise.
problem Risk bounds for Kolmogorov-Arnold Networks trained by DP-SGD with correlated noise.
method Established new optimization and population risk analysis for KANs trained with DP-SGD, addressing correlated noise.
result First optimization and population risk analysis of correlated-noise mechanisms for DP training in non-convex settings, including neural networks.
There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in …
Improved DP SO with large Lipschitz parameters, handling outliers and heavy-tailed data.
problem Differential privacy in stochastic optimization with large Lipschitz parameters.
method Assumes bounded k-th order moments, provides linear-time algorithms for smooth convex and non-smooth convex losses.
result Improved risk bounds scaling with k-th moment, not uniform Lipschitz parameter.
Geometric approach combines asset returns and investor views for better portfolio optimization.
problem Optimizing portfolios with investor-specific views.
method Generalized Wasserstein barycenter (GWB) to integrate statistical asset returns and investor views.
result The geometric approach offers more flexibility and rewards for correct investor views.
Paper proposes an alternative method to price American options using HJM approach.
problem Price American options efficiently and accurately.
method Utilizes HJM technique to model term structure of volatility for equity markets.
result Proposes a new value function, stopping criteria, and stopping time for American options.
We study inference and learning based on a sparse coding model with `spike-and-slab' prior. As in standard sparse coding, the model used assumes independent latent sources that linearly combine to generate data points. However, instead of using a standard sparse prior such as a Laplace distribution, we study the applic…
Quantum machine learning: Adiabatic quantum SVM outperforms classical methods.
problem Training support vector machines efficiently on large datasets.
method Adiabatic quantum computing for SVM training.
result Quantum approach outperforms classical methods in accuracy and scalability.
We develop a semi-analytic approach to the valuation of auto-callable structures with accrual features subject to barrier conditions. Our approach is based on recent studies of multi-assed binaries, present in the literature. We extend these studies to the case of time-dependent parameters. We compare numerically the s…
Two ML approaches learn local volatility surfaces from option prices, with GP being arbitrage-free.
problem Interpolating European vanilla option prices to create a local volatility surface.
method Gaussian process regression and neural net with arbitrage penalties.
result GP approach is arbitrage-free and yields best out-of-sample calibration error.
This paper critiques the Standardized Measurement Approach (SMA) for operational risk and recommends maintaining Advanced Measurement Approach (AMA).
problem Weaknesses and failures of the Standardized Measurement Approach (SMA) in operational risk.
method Critical review and analysis of SMA and AMA approaches.
result SMA is unstable, insensitive to risk, and implicitly related to systemic risk in the banking sector.
Distributed Acoustic Sensing (DAS) using fiber optic cables is a promising new technology for pipeline monitoring and protection. In this work, we applied and compared two approaches for event detection using DAS: Classic machine learning approach and the approach based on image processing and deep learning. Although w…
A new Euclidean approach reveals the pentagram map's beauty.
problem Exploring the pentagram map through classical geometry.
method Introducing an alternative Euclidean approach.
result Demonstrates the pentagram map's elegance through classical geometry.
Two approaches extend knowledge distillation to Gaussian Processes, showing relationships to existing methods.
problem Applying knowledge distillation to Gaussian Processes for regression and classification.
method Data-centric and distribution-centric approaches to extend distillation to GPR and GPC.
result Distribution-centric approach for GPC approximately corresponds to data duplication and scaling.
We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…