New framework improves worst-case generalization bounds for stochastic optimization.
problem Challenges in providing generalization guarantees for stochastic optimization algorithms.
method Introduces random set stability and empirically relevant complexity measures to avoid intractable mutual information terms.
result Bounded worst-case generalization error in terms of random set stability and empirically relevant complexity measures.
We derive Gaussian approximations for random forest predictions using region-based stabilization.
problem Improving the accuracy of random forest predictions for Poisson process data.
method Region-based stabilization and Malliavin-Stein method for multivariate Gaussian approximation.
result Established Gaussian approximation bounds for random forest predictions under Poisson process.
A stability metric compares feature selection algorithms in machine learning.
problem Stability of feature selection algorithms in machine learning.
method Rank-based instability index to compare MDA, LIME, and SHAP algorithms.
result LIME and SHAP are more stable than MDA, with LIME being best for human interpretability.
GCNs converge and remain stable on large random graphs, revealing geometric insights.
problem Understanding the behavior of GCNs on large, sparse random graphs.
method Analysis of GCNs on random graph models with latent variables and geometric edge probabilities.
result GCNs converge to their continuous counterparts as graph size increases, and are stable to small graph deformations.
Randomness is crucial for stability in learning and statistics, especially for differential privacy.
problem Quantifying the amount of randomness needed for algorithmic stability.
method Weak-to-strong boosting theorem for stability, characterizing randomness complexity of PAC Learning.
result Randomness complexity is tightly controlled by the best replication probability of any deterministic algorithm solving the task.
The paper studies random dynamical systems of polynomial automorphisms on C^2 and finds mean stability.
problem Random dynamical systems of polynomial automorphisms on C^2.
method Generic random dynamical systems of polynomial automorphisms are shown to have mean stability.
result A generic random dynamical system of polynomial automorphisms on C^2 has mean stability.
Optimal algorithms for online convex optimization with random order.
problem Online convex optimization with random order and non-convex loss functions.
method Stochastic gradient descent and algorithmic stability analysis.
result Achieves optimal bounds and significantly outperforms previous methods.
New method stabilizes machine learning predictions across random seeds.
problem Machine learning predictions vary across random seeds, causing instability.
method Introduces adaptive cross-bagging to eliminate seed dependence.
result Adaptive cross-bagging achieves targeted stability in debiased machine learning.
New model stabilizes asynchronous LTI systems, independent of synchronous stability.
problem Stability of asynchronous LTI systems under randomization and asynchrony.
method Introduced a new model for random asynchronous LTI systems and developed a method for system identification.
result Stability of random asynchronous LTI systems is independent of synchronous stability.
Batch normalization prevents rank collapse in deep networks, improving training stability.
problem Rank collapse in randomly initialized deep networks with increasing depth.
method Investigates spectral instabilities in random matrices and uses batch normalization to avoid rank collapse.
result Batch normalization prevents rank collapse in both linear and ReLU networks, improving training stability.
Random forests are stable and provide reliable prediction intervals.
problem Stability and reliability of random forest prediction intervals.
method Established stability under mild conditions and proved coverage bounds.
result Non-asymptotic lower and upper bounds for prediction interval coverage.
Stabilizing black-box algorithms through task-oriented randomization
problem Ensuring stability of black-box models
method Task-oriented randomization
result Established rigorous theoretical foundations and demonstrated effectiveness through simulations and real-world applications
The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.
problem The tension between uniform stability and L2-stability in generalization bounds. method Establishes in-expectation first moment generalization error bounds for L2-stable randomized learning algorithms and uses subbagging to achieve near-tight exponential bounds. result Improves generalization bounds for convex and non-convex optimization problems with SGD.
New study on replicability and stability in machine learning algorithms.
problem Ensuring consistent results in machine learning models without fixing randomness.
method Introduced global stability and list replicability concepts, proving their equivalence and boosting list replicability.
result Global stability can only be achieved weakly, while list replicability can be boosted to achieve high probability of consistent results.
Polynomial invariant of quandles counts random link colorings.
problem Counting Q-colorings of random braids in quandles. method Average number of Q-colorings for large n. result The average number of Q-colorings coincides with a polynomial PQ. Paper shows ERM's suboptimality due to bias, not variance.
problem Understanding why ERM fails to achieve optimal rates.
method Probabilistic and admissibility proofs for ERM in various settings.
result ERM's suboptimality is due to bias, not variance.
New method stabilizes deep neural networks by setting Lyapunov exponent to zero.
problem Stability issues in deep neural networks with low width.
method Lyapunov initialization method to set Lyapunov exponent to zero.
result Lyapunov exponent governs stability of deep networks; standard methods fail for low width.
Investment diversification affects financial stability, depending on network connectivity.
problem Analyzing stability of financial networks with diversified portfolios.
method Random matrix dynamical model with portfolio rebalancing, considering heterogeneity and diversification effects.
result Stability/instability transition depends on the largest eigenvalue of the random matrix.
Stability of the utility maximization problem with random endowment and indifference prices is studied for a sequence of financial markets in an incomplete Brownian setting. Our novelty lies in the nonequivalence of markets, in which the volatility of asset prices (as well as the drift) varies. Degeneracies arise from …
Study finds significant instability in node embeddings due to randomness.
problem Stability of node embeddings under random variations.
method Evaluated five node embedding algorithms (HOPE, LINE, node2vec, SDNE, GraphSAGE) on synthetic and empirical graphs.
result Significant instability in embedding spaces and downstream task accuracy.
FIRES framework selects stable features from online data.
problem Efficiently selecting features in online settings with limited data.
method FIRES framework uses model parameter importance for feature selection.
result FIRES selects stable feature sets with minimal model complexity.
Random quotients of hyperbolic cubulated groups remain cubulated.
problem Understanding properties of random quotients of hyperbolic cubulated groups.
method Cubical small-cancellation theory, exponential growth of conjugacy classes, and hyperplane stabilizers' growth.
result Low-density random quotients of cubulated hyperbolic groups are cubulated and hyperbolic.
AKO improves stability and power of Knockoff inference.
problem Instability of Knockoff-based inference.
method Aggregation of Multiple Knockoffs (AKO) method.
result AKO maintains FDR control while improving stability and power.
In this paper, we focus on quantifying model stability as a function of random seed by investigating the effects of the induced randomness on model performance and the robustness of the model in general. We specifically perform a controlled study on the effect of random seeds on the behaviour of attention, gradient-bas…
New approach to concentration inequalities for unbounded state space dynamical systems.
problem Concentration inequalities for unbounded state space dynamical systems.
method Functional analytic framework, transport-entropy inequality.
result Exponential concentration inequalities for sampling from stationary distribution.
Stabilization of linear systems with unknown dynamics is a canonical problem in adaptive control. Since the lack of knowledge of system parameters can cause it to become destabilized, an adaptive stabilization procedure is needed prior to regulation. Therefore, the adaptive stabilization needs to be completed in finite…
We analyze why some models resist unlearning using linear stability theory.
problem Understanding and predicting when machine learning models resist unlearning.
method Linear stability theory applied to machine learning models, focusing on data coherence and optimization dynamics.
result Data coherence and signal-to-noise ratio (SNR) influence unlearning resistance; lower SNR makes unlearning easier.
Stability is a general notion that quantifies the sensitivity of a learning algorithm's output to small change in the training dataset (e.g. deletion or replacement of a single training sample). Such conditions have recently been shown to be more powerful to characterize learnability in the general learning setting und…
New methods ensure feature importance rankings are correct with high probability.
problem Stability issues in feature importance scores due to random sampling.
method Hypothesis testing-based techniques to assess and verify the stability of top-ranked features.
result Ensures the most important features are correct with high-probability guarantees.
Study designs for estimating treatment effects in adaptive experiments.
problem Estimating treatment effects under adaptive treatment assignment.
method Propose and analyze IPW and AIPW estimators, establish CLTs under design stability.
result Central limit theorems for IPW and AIPW estimators under design stability.
In this paper we measured the stability of stochastic gradient method (SGM) for learning an approximated Fourier primal support vector machine. The stability of an algorithm is considered by measuring the generalization error in terms of the absolute difference between the test and the training error. Our problem is to…
The paper analyzes stability of random matrix products with Markovian noise.
problem Analyzing stability of random matrix products with Markovian noise.
method Using a super-Lyapunov drift condition and controlled growth of matrix-valued functions, the paper provides an exponential stability result for the p-th moment of random matrix product.
result Finite-time p-th moment bounds for linear stochastic approximation and TD learning algorithms.
We prove a motivic stabilization result for the cohomology of the local systems on configuration spaces of varieties over C attached to character polynomials. Our approach interprets the stabilization as a probabilistic phenomenon based on the asymptotic independence of certain *motivic random variables*, an…
New stability framework relaxes boundedness assumptions for generalization bounds.
problem Overly restrictive assumptions for modern learning settings with heavy-tailed or unbounded losses.
method Develops a stability-based framework requiring only finite Lp moment conditions. result Sharp generalization bounds derived for various learning paradigms.
Randomized hierarchical clustering tests for stability and detects clusters.
problem Greedy hierarchical clustering's sensitivity to data perturbations.
method Randomization scheme and p-values at each node.
result Valid hypothesis testing procedures for clustering results.
Random forests are among the most popular classification and regression methods used in industrial applications. To be effective, the parameters of random forests must be carefully tuned. This is usually done by choosing values that minimize the prediction error on a held out dataset. We argue that error reduction is o…
We provide a dual characterisation of the weak∗-closure of a finite sum of cones in L∞ adapted to a discrete time filtration Ft: the tth cone in the sum contains bounded random variables that are Ft-measurable. Hence we obtain a generalisation of Delbaen's m-stability condition…
Exclusive Group Lasso improves feature selection in correlated biological data.
problem Correlated features hinder Lasso performance in biological classification problems.
method Proposes and solves the exclusive group Lasso, combining stability selection and random group allocation.
result Exclusive Group Lasso outperforms Lasso in comprehensive selection of informative features.
New findings show that common optimization algorithms struggle with random problems.
problem Finding near-optimal solutions to random optimization problems.
method Low-degree polynomials, Boolean circuits, and Langevin dynamics.
result These algorithms fail to produce nearly optimal solutions with high probability.
New findings on maximizing noise stability in partitions of Gaussian space.
problem Maximizing noise stability in partitions of Gaussian space.
method Analyzing the correlation between sets and their noise stability, proving conditional conjectures and hardness results.
result Hyperstable partitions maximize noise stability and have specific properties.
New method embeds dynamic networks with stability for node behavior.
problem Embed time-evolving node representations with stability.
method Unfolded adjacency spectral embedding for dynamic networks.
result Method satisfies cross-sectional and longitudinal stability.
PAC-Bayes bounds have been proposed to get risk estimates based on a training sample. In this paper the PAC-Bayes approach is combined with stability of the hypothesis learned by a Hilbert space valued algorithm. The PAC-Bayes setting is used with a Gaussian prior centered at the expected output. Thus a novelty of our …
Random SNNs are stable and simple, with low-frequency Fourier spectra.
problem Stability and robustness of spiking neural networks.
method Boolean function analysis and Fourier spectrum concentration.
result Random LIF-SNNs are stable and biased towards simple functions.
We consider the problem of learning a set from random samples. We show how relevant geometric and topological properties of a set can be studied analytically using concepts from the theory of reproducing kernel Hilbert spaces. A new kind of reproducing kernel, that we call separating kernel, plays a crucial role in our…
Regularization improves stability and consistency of sparse autoencoders.
problem Varying features across random seeds and training choices in SAEs.
method Added L1 or L2 penalties on encoder and decoder weights.
result L2 regularization increases cross-seed feature consistency.
Strong stability of ergodic iterations proven without ergodic driving sequence.
problem Ensuring strong stability of ergodic iterations under non-ergodic driving sequences.
method Revisiting processes driven by stationary ergodic sequences, proving strong stability under mild conditions on recursive maps.
result Strong stability of iterations proven without ergodic driving sequence.
We establish novel generalization bounds for learning algorithms that converge to global minima. We do so by deriving black-box stability results that only depend on the convergence of a learning algorithm and the geometry around the minimizers of the loss function. The results are shown for nonconvex loss functions sa…
In this paper, we consider voxel selection for functional Magnetic Resonance Imaging (fMRI) brain data with the aim of finding a more complete set of probably correlated discriminative voxels, thus improving interpretation of the discovered potential biomarkers. The main difficulty in doing this is an extremely high di…