New model shows SGD can prefer sharp or flat solutions based on label noise.
problem Understanding SGD's preference for flat or sharp solutions during training.
method Solved an analytically solvable model to explore SGD behavior.
result Data distribution determines sharpness at convergence; isotropic label noise leads to flat minimum preference.
Sharp bounds on uniform generalization errors in binary linear classification.
problem Understanding the uniform generalization errors in binary linear classification.
method Isoperimetric arguments, Poincaré and log-Sobolev inequalities for joint distributions.
result Sharp concentration bounds on uniform generalization errors, almost sure convergence in broad settings.
Improves model generalization by minimizing loss sharpness.
problem Overparameterized models often fail to generalize well despite low training loss.
method Sharpness-Aware Minimization (SAM) minimizes both loss value and sharpness.
result SAM improves model generalization across various datasets and models.
Label noise SGD converges to a simple model with a single linear feature.
problem Understanding the simplicity bias in neural network training.
method Analyzing the convergence of label noise SGD on two-layer neural networks.
result Label noise SGD converges to a model with a single linear feature.
A labeled oriented graph (LOG) is an oriented graph with a labeling function from the edge set into the vertex set. The complexity of a LOG is the minimal cardinality of an initial set S of vertices such that every vertex can be reached successively from S only using edges with labels in S or already visited vert…
Overparameterization enhances SAM's effectiveness in minimizing sharpness.
problem Improving generalization in deep neural networks.
method Analysis of Sharpness-Aware Minimization (SAM) under varying degrees of overparameterization.
result Overparameterization significantly improves SAM's performance, particularly in noisy and sparse settings.
Study community detection in multi-view data with various types of information.
problem Community detection in multi-view data with different types of information.
method Unified theoretical framework, mutual information analysis, sharp thresholds, iterative algorithms.
result Sharp thresholds for community recovery in various multi-view settings.
This paper introduces minimum-risk recalibration for probabilistic classifiers, improving their reliability and accuracy.
problem Improving the reliability and accuracy of probabilistic classifiers.
method Minimum-risk recalibration within the MSE decomposition framework, analyzing UMB method and label shift adaptation.
result The optimal number of bins for UMB scales with n1/3, resulting in a risk bound of approximately O(n−2/3). Researchers found the first and second eigenvalues are Courant-sharp on a Möbius strip.
problem Determining Courant-sharp eigenvalues on a Möbius strip.
method Analyzing the eigenvalues and nodal patterns of the Möbius strip.
result Only the first and second eigenvalues are Courant-sharp on the Möbius strip.
This paper investigates the computational complexity of sparse label propagation which has been proposed recently for processing network structured data. Sparse label propagation amounts to a convex optimization problem and might be considered as an extension of basis pursuit from sparse vectors to network structured d…
SAM improves neural network generalization better than SGD, especially in noisy data.
problem Overfitting in large neural networks with label noise.
method Sharpness-Aware Minimization (SAM) compared to Stochastic Gradient Descent (SGD).
result SAM prevents noise learning and facilitates feature learning better than SGD.
Enhanced financial forecasting using supervised autoencoders with noise augmentation and triple labeling.
problem Improving investment strategy performance on noisy financial data.
method Supervised autoencoders with noise augmentation and triple barrier labeling.
result Supervised autoencoders with balanced noise augmentation and bottleneck size significantly boost strategy effectiveness.
Study finds only first and second eigenvalues are Courant-sharp for flat Klein bottle and cylinders.
problem Determining Courant-sharp eigenvalues for compact flat surfaces.
method Analyzing flat Klein bottle and cylinders, proving only first and second eigenvalues are Courant-sharp.
result Only first and second eigenvalues are Courant-sharp for flat Klein bottle and cylinders.
Sharp sample complexity for multiclass PAC learning with bandit feedback.
problem Multiclass PAC learning with bandit feedback.
method Characterization of optimal sample complexity using a new combinatorial dimension (bandit DS dimension) and a learning algorithm (ListCascade).
result Sharp sample complexity characterization for every concept class up to logarithmic factors.
Bayesian SSR on graphs improves regression with noisy labels.
problem Estimating function values on graphs from noisy labeled data.
method Bayesian approach using graph Laplacian and Gaussian prior.
result Rates of contraction of posterior measure around ground truth.
Label noise in SGD helps converge to flatter minima.
problem Improving generalization in overparametrized models.
method Analyzes SGD with label noise, showing convergence to regularized minima.
result SGD with label noise converges to flatter minima, improving generalization.
Non-convex SGD learns halfspaces with adversarial label noise efficiently.
problem Agnostically learning halfspaces in adversarial label noise settings.
method Non-convex SGD optimization for halfspace learning.
result Non-convex SGD achieves misclassification error close to optimal with adversarial noise.
We determine the information-theoretic cutoff value on separation of cluster centers for exact recovery of cluster labels in a K-component Gaussian mixture model with equal cluster sizes. Moreover, we show that a semidefinite programming (SDP) relaxation of the K-means clustering method achieves such sharp threshol…
Study sharpens threshold for matching correlated graphs without labels.
problem Matching latent vertex correspondences in correlated random graphs.
method Analyzes information-theoretic limits for correct vertex matching in sub-sampled graphs.
result Establishes a sharp information-theoretic threshold for vertex matching recovery.
The paper explores how neural networks generalize differently from natural and medical images.
problem Discrepancies in generalization error between natural and medical images.
method Established and empirically validated a generalization scaling law with respect to intrinsic dataset properties.
result Higher intrinsic 'label sharpness' of medical images leads to higher adversarial vulnerability.
New framework learns from partial feedback in multi-label tasks.
problem Learning from one-sided feedback in multi-label tasks.
method Probably Approximately Correct (PAC) framework for set functions.
result Achieves optimal sample complexity in realizable case, multiplicative approximation guarantees in agnostic case.
Self-training improves generalization by fitting to reliable pseudo-labels or gradually improving the classification plane.
problem Understanding how self-training improves generalization in high-dimensional Gaussian mixtures.
method Analyzing iterative self-training on binary Gaussian mixtures in the asymptotic limit.
result ST improves generalization by fitting to reliable pseudo-labels or gradually improving the classification plane.
This paper tackles exact recovery of clusters in a stochastic Ising model on a SBM graph.
problem Recovering clusters in a stochastic Ising model on a SBM graph.
method Proposes a Stochastic Ising Block Model (SIBM) and establishes a sharp threshold for exact recovery.
result Sharp threshold m∗ for exact recovery of clusters in SIBM, with O(n) time complexity for m≥m∗. The task of aggregating and denoising crowd-labeled data has gained increased significance with the advent of crowdsourcing platforms and massive datasets. We propose a permutation-based model for crowd labeled data that is a significant generalization of the classical Dawid-Skene model, and introduce a new error metri…
Deep networks can interpolate noisy data without losing generalization.
problem Characterizing the relationship between interpolation and generalization in overparameterized deep networks.
method Analyzing the loss landscape of neural network functions over volumes around training data points, varying model parameters and training epochs.
result Loss sharpness in the input space follows a double descent, with large models predicting noisy targets over larger volumes around training data points.
Paper tackles noisy labels in deep learning networks.
problem Learning with noisy labels in deep neural networks.
method Sparse regularization strategy to approximate one-hot constraint.
result Improves performance of commonly-used loss functions in noisy labels and class imbalance.
The study examines when to trust confidence thresholding in pseudo-labelling regression.
problem Calibrated probabilities from classifiers used for pseudo-labelling need careful handling to avoid bias in downstream regression.
method Developed a diagnostic apparatus to predict and bound the bias induced by confidence thresholding, derived a closed-form expression for the attenuation bias.
result The bias can be predicted from the residual score variance V∗, motivating a structural separation between classifier features and downstream controls. Sharp analysis of knowledge distillation for high-dimensional regression.
problem Characterizing the risk of target models in high-dimensional settings.
method Sharp non-asymptotic bounds for ridgeless regression under model and distribution shifts.
result Identifies optimal surrogate models and reveals benefits and limitations of discarding weak features.
In several natural language tasks, labeled sequences are available in separate domains (say, languages), but the goal is to label sequences with mixed domain (such as code-switched text). Or, we may have available models for labeling whole passages (say, with sentiments), which we would like to exploit toward better po…
Study uses LLMs to categorize financial tweets, revealing useful sentiment signals.
problem Discovering meaningful sentiment signals from unstructured financial social media data.
method Leveraged LLMs to automatically label financial tweets with event categories and aligned with returns.
result Certain event labels consistently yield negative alpha, with statistically significant Sharpe ratios and information coefficients.
Paper confirms winning tickets have sharp minima, useful for generalization.
problem Explaining why over-parameterized models generalize well despite having sharp minima.
method PAC-Bayesian theory applied to analyze winning tickets and their generalization behavior.
result PAC-Bayesian theory confirms winning tickets have relatively sharp minima, a disadvantage for generalization.
Self-training improves model accuracy by refining pseudo-labels.
problem Improving semi-supervised learning with self-training.
method Theoretical insights into self-training algorithm with a focus on linear classifiers.
result Self-training iterations can improve model accuracy even if stuck in sub-optimal fixed points.
Study on online regression with noise, achieving near-optimal regret bounds.
problem Online generalized linear regression with stochastic noise.
method Sharp analysis of FTRL algorithm for stochastic label noise.
result Achieved near-optimal regret bounds for O(σ2dlogT)+o(logT). We present a graph-based variational algorithm for multiclass classification of high-dimensional data, motivated by total variation techniques. The energy functional is based on a diffuse interface model with a periodic potential. We augment the model by introducing an alternative measure of smoothness that preserves s…
ATLAS separates invariant and transferable latent factors across diverse environments.
problem Transfer learning and robust prediction in heterogeneous environments.
method ATLAS leverages invariance principle to disentangle latent factors and uses auxiliary labels for robust prediction.
result Near-oracle performance and robust transferable prediction in new environments.
We propose self-adaptive training---a new training algorithm that dynamically corrects problematic training labels by model predictions without incurring extra computational cost---to improve generalization of deep learning for potentially corrupted training data. This problem is crucial towards robustly learning from …
Efficiently learns a single neuron with adversarial noise, improving on prior work.
problem Learning a single neuron with adversarial label noise.
method Efficient algorithm using local error bounds from optimization theory.
result Approximates optimal L22-error within a constant factor. Sharp threshold for exact recovery in non-uniform hypergraph stochastic block model.
problem Community detection in random hypergraphs with non-uniform hyperedge probabilities.
method Sharp threshold established; two efficient algorithms for exact recovery.
result Sharp threshold for exact recovery; information-theoretic lower bound on misclassification.
The problem of devising learning strategies for discrete losses (e.g., multilabeling, ranking) is currently addressed with methods and theoretical analyses ad-hoc for each loss. In this paper we study a least-squares framework to systematically design learning algorithms for discrete losses, with quantitative character…
Lower bound proves ridgeless regression performs poorly near interpolation threshold.
problem Proving performance of ridgeless regression near interpolation threshold.
method Distribution-independent lower bound for mean squared error in noisy ridgeless linear regression.
result Lower bound implies ridgeless regression performs poorly near interpolation threshold.
Sharp-SSL uses random projections to identify important variables for semi-supervised learning.
problem High-dimensional semi-supervised learning problems.
method Careful aggregation of low-dimensional results from many axis-aligned random projections.
result Sharp-SSL algorithm can recover signal coordinates with high probability.
GNNs improve semi-supervised node regression, but why? We explain.
problem Understanding when and why GNNs succeed in semi-supervised node regression.
method Aggregate-and-readout model encompassing message passing architectures, least-squares estimation over GNNs with linear graph convolutions and a deep ReLU readout.
result Sharp non-asymptotic risk bound separating approximation, stochastic, and optimization errors.
Study shows multi-distribution learning has slower rates than single-task learning.
problem Understanding the statistical complexity of learning from heterogeneous sources.
method Structured hypothesis-testing framework to capture the statistical cost of certifying near-optimality under bounded noise.
result Learning across multiple distributions incurs slow rates scaling with k/ε2, even under constant noise levels. Paper introduces DMPMs for efficient discrete data generation with sharp convergence bounds.
problem Efficient generation of discrete data with theoretical guarantees.
method Discrete Markov Probabilistic Models (DMPMs) operating in bit space with time-reversal process.
result Sharp convergence bounds established under minimal assumptions, competitive performance in discrete data generation.
Study proposes an active subsampling method for estimating individualized thresholds in high-dimensional data.
problem Estimating optimal individualized thresholds in high-dimensional data with limited labeled samples.
method Developed a K-step active subsampling algorithm to iteratively select and label the most informative data points.
result Revealed a phase transition phenomenon in the estimation of θ with respect to the smoothness of the conditional density. Kronheimer and Mrowka introduced a new knot invariant, called s♯, which is a gauge theoretic analogue of Rasmussen's s invariant. In this article, we compute Kronheimer and Mrowka's invariant for some classes of knots, including algebraic knots and the connected sums of quasi-positive knots with non-trivial r…
Classical multidimensional scaling is an important dimension reduction technique. Yet few theoretical results characterizing its statistical performance exist. This paper provides a theoretical framework for analyzing the quality of embedded samples produced by classical multidimensional scaling. This lays the foundati…
We present a graph-based variational algorithm for classification of high-dimensional data, generalizing the binary diffuse interface model to the case of multiple classes. Motivated by total variation techniques, the method involves minimizing an energy functional made up of three terms. The first two terms promote a …