Improves SGD for convex functions with mini-batches, proving linear convergence.
problem Minimizing convex functions with constraints.
method Projected semi-stochastic gradient descent with mini-batches.
result Linear convergence under weak strong convexity assumption.
Paper proves linear convergence of R-FDM and RC-FDM under weak strong convexity.
problem Optimizing SVM dual problem and LASSO problem.
method Randomized feasible descent method (R-FDM) and coordinate-wise random feasible descent method (RC-FDM).
result Both R-FDM and RC-FDM converge linearly under weak strong convexity assumption.
Boosts weak online learners to strong ones with sublinear regret.
problem Online learning agnostic setting without strong guarantees.
method Reduction to online convex optimization, boosting via marginally-better-than-trivial regret guarantees.
result First agnostic online boosting algorithm with sublinear regret.
Boosting is a popular way to derive powerful learners from simpler hypothesis classes. Following previous work (Mason et al., 1999; Friedman, 2000) on general boosting frameworks, we analyze gradient-based descent algorithms for boosting with respect to any convex objective and introduce a new measure of weak learner p…
Proposes a new regression framework for mixed strong and weak guidance.
problem Current regression frameworks cannot use both strong and weak guidance.
method Introduces a probabilistic formulation for weak guidance based on relative orderings, bounds, and similarity relations.
result Optimization problems with weak guidance are convex.
New algorithm solves saddle point problems in Banach spaces.
problem Solving saddle point problems in real reflexive Banach spaces.
method Stochastic Bregman Primal-Dual Splitting Algorithm with relative smoothness and strong convexity assumptions.
result Almost sure convergence to saddle points under various conditions.
New connection between subset selection and submodular maximization.
problem Subset selection and submodular maximization in high-dimensional settings.
method Greedy algorithms and weak submodularity.
result Greedy algorithms perform within a constant factor of the best possible subset-selection solution.
Extends boosting to multiclass online agnostic classification.
problem Online multiclass classification with weak learners.
method Reduces multiclass online agnostic boosting to online convex optimization.
result First boosting algorithm for online agnostic multiclass classification.
New guarantees for Group LASSO in sparse convex optimization.
problem Sparse convex optimization with vector-valued features.
method Group LASSO regularization and analysis of gradient norms.
result Group LASSO selects the same features as Orthogonal Matching Pursuit.
We study the task of online boosting--combining online weak learners into an online strong learner. While batch boosting has a sound theoretical foundation, online boosting deserves more study from the theoretical perspective. In this paper, we carefully compare the differences between online and batch boosting, and pr…
Defines certainty equivalent and utility indifference pricing for incomplete preferences.
problem Incomplete preferences represented by multiple priors and utility functions.
method Defines certainty equivalent and utility buy/sell prices as set-valued functions of claims, proves monotonicity and convexity properties, approximates bounds via convex vector optimization.
result Certainty equivalent and indifference price bounds can be computed or approximated by convex vector optimization.
RAVEN improves weak-to-strong generalization under distribution shifts.
problem Weak models fail to supervise strong models effectively under distribution shifts.
method RAVEN dynamically learns optimal combinations of weak models and strong model parameters.
result RAVEN outperforms existing methods by over 30% on out-of-distribution tasks.
New model shows weak teachers can help strong students learn even with imperfect labels.
problem Improving strong student's performance with weak teacher's imperfect pseudolabels.
method Stylized overparameterized spiked covariance model with Gaussian covariates, proving two phases of generalization.
result Provable successful and random guessing phases of strong student's generalization.
Optimal algorithm converts weak to strong learner with less data.
problem Constructing a strong learner from a weak learner with minimal data.
method New algorithm that uses less training data than AdaBoost.
result Optimal sample complexity for converting weak to strong learner.
Formalizes weak and strong verification for LLMs, controlling errors without assumptions.
problem Balancing cost and reliability in reasoning with LLMs.
method Formalizes weak-strong verification policies, introduces metrics, develops online algorithm.
result Optimal policies admit a two-threshold structure, and calibration and sharpness govern value of weak verifiers.
The study explores the strengths and weaknesses of models that generalize from weak to strong supervision.
problem Understanding the limitations and capabilities of models that generalize from weak to strong supervision.
method Theoretical analysis and experimental validation in both classification and regression settings.
result Theoretical bounds reveal the importance of strong generalization and calibration of the weak model and a careful balance in the training process.
Random feature models can outperform a weak teacher with early stopping.
problem Generalization from a weak to a strong model in random feature networks.
method Random feature models, early stopping, proving weak-to-strong generalization.
result Random feature models can outperform a weak teacher with early stopping.
Weak labels can significantly speed up learning for strong tasks.
problem Learning with limited strong labels.
method Using weak labels to accelerate learning of strong tasks.
result Weak labels can accelerate learning to O ( i c e f r a c 1 n ) \mathcal{O}(
icefrac{1}{n}) O ( i ce f r a c 1 n ) rate. Paper analyzes weak-to-strong generalization in CNNs, identifying data-scarce and data-abundant regimes.
problem Weak-to-strong generalization in CNNs trained on weak models.
method Formal analysis of gradient descent dynamics in data-scarce and data-abundant regimes.
result Identifies two regimes and distinct mechanisms of generalization in each.
The study examines different types of equilibria for stopping problems in one-dimensional diffusion processes.
problem Characterizing and comparing different types of equilibria for time-inconsistent stopping problems.
method Analyzes log sub-additive discount functions and one-dimensional diffusion processes to derive necessary and sufficient conditions for weak equilibria and other types of equilibria.
result Conditions for weak equilibria and their implications for other types of equilibria are provided.
New mechanism detects overlap density for weak-to-strong generalization.
problem Understanding what aspects of data enable weak-to-strong generalization.
method Data-centric mechanism and overlap detection algorithm.
result Overlap density is a key factor in weak-to-strong generalization.
We identify a condition for regularity of optimal transport maps that requires only three derivatives of the cost function, for measures given by densities that are only bounded above and below. This new condition is equivalent to the weak Ma-Trudinger-Wang condition when the cost is C 4 C^4 C 4 . Moreover, we only require (n…
New framework transfers latent knowledge from weak to strong models.
problem Aligning superhuman LLMs with human feedback.
method Transfer learning framework using refinement approach.
result Proves weak-to-strong generalization is possible.
W2S FT often outperforms weak teachers due to low intrinsic dimensionality.
problem Understanding why weak-to-strong finetuning outperforms weak models.
method Analyzing W2S in ridgeless regression setting, focusing on variance reduction.
result Weak teacher's variance is inherited by strong student in shared feature subspace, reduced in discrepancy subspace.
The paper defines a new equivalence relation for knot projections and finds an infinite number of distinct classes.
problem Classifying knot projections based on weak homotopy equivalence.
method Defining weak (1, 2, 3) homotopy and using it to find an invariant.
result There are an infinite number of weak (1, 2, 3) homotopy equivalence classes of knot projections.
New theory explains how strong models can learn from weak ones.
problem Learning from weak, incomplete, or incorrect labels.
method New bounds based on data distribution and student hypothesis class.
result Existing weak supervision theory fails to account for pseudolabel correction and coverage expansion.
Active learning with weak and strong labelers reduces label queries.
problem Learning from weak and strong labelers with low error.
method Active learning algorithm for weak and strong labelers, statistical consistency, label complexity analysis.
result Reduces label queries compared to using strong labelers alone.
Paper proposes DC functions for better regularization of inverse problems with theoretical guarantees.
problem Improving regularization for ill-posed inverse problems.
method Introduces difference-of-convex (DC) functions and uses them with optimization algorithms like DCA and PSM.
result DC functions yield improved performance and theoretical guarantees compared to weakly convex functions.
The paper proves stability of curvature bounds in geometric analysis.
problem Stability of local Riemannian Ricci curvature bounds under convergence.
method Gromov-Hausdorff convergence, Lagrangian approach, heat flow, weak gradients, Evolution Variational Inequality.
result Almost everywhere existence of Euclidean weak tangents.
Boosting weak learners to strong ones from aggregate labels is possible for LLP but not for MIL.
problem Boosting weak learners to strong ones from aggregate labels in learning from label proportions (LLP).
method Using a weak learner on large enough bags to obtain a strong learner for small bags in polynomial time.
result Boosting is possible for LLP but not for MIL.
New homotopy types and invariants defined for knots.
problem Defining and characterizing different homotopy types of knot projections.
method Introducing strong and weak (1, 2) homotopies and defining new invariants.
result New necessary and sufficient conditions for homotopy equivalence of knot projections.
New algorithms solve convex-concave problems faster than previous methods.
problem Solving min-max problems without bilinear structure.
method Stochastic primal-dual algorithms with logarithmic dual updates.
result Faster convergence rates than O ( 1 / T ) O(1/\sqrt{T}) O ( 1/ T ) for certain problems. Introduces new equilibrium concepts for time-inconsistent stochastic control.
problem Time-inconsistent stochastic control in continuous time.
method New equilibrium definitions and asymptotic analysis for Markov chains.
result Characterizations and existence of strong and weak equilibria.
We discuss general notions of metrics and of Finsler structures which we call weak metrics and weak Finsler structures. Any convex domain carries a canonical weak Finsler structure, which we call its tautological weak Finsler structure. We compute distances in the tautological weak Finsler structure of a domain and we …
Paper proposes sparse classification method for high-dimensional data.
problem Sparse classification in high-dimensional data with positive-confidence samples.
method Developed a novel sparse-penalization framework using L1, SCAD, and MCP penalties for convex and non-convex shrinkage.
result Proved near minimax-optimal sparse recovery rates under Restricted Strong Convexity condition.
Study uses weak transport for non-convex costs in fixed-income markets.
problem Characterizing optimal caplet pricing in fixed-income markets.
method Introduced weak optimal transport for non-convex costs, reduced general costs to convex problems.
result Established robust super-replication results for fixed-income markets.
Study shows how a strong model can learn a task's feature while retaining other capabilities.
problem How to align superhuman AI systems using weak-to-strong generalization.
method Two-layer neural networks, reward-model learning, multi-step SGD, feature learning.
result The strong model efficiently learns task features while retaining general capabilities.
The paper defines new homotopy relations on knot projections and classifies certain knot types.
problem Defining and classifying knot homotopy relations.
method Introducing cross chord numbers and using them to define strong and weak (1, 3) homotopies.
result Complete classification of knot projections with trivializing number two.
Proves inextendibility of weak null singularities from curvature blow-up.
problem Inextendibility of weak null singularities in the context of curvature blow-up.
method Introduces a new strategy to infer C l o c 0 , 1 C^{0,1}_{\mathrm{loc}} C loc 0 , 1 -inextendibility from curvature blow-up. result Expected to contribute to the resolution of strong cosmic censorship conjecture.
Improved machine learning models outperform their simpler counterparts by using imperfect labels.
problem Improving model performance using imperfect labels.
method Random feature ridge regression (RFRR) with a deterministic equivalent for excess test error.
result The student model can outperform the teacher model regardless of the teacher's scaling law, achieving the minimax optimal rate.
New research shows label refinement and weak training have limitations for aligning LLMs.
problem Limitations of refinement methods for aligning large language models.
method Analyzed probabilistic assumptions and alternative approaches to label refinement and weak training.
result Label refinement and weak training suffer from irreducible error, leaving a performance gap.
Paper improves algorithms for convex-concave minimax optimization problems.
problem Minimizing convex-concave functions with strong convexity and concavity properties.
method Proposes a new algorithm with improved gradient complexity.
result Improves gradient complexity upper bound for minimax optimization.
Boosting improves accuracy with fewer calls to weak learners for certain concept classes.
problem Improving accuracy of learning algorithms with limited weak learner calls.
method Combines boosting and list-decodable codes to achieve better performance for specific concept classes.
result A new boosting algorithm that achieves strong learning with fewer calls to weak learners and additional samples.
New varifold solutions for mean curvature flow converge and are unique.
problem Mean curvature flow and Allen-Cahn equation convergence and uniqueness.
method Evolving varifolds coupled to phase volumes, weak-strong uniqueness principle.
result Limits of Allen-Cahn solutions are varifold solutions, and classical flows are unique.
We consider an abstract compact orientable Cauchy-Riemann manifold endowed with a Cauchy-Riemann complex line bundle. We assume that the manifold satisfies condition Y(q) everywhere. In this paper we obtain a scaling upper-bound for the Szegö kernel on (0, q)-forms with values in the high tensor powers of the line bund…
We propose a weak formulation for the binormal curvature flow of curves in R 3 . \R^3. R 3 . This formulation is sufficiently broad to consider integral currents as initial data, and sufficiently strong for the weak-strong uniqueness property to hold, as long as self-intersections do not occur. We also prove a global existence t…
The paper reveals three mechanisms for weak-to-strong generalization.
problem Understanding the mechanisms behind weak-to-strong generalization in imperfect labeling scenarios.
method Theoretical analysis of simple models including ridge regression and weighted ridge regression, and a nonlinear multi-index setting.
result A student model can compensate for a teacher's under-regularization and achieve lower test error.
The strong maximum principle is proved to hold for weak (in the sense of support functions) sub- and super-solutions to a class of quasi-linear elliptic equations that includes the mean curvature equation for C 0 C^0 C 0 spacelike hypersurfaces in a Lorentzian manifold. As one application a Lorentzian warped product splittin…