New tools extend weak approximation of SGD algorithms to infinite time horizon.
problem Weak approximation of stochastic gradient descent algorithms in infinite time horizon.
method Backward error analysis of numerical stochastic differential equations and truncated formal power expansion.
result Characterization of asymptotic behavior of SGD algorithms for strongly convex functions.
This article analyzes the weak error of SGD optimization schemes.
problem Analyzing the error in SGD optimization schemes with respect to a test function.
method Weak error analysis for SGD type optimization schemes.
result The weak error decays at the same speed as in the strong sense.
Faster weak supervision framework using triplet methods.
problem Computational inefficiency in weak supervision models.
method Closed-form solution for latent variable models, avoiding iterative methods.
result Orders of magnitude faster than previous approaches.
New theory explains how strong models can learn from weak ones.
problem Learning from weak, incomplete, or incorrect labels.
method New bounds based on data distribution and student hypothesis class.
result Existing weak supervision theory fails to account for pseudolabel correction and coverage expansion.
Improved approximations for rough Heston model reduce errors.
problem Lack of Markov and semimartingale properties in rough Heston model.
method Markovian approximations with weak error analysis.
result Super-polynomial convergence of new approximations.
Improves label propagation for weakly supervised learning.
problem Reducing the need for labeled data in machine learning.
method Label Propagation with Weak Supervision (LPA) analysis.
result Demonstrated improvements over existing methods on weakly supervised classification tasks.
We introduce cylindrical projections to simulate infinite-dimensional occupation flows of diffusions.
problem Computational intractability of infinite-dimensional occupation flows of diffusions.
method Introduce cylindrical projections to approximate the occupation flow via a finite-dimensional system.
result Strong convergence of cylindrical projections to the initial process with derived rates.
W2S FT often outperforms weak teachers due to low intrinsic dimensionality.
problem Understanding why weak-to-strong finetuning outperforms weak models.
method Analyzing W2S in ridgeless regression setting, focusing on variance reduction.
result Weak teacher's variance is inherited by strong student in shared feature subspace, reduced in discrepancy subspace.
Develops weak PINNs for efficient manifold solutions of hyperbolic equations.
problem Challenges in approximating weak solutions of nonlinear hyperbolic equations on manifolds.
method Introduces a novel weak PINN (wPINN) formulation on manifolds leveraging well-posedness theory.
result Demonstrates efficient approximation of entropy solutions on manifolds with a complexity independent of ambient space dimension.
The significance of the study of the theoretical and practical properties of AdaBoost is unquestionable, given its simplicity, wide practical use, and effectiveness on real-world datasets. Here we present a few open problems regarding the behavior of "Optimal AdaBoost," a term coined by Rudin, Daubechies, and Schapire …
A network may have weak signals and severe degree heterogeneity, and may be very sparse in one occurrence but very dense in another. SCORE (Jin, 2015) is a recent approach to network community detection. It accommodates severe degree heterogeneity and is adaptive to different levels of sparsity, but its performance for…
Formalizes weak and strong verification for LLMs, controlling errors without assumptions.
problem Balancing cost and reliability in reasoning with LLMs.
method Formalizes weak-strong verification policies, introduces metrics, develops online algorithm.
result Optimal policies admit a two-threshold structure, and calibration and sharpness govern value of weak verifiers.
New bounds for RFM-KRR with weak assumptions and easy verification.
problem Establishing accurate out-of-sample bounds for RFM-KRR.
method Elementary linear algebra and weak assumptions.
result Novel out-of-sample error upper and lower bounds with weak assumptions.
Paper uses EXIT analysis for community detection with side information.
problem Community detection in the presence of side information.
method Imported EXIT method from iterative decoding of error control codes.
result Predicts asymptotic phase transition and residual errors for community detection.
This paper extends the convergence analysis of Langevin Monte Carlo beyond Poincaré inequalities.
problem Analyzing convergence of Langevin Monte Carlo under various functional inequalities.
method Establishing upper and lower bounds for Langevin diffusions and LMC under weak Poincaré inequalities.
result Explicitly quantifies the effect of the initializer on the performance of LMC algorithm.
The study explores the strengths and weaknesses of models that generalize from weak to strong supervision.
problem Understanding the limitations and capabilities of models that generalize from weak to strong supervision.
method Theoretical analysis and experimental validation in both classification and regression settings.
result Theoretical bounds reveal the importance of strong generalization and calibration of the weak model and a careful balance in the training process.
Study approximates weak error for specific stochastic models with rough and Gaussian mean-reverting volatility.
problem Approximating weak error for specific stochastic models with rough and Gaussian mean-reverting volatility.
method Used Euler type scheme with integrated kernels to study weak convergence rate.
result Obtained weak convergence rate of min(3α−1,1) for discretised rough Ornstein-Uhlenbeck process and stochastic rough volatility model. Study on error rates for approximating rough volatility models.
problem Simulation of rough volatility models with fractional Brownian motion.
method Analysis of weak error rates for numerical schemes, focusing on fBm and cubic test functions.
result Convergence rates for approximations are (3H+21)∧1 for exact left-point discretization and H+21 for hybrid schemes. Study improves weak error estimates for rough volatility models.
problem Efficient numerical schemes for non-Markovian stochastic processes with rough volatility.
method Analyzes weak rates for a class of stochastic processes with rough stochastic volatility.
result Weak rate is of order min{3H+0.5, 1} for a large class of test functions.
Proposes a method to improve deep active learning for NER tasks.
problem Weaknesses of existing deep active learning algorithms in practice.
method Estimates error decay curves of feature-defined subsets to improve sampling efficiency and robustness.
result Significantly outperforms diversification-based methods for black-box NER taggers and makes sampling more robust to labeling noise.
Attention mechanism learns to focus on sparse tokens efficiently.
problem Detecting weak, rare, and sparsely located features in long sequences.
method Theoretical analysis and training of a single-layer attention classifier in a sparse-token classification model.
result A single-layer attention classifier can achieve vanishing test error with logarithmic signal strength growth, unlike linear classifiers requiring linear growth.
Study shows how neural networks generalize with minimal training data.
problem Understanding how neural networks generalize with limited data.
method Mean-field analysis of KL-regularized empirical risk minimization.
result Generalization error rate is O(1/n) for large n. We study learning latent models with multi-instance weak supervision.
problem Learning latent models with multi-instance weak supervision.
method Formulated as multi-instance Partial Label Learning (multi-instance PLL), proposed a necessary and sufficient condition for learnability, derived Rademacher-style error bounds.
result First theoretical study of multi-instance PLL with unknown transition function, aligns with empirical results but highlights scalability issues.
Improved multi-class AdaBoost algorithm with stronger weak learnability condition.
problem Multi-class classification problem with at least two labels.
method Recursive ensemble algorithm inspired by SAMME, strengthening weak learnability condition.
result Final hypothesis converges to correct label with probability 1 and generalization error bounds exponentially.
Improved machine learning models outperform their simpler counterparts by using imperfect labels.
problem Improving model performance using imperfect labels.
method Random feature ridge regression (RFRR) with a deterministic equivalent for excess test error.
result The student model can outperform the teacher model regardless of the teacher's scaling law, achieving the minimax optimal rate.
Proposes a constrained labeling method for weakly supervised learning.
problem Combining weak supervision signals while navigating misleading correlations.
method Randomized constrained labeling within a defined space.
result Randomized constrained labeling converges after few iterations and outperforms other methods.
The paper reveals three mechanisms for weak-to-strong generalization.
problem Understanding the mechanisms behind weak-to-strong generalization in imperfect labeling scenarios.
method Theoretical analysis of simple models including ridge regression and weighted ridge regression, and a nonlinear multi-index setting.
result A student model can compensate for a teacher's under-regularization and achieve lower test error.
Establishes a microstructural foundation for a rough log-normal volatility model.
problem Developing a robust model for financial volatility under microstructural effects.
method Introduced a sequence of order-driven financial market models with Poisson process arrivals and analyzed their convergence to a log-normal rough volatility model.
result Weak convergence of price-volatility process to a log-normal rough volatility model with established weak error rates.
New method improves Euler approximation for local stochastic volatility models.
problem Well-posedness of Euler approximation for local stochastic volatility models.
method Start with a well-defined Euler approximation to the formal McKean-Vlasov equation, followed by a half-step scheme.
result Showed weak order one for the Euler discretization, plus error terms.
In this paper we consider the cluster estimation problem under the Stochastic Block Model. We show that the semidefinite programming (SDP) formulation for this problem achieves an error rate that decays exponentially in the signal-to-noise ratio. The error bound implies weak recovery in the sparse graph regime with bou…
Hybrid model improves weather forecasting accuracy.
problem Combining physics and machine learning for better weather predictions.
method Offline and online training of a neural network to correct model errors in the IFS.
result The hybrid model reduces forecast errors and improves accuracy.
We recall the Chernoff-Marsden definition of weak symplectic structure and give a rigorous treatment of the functional analysis and geometry of weak symplectic Banach spaces. We define the Maslov index of a continuous path of Fredholm pairs of Lagrangian subspaces in continuously varying Banach spaces. We derive basic …
Optimizes shapes on non-standard manifolds.
problem Optimization on non-standard infinite-dimensional manifolds.
method Develops gradient descent on weak Riemannian manifolds.
result Establishes foundational properties for optimization on various weak Riemannian manifolds.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
New research shows label refinement and weak training have limitations for aligning LLMs.
problem Limitations of refinement methods for aligning large language models.
method Analyzed probabilistic assumptions and alternative approaches to label refinement and weak training.
result Label refinement and weak training suffer from irreducible error, leaving a performance gap.
In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis. Oja's iteration maintains a running estimate of the true principal component from streaming data and enjoys less tempo…
Topological autoencoders preserve complex structures in latent spaces.
problem Preserving topological structures in latent representations of autoencoders.
method Using persistent homology, a topological data analysis technique, to calculate topological signatures of input and latent spaces and derive a differentiable topological loss term.
result Our approach preserves multi-scale connectivity information in latent representations, leading to favorable latent representations on synthetic and real-world data.
We provide sharp empirical estimates of expectation, variance and normal approximation for a class of statistics whose variation in any argument does not change too much when another argument is modified. Examples of such weak interactions are furnished by U- and V-statistics, Lipschitz L-statistics and various error f…
Gradient descent on shallow neural networks achieves near-optimal generalization error.
problem Optimizing shallow neural networks with minimal width for generalization and stability.
method Gradient descent in the interpolating regime with minimal width.
result Gradient descent achieves near-optimal generalization error with minimal width.
Boosting improves accuracy by combining weak learners into a voting classifier.
problem Boosting's theoretical performance is sub-optimal, especially for voting classifiers.
method Proposes a randomized boosting algorithm that outputs voting classifiers with a single logarithmic dependency on sample size.
result Randomized boosting achieves a generalization error with a single logarithmic dependency on the sample size.
We establish the weak continuity of the Gauss-Coddazi-Ricci system for isometric embedding with respect to the uniform Lp-bounded solution sequence for p>2, which implies that the weak limit of the isometric embeddings of the manifold is still an isometric embedding. More generally, we establish a compensated comp…
In his seminal work, Schapire (1990) proved that weak classifiers could be improved to achieve arbitrarily high accuracy, but he never implied that a simple majority-vote mechanism could always do the trick. By comparing the asymptotic misclassification error of the majority-vote classifier with the average individual …
Self-training improves weak classifiers in mixture models.
problem Improving weak classifiers in mixture models.
method Iterative self-training algorithm using pseudolabels and unlabeled data.
result Self-training converts weak learners to strong learners in mixture models.
Study tests adequacy of FARIMA models with uncorrelated but non-independent errors.
problem Testing adequacy of FARIMA models with specific error characteristics.
method Derive asymptotic distributions of residual autocovariances and autocorrelations, propose self-normalization approach.
result Asymptotic distributions of modified portmanteau statistics for weak FARIMA models.
Weak diffusion priors can still perform well in inverse problems.
problem Using mismatched or low-fidelity diffusion priors in inverse problems.
method Extensive experiments and theoretical analysis combining Bayesian-consistency theory and local-correlation analysis.
result Weak priors succeed when measurements are highly informative, and they fail in other regimes.
Geometric analysis proves weak KAM solutions constant under specific conditions.
problem Conditions for weak KAM solutions to be constant.
method Geometric and differential analysis of Hamilton-Jacobi equations.
result Weak KAM solutions are constant if and only if the 1-form is harmonic.
Deep learning reduces noise in weak lensing mass maps using GANs.
problem Noise reduction in weak lensing mass maps.
method Generative adversarial networks (GANs) applied to Subaru Hyper Suprime-Cam data.
result GANs successfully reproduce non-Gaussian information in denoised maps, showing stronger cosmological dependence.
New bounds for generative models under weaker assumptions.
problem Establishing convergence guarantees for generative models under weak assumptions.
method Non-asymptotic 2-Wasserstein distance bounds for probability flow ODEs under weak log-concavity and Lipschitz continuity.
result Concrete convergence rates for generative models, including non-log-concave distributions.