Formalizes weak and strong verification for LLMs, controlling errors without assumptions.
problem Balancing cost and reliability in reasoning with LLMs.
method Formalizes weak-strong verification policies, introduces metrics, develops online algorithm.
result Optimal policies admit a two-threshold structure, and calibration and sharpness govern value of weak verifiers.
The study explores the strengths and weaknesses of models that generalize from weak to strong supervision.
problem Understanding the limitations and capabilities of models that generalize from weak to strong supervision.
method Theoretical analysis and experimental validation in both classification and regression settings.
result Theoretical bounds reveal the importance of strong generalization and calibration of the weak model and a careful balance in the training process.
Study approximates weak error for specific stochastic models with rough and Gaussian mean-reverting volatility.
problem Approximating weak error for specific stochastic models with rough and Gaussian mean-reverting volatility.
method Used Euler type scheme with integrated kernels to study weak convergence rate.
result Obtained weak convergence rate of min(3α−1,1) for discretised rough Ornstein-Uhlenbeck process and stochastic rough volatility model. Study on error rates for approximating rough volatility models.
problem Simulation of rough volatility models with fractional Brownian motion.
method Analysis of weak error rates for numerical schemes, focusing on fBm and cubic test functions.
result Convergence rates for approximations are (3H+21)∧1 for exact left-point discretization and H+21 for hybrid schemes. Study improves weak error estimates for rough volatility models.
problem Efficient numerical schemes for non-Markovian stochastic processes with rough volatility.
method Analyzes weak rates for a class of stochastic processes with rough stochastic volatility.
result Weak rate is of order min{3H+0.5, 1} for a large class of test functions.
This article analyzes the weak error of SGD optimization schemes.
problem Analyzing the error in SGD optimization schemes with respect to a test function.
method Weak error analysis for SGD type optimization schemes.
result The weak error decays at the same speed as in the strong sense.
Improved multi-class AdaBoost algorithm with stronger weak learnability condition.
problem Multi-class classification problem with at least two labels.
method Recursive ensemble algorithm inspired by SAMME, strengthening weak learnability condition.
result Final hypothesis converges to correct label with probability 1 and generalization error bounds exponentially.
Proposes a constrained labeling method for weakly supervised learning.
problem Combining weak supervision signals while navigating misleading correlations.
method Randomized constrained labeling within a defined space.
result Randomized constrained labeling converges after few iterations and outperforms other methods.
Improved machine learning models outperform their simpler counterparts by using imperfect labels.
problem Improving model performance using imperfect labels.
method Random feature ridge regression (RFRR) with a deterministic equivalent for excess test error.
result The student model can outperform the teacher model regardless of the teacher's scaling law, achieving the minimax optimal rate.
Diffusion approximation provides weak approximation for stochastic gradient descent algorithms in a finite time horizon. In this paper, we introduce new tools motivated by the backward error analysis of numerical stochastic differential equations into the theoretical framework of diffusion approximation, extending the …
New theory explains how strong models can learn from weak ones.
problem Learning from weak, incomplete, or incorrect labels.
method New bounds based on data distribution and student hypothesis class.
result Existing weak supervision theory fails to account for pseudolabel correction and coverage expansion.
Improved approximations for rough Heston model reduce errors.
problem Lack of Markov and semimartingale properties in rough Heston model.
method Markovian approximations with weak error analysis.
result Super-polynomial convergence of new approximations.
Establishes a microstructural foundation for a rough log-normal volatility model.
problem Developing a robust model for financial volatility under microstructural effects.
method Introduced a sequence of order-driven financial market models with Poisson process arrivals and analyzed their convergence to a log-normal rough volatility model.
result Weak convergence of price-volatility process to a log-normal rough volatility model with established weak error rates.
New method improves Euler approximation for local stochastic volatility models.
problem Well-posedness of Euler approximation for local stochastic volatility models.
method Start with a well-defined Euler approximation to the formal McKean-Vlasov equation, followed by a half-step scheme.
result Showed weak order one for the Euler discretization, plus error terms.
Improves label propagation for weakly supervised learning.
problem Reducing the need for labeled data in machine learning.
method Label Propagation with Weak Supervision (LPA) analysis.
result Demonstrated improvements over existing methods on weakly supervised classification tasks.
New research shows label refinement and weak training have limitations for aligning LLMs.
problem Limitations of refinement methods for aligning large language models.
method Analyzed probabilistic assumptions and alternative approaches to label refinement and weak training.
result Label refinement and weak training suffer from irreducible error, leaving a performance gap.
We provide sharp empirical estimates of expectation, variance and normal approximation for a class of statistics whose variation in any argument does not change too much when another argument is modified. Examples of such weak interactions are furnished by U- and V-statistics, Lipschitz L-statistics and various error f…
The significance of the study of the theoretical and practical properties of AdaBoost is unquestionable, given its simplicity, wide practical use, and effectiveness on real-world datasets. Here we present a few open problems regarding the behavior of "Optimal AdaBoost," a term coined by Rudin, Daubechies, and Schapire …
A network may have weak signals and severe degree heterogeneity, and may be very sparse in one occurrence but very dense in another. SCORE (Jin, 2015) is a recent approach to network community detection. It accommodates severe degree heterogeneity and is adaptive to different levels of sparsity, but its performance for…
Boosting improves accuracy by combining weak learners into a voting classifier.
problem Boosting's theoretical performance is sub-optimal, especially for voting classifiers.
method Proposes a randomized boosting algorithm that outputs voting classifiers with a single logarithmic dependency on sample size.
result Randomized boosting achieves a generalization error with a single logarithmic dependency on the sample size.
We establish the weak continuity of the Gauss-Coddazi-Ricci system for isometric embedding with respect to the uniform Lp-bounded solution sequence for p>2, which implies that the weak limit of the isometric embeddings of the manifold is still an isometric embedding. More generally, we establish a compensated comp…
In his seminal work, Schapire (1990) proved that weak classifiers could be improved to achieve arbitrarily high accuracy, but he never implied that a simple majority-vote mechanism could always do the trick. By comparing the asymptotic misclassification error of the majority-vote classifier with the average individual …
Self-training improves weak classifiers in mixture models.
problem Improving weak classifiers in mixture models.
method Iterative self-training algorithm using pseudolabels and unlabeled data.
result Self-training converts weak learners to strong learners in mixture models.
New bounds for generative models under weaker assumptions.
problem Establishing convergence guarantees for generative models under weak assumptions.
method Non-asymptotic 2-Wasserstein distance bounds for probability flow ODEs under weak log-concavity and Lipschitz continuity.
result Concrete convergence rates for generative models, including non-log-concave distributions.
Efficiently plans large MDPs with weak function approximations.
problem Planning in large MDPs with limited function approximation capabilities.
method Uses linear value function approximation with weak requirements and a generative oracle.
result Produces almost-optimal actions for any state with polynomial computation time.
We consider the weak detection problem in a rank-one spiked Wigner data matrix where the signal-to-noise ratio is small so that reliable detection is impossible. We propose a hypothesis test on the presence of the signal by utilizing the linear spectral statistics of the data matrix. The test is data-driven and does no…
W2S FT often outperforms weak teachers due to low intrinsic dimensionality.
problem Understanding why weak-to-strong finetuning outperforms weak models.
method Analyzing W2S in ridgeless regression setting, focusing on variance reduction.
result Weak teacher's variance is inherited by strong student in shared feature subspace, reduced in discrepancy subspace.
New proof shows how to identify DAGs with weakly increasing errors.
problem Identifying the true DAG in models with weakly increasing error variances.
method Minimum-trace DAG method and hill climbing algorithm with R2R neighborhood.
result Hill climbing algorithm without strict local optima under weakly increasing error variances.
Gradient descent benefits from tangent kernel advantages under specific conditions.
problem Comparing gradient descent with tangent kernel methods in learning.
method Analysis of gradient descent and tangent kernel methods under different conditions.
result Gradient descent can achieve small error only if tangent kernel methods have a non-trivial advantage, but this advantage can be very small.
Weak supervision is a popular method for building machine learning models without relying on ground truth annotations. Instead, it generates probabilistic training labels by estimating the accuracies of multiple noisy labeling sources (e.g., heuristics, crowd workers). Existing approaches use latent variable estimation…
We introduce cylindrical projections to simulate infinite-dimensional occupation flows of diffusions.
problem Computational intractability of infinite-dimensional occupation flows of diffusions.
method Introduce cylindrical projections to approximate the occupation flow via a finite-dimensional system.
result Strong convergence of cylindrical projections to the initial process with derived rates.
Unified approach for multicalibration in weakly supervised learning.
problem Existing multicalibration methods require clean input-label pairs, which are unavailable in weakly supervised learning.
method Developed estimators and post-hoc correction methods for multicalibration under weak supervision.
result Unified framework for estimating and correcting multicalibration under weak supervision with finite-sample guarantees.
Mixed-SCORE+ improves community detection in weak signal networks.
problem Detecting communities in weak signal networks.
method Proposes Mixed-SCORE+ combining properties of Mixed-SCORE and SCORE+.
result Significantly improves detection error rates on Polblogs and weak signal networks.
We consider the task of training classifiers without labels. We propose a weakly supervised method---adversarial label learning---that trains classifiers to perform well against an adversary that chooses labels for training data. The weak supervision constrains what labels the adversary can choose. The method therefore…
An active learner is given a hypothesis class, a large set of unlabeled examples and the ability to interactively query labels to an oracle of a subset of these examples; the goal of the learner is to learn a hypothesis in the class that fits the data well by making as few label queries as possible. This work addresses…
Paper relaxes symmetry conditions for universal feature selection in noisy data.
problem Feature selection in noisy data with weak symmetry.
method Developed a universal feature selection framework using singular value decomposition of canonical dependence matrix.
result Selected features achieve asymptotically optimal error exponents up to a residual term.
This work considers the problem of modified portmanteau tests for testing the adequacy of FARIMA models under the assumption that the errors are uncorrelated but not necessarily independent (i.e. weak FARIMA). We first study the joint distribution of the least squares estimator and the noise empirical autocovariances. …
Develops weak PINNs for efficient manifold solutions of hyperbolic equations.
problem Challenges in approximating weak solutions of nonlinear hyperbolic equations on manifolds.
method Introduces a novel weak PINN (wPINN) formulation on manifolds leveraging well-posedness theory.
result Demonstrates efficient approximation of entropy solutions on manifolds with a complexity independent of ambient space dimension.
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and error functionals of l2-regularized algorithms.
The paper improves confidence intervals for test error using cross-validation.
problem Improving confidence intervals for test error in machine learning.
method Develops central limit theorems and consistent estimators for cross-validation.
result Provides asymptotically-exact confidence intervals and hypothesis tests.
Improved volatility models for option pricing with weak error rates.
problem Improving volatility models to fit market data better.
method Developed a weak convergence analysis for the Euler method applied to linear rough volatility models.
result Proved weak convergence rates of 1/2 + H for linear models and 1 for quadratic payoffs.
Existing deep active learning algorithms achieve impressive sampling efficiency on natural language processing tasks. However, they exhibit several weaknesses in practice, including (a) inability to use uncertainty sampling with black-box models, (b) lack of robustness to labeling noise, and (c) lack of transparency. I…
Study rough volatility models using path-dependent PDEs and fractional Brownian motions.
problem Modeling and analyzing rough volatility in financial markets.
method Showed conditional expectations are unique classical solutions to path-dependent PDEs derived from functional Itô formula. Leverage these to study weak rates of convergence for discretized stochastic integrals.
result Obtained optimal weak error rates for approximating log-stock prices in rough volatility models.
Conformal C2ST turns weak classifiers into reliable two-sample tests.
problem Determining if two distributions are identical using weak classifiers.
method Developed conformal variants of the C2ST to convert any classifier scores into reliable p-values.
result Even weak classifiers can yield powerful and reliable two-sample tests.
The rough Heston model emerges from scaling bivariate INAR processes, linking microstructure to option pricing.
problem Modeling and pricing financial options with heavy-tailed and cumulative processes.
method Scaling limit of bivariate INAR processes converging to rough Heston model, explicit formulas linking asymmetry parameters to volatility.
result Weak-error estimates and FFT-accelerated simulation for European and path-dependent options.
WENDy now estimates nonlinear ODEs with noisy data.
problem Estimating parameters of nonlinear ODEs with noisy data.
method WENDy-MLE algorithm for maximum likelihood estimation of nonlinear-in-parameters ODEs.
result WENDy-MLE outperforms other methods in accuracy, speed, and domain of convergence.
Study on consistency of ML methods for moving objects in non-stationary environments.
problem Consistency of machine learning methods for moving objects in non-stationary environments.
method Least squares, ridge regression, and ℓs-penalized least squares methods under non-stationary spatial-temporal sampling. result Consistency and asymptotic normality of the estimates under weak conditions.
Boosting theory extended to handle cost-sensitive and multi-objective losses.
problem Real-world prediction problems with different error costs.
method Developed a comprehensive theory of cost-sensitive and multi-objective boosting.
result Established a dichotomy for binary classification and a more intricate landscape for multiclass settings.