New research shows label refinement and weak training have limitations for aligning LLMs.
problem Limitations of refinement methods for aligning large language models.
method Analyzed probabilistic assumptions and alternative approaches to label refinement and weak training.
result Label refinement and weak training suffer from irreducible error, leaving a performance gap.
ASTRA uses unlabeled data and weak rules to train deep models effectively.
problem Learning with weak supervision rules is challenging due to their heuristic and noisy nature.
method ASTRA framework that considers contextualized representations and pseudo-labels for unlabeled data, and a rule attention network to aggregate labels.
result Significant improvements over state-of-the-art baselines on text classification benchmarks.
Subset selection improves weak supervision performance.
problem Optimizing the use of weakly-labeled data.
method Combining pretrained data representations with the cut statistic for subset selection.
result Subset selection improves weak supervision performance by up to 19%.
Optimal algorithm converts weak to strong learner with less data.
problem Constructing a strong learner from a weak learner with minimal data.
method New algorithm that uses less training data than AdaBoost.
result Optimal sample complexity for converting weak to strong learner.
Random feature models can outperform a weak teacher with early stopping.
problem Generalization from a weak to a strong model in random feature networks.
method Random feature models, early stopping, proving weak-to-strong generalization.
result Random feature models can outperform a weak teacher with early stopping.
A new method for weakly supervised learning that improves model accuracy.
problem Training machine learning models with precise labels is expensive; weak supervision provides a low-cost alternative.
method Data consistent weak supervision algorithm that searches over classifiers to find plausible labelings, considering features of the training data and estimating labels for low/no coverage data.
result Empirically, the method significantly outperforms state-of-the-art weak supervision methods on text and image classification tasks.
New framework transfers latent knowledge from weak to strong models.
problem Aligning superhuman LLMs with human feedback.
method Transfer learning framework using refinement approach.
result Proves weak-to-strong generalization is possible.
Boosting weak learners to strong ones from aggregate labels is possible for LLP but not for MIL.
problem Boosting weak learners to strong ones from aggregate labels in learning from label proportions (LLP).
method Using a weak learner on large enough bags to obtain a strong learner for small bags in polynomial time.
result Boosting is possible for LLP but not for MIL.
Improves understanding of PWS by calculating influence of sources and data.
problem Understanding the influence of each component in PWS.
method Proposes source-aware Influence Function (IF) to decompose and calculate influence.
result Improves end model's generalization performance and identifies mislabeling.
Weak diffusion priors can still perform well in inverse problems.
problem Using mismatched or low-fidelity diffusion priors in inverse problems.
method Extensive experiments and theoretical analysis combining Bayesian-consistency theory and local-correlation analysis.
result Weak priors succeed when measurements are highly informative, and they fail in other regimes.
PLRM synthesizes labels from mismatched sources for better training sets.
problem Creating labeled training sets is a major challenge in machine learning.
method PLRM uses probabilistic modeling to synthesize labels from indirect supervision sources with different output spaces.
result PLRM outperforms baselines by 2%-9% on various tasks.
While there has been much recent progress using deep learning techniques to separate speech and music audio signals, these systems typically require large collections of isolated sources during the training process. When extending audio source separation algorithms to more general domains such as environmental monitori…
ProbKT uses probabilistic logical reasoning to train object detection models with weak supervision.
problem Training object detection models requires instance-level annotations, which are often unavailable.
method ProbKT, a framework based on probabilistic logical reasoning, uses arbitrary types of weak supervision.
result ProbKT leads to significant improvement and better generalization compared to existing baselines.
Training deep neural networks requires massive amounts of training data, but for many tasks only limited labeled data is available. This makes weak supervision attractive, using weak or noisy signals like the output of heuristic methods or user click-through data for training. In a semi-supervised setting, we can use a…
Proposes a constrained labeling method for weakly supervised learning.
problem Combining weak supervision signals while navigating misleading correlations.
method Randomized constrained labeling within a defined space.
result Randomized constrained labeling converges after few iterations and outperforms other methods.
New theory explains how strong models can learn from weak ones.
problem Learning from weak, incomplete, or incorrect labels.
method New bounds based on data distribution and student hypothesis class.
result Existing weak supervision theory fails to account for pseudolabel correction and coverage expansion.
Paper analyzes weak-to-strong generalization in CNNs, identifying data-scarce and data-abundant regimes.
problem Weak-to-strong generalization in CNNs trained on weak models.
method Formal analysis of gradient descent dynamics in data-scarce and data-abundant regimes.
result Identifies two regimes and distinct mechanisms of generalization in each.
As machine learning models continue to increase in complexity, collecting large hand-labeled training sets has become one of the biggest roadblocks in practice. Instead, weaker forms of supervision that provide noisier but cheaper labels are often used. However, these weak supervision sources have diverse and unknown a…
Interactive machine learning with weak supervision and pre-trained embeddings.
problem Training machine learning models with limited labeled data.
method Use pre-trained embeddings to define a distance function and extend source votes to nearby points.
result Significantly outperforms traditional weakly-supervised and fully-supervised methods.
Large learning rates cause oscillations in NN weights that improve generalization.
problem Improving generalization of neural networks trained with large learning rates.
method Theoretical analysis and feature-noise data generation model.
result Oscillating SGD with large learning rates benefits NN generalization by effectively learning weak features.
Improved machine learning models outperform their simpler counterparts by using imperfect labels.
problem Improving model performance using imperfect labels.
method Random feature ridge regression (RFRR) with a deterministic equivalent for excess test error.
result The student model can outperform the teacher model regardless of the teacher's scaling law, achieving the minimax optimal rate.
Framework detects fake news using weak social signals from multiple sources.
problem Lack of annotated data for early fake news detection.
method Jointly uses weak social signals and clean data to train deep neural networks in a meta-learning framework.
result Framework outperforms state-of-the-art baselines for early fake news detection.
We consider the task of training classifiers without labels. We propose a weakly supervised method---adversarial label learning---that trains classifiers to perform well against an adversary that chooses labels for training data. The weak supervision constrains what labels the adversary can choose. The method therefore…
Interactive weak supervision learns useful heuristics from user feedback.
problem Creating useful heuristics for large labeled datasets is tedious and subjective.
method Develops an interactive framework for learning heuristics from user feedback.
result Only a few feedback iterations are needed to train models without ground truth labels.
New method estimates model performance bounds without ground truth labels.
problem Evaluation of weakly supervised models without direct access to ground truth labels.
method Formulates model evaluation as a partial identification problem and uses Fréchet bounds for performance estimation.
result Derives accurate and computationally efficient bounds for key metrics like accuracy, precision, recall, and F1-score.
Self-training improves weak classifiers in mixture models.
problem Improving weak classifiers in mixture models.
method Iterative self-training algorithm using pseudolabels and unlabeled data.
result Self-training converts weak learners to strong learners in mixture models.
End-to-end approach for weak supervision improves downstream model performance.
problem Data-labeling bottleneck in machine learning applications.
method Directly learning the downstream model by maximizing its agreement with probabilistic labels generated from weak supervision sources.
result Improved performance over prior work in terms of downstream model performance and robustness.
ProBoost boosts probabilistic classifiers by focusing on uncertain samples.
problem Improving probabilistic classifiers through targeted learning.
method ProBoost uses epistemic uncertainty to select challenging samples, increasing their weight for subsequent learners.
result ProBoost significantly improves classifier performance, especially with few weak learners.
Unified approach for learning with weak labels across various tasks.
problem Learning with noisy or incomplete labels in diverse machine learning settings.
method Implicit posterior models for joint label inference.
result Unified training objective for various machine learning tasks.
Study shows how a strong model can learn a task's feature while retaining other capabilities.
problem How to align superhuman AI systems using weak-to-strong generalization.
method Two-layer neural networks, reward-model learning, multi-step SGD, feature learning.
result The strong model efficiently learns task features while retaining general capabilities.
Let X be an infinite Riemann surface equipped with its conformal hyperbolic metric such that the action of the covering group π1(X) on X~ is of the first kind-i.e., the surface X is equal to its convex core. We first prove that any geodesic lamination on X is nowhere dense. Given a fixed geodesic pant…
Improved multi-class AdaBoost algorithm with stronger weak learnability condition.
problem Multi-class classification problem with at least two labels.
method Recursive ensemble algorithm inspired by SAMME, strengthening weak learnability condition.
result Final hypothesis converges to correct label with probability 1 and generalization error bounds exponentially.
Unified framework for policy learning using weak supervision.
problem High-quality supervision is often infeasible or expensive in practice.
method Treat weak supervision as imperfect peer information and evaluate policies based on correlated agreement.
result Substantial performance improvements, especially in complex or noisy environments.
Paper presents a method to train NER models without labelled data using weak supervision.
problem Dealing with NER performance drop in new domains without labelled data.
method Weak supervision through automatic annotation and hidden Markov model integration.
result Improvement of about 7 percentage points in entity-level F1 scores. The study explores the strengths and weaknesses of models that generalize from weak to strong supervision.
problem Understanding the limitations and capabilities of models that generalize from weak to strong supervision.
method Theoretical analysis and experimental validation in both classification and regression settings.
result Theoretical bounds reveal the importance of strong generalization and calibration of the weak model and a careful balance in the training process.
Combines foundation models with weak supervision to improve NLP and video tasks.
problem Leveraging weak supervision with foundation models without labeled data.
method Liger, a combination of foundation model embeddings and weak supervision techniques.
result Liger outperforms existing weak supervision methods by 14.1 points on benchmark NLP and video tasks.
Since manually labeling training data is slow and expensive, recent industrial and scientific research efforts have turned to weaker or noisier forms of supervision sources. However, existing weak supervision approaches fail to model multi-resolution sources for sequential data, like video, that can assign labels to in…
Deep networks have gained immense popularity in Computer Vision and other fields in the past few years due to their remarkable performance on recognition/classification tasks surpassing the state-of-the art. One of the keys to their success lies in the richness of the automatically learned features. In order to get ver…
Weak correlations explain linear dynamics in deep learning models.
problem Understanding the linear structure in gradient-based learning algorithms.
method Characterization of weak correlations between derivatives and parameters.
result Weak correlations are the underlying principle for linearization in deep learning models.
New methods lift weak supervision to structured prediction, providing robustness guarantees.
problem Applying weak supervision techniques to structured prediction problems.
method Introducing pseudo-Euclidean embeddings, tensor decompositions, and invariants for consistent noise rate estimation.
result Generalization guarantees nearly identical to those for models trained on clean data.
Neural networks trained with actor-critic algorithms converge to ODEs under weak convergence analysis.
problem Challenges in convergence analysis due to changing data distributions in online learning.
method Geometric ergodicity of data samples, Poisson equation, weak convergence techniques.
result Actor and critic networks converge to solutions of ODEs with random initial conditions.
Faster weak supervision framework using triplet methods.
problem Computational inefficiency in weak supervision models.
method Closed-form solution for latent variable models, avoiding iterative methods.
result Orders of magnitude faster than previous approaches.
W2S FT often outperforms weak teachers due to low intrinsic dimensionality.
problem Understanding why weak-to-strong finetuning outperforms weak models.
method Analyzing W2S in ridgeless regression setting, focusing on variance reduction.
result Weak teacher's variance is inherited by strong student in shared feature subspace, reduced in discrepancy subspace.
Sharp analysis of knowledge distillation for high-dimensional regression.
problem Characterizing the risk of target models in high-dimensional settings.
method Sharp non-asymptotic bounds for ridgeless regression under model and distribution shifts.
result Identifies optimal surrogate models and reveals benefits and limitations of discarding weak features.
Expands weak supervision by allowing partial labels from multiple noisy sources.
problem Creating models without labeled data using heuristic labelers.
method Probabilistic generative model estimating partial label accuracies.
result Improved model accuracy on various tasks (8.6% on text, comparable to zero-shot methods on images).
In this paper, we propose a method for training neural networks when we have a large set of data with weak labels and a small amount of data with true labels. In our proposed model, we train two neural networks: a target network, the learner and a confidence network, the meta-learner. The target network is optimized to…
The paper reveals three mechanisms for weak-to-strong generalization.
problem Understanding the mechanisms behind weak-to-strong generalization in imperfect labeling scenarios.
method Theoretical analysis of simple models including ridge regression and weighted ridge regression, and a nonlinear multi-index setting.
result A student model can compensate for a teacher's under-regularization and achieve lower test error.
Mitigates bias in weakly supervised datasets.
problem Bias in weakly supervised datasets.
method Proposes a counterfactual fairness-based technique to mitigate bias.
result Improves accuracy by up to 32% while reducing demographic parity gap by 82.5%.