Paper tackles high-accuracy list-decodable learning for mean estimation.
problem Estimate the mean of a distribution with a small fraction coming from a nice distribution.
method Developed a novel proof of identifiability and a new algorithmic approach.
result Achieved high-accuracy guarantees for list-decodable mean estimation.
New algorithm for robust mean estimation in high dimensions with nearly-PCA time complexity.
problem Robust mean estimation in high-dimensional data with a minority of contaminated data.
method List-decodable mean estimation algorithm using a novel soft downweighting method, SIFT, and a Ky Fan matrix multiplicative weights procedure.
result Optimal sample complexity and error rate for list-decodable mean estimation with runtime nearly matching PCA.
New algorithm recovers mixture means even with many outliers.
problem Estimating mixture means when outliers overwhelm small groups.
method Proposes an algorithm for robust mixture learning with minimal list-size overhead.
result Order-optimal error guarantees for each mixture mean.
Algorithm estimates covariance from noisy data efficiently.
problem Estimating covariance from a noisy set of points.
method Spectral techniques for list-decodable covariance estimation.
result Efficient algorithm with poly(1/α) sample and time complexity.
Polynomial-time algorithm for list-decodable linear regression with batches.
problem Efficiently decoding linear regression with a fraction of adversarial data.
method Polynomial time algorithm using batches of i.i.d. samples.
result Returns a list of size O(1/α^2) with one item close to true parameter.
Boosting improves accuracy with fewer calls to weak learners for certain concept classes.
problem Improving accuracy of learning algorithms with limited weak learner calls.
method Combines boosting and list-decodable codes to achieve better performance for specific concept classes.
result A new boosting algorithm that achieves strong learning with fewer calls to weak learners and additional samples.
New method for estimating sparse means in noisy data.
problem Estimating the mean of a sparse distribution in the presence of outliers.
method Difference-of-Pairs Filtering technique for list-decodable sparse mean estimation.
result First sample and computationally efficient algorithm for list-decodable sparse mean estimation.
Statistical Query lower bound shows difficulty in list-decodable linear regression.
problem List-decodable linear regression with adversarial corruption.
method Statistical Query (SQ) lower bound analysis.
result Lower bound of d p o l y ( 1 / α ) d^{\mathrm{poly}(1/α)} d poly ( 1/ α ) for list-decodable linear regression. Polynomial-time algorithm for estimating covariance in corrupted Gaussian data.
problem Estimating covariance in data with up to 1-α fraction of adversarial corruptions.
method Uses low-degree sum-of-squares certificates for anti-concentration and hypercontractivity.
result Outputs a list of candidate parameters with high probability containing a nearly correct covariance.
We give the first polynomial-time algorithm for robust regression in the list-decodable setting where an adversary can corrupt a greater than 1 / 2 1/2 1/2 fraction of examples. For any α < 1 α< 1 α < 1 , our algorithm takes as input a sample { ( x i , y i ) } i ≤ n \{(x_i,y_i)\}_{i \leq n} {( x i , y i ) } i ≤ n of n n n linear equations where α n αn α n of the equations satisfy $y_i = \l…
New method certifies anti-concentration for various non-Gaussian distributions.
problem Efficiently certifying anti-concentration for non-Gaussian distributions.
method Sum-of-Squares relaxation of integer program for anti-concentration.
result Quasi-polynomial time certificates for non-Gaussian distributions.
A simple model explains phase transition in large language models.
problem Understanding the emergence of abilities in large language models.
method Modeling LLM as a sequence-to-sequence random function and using a list decoder.
result A critical threshold exists where the expected number of erroneous sequences grows exponentially.
New method estimates mean from noisy data with few outliers.
problem Estimate mean from noisy data with few outliers.
method Iterative multi-filtering approach.
result Achieves near-optimal error with practical efficiency.
New algorithm clusters data with almost-linear time, robust to corruption.
problem Clustering mixtures of distributions with adversarial corruption.
method List-decodable mean estimation, robust mean estimation, iterative multi-filtering.
result First almost-linear time algorithms for clustering mixtures of k well-behaved distributions.
New algorithm for batch list-decodable linear regression with stronger guarantees.
problem Efficiently list-decoding linear regression with a fraction of corrupted batches.
method Uses higher-order moments and Sum-of-Squares (SoS) certification to achieve better guarantees.
result Achieves substantially smaller minimum batch size and final error, with optimal list size.
Paper tackles MLR prediction error without assuming realizable models.
problem Prediction error in mixture of linear regressions without realizable assumptions.
method Developed algorithms for list-decoding MLR predictions and minimized empirical risk.
result Alternating minimization algorithm finds best fit lines in non-realizable settings.
Study on list learning with noisy data, showing limits and some learnable cases.
problem Learning from noisy data in a list learning context.
method Inspired by coding theory, extends list learning model to study sparse conjunctions and parities/majors.
result Sparse conjunctions can be efficiently list learned under certain conditions, but parities and majors cannot be efficiently learned.
In list-decodable subspace recovery, the input is a collection of n n n points α n αn α n (for some α ≪ 1 / 2 α\ll 1/2 α ≪ 1/2 ) of which are drawn i.i.d. from a distribution D \mathcal{D} D with a isotropic rank r r r covariance Π ∗ Π_* Π ∗ (the \emph{inliers}) and the rest are arbitrary, potential adversarial outliers. The goal is to recover a $O(1/α)…
Study shows private learning of mixtures of Gaussians is possible with polynomial samples.
problem Estimating mixtures of Gaussians under differential privacy constraints.
method Developed a new framework for privately learning mixtures of Gaussians without structural assumptions.
result Polynomial number of samples (poly(k,d,1/α,1/ε,log(1/δ))) sufficient for estimation up to total variation distance α with (ε, δ)-DP.
Proves subgaussian distributions are SoS-certifiably subgaussian, enabling efficient algorithms for various statistical tasks.
problem Efficiently learning from subgaussian distributions in high dimensions.
method Universal constant C C C and polynomial sum of squares (SoS) approach. result Proves subgaussian distributions are SoS-certifiably subgaussian.
New algorithm reduces contamination in supervised learning.
problem Learning with contamination in supervised learning.
method Iterative polynomial filtering.
result Efficient learning of functions with contamination.
Smoothed analysis shows that many classes become learnable from positive-only samples.
problem Learning from positive-only samples is challenging due to negative results in worst-case settings.
method Smoothed analysis of positive-only learning, assuming samples from a reference distribution smooth with respect to the true distribution.
result All VC classes become learnable in the smoothed model with O ( V C / ε 2 ) O(VC/ε^2) O ( V C / ε 2 ) positive samples for ε ε ε classification error. Efficiently estimates common mean in high-dimensional data with noisy subset.
problem Estimating common mean in high-dimensional data with noisy subset.
method Iterative refinement strategy with rejection sampling and dimension-reduction.
result Achieves near-optimal error of f ( α , N ) + D / ( α N ) f(α,N) + \sqrt{D/(αN)} f ( α , N ) + D / ( α N ) . This paper bridges outlier-robust estimation in robotics and computer vision with robust statistics.
problem Outlier-robust estimation for geometric perception in robotics and computer vision.
method Adapting and extending robust linear regression and list-decodable regression to non-convex domains and vector-valued measurements.
result Performance guarantees for modern estimation algorithms in the presence of outliers.
Meta-learning adapts models for unseen tasks across AI, robotics, and NLP.
problem Adapting models to unseen tasks efficiently and accurately.
method Black-box, metric-based, layered, and Bayesian approaches.
result Meta-learning enhances model generalization and adaptation to unseen tasks.
Meta-learning improves neural networks by adapting learning algorithms.
problem Conventional AI approaches solve tasks from scratch, but meta-learning aims to improve the learning algorithm.
method Meta-learning adapts a learning algorithm based on multiple learning episodes.
result Meta-learning can tackle deep learning challenges like data and computation bottlenecks.
Survey explores how transfer learning improves deep reinforcement learning.
problem Challenges in reinforcement learning efficiency and effectiveness.
method Categorizes and analyzes transfer learning approaches.
result Transfer learning enhances reinforcement learning performance.
Study Whittle index learning algorithms for restless bandits with constant stepsizes.
problem Optimizing decisions in restless multi-armed bandits with constant stepsizes.
method Developed Q-learning algorithms with constant stepsizes for index learning in restless bandits, extending to DQN and function approximations.
result The algorithms learn the Whittle index effectively.
Poisson learning doesn't solve graph semi-supervised learning issues.
problem Global information loss in graph-based semi-supervised learning.
method Poisson learning is Laplace regularization with thresholding.
result Poisson learning cannot overcome the global information loss problem.
New unsupervised learning technique learns independent kernels for better machine learning tasks.
problem Improving unsupervised representation learning for machine learning tasks.
method Stacking convolutional transforms using alternating proximal minimization scheme.
result DCTL outperforms shallow version CTL on benchmark datasets.
Meta-learning helps models learn quickly from few samples.
problem Deep learning requires many samples, which are hard to get.
method Meta-learning optimizes models to adapt quickly to new tasks.
result Meta-learning can improve model efficiency and adaptability.
New self-imitation learning method improves performance in continuous control tasks.
problem Improving off-policy learning in continuous control tasks.
method Proposes a n-step lower bound to generalize lower-bound Q-learning and introduces a new family of self-imitation learning algorithms.
result n-step lower bound Q-learning achieves a better trade-off between bias and contraction rate, leading to improved performance.
Deep reinforcement learning finds optimal learning policies for adaptive systems.
problem Finding individualized learning plans for learners with unknown latent traits.
method Formulated as a Markov decision process, applied deep Q-learning with a transition model estimator.
result The algorithm efficiently discovers optimal learning policies with small data sets.
Unified framework explains all types of learning, including brain.
problem Lack of clear explanation for deep learning success.
method Constructing a learning principle that equates all learning to probability estimation.
result Unified understanding of learning across different fields.
Cyclical learning rates improve DRL performance without manual tuning.
problem Manual hyperparameter tuning in DRL is time-consuming and error-prone.
method Proposes cyclical learning rates for DRL problems.
result Cyclical learning achieves similar or better results than fixed learning rates.
Study batch reinforcement learning methods for personalized medical treatments.
problem Batch reinforcement learning for personalized medical treatments.
method Direct policy learning and model-based learning approaches.
result Model-based learning is impossible with finite model classes but feasible with relaxed conditions.
A new meta-meta classification method tackles few-shot learning tasks.
problem Learning with limited data in small-data settings.
method Designing an ensemble of learners for a large set of problems, then learning how to combine them for a new problem.
result Meta-meta classification outperforms traditional meta-learning and ensembling approaches in one-shot learning tasks.
The paper proposes a learning algorithm that improves adaptability and generalization.
problem Improving adaptability and generalization in learning models.
method Learning to meta-learn by meta-finetuning on related tasks before adapting to specific tasks.
result Learning to meta-learn improves adaptability and generalization across various tasks.
A common strategy in modern learning systems is to learn a representation that is useful for many tasks, a.k.a. representation learning. We study this strategy in the imitation learning setting for Markov decision processes (MDPs) where multiple experts' trajectories are available. We formulate representation learning …
This paper surveys meta-learning, online, and continual learning.
problem Combining and understanding meta-learning, online, and continual learning.
method Organizing various problem settings using consistent terminology and formal descriptions.
result Fosters further advancements in meta-learning, online, and continual learning.
Relational logistic regression (RLR) is a representation of conditional probability in terms of weighted formulae for modelling multi-relational data. In this paper, we develop a learning algorithm for RLR models. Learning an RLR model from data consists of two steps: 1- learning the set of formulae to be used in the m…
Survey on curriculum learning for reinforcement learning.
problem Efficiently learning complex tasks in reinforcement learning.
method Framework for curriculum learning in reinforcement learning.
result Survey and classification of existing CL methods.
Contrastive learning works well with redundant data views.
problem Improving unsupervised learning with redundant data views.
method Theoretical analysis of contrastive learning in a multi-view setting.
result Linear functions of learned representations are nearly optimal on prediction tasks when views are redundant.
Paper analyzes iterative learning for concept classes and learns half-spaces.
problem Learning concept classes efficiently with iterative learners.
method Analyzes various settings of iterative learning and provides a constructive algorithm for half-spaces.
result Constructive iterative algorithm for learning half-spaces from informant.
This paper introduces Dex, a reinforcement learning environment toolkit specialized for training and evaluation of continual learning methods as well as general reinforcement learning problems. We also present the novel continual learning method of incremental learning, where a challenging environment is solved using o…
Study shows cliff-learning in transfer learning from foundation models.
problem Data-scaling of transfer learning from foundation models in low data regimes.
method Investigation of cliff-learning phenomenon through foundation-model analysis and toy models.
result Cliff-learning reflects compatibility between priors and tasks.
Meta-strategy learns tuning parameters for online learning methods.
problem Difficulty in setting tuning parameters for online learning methods.
method Meta-learning approach to learn parameters from past tasks.
result Meta-strategy improves on learning each task in isolation.
We discuss a general method to learn data representations from multiple tasks. We provide a justification for this method in both settings of multitask learning and learning-to-learn. The method is illustrated in detail in the special case of linear feature learning. Conditions on the theoretical advantage offered by m…