A new learning method uses data to learn from large model sets.
problem Learning with large sets of candidate models where uniform convergence is hard.
method Data-dependent learning that incorporates empirical data less reliant on prior assumptions.
result Demonstrates improved generalization in various learning assumptions.
For many interesting tasks, such as medical diagnosis and web page classification, a learner only has access to some positively labeled examples and many unlabeled examples. Learning from this type of data requires making assumptions about the true distribution of the classes and/or the mechanism that was used to selec…
There is a large body of work on convergence rates either in passive or active learning. Here we first outline some of the main results that have been obtained, more specifically in a nonparametric setting under assumptions about the smoothness of the regression function (or the boundary between classes) and the margin…
New assumptions and algorithm solve offline two-player zero-sum Markov games.
problem Solving offline two-player zero-sum Markov games under insufficient assumptions.
method Proposed unilateral concentration assumption and pessimism-type algorithm.
result Algorithm efficiently learns Nash equilibrium under unilateral concentration.
There is a large body of work on convergence rates either in passive or active learning. Here we outline some of the results that have been obtained, more specifically in a nonparametric setting under assumptions about the smoothness and the margin noise. We also discuss the relative merits of these underlying assumpti…
The paper defines conditions for learning causal graphs from data with unobserved variables.
problem Learning causal graphs from data with unobserved variables.
method Formalizes constraint-based structure learning algorithms under conditions and assumptions.
result Natural family of algorithms output Markov equivalent graphs to the causal graph under faithfulness assumption.
Machine learning experiments show IID assumption is flawed for bathymetry editing.
problem Flawed IID assumption in machine learning for bathymetry editing.
method Real-world computer-assisted labeling task, IID assumption analysis.
result Common random split leads to poor performance in machine learning.
New active learning algorithm adapts to data without strict assumptions.
problem Efficiently label data with expensive labeling costs.
method Nonparametric adaptive active learning under local smoothness condition.
result Achieves minimax rate of convergence, performs almost as well as best non-adaptive algorithms.
Emputation learns imputation models guided by missingness assumptions.
problem Learning imputation models for missing data given observed data.
method Guided by specific missingness assumptions, Emputation trains a deep generative model to learn the extrapolation distribution of missing variables.
result The population minimizer of the emputation risk recovers the target extrapolation distribution under various identification assumptions.
Nowadays, machine learning methods have been widely used in stock prediction. Traditional approaches assume an identical data distribution, under which a learned model on the training data is fixed and applied directly in the test data. Although such assumption has made traditional machine learning techniques succeed i…
Semi-supervised learning is a setting in which one has labeled and unlabeled data available. In this survey we explore different types of theoretical results when one uses unlabeled data in classification and regression tasks. Most methods that use unlabeled data rely on certain assumptions about the data distribution.…
This paper introduces PM and PMLP to enhance SSL by considering probability density and cluster assumptions.
problem Insufficient utilization of unlabeled data in SSL.
method Introduces PM to discern similarity and PMLP to consider cluster assumption in label propagation.
result PMLP outperforms other methods in SSL tasks.
Improves online learning algorithms for functional models with capacity assumptions.
problem Convergence rates of online stochastic gradient descent algorithms for functional linear models.
method Characterizations of slope function regularity, kernel space capacity, and sampling process covariance operator.
result Capacity assumptions can alleviate saturation of convergence rates as function regularity increases.
Bagging stabilizes models without distributional assumptions.
problem Stability of machine learning models without distributional assumptions.
method Derives a finite-sample guarantee on bagging stability for any model.
result Guarantee applies to many bagging variants and is optimal.
GALA framework learns invariant graph representations via environment augmentation with minimal assumptions.
problem Learning invariant graph representations from different environments without additional assumptions.
method Developed GALA framework with minimal assumptions of variation sufficiency and consistency. Uses an assistant model to differentiate graph environment changes.
result Extracting maximally invariant subgraphs to proxy predictions identifies underlying invariant subgraphs for successful out-of-distribution generalization.
This paper broadens contrastive learning for disentangled representations without strict data distribution assumptions.
problem Learning disentangled representations from data with specific assumptions.
method Extends theoretical guarantees for disentanglement to a broader family of contrastive methods, relaxing data distribution assumptions.
result Identifiability of true latents for four contrastive losses proved without common independence assumptions.
The paper proposes using low rank assumption to improve causal structure learning in DAGs.
problem Challenges in learning causal structures in high-dimensional, non-sparse DAGs.
method Exploits low rank assumption of DAG adjacency matrix to adapt causal structure learning methods.
result Maximum rank is highly related to hubs, suggesting low rank for scale-free networks.
Characterizes concept classes for optimistic online learning.
problem Understanding minimal assumptions for online learnability.
method Investigates two questions about concept classes' learnability.
result Characterizes all concept classes for optimistically universal online learnability.
Paper tackles offline RL with weak assumptions on both function classes and data coverage.
problem Achieve sample-efficient offline RL with weak assumptions on both factors.
method Simple algorithm based on primal-dual formulation of MDPs, with density-ratio function modeling dual variables.
result Polynomial sample complexity achieved under realizability and single-policy concentrability.
Algorithm mines environment assumptions for cyber-physical systems.
problem Modeling and verifying complex cyber-physical systems.
method Supervised learning to mine STL formulas for input signals.
result Algorithm learns both the structure and constants of STL formulas.
This paper analyzes the convergence of Federated Average under relaxed assumptions.
problem Lack of theoretical analysis for Federated Average under assumptions beyond smoothness.
method Relaxing assumptions of strong smoothness to semi-smoothness and semi-Lipschitz properties, and introducing a bound on the gradient.
result Provides a theoretical convergence study on Federated Learning under new assumptions.
CausalCompass evaluates TSCD robustness under violations of modeling assumptions.
problem Widespread adoption of TSCD is hindered by untestable causal assumptions and lack of robustness evaluation.
method CausalCompass is a flexible benchmark framework for assessing TSCD robustness under violations of modeling assumptions.
result No single method consistently attains optimal performance across all settings, but deep learning-based methods perform well.
New offline RL method works with limited data and function approximators.
problem Sample efficiency with limited data and weak function approximators.
method Pessimistic algorithm based on version space formed by marginalized importance sampling (MIS), with gap assumption.
result Guarantees sample efficiency for simple algorithm under specific assumptions.
New research challenges the independence assumption in neurosymbolic learning, leading to overconfident predictions and unrepresentable uncertainty.
problem The independence assumption in neurosymbolic learning systems can lead to overconfident predictions and hinder uncertainty quantification.
method The study proves the limitations of the independence assumption and introduces new loss functions that are non-convex and difficult to optimise.
result Neurosymbolic learning systems using the independence assumption are prone to overconfidence and cannot represent uncertainty over multiple valid options.
New active learning framework for multiclass classification beyond realizability assumption.
problem Active learning in non-realizable settings with convex model classes.
method Surrogate risk minimization, epoch-based fitting, aggregation of models.
result Achieves label and sample complexity comparable to prior work in non-realizable settings.
New framework provides privacy guarantees for practical federated learning.
problem Inadequate privacy guarantees for federated learning due to restrictive assumptions.
method Fed-α-NormEC, integrating multiple local updates, partial client participation, and standard assumptions. result Provably convergent and differentially private federated learning framework.
Many machine learning algorithms are based on the assumption that training examples are drawn independently. However, this assumption does not hold anymore when learning from a networked sample where two or more training examples may share common features. We propose an efficient weighting method for learning from netw…
Statistical arbitrage is a class of financial trading strategies using mean reversion models. The corresponding techniques rely on a number of assumptions which may not hold for general non-stationary stochastic processes. This paper presents an alternative technique for statistical arbitrage based on online learning w…
New method bounds hardware noise without assumptions.
problem Estimating hardware noise without assumptions.
method Machine Learning and Conformal Prediction.
result Theoretical upper bounds of fidelity.
Study reward-free RL in non-linear settings, improving efficiency and removing assumptions.
problem Improving sample efficiency in reward-free reinforcement learning for non-linear function approximation.
method Proposed RFOLIVE algorithm for minimal structural assumptions, analyzed hardness results for reward-free and reward-aware exploration.
result Statistical efficiency and hardness results under various structural assumptions, no need for reachability or explorability assumptions.
Test verifies if data meets SCAR assumption for PU learning.
problem Verify if labeling mechanism follows SCAR assumption in PU learning.
method Generate artificial labels, mimic distribution of test statistic.
result Test detects deviations from SCAR and controls type I error.
The paper explores learning good policies from past data in large state spaces.
problem Learning good policies from historical data in large state spaces.
method Introduces expressivity assumptions and data coverage for function approximation and algorithmic design.
result A variety of algorithms and their guarantees are presented based on assumptions and desired complexity.
This thesis studies domain adaptation under minimal distribution similarity assumptions using moments.
problem Learning from samples with distributions different from training samples.
method Uses minimal similarity assumptions modeled by moments.
result Establishes learning bounds and algorithms for domain adaptation.
A new method selects covariates for causal effect estimation without strong assumptions.
problem Estimating causal effects without global causal structure learning and strong assumptions.
method Local covariate selection method that avoids pretreatment and causal sufficiency assumptions.
result The method achieves accurate causal effect estimation with improved computational efficiency.
For binary classification we establish learning rates up to the order of n−1 for support vector machines (SVMs) with hinge loss and Gaussian RBF kernels. These rates are in terms of two assumptions on the considered distributions: Tsybakov's noise assumption to establish a small estimation error, and a new geometr…
Although stochastic approximation learning methods have been widely used in the machine learning literature for over 50 years, formal theoretical analyses of specific machine learning algorithms are less common because stochastic approximation theorems typically possess assumptions which are difficult to communicate an…
New findings show learning deeper neural networks is hard even with Gaussian inputs and non-degenerate weights.
problem The computational complexity of learning neural networks, especially deeper ones.
method Smoothed analysis framework and local pseudorandom generators.
result Learning depth-3 ReLU networks under Gaussian input distribution is hard even if weight matrices are non-degenerate.
Learning a causal effect from observational data is not straightforward, as this is not possible without further assumptions. If hidden common causes between treatment X and outcome Y cannot be blocked by other measurements, one possibility is to use an instrumental variable. In principle, it is possible under some…
Value-function approximation methods that operate in batch mode have foundational importance to reinforcement learning (RL). Finite sample guarantees for these methods often crucially rely on two types of assumptions: (1) mild distribution shift, and (2) representation conditions that are stronger than realizability. H…
Unsupervised Domain Adaptation aims to learn a model on a source domain with labeled data in order to perform well on unlabeled data of a target domain. Current approaches focus on learning \textit{Domain Invariant Representations}. It relies on the assumption that such representations are well-suited for learning the …
In cheminformatics, compound-target binding profiles has been a main source of data for research. For data repositories that only provide positive profiles, a popular assumption is that unreported profiles are all negative. In this paper, we caution audience not to take this assumption for granted, and present empirica…
Given only positive (P) and unlabeled (U) data, PU learning can train a binary classifier without any negative data. It has two building blocks: PU class-prior estimation (CPE) and PU classification; the latter has been well studied while the former has received less attention. Hitherto, the distributional-assumption-f…
Responds to critiques on tests for causal parameter confidence intervals.
problem Testing nominal confidence interval coverage for causal parameters estimated by machine learning.
method Rejoinder to critiques on nearly assumption-free tests.
result Clarifies and supports the original research's approach.
AFA evaluates AI feature acquisition strategies in domains with high costs.
problem Evaluate AI feature acquisition strategies in domains with high costs.
method Apply missing data methods and offline reinforcement learning under NDE and NUC assumptions.
result Propose a novel semi-offline reinforcement learning framework with three new estimators.
Theory extends optimal learning rates without realizability assumption.
problem Agnostic binary classification without realizability assumption.
method Identifies tetrachotomy of optimal rates and combinatorial structures.
result Optimal universal rates for binary classification in agnostic setting.
Hardness proof for agnostically learning halfspaces from worst-case lattice problems.
problem Agnostically learning halfspaces in the presence of noise.
method Reduction to worst-case lattice problems (GapSVP, SIVP).
result No efficient algorithm can achieve misclassification error better than 1/2 - γ under given hardness assumptions.
We derive and analyze a new, efficient, pool-based active learning algorithm for halfspaces, called ALuMA. Most previous algorithms show exponential improvement in the label complexity assuming that the distribution over the instance space is close to uniform. This assumption rarely holds in practical applications. Ins…
We consider to learn a causal ordering of variables in a linear non-Gaussian acyclic model called LiNGAM. Several existing methods have been shown to consistently estimate a causal ordering assuming that all the model assumptions are correct. But, the estimation results could be distorted if some assumptions actually a…