Polynomial-time tester-learner for general halfspaces with Gaussian adversarial noise.
problem Learning general halfspaces with adversarial label noise.
method Reduction to testable learning of nearly homogeneous halfspaces.
result First polynomial time tester-learner for general halfspaces with dimension-independent misclassification error.
Efficient algorithms for monophonic halfspaces in graphs simplify learning and compression.
problem Learning and compressing monophonic halfspaces in graphs.
method 2-satisfiability based decomposition theorem, efficient algorithms for various learning problems.
result Achieved efficient and nearly optimal algorithms for various learning problems.
Study efficient learning of robust halfspaces with noise.
problem Learning robust halfspaces in the presence of adversarial perturbations and random label noise.
method Provides conditions for robust learnability and a simple algorithm for any ℓ_p perturbation.
result Simple computationally efficient algorithm for robust learning with random label noise.
New lower bounds show learning intersections of halfspaces is hard even for a few halfspaces.
problem Learning intersections of halfspaces in polynomial time under standard assumptions.
method Unified connection to parallel pancakes distribution for proving hardness.
result Learning ω ( log log N ) ω(\log \log N) ω ( log log N ) halfspaces in dimension N N N requires super-polynomial time under standard assumptions. Efficiently learns monophonic halfspaces in graph vertices.
problem Learning binary classifiers on graph vertices using monophonic halfspaces.
method Polynomial-time algorithm for consistent hypothesis checking, based on structural insights and reduction to 2-satisfiability.
result Near-optimal passive sample complexity for monophonic halfspaces in polynomial time.
New algorithm for reliable learning of Gaussian halfspaces with improved sample and computational complexity.
problem Learning halfspaces under Gaussian marginals with reliable agnostic model.
method Developed a new algorithm for reliable learning of Gaussian halfspaces with specific sample and computational complexity.
result Achieved a new algorithm with improved sample and computational complexity for reliable learning of Gaussian halfspaces.
Study privacy and robustness in learning halfspaces, proving hard trade-offs.
problem Balancing privacy and robustness in learning halfspaces.
method Proves nearly tight bounds on sample complexity for robust private learning of halfspaces.
result Robust and private learning is harder than robust or private learning alone.
First proper learning algorithm for Gaussian halfspaces with matching sample and computational complexity.
problem Agnostically learning halfspaces under Gaussian distribution.
method First proper learning algorithm with matching sample and computational complexity.
result First proper learning algorithm for agnostically learning halfspaces under Gaussian distribution with matching sample and computational complexity.
Non-convex SGD learns halfspaces with adversarial label noise efficiently.
problem Agnostically learning halfspaces in adversarial label noise settings.
method Non-convex SGD optimization for halfspace learning.
result Non-convex SGD achieves misclassification error close to optimal with adversarial noise.
Polynomial-time algorithm for learning halfspaces with Gaussian-distributed data and adversarial noise.
problem Learning halfspaces in the presence of adversarial label noise.
method Iterative soft localization technique enhanced with appropriate testers.
result Output a halfspace with misclassification error $O(\opt)+\eps$ .
Study learning halfspaces with Massart noise under Gaussian distribution, improving previous results.
problem Learning halfspaces with Massart noise under Gaussian distribution, especially when the parameter is 1/2.
method Developed algorithms for general and homogeneous halfspaces with sample and computational complexities.
result Established qualitatively matching lower bounds for the complexities of learning algorithms.
Gradient descent finds halfspaces with low error for agnostic learning.
problem Agnostic learning of linear halfspaces with convex surrogates.
method Gradient descent on convex surrogates for zero-one loss.
result Gradient descent finds halfspaces with error O ( O P T 1 / 2 + ε ) O(\mathsf{OPT}^{1/2} + \varepsilon) O ( OPT 1/2 + ε ) in poly time and sample complexity. Study on learning halfspaces under adversarial perturbations, finding computational hardness.
problem Learning halfspaces in the presence of adversarial noise.
method Introduced an efficient learning algorithm and proved a nearly matching computational hardness result.
result The L ∞ L_{\infty} L ∞ perturbations case is provably computationally harder than 2 ≤ p < ∞ 2 \leq p < \infty 2 ≤ p < ∞ . Strongly polynomial algorithm for approximate Forster transforms and halfspace learning.
problem Computing approximate Forster transforms and halfspace learning.
method Strongly polynomial time algorithm for approximate Forster transforms and halfspace learning.
result First strongly polynomial time algorithm for distribution-free PAC learning of halfspaces.
New algorithm learns halfspaces over hypercube with random bit flips.
problem Agnostic learning of Boolean halfspaces over discrete domains is computationally hard.
method Smoothed analysis with random bit flips for discrete inputs.
result First efficient algorithm for smoothed agnostic learning of halfspaces over Boolean hypercube.
We study the problem of learning halfspaces with Massart noise in the distribution-specific PAC model. We give the first computationally efficient algorithm for this problem with respect to a broad family of distributions, including log-concave distributions. This resolves an open question posed in a number of prior wo…
Study efficient active learning for halfspaces with Tsybakov noise using non-convex optimization.
problem Efficiently learn halfspaces with Tsybakov noise under structured unlabeled data.
method Non-convex optimization approach to find approximate first-order stationary points.
result Designs an algorithm with improved label complexity compared to previous methods.
New algorithms for privately learning decision lists and halfspaces.
problem Private learning of decision lists and halfspaces.
method Differentially private algorithms for PAC and online models.
result Private algorithms match or surpass non-private guarantees.
Near-optimal SQ hardness shows learning halfspaces with Massart noise is hard.
problem Learning halfspaces with Massart noise in the presence of label corruption.
method Statistical Query (SQ) model analysis.
result No efficient SQ algorithm can achieve better than Ω ( η ) Ω(η) Ω ( η ) error, even for optimal noise levels. Study near-optimal bounds for learning Gaussian halfspaces with random noise.
problem Learning general halfspaces with Gaussian distribution and random classification noise.
method Established nearly-matching algorithmic and SQ lower bounds, developed a computationally efficient learning algorithm.
result Sample complexity of learning algorithm is O ( d / ε + d / ( max { p , ε } ) 2 ) O(d/ε + d/(\max\{p, ε\})^2) O ( d / ε + d / ( max { p , ε } ) 2 ) , SQ lower bound is Ω ( d 1 / 2 / ( max { p , ε } ) 2 ) Ω(d^{1/2}/(\max\{p, ε\})^2) Ω ( d 1/2 / ( max { p , ε } ) 2 ) . Adversarial training improves robustness of halfspaces in noisy data.
problem Learning robust halfspaces in the presence of label noise.
method Adversarial training with binary cross-entropy or nonconvex sigmoidal loss.
result Adversarial training yields robust halfspaces with improved classification error.
Algorithm learns halfspaces with Tsybakov noise in polynomial time.
problem PAC learning halfspaces with adversarial noise.
method Reduction to certifying non-optimality, iterative process, warm-start algorithm.
result First polynomial-time algorithm for learning halfspaces with Tsybakov noise.
New algorithm learns halfspaces with adversarial noise efficiently.
problem Learning halfspaces in the presence of adversarial noise.
method Polynomial-time Perceptron-like online active learning algorithm.
result Near-optimal label and sample complexity with isotropic log-concave marginal distribution.
Algorithm learns halfspaces in noisy data efficiently.
problem Learning halfspaces with Tsybakov noise.
method Novel semi-definite programming and online convex optimization.
result First non-trivial PAC learning algorithm for Tsybakov noise.
Efficient algorithm for halfspaces with specific noise conditions.
problem Learning halfspaces with Massart and Tsybakov noise.
method PAC active learning algorithm for d-dimensional halfspaces.
result Near-optimal label complexity under Massart noise and lower than passive learning under Tsybakov noise.
New algorithm learns halfspaces with membership queries, achieving near optimal label complexity.
problem Learning halfspaces with membership queries.
method Proposed a new algorithm for learning halfspaces with membership queries, proving near optimal label complexity.
result Achieves near optimal label complexity for learning halfspaces.
Efficiently learns halfspaces with malicious noise, near-optimal label complexity.
problem Learning s s s -sparse halfspaces under malicious label noise. method Active learning algorithm with instance reweighting and empirical risk minimization.
result Near-optimal label complexity of O ( s log 4 d / ε ) O(s \log^4 d / ε) O ( s log 4 d / ε ) and noise tolerance Ω ( ε ) Ω(ε) Ω ( ε ) . New algorithm learns halfspaces with noise using Forster decomposition.
problem Learning halfspaces in noisy data.
method Forster decomposition and efficient mixture of distributions.
result First polynomial-time algorithm with strongly polynomial sample complexity.
Self-training algorithm improves classifier performance with labeled and unlabeled data.
problem Improving classifier performance with limited labeled data.
method Iterative learning of halfspaces, exploration and pruning phases.
result Misclassification error is bounded and never degrades compared to initial labeled set.
We study the problem of efficient PAC active learning of homogeneous linear classifiers (halfspaces) in R d \mathbb{R}^d R d , where the goal is to learn a halfspace with low error using as few label queries as possible. Under the extra assumption that there is a t t t -sparse halfspace that performs well on the data ( t ≪ d t \ll d t ≪ d )…
Generalizes halfspace theorems to higher dimensions for self-shrinkers.
problem Limitations of halfspace theorems in higher dimensions for self-shrinkers.
method Extends codimension 1 results to arbitrary codimension.
result Establishes new halfspace theorems for self-shrinkers in arbitrary codimension.
Improved private learning of halfspaces with reduced sample complexity.
problem Private learning of halfspaces with reduced sample complexity.
method Iterative algorithm for solving linear feasibility problem, improving state-of-the-art results.
result Sample complexity reduced to d 2.5 ⋅ 2 log ∗ ∣ G ∣ d^{2.5} \cdot 2^{\log^*|G|} d 2.5 ⋅ 2 l o g ∗ ∣ G ∣ , improving d 2 d^2 d 2 factor. Study efficient learning of halfspaces with constant noise tolerance.
problem Learning halfspaces in the presence of both instance and label corruption.
method Develops an algorithm to minimize reweighted hinge loss for robustness.
result Achieves constant noise tolerance for halfspace learning.
Study selective classification with halfspaces, achieving error bounds under Gaussian distributions.
problem Modeling relationships in subsets of data defined by selection rules.
method Sparse linear classifiers for subsets defined by halfspaces, focusing on Gaussian feature distributions.
result First PAC-learning algorithm for homogeneous halfspace selectors with error guarantee $\bigO*{\sqrt{\mathrm{opt}}}$ .
Polynomial-time algorithm learns high-dimensional halfspaces without labels.
problem Learning high-dimensional halfspaces with margins in polynomial time.
method Contrastive moments and polynomial-time algorithm.
result Establishes the unique and efficient identifiability of the hidden halfspace.
New algorithm learns halfspaces with near-optimal sample complexity in noisy conditions.
problem Learning margin halfspaces with Massart noise.
method Computational efficient algorithm using online SGD on carefully selected convex losses.
result Sample complexity of Θ ~ ( 1 / ( γ 2 ε 2 ) ) \widetilde{\Theta}(1/(γ^2 ε^2)) Θ ( 1/ ( γ 2 ε 2 )) , nearly matching lower bound. Average teaching complexity for locating target regions among halfspace intersections is Θ(d).
problem Teaching the location of a target region among intersections of halfspaces.
method Novel insights from computational geometry to count convex polytopes and faces.
result Average-case teaching complexity is Θ(d), contrasting with Θ(n) worst-case complexity.
The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.
problem Agnostically learning halfspaces and ReLUs under Gaussian marginals.
method Statistical Query (SQ) lower bounds analysis.
result Proves SQ lower bounds of d p o l y ( 1 / ε ) d^{\mathrm{poly}(1/ε)} d poly ( 1/ ε ) for both problems. We develop the Lorentzian geometry of a crooked halfspace in 2+1-dimensional Minkowski space. We calculate the affine, conformal and isometric automorphism groups of a crooked halfspace, and discuss its stratification into orbit types, giving an explicit slice for the action of the automorphism group. The set of parall…
Optimal algorithm learns Gaussian under halfspace truncation with minimal samples.
problem Learning a Gaussian distribution truncated to an unknown halfspace.
method Efficient algorithm using n = i l d e O ( d 2 / ε 2 ) n = ilde{O}(d^2/\varepsilon^2) n = i l d e O ( d 2 / ε 2 ) samples and runtime dominated by empirical covariance matrix computation. result Optimal sample and time complexity bounds for learning a Gaussian under halfspace truncation.
Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.
problem PAC learning γ-margin halfspaces with Random Classification Noise.
method Established an information-computation tradeoff and provided a simple efficient algorithm with sample complexity O(1/(γ^2 ε^2)). Also, proved lower bounds for SQ algorithms and low-degree polynomial tests.
result Inherent gap between sample complexity and computational efficiency for learning halfspaces with random noise.
Efficient algorithms improve learning of large-margin halfspaces.
problem Learning large-margin halfspaces efficiently and reproducibly.
method Design of efficient, dimension-independent, polynomial-time algorithms; SGD-based approach; DP-to-Replicability reduction.
result Improved sample complexity compared to previous algorithms, with optimal sample complexity for one algorithm.
Paper proves hardness of learning various complex models under local pseudorandom generators.
problem Hardness of learning various complex models.
method Existence of local pseudorandom generators.
result Proves hardness of learning shallow ReLU neural networks and other models.
Hardness proof for agnostically learning halfspaces from worst-case lattice problems.
problem Agnostically learning halfspaces in the presence of noise.
method Reduction to worst-case lattice problems (GapSVP, SIVP).
result No efficient algorithm can achieve misclassification error better than 1/2 - γ under given hardness assumptions.
Paper addresses classification under misspecification, providing simpler algorithms and resolving open questions.
problem Learning halfspaces and generalized linear models under Massart noise and other corruption models.
method Developed simpler algorithms and used blackbox knowledge distillation to convert complex classifiers to proper ones. Leveraged evolvability for theoretical insights.
result First efficient algorithm for learning Massart halfspaces with η + ε η+ ε η + ε accuracy, and general algorithm for generalized linear models. New research shows logistic regression can achieve optimal error rate for agnostic learning of halfspaces.
problem Agnostic learning of homogeneous halfspaces with logistic loss.
method Constructing a well-behaved distribution and using logistic regression with additional convex optimization steps.
result Logistic regression can achieve Ω ( e x t r m O P T ) Ω(\sqrt{ extrm{OPT}}) Ω ( e x t r m O P T ) misclassification risk, matching the upper bound. New algorithm learns halfspaces almost optimally with fewer queries.
problem Learning halfspaces with minimal queries.
method Randomized linear decision tree of depth O(d log |X|).
result First nearly optimal solution for active learning of halfspaces.
Improved sample complexity for learning halfspaces with malicious noise.
problem Efficiently learning halfspaces in the presence of malicious noise.
method New analysis of Awasthi et al. algorithm with matrix Chernoff inequality and localization schemes.
result Achieved near-optimal sample complexity of i l d e O ( d ) ilde{O}(d) i l d e O ( d ) for isotropic log-concave distributions.