Paper proves hardness of learning various complex models under local pseudorandom generators.
problem Hardness of learning various complex models.
method Existence of local pseudorandom generators.
result Proves hardness of learning shallow ReLU neural networks and other models.
This study examines the topology of singularities in optimal semicouplings between unequal spaces.
problem Topology of singularities in optimal semicouplings between unequal spaces.
method Continuous strong deformation retracts and Uniform Halfspace condition.
result Homotopy-reductions from a source space onto singularities of c c c -optimal semicouplings. Efficiently learns halfspaces with malicious noise, near-optimal label complexity.
problem Learning s s s -sparse halfspaces under malicious label noise. method Active learning algorithm with instance reweighting and empirical risk minimization.
result Near-optimal label complexity of O ( s log 4 d / ε ) O(s \log^4 d / ε) O ( s log 4 d / ε ) and noise tolerance Ω ( ε ) Ω(ε) Ω ( ε ) . We derive and analyze a new, efficient, pool-based active learning algorithm for halfspaces, called ALuMA. Most previous algorithms show exponential improvement in the label complexity assuming that the distribution over the instance space is close to uniform. This assumption rarely holds in practical applications. Ins…
It has been a long-standing problem to efficiently learn a halfspace using as few labels as possible in the presence of noise. In this work, we propose an efficient Perceptron-based algorithm for actively learning homogeneous halfspaces under the uniform distribution over the unit sphere. Under the bounded noise condit…
Generalizes halfspace theorems to higher dimensions for self-shrinkers.
problem Limitations of halfspace theorems in higher dimensions for self-shrinkers.
method Extends codimension 1 results to arbitrary codimension.
result Establishes new halfspace theorems for self-shrinkers in arbitrary codimension.
Polynomial-time tester-learner for general halfspaces with Gaussian adversarial noise.
problem Learning general halfspaces with adversarial label noise.
method Reduction to testable learning of nearly homogeneous halfspaces.
result First polynomial time tester-learner for general halfspaces with dimension-independent misclassification error.
Efficient algorithms for monophonic halfspaces in graphs simplify learning and compression.
problem Learning and compressing monophonic halfspaces in graphs.
method 2-satisfiability based decomposition theorem, efficient algorithms for various learning problems.
result Achieved efficient and nearly optimal algorithms for various learning problems.
Study efficient learning of robust halfspaces with noise.
problem Learning robust halfspaces in the presence of adversarial perturbations and random label noise.
method Provides conditions for robust learnability and a simple algorithm for any ℓ_p perturbation.
result Simple computationally efficient algorithm for robust learning with random label noise.
New lower bounds show learning intersections of halfspaces is hard even for a few halfspaces.
problem Learning intersections of halfspaces in polynomial time under standard assumptions.
method Unified connection to parallel pancakes distribution for proving hardness.
result Learning ω ( log log N ) ω(\log \log N) ω ( log log N ) halfspaces in dimension N N N requires super-polynomial time under standard assumptions. We develop the Lorentzian geometry of a crooked halfspace in 2+1-dimensional Minkowski space. We calculate the affine, conformal and isometric automorphism groups of a crooked halfspace, and discuss its stratification into orbit types, giving an explicit slice for the action of the automorphism group. The set of parall…
Efficiently learns monophonic halfspaces in graph vertices.
problem Learning binary classifiers on graph vertices using monophonic halfspaces.
method Polynomial-time algorithm for consistent hypothesis checking, based on structural insights and reduction to 2-satisfiability.
result Near-optimal passive sample complexity for monophonic halfspaces in polynomial time.
Non-convex SGD learns halfspaces with adversarial label noise efficiently.
problem Agnostically learning halfspaces in adversarial label noise settings.
method Non-convex SGD optimization for halfspace learning.
result Non-convex SGD achieves misclassification error close to optimal with adversarial noise.
New algorithm for reliable learning of Gaussian halfspaces with improved sample and computational complexity.
problem Learning halfspaces under Gaussian marginals with reliable agnostic model.
method Developed a new algorithm for reliable learning of Gaussian halfspaces with specific sample and computational complexity.
result Achieved a new algorithm with improved sample and computational complexity for reliable learning of Gaussian halfspaces.
Study learning halfspaces with Massart noise under Gaussian distribution, improving previous results.
problem Learning halfspaces with Massart noise under Gaussian distribution, especially when the parameter is 1/2.
method Developed algorithms for general and homogeneous halfspaces with sample and computational complexities.
result Established qualitatively matching lower bounds for the complexities of learning algorithms.
Study privacy and robustness in learning halfspaces, proving hard trade-offs.
problem Balancing privacy and robustness in learning halfspaces.
method Proves nearly tight bounds on sample complexity for robust private learning of halfspaces.
result Robust and private learning is harder than robust or private learning alone.
Polynomial-time algorithm for learning halfspaces with Gaussian-distributed data and adversarial noise.
problem Learning halfspaces in the presence of adversarial label noise.
method Iterative soft localization technique enhanced with appropriate testers.
result Output a halfspace with misclassification error $O(\opt)+\eps$ .
Gradient descent finds halfspaces with low error for agnostic learning.
problem Agnostic learning of linear halfspaces with convex surrogates.
method Gradient descent on convex surrogates for zero-one loss.
result Gradient descent finds halfspaces with error O ( O P T 1 / 2 + ε ) O(\mathsf{OPT}^{1/2} + \varepsilon) O ( OPT 1/2 + ε ) in poly time and sample complexity. First proper learning algorithm for Gaussian halfspaces with matching sample and computational complexity.
problem Agnostically learning halfspaces under Gaussian distribution.
method First proper learning algorithm with matching sample and computational complexity.
result First proper learning algorithm for agnostically learning halfspaces under Gaussian distribution with matching sample and computational complexity.
Study proves existence of a specific type of flow in geometry.
problem Existence of canonical multi-phase free boundary Brakke flows.
method Global-in-time existence established using Brakke flow and uniform density ratio assumption.
result Existence of the flow with no positive mass on the free boundary for some short time.
Study on learning halfspaces under adversarial perturbations, finding computational hardness.
problem Learning halfspaces in the presence of adversarial noise.
method Introduced an efficient learning algorithm and proved a nearly matching computational hardness result.
result The L ∞ L_{\infty} L ∞ perturbations case is provably computationally harder than 2 ≤ p < ∞ 2 \leq p < \infty 2 ≤ p < ∞ . Adversarial training improves robustness of halfspaces in noisy data.
problem Learning robust halfspaces in the presence of label noise.
method Adversarial training with binary cross-entropy or nonconvex sigmoidal loss.
result Adversarial training yields robust halfspaces with improved classification error.
We study the problem of learning halfspaces with Massart noise in the distribution-specific PAC model. We give the first computationally efficient algorithm for this problem with respect to a broad family of distributions, including log-concave distributions. This resolves an open question posed in a number of prior wo…
Strongly polynomial algorithm for approximate Forster transforms and halfspace learning.
problem Computing approximate Forster transforms and halfspace learning.
method Strongly polynomial time algorithm for approximate Forster transforms and halfspace learning.
result First strongly polynomial time algorithm for distribution-free PAC learning of halfspaces.
New algorithm learns halfspaces over hypercube with random bit flips.
problem Agnostic learning of Boolean halfspaces over discrete domains is computationally hard.
method Smoothed analysis with random bit flips for discrete inputs.
result First efficient algorithm for smoothed agnostic learning of halfspaces over Boolean hypercube.
Study efficient active learning for halfspaces with Tsybakov noise using non-convex optimization.
problem Efficiently learn halfspaces with Tsybakov noise under structured unlabeled data.
method Non-convex optimization approach to find approximate first-order stationary points.
result Designs an algorithm with improved label complexity compared to previous methods.
While it is well known from examples that no interesting `halfspace theorem' holds for properly immersed complete n n n -dimensional self-translating mean curvature flow solitons in Euclidean space R n + 1 \mathbb{R}^{n+1} R n + 1 , we show that they must all obey a general `bi-halfspace theorem': Two transverse vertical halfspaces can …
Near-optimal SQ hardness shows learning halfspaces with Massart noise is hard.
problem Learning halfspaces with Massart noise in the presence of label corruption.
method Statistical Query (SQ) model analysis.
result No efficient SQ algorithm can achieve better than Ω ( η ) Ω(η) Ω ( η ) error, even for optimal noise levels. Self-training algorithm improves classifier performance with labeled and unlabeled data.
problem Improving classifier performance with limited labeled data.
method Iterative learning of halfspaces, exploration and pruning phases.
result Misclassification error is bounded and never degrades compared to initial labeled set.
New algorithms for privately learning decision lists and halfspaces.
problem Private learning of decision lists and halfspaces.
method Differentially private algorithms for PAC and online models.
result Private algorithms match or surpass non-private guarantees.
Algorithm learns halfspaces with Tsybakov noise in polynomial time.
problem PAC learning halfspaces with adversarial noise.
method Reduction to certifying non-optimality, iterative process, warm-start algorithm.
result First polynomial-time algorithm for learning halfspaces with Tsybakov noise.
Study near-optimal bounds for learning Gaussian halfspaces with random noise.
problem Learning general halfspaces with Gaussian distribution and random classification noise.
method Established nearly-matching algorithmic and SQ lower bounds, developed a computationally efficient learning algorithm.
result Sample complexity of learning algorithm is O ( d / ε + d / ( max { p , ε } ) 2 ) O(d/ε + d/(\max\{p, ε\})^2) O ( d / ε + d / ( max { p , ε } ) 2 ) , SQ lower bound is Ω ( d 1 / 2 / ( max { p , ε } ) 2 ) Ω(d^{1/2}/(\max\{p, ε\})^2) Ω ( d 1/2 / ( max { p , ε } ) 2 ) . New algorithm learns halfspaces with adversarial noise efficiently.
problem Learning halfspaces in the presence of adversarial noise.
method Polynomial-time Perceptron-like online active learning algorithm.
result Near-optimal label and sample complexity with isotropic log-concave marginal distribution.
Algorithm learns halfspaces in noisy data efficiently.
problem Learning halfspaces with Tsybakov noise.
method Novel semi-definite programming and online convex optimization.
result First non-trivial PAC learning algorithm for Tsybakov noise.
A new depth measure for non-convex data supports, faster than halfspace depth.
problem Non-convex data supports in multivariate statistics.
method Extending halfspace depth to Reproducing Kernel Hilbert Space (RKHS).
result The new depth measure is consistent and can be computed faster.
Study selective classification with halfspaces, achieving error bounds under Gaussian distributions.
problem Modeling relationships in subsets of data defined by selection rules.
method Sparse linear classifiers for subsets defined by halfspaces, focusing on Gaussian feature distributions.
result First PAC-learning algorithm for homogeneous halfspace selectors with error guarantee $\bigO*{\sqrt{\mathrm{opt}}}$ .
Efficient algorithm for halfspaces with specific noise conditions.
problem Learning halfspaces with Massart and Tsybakov noise.
method PAC active learning algorithm for d-dimensional halfspaces.
result Near-optimal label complexity under Massart noise and lower than passive learning under Tsybakov noise.
We study the problem of efficient PAC active learning of homogeneous linear classifiers (halfspaces) in R d \mathbb{R}^d R d , where the goal is to learn a halfspace with low error using as few label queries as possible. Under the extra assumption that there is a t t t -sparse halfspace that performs well on the data ( t ≪ d t \ll d t ≪ d )…
We provide new results for noise-tolerant and sample-efficient learning algorithms under s s s -concave distributions. The new class of s s s -concave distributions is a broad and natural generalization of log-concavity, and includes many important additional distributions, e.g., the Pareto distribution and t t t -distribution.…
New algorithm learns halfspaces with noise using Forster decomposition.
problem Learning halfspaces in noisy data.
method Forster decomposition and efficient mixture of distributions.
result First polynomial-time algorithm with strongly polynomial sample complexity.
Improved private learning of halfspaces with reduced sample complexity.
problem Private learning of halfspaces with reduced sample complexity.
method Iterative algorithm for solving linear feasibility problem, improving state-of-the-art results.
result Sample complexity reduced to d 2.5 ⋅ 2 log ∗ ∣ G ∣ d^{2.5} \cdot 2^{\log^*|G|} d 2.5 ⋅ 2 l o g ∗ ∣ G ∣ , improving d 2 d^2 d 2 factor. Average teaching complexity for locating target regions among halfspace intersections is Θ(d).
problem Teaching the location of a target region among intersections of halfspaces.
method Novel insights from computational geometry to count convex polytopes and faces.
result Average-case teaching complexity is Θ(d), contrasting with Θ(n) worst-case complexity.
New algorithm learns halfspaces with membership queries, achieving near optimal label complexity.
problem Learning halfspaces with membership queries.
method Proposed a new algorithm for learning halfspaces with membership queries, proving near optimal label complexity.
result Achieves near optimal label complexity for learning halfspaces.
Polynomial-time algorithm learns high-dimensional halfspaces without labels.
problem Learning high-dimensional halfspaces with margins in polynomial time.
method Contrastive moments and polynomial-time algorithm.
result Establishes the unique and efficient identifiability of the hidden halfspace.
Study efficient learning of halfspaces with constant noise tolerance.
problem Learning halfspaces in the presence of both instance and label corruption.
method Develops an algorithm to minimize reweighted hinge loss for robustness.
result Achieves constant noise tolerance for halfspace learning.
The study establishes SQ lower bounds for learning halfspaces and ReLUs under Gaussian marginals.
problem Agnostically learning halfspaces and ReLUs under Gaussian marginals.
method Statistical Query (SQ) lower bounds analysis.
result Proves SQ lower bounds of d p o l y ( 1 / ε ) d^{\mathrm{poly}(1/ε)} d poly ( 1/ ε ) for both problems. Efficient algorithms improve learning of large-margin halfspaces.
problem Learning large-margin halfspaces efficiently and reproducibly.
method Design of efficient, dimension-independent, polynomial-time algorithms; SGD-based approach; DP-to-Replicability reduction.
result Improved sample complexity compared to previous algorithms, with optimal sample complexity for one algorithm.
New algorithm learns halfspaces with near-optimal sample complexity in noisy conditions.
problem Learning margin halfspaces with Massart noise.
method Computational efficient algorithm using online SGD on carefully selected convex losses.
result Sample complexity of Θ ~ ( 1 / ( γ 2 ε 2 ) ) \widetilde{\Theta}(1/(γ^2 ε^2)) Θ ( 1/ ( γ 2 ε 2 )) , nearly matching lower bound.