arXiv research

First proper learning algorithm for Gaussian halfspaces with matching sample and computational complexity.

Paper characterizes classes for which optimal sample complexity can be achieved by proper learning algorithms.

There has been much recent interest in application of the pool-adjacent-violators (PAV) algorithm for the purpose of calibrating the probabilistic outputs of automatic pattern recognition and machine learning algorithms. Special cost functions, known as proper scoring rules form natural objective functions to judge the…

Optimal multiclass U-calibration error found to be Θ(√KT).

Proper learning is possible with labeled data, but unlabeled data can improve performance.

Study on learning halfspaces under adversarial perturbations, finding computational hardness.

New algorithms minimize dynamic regret for strongly convex losses.

Efficiently learns complex Boolean functions under Gaussian distributions.

Study on proper learning under relaxed worst-case robust loss for VC classes.

Efficient algorithms for monophonic halfspaces in graphs simplify learning and compression.

The paper provides an almost optimal learning and testing algorithm for sparse polynomials.

Paper analyzes proper losses and their performance in machine learning tasks.

Paper develops proper, lower-bounded losses for weakly supervised classification.

A novel framework quantifies uncertainty using proper scores for various tasks.

MLR-SNet learns flexible LR schedules for diverse tasks.

Efficient active learning with abstention reduces label complexity exponentially.

We study the problem of {\em properly} learning large margin halfspaces in the agnostic PAC model. In more detail, we study the complexity of properly learning $d$-dimensional halfspaces on the unit ball within misclassification error $α\cdot \mathrm{OPT}_γ + ε$, where $\mathrm{OPT}_γ$ is the optimal $γ$-margin error r…

New algorithm reduces prediction errors across various loss functions.

The goal of online prediction with expert advice is to find a decision strategy which will perform almost as well as the best expert in a given pool of experts, on any sequence of outcomes. This problem has been widely studied and $O(\sqrt{T})$ and $O(\log{T})$ regret bounds can be achieved for convex losses (\cite{zin…

New algorithms improve agnostic learning for triangles and polygons, reducing time complexity.

New binary classification techniques help multiclass classification by aggregating proper learners.

The paper briefly introduces multiple classifier systems and describes a new algorithm, which improves classification accuracy by means of recommendation of a proper algorithm to an object classification. This recommendation is done assuming that a classifier is likely to predict the label of the object correctly if it…

New insights into learning from only positive examples.

Paper proposes fitting loss functions to data using source functions from information geometry.

New algorithms achieve near-optimal cumulative loss in nonparametric online learning and games.

How many bits of information are required to PAC learn a class of hypotheses of VC dimension $d$? The mathematical setting we follow is that of Bassily et al. (2018), where the value of interest is the mutual information $\mathrm{I}(S;A(S))$ between the input sample $S$ and the hypothesis outputted by the learning algo…

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

A new method learns proper multiclass losses and probabilities.

Optimizing proper loss yields calibrated models under specific conditions.

Algorithm learns decision trees from noisy data.

The problem of bipartite ranking, where instances are labeled positive or negative and the goal is to learn a scoring function that minimizes the probability of mis-ranking a pair of positive and negative instances (or equivalently, that maximizes the area under the ROC curve), has been widely studied in recent years. …

This work examines uncertainty sampling in binary classification using equivalent loss.

New algorithm tackles smooth online learning with optimal regret.

New framework models echo chamber learning, proving tight bounds on algorithm performance.

Empirical studies show that gradient-based methods can learn deep neural networks (DNNs) with very good generalization performance in the over-parameterization regime, where DNNs can easily fit a random labeling of the training data. Very recently, a line of work explains in theory that with over-parameterization and p…

New concept of proper-calibeating extends classic calibrated forecasts to proper scoring rules.

The article reviews scoring rules for estimating and evaluating forecasts.

New method improves decision-making accuracy without complex calculations.

We study the problem of learning latent variables in Gaussian graphical models. Existing methods for this problem assume that the precision matrix of the observed variables is the superposition of a sparse and a low-rank component. In this paper, we focus on the estimation of the low-rank component, which encodes the e…

Paper addresses classification under misspecification, providing simpler algorithms and resolving open questions.

Complex-valued signals are used in the modeling of many systems in engineering and science, hence being of fundamental interest. Often, random complex-valued signals are considered to be proper. A proper complex random variable or process is uncorrelated with its complex conjugate. This assumption is a good model of th…

New method for simplifying knots with specific properties.

This is a report on our long term project to find an algorithm to decide if a finitely presented group has a non-trivial action on a tree.

Proper regularization is critical for speeding up training, improving generalization performance, and learning compact models that are cost efficient. We propose and analyze regularized gradient descent algorithms for learning shallow neural networks. Our framework is general and covers weight-sharing (convolutional ne…

Efficiently learns polytrees with known skeleton in polynomial time and sample complexity.

There has been much recent progress in forecasting the next observation of a linear dynamical system (LDS), which is known as the improper learning, as well as in the estimation of its system matrices, which is known as the proper learning of LDS. We present an approach to proper learning of LDS, which in spite of the …

The paper explores proper actions and their relation to representation theory, with new quantitative methods.

We provide an effective algorithm for determining whether an element of the outer automorphism group of a free group is fully irreducible. Our method produces a finite list which can be checked for periodic proper free factors.