arXiv research

This paper studies universal rates of ERM for binary classification under agnostic learning.

A universal learner achieves best rates for all distributions.

Theory extends optimal learning rates without realizability assumption.

Paper explores universal rates of ERM in machine learning.

Prototype rules simplify multiclass classification in metric spaces, achieving consistency and reduced complexity.

Paper establishes universal lower bounds and optimal rates for clustering sub-exponential mixture models.

Paper establishes a universal growth rate for smooth surrogate losses in classification.

We establish minimax optimal rates of convergence for estimation in a high dimensional additive model assuming that it is approximately sparse. Our results reveal an interesting phase transition behavior universal to this class of high dimensional problems. In the {\it sparse regime} when the components are sufficientl…

New theorem for generalized group sparsity improves consistency and convergence rates.

We study the problem of minimizing a strongly convex, smooth function when we have noisy estimates of its gradient. We propose a novel multistage accelerated algorithm that is universally optimal in the sense that it achieves the optimal rate both in the deterministic and stochastic case and operates without knowledge …

In this paper we study the growth rates of Artin monoids and we show that 4 is a universal upper bound. We also show that the generating functions of the associated right-angled Artin monoids are given by families of Chebyshev polynomials. Applications to Artin groups and positive braids are given.

Unified framework for comparing classification metrics across different imbalance rates.

GCNNs gain rotation invariance with more training augmentation, making SVD-Universal more effective.

We analyse the effectiveness of modern deep learning techniques in predicting credit ratings over a universe of thousands of global corporate entities obligations when compared to most popular, traditional machine-learning approaches such as linear models and tree-based classifiers. Our results show a adequate accuracy…

OptiNet achieves near-minimax error rates with compression in Euclidean space.

The paper proves neural networks' consistency and optimal convergence rates for various function classes.

We present an algorithm for computing class-specific universal adversarial perturbations for deep neural networks. Such perturbations can induce misclassification in a large fraction of images of a specific class. Unlike previous methods that use iterative optimization for computing a universal perturbation, the propos…

Transfer learning improves chaotic dynamics predictions with less data.

Study expands multiclass classification models with new rates and partial concept classes.

To estimate a sparse linear model from data with Gaussian noise, consilience from lasso and compressed sensing literatures is that thresholding estimators like lasso and the Dantzig selector have the ability in some situations to identify with high probability part of the significant covariates asymptotically, and are …

New proof shows incremental flow models are essential for universal generation.

Simple technique turns any adversarial attack into a universal one using few test examples.

This note provides a neat and enjoyable expansion and application of the magnificent Ordentlich-Cover theory of "universal portfolios." I generalize Cover's benchmark of the best constant-rebalanced portfolio (or 1-linear trading strategy) in hindsight by considering the best bilinear trading strategy determined in hin…

FibQuant improves KV-cache compression for long-context inference.

Convolutional neural networks (CNN) have become one of the most popular machine learning tools and are being applied in various tasks, however, CNN models are vulnerable to universal perturbations, which are usually human-imperceptible but can cause natural images to be misclassified with high probability. One of the s…

Deep neural networks without regularization can achieve consistent estimates with good convergence rates.

PLN-Nets with two linear layers and parallel LN achieve universal approximation.

The paper introduces a new learning model that explains practical aspects of machine learning.

Study optimal rates for multiclass classification, resolving open questions.

Consider a family of portfolio strategies with the aim of achieving the asymptotic growth rate of the best one. The idea behind Cover's universal portfolio is to build a wealth-weighted average which can be viewed as a buy-and-hold portfolio of portfolios. When an optimal portfolio exists, the wealth-weighted average c…

New analysis shows halting time is predictable for large models, improving optimization efficiency.

Improved rates for continual learning using SGD and last-iterate analysis.

Develops a new model-free approach to portfolio theory using rough paths.

The paper shows neural networks can approximate functions over non-compact domains with non-polynomial activation.

The study shows how nonnegative Ricci curvature and metric cones imply the existence of abelian subgroups in the fundamental group of open manifolds.

Unified framework proves neural networks' ability to mimic complex tasks.

In this paper, we study adaptive online convex optimization, and aim to design a universal algorithm that achieves optimal regret bounds for multiple common types of loss functions. Existing universal methods are limited in the sense that they are optimal for only a subclass of loss functions. To address this limitatio…

Hysteresis phenomena have been observed in different branches of physics and engineering sciences. Therefore, several models have been proposed for hysteresis simulation in different fields; however, almost neither of them can be utilized universally. In this paper by inspiring of Preisach Neural Network which was insp…

The paper addresses the gap between theoretical and practical confidence set widths in universal inference.

Random feature models approximate functions in Banach spaces efficiently.

Adversarial attacks on deep neural networks (DNNs) have been found for several years. However, the existing adversarial attacks have high success rates only when the information of the victim DNN is well-known or could be estimated by the structure similarity or massive queries. In this paper, we propose to Attack on A…

New bounds for optimal transport using Gaussian processes and rate-distortion functions.

New methods for inferring, predicting, and estimating continuous-time, discrete-event processes.

New method generates universal adversarial perturbations across different image sources.

A new method for faster optimization of noisy functions.

Empirical study on UEEs reveals liquidity's role and universal recovery patterns.

Nearest neighbor methods are a popular class of nonparametric estimators with several desirable properties, such as adaptivity to different distance scales in different regions of space. Prior work on convergence rates for nearest neighbor classification has not fully reflected these subtle properties. We analyze the b…

New method achieves both universality and adaptivity in online convex optimization.