arXiv research

Paper proves a new lower bound on calibration error for binary prediction.

Paper develops conformalized survival analysis method for better prediction.

Paper proves tight lower bounds for online multicalibration, separating it from marginal calibration.

The paper explores high-dimensional learning in finance, proving key aspects and setting lower bounds.

Improved Gaussian process regression with tighter log marginal likelihood bounds.

Improved upper bound for online calibrated forecasting of binary sequences.

Proposes a method to compute valid lower confidence bounds for multiple models selected based on their performance.

The paper studies the distance from calibration in sequential prediction, proving upper and lower bounds.

This work addresses the classic machine learning problem of online prediction with expert advice. A new potential-based framework for the fixed horizon version of this problem has been recently developed using verification arguments from optimal control theory. This paper extends this framework to the random (geometric…

New method uses unlabelled data to improve Bayesian Neural Networks.

The study sets lower bounds on MMSE for inferring sensitive features from noisy data.

Proposes an efficient lower bound for Gromov-Wasserstein discrepancy.

Optimizes experimental designs for intractable models using mutual information bounds.

BILBO optimizes bilevel problems without repeated lower-level optimizations.

The paper proves ACC for local volumes under boundedness conditions.

In this paper, we obtain fundamental $\mathcal{L}_{p}$ bounds in sequential prediction and recursive algorithms via an entropic analysis. Both classes of problems are examined by investigating the underlying entropic relationships of the data and/or noises involved, and the derived lower bounds may all be quantified in…

Exact bounds derived for neural network outputs with noisy inputs.

Lower bound on BART's mixing time increases with data points.

This paper considers the quantification of the prediction performance in Gaussian process regression. The standard approach is to base the prediction error bars on the theoretical predictive variance, which is a lower bound on the mean square-error (MSE). This approach, however, does not take into account that the stat…

New algorithms improve online prediction from experts with privacy constraints.

Measuring Mutual Information (MI) between high-dimensional, continuous, random variables from observed samples has wide theoretical and practical applications. Recent work, MINE (Belghazi et al. 2018), focused on estimating tight variational lower bounds of MI using neural networks, but assumed unlimited supply of samp…

Improved efficient robust regression with near-linear time and subquadratic samples.

In this note we consider setups in which variational objectives for Bayesian neural networks can be computed in closed form. In particular we focus on single-layer networks in which the activation function is piecewise polynomial (e.g. ReLU). In this case we show that for a Normal likelihood and structured Normal varia…

In classic papers, Zellner demonstrated that Bayesian inference could be derived as the solution to an information theoretic functional. Below we derive a generalized form of this functional as a variational lower bound of a predictive information bottleneck objective. This generalized functional encompasses most moder…

For the problem of high-dimensional sparse linear regression, it is known that an $\ell_0$-based estimator can achieve a $1/n$ "fast" rate on the prediction error without any conditions on the design matrix, whereas in absence of restrictive conditions on the design matrix, popular polynomial-time methods only guarante…

Study optimizes prediction error for growing-dimensional PFLM models.

The paper sets limits on the accuracy of macroeconomic forecasts based on statistical moments and trade volumes.

We show that fundamental learning tasks, such as finding an approximate linear separator or linear regression, require memory at least \emph{quadratic} in the dimension, in a natural streaming setting. This implies that such problems cannot be solved (at least in this setting) by scalable memory-efficient streaming alg…

We consider random-design linear prediction and related questions on the lower tail of random matrices. It is known that, under boundedness constraints, the minimax risk is of order $d/n$ in dimension $d$ with $n$ samples. Here, we study the minimax expected excess risk over the full linear class, depending on the dist…

New algorithms avoid a dominant lower-order term in heavy-tailed loss settings.

Investigates Lipschitz continuity in neural networks across various settings.

In this paper, we examine the fundamental performance limits of prediction, with or without side information. More specifically, we derive generic lower bounds on the $\mathcal{L}_p$ norms of the prediction errors that are valid for any prediction algorithms and for any data distributions. Meanwhile, we combine the ent…

This work creates a CS for non-negative heavy-tailed data with bounded mean.

A new multi-label CPC method improves mutual information estimation and representation learning.

Standard methods in supervised learning separate training and prediction: the model is fit independently of any test points it may encounter. However, can knowledge of the next test point $\mathbf{x}_{\star}$ be exploited to improve prediction accuracy? We address this question in the context of linear prediction, show…

A new method for multi-objective Bayesian optimization using entropy search and variational lower bound maximization.

The paper analyzes the robustness of a minimum $\ell_2$ interpolator in high-dimensional linear regression.

The paper shows optimal robustness against adversarial corruption in sequential decision-making problems.

Translation-based embedding models have gained significant attention in link prediction tasks for knowledge graphs. TransE is the primary model among translation-based embeddings and is well-known for its low complexity and high efficiency. Therefore, most of the earlier works have modified the score function of the Tr…

The paper tightens bounds on covering numbers for deep ReLU networks.

We analyze the problem of sequential probability assignment for binary outcomes with side information and logarithmic loss, where regret---or, redundancy---is measured with respect to a (possibly infinite) class of experts. We provide upper and lower bounds for minimax regret in terms of sequential complexities of the …

Improved estimator for least squares using random projections achieves smaller error.

We develop a method to estimate the time to unsafe responses in LLMs.

An algorithm learns from multiple models to match an oracle's risk.

This work addresses the classic machine learning problem of online prediction with expert advice. We consider the finite-horizon version of this zero-sum, two-person game. Using verification arguments from optimal control theory, we view the task of finding better lower and upper bounds on the value of the game (regret…

New language model shows context length impacts generation quality and reasoning ability.

Machine learning's predictive power is limited by sample size, as shown by the Limits-to-Learning Gap.

Study tightens bounds for interpolating noisy data using minimum l1-norm.