arXiv research

A new framework for information theory considers computational constraints.

In this paper, we examine the fundamental performance limits of prediction, with or without side information. More specifically, we derive generic lower bounds on the $\mathcal{L}_p$ norms of the prediction errors that are valid for any prediction algorithms and for any data distributions. Meanwhile, we combine the ent…

Combines Bézier curves with Gaussian processes for better sequential data modeling.

Physics-informed GP regression solves eigenvalue problems by identifying non-trivial eigenspaces.

We study how to adapt to smoothly-varying ('easy') environments in well-known online learning problems where acquiring information is expensive. For the problem of label efficient prediction, which is a budgeted version of prediction with expert advice, we present an online algorithm whose regret depends optimally on t…

The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.

Study learns optimal strategies in imperfect information games with self-play.

The paper addresses fairness in online learning by extending auditing schemes and presenting efficient algorithms.

A new Helmholtzian operator from point clouds for flow analysis.

LeanML reduces machine learning project waste by estimating best performance without training models.

Stokes' theorem's boundary maximizes entropy.

Bayesian framework calibrates imperfect models using physics-informed priors and Hamiltonian Monte Carlo.

We give an online algorithm and prove novel mistake and regret bounds for online binary matrix completion with side information. The mistake bounds we prove are of the form $\tilde{O}(D/γ^2)$. The term $1/γ^2$ is analogous to the usual margin term in SVM (perceptron) bounds. More specifically, if we assume that there i…

How many bits of information are required to PAC learn a class of hypotheses of VC dimension $d$? The mathematical setting we follow is that of Bassily et al. (2018), where the value of interest is the mutual information $\mathrm{I}(S;A(S))$ between the input sample $S$ and the hypothesis outputted by the learning algo…

The paper connects Bergman geometry with information geometry.

This work uses QPGPs to improve ILC performance in repetitive tasks.

Unified CMI bounds for meta learning, bridging information-theoretic and classical learning theory.

Jordan algebras in information geometry linked to metrics on probability distributions.

This paper proves the theoretical advantage of unsupervised pretraining for machine learning tasks.

Assume that $M(\mathcal{T})$ is a rational homology sphere plumbed 3-manifold associated with a connected negative definite graph $\mathcal{T}$. We consider the combinatorial multivariable Poincaré series associated with $\mathcal{T}$ and its counting functions, which encode rich topological information. Using the `per…

PASTIS method selects simple models from noisy data.

Identifies minimal training subset to flip a prediction.

Improved mean estimation for symmetric distributions with finite-sample guarantees.

The space of all probability measures having positive density function on a connected compact smooth manifold $M$, denoted by $\mathcal{P}(M)$, carries the Fisher information metric $G$. We define the geometric mean of probability measures by the aid of which we investigate information geometry of $\mathcal{P}(M)$, equ…

New algorithm reduces regret and constraint violation in online convex optimization with predictions.

Paper justifies ideal point forecasts as measurable, clarifying conditions for their existence.

Bayesian active learning method improved for censored regression data.

Gaussian processes (GPs) provide a powerful framework for extrapolation, interpolation, and noise removal in regression and classification. This paper considers constraining GPs to arbitrarily-shaped domains with boundary conditions. We solve a Fourier-like generalised harmonic feature representation of the GP prior in…

Accurately annotating large scale dataset is notoriously expensive both in time and in money. Although acquiring low-quality-annotated dataset can be much cheaper, it often badly damages the performance of trained models when using such dataset without particular treatment. Various methods have been proposed for learni…

Physics-informed DeepONets solve PDEs without paired data, predicting solutions quickly.

PeL separates sensory interface optimization from decision learning.

The paper characterizes the efficiency of transferring knowledge from a teacher to a student classifier over finite domains.

Improved Thompson Sampling for logistic bandits with information-theoretic analysis.

Study of psc metrics via block diffeomorphisms and cubical sets.

Paper solves learning imperfect-information games with fewer episodes.

The information bottleneck (IB) problem tackles the issue of obtaining relevant compressed representations $T$ of some random variable $X$ for the task of predicting $Y$. It is defined as a constrained optimization problem which maximizes the information the representation has about the task, $I(T;Y)$, while ensuring t…

New algorithms achieve optimal regret in sliding window model with limited memory.

Estimates KL divergence with fairness considerations for sub-populations.

Let $\mathcal M= (M,\mathcal O_\mathcal M)$ be a smooth supermanifold with connection $\nabla$ and Batchelor model $\mathcal O_\mathcal M\congΓ_{ΛE^\ast}$. From $(\mathcal M,\nabla)$ we construct a connection on the total space of the vector bundle $E\to{M}$. This reduction of $\nabla$ is well-defined independently of …

New bounds on efficiency for conformalized regression methods.

Hybrid GP/NN framework for operator learning improves performance and enables zero-shot predictions.

New method finds balanced clusters in graphs using auxiliary information.

Improved bounds for continuous functions in online learning.

The paper proposes multicalibration to improve matching in graphs with imperfect predictors.

Numerous control and learning problems face the situation where sequences of high-dimensional highly dependent data are available but no or little feedback is provided to the learner, which makes any inference rather challenging. To address this challenge, we formulate the following problem. Given a series of observati…

NeuralChaos efficiently approximates complex stochastic processes.

Paper explores how uncertainty quantification improves Transformer's in-context learning ability.

Improved computational complexity in statistical models using second-order information.