We give the proof of a tight lower bound on the probability that a binomial random variable exceeds its expected value. The inequality plays an important role in a variety of contexts, including the analysis of relative deviation bounds in learning theory and generalization bounds for unbounded loss functions.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,932 papers · 148 categories
Trend · papers per month
48 results for “Expected Tight Bounds”
Proposes ETB for tighter bounds in robust training of deep networks.
problem Training robust deep neural networks to adversarial attacks.
method Proposes ETB, a provably tighter bound approach for interval bound propagation.
result Achieves orders of magnitudes tighter bounds compared to IBP.
Upper and lower bounds on regret for noisy optimization of Brownian motion.
problem Optimizing a one-dimensional Brownian motion with noisy observations.
method Upper bound uses confidence bounds and Markov property; lower bound uses hypothesis testing reduction.
result Upper and lower bounds are tight up to a factor of .
Study finds optimal regret bound for multi-armed bandit problem with expert advice.
problem Optimizing decision-making in a multi-armed bandit problem with expert advice.
method Proved a tight lower bound matching the upper bound of Kale (2014) for minimax expected regret.
result The minimax optimal expected regret is Θ(√(T K log (N/K))) for the problem.
A stochastic combinatorial semi-bandit is an online learning problem where at each step a learning agent chooses a subset of ground items subject to constraints, and then observes stochastic weights of these items and receives their sum as a payoff. In this paper, we close the problem of computationally and sample effi…
In the Bayesian approach to sequential decision making, exact calculation of the (subjective) utility is intractable. This extends to most special cases of interest, such as reinforcement learning problems. While utility bounds are known to exist for this problem, so far none of them were particularly tight. In this pa…
Many important optimization problems, such as the minimum spanning tree and minimum-cost flow, can be solved optimally by a greedy method. In this work, we study a learning variant of these problems, where the model of the problem is unknown and has to be learned by interacting repeatedly with the environment in the ba…
Unified framework for differentiable graph partitioning with probabilistic cuts.
problem Lack of general guarantees and principled gradients in prior probabilistic relaxations of graph cuts.
method Unified probabilistic framework covering a wide class of cuts, including Normalized Cut, with tight analytic upper bounds.
result Rigorous, numerically stable foundation for scalable, differentiable graph partitioning.
Investigates tight PAC-Bayes bounds for small datasets.
problem Tightening PAC-Bayes bounds for small data.
method Generic PAC-Bayes theorem, meta-learning, synthetic tasks.
result PAC-Bayes bounds are competitive with Chernoff bounds but not as tight.
The study identifies conditions for algorithms to have tight generalization bounds.
problem Understanding which algorithms have tight generalization bounds.
method Analyzing conditions that preclude tight generalization bounds and identifying stable algorithms.
result Stable algorithms have tight generalization bounds, while unstable ones do not.
Paper proves tight lower bounds for online multicalibration, separating it from marginal calibration.
problem Proving lower bounds for online multicalibration in relation to marginal calibration.
method Information-theoretic approach, constructing group families from orthonormal bases.
result Establishes tight lower bounds for online multicalibration, matching upper bounds up to logarithmic factors.
Conformal Risk Controlstat.ME
Extends conformal prediction for controlling expected risk of monotone loss functions.
problem Controlling expected risk of monotone loss functions.
method Generalizes split conformal prediction with coverage guarantee, extending to distribution shift, quantile risk, multiple, adversarial, and expectations of U-statistics.
result Tight up to an factor, with worked examples in computer vision and natural language processing.
Researchers add random metrics to data models to enable operations.
problem Lack of meaningful operations in learned low-dimensional representations.
method Endow latent space of generative models with a random Riemannian metric.
result Derived tight error bounds on expected distances in deterministic approximations.
We consider distributed statistical optimization in one-shot setting, where there are machines each observing i.i.d. samples. Based on its observed samples, each machine sends a -bit-long message to a server. The server then collects messages from all machines, and estimates a parameter that minimizes an exp…
This paper develops upper and lower bounds on the influence measure in a network, more precisely, the expected number of nodes that a seed set can influence in the independent cascade model. In particular, our bounds exploit nonbacktracking walks, Fortuin-Kasteleyn-Ginibre (FKG) type inequalities, and are computed by m…
MRCs minimize worst-case expected 0-1 loss and provide performance guarantees.
problem Minimizing expected 0-1 loss in classification.
method Minimizes worst-case expected 0-1 loss over uncertainty sets defined by linear constraints.
result Achieves efficient learning and generalization with performance guarantees.
Jiang et al. (2020) found no uniformly tight generalization bounds for neural networks in the overparameterized setting.
problem Finding uniformly tight generalization bounds for neural networks in the overparameterized setting.
method Examined more than a dozen generalization bounds, proving that no bounds can be uniformly tight in the overparameterized setting.
result No generalization bounds can be uniformly tight in the overparameterized setting.
HAMBO estimates policy performance by hallucinating worst-case trajectories, providing valid lower bounds.
problem Conservative off-policy evaluation of policies in real-world applications.
method HAMBO hallucinates worst-case trajectories based on learned model uncertainty.
result Valid lower bounds on policy performance, converging to true expected return under regular conditions.
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …
One fundamental goal in any learning algorithm is to mitigate its risk for overfitting. Mathematically, this requires that the learning algorithm enjoys a small generalization risk, which is defined either in expectation or in probability. Both types of generalization are commonly used in the literature. For instance, …
In a financial market model, we consider variations of the problem of minimizing the expected time to upcross a certain wealth level. For exponential Levy markets, we show the asymptotic optimality of the growth-optimal portfolio for the above problem and obtain tight bounds for the value function for any wealth level.…
Study MNL-Bandit in non-stationary settings with optimal regret bound.
problem Optimizing decisions in a non-stationary environment for multi-armed bandit problems.
method Develops an algorithm with worst-case expected regret bound and introduces new techniques to handle non-stationarity.
result Optimal regret bound proven for the MNL-Bandit problem in non-stationary environments.
Exactly Tight Information-Theoretic Generalization Error Bound for the Quadratic Gaussian Problemcs.IT
New bound matches exact generalization error for quadratic Gaussian problem.
problem Understanding generalization error in quadratic Gaussian problems.
method Information-theoretic approach with new ingredients.
result Exact tight bound for generalization error.
Classifies tight contact structures on specific Seifert fibered manifolds.
problem Classifying tight contact structures on Seifert fibered manifolds.
method Constructed contact structures using Legendrian surgery and used convex surface theory for the upper bound.
result Found the lower and upper bounds for tight contact structures.
Bayesian sequence prediction is a simple technique for predicting future symbols sampled from an unknown measure on infinite sequences over a countable alphabet. While strong bounds on the expected cumulative error are known, there are only limited results on the distribution of this error. We prove tight high-probabil…
A new stopping criterion for active learning based on deterministic generalization bounds.
problem Determining the optimal stopping point for active learning when data acquisition is costly.
method The proposed stopping criterion is based on the difference in expected generalization errors and hypothesis testing, derived from PAC-Bayesian theory.
result The proposed stopping criterion effectively stops active learning by combining an upper bound with a statistical test.
Develops bounds for deep learning risk via Hilbert coresets.
problem Risk estimation for complex deep learning models.
method Hilbert coreset approach for transductive risk bounds.
result Effective and meaningful bounds for deep neural networks.
New algorithms minimize PAC-Bayesian C-Bound for majority voting, leading to scalable and accurate predictors.
problem Improving majority vote classifiers using PAC-Bayesian bounds.
method Directly optimizing PAC-Bayesian guarantees on the C-Bound with gradient descent.
result Self-bounding majority vote learning algorithms with scalable and accurate predictors.
Paper proposes a cost-sensitive conformal training method with provably controllable learning bounds.
problem Uncertainty quantification and learning bounds in conformal prediction.
method Cost-sensitive conformal training algorithm that minimizes the expected size of prediction sets using rank weighting.
result Theoretical analysis shows tightness between weighted objective and expected size of conformal prediction sets.
The paper relaxes the stability condition to boost confidence in generalization for randomized learning algorithms.
problem The tension between uniform stability and -stability in generalization bounds.
method Establishes in-expectation first moment generalization error bounds for -stable randomized learning algorithms and uses subbagging to achieve near-tight exponential bounds.
result Improves generalization bounds for convex and non-convex optimization problems with SGD.
Optimizing rewards under budget constraints with correlated costs and rewards.
problem Maximizing total expected reward under a budget constraint on total cost with correlated and potentially heavy-tailed cost-reward pairs.
method Proposes algorithms exploiting correlation between cost and reward via linear minimum mean-square error estimation to achieve tight regret bounds.
result Achieves regret for a budget under certain moment conditions.
We study the multiclass online learning problem where a forecaster makes a sequence of predictions using the advice of experts. Our main contribution is to analyze the regime where the best expert makes at most mistakes and to show that when , the expected number of mistakes made by the optima…
Bounds on treatment effect sensitivity in causal reasoning using Hölder's inequality.
problem Estimating treatment effects in presence of unobserved confounders.
method Using Hölder's inequality, derived bounds on confounding bias based on unmeasured confounding strength.
result Bounds are tight under specific conditions of independence between U and T/Y.
Sharp bounds on Euler characteristics for lens space fillings.
problem Limits on the Euler characteristics of symplectic fillings of tight contact structures on lens spaces.
method Topological analysis of lens spaces and their coverings.
result Sharp upper and lower bounds on Euler characteristics for minimal fillings.
The paper explores the limits of tight PAC-Bayes bounds for cheap models in robust statistics.
problem The challenge of obtaining meaningful bounds on the error of learning algorithms without prior assumptions.
method Investigates tight PAC-Bayes bounds for robust models with minimal cost.
result Demonstrates the limits of obtaining tight PAC-Bayes bounds for cheap models.
A new method optimizes robustness measures under input uncertainty using randomized Gaussian process upper confidence bound.
problem Optimizing robustness measures under input uncertainty.
method Randomized robustness measure GP-UCB (RRGP-UCB) that samples β from a chi-squared-based distribution.
result RRGP-UCB provides tight bounds on expected regret.
There has been renewed recent interest in developing effective lower bounds for Dynamic Time Warping (DTW) distance between time series. These have many applications in time series indexing, clustering, forecasting, regression and classification. One of the key time series classification algorithms, the nearest neighbo…
Tight geodesics were introduced by Masur-Minsky in [17]. They and their hierarchies have been a powerful tool in the study of the curve complex, mapping class groups, Teichmüller spaces, and hyperbolic 3-manifolds. In the same paper, they showed that there are at least one and at most finitely many tight geodesics betw…
We introduce the notion of tight homomorphism into a locally compact group with nonvanishing bounded cohomology and study these homomorphisms in detail when the target is a Lie group of Hermitian type. Tight homomorphisms between Lie groups of Hermitian type give rise to tight totally geodesic maps of Hermitian symmetr…
Classifies real tight contact structures on lens spaces and solid tori.
problem Classifying real tight contact structures on specific 3-manifolds.
method Equivariant contact isotopy, real open book decompositions, and isolated real algebraic surface singularities.
result Unique real tight structures on and , at most one on , and bounds on the count.
Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms correspond to extrema of the Bethe and Kikuchi free energy. However, belief propagation does not always converge, which explains the need for approach…
Optimal sequential testing for Markovian data with lower and upper bounds.
problem Sequential hypothesis testing for Markovian data.
method Non-asymptotic lower bounds and optimal test design.
result Optimal test matches lower bound asymptotically.
The paper studies a stochastic majority vote approach to improve classifier accuracy.
problem Improving classifier accuracy over ensembles of classifiers.
method Minimizing a PAC-Bayes generalization bound with Dirichlet distributions.
result Achieves state-of-the-art accuracy and tight generalization bounds.
We study a sequential resource allocation problem between a fixed number of arms. On each iteration the algorithm distributes a resource among the arms in order to maximize the expected success rate. Allocating more of the resource to a given arm increases the probability that it succeeds, yet with a cut-off. We follow…
Study on gradient complexity of private optimization with private oracles.
problem Analyzing the efficiency of differentially private optimization algorithms.
method Lower bounds on the number of first-order oracle queries for private optimization.
result Lower bounds on the number of queries for private optimization algorithms, showing a dimension-dependent runtime penalty.
Between Stochastic and Adversarial Online Convex Optimization: Improved Regret Bounds via Smoothnesscs.LG
Improved online convex optimization bounds between stochastic and adversarial settings.
problem Understanding optimization tasks that are neither i.i.d. nor fully adversarial.
method Establishing novel regret bounds exploiting smoothness of expected losses.
result Regret bounds improve on previous results by reducing dependence on maximum gradient length to variance of gradients.
In-Context Learning as Nonparametric Conditional Probability Estimation: Risk Bounds and Optimalitystat.ML
Paper analyzes risk bounds for in-context learning in multiclass classification.
problem Risk bounds for in-context learning in multiclass classification.
method Formalizes tasks as sequences of labeled examples and queries, estimates conditional class probabilities, establishes oracle inequality for KL divergence.
result ICL achieves minimax optimal rate for conditional probability estimation.
CRS model improves ranking data modeling with theoretical guarantees.
problem Lack of rich, multimodal models for ranking data.
method Contextual Repeated Selection (CRS) model for multimodal ranking data.
result CRS model significantly outperforms existing methods in various ranking contexts.