A note on extending Chernoff bound for unit interval random variables.
problem Extending the Chernoff bound to random variables in the unit interval.
method Proof of extension of the Chernoff bound.
result A proof provided for the extension of the Chernoff bound.
New bounds on continuous random variables' right-tail probabilities.
problem Finding precise upper and lower limits for right-tail probabilities of continuous random variables.
method Developed new bounds based on PDF, first derivative, and two parameters.
result The new bounds are tight for various continuous random variables.
The paper shows how neural networks with less decision boundary variability generalize better.
problem Improving neural network generalizability by reducing decision boundary variability.
method Introduces new measures (algorithm DB variability and (ε,η)-data DB variability) to quantify decision boundary variability and proves theoretical bounds on generalizability. result Neural networks with lower decision boundary variability have better generalizability, as shown by extensive experiments and theoretical bounds.
We prove new concentration inequalities for random variables.
problem Concentration of random variables in nonlinear functions.
method Efron-Stein inequalities and PAC-Bayesian approach.
result User-friendly concentration bounds for various applications.
Coordinate ascent variational inference is an important algorithm for inference in probabilistic models, but it is slow because it updates only a single variable at a time. Block coordinate methods perform inference faster by updating blocks of variables in parallel. However, the speed and stability of these algorithms…
Paper introduces new bounds linking data compressibility to generalization error.
problem Establishing data-dependent generalization bounds.
method Variable-size compressibility framework linking generalization error to compression rate of input data.
result New bounds depend on empirical data measure, subsuming existing PAC-Bayes and intrinsic dimension bounds.
Valid p-value for bounded random variables without distributional assumptions.
problem Calibration of predictive algorithms in a distribution-free setting.
method Built a super-uniform p-value based on a concentration inequality.
result Super-uniform p-value is tighter than existing alternatives.
Note improves confidence bounds for random variables.
problem Improving confidence bounds for random variables with unbounded ranges and different distributions.
method PAC-Bayes-ification of a derived confidence bound.
result Streamlined proofs for existing results.
Sharp concentration bounds for i.i.d. variables.
problem Controlling the tail probabilities of independent variables.
method Extension of Sanov's theorem using large deviations and information theory.
result Matching concentration and anti-concentration bounds for i.i.d. samples of any size.
New framework for tracking varying bounds in time series forecasting.
problem Forecasting bounded time series with varying bounds.
method Extended log-likelihood estimation, online maximum likelihood estimation, Normalized Gradient Descent (NGD) for quasiconvex optimization.
result Derive an Online Normalized Gradient Descent algorithm for online bound tracking.
This paper addresses sampling from bounded distributions using SGLD.
problem Sampling from models with bounded variables using SGLD.
method Introduces and evaluates various mapping techniques to transform unbounded samples into bounded ones.
result Invertible Lipschitz mappings overcame the pitfalls of existing methods and achieved weak convergence.
The paper sets limits on the accuracy of macroeconomic forecasts based on statistical moments and trade volumes.
problem Uncertainty in predicting macroeconomic variables like prices and returns.
method Defines theoretical lower bounds of uncertainty and upper limits on forecast accuracy based on statistical moments and trade volumes.
result Accuracy of forecasts of probabilities of macroeconomic variables doesn't exceed Gaussian approximations.
Novel Orlicz regrets consistently bound environmental variable statistics.
problem Consistent evaluation of stochastic environmental variables like water quality indices.
method Proposed novel Orlicz regrets for upper and lower bounds.
result Explicit linkage between Orlicz regrets and divergence risk measures.
UCB algorithms estimate uplifts in multi-variable reward systems.
problem Marketing campaigns and recommender systems with multiple outcome variables.
method UCB-style algorithms for estimating uplifts over a baseline.
result Sublinear regret bounds for multiple problem variants.
New analysis shows how temporal variability affects online learning performance.
problem Understanding the impact of temporal variability on online learning performance.
method Careful regret analysis and adaptive algorithm development.
result Proved a novel static regret bound that depends on temporal variability.
New model tackles causal bandits with dependent variables.
problem Understanding reward-maximizing interventions in causal networks with dependent variables.
method Introduces hierarchical causal bandit model with a contextual variable capturing interactions among variables.
result Derives nearly matching regret bounds for binary context in causal bandits with dependent arms.
Powerful generative models, particularly in Natural Language Modelling, are commonly trained by maximizing a variational lower bound on the data log likelihood. These models often suffer from poor use of their latent variable, with ad-hoc annealing factors used to encourage retention of information in the latent variab…
Develops ELBD for efficient feature selection in VAE latent variables.
problem Feature selection in latent variables of VAE and its variants.
method ELBD score algorithm and weak convergence approximation for optimization.
result Effective feature selection and optimization of VAE models.
Near-optimal confidence intervals for bounded data.
problem Online inference for sequential decision problems like A/B testing.
method Utilizing Bentkus' concentration results to improve on existing methods.
result Near-optimal confidence intervals confirmed favorable in synthetic and practical applications.
New bound on machine learning model performance using Jensen-Shannon information.
problem Understanding the performance of machine learning models.
method Proposes a new information-theoretic bound on generalization error.
result Shows that the new bound can be tighter than mutual information-based bounds under certain conditions.
This paper refines learning algorithm bounds, proving sharper generalization and lower bounds.
problem Improving generalization bounds for uniformly stable learning algorithms.
method Developed a new concentration inequality for weakly correlated random variables and proved sharper generalization and lower bounds.
result The new concentration inequality implies a stronger generalization bound than previous results.
A new method reduces CI tests for causal structure learning.
problem Exponential CI tests in constraint-based methods.
method Recursive Markov boundary-based approach.
result Significantly reduces CI tests compared to existing methods.
Maximizing high-dimensional, non-convex functions through noisy observations is a notoriously hard problem, but one that arises in many applications. In this paper, we tackle this challenge by modeling the unknown function as a sample from a high-dimensional Gaussian process (GP) distribution. Assuming that the unknown…
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
The paper accelerates regression algorithms by identifying saturated coordinates.
problem Non-negative and bounded-variable linear regression problems.
method Safe screening technique to identify saturated coordinates.
result The approach provides theoretical guarantees for identifying saturated coordinates.
We propose a new localized inference algorithm for answering marginalization queries in large graphical models with the correlation decay property. Given a query variable and a large graphical model, we define a much smaller model in a local region around the query variable in the target model so that the marginal dist…
Extends inequality for Rademacher complexities using p-stable variables.
problem Improving Rademacher complexity bounds using p-stable variables. method Extends contraction inequality to p-stable variables for 1<p<2. result New bounds for Rademacher complexities with p-stable variables. Upper bounds for CV errors apply to lasso and other models.
problem Bounding CV errors for lasso and similar models.
method Rademacher complexity and Orlicz-Ψν norm. result Upper bounds are tight and stable for lasso.
The study uses information theory to set lower bounds on model likelihoods.
problem Improving latent variable models by optimizing priors or likelihoods.
method Applying rate-distortion theory to find lower bounds on negative log likelihood.
result Rate-distortion theory can be used to optimize priors and likelihoods in latent variable models.
A hybrid model for Bayesian optimization handles mixed variables using MCTS for categorical and GP for continuous.
problem Optimizing functions with mixed variable types (continuous, integer, categorical).
method Merges MCTS for categorical and GP for continuous variables, integrates UCTS search strategy, and dynamically selects kernels.
result Hybrid models outperform traditional methods in Bayesian optimization.
The paper analyzes graph Laplacians on manifolds with curvature bounds and applies to non-collapsed spaces.
problem Analyzing spectral properties of graph Laplacians on manifolds with curvature constraints.
method Quantitative bounds on eigenvalues and eigenfunctions of graph Laplacians constructed from random variables on manifolds with uniform lower Ricci curvature bounds.
result Spectral convergence of graph Laplacians on manifolds with curvature bounds and in non-collapsed spaces.
Gradient matching with Gaussian processes is a promising tool for learning parameters of ordinary differential equations (ODE's). The essence of gradient matching is to model the prior over state variables as a Gaussian process which implies that the joint distribution given the ODE's and GP kernels is also Gaussian di…
We present two alternative ways to apply PAC-Bayesian analysis to sequences of dependent random variables. The first is based on a new lemma that enables to bound expectations of convex functions of certain dependent random variables by expectations of the same functions of independent Bernoulli random variables. This …
A refinement of Bennett's inequality is introduced which is strictly tighter than the classical bound. The new bound establishes the convergence of the average of independent random variables to its expected value. It also carefully exploits information about the potentially heterogeneous mean, variance, and ceiling of…
Study develops smart contract framework for procurement under demand variability.
problem Operational and economic implications of smart contract adoption under moderate uncertainty.
method Multi-supplier model with endogenized adoption costs, supplier readiness, and inventory penalties; analytical and numerical results.
result Partial adoption strategies support moderate demand variability, while excessive digital investment reduces profitability.
Graph neural nets learn better branch-and-bound policies.
problem Combinatorial optimization problems, especially hard ones.
method Graph convolutional neural network model trained via imitation learning.
result Improves over state-of-the-art methods and expert-designed rules.
In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…
We introduce a Bayesian Gaussian process latent variable model that explicitly captures spatial correlations in data using a parameterized spatial kernel and leveraging structure-exploiting algebra on the model covariance matrices for computational tractability. Inference is made tractable through a collapsed variation…
The paper examines bounds for stop-loss payoffs using transformed random variables.
problem Bounding stop-loss payoffs for a difference of two random variables.
method Analyzes crossing points of cdfs of original and transformed random variables.
result Unique pairwise crossing points for mortality-linked securities under symmetric copulas.
Variational autoencoders learn distributions of high-dimensional data. They model data with a deep latent-variable model and then fit the model by maximizing a lower bound of the log marginal likelihood. VAEs can capture complex distributions, but they can also suffer from an issue known as "latent variable collapse," …
New PAC-Bayes bounds for unbounded losses using Cramér-Chernoff techniques.
problem Developing bounds for unbounded losses in PAC-Bayesian settings.
method Introducing a new PAC-Bayes oracle bound using Cramér-Chernoff bounds and controlling random variable tails.
result Our bounds generalize and improve upon previous results, providing more informative and potentially tighter bounds.
We establish general versions of a variety of results for quasiconvex, lower-semicontinuous, and law-invariant functionals. Our results extend well-known results from the literature to a large class of spaces of random variables. We sometimes obtain sharper versions, even for the well-studied case of bounded random var…
Develops a method to estimate policy values robustly in the presence of confounding variables.
problem Infinite-horizon reinforcement learning with unobserved confounding variables makes policy evaluation unidentifiable.
method Robust approach estimating sharp bounds on policy value using optimization over state-occupancy ratios and sensitivity model.
result Proves convergence to sharp bounds as more confounded data is collected.
The standard margin-based structured prediction commonly uses a maximum loss over all possible structured outputs. The large-margin formulation including latent variables not only results in a non-convex formulation but also increases the search space by a factor of the size of the latent space. Recent work has propose…
The paper bounds and identifies joint probabilities in causal inference with monotonicity assumptions.
problem Bounding and identifying joint probabilities of potential outcomes and observed variables under monotonicity assumptions.
method Proposes new families of monotonicity assumptions, formulates bounding problem as linear programming, introduces new monotonicity assumption for identification.
result Validated methods through numerical experiments and applied to real-world datasets.
Recent work in unsupervised representation learning has focused on learning deep directed latent-variable models. Fitting these models by maximizing the marginal likelihood or evidence is typically intractable, thus a common approximation is to maximize the evidence lower bound (ELBO) instead. However, maximum likeliho…
Lower bounds found for nonconvex-strongly-concave min-max optimization problems.
problem Finding stationary points in nonconvex-strongly-concave min-max optimization.
method Provided lower bounds for first-order oracle complexity.
result Lower bounds of Ω(√κε⁻²) for deterministic oracles and Ω(√κε⁻² + κ¹/₃ε⁻⁴) for stochastic oracles.
New algorithm for multiarmed bandits with variable, unbounded delays achieves similar regret bounds.
problem Variable, unbounded delays in multiarmed bandits.
method Introduces a new algorithm that skips rounds with excessively large delays and uses a doubling scheme.
result Achieves the same regret bound as Exp3 with variable, unbounded delays.