This work proposes an online learning approach to tighten constraints in stochastic control problems.
problem Solving chance-constrained stochastic optimal control problems is computationally challenging.
method Reformulate chance constraints as a binary regression problem and use a GP model to learn constraint-tightening parameters online.
result The approach tightens constraints more effectively, leading to lower costs in numerical experiments.
In this paper we present a new approach for tightening upper bounds on the partition function. Our upper bounds are based on fractional covering bounds on the entropy function, and result in a concave program to compute these bounds and a convex program to tighten them. To solve these programs effectively for general r…
Unified framework for hard affine SDP constraints in vRKHSs.
problem Incorporating shape constraints into predictive models for rich function classes.
method Unified convex optimization framework using second-order cone tightening.
result Unified and modular approach for handling multiple shape constraints.
We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…
Paper tightens optimization bounds using conformal prediction.
problem Lack of practical informiveness in dual bounds from optimization solvers.
method Introduces conformal prediction framework to tighten loose primal and dual bounds.
result Proposed method produces tighter, more informative prediction intervals.
Improved neural network robustness certification through tighter convex relaxations.
problem Certifying neural network robustness to perturbed and adversarial inputs.
method Exploiting ReLU network structure, novel partition-based certification procedure.
result Tightens existing linear programming relaxations to achieve zero relaxation error asymptotically.
Paper improves variational inference by tightening bounds using perturbation theory.
problem Improving variational inference's bias and KL divergence approximation.
method Revisits perturbation theory to derive corrections that tighten variational bounds.
result New bounds are tighter and more mass-covering, leading to higher likelihoods.
Safety filter for unknown discrete-time systems with learned models and noise covariance.
problem Ensuring safety for unknown discrete-time linear systems with Gaussian noise.
method Develops a learning-based safety filter using empirical model and noise covariance, optimizing control actions to stay within safety constraints.
result Minimally modifies nominal control actions to ensure safety with high probability, tightening constraints as more data is collected.
New bounds tighten the generalization error of Gibbs algorithm.
problem Bounding the generalization error of Gibbs algorithm.
method Characterization of generalization error in terms of symmetrized KL information.
result Exact characterization of Gibbs algorithm's expected generalization error.
Sparse principal component analysis (PCA) involves nonconvex optimization for which the global solution is hard to obtain. To address this issue, one popular approach is convex relaxation. However, such an approach may produce suboptimal estimators due to the relaxation effect. To optimally estimate sparse principal su…
Optimal experiments tighten causal effect bounds efficiently.
problem Selecting experiments to tighten causal effect bounds from observational data.
method Formalized as max-potency problem, NP-hard. Polynomial-programming framework with graphical pruning criteria.
result Pruning criteria reduce search space significantly, enabling efficient experiment selection.
New framework tightens certified robustness gaps in machine learning models.
problem Persistent gap between theoretical certified robustness and empirical accuracy.
method Leverages Lipschitz continuity and novel confidence intervals.
result Improves robust accuracy, compressing the gap between theory and practice.
Polynomial bound on tightening curves on surfaces without increasing crossings.
problem Proving a polynomial bound on the number of monotonic homotopy moves for curves on surfaces.
method Combining tools from hyperbolic geometry and graph drawing algorithms.
result First polynomial bound on the number of monotonic homotopy moves, improving from exponential.
Bounding the generalization error of learning algorithms has a long history, which yet falls short in explaining various generalization successes including those of deep learning. Two important difficulties are (i) exploiting the dependencies between the hypotheses, (ii) exploiting the dependence between the algorithm'…
In this note we establish estimates for the harmonic map heat flow from S1 into a closed manifold, and use it to construct sweepouts with the following good property: each curve in the tightened sweepout, whose energy is close to the maximal energy of curves in the sweepout, is itself close to a closed geodesic.
FROWN optimizes neural network robustness, improving safety in deep learning.
problem Ensuring robustness of neural networks against adversarial attacks.
method Optimization-based approach to tighten robustness certificates.
result Deterministic CROWN solutions are optimal under mild constraints.
Paper tightens statistical aggregation results using local complexity.
problem Combining predictors to achieve nearly optimal predictions.
method Replacing global complexity with local complexity, using PAC-Bayes localization.
result Localized versions of classical aggregation bounds proven, improving previous results.
Improved bounds on geodesic lengths in Riemannian surfaces.
problem Finding precise lengths of geodesics in Riemannian surfaces.
method Proved curvature-free linear length bounds on geodesics.
result Length of kextth-shortest geodesic is at most 8kd. We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on t…
New method tightens bounds on causation probabilities using independent datasets.
problem Challenging point identification of causation probabilities without strong assumptions.
method Imposes counterfactual consistency between SCMs constructed from independent datasets and uses conditional mutual information.
result Significantly tighter bounds on causation probabilities are established.
UCRL3 improves UCRL2's efficiency in reinforcement learning by reducing exploration.
problem Long burn-in phases in numerical experiments of UCRL2.
method UCRL3 uses state-of-the-art time-uniform concentration inequalities and adaptive support computation to tighten exploration.
result UCRL3 achieves a better numerical improvement over UCRL2 in standard environments.
New method improves neural network verification by considering multivariate input space of ReLU neurons.
problem Improving the effectiveness of neural network verification algorithms.
method A new tightened convex relaxation for ReLU neurons considering multivariate input space.
result Our convex relaxation is significantly stronger than the commonly used univariate-input relaxation.
We consider a discrete-time approximation of paths of an Ornstein--Uhlenbeck process as a mean for estimation of a price of European call option in the model of financial market with stochastic volatility. The Euler--Maruyama approximation scheme is implemented. We determine the estimates for the option price for prede…
The study tightens bounds on binomial probabilities and minimums using KL-divergence.
problem Tightening bounds on binomial probabilities and minimums of i.i.d. Binomials.
method Applied Sanov's theorem to derive upper and lower bounds on binomial tail probabilities and minimums, expressed in terms of KL-divergence.
result High probability upper and lower bounds on the minimum of i.i.d. Binomial random variables, finite sample, asymptotically tight.
Optimizes decisions in time-varying distributions using online stochastic methods and Wasserstein distance.
problem Optimizing decisions in time-varying distributions using Wasserstein distance.
method Online proximal-gradient method, exact penalty method, constraint-tightening approach.
result Dynamic regret bounds for tracking and estimation error.
Adaptive uncertainty quantification improves black-box model predictions in generative AI.
problem Improving uncertainty quantification for black-box models in generative AI.
method Adaptive partitioning and local calibration of conformity scores.
result Local tightening of uncertainty sets with adaptive bands.
We describe a new technique for computing lower-bounds on the minimum energy configuration of a planar Markov Random Field (MRF). Our method successively adds large numbers of constraints and enforces consistency over binary projections of the original problem state space. These constraints are represented in terms of …
Safe learning in uncertain systems with state measurements and optimization.
problem Safe learning in nonlinear control-affine systems with unknown additive uncertainty.
method Model uncertainty as Gaussian noise, learn mean and covariance, use optimization to adjust control input.
result Guaranteed safety with arbitrarily large probability while learning and control proceed simultaneously.
The superposition of temporal point processes has been studied for many years, although the usefulness of such models for practical applications has not be fully developed. We investigate superposed Hawkes process as an important class of such models, with properties studied in the framework of least squares estimation…
New method improves deep learning by sampling worst-performing data.
problem Overfitting and poor generalization in deep learning.
method Distributional robust optimization to modify sample contributions.
result Faster convergence and higher accuracy in different scenarios.
New regularizers tighten convex relaxation bounds for neural networks.
problem Large gap between certifiable and empirical robustness in neural networks.
method Two regularizers to train neural networks yielding tighter convex relaxation bounds.
result Higher certified accuracy with proposed regularizers.
The paper tightens bounds on distances between Reeb graphs.
problem Certifying quasi-universality of distances between Reeb graphs.
method Establishes tight bi-Lipschitz bounds for various distances.
result Proves strict universality of the functional contortion distance for contour trees and coincides with interleaving distance for merge trees.
We introduce a new class of lower bounds on the log partition function of a Markov random field which makes use of a reversed Jensen's inequality. In particular, our method approximates the intractable distribution using a linear combination of spanning trees with negative weights. This technique is a lower-bound count…
New method speeds up solving L0-regularized least-squares problems.
problem Solving L0-regularized least-squares problems efficiently.
method Safe peeling for Branch-and-Bound algorithm.
result Significant gains in solving time and node exploration.
Study tightens bounds for interpolating noisy data using minimum l1-norm.
problem Predicting noisy data with minimum l1-norm interpolation.
method Provided matching upper and lower bounds for prediction error.
result Tight consistency up to negligible terms for d≫n. We consider the learning of multi-agent Hawkes processes, a model containing multiple Hawkes processes with shared endogenous impact functions and different exogenous intensities. In the framework of stochastic maximum likelihood estimation, we explore the associated risk bound. Further, we consider the superposition o…
We study the problem of instance segmentation in biological images with crowded and compact cells. We formulate this task as an integer program where variables correspond to cells and constraints enforce that cells do not overlap. To solve this integer program, we propose a column generation formulation where the prici…
This paper tightens information-theoretic bounds on generalization errors.
problem Understanding the discrepancy between training and testing data losses.
method Investigates the tightness of information-theoretic bounds on generalization error.
result The individual sample mutual information bound can be asymptotically tight under specific assumptions.
We propose a new complexity measure for Markov decision processes (MDPs), the maximum expected hitting cost (MEHC). This measure tightens the closely related notion of diameter [JOA10] by accounting for the reward structure. We show that this parameter replaces diameter in the upper bound on the optimal value span of a…
Improved PAC-Bayesian bounds by considering example difficulty.
problem Improving generalization bounds in machine learning.
method Introducing a modified excess risk that leverages example difficulty to reduce variance and tighten PAC-Bayesian bounds.
result Tighter PAC-Bayesian generalization bounds for machine learning models.
We give an algorithm to compute the stable lengths of pseudo-Anosovs on the curve graph, answering a question of Bowditch. We also give a procedure to compute all invariant tight geodesic axes of pseudo-Anosovs. Along the way we show that there are constants 1<a1<a2 such that the minimal upper bound on `slices' of …
New algorithm learns and unlearns from streaming data efficiently.
problem Continuous learning and unlearning from production data streams.
method Translated batch unlearning techniques to online setting using regret, sample complexity, and deletion capacity.
result Achieved logarithmic regret bound of O(lnT) for online unlearning. We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope. The algorithm is based on the conditional gradient method (Frank-Wolfe) and moves pseudomarginals within the marginal polytope through repeated maximum a posteriori (MAP) calls. This m…
News on inflation and monetary policy impacts US household inflation expectations.
problem Understanding how news affects inflation expectations.
method Monthly disaggregated US data from 1978 to 2016, controlling for various factors.
result News on rising inflation and easier monetary policy has a stronger impact on inflation expectations.
Strong theoretical guarantees of robustness can be given for ensembles of classifiers generated by input randomization. Specifically, an ℓ2 bounded adversary cannot alter the ensemble prediction generated by an additive isotropic Gaussian noise, where the radius for the adversary depends on both the variance of t…
The study tightens risk bounds for mixtures of experts using local differential privacy.
problem Improving risk bounds for mixtures of experts.
method Imposing local differential privacy (LDP) on the gating mechanism of mixtures of experts.
result Theoretical bounds exhibit logarithmic dependence on the number of experts and tighter than existing bounds.
Paper tightens lower bounds on decentralized training complexity.
problem Understanding and optimizing iteration complexity in decentralized training.
method Proved a tight lower bound on iteration complexity and proposed DeTAG algorithm.
result DeTAG achieves the theoretical lower bound with only a logarithmic gap.
We refine toxicity bounds for dynamic liquidation incentives in CP-AMM systems.
problem Ensuring stability in dynamic liquidation incentives in automated market makers.
method Derived state-dependent toxicity bounds for dynamic liquidation incentives, reconciling them with CP-AMM price dynamics.
result State-dependent bounds and liquidity-depth-only condition for dynamic liquidation incentives.