We show that the variational representations for f-divergences currently used in the literature can be tightened. This has implications to a number of methods recently proposed based on this representation. As an example application we use our tighter representation to derive a general f-divergence estimator based on t…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “tighter representations”
Optimizing Variational Representations of Divergences and Accelerating their Statistical Estimationcs.LG
New method tightens variational representations of divergences for faster learning.
problem Improving tightness of variational representations of divergences for faster statistical estimation.
method Improved objective functionals constructed via an auxiliary optimization problem, leveraging neural network approximation.
result Tighter variational representations can result in significantly faster learning and more accurate estimation of divergences.
Paper develops tighter risk certificates for contrastive learning models.
problem Statistical theory for contrastive learning is lacking, especially for practical models like SimCLR.
method Develops non-vacuous PAC-Bayesian risk certificates considering practical SimCLR factors.
result Risk certificates for contrastive loss and downstream prediction are much tighter than previous results.
This paper improves Bayesian optimization methods with tighter regret bounds and practical solutions.
problem Improving Bayesian optimization methods with tighter regret bounds and practical solutions.
method The paper analyzes and compares different acquisition functions (GP-UCB, TS, PIMS) to achieve tighter Bayesian cumulative regret bounds and address practical issues.
result PIMS achieves the tighter BCR bound and avoids hyperparameter tuning, unlike GP-UCB and TS.
This paper improves DNN generalization by accurately estimating mutual information.
problem Intractability of estimating mutual information in DNNs.
method Introduces a probabilistic representation of DNNs to accurately estimate mutual information.
result Derives a tighter generalization bound than previous relaxations.
The paper accelerates ISTA and FISTA algorithms for composite optimization problems.
problem Improving convergence rates of ISTA and FISTA for composite optimization.
method Improved proximal subgradient norm minimization using Lyapunov function.
result Convergence rates of ISTA and FISTA are accelerated.
This work improves VAEs using MCMC methods for better variational bounds.
problem Improving the expressiveness of variational distributions in VAEs.
method Entropy-based adaptation for MALA/HMC chains to optimize tighter variational bounds.
result Higher held-out log-likelihoods and improved generative metrics.
In this paper, we reformulate the forest representation learning approach as an additive model which boosts the augmented feature instead of the prediction. We substantially improve the upper bound of generalization gap from to , while - the margin r…
Large and Deep Factor Modelsq-fin.ST
Deep neural networks decompose SDF into linear and nonlinear components.
problem Constructing accurate stochastic discount factors (SDFs) for pricing.
method Additive decomposition of a deep neural network trained to construct SDFs.
result The PTK representation delivers significant performance gains in equity data.
Paper shows linear convergence of ISTA and FISTA for ill-conditioned images.
problem Solving linear inverse problems with sparse representation in signal and image processing.
method Revisits iterative shrinkage-thresholding algorithms (ISTA) and improves their convergence properties.
result Linear convergence of ISTA and FISTA for strongly convex smooth parts, even in ill-conditioned cases.
We show that the stick-breaking construction of the beta process due to Paisley, et al. (2010) can be obtained from the characterization of the beta process as a Poisson process. Specifically, we show that the mean measure of the underlying Poisson process is equal to that of the beta process. We use this underlying re…
New tighter confidence bounds for sequential kernel regression.
problem Quantifying uncertainty in sequential learning algorithms.
method Martingale tail inequalities and conic programming.
result New confidence bounds are tighter than existing ones.
The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribut…
A new method to measure neural network expressiveness using tighter upper bounds.
problem Measuring the expressiveness of deep neural networks (DNNs).
method Proposes a new tighter upper bound for the number of linear regions in rectifier networks, using matrix computation.
result The proposed upper bound is tighter than existing ones and explains the performance improvements of skip connections and residual structures.
Improved PAC-Bayesian bounds by considering example difficulty.
problem Improving generalization bounds in machine learning.
method Introducing a modified excess risk that leverages example difficulty to reduce variance and tighten PAC-Bayesian bounds.
result Tighter PAC-Bayesian generalization bounds for machine learning models.
We present a representation learning algorithm that learns a low-dimensional latent dynamical system from high-dimensional \textit{sequential} raw data, e.g., video. The framework builds upon recent advances in amortized inference methods that use both an inference network and a refinement procedure to output samples f…
New tighter bounds for learning algorithms from Steinke & Zakynthinou's supersample setting.
problem Improving generalization bounds for machine learning algorithms.
method Information-theoretic approach using projected loss and Rademacher sequence.
result The new bounds are tighter than previous information-theoretic bounds.
RER improves sample complexity by updating in reverse order.
problem Theoretical analysis limits RER's convergence rate.
method Tighter analysis for larger learning rates and longer sequences.
result RER converges faster with larger learning rates and longer sequences.
Improved median of means estimator with tighter bounds.
problem Improving the efficiency and reliability of median of means estimator.
method Modification of the median of means estimator with sub-Gaussian deviation bounds.
result Achieves nearly optimal constants under minimal assumptions.
We provide theoretical and empirical evidence that using tighter evidence lower bounds (ELBOs) can be detrimental to the process of learning an inference network by reducing the signal-to-noise ratio of the gradient estimator. Our results call into question common implicit assumptions that tighter ELBOs are better vari…
American options in a multi-asset market model with proportional transaction costs are studied in the case when the holder of an option is able to exercise it gradually at a so-called mixed (randomised) stopping time. The introduction of gradual exercise leads to tighter bounds on the option price when compared to the …
In statistical inference problems, we wish to obtain lower bounds on the minimax risk, that is to bound the performance of any possible estimator. A standard technique to obtain risk lower bounds involves the use of Fano's inequality. In an information-theoretic setting, it is known that Fano's inequality typically doe…
Proposes methods to identify and estimate counterfactual distributions with confounding.
problem Estimating counterfactual distributions in the presence of confounding.
method Nonparametric identification and semiparametric estimation using conditional copulas and machine learning.
result Valid inference for individual-level effects and nonparametric identifiability of latent confounding subspace.
Ahpatron improves online kernel learning with tighter mistake bounds.
problem Improving mistake bounds in online kernel learning with budget constraints.
method Introducing Ahpatron, a new model that uses an aggressive updating rule and a budget maintenance mechanism to approximate AVP.
result Ahpatron achieves tighter mistake bounds compared to previous models.
The variational autoencoder (VAE; Kingma, Welling (2014)) is a recently proposed generative model pairing a top-down generative network with a bottom-up recognition network which approximates posterior inference. It typically makes strong assumptions about posterior inference, for instance that the posterior distributi…
A new method for tighter privacy loss accounting in adaptive analyses.
problem Ensuring individual privacy in adaptive analyses while staying within a privacy budget.
method A personalized privacy loss estimate and a Rényi differential privacy filter.
result Personalized privacy loss accounting can be practical and tighter than existing methods.
High-dimensional representations often have a lower dimensional underlying structure. This is particularly the case in many decision making settings. For example, when the representation of actions is generated from a deep neural network, it is reasonable to expect a low-rank structure whereas conventional structures l…
This work improves multi-modal generative models by using permutation-invariant neural networks.
problem Improving multi-modal generative models with tighter variational objectives.
method Developed more flexible aggregation schemes based on permutation-invariant neural networks.
result Our variational objective and flexible aggregation models can better approximate the true joint distribution.
We investigate in this paper the architecture of deep convolutional networks. Building on existing state of the art models, we propose a reconfiguration of the model parameters into several parallel branches at the global network level, with each branch being a standalone CNN. We show that this arrangement is an effici…
Game (Israeli) options in a multi-asset market model with proportional transaction costs are studied in the case when the buyer is allowed to exercise the option and the seller has the right to cancel the option gradually at a mixed (or randomised) stopping time, rather than instantly at an ordinary stopping time. Allo…
Insights in power grid pixel maps (PGPMs) refer to important facility operating states and unexpected changes in the power grid. Identifying insights helps analysts understand the collaboration of various parts of the grid so that preventive and correct operations can be taken to avoid potential accidents. Existing sol…
Paper tightens variational GP approximations for large datasets.
problem Scaling Gaussian processes to large datasets.
method Relaxing the standard assumption about inducing points' posterior matching the prior, leading to a tighter variational approximation.
result The proposed approximation consistently matches or outperforms standard sparse variational GPs while maintaining computational cost.
New method prevents posterior collapse in iVAE models.
problem Posterior collapse in iVAE models where observations and ICs are independent given covariates.
method Developed CI-iVAE by considering a mixture of encoder and posterior distributions in the objective function.
result Prevents posterior collapse, resulting in latent representations with more information of the observations.
Introduces Cauchy-Schwarz divergence for domain adaptation.
problem Evaluating discrepancy between source and target domains in unsupervised domain adaptation.
method Introduces Cauchy-Schwarz divergence as a measure for evaluating discrepancy between marginal and conditional distributions.
result CS divergence offers a tighter generalization error bound than Kullback-Leibler divergence.
Improved algorithms for stochastic linear bandits using tighter confidence sequences.
problem Stochastic linear bandits with improved worst-case regret guarantees.
method Novel tail bound for adaptive martingale mixtures to construct tighter confidence sequences.
result Linear bandit algorithm achieves competitive worst-case regret.
STanHop predicts multivariate time series with memory-enhanced capabilities.
problem Predicting multivariate time series with memory-enhanced capabilities.
method Sparse Tandem Hopfield Network (STanHop) with two external memory modules.
result STanHop outperforms dense Hopfield models in memory retrieval error.
In this paper we derive robust super- and subhedging dualities for contingent claims that can depend on several underlying assets. In addition to strict super- and subhedging, we also consider relaxed versions which, instead of eliminating the shortfall risk completely, aim to reduce it to an acceptable level. This yie…
Paper improves neural network robustness certification with tighter radii estimates.
problem Certifying neural networks' robustness against adversarial attacks.
method Advanced algorithms for discrete and continuous domains, optimizing sample size, standard deviation, and temperature.
result Significant improvement in certified test-set accuracy with tighter certified radii bounds.
The CUR matrix decomposition is an important extension of Nyström approximation to a general matrix. It approximates any data matrix in terms of a small number of its columns and rows. In this paper we propose a novel randomized CUR algorithm with an expected relative-error bound. The proposed algorithm has the advanta…
This paper tackles non-vacuous generalization bounds in ReLU networks by resolving rescaling invariances.
problem Non-vacuous generalization guarantees for ReLU networks with rescaling invariances.
method Proposes a lifted representation to resolve rescaling invariances and studies KL-based rescaling-invariant PAC-Bayes bounds.
result KL-based rescaling-invariant PAC-Bayes bounds provide tighter guarantees and resolve discrepancies in network complexity.
Tighter 'uniform bounds for Black-Scholes implied volatility' and the applications to root-findingq-fin.MF
Improved bounds for Black-Scholes volatility lead to faster root-finding.
problem Finding accurate implied volatility for Black-Scholes model.
method Systematic use of option delta to derive tighter bounds, proposing a Newton-Raphson algorithm.
result Proposed algorithm converges rapidly for all price ranges, especially useful for extreme option prices.
New analysis explains pathology of deep Gaussian processes.
problem Pathology of deep Gaussian processes reduces learning capacities with increased layers.
method Study nonlinear dynamic systems corresponding to DGPs, derive recurrence relations.
result Provide tighter bounds and rate of convergence for dynamic systems.
Improved bounds on learning algorithms' performance using conditional mutual information.
problem Bounding the generalization error of learning algorithms.
method Introducing conditional mutual information and disintegrated mutual information to tighten bounds.
result New bounds are tighter than previous ones, especially for noisy, iterative algorithms.
Matrix rank minimization problem is in general NP-hard. The nuclear norm is used to substitute the rank function in many recent studies. Nevertheless, the nuclear norm approximation adds all singular values together and the approximation error may depend heavily on the magnitudes of singular values. This might restrict…
New model generates graphs with tighter likelihood bounds and better quality.
problem Intractable likelihood of autoregressive graph models.
method Derive exact joint probability, approximate node orderings, variational inference.
result Lower bound on log-likelihood is significantly tighter than previous methods.
New research limits what GNNs can compute and generalizes their performance.
problem Limits of GNNs in computing graph properties and generalization bounds.
method Novel graph-theoretic formalism and data-dependent generalization bounds.
result Proves GNNs can't compute certain graph properties and provides tighter generalization bounds.
Paper improves robustness of GNNs against adversarial attacks.
problem Understanding robust generalization of GNNs in adversarial settings.
method Develops a sensitivity-aware PAC-Bayesian framework for MPGNNs.
result Derives tighter robust generalization bounds for MPGNNs.
Convolutional neural networks (CNNs) have achieved breakthrough performances in a wide range of applications including image classification, semantic segmentation, and object detection. Previous research on characterizing the generalization ability of neural networks mostly focuses on fully connected neural networks (F…