Stochastic trace estimation with tensor train random vectors
problem Stochastic trace estimation for large-scale matrices
method Gaussian random tensor train vectors
result Median-of-means variant achieves dimension-independent guarantees
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Stochastic trace approximations”
This paper optimizes matrix-based Renyi's entropy computation for large datasets.
problem Efficiently calculating matrix-based Renyi's entropy for large-scale applications.
method Develops randomized approximations for matrix-based Renyi's entropy with arbitrary α orders.
result Achieves a significant reduction in time complexity from O(n^3) to O(n^2sm), where s, m << n.
Quantum stochastic flow computes heat kernel traces for Ricci flat manifolds.
problem Computing heat kernel traces for Ricci flat manifolds.
method Quantum stochastic differential equation (qsde) on Fock space over differential 1-forms, adapted flow construction.
result Trace of the connection Laplacian heat kernel can be computed over any compact Ricci-flat Riemannian manifold.
Bayesian generative models can flag performance loss, bias, and out-of-distribution image contentcs.LG
SLUG method detects bias and out-of-distribution content in generative models.
problem Generative models can underrepresent certain groups and fail on out-of-distribution data.
method SLUG: A new uncertainty quantification method for VAEs combining Laplace approximations and stochastic trace estimators.
result SLUG's UQ score correlates with bias and out-of-distribution content.
A new model captures forward curve dynamics with stochastic volatility.
problem Modeling continuous-time evolution of forward curves in financial markets.
method Affine stochastic volatility model with modulated dynamics.
result Model allows for maturity-specific risk and volatility clustering.
New algorithms for collaborative reinforcement learning with limited communication.
problem Efficiently learning value functions in multi-agent systems with strict information constraints.
method Distributed gradient-based temporal difference algorithms with consensus schemes.
result Parameter estimates converge to ODEs with defined invariant sets under general assumptions.
A new method for optimizing deep neural networks using TKFAC.
problem Optimizing deep neural networks with second-order methods.
method Proposes Trace-restricted Kronecker-factored Approximate Curvature (TKFAC) for Fisher information matrix approximation.
result TKFAC improves performance on deep network architectures compared to state-of-the-art algorithms.
The paper introduces Causal Neural Operators to approximate operators in stochastic analysis.
problem Leveraging temporal structure in non-linear operators for deep learning models.
method Designing a deep learning model framework for infinite-dimensional linear metric spaces.
result Causal Neural Operators can uniformly approximate Hölder or smooth trace class operators.
Method uses deep learning to estimate traffic intensity.
problem Estimating stochastic intensity of traffic processes.
method Deep neural networks for nonlinear filtering.
result Deep learning method accurately estimates traffic intensity.
Global convergence of an online (stochastic) limited memory version of the Broyden-Fletcher- Goldfarb-Shanno (BFGS) quasi-Newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established. Lower and upper bounds on the Hessian eigenvalues of the sample …
This study improves convergence of two-timescale SA under Markovian noise in reinforcement learning.
problem Stability and convergence of two-timescale stochastic approximations under Markovian noise.
method Introduced a new control strategy for the fast timescale parameter.
result Established almost sure convergence of TDC with eligibility traces under off-policy learning with linear function approximation.
Ray tracing sampler improves neural network sampling efficiency and resilience.
problem Sampling neural network posterior distributions efficiently and robustly.
method Markov Chain Monte Carlo using ray tracing through likelihood space.
result Significantly higher resilience to gradient heating compared to HMC.
Policy evaluation with linear function approximation is an important problem in reinforcement learning. When facing high-dimensional feature spaces, such a problem becomes extremely hard considering the computation efficiency and quality of approximations. We propose a new algorithm, LSTD()-RP, which leverages rando…
Hutch++ optimizes trace estimation for generative models, reducing variance and improving quality.
problem High variance and scalability issues in Hutchinson estimators for generative models.
method Hutch++ is an optimal stochastic trace estimator designed to minimize training variance while maintaining transport optimality.
result Hutch++ leads to higher quality generations and effective variance reduction in various applications.
We introduce a method called TracIn that computes the influence of a training example on a prediction made by the model. The idea is to trace how the loss on the test point changes during the training process whenever the training example of interest was utilized. We provide a scalable implementation of TracIn via: (a)…
Stochastic Approximation (SA) is a popular approach for solving fixed-point equations where the information is corrupted by noise. In this paper, we consider an SA involving a contraction mapping with respect to an arbitrary norm, and show its finite-sample error bounds while using different stepsizes. The idea is to c…
We present a new trace estimator of the matrix whose explicit form is not given but its matrix multiplication to a vector is available. The form of the estimator is similar to the Hutchison stochastic trace estimator, but instead of the random noise vectors in Hutchison estimator, we use small number of probing vectors…
-adic framed braids IImath.GT
The Yokonuma-Hecke algebras are quotients of the modular framed braid group and they support Markov traces. In this paper, which is sequel to Juyumaya and Lambropoulou (2007), we explore further the structures of the -adic framed braids and the -adic Yokonuma-Hecke algebras constructed in Juyumaya and Lambropoulo…
New pivoting strategy improves trace norm contraction in low-rank approximation.
problem Finding good low-rank approximations of symmetric, positive-definite matrices.
method Choosing rows with likelihood proportional to for randomly pivoted partial Cholesky algorithm.
result Same trace norm contraction result in Frobenius norm for improved pivoting strategy.
The paper analyzes off-policy TD-learning using generalized Bellman operators and provides finite-sample bounds.
problem High variance in off-policy TD-learning due to importance sampling.
method Derives finite-sample bounds for off-policy TD-like algorithms using generalized Bellman operators.
result First-known finite-sample guarantees for several off-policy TD algorithms.
Spectral regularization simplifies sequence models by focusing on grammatical simplicity.
problem Sequence modeling challenges in learning tasks.
method Introduces spectral regularization based on Hankel matrices and trace norm, addressing bi-infinite matrices with an unbiased estimator.
result Demonstrates spectral regularization's potential benefits on Tomita grammars.
Stochastic Trace Optimization of Parameter Dependent Matrices Based on Statistical Learning Theorystat.ML
Estimates matrix trace optimization with statistical learning theory.
problem Optimizing trace of parameter-dependent matrices.
method Monte Carlo estimator with bounds derived from epsilon nets and generic chaining.
result Predicts small sampling amount for matrices with small off-diagonal mass.
A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variantscs.LG
Unified framework for finite-sample RL algorithms using Lyapunov theory.
problem Finite-sample convergence guarantees of asynchronous RL algorithms.
method Reformulate RL algorithms as Markovian SA, develop Lyapunov analysis.
result Mean-square error bounds and convergence for various RL algorithms.
We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, a distributed Frank-Wolfe algorithm which leverages the low-rank structure of its updates to ac…
We consider the problem of approximately reconstructing a partially-observed, approximately low-rank matrix. This problem has received much attention lately, mostly using the trace-norm as a surrogate to the rank. Here we study low-rank matrix reconstruction using both the trace-norm, as well as the less-studied max-no…
Guaranteed bounds for posterior inference in probabilistic programs.
problem Approximating the posterior distribution of probabilistic programs with provable correctness.
method Interval-based trace semantics, soundness and completeness proofs, weight-aware interval type system.
result Guaranteed bounds on the posterior distribution of probabilistic programs are computed and proven to be correct.
Temporal difference learning (TD) is a simple iterative algorithm used to estimate the value function corresponding to a given policy in a Markov decision process. Although TD is one of the most widely used algorithms in reinforcement learning, its theoretical analysis has proved challenging and few guarantees on its s…
Bayesian inference for deep neural networks using trace-class priors and MLMC.
problem Efficient Bayesian inference for deep neural networks.
method Trace-class neural network priors and Multilevel Monte Carlo method.
result Optimal computational complexity for Bayesian inference of TNN models.
Early training phase affects deep neural network optimization and generalization.
problem The choice of learning rate influences generalization in deep learning models.
method Showed that SGD implicitly penalizes the trace of the Fisher Information Matrix (FIM) from the start of training, and explicitly penalizing the trace of FIM improves generalization.
result Catastrophic Fisher explosion (large trace of FIM early in training) is linked to poor generalization.
In a noncommutative torus, effect of perturbation by inner derivation on the associated quantum stochastic process and geometric parameters like volume and scalar curvature have been studied. Cohomological calculations show that the above perturbation produces new spectral triples. Also for the Weyl C^*-algebra, the La…
We propose SPARFA-Trace, a new machine learning-based framework for time-varying learning and content analytics for education applications. We develop a novel message passing-based, blind, approximate Kalman filter for sparse factor analysis (SPARFA), that jointly (i) traces learner concept knowledge over time, (ii) an…
Study multi-task learning with low-rank representation in stochastic linear bandits.
problem Transfer learning across multiple linear bandit tasks with shared low-dimensional representation.
method Proposes a greedy policy with trace norm regularization to implicitly learn a low-rank representation without knowing the rank.
result Upper bound on multi-task regret of , showing benefit over independent task solving.
In this paper, we study a refined L2 version of the semiclassical approximation of projectively invariant elliptic operators with invariant Morse type potentials on covering spaces of compact manifolds. We work on the level of spectral projections (and not just their traces) and obtain an information about classes of t…
Efficiently learns distributions corrupted by both global and local adversarial modifications.
problem Learning distributions with both global and local adversarial corruptions.
method Develops an efficient algorithm to minimize Wasserstein distance with orthogonal projections.
result Achieves optimal risk bounds with error .
It is known that evaluating a certain approximation to the Jones polynomial for the plat closure of a braid is a BQP-complete problem. That is, this problem exactly captures the power of the quantum circuit model. The one clean qubit model is a model of quantum computation in which all but one qubit starts in the maxim…
New method estimates log-determinant using trace powers, avoiding classical limitations.
problem Estimating log-determinant of large matrices efficiently and accurately.
method Interpolating moment-generating function and its derivative at zero using trace powers.
result No continuous estimator using finite moments can be uniformly accurate over unbounded conditioning.
The Schatten quasi-norm was introduced to bridge the gap between the trace norm and rank function. However, existing algorithms are too slow or even impractical for large-scale problems. Motivated by the equivalence relation between the trace norm and its bilinear spectral penalty, we define two tractable Schatten norm…
We describe Venture, an interactive virtual machine for probabilistic programming that aims to be sufficiently expressive, extensible, and efficient for general-purpose use. Like Church, probabilistic models and inference problems in Venture are specified via a Turing-complete, higher-order probabilistic language desce…
It is the main purpose of this paper to introduce a graph-valued stochastic process in order to model the spread of a communicable infectious disease. The major novelty of the SIR model we promote lies in the fact that the social network on which the epidemics is taking place is not specified in advance but evolves thr…
The log-determinant of a kernel matrix appears in a variety of machine learning problems, ranging from determinantal point processes and generalized Markov random fields, through to the training of Gaussian processes. Exact calculation of this term is often intractable when the size of the kernel matrix exceeds a few t…
The scalable calculation of matrix determinants has been a bottleneck to the widespread application of many machine learning methods such as determinantal point processes, Gaussian processes, generalised Markov random fields, graph models and many others. In this work, we estimate log determinants under the framework o…
We demonstrate new applications of the trace embedding lemma to the study of piecewise-linear surfaces and the detection of exotic phenomena in dimension four. We provide infinitely many pairs of homeomorphic 4-manifolds and homotopy equivalent to which have smooth structures distinguished by several for…
DiAMoNDBack models protein backmapping from coarse-grained Cα traces.
problem Restoring all-atom details from coarse-grained protein representations.
method Autoregressive denoising diffusion model for residue-by-residue backmapping.
result Achieves state-of-the-art reconstruction performance in diverse applications.
Matrix completion has been well studied under the uniform sampling model and the trace-norm regularized methods perform well both theoretically and numerically in such a setting. However, the uniform sampling model is unrealistic for a range of applications and the standard trace-norm relaxation can behave very poorly …
High-probability bound for distributed stochastic approximation tracking error.
problem Analyzing the convergence of distributed stochastic approximation schemes.
method Analysis using ODE approach to stochastic approximation.
result High probability bound for tracking error between iterates and limiting differential equation.
Stochastic approximation algorithms show exponential progress bounds.
problem Analyzing the convergence of stochastic approximation algorithms.
method Developed geometric ergodicity proofs to establish exponential concentration bounds.
result Proved faster convergence rates for specific algorithms.
In many environments only a tiny subset of all states yield high reward. In these cases, few of the interactions with the environment provide a relevant learning signal. Hence, we may want to preferentially train on those high-reward states and the probable trajectories leading to them. To this end, we advocate for the…
Deviation inequalities for stochastic approximation methods.
problem Establishing bounds on the deviation of stochastic approximation methods.
method Martingale approximation method for separately Lipschitz functions.
result Established various deviation inequalities for stochastic approximation by averaging and minimization.