New algorithms estimate matrix norms without matrix multiplication.
problem Estimating matrix norms efficiently in a matrix-free setting.
method Randomized algorithms based on Hutchinson's estimator modifications.
result Oracle complexity bounds for two-to-infinity and one-to-two norms.
New model improves option pricing with faster convergence and better generalization.
problem Improving classical option pricing models.
method Introducing a time value related decision function and proving a universal approximation theorem.
result The new decision function approximates on the entire domain of definition by neural networks.
Generative model for condensed matter using Riemannian flow matching.
problem Sampling equilibrium distributions in condensed-phase systems.
method Riemannian flow matching to incorporate periodicity, using Hutchinson's trace estimator and cumulant expansion for bias correction.
result Highly accurate free energy estimates on monatomic ice without multistage estimators.
SCALLOP improves likelihood flow maps for efficient Boltzmann generation.
problem Efficient estimation of model likelihood in flow-based generative models.
method SCALLOP introduces a Hutchinson-free likelihood distillation objective for scalable flow-based models.
result SCALLOP achieves up to 10x inference speedup while improving performance.
Estimates metric tensor on neuromanifolds using Fisher information and random methods.
problem Computing the metric tensor on high-dimensional neuromanifolds efficiently and accurately.
method Deterministic bounds and unbiased random estimators based on Hutchinson's trace method.
result An efficient random estimator with bounded standard deviation.
StAD predicts divergence of diffusion and flow models without Jacobian computation.
problem Computing likelihood from diffusion and flow models is computationally expensive.
method Introduces StAD, a distillation method to predict divergence using Langevin-Stein operator.
result StAD predicts divergence with competitive variance and speed compared to existing methods.
We consider the problem of performing matrix completion with side information on row-by-row and column-by-column similarities. We build upon recent proposals for matrix estimation with smoothness constraints with respect to row and column graphs. We present a novel iterative procedure for directly minimizing an informa…
A promising class of generative models maps points from a simple distribution to a complex distribution through an invertible neural network. Likelihood-based training of these models requires restricting their architectures to allow cheap computation of Jacobian determinants. Alternatively, the Jacobian trace can be u…
Hutch++ optimizes trace estimation for generative models, reducing variance and improving quality.
problem High variance and scalability issues in Hutchinson estimators for generative models.
method Hutch++ is an optimal stochastic trace estimator designed to minimize training variance while maintaining transport optimality.
result Hutch++ leads to higher quality generations and effective variance reduction in various applications.
Hessian alignment improves OOD generalization in deep learning.
problem Improving deep learning models' ability to generalize to out-of-distribution data.
method Analyzed Hessian and gradient alignment for domain generalization using recent OOD theory.
result Hessian alignment methods achieve promising performance on various OOD benchmarks.
HTE improves PINNs for high-dimensional, high-order PDEs by reducing computational cost and memory usage.
problem Challenges in solving high-dimensional, high-order PDEs with PINNs due to computational cost and memory constraints.
method Introduces Hutchinson Trace Estimation (HTE) to transform Hessian matrix calculations into Hessian vector products (HVP), reducing computational cost and memory usage.
result HTE significantly reduces memory consumption and computational cost, enabling faster and more efficient solution of high-dimensional and high-order PDEs.
Study geometric properties of loss functions to understand neural network performance.
problem Understanding the geometric properties of high-dimensional loss functions to improve neural network performance.
method Combine concepts from high-dimensional probability and differential geometry to study curvature properties in lower-dimensional loss representations.
result Mean curvature in the original loss space determines if saddle points appear as minima, maxima, or flat regions.
Develops a PDE approach to constructing nontrivial anisotropic surfaces.
problem Min-max construction of anisotropic surfaces.
method PDE-based approach to anisotropic surface energies.
result Construction of an anisotropic min-max hypersurface.
A new method speeds up sampling of Boltzmann distribution in high-dimensional systems.
problem High computational cost of obtaining Jacobian of flow-based models in high dimensions.
method Flow perturbation method that incorporates stochastic perturbations and reweighting.
result Achieves unbiased sampling of Boltzmann distribution with orders of magnitude speedup.
Paper proposes a method to reduce hallucinations in diffusion models using Laplacian score sharpening.
problem Hallucinations in diffusion models create incoherent or unrealistic samples.
method Post-hoc adjustment to the score function during inference using Laplacian approximation.
result Significantly reduces the rate of hallucinated samples across various data types.
Stochastic trace estimation with tensor train random vectors
problem Stochastic trace estimation for large-scale matrices
method Gaussian random tensor train vectors
result Median-of-means variant achieves dimension-independent guarantees
Counterexample and new proof for curvature varifolds.
problem Counterexample to Hutchinson's proof and new proof of C1,α representation. method Alternative proof method and decomposition of varifolds.
result Structure theorem for curvature varifolds with null second fundamental form.
Study the singular limit of a boundary reaction equation, showing energy concentration and varifold support.
problem Analyzing the singular limit of a boundary reaction equation.
method Investigates the critical points of the boundary reaction equation \((-Δ)^{\frac{1}{2}}u = \frac{1}{\varepsilon}(u-u^3)\) in \(U \subset \mathbb{R}^n\).
result Shows existence of an (n−1)-rectifiable energy concentration set and associates limit energy measures to a stationary varifold. CWGD measures gradient diversity weighted by curvature, improving SGD convergence.
problem Gradient noise in high-curvature directions is underestimated by standard methods.
method CWGD weights gradient diversity by the inverse square root of the Hessian.
result CWGD-Cosine reduces optimization error by up to 20% compared to standard cosine annealing.
An iterated function system Φ consisting of contractive similarity mappings has a unique attractor F⊆Rd which is invariant under the action of the system, as was shown by Hutchinson [Hut]. This paper shows how the action of the function system naturally produces a tiling T of the con…
The combined work of Guaraco, Hutchinson, Tonegawa and Wickramasekera has recently produced a new proof of the classical theorem that any closed Riemannian manifold of dimension n+1≥3 contains a minimal hypersurface with a singular set of Hausdorff dimension at most n−7. This proof avoids the Almgren--Pitts …
ADAHESSIAN optimizes machine learning models with adaptive second-order methods.
problem Efficiently optimizing machine learning models with second-order methods.
method Dynamic Hessian estimation via adaptive estimates, incorporating fast approximations and moving averages.
result ADAHESSIAN achieves state-of-the-art performance across various tasks.
Study boundary behavior of limit interfaces in Riemannian manifolds without convexity assumptions.
problem Boundary behavior of limit interfaces in Riemannian manifolds.
method Proves limit-interface is a free boundary varifold, integer rectifiable up to boundary.
result No convexity assumption required; valid even when limit-interface clusters near boundary.
Proves existence of a single-valued minimal hypersurface in compact manifolds.
problem Existence of multiplicity-1 minimal hypersurfaces in compact Riemannian manifolds.
method Modified minmax construction with Allen-Cahn approximation and valley point optimization.
result Existence of a smooth, closed minimal hypersurface with multiplicity 1 in bumpy metrics.
Compactness results for hypersurfaces with mean curvature prescribed by an ambient function.
problem Proving compactness for hypersurfaces with prescribed mean curvature.
method Using oriented integral varifolds and a weak notion of curvature coefficients.
result Locally uniform bounds on second fundamental form lead to compactness.
The paper studies critical sections of the Allen-Cahn functional and their relation to minimal hypersurfaces.
problem Minimal hypersurfaces with boundary equal to a given submanifold.
method Analysis of the Allen-Cahn functional and its critical sections.
result The limit of critical sections converges to a stationary varifold, which is a minimal hypersurface away from the boundary.
New estimators outperform maximum likelihood without hyper-parameter estimation.
problem Improving system identification performance without hyper-parameter estimation.
method Developed generalized Bayes and closed-form biased estimators using excess MSE.
result New estimators have comparable performance to empirical-Bayes-based regularized estimator.
New estimator reduces kernel mean estimation error.
problem Kernel mean estimation in reproducing kernel Hilbert spaces.
method Corrupt data with known distributions and estimate kernel mean under the corrupted distribution.
result The marginalized kernel mean estimator achieves lower estimation error.
Dual Bayesian Affine Estimators for Wiener-type state-space models
problem Estimating parameters in Wiener-type state-space models
method Fixed-point architecture combining two affine estimators
result Dual basis-parameter estimator achieves comparable parameter MSE to purely affine estimator
Enhances gradient estimates for Hermitian Monge-Ampère equations.
problem Improving estimates for Hermitian Monge-Ampère equations.
method Improves gradient estimates using Evans-Krylov and third derivatives estimates.
result Enhanced estimates for second and third order derivatives.
Paper proposes robust estimators for GANs under Wasserstein contamination.
problem Robust estimation of distributions under contamination.
method Wasserstein GAN-based estimators for location, covariance, and regression.
result Proposed estimators are minimax optimal in many scenarios.
New framework converts offline to online estimation using black-box offline estimators.
problem Convert offline estimation algorithms to online estimation algorithms.
method Oracle-Efficient Online Estimation (OEOE) framework.
result Achieves near-optimal online estimation error via black-box offline estimators.
Proposes variational autoencoder for efficient MMSE estimation.
problem Efficient parameterized MMSE estimation for noisy observations.
method Variational autoencoder models data distribution, approximates MMSE.
result Proposed estimator performs well compared to state-of-the-art.
Paper improves Fisher information estimation methods.
problem Estimating Fisher information for location parameters.
method Revisits and improves Bhattacharya estimator, introduces clipped estimator.
result Clipped estimator shows superior convergence rates in Gaussian noise.
Proposes a robust estimator for RD designs.
problem Estimating treatment effects in RD designs.
method Doubly robust estimator combining two estimators.
result Enhances robustness of treatment effect estimators.
New estimator reduces variance in discrete random variables.
problem Estimating gradients for discrete random variables with reduced variance.
method Sampling without replacement and Rao-Blackwellization.
result Our estimator is the most consistent gradient estimator across different entropy settings.
SCOPE estimator improves covariance and precision matrix estimation.
problem Estimating covariance and precision matrices accurately.
method Distributionally robust optimization with convex spectral divergence.
result SCOPE estimator reduces spectral bias and improves condition number.
We present a multi-task learning approach to jointly estimate the means of multiple independent data sets. The proposed multi-task averaging (MTA) algorithm results in a convex combination of the single-task maximum likelihood estimates. We derive the optimal minimum risk estimator and the minimax estimator, and show t…
Obtaining more accurate equity value estimates is the starting point for stock selection, value-based indexing in a noisy market, and beating benchmark indices through tactical style rotation. Unfortunately, discounted cash flow, method of comparables, and fundamental analysis typically yield discrepant valuation estim…
The maximum mean discrepancy (MMD) is a kernel-based distance between probability distributions useful in many applications (Gretton et al. 2012), bearing a simple estimator with pleasing computational and statistical properties. Being able to efficiently estimate the variance of this estimator is very helpful to vario…
Stochastic volatility modelling of financial processes has become increasingly popular. The proposed models usually contain a stationary volatility process. We will motivate and review several nonparametric methods for estimation of the density of the volatility process. Both models based on discretely sampled continuo…
A new copula estimation method using classification.
problem Estimating copula density from joint and marginal distributions.
method Train a classifier to distinguish joint density from product of marginals.
result Empirically outperforms existing copula estimators.
This paper reviews SDR methods for multivariate response regression.
problem Handling sufficient dimension reduction for multivariate response regression.
method Characterizes SDR estimators as inverse or forward regression methods.
result Pooled marginal, projective resampling, distance-based, ordinary least squares, partial least squares, and semiparametric SDR estimators are discussed.
Density ratio estimation is a vital tool in both machine learning and statistical community. However, due to the unbounded nature of density ratio, the estimation procedure can be vulnerable to corrupted data points, which often pushes the estimated ratio toward infinity. In this paper, we present a robust estimator wh…
TAKDE optimizes kernel density estimation for real-time dynamic processes.
problem Real-time density estimation in applications like computer vision and signal processing.
method Derives asymptotic mean integrated squared error (AMISE) upper bound for 'sliding window' kernel density estimator and proposes TAKDE as a novel, theoretically optimal estimator.
result TAKDE outperforms other dynamic density estimators in terms of test log-likelihood and runtime.
We introduce two new estimators of the bivariate Hurst exponent in the power-law cross-correlations setting -- the cross-periodogram and local X-Whittle estimators -- as generalizations of their univariate counterparts. As the spectrum-based estimators are dependent on a part of the spectrum taken into consideration …
Paper bridges score estimation to parameter and density estimation in DDPMs.
problem Efficiently estimating scores for generative models.
method Introduces a framework linking score estimation to parameter and density estimation.
result Denoising score-matching in DDPMs is asymptotically efficient for parameter estimation.
New method for fast volatility estimation robust to change points.
problem Robust high-frequency volatility estimation with change points.
method ℓ1-regularized power variation estimators using LARS for sparse estimation and dynamic programming for change point refinement.
result Minimax rates achieved for volatility estimators, providing accurate and smooth forecasts.