New algorithm for computing Wasserstein barycenters with guarantees.
problem Computing Wasserstein barycenters with varying regularization strengths.
method Damped Sinkhorn iterations followed by exact maximization/minimization steps.
result First non-asymptotic convergence guarantees for approximating Wasserstein barycenters.
For massive and heterogeneous modern datasets, it is of fundamental interest to provide guarantees on the accuracy of estimation when computational resources are limited. In the application of learning to rank, we provide a hierarchy of rank-breaking mechanisms ordered by the complexity in thus generated sketch of the …
seMCD computes depth functions with statistical guarantees using sequential Monte Carlo.
problem Computing depth functions is computationally challenging, especially in high dimensions.
method Sequential Monte Carlo methodology with theoretical and empirical guarantees.
result The seMCD method provides accurate depth approximations with fewer samples than traditional methods.
DPOT uses deep learning to compute optimal transport efficiently.
problem Computing optimal transport between continuous distributions from unpaired samples.
method DeepParticle methods for min-min optimization without network structure restrictions.
result Established weak convergence and error bounds between learned and optimal maps.
New insights into privacy guarantees for subsampled mechanisms under composition.
problem Tight privacy guarantees for the composition of subsampled differentially private mechanisms.
method Addressed confusion points in privacy accounting for subsampled mechanisms, providing examples and counterexamples.
result Privacy guarantees for subsampled mechanisms differ significantly between Poisson subsampling and sampling without replacement.
RGBM improves GBM efficiency with randomization.
problem Limited theoretical understanding of Gradient Boosting Machine.
method Randomization scheme to reduce search in weak-learners space.
result RGBM leads to substantial computational gains compared to GBM.
Paper extends FFT-based differential privacy method to heterogeneous compositions.
problem Computing accurate differential privacy guarantees for mixed mechanisms.
method Uses Fast Fourier Transform (FFT) for error analysis and parameter selection.
result Provides tighter bounds for heterogeneous compositions compared to homogeneous cases.
FAVANO improves federated learning for resource-constrained environments.
problem Asynchronous communication in federated learning leads to bias and scalability issues.
method FAVANO is a novel asynchronous federated learning framework for resource-constrained environments.
result FAVANO outperforms existing methods on standard benchmarks.
Efficient PI for neural networks without distributional assumptions.
problem No distributional assumptions for efficient predictive inference.
method Differential privacy and linear approximation for leave-one-out models.
result Rigorous coverage guarantees with reduced computation.
New method calibrates probabilistic linear solver for online coverage guarantees.
problem Uncertainty in probabilistic linear solver solutions without coverage guarantees.
method Online conformal prediction-PLS (OCP-PLS) method to calibrate uncertainty thresholds.
result Validates online calibration of uncertainty thresholds via online conformal prediction.
LOT Wassmap speeds up Wasserstein space manifold learning.
problem Finding low-dimensional structures in Wasserstein space datasets.
method Linearized optimal transport and approximation schemes.
result LOT Wassmap provides accurate embeddings with computational efficiency.
SAMPa speeds up SAM by parallelizing its computations.
problem Improving neural network generalization through SAM.
method Parallelizing the two gradient computations in SAM.
result Achieves a twofold speedup of SAM.
This work improves scalability of Wasserstein distances in high dimensions.
problem Scalability issues in computing Wasserstein distances in high dimensions.
method Empirical convergence rates, robustness to data contamination, and computational methods.
result Established fast rates and robust estimation risks for sliced Wasserstein distances.
The paper tackles statistical and computational challenges in learning correlated reward models.
problem The Independence of Irrelevant Alternatives (IIA) assumption collapses human preferences into a universal utility function, leading to coarse approximations.
method The paper investigates the statistical and computational challenges of learning a correlated probit model using best-of-three preference data.
result Best-of-three preference data overcomes the limitations of pairwise preference data, allowing for more fine-grained modeling of human preferences.
Random feature method approximates operators with theoretical guarantees and reduced computation.
problem Approximating operators between infinite dimensional Banach spaces using machine learning.
method Random feature operator learning method with theoretical guarantees and error bounds.
result The random feature method can achieve similar or better test errors than kernel-based methods and neural networks with significantly reduced training times.
The Hidden Markov Model (HMM) is one of the mainstays of statistical modeling of discrete time series, with applications including speech recognition, computational biology, computer vision and econometrics. Estimating an HMM from its observation process is often addressed via the Baum-Welch algorithm, which is known t…
Small neural networks embed arbitrary metric spaces into Gaussian mixtures.
problem Embedding arbitrary metric spaces into a fixed space with low distortion.
method Probabilistic transformers of small depth and width.
result Embeddings with low metric distortion for various metric spaces.
We simplify deep learning convergence analysis using basic math.
problem Ensuring reliable convergence of optimization algorithms in deep networks.
method Elementary arguments and computations to estimate smoothness constants.
result Systematic computation of smoothness constants for first-order algorithms.
New method reduces uncertainty in deep neural networks with minimal computation.
problem Uncertainty in over-parameterized neural networks hinders reliability and statistical guarantees.
method Procedural-noise-correcting (PNC) predictor and resampling methods.
result Asymptotically exact-coverage confidence intervals constructed with minimal computation.
We present safe active incremental feature selection~(SAIF) to scale up the computation of LASSO solutions. SAIF does not require a solution from a heavier penalty parameter as in sequential screening or updating the full model for each iteration as in dynamic screening. Different from these existing screening methods,…
The study provides risk bounds for reservoir computing systems.
problem Analyzing the generalization error of reservoir computing systems.
method Deriving finite sample upper bounds for generalization error using statistical learning theory.
result Explicit bounds on the number of observations needed for estimation accuracy.
CIR method constructs efficient prediction intervals with guaranteed coverage.
problem Efficiently constructing near-minimal prediction intervals with guaranteed coverage.
method Conditional Interquantile Regression (CIR) and CIR+ (enhanced version).
result Optimal balance between predictive accuracy and computational efficiency.
Adaptive learning rates improve FTPL's BOBW guarantees in bandit problems.
problem Improving Follow-the-Perturbed-Leader's BOBW guarantees in bandit problems.
method Introducing surrogate probability functions to compute adaptive learning rates without exact probabilities.
result BOBW guarantees for FTPL with Pareto perturbations for any α>1. Distributed model training is vulnerable to byzantine system failures and adversarial compute nodes, i.e., nodes that use malicious updates to corrupt the global model stored at a parameter server (PS). To guarantee some form of robustness, recent work suggests using variants of the geometric median as an aggregation r…
The paper evaluates Bayesian neural networks for safety in autonomous driving.
problem Safety guarantees for deep neural network controllers in autonomous driving.
method Developed a framework using a state-of-the-art simulator to evaluate Bayesian controllers.
result Bayesian inference methods can provide statistical guarantees for uncertainty computation in autonomous driving.
The pathwise coordinate optimization is one of the most important computational frameworks for high dimensional convex and nonconvex sparse learning problems. It differs from the classical coordinate optimization algorithms in three salient features: {\it warm start initialization}, {\it active set updating}, and {\it …
Paper provides exponential convergence guarantees for Iterative Markovian Fitting.
problem Addressing the Schrödinger Bridge problem in computational optimal transport and generative modeling.
method Develops non-asymptotic exponential convergence guarantees for Iterative Markovian Fitting.
result First non-asymptotic exponential convergence guarantees for IMF under mild structural assumptions.
Deployment of deep neural networks (DNNs) in safety- or security-critical systems requires provable guarantees on their correct behaviour. A common requirement is robustness to adversarial perturbations in a neighbourhood around an input. In this paper we focus on the L0 norm and aim to compute, for a trained DNN an…
Empirical error estimates improve graph sparsification reliability.
problem Uncertainty in sparsification error limits downstream computations reliability.
method Data-driven approach to compute empirical error estimates.
result Empirical error estimates provide theoretical guarantees and are computationally feasible.
BlinkML speeds up ML training by 6x-600x with probabilistic guarantees.
problem Ad-hoc sampling in ML analysis leads to unreliable model quality.
method BlinkML uses probabilistic guarantees for approximate model training.
result BlinkML can speed up ML training by 6.26x-629x with 95% prediction accuracy.
This paper considers stochastic optimization problems for a large class of objective functions, including convex and continuous submodular. Stochastic proximal gradient methods have been widely used to solve such problems; however, their applicability remains limited when the problem dimension is large and the projecti…
Linear mixed models (LMMs) are used extensively to model dependecies of observations in linear regression and are used extensively in many application areas. Parameter estimation for LMMs can be computationally prohibitive on big data. State-of-the-art learning algorithms require computational complexity which depends …
The kernel method is a potential approach to analyzing structured data such as sequences, trees, and graphs; however, unordered trees have not been investigated extensively. Kimura et al. (2011) proposed a kernel function for unordered trees on the basis of their subpaths, which are vertical substructures of trees resp…
Decentralized ranking consensus via gossip for robust and scalable systems.
problem Achieving reliable and resilient consensus on collective rankings in a decentralized setting.
method Random gossip communication for decentralized computation of global rankings.
result Robust and scalable consensus on collective rankings achieved through decentralized, local interactions.
We present a generic compact computational framework relying on structured random matrices that can be applied to speed up several machine learning algorithms with almost no loss of accuracy. The applications include new fast LSH-based algorithms, efficient kernel computations via random feature maps, convex optimizati…
We consider the minimization of non-convex functions that typically arise in machine learning. Specifically, we focus our attention on a variant of trust region methods known as cubic regularization. This approach is particularly attractive because it escapes strict saddle points and it provides stronger convergence gu…
Accelerates signature kernel computation for sequences.
problem Severe computational bottleneck in computing signature kernel.
method Random Fourier features to accelerate signature kernel computation.
result Uniform approximation guarantees for unbiased estimator with linear computation time.
We simplify debiasing of LASSO estimates for practical computation.
problem Bias in LASSO estimates for high-dimensional data.
method Reformulate optimization problem to compute debiasing matrix W directly.
result Efficient closed-form solution for W with unique optimal solution.
Optimizes policies for reinforcement learning with limited data.
problem Computing reliable policies with high confidence in reinforcement learning problems.
method Robust MDPs (RMDPs) with weighted L1 and L∞ norms to minimize ambiguity set spans.
result Optimized ambiguity sets improve policy performance significantly.
We consider the problem of recovering low-rank matrices from random rank-one measurements, which spans numerous applications including covariance sketching, phase retrieval, quantum state tomography, and learning shallow polynomial neural networks, among others. Our approach is to directly estimate the low-rank factor …
Novel algorithm reduces computational burden in IRL with finite-time guarantees.
problem Efficiently recover reward function and optimal policy from expert behavior.
method Single-loop algorithm that maximizes likelihood after each policy improvement step.
result Algorithm provably converges to a stationary solution with finite-time guarantees.
This work improves algorithm design for structured Pfaffian settings.
problem Designing algorithms for specific application domains with theoretical guarantees.
method Data-driven algorithm design using hyperparameter tuning and learning guarantees.
result Introduced the Pfaffian GJ framework for providing learning guarantees for Pfaffian function classes.
MPNN improves on UniFL approximation with provable guarantees.
problem Uniform Facility Location (UniFL) optimization problem.
method Graph Neural Network (MPNN) incorporating approximation-algorithmic principles.
result Empirically outperforms standard approximation algorithms.
We propose a fast method with statistical guarantees for learning an exponential family density model where the natural parameter is in a reproducing kernel Hilbert space, and may be infinite-dimensional. The model is learned by fitting the derivative of the log density, the score, thus avoiding the need to compute a n…
Algorithm constructs prediction sets with PAC guarantees in label shift settings.
problem Reliable uncertainty quantification in the face of distribution shift.
method Estimates predicted probabilities and confusion matrix, then propagates uncertainty through Gaussian elimination to compute confidence intervals and construct prediction sets.
result Satisfies PAC guarantees and produces smaller, more informative prediction sets.
Paper proposes a method to numerically approximate differential privacy guarantees using FFT.
problem Computing tight differential privacy guarantees for machine learning algorithms.
method The method uses numerical approximation of an integral formula, discretization, and the fast Fourier transform algorithm.
result Significant improvements in bound tightness and computation time compared to state-of-the-art techniques.
New geometric methods improve optimization and data science problems.
problem Improving optimization and data science problems.
method Geometric tools for high-dimensional optimization and statistical data science.
result New algorithms and statistical guarantees for optimization and data science.
Signature kernel handles sequential data with theoretical and practical advantages.
problem Handling sequential data efficiently and accurately.
method Positive definite kernel based on stochastic analysis with efficient computation.
result Strong empirical performance and theoretical guarantees.