New algorithms for optimizing large datasets converge globally from any starting point.
problem Optimizing large-scale machine learning problems efficiently.
method Sub-sampled Newton methods for solving optimization problems.
result Global convergence of sub-sampled Newton methods from any initial point.
New methods for non-convex optimization using inexact Hessian approximations.
problem Optimization of non-convex functions with inexact Hessian information.
method Trust-region and cubic regularization methods with inexact Hessian approximations.
result Iteration complexity to achieve ε-approximate second-order optimality.
New optimization methods reduce computational complexity for large-scale problems.
problem Optimizing large sums of functions with high-dimensional parameters.
method Sub-sampling Hessian and gradient, using random matrix concentration and RandNLA.
result Locally Q-linear and Q-superlinear convergence rates for sub-sampled Newton methods.
A new optimization method for faster convergence.
problem Optimization problems, especially those involving self-concordant functions.
method Newton Sketch: approximate Newton step using randomly projected Hessian.
result Super-linear convergence with exponential probability for self-concordant functions.
A new method uses sub-sampling to make cubic regularization feasible for large-scale learning.
problem Minimizing non-convex functions in machine learning.
method Sub-sampled cubic regularization.
result Global convergence guarantees for non-convex functions.
Novel Newton method for large-scale kernel methods using random features.
problem Efficiently solving large-scale finite-sum minimization problems in RKHS.
method Randomized feature-based Newton method for empirical risk minimization.
result Local superlinear and global linear convergence of the method.
New algorithms reduce Newton method complexity by non-uniformly sampling Hessians.
problem Finding minimizers of convex functions with high-rank Hessians.
method Randomized Newton-type algorithms with non-uniform sub-sampling.
result Non-uniform sampling of Hessians at most O(d log d) terms achieves linear-quadratic convergence.
New insights into learning rates and batch sizes for neural networks using random matrix theory.
problem Understanding how batch size affects learning rates in neural networks.
method Random matrix theory applied to spiked, field-dependent random matrices.
result Analytical expressions for maximal learning rates as a function of batch size.
Boosts change-point detection power with optimal sub-sampling.
problem Power loss in sequential change-point detection from large history data.
method Optimal sub-sampling of history data before kernel-based detection procedures.
result Improved detection performance in extensive experiments.
Optimal sub-sampling method improves statistical model performance.
problem No well-motivated approach for non-uniform sub-sampling in statistical models.
method Asymptotically linear estimator and influence function.
result Optimal influence function based method outperforms previous approaches for linear regression models.
Active covariance estimation using random sub-sampling of variable subsets.
problem Estimating covariance matrices for partially observed random vectors.
method Unbiased covariance estimator under a model of partially observed variables and active learning framework.
result Derivation of error bounds revealing relations between sub-sampling probabilities and covariance matrix entries.
Paper analyzes Nyström regularization for time series forecasting with sequential sub-sampling.
problem Learning rate analysis of Nyström regularization for τ τ τ -mixing time series. method Banach-valued Bernstein inequality and integral operator approach for τ τ τ -mixing sequences. result Almost optimal learning rates for Nyström regularization with sequential sub-sampling.
LOUPE optimizes MRI sub-sampling patterns using machine learning.
problem Optimizing sub-sampling patterns for MRI scans to improve reconstruction accuracy.
method End-to-end learning strategy combining sub-sampling pattern optimization and reconstruction model training.
result LOUPE yields more accurate reconstructions compared to standard under-sampling schemes.
Efficient bandit exploration for various distributions without distribution-specific tuning.
problem Optimizing exploration in multi-armed bandit models for different distributions.
method Sub-sampling Duelling Algorithms (SDA) with Random Block sampling for efficient exploration.
result Achieves asymptotically optimal regret for Bernoulli, Gaussian, and Poisson distributions.
This study addresses the challenges of dynamic mini-batch sub-sampling in neural network training.
problem Challenges in training neural networks due to dynamic mini-batch sub-sampling.
method Distinguishes between static and dynamic sub-sampling, recasting optimization to find SNN-GPPs.
result SNN-GPPs are less susceptible to sub-sampling-induced discontinuities and better approximate true optima.
This paper shows using sub-sample estimates can improve optimization results in large-scale problems.
problem Large-scale optimization problems with uncertain parameters often lead to suboptimal solutions due to mis-specifications or extreme sample characteristics.
method The paper introduces the use of sub-sample estimates to reduce errors in stochastic optimization models, providing theoretical analysis and numerical examples.
result Sub-sample optimization can achieve improved results over full-sample solution estimates in large-scale problems.
The question of how to incorporate curvature information in stochastic approximation methods is challenging. The direct application of classical quasi- Newton updating techniques for deterministic optimization leads to noisy curvature estimates that have harmful effects on the robustness of the iteration. In this paper…
Deep learning reduces data rate for clearer ultrasound images.
problem Reducing data rate for high-quality ultrasound images from sub-sampled RF data.
method Deep learning approach that interpolates missing RF data.
result Effective reduction in data rate without compromising image quality.
New method uses PDMPs with sub-sampling for efficient sampling from posterior distributions.
problem Efficient sampling from posterior distributions with limited data access.
method Approximate simulation of PDMPs with sub-sampling and stochastic gradient estimation.
result Stochastic-gradient PDMPs are efficient and robust compared to Langevin dynamics.
Efficiently estimate risk of large portfolios using MLMC and sub-sampling.
problem Estimating risk of large portfolios with high computational cost.
method Apply Multilevel Monte Carlo (MLMC) with adaptive inner sampling and sub-sampling strategy.
result Sub-sampling strategy reduces computational complexity without portfolio size increase.
Linear reconstruction works for MRI compression without prior signal knowledge.
problem Compressive MRI reconstruction without signal structure knowledge.
method Learn sub-sampling pattern from training data, use linear reconstruction.
result Theoretical and experimental validation of linear reconstruction effectiveness.
New algorithm reduces Newton's method's per-iteration cost while maintaining similar convergence rates.
problem Minimizing a sum of functions over a convex set with large n n n and p p p . method Sub-sampling and low-rank approximation for a randomized batch algorithm.
result Comparable convergence rate to Newton's method with much smaller per-iteration cost.
Paper introduces efficient online sub-sampling for RL with function approximation, reducing policy updates.
problem Efficiently managing computation complexity in RL with general function approximation.
method Online sub-sampling framework that measures information gain and guides exploration.
result Policy updates reduced to ∝ poly log ( K ) \propto\operatorname{poly}\log(K) ∝ poly log ( K ) times for near-optimal regret. The Inverse Bagging Algorithm detects anomalies by identifying sub-samples rich in known data.
problem Detecting anomalies in data sets with a well-modeled process and an unknown PDF.
method Uses inverse bootstrap aggregating to identify sub-samples rich in the known process and classify events.
result The method avoids modifying the kinematic distributions of the well-modeled process.
Improved statistical computation through efficient matrix sampling.
problem Reducing computational cost in large-scale statistical methods.
method Accumulative sub-sampling method to improve statistical efficiency.
result Effective matrix size control improves computational efficiency.
Efficiently approximates statistical leverage scores for faster KRR.
problem Accurately estimating statistical leverage scores for fast KRR.
method Analytic formula for statistical leverage scores, leveraging kernel spectral density.
result Linear time approximation with theoretical guarantees, significantly faster than existing methods.
Study sharpens threshold for matching correlated graphs without labels.
problem Matching latent vertex correspondences in correlated random graphs.
method Analyzes information-theoretic limits for correct vertex matching in sub-sampled graphs.
result Establishes a sharp information-theoretic threshold for vertex matching recovery.
Adapts to high dimensions for estimating conditional moments.
problem Estimation and inference in high-dimensional settings with unknown intrinsic dimension.
method Sub-sampled k k k -NN Z Z Z -estimator, adaptive data-driven sub-sampling. result Estimation error of n − 1 / ( d + 2 ) n^{-1/(d+2)} n − 1/ ( d + 2 ) and asymptotic normality with n 1 / ( d + 2 ) n^{1/(d+2)} n 1/ ( d + 2 ) rate. Improved MCMC for rare events in hidden Markov models.
problem Slow inference and prediction for rare latent states in hidden Markov models.
method Targeted sub-sampling (TASS) over-samples rare latent states, reducing variance in gradient estimation.
result Substantial gains in predictive and inferential accuracy on real and synthetic examples.
Bayesian realized EGARCH models improve tail risk forecasting.
problem Forecasting tail risks in financial markets.
method Developed a Bayesian framework for realized EGARCH models, incorporating multiple realized volatility measures and using robust adaptive Metropolis algorithm for estimation.
result Standardized skewed Student-t distribution and sub-sampled realized range models outperform other models in tail risk forecasting.
Bayesian realized-GARCH models forecast financial tail risks using two-sided Weibull distribution.
problem Forecasting financial tail risks in volatile markets.
method Adaptive Bayesian Markov Chain Monte Carlo for estimation and forecasting, incorporating sub-sampled realized range and variance.
result Realized-GARCH models with two-sided Weibull distribution outperform other models in tail risk forecasting.
Corrects bias in random sampling matrices for improved ML methods.
problem Inversion bias in random sampling matrices hampers ML applications.
method Corrects inversion bias for various random sampling methods.
result Establishes local convergence rates for sub-sampled Newton methods.
StreaMRAK improves KRR for streaming data.
problem Streaming data with memory constraints.
method Divides problem into levels of resolution, sub-sampling.
result Efficiently integrates new samples, reduces memory and complexity.
End-to-end analysis of SGD for STL with adaptive sub-sampling.
problem Designing SGD for STL with statistical guarantees without prior knowledge of source quality.
method Mixed-sample SGD procedure that alternates between source and target data, maintaining transfer guarantees.
result Mixed-sample SGD converges to a target-adaptive solution with 1 / T 1/\sqrt{T} 1/ T rate. This paper extends Newton's method to distributed learning, avoiding saddle points and handling Byzantine workers.
problem Avoiding saddle points in distributed non-convex optimization, especially in the presence of Byzantine workers.
method Extends cubic-regularized Newton method to distributed framework, addressing communication bottlenecks and Byzantine attacks.
result The method achieves improved iteration complexity compared to first-order methods, with a 25% improvement in experiments.
Paper proposes an efficient bandit-based algorithm for hyperparameter optimization.
problem Efficiently evaluating hyperparameters in deep learning models with large search spaces.
method Sub-Sampling (SS) algorithm combined with Bayesian Optimization (BOSS).
result Theoretical proof of optimality and empirical validation of superior performance.
Granger causality reviewed and advanced for complex data.
problem Validity of inferring causal relationships from time series data.
method Recent advances in models for high-dimensional time series, accounting for nonlinear and non-Gaussian observations, and sub-sampled data.
result Improved computational tools for Granger causality.
New criteria detect anomaly detection algorithms without labeled data.
problem Lack of labeled data for evaluating anomaly detection algorithms.
method Developed two new criteria based on Excess-Mass and Mass-Volume curves, and a feature sub-sampling methodology.
result Empirically validated new criteria outperform classical ROC and PR curves in non-labeled data scenarios.
A novel distributed adaptive NN classifier for large data sets.
problem Handling large and distributed data for efficient classification.
method Distributed adaptive nearest neighbor classifier with stochastic tuning parameter selection and early stopping rule.
result Achieves nearly optimal convergence rate under large sub-sample sizes.
The study proves that certain noncompact Hessian manifolds are diffeomorphic to R^n.
problem Characterizing complete noncompact Hessian manifolds with nonnegative Hessian sectional curvature.
method Using a geometric flow on noncompact affine Riemannian manifolds, constructing Hessian metrics, and proving diffeomorphism.
result Complete noncompact Hessian manifolds with nonnegative Hessian sectional curvature are diffeomorphic to R^n if their tangent bundle has maximal volume growth.
New metric for probability measures connects physics and geometry.
problem Developing a new metric for probability measures.
method Transport Hessian metric, formulated dynamical systems.
result Connections to physics equations and mathematical models.
Investigates regularity of solutions to complex Hessian equation.
problem Regularity of solutions to complex Hessian equation.
method Analyzes solutions to Dirichlet problem with specific density condition.
result Establishes conditions for regularity of solutions.
Abstract: Survey on quadratic Hessian equations, their properties, and open problems.
problem Understanding quadratic Hessian equations and their solutions.
method Survey and review of existing research.
result Survey of entire solutions, viscosity solutions, and Hessian estimates.
Paper solves Hessian equations on Kähler manifolds.
problem Solving Hessian equations on Kähler manifolds.
method Combines elementary symmetric functions; provides sufficient and necessary condition.
result Generalizes results for Hessian and Hessian quotient equations.
New Hessian estimates for heat equations on manifolds.
problem Estimating Hessian matrices for heat-type equations on Riemannian manifolds.
method Using Bismut-Stroock Hessian formula, with explicit coefficients and delay/growth rate functions.
result Novel backward weak Harnack inequality and precise pointwise Hessian estimates for eigenfunctions.
Characterizes and describes selfsimilar Hessian manifolds with homothetic vector fields.
problem Understanding the structure and properties of selfsimilar Hessian manifolds.
method Characterization and description of selfsimilar manifolds with homothetic vector fields.
result Any selfsimilar Hessian manifold with a potential homothetic vector field is locally isomorphic to a product of radiant Hessian manifolds.
This paper enhances stability selection by evaluating overall results robustness and identifying optimal regularization values.
problem Improving the robustness and reliability of high-dimensional variable selection.
method Developed a stability estimator to evaluate stability of stability selection results, calibrating key parameters.
result Identified optimal regularization value and improved stability of variable selection.
Study calculates Hessian of Busemann function on Damek-Ricci spaces.
problem Calculating Hessian of Busemann function on Damek-Ricci spaces.
method Calculates eigenvalues of Hessian and proves positive definiteness.
result Hessian of Busemann function is positive definite.