A new TwinGP framework for efficient large-scale GP modeling.
problem Efficiently modeling large-scale Gaussian processes with computational constraints.
method Combines global and local approximations using a subset-of-data approach.
result TwinGP framework performs on par or better than state-of-the-art methods at a fraction of the computational cost.
The study provides conditions for approximating Riemannian manifolds with polyhedral metrics.
problem Approximating Riemannian manifolds with polyhedral metrics.
method Conditions on curvature tensors for Lipschitz and local polyhedral approximations.
result Conditions are sufficient for local polyhedral approximations, conjectured to be sufficient for global approximations.
Global solution found for biharmonic wave maps into spheres.
problem Solving biharmonic wave maps into spherical targets.
method Reformulated as a conservation law, solved with Ginzburg-Landau approximation.
result Global weak solution constructed in the energy space.
Global approximation for piecewise linear paths via signatures.
problem Global approximation theorems for piecewise linear paths.
method Using signatures of piecewise linear paths and their density in Lp-norms. result Linear functionals of signatures are dense in Lp-norms under an integrability condition. Two algorithms approximate CMI for feature selection, improving classifier model development.
problem Estimating MI for feature subsets is hard and combinatorial.
method Truncated Power Method (TPower) and Low Rank Bilinear Approximation (LowRank).
result Approximations to NP-hard CMI feature selection are very effective.
Develops a chain rule for ReLU networks and extends approximation theory to global error estimates.
problem Applying standard chain rule to ReLU networks and extending approximation results globally.
method Introduces a derivative for ReLU networks and converts bounded domain results to global estimates.
result Extends neural network approximation theory to include regularity properties for ReLU networks.
Newton's method converges linearly for stable Hessians, even with approximations.
problem Finding global linear convergence for functions without strong convexity or Lipschitz gradients.
method Global linear convergence of Newton's method for stable Hessians, using approximate Hessians and subproblems.
result Global linear convergence rate for a broad class of functions, superior to first-order methods.
Optimizers find approximate global minima in non-convex problems.
problem Understanding why local methods solve non-convex optimization problems.
method Formalizing the hypothesis that many local minima are approximately global minima.
result Most local minima of practical non-convex objectives are approximately global minima.
We study local and global approximations of smooth nets of curvature lines and smooth conjugate nets by respective discrete nets (circular nets and planar quadrilateral nets) with infinitesimal quads. It is shown that choosing the points of discrete nets on the smooth surface one can obtain second-order approximation g…
New quasi-Newton method guarantees global superlinear convergence.
problem Global convergence and superlinear convergence of quasi-Newton methods.
method Hybrid proximal extragradient method with online learning for Hessian approximation.
result First globally convergent quasi-Newton method with explicit superlinear convergence rate.
GRAF uses global partitioning to improve ensemble classifier performance.
problem Improving ensemble classifier performance.
method GRAF extends oblique decision trees to global partitioning.
result GRAF reduces generalization error and improves performance on benchmark datasets.
New method finds global minima using function evaluations and kernel approximations.
problem Finding global minima of smooth functions with limited evaluations.
method Approximates the function using infinite sums of square smooth functions and solves the optimization problem with polynomial time complexity.
result Achieves optimal number of function evaluations with theoretical guarantees and nearly optimal convergence rate.
Gradients help find global optima in complex functions.
problem Finding global optima in functions with many local minima.
method A principle for generating search directions from non-local quadratic approximants based on gradients.
result The proposed algorithm and CMA-ES perform better than random reinitialized BFGS.
Single-timescale actor-critic finds globally optimal policy.
problem Finding globally optimal policy in reinforcement learning.
method Simultaneous actor and critic updates with linear or deep neural network approximations.
result Actor sequence converges to globally optimal policy at O(K−1/2) rate. New method finds global optima in variational inference.
problem Uncertainty in finding global optima in variational inference.
method Deterministic optimization algorithm for variational inference.
result Always converges to globally optimal variational lower bound.
A new method for decomposing non-negative tensors using energy-based modeling.
problem Challenges in traditional tensor decomposition methods, especially global optimization and rank selection.
method Energy-based modeling of tensors, considering interactions between modes for global optimization.
result Demonstrates effectiveness in tensor completion and approximation, revealing a relationship between many-body and low-rank approximations.
Universal approximation for stochastic processes using Brownian motion.
problem Approximating stochastic processes with linear functionals.
method Establishing Lp-type universal approximation theorems for rough path spaces. result Linear functionals on the signature of time-extended Brownian motion can approximate any p-integrable stochastic process. Distributed algorithm finds global solutions for low-rank matrices.
problem Finding global solutions for low-rank matrices in distributed systems.
method Distributed Gradient Descent (DGD+) with LOCAL variables.
result DGD+LOCAL converges to global minimizer with exact consensus.
New algorithms improve likelihood of finding global optima in Bayesian inference.
problem Finding global optima in Bayesian inference is difficult due to nonconvexity.
method Developed two algorithms: consistent Laplace approximation (CLA) and consistent stochastic variational inference (CSVI).
result Both CSVI and CLA improve likelihood of obtaining global optima compared to standard methods.
Functional input neural networks approximate continuous functions on weighted spaces.
problem Approximating continuous functions on infinite-dimensional weighted spaces.
method Additive family mapping, non-linear activation, linear readouts, Stone-Weierstrass theorem.
result Global universal approximation of continuous functions on weighted spaces.
PNNs ensure most local optima are global for 2-layer networks.
problem Non-convex optimization challenges in neural networks.
method Constrain weights to lie over a finite set of lines.
result Most local optima are global in PNN optimizations.
Develops a curvature-corrected tangent space method for manifold-valued data.
problem Generalizing real-valued data approximation to manifold-valued data.
method Systematic approach to developing global-geometry aware, computationally feasible approximation schemes.
result Proposes CC-tHOSVD for low-rank approximation of manifold-valued data.
Paper studies Transformer learning theory for Euclidean and Riemannian domains.
problem Understanding and optimizing Transformer networks for regression tasks.
method Constructive approximation framework using softmax partition of unity and attention mechanism.
result Transformer can achieve uniform ε-approximation error with minimal parameters.
Optimizes piecewise local-linear approximations for global model understanding.
problem Global model behavior interpretation for black-box models.
method Dynamic programming framework for piecewise local-linear approximations with fidelity guarantees.
result Polynomial time algorithm for optimal clustering.
Recently several methods were proposed for sparse optimization which make careful use of second-order information [10, 28, 16, 3] to improve local convergence rates. These methods construct a composite quadratic approximation using Hessian information, optimize this approximation using a first-order method, such as coo…
NormLIME improves feature importance explanations for deep neural networks.
problem Improving local feature explanations for deep learning models.
method NormLIME aggregates local models into global and class-specific interpretations.
result NormLIME outperforms other feature importance metrics in human user studies and numerical experiments.
We propose a strategy for approximating Pareto optimal sets based on the global analysis framework proposed by Smale (Dynamical systems, New York, 1973, pp. 531-544). The method highlights and exploits the underlying manifold structure of the Pareto sets, approximating Pareto optima by means of simplicial complexes. Th…
Global inducing points improve Bayesian neural network performance.
problem Improving Bayesian neural network performance.
method Adapting correlated approximate posterior to all layers in a Bayesian neural network and deep Gaussian processes using learned global inducing points.
result State-of-the-art performance on CIFAR-10 (86.7%) without data augmentation or tempering.
Paper approximates free boundary for optimal investment stopping problems.
problem Optimal investment stopping problems with utility maximization.
method Dual control method to derive asymptotic properties and construct a global closed-form approximation.
result Global closed-form approximation of dual free boundary reduces computational cost.
New nonconvex approach for multiview learning improves efficiency and scalability.
problem Efficiently solve multiview representation learning problems.
method Developed a nonconvex formulation and used SGD for efficient solution.
result Nonconvex approach converges to global optima with simple SGD.
New algorithm trains deep neural networks without global optimization.
problem Training deep neural networks efficiently and without global optimization.
method Uses random complex exponential activation functions and Markov Chain Monte Carlo sampling.
result Consistently attains theoretical approximation rate for residual networks.
Paper proposes a quasi-Newton method for nonlinear equations with global convergence guarantees.
problem Solving smooth and monotone nonlinear equations efficiently and globally.
method Hybrid proximal extragradient framework combined with online learning for Jacobian approximation.
result First global convergence results showing quasi-Newton method's advantage over extragradient method.
The paper tackles learning smooth distance functions using query-based methods.
problem Learning smooth distance functions under query constraints.
method Global and local approaches using Mahalanobis distance functions.
result Quadratic query complexity for both additive and multiplicative approximations.
Paper proves convergence of SA algorithm via martingale and converse Lyapunov methods.
problem Proves convergence of stochastic approximation algorithm.
method Uses martingale and converse Lyapunov methods to prove convergence.
result Provides alternate proof of convergence for SA algorithm.
New perspective on federated learning as posterior inference, improving optimization.
problem Optimizing global models in distributed learning settings.
method Formulated as posterior inference problem, using MCMC for approximate inference and federated averaging for refinement.
result Federated posterior averaging (FedPA) outperforms existing methods on benchmarks.
New algorithms solve feature-sparsity constrained PCA problem.
problem Feature selection and PCA simultaneously for low-rank covariance.
method Two algorithms: Algorithm 1 for low-rank covariance, Algorithm 2 for general covariance.
result Global convergence and approximation guarantees for new algorithms.
We are analysing the convexity and continuity properties of the Mabuchi functional along weak geodesics. The key technical point in our paper is the global approximation of weak geodesics obtained via a well-chosen family of Monge-Ampère equations.
New bounds for SMC show its advantage over MCMC in multimodal distributions.
problem Estimating expectations under multimodal distributions with slow global mixing.
method Proves finite sample complexities for SMC with local mixing times, addressing bias through sequential resampling.
result SMC provides fully polynomial time approximation for multimodal problems.
Boosting Variational Inference improves posterior approximations with adaptive step-sizes.
problem Limited resources hinder the widespread adoption of Boosting Variational Inference.
method Characterized global curvature impact, introduced local curvature, and developed an approximate backtracking algorithm.
result New theoretical convergence rates and experimental validation demonstrate improved performance.
Proposes method for global explanations of credit risk models.
problem Lack of interpretability in credit risk scoring models.
method Sampling decision function to learn interpretable models.
result Unified solution to approximate complex decision boundaries.
Proposes a filtering method for cluster analysis using ℓ0-norm regularization.
problem Improving cluster analysis by filtering data.
method Minimizes a least squares function with a weighted ℓ0-norm penalty, approximated by smooth non-convex functions. result The proposed method can enhance existing clustering techniques.
Neural networks can approximate complex stochastic equations well.
problem Approximating general stochastic differential equations.
method Identified neural network classes approximating continuous functions.
result Neural stochastic differential equations can approximate general stochastic differential equations arbitrarily well.
New method for estimating global and local parameters using regularized Riesz representers.
problem Estimating global and local parameters in complex models robustly.
method Adaptive inference methods based on ℓ1 regularization, including Riesz representer as a nuisance parameter.
result Non-asymptotic and asymptotic uniform validity for honest confidence bands.
We introduce a new graph kernel combining local and global properties.
problem Graph kernels focusing on local properties often fail on large graphs.
method Weisfeiler-Lehman algorithm with stochastic approximation.
result Our kernel outperforms state-of-the-art on graph classification benchmarks.
One-round FL method improves robustness and reduces communication rounds.
problem Making predictions robust and reducing FL communication rounds in heterogeneous data.
method Bayesian predictive space aggregation of client posteriors in one round.
result One-round FL method outperforms other techniques on heterogeneous settings.
We investigate a generalization of the so-called metric splitting of globally hyperbolic space-times to non-smooth Lorentzian manifolds and show the existence of this metric splitting for a class of wave-type space-times. Our approach is based on smooth approximations of non-smooth space-times by families (or sequences…
Sparse optimization refers to an optimization problem involving the zero-norm in objective or constraints. In this paper, nonconvex approximation approaches for sparse optimization have been studied with a unifying point of view in DC (Difference of Convex functions) programming framework. Considering a common DC appro…
New methods improve global optimisation for expensive functions using lookahead strategies.
problem Optimising expensive functions without gradient info in high dimensions.
method Nonmyopic acquisition strategies based on approximate dynamic programming.
result Nonmyopic methods outperform myopic approaches in various applications.