Maximal acceleration metrics limit spacetime curvature.
problem Bounding spacetime curvature under maximal acceleration.
method Developed a geometric framework for maximal acceleration metrics and associated connections, proving curvature bounds.
result Uniform bounds on curvature components follow from uniform bounds on maximal acceleration.
New algorithms solve monotone inclusions and convex-concave minimax problems.
problem Solving maximally monotone equations and inclusions.
method Developed new accelerated algorithms based on Halpern-type fixed-point iteration and Popov's past extra-gradient method.
result Achieved O(1/k) convergence rates for various problems. A fast method for training linear classifiers maximizes margins.
problem Training linear classifiers with maximum margins.
method Momentum-based gradient method derived from convex dual with Nesterov acceleration.
result Exponentially faster convergence rate compared to standard methods.
New accelerators for EM improve convergence speed in complex mixture models.
problem Improving the convergence speed of the EM algorithm for complex mixture models.
method Derive a new operator connecting global descent and local convergence, and use it to develop two acceleration strategies.
result Two new acceleration strategies (G-Accelerator and Geo-Adaptive) significantly improve EM algorithm performance.
Unified framework for accelerated Perceptron and related problems.
problem Finding optimal linear threshold functions for classification.
method Modern acceleration techniques, specifically optimistic online learning.
result Improved convergence rates for various Perceptron-related problems.
Motivated by obtaining a consistent mathematical description for the radiation reaction of point charged particles in linear classical electrodynamics, a theory of generalized higher order tensors and differential forms is introduced. The generalization of some fundamental notions of the differential geometry and the t…
A faster EM algorithm for unsupervised Gaussian mixture models.
problem Efficiently determining the number of components in Gaussian mixture models.
method Adaptive Anderson Acceleration (AA) for EM algorithm, with novel monotonicity control and covariance matrix preservation.
result Significantly faster convergence compared to non-accelerated EM, up to 60X in some cases.
The Hawkes process (HP) has been widely applied to modeling self-exciting events including neuron spikes, earthquakes and tweets. To avoid designing parametric triggering kernel and to be able to quantify the prediction confidence, the non-parametric Bayesian HP has been proposed. However, the inference of such models …
New method accelerates Bayesian imaging using Langevin sampling.
problem Bayesian inference in imaging inverse problems with convex geometry.
method Stochastic relaxed proximal-point iteration targeting posterior distribution.
result Accelerated convergence for κ-strongly log-concave targets. GOCPD detects change points by maximizing the probability of two independent models.
problem Large false discovery rates in online change point detection methods.
method GOCPD uses ternary search to find change points by maximizing the probability of two independent models.
result GOCPD accelerates CPD with logarithmic complexity for single change point detection.
Paper proposes NASAIC framework for co-designing neural architectures and heterogeneous ASICs.
problem Designing efficient neural architectures and ASICs for multiple tasks.
method Build ASIC templates and propose NASAIC framework for simultaneous design of architectures and ASICs.
result NASAIC ensures design specifications and maximizes accuracy with minimal performance loss.
Proposes a curriculum learning algorithm to maximize cumulative return in reinforcement learning.
problem Maximizing cumulative return in reinforcement learning tasks.
method Task sequencing algorithm maximizing cumulative return, using curriculum learning to minimize suboptimal actions.
result Significantly better performance on cumulative return maximization compared to metaheuristic algorithms.
In modern large-scale machine learning applications, the training data are often partitioned and stored on multiple machines. It is customary to employ the "data parallelism" approach, where the aggregated training loss is minimized without moving data across machines. In this paper, we introduce a novel distributed du…
Efficient kernel methods for large datasets using GPU acceleration.
problem Handling large-scale nonparametric learning problems efficiently.
method Preconditioned gradient solver, GPU acceleration, parallelization, out-of-core linear algebra, numerical precision optimization.
result Dramatic speedups on datasets with billions of points, maintaining state-of-the-art performance.
Constrained Markov Decision Process (CMDP) is a natural framework for reinforcement learning tasks with safety constraints, where agents learn a policy that maximizes the long-term reward while satisfying the constraints on the long-term cost. A canonical approach for solving CMDPs is the primal-dual method which updat…
PABO optimizes DNN and hardware hyperparameters for efficient edge device acceleration.
problem Joint optimization of DNN accuracy and hardware cost for edge devices.
method Bayesian optimization with pseudo agent-based approach for memristive crossbar accelerators.
result PABO achieves significant speed-ups and superior performance compared to state-of-the-art methods.
EM algorithm speeds up convergence in federated learning with heterogenous data.
problem Understanding convergence rates of federated learning algorithms under data heterogeneity.
method Characterized convergence rate of EM algorithm for FMLR model under various regimes.
result EM algorithm converges to ground truth with SNR ≥ √K in all regimes.
This work uses a scalable approach to identify partially observed nonlinear systems.
problem Offline identification of partially observed nonlinear systems.
method Certainty-equivalent expectation-maximization (CEEM) as block coordinate-ascent.
result The CEEM approach can identify high-dimensional systems reliably and efficiently.
MONSTOR estimates influence in unseen networks with high accuracy.
problem Estimating and maximizing influence in social networks.
method Inductive machine learning approach replacing Monte Carlo simulations.
result Highly accurate estimates with strong correlation to actual influence.
Using supervised machine learning approaches to recognize human activities from on-body wearable accelerometers generally requires a large amount of labelled data. When ground truth information is not available, too expensive, time consuming or difficult to collect, one has to rely on unsupervised approaches. This pape…
New algorithm finds global maxima in multi-modal functions.
problem Finding global maxima in multi-modal functions with saddle points.
method G-PFSO algorithm using particle filter and averaging.
result G-PFSO efficiently finds global maximizer at optimal rate.
Two spectral clustering methods for multi-layer networks are analyzed and compared.
problem Community detection in multi-layer networks.
method Sum and debiased sum of squared adjacency matrices for spectral clustering.
result Debiased sum of squared adjacency matrices outperforms sum of adjacency matrices.
CAQL tackles continuous action maximization in Q-learning.
problem Maximizing continuous actions in Q-learning.
method Developed CAQL algorithms using plug-and-play optimizers and MIP for optimal max-Q.
result CAQL outperforms policy-based methods in heavily constrained environments.
The problem of human activity recognition is central for understanding and predicting the human behavior, in particular in a prospective of assistive services to humans, such as health monitoring, well being, security, etc. There is therefore a growing need to build accurate models which can take into account the varia…
Proposes qPO, a new acquisition strategy for batched Bayesian optimization that maximizes the probability of including the optimum.
problem Efficiently identifying top-performing compounds from a large chemical library.
method qPO (multipoint Probability of Optimality) acquisition strategy that maximizes the probability of including the true optimum.
result Empirical evidence shows that qPO is competitive with and complements other state-of-the-art methods in batched Bayesian optimization.
New algorithms speed up learning from large screens of proteins.
problem Lack of scaled data hampers biological machine learning.
method Optimized high throughput screens and generative models.
result Maximized information gain with consistent estimates of p(y∣x). Gradient boosting decision trees (GBDTs) have seen widespread adoption in academia, industry and competitive data science due to their state-of-the-art performance in many machine learning tasks. One relative downside to these models is the large number of hyper-parameters that they expose to the end-user. To maximize …
Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
New framework analyzes regret in guided diffusion for optimizing structured inputs.
problem Understanding regret behavior in guided-diffusion black-box optimization for structured design problems.
method Developed a certificate-based expected simple-regret framework that avoids assumptions breaking down in modern diffusion BO pipelines.
result Explains how exponential and polynomial convergence can arise from mass lift in near-optimal designs.
A faster method for optimizing DNA and protein sequences using machine learning.
problem Designing DNA and protein sequences with improved function.
method Activation maximization with a straight-through approximation and adaptive entropy variable.
result Fast SeqProp achieves up to 100-fold faster convergence and improved fitness optima.
Solves imaging inverse problems using a VAE prior and joint MAP optimization.
problem Solving ill-posed inverse problems in imaging.
method Joint Posterior Maximization with a VAE prior, using alternate optimization algorithms and stochastic encoding.
result Converges to high-quality solutions close to bi-convex, outperforming non-convex MAP approaches.
New AD methods improve likelihood estimation for partially observed systems.
problem Estimating likelihood functions for partially observed nonlinear systems.
method Embedding AD particle filter methods in a theoretical framework, developing new algorithms for likelihood maximization.
result Mean squared error significantly lower than existing algorithms.
Enhanced decision-making through Dreamer's anticipatory trajectories and Online Decision Transformer.
problem Efficiently integrating world models with decision transformers.
method Combining Dreamer's trajectory forecasting with Online Decision Transformer's adaptive learning.
result Notable improvements in sample efficiency and reward maximization.
The wide adoption of DNNs has given birth to unrelenting computing requirements, forcing datacenter operators to adopt domain-specific accelerators to train them. These accelerators typically employ densely packed full precision floating-point arithmetic to maximize performance per area. Ongoing research efforts seek t…
Policy gradient algorithms typically combine discounted future rewards with an estimated value function, to compute the direction and magnitude of parameter updates. However, for most Reinforcement Learning tasks, humans can provide additional insight to constrain the policy learning. We introduce a general method to i…
Paper refutes EM convergence theory and introduces a new EM algorithm.
problem The convergence theory of the EM algorithm is incorrect and affects its performance.
method Proposes a new EM algorithm called the Channel Matching (CM) EM algorithm and provides an initialization map.
result The locally maximal Q can affect the convergent speed but not the global convergence.
Accelerates optimization in asynchronous systems with sparse updates.
problem Optimizing finite-sum objectives in asynchronous lock-free environments.
method New accelerated SVRG variant with sparse updates.
result Achieves optimal incremental gradient complexity.
A new method for graph neural networks speeds up inference and training.
problem Challenges in constructing mini-batches for large graphs in graph neural networks.
method Theoretical model of batch construction via maximizing influence score of nodes on outputs.
result Accelerates inference by up to 130x compared to previous methods.
We analyze Riemannian accelerated methods using a new framework.
problem Understanding Riemannian accelerated gradient methods.
method Riemannian A-HPE framework, focusing on Euclidean A-HPE insights and metric distortion control.
result Characterization of acceleration for various Riemannian methods.
Convolutional sparse coding (CSC) can learn representative shift-invariant patterns from multiple kinds of data. However, existing CSC methods can only model noises from Gaussian distribution, which is restrictive and unrealistic. In this paper, we propose a general CSC model capable of dealing with complicated unknown…
Super-acceleration of gradient descent with momentum improves loss function minimization.
problem Minimizing loss functions in machine learning.
method Extending Nesterov acceleration by using gradients at multiple steps ahead.
result Super-acceleration of the momentum algorithm is beneficial for various loss landscapes and tasks.
We consider the problem of estimating the inverse covariance matrix by maximizing the likelihood function with a penalty added to encourage the sparsity of the resulting matrix. We propose a new approach based on the split Bregman method to solve the regularized maximum likelihood estimation problem. We show that our m…
PF-LaCG removes the need for knowing smoothness and strong convexity parameters for locally accelerated CG.
problem Locally accelerated CG requires knowledge of smoothness and strong convexity parameters.
method Parameter-Free Locally Accelerated CG (PF-LaCG) algorithm.
result PF-LaCG achieves local acceleration without requiring knowledge of smoothness and strong convexity parameters.
Accelerates coordinate descent methods for machine learning problems.
problem Slowness of coordinate descent methods in machine learning.
method Extrapolation-based accelerated coordinate descent.
result Significant speed-up in practice compared to existing methods.
Acceleration in Hilbert spaces reduces computations but not accuracy.
problem Improving learning accuracy with fewer computations.
method Analysis of Nesterov acceleration and heavy-ball methods in Hilbert spaces.
result Acceleration can reduce computations but not improve accuracy with respect to gradient descent.
Continuized Nesterov acceleration accelerates stochastic gradient descent and gossip algorithms.
problem Improving the convergence rate of stochastic gradient descent and gossip algorithms.
method Introducing a continuized variant of Nesterov acceleration, which mixes variables continuously and takes gradient steps at random times.
result The continuized Nesterov acceleration achieves convergence rates similar to Nesterov's original acceleration but with random parameters.
Accelerated gradient methods play a central role in optimization, achieving optimal rates in many settings. While many generalizations and extensions of Nesterov's original acceleration method have been proposed, it is not yet clear what is the natural scope of the acceleration concept. In this paper, we study accelera…
Develops accelerated methods for optimization using low-dimensional projected-gradient information.
problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.