arXiv research

Deep learning methods are useful for high-dimensional data and are becoming widely used in many areas of software engineering. Deep learners utilizes extensive computational power and can take a long time to train-- making it difficult to widely validate and repeat and improve their results. Further, they are not the b…

Scalable algorithms to solve optimization and regression tasks even approximately, are needed to work with large datasets. In this paper we study efficient techniques from matrix sketching to solve a variety of convex constrained regression problems. We adopt "Iterative Hessian Sketching" (IHS) and show that the fast C…

Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce a new method in this class with a theoretical convergence rate four times faster than existing methods, for sums with sufficie…

Matrix completion is a widely used technique for image inpainting and personalized recommender system, etc. In this work, we focus on accelerating the matrix completion using faster randomized singular value decomposition (rSVD). Firstly, two fast randomized algorithms (rSVD-PI and rSVD- BKI) are proposed for handling …

New method reduces inference variance for faster optimization.

The time complexity of support vector machines (SVMs) prohibits training on huge data sets with millions of data points. Recently, multilevel approaches to train SVMs have been developed to allow for time-efficient training on huge data sets. While regular SVMs perform the entire training in one -- time consuming -- op…

Nesterov's accelerated gradient descent (AGD), an instance of the general family of "momentum methods", provably achieves faster convergence rate than gradient descent (GD) in the convex setting. However, whether these methods are superior to GD in the nonconvex setting remains open. This paper studies a simple variant…

New machine learning model faster, more accurate, and can identify hard-to-classify samples.

Exploiting sparsity enables hardware systems to run neural networks faster and more energy-efficiently. However, most prior sparsity-centric optimization techniques only accelerate the forward pass of neural networks and usually require an even longer training process with iterative pruning and retraining. We observe t…

We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same…

FIRE PBT improves neural network training by focusing on long-term performance.

A faster algorithm for ranking from pairwise comparisons.

Deeper neural networks learn lower frequency functions faster, according to a new principle.

Paper proposes faster method to find local minima in nonconvex optimization.

Faster training of neural ODEs using Gauß-Legendre quadrature.

In this paper, we present the Bennett-type generalization bounds of the learning process for i.i.d. samples, and then show that the generalization bounds have a faster rate of convergence than the traditional results. In particular, we first develop two types of Bennett-type deviation inequality for the i.i.d. learning…

We present a unified framework for low-rank matrix estimation with nonconvex penalties. We first prove that the proposed estimator attains a faster statistical rate than the traditional low-rank matrix estimator with nuclear norm penalty. Moreover, we rigorously show that under a certain condition on the magnitude of t…

Faster Tsetlin Machines use clause indexing to speed inference and learning.

This is our second paper in a series to study gravitational instantons, i.e. complete hyperkäler 4-manifolds with faster than quadratic curvature decay. We prove two main theorems: 1.The asymptotic rate of gravitational instantons to the standard models can be improved automatically. 2.Any ALF-D_k gravitational instant…

learn2mix trains neural nets faster by adjusting class proportions dynamically.

New methods test discrete distributions faster with local privacy constraints.

We propose a novel accelerated exact k-means algorithm, which performs better than the current state-of-the-art low-dimensional algorithm in 18 of 22 experiments, running up to 3 times faster. We also propose a general improvement of existing state-of-the-art accelerated exact k-means algorithms through better estimate…

Paper proposes faster adaptation to distribution shifts in online settings.

A faster ADMM method for nonconvex optimization with improved complexity.

Graphs with stronger curvature grow faster.

Faster algorithms for solving multichain MDPs under average-reward criterion.

Faster policy learning via continuous-time gradients.

We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…

Framework improves gradient estimation for faster training convergence.

Increasing urban concentration raises operational challenges that can benefit from integrated monitoring and decision support. Such complex systems need to leverage the full stack of analytical methods, from state estimation using multi-sensor fusion for situational awareness, to prediction and computation of optimal r…

Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…

TOLD++ improves convergence of diffusion models by critically damping the forward transition matrix.

Many currently deployed Reinforcement Learning agents work in an environment shared with humans, be them co-workers, users or clients. It is desirable that these agents adjust to people's preferences, learn faster thanks to their help, and act safely around them. We argue that most current approaches that learn from hu…

We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which scales linearly in the underlying dimension and the number of training examples. The time complexity of our algorithm to find an approximate local minimum is even faster than that of gradie…

Vectors of data are at the heart of machine learning and data mining. Recently, vector quantization methods have shown great promise in reducing both the time and space costs of operating on vectors. We introduce a vector quantization algorithm that can compress vectors over 12x faster than existing techniques while al…

Debona improves neural network verification by faster and tighter bounds.

Generalised matrix-matrix multiplication forms the kernel of many mathematical algorithms. A faster matrix-matrix multiply immediately benefits these algorithms. In this paper we implement efficient matrix multiplication for large matrices using the floating point Intel Pentium SIMD (Single Instruction Multiple Data) a…

In this paper, we study gravitational instantons (i.e., complete hyperkäler 4-manifolds with faster than quadratic curvature decay). We prove three main theorems: 1.Any gravitational instanton must have known end----ALE, ALF, ALG or ALH. 2.In ALG and ALH-non-splitting cases, it must be biholomorphic to a compact comple…

BPNNs learn to solve combinatorial problems faster and more accurately.

This paper investigates the nonparametric regression problem using SVMs with anisotropic Gaussian RBF kernels. Under the assumption that the target functions are resided in certain anisotropic Besov spaces, we establish the almost optimal learning rates, more precisely, optimal up to some logarithmic factor, presented …

Random permutations can offer faster convergence than with-replacement sampling for some functions.

Motivated by the study of billiards in polygons, we prove fine results for the distribution of gaps of directions of saddle connections on translation surfaces. As an application we prove that for almost every holomorphic differential $ω$ on a Riemann surface of genus $g \geq 2$ the smallest gap between saddle connecti…

New methods boost first-order optimization with faster rates.

Faster algorithm reduces contextual bandit regret with fewer offline regression calls.

Faster algorithms solve convex function learning problems.

We provide improved convergence rates for various \emph{non-smooth} optimization problems via higher-order accelerated methods. In the case of $\ell_\infty$ regression, we achieves an $O(ε^{-4/5})$ iteration complexity, breaking the $O(ε^{-1})$ barrier so far present for previous methods. We arrive at a similar rate fo…

Learning-based hashing algorithms are ``hot topics" because they can greatly increase the scale at which existing methods operate. In this paper, we propose a new learning-based hashing method called ``fast supervised discrete hashing" (FSDH) based on ``supervised discrete hashing" (SDH). Regressing the training exampl…

We show the rank (i.e. minimal size of a generating set) of lattices cannot grow faster than the volume.