arXiv research

New method reduces neural image compression run-time by 50%.

New compression technique reduces RNN size by 2-4x without sacrificing accuracy.

Efficiently learns complex Boolean functions under Gaussian distributions.

This paper develops coding techniques to reduce the running time of distributed learning tasks. It characterizes the fundamental tradeoff to compute gradients (and more generally vector summations) in terms of three parameters: computation load, straggler tolerance and communication cost. It further gives an explicit c…

In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running …

The paper predicts run times for Gaussian chemistry code.

We describe two techniques that significantly improve the running time of several standard machine-learning algorithms when data is sparse. The first technique is an algorithm that effeciently extracts one-way and two-way counts--either real or expected-- from discrete data. Extracting such counts is a fundamental step…

We introduce dropout compaction, a novel method for training feed-forward neural networks which realizes the performance gains of training a large model with dropout regularization, yet extracts a compact neural network for run-time efficiency. In the proposed method, we introduce a sparsity-inducing prior on the per u…

We study the fundamental problem of learning the parameters of a high-dimensional Gaussian in the presence of noise -- where an $\varepsilon$-fraction of our samples were chosen by an adversary. We give robust estimators that achieve estimation error $O(\varepsilon)$ in the total variation distance, which is optimal up…

Optimal transport relaxations improve Wasserstein GAN training efficiency.

While matrix factorisation models are ubiquitous in large scale recommendation and search, real time application of such models requires inner product computations over an intractably large set of item factors. In this manuscript we present a novel framework that uses the inverted index representation to exploit struct…

DASNet improves neural network efficiency by dynamically pruning activations.

Efficiently estimates binary product distributions with privacy.

New framework reduces private mean estimation error with optimal efficiency.

This article introduces the concepts around Online Bandit Linear Optimization and explores an efficient setup called SCRiBLe (Self-Concordant Regularization in Bandit Learning) created by Abernethy et. al.\cite{abernethy}. The SCRiBLe setup and algorithm yield a $O(\sqrt{T})$ regret bound and polynomial run time comple…

Improved SVRG for quadratic functions achieves better performance and running times.

We propose a novel and flexible rank-breaking-then-composite-marginal-likelihood (RBCML) framework for learning random utility models (RUMs), which include the Plackett-Luce model. We characterize conditions for the objective function of RBCML to be strictly log-concave by proving that strict log-concavity is preserved…

The rising popularity of intelligent mobile devices and the daunting computational cost of deep learning-based models call for efficient and accurate on-device inference schemes. We propose a quantization scheme that allows inference to be carried out using integer-only arithmetic, which can be implemented more efficie…

This work optimizes mean estimation under varying user privacy demands.

In this paper, a modification to the training process of the popular SPLICE algorithm has been proposed for noise robust speech recognition. The modification is based on feature correlations, and enables this stereo-based algorithm to improve the performance in all noise conditions, especially in unseen cases. Further,…

AITuning uses reinforcement learning to optimize communication libraries.

We study $k$-GenEV, the problem of finding the top $k$ generalized eigenvectors, and $k$-CCA, the problem of finding the top $k$ vectors in canonical-correlation analysis. We propose algorithms $\mathtt{LazyEV}$ and $\mathtt{LazyCCA}$ to solve the two problems with running times linearly dependent on the input size and…

We introduce the simulation tool SABCEMM (Simulator for Agent-Based Computational Economic Market Models) for agent-based computational economic market (ABCEM) models. Our simulation tool is implemented in C++ and we can easily run ABCEM models with several million agents. The object-oriented software design enables th…

Efficient algorithms improve learning of large-margin halfspaces.

Variable selection is one of the most important tasks in statistics and machine learning. To incorporate more prior information about the regression coefficients, the constrained Lasso model has been proposed in the literature. In this paper, we present an inexact augmented Lagrangian method to solve the Lasso problem …

We prove that stochastic gradient descent efficiently converges to the global optimizer of the maximum likelihood objective of an unknown linear time-invariant dynamical system from a sequence of noisy observations generated by the system. Even though the objective function is non-convex, we provide polynomial running …

We develop an efficient algorithm for low-rank approximation with improved approximation guarantees.

We address the problem of learning in an online setting where the learner repeatedly observes features, selects among a set of actions, and receives reward for the action taken. We provide the first efficient algorithm with an optimal regret. Our algorithm uses a cost sensitive classification learner as an oracle and h…

The paper proposes using function approximations to reduce the computational burden in measuring counterparty credit exposure.

In dictionary selection, several atoms are selected from finite candidates that successfully approximate given data points in the sparse representation. We propose a novel efficient greedy algorithm for dictionary selection. Not only does our algorithm work much faster than the known methods, but it can also handle mor…

Semantic segmentation remains a computationally intensive algorithm for embedded deployment even with the rapid growth of computation power. Thus efficient network design is a critical aspect especially for applications like automated driving which requires real-time performance. Recently, there has been a lot of resea…

New algorithm estimates Gaussian means and covariances efficiently and privately.

Algorithm efficiently learns deep ReLU networks with polynomial runtime in depth and parameters.

Non-linear kernel methods can be approximated by fast linear ones using suitable explicit feature maps allowing their application to large scale problems. We investigate how convolution kernels for structured data are composed from base kernels and construct corresponding feature maps. On this basis we propose exact an…

APMSqueeze improves Adam for faster training with less communication.

Pareto Testing optimizes model performance under multiple constraints.

ATSDLN adapts to time series data for anomaly detection.

New algorithms improve agnostic learning for triangles and polygons, reducing time complexity.

Belief propagation and its variants are popular methods for approximate inference, but their running time and even their convergence depend greatly on the schedule used to send the messages. Recently, dynamic update schedules have been shown to converge much faster on hard networks than static schedules, namely the res…

Long short-term memory (LSTM) has been widely used for sequential data modeling. Researchers have increased LSTM depth by stacking LSTM cells to improve performance. This incurs model redundancy, increases run-time delay, and makes the LSTMs more prone to overfitting. To address these problems, we propose a hidden-laye…

Unified approach for robust and heavy-tailed mean estimation in high dimensions.

Efficient method for learning continuous exponential families beyond Gaussian.

Randomly chosen support makes sparse linear regression easy.

New algorithm speeds up polynomial kernel approximations.

Improved K-Means++ and K-Means$\|$ with faster run-time.

Efficient EKF-based algorithm improves LSTM adaptive learning accuracy.

The colored Jones polynomial is a knot invariant that plays a central role in low dimensional topology. We give a simple and an efficient algorithm to compute the colored Jones polynomial of any knot. Our algorithm utilizes the walks along a braid model of the colored Jones polynomial that was refined by Armond from th…

The kernel method is a potential approach to analyzing structured data such as sequences, trees, and graphs; however, unordered trees have not been investigated extensively. Kimura et al. (2011) proposed a kernel function for unordered trees on the basis of their subpaths, which are vertical substructures of trees resp…