arXiv research

Memory split advantage: thinner networks outperform a single wide network.

This work investigates power laws in deep neural network ensembles and predicts their performance.

Memory bandwidth bottleneck is a major challenges in processing machine learning (ML) algorithms. In-memory acceleration has potential to address this problem; however, it needs to address two challenges. First, in-memory accelerator should be general enough to support a large set of different ML algorithms. Second, it…

Backpropagation-free trunk training improves model performance on various benchmarks.

Split conformal prediction works well for time series despite temporal dependence.

PSI-LinUCB improves scalability for large recommender systems.

Neural network tackles continual learning with neuromodulation and local error signals.

Study validates Lillo-Mike-Farmer model predicting financial market long-range correlations.

Recent empirical studies have demonstrated long-memory in the signs of orders to buy or sell in financial markets [2, 19]. We show how this can be caused by delays in market clearing. Under the common practice of order splitting, large orders are broken up into pieces and executed incrementally. If the size of such lar…

Ensembles, where multiple neural networks are trained individually and their predictions are averaged, have been shown to be widely successful for improving both the accuracy and predictive uncertainty of single neural networks. However, an ensemble's cost for both training and testing increases linearly with the numbe…

Delay-SDE-net models time series with memory and uncertainty, outperforming other models.

Deep learning typically requires training a very capable architecture using large datasets. However, many important learning problems demand an ability to draw valid inferences from small size datasets, and such problems pose a particular challenge for deep learning. In this regard, various researches on "meta-learning…

We introduce the C++ application and R package ranger. The software is a fast implementation of random forests for high dimensional data. Ensembles of classification, regression and survival trees are supported. We describe the implementation, provide examples, validate the package with a reference implementation, and …

A new algorithm splits Gaussian processes for efficient streaming data.

A new approach for efficient data compression in split DNN computing.

Divide-and-conquer method splits large data sets for efficient analysis.

Regularizes decision trees to reduce inference time by up to 4x with minimal accuracy loss.

Along with developing of Peaceman-Rachford Splittling Method (PRSM), many batch algorithms based on it have been studied very deeply. But almost no algorithm focused on the performance of stochastic version of PRSM. In this paper, we propose a new stochastic algorithm based on PRSM, prove its convergence rate in ergodi…

Estimating volatility from recent high frequency data, we revisit the question of the smoothness of the volatility process. Our main result is that log-volatility behaves essentially as a fractional Brownian motion with Hurst exponent H of order 0.1, at any reasonable time scale. This leads us to adopt the fractional s…

Study non-oblivious adversarial bandits with delayed feedback and propose algorithms with improved regret bounds.

A new model decomposes equity returns and volatilities into memory components.

Batch-splitting (data-parallelism) is the dominant distributed Deep Neural Network (DNN) training strategy, due to its universal applicability and its amenability to Single-Program-Multiple-Data (SPMD) programming. However, batch-splitting suffers from problems including the inability to train very large models (due to…

This paper studies the inference problem in quantile regression (QR) for a large sample size $n$ but under a limited memory constraint, where the memory can only store a small batch of data of size $m$. A natural method is the naïve divide-and-conquer approach, which splits data into batches of size $m$, computes the l…

This paper proposes an alternative to E2E training for deep networks, reducing memory footprint.

In this paper, we present a novel massively parallel algorithm for accelerating the decision tree building procedure on GPUs (Graphics Processing Units), which is a crucial step in Gradient Boosted Decision Tree (GBDT) and random forests training. Previous GPU based tree building algorithms are based on parallel multi-…

NHC learns scalable algorithmic solutions from diverse tasks.

We develop an online learning method for prediction, which is important in problems with large and/or streaming data sets. We formulate the learning approach using a covariance-fitting methodology, and show that the resulting predictor has desirable computational and distribution-free properties: It is implemented onli…

SpaceNet improves continual learning by intelligently compressing neural connections.

Study evaluates stock price forecasting models during the pandemic.

In big data image/video analytics, we encounter the problem of learning an overcomplete dictionary for sparse representation from a large training dataset, which can not be processed at once because of storage and computational constraints. To tackle the problem of dictionary learning in such scenarios, we propose an a…

Several deep models, esp. the generative, compare the samples from two distributions (e.g. WAE like AutoEncoder models, set-processing deep networks, etc) in their cost functions. Using all these methods one cannot train the model directly taking small size (in extreme -- one element) batches, due to the fact that samp…

SPlit optimizes dataset splitting for better model performance.

The paper extends keenness concept to bridge splittings and finds conditions for existence.

Non-split almost complex supermanifolds and non-split Riemannian supermanifolds are studied. The first obstacle for a splitting is parametrized by group orbits on an infinite dimensional vector space. Further it is shown that non-split structures appear in the first case as deformations of a split reduction and in the …

Study flippable Heegaard splittings in Seifert fibered spaces.

We study the self-dual Yang-Mills equations in split signature. We give a special solution, called the basic split instanton, and describe the ADHM construction in the split signature. Moreover a split version of t'Hooft ansatz is described.

Paper proposes a novel SVM method for creating survival trees.

New methods improve prediction regions for high-dimensional data.

Heretofore, neural networks with external memory are restricted to single memory with lossy representations of memory interactions. A rich representation of relationships between memory pieces urges a high-order and segregated relational memory. In this paper, we propose to separate the storage of individual experience…

In this paper, we propose a dual memory structure for reinforcement learning algorithms with replay memory. The dual memory consists of a main memory that stores various data and a cache memory that manages the data and trains the reinforcement learning agent efficiently. Experimental results show that the dual memory …

New spheres can split a 4D link in ways not possible in 3D.

This paper studies properties of weak reducing pairs in critical Heegaard splittings.

We show that if a split link is obtained from a split link $L$ in $S^3$ by $1/n$-Dehn surgery along a trivial knot $C$, then the link $L\cup C$ is splittable. That is to say, it is impossible to obtain a split link from a split link via a non-trivial twisting. As its corollary, we completely determine when a trivial li…

New methods convert complex link presentations to simpler, recognizable forms.

Stable Hadamard Memory improves reinforcement learning by efficiently managing memory.

The splitting number of a link is the minimal number of crossing changes between different components required, on any diagram, to convert it to a split link. We introduce new techniques to compute the splitting number, involving covering links and Alexander invariants. As an application, we completely determine the sp…

New proof of Lorentzian splitting theorems using elliptic operators.

Little is known on the classification of Heegaard splittings for hyperbolic 3-manifolds. Although Kobayashi gave a complete classification of Heegaard splittings for the exteriors of 2-bridge knots, our knowledge of other classes is extremely limited. In particular, there are very few hyperbolic manifolds that are know…