A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Capsule networks improve performance on image classification tasks with fewer parameters.
Quantum model generates financial data with fewer parameters.
DP-SGD can update fewer coordinates while maintaining privacy.
Improved NLP performance with fewer parameters and less data using conditional multi-task learning.
We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural network (CNN) and a highway network over characters, whose output is given to a long short-term memory (LSTM) recurrent neural network language mo…
Recurrent Neural Networks (RNNs) are powerful sequence modeling tools. However, when dealing with high dimensional inputs, the training of RNNs becomes computational expensive due to the large number of model parameters. This hinders RNNs from solving many important computer vision tasks, such as Action Recognition in …
RDL-Net improves speech enhancement with fewer parameters and better performance.
AUTOMAP is a promising generalized reconstruction approach, however, it is not scalable and hence the practicality is limited. We present dAUTOMAP, a novel way for decomposing the domain transformation of AUTOMAP, making the model scale linearly. We show dAUTOMAP outperforms AUTOMAP with significantly fewer parameters.
Bonsai-Net efficiently discovers state-of-the-art models with fewer parameters.
Neural networks fit fewer samples than their parameters suggest in practice.
Syllabification does not seem to improve word-level RNN language modeling quality when compared to character-based segmentation. However, our best syllable-aware language model, achieving performance comparable to the competitive character-aware model, has 18%-33% fewer parameters and is trained 1.2-2.2 times faster.
Pixel-space diffusion models outperform latent models on high-resolution image synthesis.
Recently, deep neural networks (DNNs) have been regarded as the state-of-the-art classification methods in a wide range of applications, especially in image classification. Despite the success, the huge number of parameters blocks its deployment to situations with light computing resources. Researchers resort to the re…
Study shows DNNs can recover functions with fewer samples than model parameters at overparameterization.
Study proposes Local Linear Encoding for better feature discretization.
New framework uses symmetry-based matrices for efficient, flexible NNs.
3D convolutional neural networks are difficult to train because they are parameter-expensive and data-hungry. To solve these problems we propose a simple technique for learning 3D convolutional kernels efficiently requiring less training data. We achieve this by factorizing the 3D kernel along the temporal dimension, r…
BayesSum improves Bayesian quadrature for discrete domains, requiring fewer samples.
Higher granularity in MoE models boosts expressivity exponentially.
Study improves fractional posterior for 1-bit matrix completion.
SHVC improves image compression with fewer parameters.
Over the past year, the emergence of transfer learning with large-scale language models (LM) has led to dramatic performance improvements across a broad range of natural language understanding tasks. However, the size and memory footprint of these large LMs makes them difficult to deploy in many scenarios (e.g. on mobi…
Estimates mode of discrete distributions with fewer samples.
BaM improves BBVI by optimizing a score-based divergence, leading to faster convergence.
Express Wavenet is an improved optical diffractive neural network. At each layer, it uses wavelet-like pattern to modulate the phase of optical waves. For input image with n2 pixels, express wavenet reduce parameter number from O(n2) to O(n). Only need one percent of the parameters, and the accuracy is still very high.…
Multi-task learning shares information between related tasks, sometimes reducing the number of parameters required. State-of-the-art results across multiple natural language understanding tasks in the GLUE benchmark have previously used transfer from a single large task: unsupervised pre-training with BERT, where a sep…
In this paper, we develop a theory about the relationship between -invariant/equivariant functions and deep neural networks for finite group . Especially, for a given -invariant/equivariant function, we construct its universal approximator by deep neural network whose layers equip -actions and each affine t…
New method finds smaller networks with similar performance to large models in fewer epochs.
Quaternion Conformer GAN (QC-GAN) is a parameter-efficient speech enhancement framework that combines a Quaternion Conformer generator with MetricGAN-based training.
A low-rank tensor model simplifies multi-dimensional Markov chains.
Over the last few years, graph autoencoders (AE) and variational autoencoders (VAE) emerged as powerful node embedding methods, with promising performances on challenging tasks such as link prediction and node clustering. Graph AE, VAE and most of their extensions rely on multi-layer graph convolutional networks (GCN) …
Kernelised flows improve density estimation and generation with fewer parameters.
Fewer data weight updates lead to faster convergence in machine learning models.
We show how the success of deep learning could depend not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can frequently be approximated through "cheap learning" with …
Proposes a neural network for efficient imbalance electricity price forecasting.
Neural network architectures found by sophistic search algorithms achieve strikingly good test performance, surpassing most human-crafted network models by significant margins. Although computationally efficient, their design is often very complex, impairing execution speed. Additionally, finding models outside of the …
We introduce a novel method that enables parameter-efficient transfer and multi-task learning with deep neural networks. The basic approach is to learn a model patch - a small set of parameters - that will specialize to each task, instead of fine-tuning the last layer or the entire network. For instance, we show that l…
Adjoined Networks trains both base and compressed networks together for efficient model compression.
Wavelets help compress neural networks efficiently.
Investigates ways to train larger models with fewer resources, finding that test loss depends only on the actual number of trainable parameters.
Transformers can efficiently approximate nonparametric regression with minimal parameters and sequences.
This paper addresses parameter estimation for wave equations with Markovian switching.
Embedding representations power machine intelligence in many applications, including recommendation systems, but they are space intensive -- potentially occupying hundreds of gigabytes in large-scale settings. To help manage this outsized memory consumption, we explore mixed dimension embeddings, an embedding layer arc…
New method improves image denoising with fewer parameters and less data.
Active learning method reduces labeling cost for regression models with aggregated data.
Probabilistic modeling enables combining domain knowledge with learning from data, thereby supporting learning from fewer training instances than purely data-driven methods. However, learning probabilistic models is difficult and has not achieved the level of performance of methods such as deep neural networks on many …
Neural models for NLP typically use large numbers of parameters to reach state-of-the-art performance, which can lead to excessive memory usage and increased runtime. We present a structure learning method for learning sparse, parameter-efficient NLP models. Our method applies group lasso to rational RNNs (Peng et al.,…
There exist various types of network block models such as the Stochastic Block Model (SBM), the Degree Corrected Block Model (DCBM), and the Popularity Adjusted Block Model (PABM). While this leads to a variety of choices, the block models do not have a nested structure. In addition, there is a substantial jump in the …