arXiv research

GOTabPFN improves tabular model performance with compact tokenization for HDLSS data.

BSTabDiff: Block-Subunit Diffusion Priors for HDLSS Tabular Data Generation

Method finds motifs in knowledge graphs, revealing their structure.

We propose that the Continual Learning desiderata can be achieved through a neuro-inspired architecture, grounded on Mountcastle's cortical column hypothesis. The proposed architecture involves a single module, called Self-Taught Associative Memory (STAM), which models the function of a cortical column. STAMs are repea…

Hierarchical causal models help understand cause and effect in nested data.

We introduce a solvable model of randomly growing systems consisting of many independent subunits. Scaling relations and growth rate distributions in the limit of infinite subunits are analysed theoretically. Various types of scaling properties and distributions reported for growth rates of complex systems in a variety…

Neuro-inspired recurrent neural network algorithms, such as echo state networks, are computationally lightweight and thereby map well onto untethered devices. The baseline echo state network algorithms are shown to be efficient in solving small-scale spatio-temporal problems. However, they underperform for complex task…

We show that the predictability of letters in written English texts depends strongly on their position in the word. The first letters are usually the least easy to predict. This agrees with the intuitive notion that words are well defined subunits in written languages, with much weaker correlations across these units t…

Neuromemristive systems (NMSs) currently represent the most promising platform to achieve energy efficient neuro-inspired computation. However, since the research field is less than a decade old, there are still countless algorithms and design paradigms to be explored within these systems. One particular domain that re…

Deep networks have enabled reinforcement learning to scale to more complex and challenging domains, but these methods typically require large quantities of training data. An alternative is to use sample-efficient episodic control methods: neuro-inspired algorithms which use non-/semi-parametric models that predict valu…

Recently, neuro-inspired episodic control (EC) methods have been developed to overcome the data-inefficiency of standard deep reinforcement learning approaches. Using non-/semi-parametric models to estimate the value function, they learn rapidly, retrieving cached values from similar past states. In realistic scenarios…

Detailed empirical studies of publicly traded business firms have established that the standard deviation of annual sales growth rates decreases with increasing firm sales as a power law, and that the sales growth distribution is non-Gaussian with slowly decaying tails. To explain these empirical facts, a theory is dev…

Study identifies biomarkers for lung cancer in female non-smokers.

Flexible framework compresses models using LC algorithm.

Reduces multiclass and regression compression schemes to binary ones.

Proposes a link between randomness and compression in deep learning.

Paper introduces a new adaptive gradient method with gradient compression for distributed training.

Galen algorithm compresses neural networks for specific hardware with reduced latency.

The (isothermic) compressibility of lattice knots can be examined as a model of the effects of topology and geometry on the compressibility of ring polymers. In this paper, the compressibility of minimal length lattice knots in the simple cubic, face centered cubic and body centered cubic lattices are determined. Our r…

Neural NCD reveals LLMs don't compress well for classification.

EGR refines and assesses protein complex structures.

BDC compresses both sample size and dimensionality of large datasets.

DoCoFL compresses model updates for cross-device federated learning.

Compressing neural nets is an active research problem, given the large size of state-of-the-art nets for tasks such as object recognition, and the computational limits imposed by mobile devices. We give a general formulation of model compression as constrained optimization. This includes many types of compression: quan…

Proposes a method to train neural networks directly on compressed text data.

Study shows LLC correlates with neural network compressibility.

Paper proposes SCALLION and SCAFCOM for compressed FL with reduced communication.

New uncertainty principle limits compression in distributed learning, suggesting optimal methods.

One of the biggest issues in deep learning theory is the generalization ability of networks with huge model size. The classical learning theory suggests that overparameterized models cause overfitting. However, practically used large deep models avoid overfitting, which is not well explained by the classical approaches…

Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size.As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy. This paper introduces a method to compress RNNs for resource constrained e…

Proposes a new linearity-based neural network compression method.

Randomized matrix compression techniques, such as the Johnson-Lindenstrauss transform, have emerged as an effective and practical way for solving large-scale problems efficiently. With a focus on computational efficiency, however, forsaking solutions quality and accuracy becomes the trade-off. In this paper, we investi…

Can textual data be compressed intelligently without losing accuracy in evaluating sentiment? In this study, we propose a novel evolutionary compression algorithm, PARSEC (PARts-of-Speech for sEntiment Compression), which makes use of Parts-of-Speech tags to compress text in a way that sacrifices minimal classification…

Unified framework compresses GANs up to 47x with minimal quality loss.

New compression theory justifies model pruning for neural networks.

SHVC improves image compression with fewer parameters.

Model compression has been introduced to reduce the required hardware resources while maintaining the model accuracy. Lots of techniques for model compression, such as pruning, quantization, and low-rank approximation, have been suggested along with different inference implementation characteristics. Adopting model com…

Compressive learning is a framework where (so far unsupervised) learning tasks use not the entire dataset but a compressed summary (sketch) of it. We propose a compressive learning classification method, and a novel sketch function for images.

COIN++ compresses multiple data types efficiently.

Paper introduces adversarial lossy compression for video artifacts reduction.

LLMs compress financial texts, but distort decision-making.

A new method, REC, compresses images by encoding their latent representations efficiently.

Wavelets help compress neural networks efficiently.

This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.

Auto-Compressing Subset Pruning reduces model size for faster inference.

DP-Net uses dynamic programming for efficient deep neural network compression.

LEAD algorithm speeds up decentralized optimization with compression.

We show that model compression can improve the population risk of a pre-trained model, by studying the tradeoff between the decrease in the generalization error and the increase in the empirical risk with model compression. We first prove that model compression reduces an information-theoretic bound on the generalizati…