arXiv research

Novel technique reduces Bayesian network complexity while preserving inference accuracy.

We develop a novel method, called PoWER-BERT, for improving the inference time of the popular BERT model, while maintaining the accuracy. It works by: a) exploiting redundancy pertaining to word-vectors (intermediate encoder outputs) and eliminating the redundant vectors. b) determining which word-vectors to eliminate …

We reframe linear dimensionality reduction as a problem of Bayesian inference on matrix manifolds. This natural paradigm extends the Bayesian framework to dimensionality reduction tasks in higher dimensions with simpler models at greater speeds. Here an orthogonal basis is treated as a single point on a manifold and is…

Solla discusses neural processing using statistical physics and Bayesian methods.

In statistical learning, high covariate dimensionality poses challenges for robust prediction and inference. To address this challenge, supervised dimension reduction is often performed, where dependence on the outcome is maximized for a selected covariate subspace with smaller dimensionality. Prevalent dimension reduc…

TERA method speeds up derivative Gaussian processes in high dimensions.

Paper improves tree probability estimation using stochastic optimization and variance reduction.

We consider active maximum a posteriori (MAP) inference problem for Hidden Markov Models (HMM), where, given an initial MAP estimate of the hidden sequence, we select to label certain states in the sequence to improve the estimation accuracy of the remaining states. We develop an analytical approach to this problem for…

U-statistics improve gradient estimation in importance-weighted variational inference.

Improves scalability and efficiency of mixture models in black-box variational inference.

Paper proposes efficient BNN inference flow to reduce computation and memory costs.

Paper interprets UMAP and t-SNE as probabilistic MAP inference.

Paper speeds up large foundation models for time series data.

This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.

ISALT uses inference to simulate SDEs with large time-steps, improving efficiency.

The wavelet Maximum Entropy on the Mean (wMEM) approach to the MEG inverse problem is revisited and extended to infer brain activity from full space-time data. The resulting dimensionality increase is tackled using a collection of techniques , that includes time and space dimension reduction (using respectively wavelet…

Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices. To achieve this goal, we introduce a novel network reparameterization based on the Kronecker-factored eigenbasis (KFE), and then apply Hessian-based structure…

ProbDR framework interprets DR algorithms as probabilistic inference.

Neural network pruning with suitable retraining can yield networks with considerably fewer parameters than the original with comparable degrees of accuracy. Typical pruning methods require large, fully trained networks as a starting point from which they perform a time-intensive iterative pruning and retraining procedu…

A new quantization strategy reduces Transformer model size and inference time.

This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the …

Capturing the dynamical properties of time series concisely as interpretable feature vectors can enable efficient clustering and classification for time-series applications across science and industry. Selecting an appropriate feature-based representation of time series for a given application can be achieved through s…

Binarized Neural Network (BNN) removes bitwidth redundancy in classical CNN by using a single bit (-1/+1) for network parameters and intermediate representations, which has greatly reduced the off-chip data transfer and storage overhead. However, a large amount of computation redundancy still exists in BNN inference. B…

A new algorithm for time series prediction intervals.

New method improves online covariance estimation for SGD.

New insights link diverse statistical problems via secret leakage planted clique.

Bayesian approach speeds up SPN learning and inference.

Survey of factor analysis, PCA, variational inference, and VAE.

Extends dimension reduction to data-driven settings without gradients.

FAQ efficiently evaluates LLMs with statistical guarantees using adaptive query selection.

This paper introduces early exits in neural networks for faster inference.

Improved GPLVM model for single-cell RNA-seq data.

Unified approach combines prediction-powered inference and variance reduction for semi-supervised optimization.

Improves inference-time alignment for diffusion models without updating weights.

Accelerates pulsar light curve inference with learned representations and optimization.

The paper extends Marsden-Weinstein reduction to mechanical presymplectic structures for time-dependent Hamiltonian systems.

A framework uses a mixture of predictors for semi-supervised inference.

A new RNN architecture reduces model size and improves performance.

CSDM integrates compressed sensing into diffusion models for faster data generation.

Paper compares Lagrangian reduction methods for rigid body systems.

PiVoT improves real-time multi-object detection and tracking in clutter.

A plethora of networks is being collected in a growing number of fields, including disease transmission, international relations, social interactions, and others. As data streams continue to grow, the complexity associated with these highly multidimensional connectivity data presents novel challenges. In this paper, we…

A new framework compresses neural networks using sparse optimization.

Bayesian sparsification reduces deep neural network complexity.

CIT and CIF improve feature selection for downstream prediction.

This work optimizes statistical inference with neural networks for high-energy physics data.

Latent dynamics discovery is challenging in extracting complex dynamics from high-dimensional noisy neural data. Many dimensionality reduction methods have been widely adopted to extract low-dimensional, smooth and time-evolving latent trajectories. However, simple state transition structures, linear embedding assumpti…

Wide adoption of complex RNN based models is hindered by their inference performance, cost and memory requirements. To address this issue, we develop AntMan, combining structured sparsity with low-rank decomposition synergistically, to reduce model computation, size and execution time of RNNs while attaining desired ac…