Novel technique reduces Bayesian network complexity while preserving inference accuracy.
problem Complexity reduction in Bayesian networks for efficient inference.
method Directed convex hull structure and polynomial-time algorithm for identifying minimum localized networks.
result High dimension reduction capability and improved inference efficiency in real networks.
PoWER-BERT speeds up BERT inference by eliminating redundant word-vectors.
problem Improving BERT inference speed without sacrificing accuracy.
method Eliminating redundant word-vectors using a self-attention-based significance measure and learning the number of vectors to eliminate.
result Up to 4.5x reduction in inference time with <1% loss in accuracy on GLUE benchmark.
We reframe linear dimensionality reduction as a problem of Bayesian inference on matrix manifolds. This natural paradigm extends the Bayesian framework to dimensionality reduction tasks in higher dimensions with simpler models at greater speeds. Here an orthogonal basis is treated as a single point on a manifold and is…
Solla discusses neural processing using statistical physics and Bayesian methods.
problem Understanding neural information processing through statistical physics.
method Bayesian inference, Gibbs description, Generalized Linear Models, dimensionality reduction.
result Connection between neural processing and statistical physics.
In statistical learning, high covariate dimensionality poses challenges for robust prediction and inference. To address this challenge, supervised dimension reduction is often performed, where dependence on the outcome is maximized for a selected covariate subspace with smaller dimensionality. Prevalent dimension reduc…
TERA method speeds up derivative Gaussian processes in high dimensions.
problem High-dimensional function evaluations and gradient computations are computationally expensive.
method TERA uses exact gradient reduction to decouple n n n and d d d from the computational cost. result TERA achieves state-of-the-art predictive accuracy with orders of magnitude faster computation.
Paper improves tree probability estimation using stochastic optimization and variance reduction.
problem Improving tree probability estimation in phylogenetic inference.
method Introduces computationally efficient methods for training SBNs and variance reduction for optimization.
result Methods outperform previous baseline methods in tree topology probability estimation and Bayesian phylogenetic inference.
We consider active maximum a posteriori (MAP) inference problem for Hidden Markov Models (HMM), where, given an initial MAP estimate of the hidden sequence, we select to label certain states in the sequence to improve the estimation accuracy of the remaining states. We develop an analytical approach to this problem for…
U-statistics improve gradient estimation in importance-weighted variational inference.
problem High variance in gradient estimation for importance-weighted variational inference.
method Use U-statistics to average base gradient estimators on overlapping batches of size m, achieving lower variance.
result U-statistic variance reduction leads to modest to significant improvements in inference performance.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
Paper proposes efficient BNN inference flow to reduce computation and memory costs.
problem High computation complexity in Bayesian Neural Networks (BNNs) limits deployment in power-constrained systems.
method Feature decomposition and memorization strategy to reduce computations and a memory-friendly computing framework to reduce memory overhead.
result Reduces computation by about half and energy consumption by 73% with 14% area overhead.
Paper interprets UMAP and t-SNE as probabilistic MAP inference.
problem Understanding and interpreting UMAP and t-SNE.
method Interprets UMAP and t-SNE as MAP inference methods corresponding to a probabilistic model of the graph Laplacian.
result Shows UMAP and t-SNE can be understood as probabilistic inference methods.
Paper speeds up large foundation models for time series data.
problem Resource-intensive foundation models limit accessibility.
method Dimensionality reduction techniques, including PCA and neural network adapters.
result Up to 10x speedup and 4.5x more datasets fit on a single GPU.
This work proposes using zero-variance control variates to reduce variance in pathwise gradient estimators for variational inference.
problem Pathwise gradient estimators in variational inference have high variance, leading to inefficient optimization.
method Apply zero-variance control variates to pathwise gradient estimators.
result Zero-variance control variates can significantly reduce the variance of pathwise gradient estimators without requiring complex assumptions.
ISALT uses inference to simulate SDEs with large time-steps, improving efficiency.
problem Efficiently simulating ergodic SDEs with large time-steps.
method Inference-based schemes adaptive to large time-steps (ISALT) from data.
result ISALT achieves significant time reduction and optimal accuracy.
The wavelet Maximum Entropy on the Mean (wMEM) approach to the MEG inverse problem is revisited and extended to infer brain activity from full space-time data. The resulting dimensionality increase is tackled using a collection of techniques , that includes time and space dimension reduction (using respectively wavelet…
Reducing the test time resource requirements of a neural network while preserving test accuracy is crucial for running inference on resource-constrained devices. To achieve this goal, we introduce a novel network reparameterization based on the Kronecker-factored eigenbasis (KFE), and then apply Hessian-based structure…
ProbDR framework interprets DR algorithms as probabilistic inference.
problem Efficiently compressing high-dimensional data into lower dimensions.
method ProbDR variational framework that treats DR as probabilistic inference.
result ProbDR unifies various DR algorithms and enables probabilistic reasoning.
A new quantization strategy reduces Transformer model size and inference time.
problem Heavy computation load and memory overhead in Transformer models for mobile devices.
method Mixed precision quantization with varying bits per word in embedding blocks.
result 11.8x smaller model size and 3.5x speed up for on-device NMT.
Real-time pruning during training reduces network size and training time.
problem Efficiently reducing neural network size without sacrificing accuracy.
method Activation density-based pruning during training.
result Up to 200x reduction in parameters and 60x reduction in inference compute operations.
This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the …
Capturing the dynamical properties of time series concisely as interpretable feature vectors can enable efficient clustering and classification for time-series applications across science and industry. Selecting an appropriate feature-based representation of time series for a given application can be achieved through s…
Binarized Neural Network (BNN) removes bitwidth redundancy in classical CNN by using a single bit (-1/+1) for network parameters and intermediate representations, which has greatly reduced the off-chip data transfer and storage overhead. However, a large amount of computation redundancy still exists in BNN inference. B…
A new algorithm for time series prediction intervals.
problem Non-exchangeability in time series data.
method Adaptive re-estimation of non-conformity scores.
result Significant reduction in interval width compared to existing methods.
New method improves online covariance estimation for SGD.
problem Improving online covariance estimation for SGD.
method Proposes a de-biased covariance estimator that eliminates second-order derivatives.
result Achieves a convergence rate of n ( α − 1 ) / 2 log n n^{(α-1)/2} \sqrt{\log n} n ( α − 1 ) /2 log n , outperforming existing methods. New insights link diverse statistical problems via secret leakage planted clique.
problem Statistical-computational gaps in inference problems.
method Secret leakage planted clique as a new hardness assumption for reductions.
result Establishes tight statistical-computational tradeoffs for various problems.
Bayesian approach speeds up SPN learning and inference.
problem Increased learning-time complexity for large SPNs.
method Bayesian posterior sampling with hyperparameter tuning.
result Significant improvement in learning-time complexity and predictive performance.
Survey of factor analysis, PCA, variational inference, and VAE.
problem Dimensionality reduction and generative modeling of data.
method Variational inference, factor analysis, probabilistic PCA, and VAE.
result Derivation and explanation of ELBO, EM, and closed-form solutions.
Extends dimension reduction to data-driven settings without gradients.
problem Gradient-based dimension reduction limitations in data-driven settings.
method Score ratio matching framework, tailored parameterization, regularization, eigenvalue deflation.
result Outperforms standard score-matching for problems with low-dimensional structure.
FAQ efficiently evaluates LLMs with statistical guarantees using adaptive query selection.
problem Efficiently evaluating many LLMs on a large suite of benchmarks is expensive.
method FAQ uses Bayesian factor models, adaptive sampling, and proactive active inference to select queries.
result FAQ delivers up to 5x effective sample size gains over baselines, matching CI width with fewer queries.
This paper introduces early exits in neural networks for faster inference.
problem Reducing inference time and preventing overfitting in neural networks.
method Designing and training multi-output neural networks with early exits.
result Significant reductions in inference time and improved robustness.
Improved GPLVM model for single-cell RNA-seq data.
problem Lack of effective scalable models for clustering cell types in large-scale single-cell RNA-seq data.
method Introduces amortized stochastic variational Bayesian GPLVM (BGPLVM) tailored for single-cell RNA-seq.
result Matches the performance of scVI on synthetic and real-world datasets and reveals more interpretable latent structures.
Unified approach combines prediction-powered inference and variance reduction for semi-supervised optimization.
problem Scarcity of labeled data in semi-supervised optimization.
method PPI-SVRG, combining PPI and SVRG methods.
result Unified convergence bound with improved performance under label scarcity.
Improves inference-time alignment for diffusion models without updating weights.
problem Aligning diffusion models without updating weights for high-reward outputs.
method Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC) for variance reduction and efficiency.
result Improves primary alignment objectives on text generation tasks.
Bayesian framework reduces high-dimensional GP modeling costs.
problem Challenges in fitting Gaussian processes to high-dimensional inputs.
method Hierarchical Bayesian model with orthonormal projection matrix, incorporating Deep Gaussian Processes.
result Improves predictive performance and uncertainty quantification.
Accelerates pulsar light curve inference with learned representations and optimization.
problem Computational expense of Markov chain Monte Carlo methods for posterior inference.
method Combining U-Net latent representations with local simulator-guided optimization.
result 120x reduction in inference time (24 hours to 12 minutes) with accuracy preserved.
The paper extends Marsden-Weinstein reduction to mechanical presymplectic structures for time-dependent Hamiltonian systems.
problem Limitations of Marsden-Weinstein reduction for cosymplectic structures in time-dependent Hamiltonian systems.
method Developed Marsden-Weinstein reduction for mechanical presymplectic structures.
result Mechanical presymplectic structures provide a more suitable framework for time-dependent Hamiltonian systems than cosymplectic structures.
A framework uses a mixture of predictors for semi-supervised inference.
problem Limited labeled data, abundant unlabeled data.
method Mixture of Experts (MOE) for semi-supervised inference.
result MOE-powered inference framework achieves smallest possible variance.
A new RNN architecture reduces model size and improves performance.
problem Overparameterization and resource limitations in RNNs.
method Jointly encodes weight matrices using tensor-train factorization.
result Reduces model size by several orders of magnitude without sacrificing performance.
CSDM integrates compressed sensing into diffusion models for faster data generation.
problem Efficiently generating synthetic data in high-dimensional spaces.
method Integrating compressed sensing into diffusion models (CSDM) to reduce dimensionality and accelerate inference.
result Achieves provably faster convergence and better latent space dimension selection.
PiVoT improves real-time multi-object detection and tracking in clutter.
problem Challenges in multi-object detection and tracking from noisy point clouds.
method Variational inference for fast, clutter-resilient multi-object tracking.
result Substantial performance improvement over existing Bayesian trackers.
A plethora of networks is being collected in a growing number of fields, including disease transmission, international relations, social interactions, and others. As data streams continue to grow, the complexity associated with these highly multidimensional connectivity data presents novel challenges. In this paper, we…
Paper compares Lagrangian reduction methods for rigid body systems.
problem Modeling and reduction of rigid body systems with rotors.
method Euler-Poincaré reduction by the whole group and reduction by stages.
result Equivalence of equations and conservation laws are tracked.
A new framework compresses neural networks using sparse optimization.
problem Efficiently reducing the size of deep neural networks for practical deployment.
method Sparse optimization for model compression, tailored for stochastic learning.
result Up to 7.2 and 2.9 times FLOPs reduction with comparable accuracy.
Bayesian sparsification reduces deep neural network complexity.
problem Complexity of deep neural networks limits their performance.
method Combines Bayesian shrinkage priors with stochastic variational inference.
result Bayesian model reduction (BMR) is a more efficient alternative for pruning model weights.
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.
Latent dynamics discovery is challenging in extracting complex dynamics from high-dimensional noisy neural data. Many dimensionality reduction methods have been widely adopted to extract low-dimensional, smooth and time-evolving latent trajectories. However, simple state transition structures, linear embedding assumpti…
This work optimizes statistical inference with neural networks for high-energy physics data.
problem Optimal dimensionality reduction with minimal loss of information in the presence of systematic uncertainties.
method Neural network optimization based on binned Poisson likelihoods with nuisance parameters.
result Estimates of parameters of interest close to optimal.