Efficiently infers neural architectures for unseen tasks.
problem High cost of neural architecture search.
method Gradient-based framework sharing information across tasks.
result Quick identification of good candidate architectures for new tasks.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “architecture inference”
DARC learns resource-efficient models by replacing expensive components with cheaper ones.
problem Resource constraints at inference time are more severe than at training time.
method Combines model compression and architecture search to learn resource-efficient models.
result Improves inference speed and memory footprint with minimal accuracy loss.
Conventional Neural Architecture Search (NAS) aims at finding a single architecture that achieves the best performance, which usually optimizes task related learning objectives such as accuracy. However, a single architecture may not be representative enough for the whole dataset with high diversity and variety. Intuit…
The emerging paradigm of Human-Machine Inference Networks (HuMaINs) combines complementary cognitive strengths of humans and machines in an intelligent manner to tackle various inference tasks and achieves higher performance than either humans or machines by themselves. While inference performance optimization techniqu…
New approaches improve uncertainty quantification in autoregressive models for sequence data.
problem Uncertainty quantification in autoregressive models for exchangeable sequences.
method Study of inferential and architectural biases for autoregressive models, focusing on multi-step inference.
result Custom architectures are necessary for multi-step inference to ensure exchangeability.
VINNAS uses variational inference to avoid mode collapse in neural architecture search.
problem Mode collapse in gradient-based NAS methods, leading to suboptimal architectures.
method Differentiable variational inference with variational dropout and automatic relevance determination.
result State-of-the-art accuracy with up to twice fewer non-zero parameters.
This paper examines how neural architectures support amortized Bayesian inference and its performance under varying conditions.
problem Understanding and evaluating amortized inference under signal-to-noise variation and distribution shift.
method Statistical analysis of neural architectures including feedforward networks, Deep Sets, and Transformers.
result Neural architectures support amortized Bayesian inference, offering controlled generalization error and robustness under varying conditions.
Unified framework DDNs for multi-label classification, improving inference efficiency.
problem Efficient inference for multi-label classification with dependency networks.
method Combining dependency networks and deep learning, proposing novel inference schemes.
result Novel inference schemes outperform basic neural architectures and Markov networks.
Efficient algorithm finds fast Transformer models.
problem Slow inference time of Transformer models.
method Decompose Transformer architecture into components, use sampling-based one-shot search.
result Achieved 10% to 30% speedup on pre-trained BERT and 70% on top of a previous state-of-the-art model.
Approximating exponential family models (not single distributions) with a two-network architecturestat.ML
A two-network architecture learns intractable exponential family models.
problem Learning a model itself, not just optimizing parameters of a single distribution.
method Two-network architecture and optimization procedure for exponential family models.
result Accurately learns exponential family models, enabling generic operations.
SOTERIA optimizes neural networks for secure inference with minimal overhead.
problem Protecting user privacy in ML-as-a-service models with low overhead.
method Neural architecture search with dual objectives of accuracy and cryptographic efficiency.
result SOTERIA constructs efficient models for secure inference.
CMPE improves SBI efficiency and accuracy.
problem Efficiently infer parameters of complex simulation models.
method CMPE is a new conditional sampler that distills a continuous probability flow.
result CMPE outperforms current state-of-the-art algorithms on hard low-dimensional benchmarks and competitive on high-dimensional problems.
Optimizes crypto-oriented neural architectures for faster secure inference.
problem Privacy conflicts between model users and providers in neural network applications.
method Proposes a novel Partial Activation layer to optimize the initial design of crypto-oriented neural architectures.
result Significant improvement in the efficiency of secure inference on common evaluation metrics.
Aux-NAS uses auxiliary labels to improve primary task performance without extra inference cost.
problem Improving primary task performance using auxiliary labels without increasing inference cost.
method Architecture-based approach with a flexible asymmetric structure for primary and auxiliary tasks, using Neural Architecture Search (NAS) to evolve networks with only primary-to-auxiliary connections.
result Achieves improved performance on multiple tasks without increasing inference cost.
Study improves posterior inference in neural processes with limited data.
problem Improving posterior predictive inference in probabilistic models with scarce conditioning data.
method Examined effects of pooling operators and variational families on posterior quality in neural processes.
result Novel neural process architectures lead to superior posterior predictive samples in image completion/in-painting tasks.
The Pachinko Allocation Machine (PAM) is a deep topic model that allows representing rich correlation structures among topics by a directed acyclic graph over topics. Because of the flexibility of the model, however, approximate inference is very difficult. Perhaps for this reason, only a small number of potential PAM …
Model-based methods and deep neural networks have both been tremendously successful paradigms in machine learning. In model-based methods, problem domain knowledge can be built into the constraints of the model, typically at the expense of difficulties during inference. In contrast, deterministic deep neural networks a…
DRNets dynamically route instances to efficient transformations.
problem High inference costs due to static model capacity.
method Dynamic Routing Networks (DRNets) with RouterNets for branch selection.
result DRNets reduce inference costs with comparable performance.
We introduce a novel uncertainty estimation for classification tasks for Bayesian convolutional neural networks with variational inference. By normalizing the output of a Softplus function in the final layer, we estimate aleatoric and epistemic uncertainty in a coherent manner. The intractable posterior probability dis…
PSI models and infers feature attributions efficiently and accurately.
problem Modeling and inferring feature attributions in flexible predictive models.
method Probabilistic Shapley inference (PSI) framework using latent random variables and a masking-based neural network architecture.
result PSI learns feature attribution distributions centered at Shapley values, revealing meaningful uncertainty.
Parsimonious Inferencestat.ML
Bayesian inference simplified for machine learning models.
problem Difficulty in specifying general prior belief in machine learning architectures.
method Parsimonious inference using information theory and Kolmogorov complexity.
result Framework quantifies model complexity and prediction information, reducing memorization.
Blog post comparing neural network methods for causal inference.
problem Estimating heterogeneous treatment effects in causal inference.
method Developed and compared a fully connected neural network implementation of Bayesian Causal Forest.
result Improvements in performance in simulation settings.
CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching
problem Inference of tabular foundation models with large training datasets
method CRUMB (Clustered Retrieval Using Minimised-MMD Batching)
result CRUMB outperforms state-of-the-art context selection strategies on the TabArena benchmark
New MIF architecture improves posterior approximations in Bayesian models.
problem Challenges in variational inference for complex hierarchical models.
method Combines VIP and autoregressive flow with prior information and hierarchical ordering.
result Empirically, MIF delivers tighter posterior approximations and state-of-the-art performance.
A novel circuit motif uses sister cells for inference with correlated priors.
problem Structured priors in neural systems pose architectural challenges.
method Proposes a novel circuit motif using sister cells to implement correlated priors without direct interactions.
result Demonstrates the efficacy of correlated priors for inference in noisy environments.
This paper compares amortized methods for Bayesian posterior estimation.
problem Bayesian inference difficulties with iterative routines.
method Amortized in-context Bayesian posterior estimation using transformers and normalizing flows.
result Reverse KL estimator superior for predictive problems.
GenSBI offers JAX-native SBI methods for natural sciences.
problem Lack of native SBI libraries in JAX for natural sciences.
method Flow and diffusion generative models in JAX.
result Near-ideal mean C2ST scores on SBIBM tasks.
We propose a recurrent extension of the Ladder networks whose structure is motivated by the inference required in hierarchical latent variable models. We demonstrate that the recurrent Ladder is able to handle a wide variety of complex learning tasks that benefit from iterative inference and temporal modeling. The arch…
InvGAN combines generative and inference models for photo-realistic image manipulation.
problem GANs lack an inference model for image editing and downstream tasks.
method Train inference and generative models together to adapt and converge.
result InvGAN embeds real images into a high-quality generative model's latent space.
The Inference-Compute Frontier and a Latency-Efficient Architecture for Limit Order Book Predictioncs.LG
A power-law fit to the empirical inference-compute frontier in LOB prediction suggests a scaling-law-style frontier.
problem Limit order book prediction
method Using a suite of models ranging from small decision trees to neural LOB architectures
result A power-law fit to the low- and mid-compute non-MLPLOB frontier extrapolates across multiple orders of magnitude and attains on the excluded high-compute MLPLOB target frontier.
TextNAS finds optimal text representation networks using neural architecture search.
problem Finding the optimal text representation networks is challenging.
method Proposes a novel search space for text representation and uses automatic neural architecture search.
result Automatic search discovers network architectures that outperform state-of-the-art models on text classification and natural language inference tasks.
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
Deep active inference agents learn complex environments using Monte-Carlo methods.
problem Understanding and modeling biological intelligence in complex, continuous state-spaces.
method Neural architecture for deep active inference agents using multiple forms of Monte-Carlo sampling.
result Deep active inference agents can learn environmental dynamics and plan future actions.
Proposes MOPED method for choosing priors in Bayesian DNNs.
problem Challenges in specifying meaningful priors for deep neural networks.
method Two-stage hierarchical modeling with empirical Bayes.
result MOPED enables scalable variational inference and reliable uncertainty quantification.
Distribution Transformers: Fast Approximate Bayesian Inference With On-The-Fly Prior Adaptationstat.ML
Fast Bayesian inference with adaptable priors for real-time applications.
problem Intractable exact posterior computation limits Bayesian inference's adoption.
method Distribution Transformer architecture that learns mappings between priors and posteriors.
result Significant reduction in computation time from minutes to milliseconds.
ABI adapts to graph data for fast, scalable inference.
problem Challenges in inference on graph-structured data.
method Amortized Bayesian Inference (ABI) framework for graph data.
result ABI successfully addresses challenges in graph data inference.
A new method uses Transformers for efficient prediction of marked point processes.
problem Efficiently predicting the next event in a sequence given its history.
method Modeling conditional inter-event times with a mixture of log-normals and marks with a Transformer architecture.
result The method achieves state-of-the-art performance and is faster during inference.
L-HNNs improve Bayesian inference efficiency by reducing gradient computation.
problem Efficient Bayesian inference with minimal gradient computation.
method Integrating L-HNNs into NUTS with online error monitoring.
result L-HNNs in NUTS outperform NUTS in complex posterior densities.
Study compares neural networks for stock selection using fundamental ratios.
problem Predicting stock performance using fundamental financial ratios.
method Comparative study of feed-forward neural network (FNN) and adaptive neural fuzzy inference system (ANFIS).
result Both FNN and ANFIS can separate winners and losers, but FNN performs better.
The recent advances in deep neural networks (DNNs) make them attractive for embedded systems. However, it can take a long time for DNNs to make an inference on resource-constrained computing devices. Model compression techniques can address the computation issue of deep inference on embedded devices. This technique is …
New neural network accelerates cancer research analysis.
problem Slow inference speeds limit cancer research analysis.
method Automatically generated, tailored neural network architecture.
result 16x faster inference on cancer pathology data.
In-network learning outperforms Federated and Split learning in wireless networks.
problem Efficiently using distributed features for inference in wireless networks.
method Proposes 'in-network learning' architecture, uses neural networks for optimization, compares with Federated and Split learning.
result In-network learning offers better accuracy and bandwidth savings.
Unified theory for deep and recurrent networks using Gaussian processes.
problem Understanding capabilities and limitations of different network architectures.
method Unified derivation of mean-field theory from statistical physics of disordered systems.
result Gaussian processes yield identical Gaussian kernels for both architectures at a single time point or layer.
VB-DeepONet uses Bayesian inference to improve DeepONet's predictions and uncertainty quantification.
problem Overfitting and lack of uncertainty quantification in DeepONet.
method Variational Bayes approach to approximate posterior distribution, reducing computational cost.
result VB-DeepONet alleviates DeepONet's limitations and provides uncertainty quantification.
Enhances GNN robustness during inference using Conditional Random Fields.
problem Vulnerability of GNNs to adversarial attacks.
method Post-hoc approach using Conditional Random Fields (CRF).
result Improves robustness of GNNs across various models.
PhysicsNAS generalizes PBL by integrating NAS into neural networks with physical priors.
problem Leveraging physical priors for robust inference in machine learning.
method Physics-based neural architecture search (PhysicsNAS) integrating physical priors and neural networks.
result PhysicsNAS is a top-performer across various physical models and dataset qualities.
MLPerf benchmarks ML inference systems across diverse hardware.
problem Challenges in evaluating ML inference systems due to variety and complexity.
method MLPerf prescribes a set of rules and best practices for comparability.
result First call for submissions yielded over 600 reproducible measurements.
Paper infers human mobility from sparse trajectories.
problem Modeling and inferring human mobility from sparse trajectory data.
method Proposes a single trajectory inference algorithm and a deep learning architecture for multiple trajectories.
result Deep learning model achieves 2x overall accuracy improvement on sparse trajectories.