Inference-aware meta-alignment of LLMs reduces computational cost.
problem Aligning LLMs to diverse human preferences is challenging due to conflicting criteria.
method IAMA trains a base model to be aligned to multiple tasks via different inference-time alignment algorithms, using non-linear GRPO for optimization.
result IAMA enables effective alignment of LLMs to multiple criteria with limited computational budget.
RNE provides a flexible framework for diffusion models, enabling inference-time control and energy-based training.
problem Insufficient knowledge of marginal densities in diffusion models.
method Introduces Radon-Nikodym Estimator (RNE) to reveal the connection between marginal densities and transition kernels.
result RNE delivers strong results in inference-time control and energy-based diffusion training.
Tutorial on optimizing diffusion model samples for specific metrics.
problem Optimizing diffusion model samples for specific downstream metrics.
method Review and exploration of inference-time guidance and alignment methods.
result Unified perspective on inference-time algorithms and novel methods.
This work shows how approximate reward models can significantly improve inference-time scaling.
problem Improving the efficiency of inference for large language models.
method Identifying the Bellman error of approximate reward models and using Sequential Monte Carlo (SMC) for inference.
result Approximate reward models can reduce computational complexity from exponential to polynomial in T. New algorithm improves inference-time alignment without reward hacking.
problem Improving quality of responses from language models with limited compute.
method Inference-time alignment, focusing on extttInferenceTimePessimism algorithm. result Optimal performance and scaling-monotonicity of extttInferenceTimePessimism. EHR-MPC optimizes sepsis treatment using digital twins and inference-time control.
problem Optimal sepsis treatment policies are contested and difficult to adapt during inference.
method EHR-MPC decouples learning patient dynamics from treatment optimization, enabling inference-time control over learned digital twins.
result EHR-MPC achieves comparable off-policy performance and improved simulation performance compared to RL baselines.
DTS improves inference-time alignment of diffusion models with less compute.
problem Inference-time alignment of diffusion models suffers from inaccurate value estimation and inefficient reuse of past computations.
method Diffusion Tree Sampling (DTS) uses a tree-based approach to propagate terminal rewards and iteratively refine value estimates.
result DTS produces asymptotically exact samples and matches the FID of best-performing baselines with up to 10x less compute.
URGE improves diffusion model quality without gradients or Hessian.
problem Improving sample quality in diffusion models without gradient evaluations.
method Path-wise importance reweighting via Girsanov change of measure.
result URGE achieves better generation quality than existing methods.
New method aligns diffusion models for inference-time properties without retraining.
problem Aligning pre-trained diffusion models for desired inference-time properties.
method Variationally stable Doob's matching for provable guidance estimation.
result Consistent estimator of guidance with non-asymptotic convergence guarantees.
Improves diffusion model performance and efficiency through classical search.
problem Tackles inference-time control in diffusion models.
method Proposes a framework combining local and global search for efficient navigation.
result Significant gains in performance and efficiency across various domains.
We develop a novel method, called PoWER-BERT, for improving the inference time of the popular BERT model, while maintaining the accuracy. It works by: a) exploiting redundancy pertaining to word-vectors (intermediate encoder outputs) and eliminating the redundant vectors. b) determining which word-vectors to eliminate …
Improves inference-time alignment for diffusion models without updating weights.
problem Aligning diffusion models without updating weights for high-reward outputs.
method Trust-Region Iterative Twisted Sequential Monte Carlo (TRI-TSMC) for variance reduction and efficiency.
result Improves primary alignment objectives on text generation tasks.
Framework disentangles deep feature uncertainty for efficient inference.
problem Inference-time uncertainty estimation for reliable decision-making.
method Uncertainty-Guided Inference-Time Selection framework.
result Significantly tighter prediction intervals and 60% compute reduction.
As neural networks become widely deployed in different applications and on different hardware, it has become increasingly important to optimize inference time and model size along with model accuracy. Most current techniques optimize model size, model accuracy and inference time in different stages, resulting in subopt…
While several approaches to face emotion recognition task are proposed in literature, none of them reports on power consumption nor inference time required to run the system in an embedded environment. Without adequate knowledge about these factors it is not clear whether we are actually able to provide accurate face e…
A new method for math reasoning that allows for iterative correction.
problem Standard reasoning models commit to each token and cannot recover from early errors.
method Generative framework with latent thought vectors for iterative self-correction.
result 30 rethinking iterations surpass baselines with 15 times more parameters.
Optimizes trading policies using future price forecasts.
problem Static reinforcement learning agents lack mechanisms for using price forecasts at inference time.
method FPILOT framework inspired by Model Predictive Control (MPC). Uses a predictive model to construct an allocation-based imagined return objective at each decision step.
result Consistent improvements in total return and risk-adjusted metrics across various policy learning algorithms.
Inference-Time Scaling can be extended to domains prone to systematic failure using intrinsic statistics.
problem Scaling inference time in domains prone to systematic failure
method Intrinsic Selection (iS), Intrinsic Particle Filtering (iPF), and Particle Distillation (dPF)
result Intrinsic Selection improves engineering design selection by 20% and pass@1 by 6.1 points on average.
New method improves grouped convolutions on edge devices.
problem Efficiently implementing grouped convolutions on edge devices.
method Grouped Spatial Pack Convolutions (GSPC) in TVM.
result GSPC outperforms existing implementations by 3.4x, 8x, and 4x on average.
We introduce a novel view to understand how dropout works as an inexplicit ensemble learning method, which doesn't point out how many and which nodes to learn a certain feature. We propose a new training method named internal node bagging, it explicitly forces a group of nodes to learn a certain feature in training tim…
DriftLite improves inference quality of diffusion models without retraining.
problem Adapting pre-trained diffusion models to new target distributions without retraining.
method Lightweight, training-free particle-based approach that steers inference dynamics with optimal stability control.
result Consistently reduces variance and improves sample quality over existing methods.
Paper organizes sampling methods for generative modeling.
problem Challenges in sampling with diffusion models.
method Reviews and organizes existing sampling methods.
result Reveals links between methods to overcome challenges.
MFMs enable efficient reward alignment for generative models.
problem Computational bottleneck in controlling generative models.
method Meta Flow Maps (MFMs) extend consistency models and flow maps to stochastic regime for efficient value function estimation.
result MFMs enable inference-time steering and unbiased, off-policy fine-tuning to general rewards efficiently.
Efficiently infers time-varying sparse MRFs with strong statistical guarantees.
problem Inference of time-varying sparse MRFs with strong statistical guarantees.
method Constrained optimization with exact ℓ0 regularization, near-linear time and memory complexity. result Sharp statistical guarantees for sparsely-changing Gaussian MRFs with as few as one sample per time.
Remasking improves the quality of discrete diffusion models for natural language and image generation.
problem Limited iterative refinement in masked discrete diffusion models.
method Introducing ReMDM sampler that allows remasking during inference.
result Remasking enables better quality outputs with increased sampling steps.
LTMs use latent vectors for efficient autoregressive generation.
problem Efficient autoregressive generation in language models.
method Dual-rate optimization in variational Bayes framework.
result LTMs achieve superior sample and parameter efficiency.
Improves scalability and efficiency of mixture models in black-box variational inference.
problem Scaling mixture models in black-box variational inference leads to high parameter and time costs.
method Introduces MISVAE for amortized mixture parameter space and new ELBO estimators.
result Achieves superior estimation performance with fewer parameters and shorter inference time.
This work makes neural sequence models more efficient by controlling computation.
problem Fixed compute for all examples in neural networks.
method Conditional computation to adapt compute to example complexity.
result Conditional Computation Transformer (CCT) improves efficiency and performance.
Gaussian processes (GP) are attractive building blocks for many probabilistic models. Their drawbacks, however, are the rapidly increasing inference time and memory requirement alongside increasing data. The problem can be alleviated with compactly supported (CS) covariance functions, which produce sparse covariance ma…
Proposes continuous convolution layers for flexible feature map resizing.
problem Fixed stride limitations in discrete convolution layers.
method Introduces Continuous Convolution (CC) layers that use learned continuous functions.
result Dynamic and consistent resizing of feature maps at any scale, non-integer and axis-dependent.
Gaussian processes (GPs) are flexible models that can capture complex structure in large-scale dataset due to their non-parametric nature. However, the usage of GPs in real-world application is limited due to their high computational cost at inference time. In this paper, we introduce a new framework, \textit{kernel di…
This paper compares self-reflection and budget tuning for LLMs, revealing domain-specific performance gains.
problem Improving inference-time performance of LLMs without retraining, balancing quality, cost, and latency.
method Systematic comparison of self-reflection and budget tuning across mathematical reasoning and translation tasks, evaluating various LLMs and model families.
result Substantial domain-dependent variation in self-reflection effectiveness, with up to 220% performance gains in mathematical reasoning.
Best-of-∞ improves LLM performance by efficiently allocating inference-time computation.
problem Achieving optimal performance in test-time LLM ensembling with infinite budget.
method Adaptive generation scheme and weighted ensembles of LLMs, formulated as mixed-integer linear program.
result Optimal ensemble weighting improves performance over individual models.
A new RNN architecture reduces model size and improves performance.
problem Overparameterization and resource limitations in RNNs.
method Jointly encodes weight matrices using tensor-train factorization.
result Reduces model size by several orders of magnitude without sacrificing performance.
ELS framework improves safety alignment by dynamically steering LLMs towards helpful responses.
problem Over-Refusal in Aligned Large Language Models
method Fine-tuning free framework using an Energy-Based Model (EBM) to dynamically steer LLMs during inference.
result Extensive experiments show a significant reduction in false refusals (from 57.3% to 82.6%) while maintaining safety performance.
The paper discusses the limitations of efficiency metrics in machine learning models.
problem Inadequate reporting of efficiency metrics leads to incomplete conclusions.
method Thoroughly discusses common cost indicators, their advantages and disadvantages, and how they contradict each other.
result Incomplete reporting of efficiency metrics can lead to partial conclusions and a blurred picture of model practical considerations.
Deep neural networks are widely used and exhibit excellent performance in many areas. However, they are vulnerable to adversarial attacks that compromise the network at the inference time by applying elaborately designed perturbation to input data. Although several defense methods have been proposed to address specific…
ePF improves PF for ITS by balancing exploration and exploitation, outperforming baselines.
problem Premature exploitation in PF leads to suboptimal solutions under constrained budgets.
method Integrates Entropic Annealing and Look-ahead Modulation to preserve diversity and evaluate potential.
result Significant improvement in task reward (up to 50% relative) on math benchmarks.
Rate-In dynamically adjusts dropout rates during inference to improve uncertainty estimation in neural networks.
problem Static dropout rates lead to suboptimal uncertainty estimates in neural networks.
method Rate-In dynamically adjusts dropout rates using information-theoretic principles.
result Rate-In improves calibration and sharpens uncertainty estimates compared to fixed or heuristic dropout rates.
Unified framework infers time-varying graphs from incomplete signals.
problem Jointly inferring time-varying network topologies and imputing missing data from partial observations.
method Unified non-convex optimization framework with Proximal Alternating Direction Method of Multipliers (PADMM) algorithm.
result Superior robustness in high missing-data regimes, demonstrated through extensive numerical experiments.
In many learning situations, resources at inference time are significantly more constrained than resources at training time. This paper studies a general paradigm, called Differentiable ARchitecture Compression (DARC), that combines model compression and architecture search to learn models that are resource-efficient a…
A new method calibrates scientific models by adding randomness to their predictions.
problem Current scientific foundation models lack calibrated uncertainty.
method Stochastic Attention, which randomizes attention weights using multinomial samples.
result Stochastic Attention achieves the strongest native calibration and sharpest prediction intervals.
A new method combines scores of individual observations to efficiently approximate posterior distributions.
problem Handling posterior distributions conditioned on multiple observations with neural methods.
method Conditional score modeling to combine learned scores from individual observations.
result Sample-efficient method that can aggregate multiple observations at inference time.
MISA detects Trojan triggers in neural networks at inference time.
problem Trojan attacks on neural networks that respond to specific trigger patterns.
method MISA uses misattributions to detect anomalous feature activations.
result MISA achieves 96% AUC in detecting Trojan triggers without assumptions.
This paper introduces early exits in neural networks for faster inference.
problem Reducing inference time and preventing overfitting in neural networks.
method Designing and training multi-output neural networks with early exits.
result Significant reductions in inference time and improved robustness.
ICEE learns new RL tasks in less time with a Transformer model.
problem Efficient in-context policy learning for reinforcement learning.
method In-context Exploration-Exploitation (ICEE) algorithm that optimizes efficiency without explicit Bayesian inference.
result ICEE solves Bayesian optimization problems as efficiently as Gaussian process biased methods but in significantly less time.
SCaSML improves PDE solvers by correcting errors efficiently.
problem Reliable and error-free high-dimensional PDE solutions.
method Defect correction method to derive a Structural-preserving Law of Defect.
result SCaSML achieves faster convergence and reduced errors in high-dimensional PDEs.
Demon aligns diffusion models without retraining or backpropagation.
problem Aligning diffusion models with user preferences.
method Stochastic optimization to control noise distribution.
result Significantly improves aesthetics scores for text-to-image generation.