This work compares regularization and constrained inference for label constraints in machine learning.
problem Improving model performance with label constraints in machine learning.
method Comparison of regularization and constrained inference strategies.
result Constrained inference reduces population risk by correcting model violations, while regularization narrows the generalization gap but introduces bias.
New method for online inference of constrained optimization problems.
problem Online inference of constrained stochastic optimization problems.
method Random scaling of Sketched Stochastic Sequential Quadratic Programming (SSQP).
result Asymptotically valid confidence intervals and matrix-free computation.
We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply …
New algorithms for sampling in constrained domains without learning rates.
problem Sampling in constrained domains with fairness constraints and post-selection inference.
method Coin betting ideas from convex optimisation and a unifying framework for constrained sampling.
result Our algorithms achieve competitive performance without hyperparameter tuning.
Cookbook transforms constrained statistical inference into unconstrained problems.
problem Transforming constrained statistical inference into unconstrained problems.
method Bijective and diffeomorphisms parametrizations.
result Maintains statistical inference properties like identifiability.
Energy savings for DNN inference on resource-constrained devices.
problem Energy efficiency in deep learning inference for constrained devices.
method Efficiently searches through equivalent DNN graphs to find the one with the least execution cost.
result Achieves 24% energy savings with minimal performance impact.
Method reformulates constrained optimization as latent space inference.
problem Optimizing black-box functions with hard constraints.
method Posterior inference in latent space using flow-based models and diffusion models.
result Method achieves superior performance across various tasks.
The paper tackles MAP inference over non-convex constraints in safety-critical settings.
problem Efficiently computing MAP predictions subject to non-convex constraints is challenging.
method The paper investigates conditions for exact and efficient MAP inference over continuous variables and devises scalable algorithms for both tractable and general cases.
result The proposed methods outperform constraint-agnostic baselines and scale to complex densities.
New method for constrained sampling using gradient flows.
problem Sampling from constrained domains.
method Introducing a boundary condition for gradient flow to confine particles within the domain.
result Provable continuous-time convergence in total variation for constrained sampling.
Develops an online method for solving constrained optimization problems with debiasing techniques.
problem Online inference of solutions to constrained optimization problems with equality and inequality constraints.
method Stochastic Sequential Quadratic Programming (SSQP) with momentum debiasing.
result Achieves global almost-sure convergence and local asymptotic normality with optimal primal-dual limiting covariance.
Controller-Augmented Hidden Markov Models (CHMMs) are a framework for constrained sequential inference.
problem Hidden Markov models fail under pathwise constraints like precedence, visitation, or monotonic state progression.
method CHMMs compile constraints into finite-state controllers, then use standard forward-backward and Viterbi recursions to compute exact constrained posteriors and paths.
result CHMMs provide exact constrained inference, monotone ascent in constrained EM, and linear complexity in controller cardinality.
New samplers minimize KL divergence for constrained and non-Euclidean geometries.
problem Efficient sampling from constrained and non-Euclidean distributions.
method Stein Variational Mirror Descent and Mirrored Stein Variational Gradient Descent.
result New samplers converge more rapidly and accurately than prior methods.
This paper optimizes AI inference on edge devices with reduced communication and computation costs.
problem Efficiently performing AI inference on resource-constrained edge devices with reduced communication and computation costs.
method A three-step framework for effective inference: model split point selection, communication-aware model compression, and task-oriented encoding of intermediate features.
result Our proposed framework achieves a better trade-off and significantly reduces inference latency compared to baseline methods.
New method solves constrained stochastic optimization problems efficiently.
problem Online statistical inference of constrained stochastic nonlinear optimization problems.
method Stochastic Sequential Quadratic Programming (StoSQP) with iterative sketching solver.
result The rescaled primal-dual sequence converges to a mean-zero Gaussian distribution.
Paper studies PSGD for constrained optimization problems and its statistical properties.
problem Online inference for constrained optimization problems.
method Stochastic gradient descent with projection (PSGD) for constrained optimization.
result Limiting distribution of PSGD-based estimates under linear-equality constraints.
New method uses constrained transport metric for robust Bayesian inference.
problem Flexible Bayesian models with many uninterpretable parameters.
method Exponentially tilted empirical likelihood with a novel Wasserstein metric, combined with a prior.
result Superior performance compared to state-of-the-art robust Bayesian inference methods.
Chance-constrained ActInf allows for small violations of constraints to drive goal-directed behavior.
problem Goal-directed behavior constrained by prior beliefs.
method Introducing chance constraints to ActInf, allowing for small violations of constraints.
result Chance-constrained ActInf allows for a trade-off between robust control and chance constraint violation.
Loopy and generalized belief propagation are popular algorithms for approximate inference in Markov random fields and Bayesian networks. Fixed points of these algorithms correspond to extrema of the Bethe and Kikuchi free energy. However, belief propagation does not always converge, which explains the need for approach…
C-Learner improves stability of plug-in estimators for causal inference.
problem Limited overlap between treatment and control groups leads to unstable estimates.
method Constrained learning framework that achieves stability and asymptotic properties.
result Constrained learning produces stable estimates with desirable asymptotic properties.
MAGI-X learns unknown dynamics from data without numerical integration.
problem Difficult to propose ODEs in closed-form for complex systems.
method MAGI-X uses neural networks within a manifold-constrained Gaussian process framework.
result MAGI-X achieves competitive accuracy in fitting and forecasting with reduced computational time.
A new pricing controller handles resource constraints to infer target prices effectively.
problem Resource constraints prevent fixed-price inference, leading to support exclusion.
method Formalizes support-exclusion failure, designs a target-aware controller, and uses a realized information clock.
result The controller can certify feasible target bands and log continuous local densities, leading to polynomial rates of inference.
New methods for parameter estimation in mechanistic models using data-consistent inversion.
problem Parameter estimation bias in Bayesian analysis for mechanistic models.
method Data-consistent inversion methods based on rejection sampling, MCMC, GANs, and constrained optimization.
result Improved parameter estimation without bias from uninformative priors.
Self-distillation improves constrained language generation by aligning models with target distributions.
problem Sparse and uninformative reward signals in constrained generation settings.
method Iteratively refining the base model through self-distillation, incorporating learned twist functions and proposals.
result Substantial gains in generation quality through improved model alignment with target distributions.
New algorithm samples constrained distributions efficiently.
problem Sampling from distributions with statistical constraints.
method Primal-dual Langevin Monte Carlo (PD-LMC) using gradient descent-ascent dynamics.
result PD-LMC algorithm successfully samples constrained distributions.
Efficiently infers time-varying sparse MRFs with strong statistical guarantees.
problem Inference of time-varying sparse MRFs with strong statistical guarantees.
method Constrained optimization with exact ℓ0 regularization, near-linear time and memory complexity. result Sharp statistical guarantees for sparsely-changing Gaussian MRFs with as few as one sample per time.
Proposes a Gaussian process model for constrained dynamics learning.
problem Challenges in identifying constrained dynamics of mechanical systems.
method Combines analytical mechanics with Gaussian process regression.
result Improves data efficiency and constraint integrity in predictions.
This paper introduces a new approach to active inference using constrained Bethe Free Energy.
problem Tackling the limitations of existing epistemic behavior models in active inference.
method Introducing a constrained Bethe Free Energy (CBFE) perspective to optimize epistemic behavior in generative models.
result CBFE optimization leads to more robust and flexible epistemic behavior compared to existing methods.
TASO optimizes CNN models for memory-constrained devices.
problem Large CNN models are too resource-intensive for mobile devices.
method Integer Linear Programming (ILP) for selecting operations and workspace allocation.
result 8x speedup and 2.2x reduction in memory usage compared to greedy algorithms.
Develops probabilistic models for gene regulatory network inference.
problem Challenges in reconstructing gene regulatory networks from genome-wide data.
method Two complementary frameworks: PMF-GRN and GLM-Prior.
result Probabilistic inference refines regulatory estimates with quantified uncertainty.
cCBO optimizes interventions in causal graphs under constraints.
problem Finding optimal interventions in causal graphs with constraints.
method Exploits graph structure, uses Gaussian processes, and sequentially selects interventions.
result Successful trade-off between fast convergence and feasibility of interventions.
Paper uses RL to solve constrained combinatorial optimization problems.
problem Constrained combinatorial optimization problems.
method Extending NCO theory to CMDPs, using RL with penalty signals.
result Proposes a superior method for solving constrained problems.
Predictive models can be used for causal inference with feature selection.
problem Limitations of predictive models in interpreting causal relationships.
method Constrained learning process by selecting features according to Pearl's backdoor adjustment criterion.
result Causal models provide near unbiased effect estimates and better generalization.
Locally learned synaptic failure enables complete Bayesian inference.
problem Bayesian inference in neural networks.
method Biologically constrained neural network with synaptic failure and lateral inhibition.
result Synaptic failure allows sampling from both epistemic and aleatoric distributions.
Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained d…
The paper studies continuous submodular functions and their optimization.
problem Maximizing continuous submodular functions in poly. time.
method Characterization of continuous submodularity, operations preserving it, and algorithms for constrained maximization.
result Continuous submodularity is equivalent to a weak DR property, leading to continuous DR-submodular functions with the full DR property.
Noninvasive reconstruction of cardiac transmembrane potential (TMP) from surface electrocardiograms (ECG) involves an ill-posed inverse problem. Model-constrained regularization is powerful for incorporating rich physiological knowledge about spatiotemporal TMP dynamics. These models are controlled by high-dimensional …
Dual-space sampling tackles ill-conditioned inverse problems with Bayesian methods.
problem Bayesian inference in constrained inverse problems with ill-conditioned solutions.
method Dual-space posterior sampling using ADMM and SVGD.
result Well-calibrated uncertainty estimates and posterior contraction with increasing data.
We present an improved Bayesian framework for performing inference of affine transformations of constrained functions. We focus on quadrature with nonnegative functions, a common task in Bayesian inference. We consider constraints on the range of the function of interest, such as nonnegativity or boundedness. Although …
New method uses neural networks to infer dark matter subhalo abundance from stellar streams.
problem Constrain warm dark matter mass using stellar streams.
method Amortized Approximate Likelihood Ratios (AALR) for likelihood-free Bayesian inference.
result Demonstrates effectiveness of new method for estimating dark matter subhalo abundance.
Parameter inference in ordinary differential equations is an important problem in many applied sciences and in engineering, especially in a data-scarce setting. In this work, we introduce a novel generative modeling approach based on constrained Gaussian processes and leverage it to build a computationally and data eff…
The paper tackles sampling from Gibbs measures with constrained support, providing a sampling guarantee.
problem Sampling from Gibbs measures with constrained support, especially in the pre-asymptotic regime.
method Analyzing the spectral gap of Langevin dynamics to provide a non-asymptotic sampling guarantee.
result The low-temperature Gibbs distribution concentrates on a neighborhood of its mode in the pre-asymptotic regime.
We propose a novel hierarchical model for multitask bipartite ranking. The proposed approach combines a matrix-variate Gaussian process with a generative model for task-wise bipartite ranking. In addition, we employ a novel trace constrained variational inference approach to impose low rank structure on the posterior m…
Bayesian neural network (BNN) priors are defined in parameter space, making it hard to encode prior knowledge expressed in function space. We formulate a prior that incorporates functional constraints about what the output can or cannot be in regions of the input space. Output-Constrained BNNs (OC-BNN) represent an int…
For successful deployment of deep neural networks on highly--resource-constrained devices (hearing aids, earbuds, wearables), we must simplify the types of operations and the memory/power resources used during inference. Completely avoiding inference-time floating-point operations is one of the simplest ways to design …
Optimal neural network approximation for Wasserstein gradient direction via convex optimization.
problem Approximating Wasserstein gradient direction with limited data.
method Two-layer networks with squared-ReLU activations, SDP relaxation.
result Optimal approximation of Wasserstein gradient direction in two-layer networks.
Bayesian coresets improve scalable Bayesian inference.
problem Efficiently approximating posterior inference with a subset of data.
method Sparsity constrained optimization and accelerated optimization methods.
result Explicit convergence rate guarantees and superior performance compared to state-of-the-art.
CANs improve GANs by enforcing structured constraints during training.
problem Generating valid structured objects like molecules and game maps from examples alone.
method Constrained Adversarial Networks (CANs) embed constraints into the model during training, penalizing invalid structures.
result CANs efficiently generate high-quality and novel valid structures.
Unified framework for constrained diffusion models on nonconvex sets with efficient landing mechanism.
problem Efficiently modeling generative models under nonconvex constraints.
method Unified framework with overdamped and underdamped dynamics, landing mechanism.
result Significantly reduces computational cost while maintaining sample quality.