In-BO optimizes complex constrained domains using SIn-GP surrogate models.
problem Optimizing in complex constrained domains with irregular shapes.
method Sparse Intrinsic Gaussian Processes (SIn-GP) on manifolds with heat kernel estimation.
result In-BO outperforms traditional BO in complex constrained domains.
We propose a class of intrinsic Gaussian processes (in-GPs) for interpolation, regression and classification on manifolds with a primary focus on complex constrained domains or irregular shaped spaces arising as subsets or submanifolds of R, R2, R3 and beyond. For example, in-GPs can accommodate spatial domains arising…
New algorithms for sampling in constrained domains without learning rates.
problem Sampling in constrained domains with fairness constraints and post-selection inference.
method Coin betting ideas from convex optimisation and a unifying framework for constrained sampling.
result Our algorithms achieve competitive performance without hyperparameter tuning.
The area of constrained clustering has been extensively explored by researchers and used by practitioners. Constrained clustering formulations exist for popular algorithms such as k-means, mixture models, and spectral clustering but have several limitations. A fundamental strength of deep learning is its flexibility, a…
Deep neural networks with more parameters and FLOPs have higher capacity and generalize better to diverse domains. But to be deployed on edge devices, the model's complexity has to be constrained due to limited compute resource. In this work, we propose a method to improve the model capacity without increasing inferenc…
Machine-learning models for security-critical applications such as bot, malware, or spam detection, operate in constrained discrete domains. These applications would benefit from having provable guarantees against adversarial examples. The existing literature on provable adversarial robustness of models, however, exclu…
New model generates data on constrained sets without losing tractability.
problem Generating data on constrained sets without losing tractability.
method Mirror Diffusion Models (MDM) learn diffusion processes in a dual space constructed from a mirror map.
result MDM generates data on convex constrained sets without losing tractability.
New method for constrained sampling using gradient flows.
problem Sampling from constrained domains.
method Introducing a boundary condition for gradient flow to confine particles within the domain.
result Provable continuous-time convergence in total variation for constrained sampling.
Deep learning struggles with out-of-distribution data, so this paper tackles domain generalization.
problem Deep learning models fail with out-of-distribution data.
method Formulates domain generalization as a constrained statistical learning problem, then uses nonconvex duality theory to develop an algorithm with convergence guarantees.
result Improves domain generalization by up to 30 percentage points on various benchmarks.
Extends Langevin dynamics for constrained domains.
problem Optimization of constrained probability measures.
method Mirror mean-field Langevin dynamics (MMFLD).
result Linear convergence guarantees and propagation of chaos results.
Proposes r2SGLD for efficient constrained exploration in non-convex learning.
problem Stagnation in high-temperature chains of reSGLD in distribution tails.
method r2SGLD: replica exchange with reflection steps in a bounded domain.
result Reflection steps enhance mixing rates with quadratic improvement in domain diameter.
The paper tackles gradual domain adaptation with manifold-constrained DRO, showing error bounds across distributions.
problem Gradual domain adaptation challenge with manifold-constrained data distributions.
method Distributionally Robust Optimization (DRO) with an adaptive Wasserstein radius.
result Theoretical bounds on classification error across distributions, demonstrating error propagation dynamics.
The paper tackles MAP inference over non-convex constraints in safety-critical settings.
problem Efficiently computing MAP predictions subject to non-convex constraints is challenging.
method The paper investigates conditions for exact and efficient MAP inference over continuous variables and devises scalable algorithms for both tractable and general cases.
result The proposed methods outperform constraint-agnostic baselines and scale to complex densities.
This paper focuses on convex constrained optimization problems, where the solution is subject to a convex inequality constraint. In particular, we aim at challenging problems for which both projection into the constrained domain and a linear optimization under the inequality constraint are time-consuming, which render …
DE-PSGLD samples from constrained distributions in a decentralized manner.
problem Sampling from log-concave distributions with constraints.
method Decentralized Proximal Stochastic Gradient Langevin Dynamics with proximal regularization.
result DE-PSGLD converges to a regularized Gibbs distribution and maintains posterior concentration.
New algorithms solve stochastic variational inequalities without bounded variance assumption.
problem Solving stochastic variational inequalities without bounded variance assumption.
method Developed algorithms for two classes of problems: monotone and structured nonmonotone VIs.
result Oracle complexity of O(ε^-4) for solving VIs with unbounded domains and possibly unbounded variance.
Paper proposes SOTL framework for improving transfer learning accuracy and efficiency.
problem Statistical bias and computational efficiency in multi-source domain adaptation.
method Sparse Optimization for Transfer Learning (SOTL) with L0-regularization.
result SOTL significantly improves estimation accuracy and computational speed, especially under adversarial conditions.
PNDEs project neural dynamics onto constraint manifolds, improving accuracy and stability.
problem Learning dynamics from data without violating known constraints.
method Projecting the learned vector field onto the tangent space of the constraint manifold.
result PNDEs outperform existing methods in learning constrained dynamical systems.
A new algorithm estimates output ranges for deep neural networks efficiently.
problem Estimating output ranges in deep neural networks with complex non-linearities.
method Integrates Simulated Annealing tailored for constrained domains and global optima.
result Guaranteed convergence and robust performance across various DNN architectures.
We use a locally constrained mean curvature flow to prove the isoperimetric inequality for spacelike domains in generalized Robertson-Walker spaces satisfying the null convergence condition.
Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained d…
Meta-gradient D4PG optimizes performance and constraint adherence in RL.
problem Balancing performance and adherence to complex constraints in RL.
method Uses meta-gradients to find a balance between expected return and minimizing constraint violations.
result Meta-gradient D4PG consistently outperforms baselines across MuJoCo domains.
In this paper, we explore creative image generation constrained by small data. To partially automate the creation of cartoon sketches consistent with a specific designer's style, where acquiring a very large original image set is impossible or cost prohibitive, we exploit domain specific knowledge for a huge reduction …
Hydra boosts efficiency for long-context reasoning in resource-constrained settings.
problem Quadratic complexity of transformers limits long-context reasoning in resource-constrained systems.
method Hydra uses a modular architecture with adaptive routing between sparse global attention, mixture-of-experts, and dual memories.
result Hydra achieves significant throughput and accuracy improvements for long-context reasoning.
New samplers minimize KL divergence for constrained and non-Euclidean geometries.
problem Efficient sampling from constrained and non-Euclidean distributions.
method Stein Variational Mirror Descent and Mirrored Stein Variational Gradient Descent.
result New samplers converge more rapidly and accurately than prior methods.
Paper tackles constrained learning with non-convex losses, overcoming challenges with new approach.
problem Challenges in learning with non-convex losses and statistical constraints.
method Learning in the empirical dual domain, bounding empirical duality gap.
result Established a constrained counterpart to classical learning theory.
This paper tackles the computational complexity of finding approximate stationary points in non-convex optimization.
problem Finding approximate stationary points in non-convex optimization problems.
method PLS-completeness, zero-order algorithms, and gradient queries.
result The query complexity of finding approximate stationary points is Θ(1/ε) for d=2.
Paper proposes risk-averse reinforcement learning algorithms.
problem Managing model uncertainty in reinforcement learning.
method Entropic risk constrained policy gradient and actor-critic algorithms.
result Demonstrates usefulness of risk-averse algorithms on various domains.
New diffusion models handle constrained domains, improving generative tasks.
problem Diffusion models struggle with manifolds defined by inequality constraints.
method Developed two noising processes: logarithmic barrier metric and reflected Brownian motion.
result Demonstrated practical utility on synthetic and real-world tasks.
Gaussian processes (GPs) provide a powerful framework for extrapolation, interpolation, and noise removal in regression and classification. This paper considers constraining GPs to arbitrarily-shaped domains with boundary conditions. We solve a Fourier-like generalised harmonic feature representation of the GP prior in…
Geometric inequalities for static convex domains in hyperbolic space proved.
problem Proving geometric inequalities for static convex domains in hyperbolic space.
method Using static convexity of flow hypersurfaces, new inequalities are derived.
result New family of geometric inequalities for static convex domains in hyperbolic space.
Algorithm mitigates performance loss in constrained reinforcement learning with model misspecification.
problem Performance loss in reinforcement learning policies due to model misspecification in constrained control systems.
method Proposes an algorithm to handle constrained model misspecification in continuous control systems.
result Algorithm successfully mitigates performance loss in real-world reinforcement learning tasks.
New algorithm enhances generative modeling for bounded domains.
problem Ad-hoc thresholding techniques for boundary enforcement in diffusion models.
method Reflected Schrödinger Bridge algorithm for entropy-regularized optimal transport.
result Generative modeling in diverse bounded domains with optimal transport properties.
Theoretical models of the strong nuclear interaction contain unknown coupling constants (parameters) that must be determined using a pool of calibration data. In cases where the models are complex, leading to time consuming calculations, it is particularly challenging to systematically search the corresponding paramete…
New method improves generative modeling on convex domains using regularized mirror maps and Student-t priors.
problem Challenges in generative modeling on convex domains with heavy-tailed targets.
method Mirror Flow Matching with regularized mirror maps and Student-t priors.
result Empirically outperforms baselines and achieves competitive sample quality.
RFM improves CNFs by adding a boundary constraint term and matching velocity fields.
problem Flow matching on constrained domains leads to unnatural samples.
method RFM adds a boundary constraint term and matches velocity fields in a simulation-free manner.
result RFM achieves comparable or better results on standard image benchmarks and produces high-quality samples.
Develops a theory to make learning solutions fair and safe.
problem Ensuring learning solutions are unbiased and safe in critical applications.
method Generates a generalization theory based on PAC learning framework, introduces constrained learning algorithm.
result Proves that constrained learning is as learnable as unconstrained learning, provides practical algorithm.
New conditions ensure Dantzig-Wolfe relaxation matches rank-constrained optimization problems.
problem Rank-constrained optimization problems with linear matrix inequalities.
method Investigates Dantzig-Wolfe relaxation and develops conditions for exactness.
result Conditions for extreme point, convex hull, and objective exactness.
Paper proposes a probabilistic alignment method for domain adaptation.
problem Latent distribution mismatch and miscalibrated uncertainty in adapting large-scale models.
method Bayesian latent transport framework with PAC-Bayesian regularization.
result Reduction in latent manifold discrepancy and improved uncertainty calibration.
Noise reduction is an important part of modern hearing aids and is included in most commercially available devices. Deep learning-based state-of-the-art algorithms, however, either do not consider real-time and frequency resolution constrains or result in poor quality under very noisy conditions. To improve monaural sp…
A new method solves complex constrained minimax problems.
problem Solving constrained minimax optimization problems.
method First-order augmented Lagrangian method.
result Established an operation complexity of O(ε−4logε−1). Adaptive algorithm AMSGrad converges for weakly convex constrained optimization problems.
problem Solving constrained stochastic optimization problems with weakly convex objectives.
method Analysis of AMSGrad algorithm for a specific class of problems.
result AMSGrad achieves a convergence rate of ildeO(t−1/4) for the norm of the gradient of the Moreau envelope. A new method solves a complex optimization problem efficiently.
problem Nonconvex-strongly-concave constrained minimax optimization.
method First-order augmented Lagrangian method with a first-order subproblem solver.
result Achieves improved operation complexity for finding solutions.
We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply …
PandaAI: A practical agent for neuro-symbolic data analysis and decision-making in finance
problem Sequential decision-making in finance
method Leveraging LLMs for market regime modeling and constrained alpha generation
result PandaAI achieves higher Rank IC and lower maximum drawdown
The cardinality constraint is an intrinsic way to restrict the solution structure in many domains, for example, sparse learning, feature selection, and compressed sensing. To solve a cardinality constrained problem, the key challenge is to solve the projection onto the cardinality constraint set, which is NP-hard in ge…
New method for sampling on constrained domains using orthogonal-space gradient flow.
problem Sampling on manifolds defined by constraints is challenging.
method Orthogonal-Space Variational Gradient Descent (O-Gradient)
result O-Gradient converges to the target constrained distribution efficiently.
MAGI-X learns unknown dynamics from data without numerical integration.
problem Difficult to propose ODEs in closed-form for complex systems.
method MAGI-X uses neural networks within a manifold-constrained Gaussian process framework.
result MAGI-X achieves competitive accuracy in fitting and forecasting with reduced computational time.