Simplifies neural network constraints with computationally efficient method.
problem Implementing hard output constraints in neural networks.
method Additional neural network layer for output constraints.
result Computational simplicity with complexity O(n*m) for linear constraints.
The paper trains neural networks with robustness guarantees using semidefinite constraints.
problem Training neural networks with robustness and stability guarantees.
method Exploiting the banded structure of semidefinite constraints, an efficient and scalable training scheme based on interior point methods is set up.
result The method allows for enforcing Lipschitz constraints in large-scale deep neural networks, as demonstrated in numerical examples.
HardNet adds hard constraints to neural networks without sacrificing performance.
problem Ensuring adherence to input-dependent constraints in neural networks.
method Appends a differentiable enforcement layer to neural networks for end-to-end training with hard constraint guarantees.
result HardNet retains neural networks' universal approximation capabilities and enables efficient optimization.
HardCoRe-NAS finds fitting neural networks adhering to hard resource constraints.
problem Finding fitting neural networks that adhere to hard resource constraints.
method Accurate formulation of resource requirement and scalable search method.
result HardCoRe-NAS generates state-of-the-art architectures strictly satisfying hard resource constraints.
SnareNet adds repair layers to neural networks to ensure outputs meet physical constraints.
problem Unconstrained neural network predictions violate physical or safety requirements.
method SnareNet appends a differentiable repair layer that navigates constraints to produce feasible outputs.
result SnareNet consistently improves objective quality while satisfying constraints more reliably.
DiffSlack learns neural networks with nonlinear constraints via learnable slack variables.
problem Enforcing nonlinear inequality constraints in neural networks.
method DiffSlack reformulates inequalities as equalities with learnable slack variables, predicting them as part of the network output.
result DiffSlack achieves higher planning success rates and stronger geometric constraint satisfaction compared to existing methods.
FISAR uses neural networks to optimize safe reinforcement learning with forward-invariant constraints.
problem Safe reinforcement learning with constraints in safety-critical environments.
method Imposing linear constraints on policy parameters' updating dynamics, using a DNN-based optimizer to satisfy these constraints.
result The policy decreases constraint violation and maximizes cumulative reward monotonically.
Proposes methods to add constraints to neural networks to improve stability and generalization.
problem Improving stability and generalization of neural networks.
method Constraint-based regularization using stochastic gradient Langevin dynamics.
result Constraints help stabilize and improve the robustness of deep neural networks.
LP algorithm optimizes neural networks with architectural constraints.
problem Training neural networks with specific architectural constraints.
method Lagrangian optimization and saddle point search in adjoint space.
result LP algorithm is fully parallelizable and feasible for training deep networks.
Wide deep neural networks are easy to optimize without constraints.
problem Optimizing wide deep neural networks.
method Analysis of optimization landscapes and empirical-risk minimization.
result Wide neural networks have no confined points, making optimization easier.
CANs improve GANs by enforcing structured constraints during training.
problem Generating valid structured objects like molecules and game maps from examples alone.
method Constrained Adversarial Networks (CANs) embed constraints into the model during training, penalizing invalid structures.
result CANs efficiently generate high-quality and novel valid structures.
Bayesian Entropy Neural Networks enforce constraints on deep learning predictions.
problem Deep learning models lack well-defined constraints in their outputs.
method Bayesian Entropy Neural Networks (BENN) using Maximum Entropy principles and the method of multipliers.
result BENN improves model robustness and reliability across various applications.
This work shows neural networks can solve non-convex constraints problems.
problem Training neural networks under non-convex constraints.
method Project stochastic gradient descent with no-regret analysis of online learning.
result Overparameterized neural networks achieve near-optimal and near-feasible solutions.
POLICE enforces linear constraints on deep neural networks efficiently.
problem Enforcing constraints on deep neural networks without affecting optimization.
method Provably optimal affine constraint enforcement method that minimally modifies DNNs.
result POLICE ensures DNNs fulfill affine constraints during training and testing.
New algorithm optimizes online network resource allocation with long-term constraints.
problem Optimal resource reservation in communication networks with job transfers and budget limits.
method Randomized exponentially weighted method for long-term constraints.
result Upper bound for regret and cumulative constraint violations established.
This work is a further study on the Generalized Constraint Neural Network (GCNN) model [1], [2]. Two challenges are encountered in the study, that is, to embed any type of prior information and to select its imposing schemes. The work focuses on the second challenge and studies a new constraint imposing scheme for equa…
Enhanced neural network framework improves constraint satisfaction with topological conditioning.
problem Maintaining semantic coherence while satisfying physical and logical constraints in neuro-symbolic reasoning.
method Integrates topological conditioning with gradient stabilization mechanisms using Forman-Ricci curvature, Deep Delta Learning, and Covariance Matrix Adaptation Evolution Strategy.
result Achieves mean energy reduction to 1.15 compared to baseline values of 11.68, with 95 percent success rate.
Neural networks learn vector fields constrained by linear operators.
problem Learning vector fields from physical systems with linear operator constraints.
method Model the target function as a linear transformation of a potential field, which is a neural network.
result Predictions of the target function satisfy the linear operator constraints.
Constraints improve deep neural network training by stabilizing and enhancing robustness.
problem Vanishing/exploding gradients and poor weight magnitudes in deep neural networks.
method Weight-constrained stochastic dynamics using Langevin dynamics framework.
result Enhanced exploration of the loss landscape and improved generalization.
Improved sample complexity for ReLU networks with norm constraints.
problem Estimating sample complexity for ReLU networks under norm constraints.
method Refined Rademacher complexity analysis for function class.
result Often no explicit depth-dependence in sample complexity bound.
Proposes a new algorithm for learning continuous-time Bayesian network structures.
problem Lack of constraint-based algorithms for continuous-time Bayesian networks.
method Develops a constraint-based algorithm using statistical tests for conditional independence.
result The proposed algorithm is more accurate with variables having more than two values.
Simplifies neural network models by explicitly enforcing constraints in Cartesian coordinates.
problem Learning dynamics of complex systems efficiently and accurately.
method Embedding systems into Cartesian coordinates and using Lagrange multipliers to enforce constraints.
result Explicitly enforcing constraints leads to a 100x improvement in accuracy and data efficiency.
Generative adversarial networks (GANs) were initially proposed to generate images by learning from a large number of samples. Recently, GANs have been used to emulate complex physical systems such as turbulent flows. However, a critical question must be answered before GANs can be considered trusted emulators for physi…
The paper explores how to learn models that respect constraints in probabilistic learning.
problem Learning models that respect declared constraints in probabilistic learning.
method Mathematical inquiry on tractable probabilistic models like sum-product networks.
result Determines conditions under which constraints can be integrated with model learning.
Optimizes multi-period portfolios with tail-risk constraints using neural networks.
problem Maximizing expected return while managing tail-risk constraints over multiple periods.
method Recurrent neural network approach to approximate optimal policy.
result Validated in financial and insurance models, capturing long-term risk dynamics.
Neural networks are increasingly used in complex (data-driven) simulations as surrogates or for accelerating the computation of classical surrogates. In many applications physical constraints, such as mass or energy conservation, must be satisfied to obtain reliable results. However, standard machine learning algorithm…
Improved machine learning with reduced tensor rank constraints and dropout.
problem Efficiently approximating large tensors in machine learning.
method Tree tensor networks with CP rank constraints and tensor dropout.
result Low-rank TTN classifier achieves 90.3% accuracy in Fashion-MNIST.
We derive a closed form solution for an optimal control problem related to an interbank lending schemes subject to terminal probability constraints on the failure of banks which are interconnected through a financial network. The derived solution applies to a real banks network by obtaining a general solution when the …
Novel framework for Bayesian neural networks incorporating task-specific constraints.
problem Task-specific constraints in supervised model deployment.
method Introduces Output-Constrained BNN (OC-BNN) framework.
result OC-BNNs effectively incorporate prior expert knowledge and desiderata like safety and fairness.
Automatically learns flexible symmetry constraints in neural networks using gradients.
problem Fixed hard constraints on neural network functions that cannot be adapted.
method Improves parameterisations of soft equivariance and optimizes marginal likelihood using differentiable Laplace approximations.
result Achieves equivalent or improved performance on image classification tasks compared to baselines with hard-coded symmetry.
In classification models fairness can be ensured by solving a constrained optimization problem. We focus on fairness constraints like Disparate Impact, Demographic Parity, and Equalized Odds, which are non-decomposable and non-convex. Researchers define convex surrogates of the constraints and then apply convex optimiz…
Paper proposes efficient network pruning method for deep neural networks.
problem High computational and memory cost of deep neural networks.
method Annealing and direct sparsity control for channel-level pruning.
result Proposed method achieves better or competitive performance compared to other methods.
New neural network smoothness constraints improve model performance.
problem Improving model sensitivity to input changes for better generalization and robustness.
method Exploring current smoothness constraints and proposing new flexible definitions.
result Current smoothness constraints lack flexibility and understanding of data, tasks, and learning.
We propose a mixed integer programming (MIP) model and iterative algorithms based on topological orders to solve optimization problems with acyclic constraints on a directed graph. The proposed MIP model has a significantly lower number of constraints compared to popular MIP models based on cycle elimination constraint…
Topological constraints improve neural network generalization.
problem Improving generalization in neural networks with limited data.
method Imposing topological constraints on internal representations of neural networks.
result Topological constraints lead to better mass concentration around training instances, improving generalization.
Convex neural networks enforce convex constraints on weights and activations, improving generalization.
problem Improving generalization and reducing overfitting in neural networks.
method Enforce convex constraints on weights and activations, using non-negative weights and non-decreasing convex activation functions.
result Convex neural networks self-regularize, outperforming base architectures and achieving similar performance to convolutional architectures.
Model explains money creation under regulatory constraints.
problem Understanding money creation dynamics under regulatory pressures.
method Agent-based model of secured interbank network.
result Excess liquidity and repurchase agreements emerge due to regulatory constraints.
In this paper, we propose a simple, versatile model for learning the structure and parameters of multivariate distributions from a data set. Learning a Markov network from a given data set is not a simple problem, because Markov networks rigorously represent Markov properties, and this rigor imposes complex constraints…
This dissertation uses ILP to learn Bayesian network structures efficiently.
problem Learning the structure of Bayesian networks from data.
method Integer Linear Programming formulation with cluster constraints and cutting planes.
result The approach finds feasible solutions for Bayesian network structures efficiently.
State-augmented algorithm optimizes wireless network resource management.
problem Optimizing resource allocation in multi-user wireless networks.
method Proposes a state-augmented algorithm using dual variables.
result Feasible and near-optimal resource decisions achieved.
Study approximates BSDEs with constraints using machine learning.
problem Approximating BSDEs with a constraint on the gains process.
method Discretization followed by machine learning approximation of the discretely constrained BSDE.
result The discretely constrained BSDE converges to the continuously constrained one as the mesh grid approaches zero.
SMGD trains low-bit neural networks with memory constraints.
problem Training large neural networks with limited memory.
method Stochastic Markov Gradient Descent (SMGD).
result Encouraging numerical results and theoretical guarantees.
S3C2 uses Siamese networks for semi-supervised clustering with pairwise constraints.
problem Semi-supervised clustering with pairwise constraints.
method S3C2 decomposes SSC into two classification tasks: first, using Siamese networks to label unlabeled pairs; second, using the labeled dataset for clustering.
result S3C2 outperforms existing SSC methods on various datasets.
Paper proposes Vertex Networks for reinforcement learning of control systems with safety guarantees.
problem Challenges in reinforcement learning with hard state and action constraints.
method Vertex Networks incorporate safety constraints into policy network architecture, ensuring safety during exploration.
result Proposed Vertex Networks outperform vanilla reinforcement learning in benchmark control tasks.
Existing information-theoretic frameworks based on maximum entropy network ensembles are not able to explain the emergence of heterogeneity in complex networks. Here, we fill this gap of knowledge by developing a classical framework for networks based on finding an optimal trade-off between the information content of a…
We propose Trusted Neural Network (TNN) models, which are deep neural network models that satisfy safety constraints critical to the application domain. We investigate different mechanisms for incorporating rule-based knowledge in the form of first-order logic constraints into a TNN model, where rules that encode safet…
Synthesizing programs using example input/outputs is a classic problem in artificial intelligence. We present a method for solving Programming By Example (PBE) problems by using a neural model to guide the search of a constraint logic programming system called miniKanren. Crucially, the neural model uses miniKanren's i…
We solve S-shaped utility portfolio selection with SD constraints using algorithms and neural networks.
problem Optimizing portfolios with S-shaped utility functions under SD constraints.
method First-order SD constraint solution, numerical algorithm for SSD, neural network approach.
result Effective numerical and neural network solutions for SSD constrained problems.