Improves neural network verification by merging abstract domains and Lagrangian methods.
problem Prove provable bounds for neural network outputs given input ranges.
method Uses zonotopes within a Lagrangian decomposition to verify deep neural networks.
result Yields bounds that improve upon existing techniques in both time and tightness.
LDDNN learns physical dynamics from data without exact solutions.
problem Learning physical dynamics from data without exact solutions.
method LDDNN topology that learns Lagrangian density from data.
result LDDNN can learn physical dynamics from data.
LNNs learn Lagrangians without canonical coordinates, conserving energy and relativity.
problem Neural networks struggle to learn physical symmetries like conservation laws.
method Lagrangian Neural Networks (LNNs) parameterize arbitrary Lagrangians using neural networks.
result LNNs conserve energy and relativity in complex systems.
Proposes TgNN-LD to improve neural network effectiveness and efficiency.
problem Limits in maintaining tradeoff between data and domain knowledge.
method Converts loss function to constrained form with PDEs, ECs, and EK as constraints, incorporating Lagrangian variables for equitable tradeoff.
result Improves prediction accuracy and conserves resources.
Stochastic approach improves neural network training for kinetic simulations.
problem Training neural networks under physical constraints in kinetic fusion simulations.
method Stochastic augmented Lagrangian approach using pyTorch.
result Higher model prediction accuracy achieved compared to fixed penalty method.
Model learns Lagrangian dynamics from images for better prediction and control.
problem Lack of interpretability and applicability to high-dimensional data like images.
method Unsupervised neural network model that learns Lagrangian dynamics from images using a coordinate-aware VAE.
result Model infers interpretable Lagrangian dynamics, enabling long-term prediction and synthesis of controllers.
Novel approach using Lagrangian Decomposition for neural network verification.
problem Computing tight bounds on neural network outputs.
method Lagrangian Decomposition, efficient supergradient ascent, proximal algorithm.
result Proves tighter bounds than previous dual algorithms.
A new Lagrangian method for graph neural networks accelerates state computation.
problem Efficiently computing states in graph neural networks for complex data.
method Lagrangian optimization for state convergence in graph neural networks.
result The proposed method accelerates state computation without iterative phases.
Bayesian framework discovers interpretable Lagrangian from data.
problem Discovering physical laws from limited data.
method Sparse Bayesian approach for learning interpretable Lagrangian.
result Automates Hamiltonian discovery from Lagrangian and provides ODE/PDE descriptions.
Simplifies neural network models by explicitly enforcing constraints in Cartesian coordinates.
problem Learning dynamics of complex systems efficiently and accurately.
method Embedding systems into Cartesian coordinates and using Lagrange multipliers to enforce constraints.
result Explicitly enforcing constraints leads to a 100x improvement in accuracy and data efficiency.
Noether's framework reveals symmetry-breaking in neural networks.
problem Understanding the role of symmetry breaking in neural networks.
method Developed a theoretical framework using Lagrangian mechanics.
result Identified 'kinetic symmetry breaking' and its effect on learning dynamics.
Improved neural network verification using Lagrangian decomposition and parallel algorithms.
problem Formally proving input-output properties of neural networks efficiently.
method Novel bounding and branching algorithms based on Lagrangian Decomposition and activation-based heuristics.
result Significant reduction in verification times, up to 50x faster on adversarial robustness properties.
Researchers develop neural optimal transport with Lagrangian costs for efficient computation.
problem Optimal transport between measures with Lagrangian costs for systems with geometric constraints.
method Neural network approach to compute geodesics and optimal transport maps efficiently.
result Efficient computation of geodesics and optimal transport maps without ODE solvers.
We introduce a generative adversarial network (GAN) model to simulate the 3-dimensional Lagrangian motion of particles trapped in the recirculation zone of a buoyancy-opposed flame. The GAN model comprises a stochastic recurrent neural network, serving as a generator, and a convoluted neural network, serving as a discr…
LP algorithm optimizes neural networks with architectural constraints.
problem Training neural networks with specific architectural constraints.
method Lagrangian optimization and saddle point search in adjoint space.
result LP algorithm is fully parallelizable and feasible for training deep networks.
LPINNs solve complex PDEs by reformulating PINNs on Lagrangian frame, reducing training complexity.
problem Complexity in training PINNs, especially for convection-diffusion equations.
method Propose LPINNs, a Lagrangian reformulation of PINNs, with two branches solving state variables and characteristics curves.
result Loss landscapes of LPINNs are less sensitive to problem complexity compared to traditional PINNs.
Proposes a new method for GNNs that avoids iterative node state convergence.
problem Iterative computation of node states in GNNs is inefficient and requires many epochs.
method Constrained optimization in the Lagrangian framework to learn transition function and node states simultaneously.
result The proposed method compares favorably with existing models on various benchmarks.
Physics-informed neural networks improve model accuracy and efficiency.
problem Accurate dynamic models for technical systems are hard to achieve.
method Physics-informed neural ordinary differential equations (PINODE) integrating Lagrangian mechanics.
result Hybrid model combines physical insight and data approximation.
We introduce a deep learning method to simulate the motion of particles trapped in a chaotic recirculating flame. The Lagrangian trajectories of particles, captured using a high-speed camera and subsequently reconstructed in 3-dimensional space, were used to train a variational autoencoder (VAE) which comprises multipl…
The paper develops spectral networks in symplectic topology and their relation to Lagrangian fillings.
problem Understanding spectral networks in symplectic topology and their role in Lagrangian fillings.
method Analytic results on adiabatic degeneration of Floer trajectories and explicit computation of continuation strips.
result Established equivalence between Family Floer functor and non-abelianization functor for Lagrangian fillings with spectral networks.
Physics-enhanced NNs improve predictive accuracy in small data scenarios.
problem Predicting physical systems dynamics with limited data.
method Integrating physical principles (Hamiltonian/Lagrangian) and regularization term based on energy level into neural networks.
result Significant gains in predictive accuracy for small data cases.
Bayesian Entropy Neural Networks enforce constraints on deep learning predictions.
problem Deep learning models lack well-defined constraints in their outputs.
method Bayesian Entropy Neural Networks (BENN) using Maximum Entropy principles and the method of multipliers.
result BENN improves model robustness and reliability across various applications.
This study compares different thermodynamic structure-informed neural networks for solving differential equations.
problem Improving the accuracy and physical consistency of neural network solutions to differential equations.
method Comprehensive evaluation of various thermodynamic formulations in physics-informed neural networks.
result Newtonian-residual-based PINNs fail to reliably recover physical quantities, while structure-preserving formulations enhance accuracy and robustness.
New method solves constrained optimization problems efficiently.
problem Equality-constrained nonlinear, nonconvex optimization problems.
method Adaptive inexact Newton method with randomized iterative sketching.
result Global almost sure convergence and local linear/superlinear convergence.
Paper protects privacy and fairness in deep learning models.
problem Ensuring fairness in deep learning models while protecting sensitive data.
method Uses differential privacy and Lagrangian duality to design fair predictors.
result Demonstrates improved model performance on prediction tasks.
Survey of diffusion and optimal transport methods in machine learning.
problem Design and analysis of time-evolving probability distributions in machine learning.
method Switch from Eulerian to Lagrangian representation through vector fields.
result Both diffusion methods and optimal transport offer computational advantages.
Formulates mechanics for probability distributions on statistical manifold.
problem Formulating mechanics for probability distributions on statistical manifold.
method Information-geometric formulation of Classical Mechanics on statistical manifold, using dually-flat connection and Hilbert bundle structure.
result Provides coherent formalism for Lagrangian and Hamiltonian mechanics on statistical bundle.
Neural network pruning is an important step in design process of efficient neural networks for edge devices with limited computational power. Pruning is a form of knowledge transfer from the weights of the original network to a smaller target subnetwork. We propose a new method for compute-constrained structured channe…
This work studies the robustness certification problem of neural network models, which aims to find certified adversary-free regions as large as possible around data points. In contrast to the existing approaches that seek regions bounded uniformly along all input features, we consider non-uniform bounds and use it to …
Unified analysis of neural networks in NPIV using 2SLS and MFLD.
problem Global convergence of neural networks in NPIV.
method Lifted perspective through MFLD, penalty gradient approach for bilevel optimization.
result First global convergence result of neural networks for 2SLS in NPIV.
New method uses neural networks to learn causal graphs from interventional data.
problem Challenges in learning causal directed acyclic graphs from data.
method Reformulates as continuous constrained optimization, uses neural networks, leverages interventional data.
result Method compares favorably to state of the art in various settings.
This paper addresses the problem of formally verifying desirable properties of neural networks, i.e., obtaining provable guarantees that neural networks satisfy specifications relating their inputs and outputs (robustness to bounded norm adversarial perturbations, for example). Most previous work on this topic was limi…
A new method solves distributed optimization problems over networks.
problem Solving optimization problems over networks with local cost functions and limited communication.
method Distributed semismooth Newton based augmented Lagrangian method.
result The method efficiently solves distributed optimization problems over networks.
TEAM generates more powerful adversarial examples for DNNs.
problem Vulnerability of DNNs to imperceptible adversarial examples.
method TEAM uses Taylor expansion and Lagrangian multiplier method to craft adversarial examples.
result TEAM generates adversarial examples with 100% attack success rate using smaller perturbations.
Develops regularity theory for Beckmann's optimal transport problem.
problem Minimizing total squared flux in continuous transport from source to target.
method Unconstrained Lagrangian formulation, variational first order optimality conditions, Schauder estimates.
result Exact Hölder regularity of potential, flux, and flow generating on bounded, regular domains.
Graph neural networks optimize radio resource management policies for wireless networks.
problem Optimizing user selection and power control in wireless networks with fairness constraints.
method Formulated as a Lagrangian dual problem, RRM policies are parameterized by a GNN architecture trained on channel conditions.
result The method achieves superior tradeoff between average and 5th percentile rates, demonstrating fairness.
Proves existence of Lagrangian mean curvature flow solutions.
problem Desingularizing transverse intersection points of immersed Lagrangians.
method Direct PDE approach using manifolds with corners and a-corners.
result Existence of Lagrangian mean curvature flow solutions with stronger convergence.
LFlows model fluid densities and velocities using invertible maps that satisfy the continuity equation.
problem Modeling fluid densities and velocities continuously in space and time.
method LFlows are based on invertible maps that satisfy the continuity equation, derived from classical theory of Lagrangian flows for smooth vector fields.
result LFlows show higher predictive accuracy in density modeling tasks compared to competing models in 2D and 3D.
This paper explores the potential of Lagrangian duality for learning applications that feature complex constraints. Such constraints arise in many science and engineering domains, where the task amounts to learning optimization problems which must be solved repeatedly and include hard physical and operational constrain…
In classification models fairness can be ensured by solving a constrained optimization problem. We focus on fairness constraints like Disparate Impact, Demographic Parity, and Equalized Odds, which are non-decomposable and non-convex. Researchers define convex surrogates of the constraints and then apply convex optimiz…
Deep learning has achieved astonishing results on many tasks with large amounts of data and generalization within the proximity of training data. For many important real-world applications, these requirements are unfeasible and additional prior knowledge on the task domain is required to overcome the resulting problems…
This paper proposes an in-depth re-thinking of neural computation that parallels apparently unrelated laws of physics, that are formulated in the variational framework of the least action principle. The theory holds for neural networks that are also based on any digraph, and the resulting computational scheme exhibits …
Paper designs Poisson integrators using machine learning.
problem Designing integrators that preserve Poisson geometry.
method Reformulated as an optimization problem in Hamilton-Jacobi PDE, solved using machine learning.
result Machine learning approximates solutions to Hamilton-Jacobi PDE.
Hebbian learning derived from maximum entropy principles.
problem Storing and retrieving information in neural networks.
method Maximum entropy extremization to derive Hebbian learning rules.
result Hebbian learning rules converge to Hopfield model in the big data limit.
AutoKE automates embedding physical knowledge into neural networks for complex engineering problems.
problem Complex physical equations in engineering problems.
method AutoKE framework using deep neural networks, equation parsing, automatic differentiation, adaptive weights, and NAS.
result Automatically embeds physical knowledge into neural networks for complex equations efficiently.
New method uses dynamic sampling to improve PINNs efficiency.
problem Improving sample efficiency and performance of PINNs.
method pdPINN, inspired by Eulerian formulation, uses dynamic Monte Carlo sampling from particle positions.
result Higher sample efficiency and improved performance of PINNs.
By and large, Backpropagation (BP) is regarded as one of the most important neural computation algorithms at the basis of the progress in machine learning, including the recent advances in deep learning. However, its computational structure has been the source of many debates on its arguable biological plausibility. In…
A scalable PyTorch framework for non-crossing quantile regression.
problem Non-crossing quantile regression to avoid impossible negative probability densities.
method CJQR-ALM combining Augmented Lagrangian Method, differentiable pinball loss, and L-BFGS optimization.
result Achieves near-zero crossing rates on large datasets within minutes.