Paper links network Lasso to network flow optimization.
problem Joint clustering and optimization of networked data.
method Exploration of duality between network Lasso and network flow optimization.
result nLasso is equivalent to a minimum-cost flow problem on the data network structure.
The robustness and integrity of IP networks require efficient tools for traffic monitoring and analysis, which scale well with traffic volume and network size. We address the problem of optimal large-scale flow monitoring of computer networks under resource constraints. We propose a stochastic optimization framework wh…
A new method computes high-dimensional optimal transport using flow neural networks.
problem Computing optimal transport for high-dimensional data.
method Optimizing a flow model to minimize transport cost between two arbitrary distributions.
result Trained optimal transport flow enables downstream tasks like DRE and domain adaptation.
Gradient flows of neural networks converge to optimal values or diverge, with thresholds and asymptotic behaviors.
problem Understanding the convergence and divergence of gradient flows in neural networks.
method Analysis of gradient flows on loss landscapes of neural networks using o-minimal structures.
result Gradient flows either converge to optimal values or diverge to infinity, with thresholds and asymptotic behaviors.
A method for optimal Bayesian filtering using progressive particle flow and optimal transport maps.
problem Optimizing Bayesian filtering with deterministic particles to avoid degeneration.
method Progressive flow of particles through a sequence of sub-steps, each using an optimal transport map to replace non-equally weighted particles with equally weighted ones.
result The method avoids particle degeneration and simplifies the filtering process by not requiring inversions or monotonicity constraints.
This work develops a particle system to approximate Fisher-Rao gradient flows in mean-field optimization.
problem Optimizing probability measures in neural network contexts.
method Constructing an interacting particle system approximating Fisher-Rao gradient flows.
result Propagation of chaos for the Fisher-Rao gradient flow in entropic mean-field optimization.
GFM models neural network training as a dynamical system to forecast final weights.
problem Computational intensity and inefficiency in training deep neural networks.
method Gradient Flow Matching (GFM) treats training as a dynamical system with learned vector fields.
result GFM achieves forecasting accuracy competitive with Transformer-based models and significantly outperforms classical baselines.
We present a unified method, based on convex optimization, for managing the power produced and consumed by a network of devices over time. We start with the simple setting of optimizing power flows in a static network, and then proceed to the case of optimizing dynamic power flows, i.e., power flows that change with ti…
Sharp results link DLN gradient flow to basis pursuit optimization and GHA phase transitions.
problem Understanding implicit regularization in Diagonal Linear Networks.
method Sharp convergence bounds and characterization of ℓ1 minimizers. result Gradient flow of DLNs with tiny initialization approximates minimizers of basis pursuit optimization problem.
CPFM integrates dimensionality reduction and reconstruction with flow networks.
problem Learning coupled continuous flows for data and embeddings.
method Coupled flow matching framework with Gromov-Wasserstein objective and dual-conditional flow network.
result CPFM preserves and recovers residual information in latent space.
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth and invertible transformation that maps a simple distribution to the desired ma…
New method uses neural nets in Hilbert space for option pricing on flow forwards.
problem Pricing options on flow forwards with neural networks in Hilbert space.
method Optimization problem in Hilbert space solved by a novel feedforward neural network architecture.
result Excellent numerical efficiency and superior performance over classical methods.
COMBO network improves optical flow estimation by combining deep learning with brightness constancy.
problem Optical flow estimation using deep learning requires complex training schemes.
method COMBO network explicitly exploits brightness constancy and combines it with a data-driven approach.
result COMBO network outperforms state-of-the-art methods on various benchmarks.
A new flow optimizes synthetic data alignment for generative modeling.
problem Optimizing synthetic data to match target distributions.
method Score difference (SD) flow to reduce KL divergence.
result SD flow provides a theoretical link between generative models.
Softmax policy gradient achieves global optimality in wide neural networks with entropy regularization.
problem Optimizing softmax policies with neural networks in the mean-field regime.
method Modeling neural networks as Wasserstein gradient flows and proving global optimality of fixed points.
result Global optimality of softmax policy gradient in wide single hidden layer neural networks with entropy regularization.
Gradient methods work well on overparameterized diagonal linear networks.
problem Understanding why gradient-based methods work well in overparameterized models.
method Study of Deep Diagonal Linear Networks with gradient flow analysis.
result Gradient flow on layer parameters induces a mirror-flow dynamic in the effective parameter space, leading to explicit convergence guarantees.
NTK neural networks are robust to adversarial attacks in nonparametric regression.
problem Adversarial robustness of neural networks in nonparametric regression.
method Gradient flow with early stopping for NTK neural networks, proving robustness in Sobolev spaces.
result NTK neural networks achieve optimal adversarial robustness rates in Sobolev spaces.
A new method uses normalizing flows to approximate optimal transport between empirical distributions.
problem Learning an optimal transport map between two empirical distributions.
method Relaxing the Monge formulation of optimal transport, using normalizing flows to approximate the solution.
result The method provides a good approximation of the true optimal transport.
Stable neural flows ensure robustness and efficiency in deep learning.
problem Ensuring robustness and stability in deep learning models.
method Introducing a stable variant of neural ODEs with a neural network parametrizing an energy functional, solving as an optimal control problem with adjoint sensitivity analysis.
result The proposed model provides robustness against input perturbations and low computational burden.
Rectified flows achieve optimal sample complexity for generating data.
problem Generating high-quality data samples efficiently.
method Rectified flows constrain transport trajectories to be linear, enabling efficient sampling.
result Achieve sample complexity of ildeO(ε−2), matching optimal rate for mean estimation. Paper establishes a generalization bound for gradient flow using a data-dependent kernel.
problem Understanding the generalization properties of gradient-based optimization methods.
method Establishes a generalization bound for gradient flow through a data-dependent kernel called the loss path kernel (LPK).
result The LPK captures the entire training trajectory and leads to tighter generalization guarantees.
Generative Flow Networks use submodular upper bounds to generate more data.
problem Generating data from unknown, complex reward functions efficiently.
method Introduce submodular upper bounds to estimate reward, use Optimism in the Face of Uncertainty principle to train GFNs.
result SUBo-GFN generates significantly more data than classical GFNs.
We study a game-theoretic variant of the maximum circulation problem. In a flow allocation game, we are given a directed flow network. Each node is a rational agent and can strategically allocate any incoming flow to the outgoing edges. Given the strategy choices of all agents, a maximal circulation that adheres to the…
Paper proves higher-order flow matching preserves optimality in generative modeling.
problem Theoretical guarantees for higher-order flow matching in generative modeling.
method Neural network approximations with controlled depth, width, and sparsity.
result Proves worst case optimality for second-order flow matching.
Deep generative networks such as GANs and normalizing flows flourish in the context of high-dimensional tasks such as image generation. However, so far exact modeling or extrapolation of distributional properties such as the tail asymptotics generated by a generative network is not available. In this paper, we address …
Proves convergence of neural networks in a two-timescale regime.
problem Training dynamics of shallow neural networks.
method Two-timescale regime analysis of gradient flow.
result Gradient flow converges to global optimum in non-convex optimization.
The curse of dimensionality affects neural network optimization, especially with smooth functions.
problem The curse of dimensionality in neural network optimization.
method Examined through the evolution of the parameter distribution under 2-Wasserstein gradient flow.
result The curse of dimensionality persists in neural network optimization, even with smooth functions.
Paper proves CFlows can approximate any diffeomorphism and applies it in Bayesian optimization.
problem Proving the universality of CFlows in approximating diffeomorphisms.
method Deriving the universality of Para-CFlows through affine coupling layers and invertible linear transforms.
result Para-CFlows can approximate any diffeomorphism in C^k-norm.
DIGRAC clusters directed graphs using flow imbalance, outperforming existing methods.
problem Clustering directed networks without label supervision.
method DIGRAC uses a graph neural network with a novel imbalance loss for directed flow imbalance.
result DIGRAC outperforms 10 state-of-the-art methods on directed graph clustering.
The paper analyzes how GANs converge using dual metric flows.
problem Understanding the convergence dynamics of GANs.
method Investigates the convergence of GANs using dual metric flows, formal definitions, and proving convergence.
result GAN learning dynamics converge to a limit when learning rate is small.
Path regularization improves GFlowNets exploration and generalization.
problem Improving GFlowNets exploration and generalization.
method Path regularization based on optimal transport theory.
result Path regularization enhances GFlowNets to generate more diverse and novel candidates.
Normalizing flows optimize Jacobian determinant for unique likelihood objective.
problem Optimizing normalizing flows for unique likelihood.
method Showed Jacobian determinant is unique for given distributions, leading to a unique global optimum. Used eigenvalues of auto-correlation matrix for explicit likelihood expression.
result Explicit expression of likelihood for flows, independent of neural network parameterization, with theoretical optimal value.
The book explains deep learning theory and how networks learn nontrivial representations.
problem Understanding and optimizing deep neural networks.
method Developed RG flow to characterize signal propagation, solved layer-to-layer equations, and analyzed representation learning.
result Predictions of trained networks are nearly-Gaussian, with depth-to-width ratio controlling deviations.
This paper analyzes neural networks for solving complex optimization problems.
problem Minimax optimization problems in infinite-dimensional function spaces.
method Mean-field analysis of stochastic gradient descent-ascent in neural networks.
result The algorithm converges to a stationary point at a sublinear rate.
MI-GAN solves OPF with renewable uncertainty using model-informed layers.
problem Optimal Power Flow (OPF) under renewable uncertainty.
method Model-Informed Generative Adversarial Network (MI-GAN) framework with three layers.
result MI-GAN improves solution feasibility and optimality.
The paper develops a new theory to understand deep learning optimization.
problem Understanding the dynamics of optimization in deep learning, especially in the edge of stability regime.
method Developed a central flow differential equation to describe the time-averaged trajectory of oscillatory optimizers.
result Central flows can predict long-term optimization trajectories with high numerical accuracy.
Flow Matching improves statistical guarantees through kernel density estimation.
problem Improving statistical guarantees for generative models.
method Connecting Flow Matching to kernel density estimation and verifying optimal rates of convergence.
result Flow Matching achieves optimal rates up to logarithmic factors for large networks and on lower-dimensional manifolds.
Study uses machine learning to optimize power generation in electrical grids.
problem Optimizing power generation in electrical grids while respecting physical and engineering constraints.
method Two formulations of ACOPF as machine learning problems: direct prediction and constraint prediction.
result Validated machine learning approaches on two benchmark grids.
We consider a class of learning problems that involve a structured sparsity-inducing norm defined as the sum of ℓ∞-norms over groups of variables. Whereas a lot of effort has been put in developing fast optimization methods when the groups are disjoint or embedded in a specific hierarchical structure, we add…
Optimizes financial auditor schedules to reduce time and costs.
problem Efficiently scheduling financial auditors with multiple constraints.
method Used Integer Linear Programming and compared two exact formulations.
result Multi-commodity network flow formulation is 24 times faster.
Variational inference improves training of generative flow networks.
problem Training generative flow networks efficiently and accurately.
method Define variational objectives in terms of KL divergences and optimize convex combinations.
result Variational inference methods can reduce the variance of gradients in training generative flow networks.
Belief Propagation algorithms are instruments used broadly to solve graphical model optimization and statistical inference problems. In the general case of a loopy Graphical Model, Belief Propagation is a heuristic which is quite successful in practice, even though its empirical success, typically, lacks theoretical gu…
iGNN tackles inverse graph prediction using invertible neural networks.
problem Inverse graph prediction problem in data analysis and machine learning.
method Developed invertible graph neural network (iGNN) to solve inverse prediction problem on graphs.
result iGNN model allows efficient generation from output labels and forward prediction.
BatchGFN uses generative flow networks for efficient batch active learning.
problem Efficiently selecting informative batches for active learning.
method Generative flow networks to sample batches proportional to a batch reward.
result Constructs highly informative batches with a single forward pass per point.
Neural execution solves complex graph problems like bipartite matching.
problem Solving complex graph algorithms like maximum bipartite matching.
method Reduces bipartite matching to a flow problem and uses Ford-Fulkerson for maximum flow.
result Neural network achieves optimal matching almost 100% of the time.
New method speeds up GAN training by solving saddle point problem.
problem Slow convergence in training Generative Adversarial Networks (GANs).
method Fluid flow mass transport formulation for strict minimization.
result Quick convergence and meaningful metrics in optimization.
A new framework solves complex optimization problems with continuous worst-case distributions.
problem Optimizing under uncertain distributions with continuous worst-case scenarios.
method Flow-based distributionally robust optimization (DRO) with Wasserstein uncertainty sets and invertible transport maps.
result The framework finds continuous worst-case distributions and samples efficiently.
The implementation of optimal power flow (OPF) methods to perform voltage and power flow regulation in electric networks is generally believed to require extensive communication. We consider distribution systems with multiple controllable Distributed Energy Resources (DERs) and present a data-driven approach to learn c…