This work replaces pixel-wise reconstruction in VAEs with real NVP transformations.
problem Shortcomings of pixel-wise reconstruction in VAEs.
method Used real-valued non-volume preserving transformations (real NVP) for conditional likelihood computation.
result A simple VAE with real NVP is competitive with complex VAE structures on image modeling tasks.
Improved flow-based models capture dependencies better with multi-scale autoregressive priors.
problem Limited expressiveness of flow-based models for long-range data dependencies.
method Introducing channel-wise dependencies through multi-scale autoregressive priors (mAR) in split coupling flow layers (mAR-SCF).
result Achieves state-of-the-art density estimation results on MNIST, CIFAR-10, and ImageNet.
Lipschitz regularization improves neural network robustness by coupling weights across layers.
problem Improving neural network robustness under random input uncertainties.
method Regularization of neural networks by their Lipschitz constant, highlighting the coupling effect on weights across layers.
result Lipschitz regularization introduces a tradeoff between robustness and expressiveness, suggesting careful implementation.
We study cascades on a two-layer multiplex network, with asymmetric feedback that depends on the coupling strength between the layers. Based on an analytical branching process approximation, we calculate the systemic risk measured by the final fraction of failed nodes on a reference layer. The results are compared with…
Proposes a model for semi-supervised learning using both labeled and unlabeled data.
problem Semi-supervised learning with limited labeled data.
method Semi-conditional normalizing flow model with conditional coupling layer.
result Model outperforms variational auto-encoders on MNIST dataset.
Unified SVD compression fails in practical tasks, highlighting the importance of per layer activation reconstruction.
problem The failure of a unified SVD compression method in practical tasks like perplexity and accuracy.
method Unified optimization problem for SVD based compression methods, focusing on cross-layer coupling.
result Downstream metrics like perplexity and accuracy degrade severely compared to standard per layer SVD LLM.
We address representational challenges in normalizing flows, particularly depth and conditioning issues.
problem Challenges in training normalizing flows, including vanishing/exploding gradients and poor conditioning.
method Analyzes representational aspects of depth and conditioning in normalizing flows, proving theoretical bounds and investigating phenomena.
result Proves that shallow affine coupling networks are universal approximators in Wasserstein distance if ill-conditioning is allowed.
New invertible transformations improve flow-based generative models.
problem Improving flow-based generative models for better performance.
method Proposed new invertible transformations and coupling layers.
result New coupling layers achieve better results in IDF.
DS2CF-Net learns hierarchical representations with deep coupled factorization and enriched prior.
problem Learning deep hierarchical representations from data.
method Dual-constrained Deep Semi-Supervised Coupled Factorization Network (DS2CF-Net) with enriched prior.
result DS2CF-Net achieves state-of-the-art performance in representation learning and clustering.
Graph-Coupled Oscillator Networks (GraphCON) tackles graph-based learning problems.
problem The oversmoothing problem in Graph Neural Networks (GNNs).
method GraphCON is a novel framework based on discretizations of ODEs modeling oscillators coupled via graph adjacency.
result GraphCON mitigates the oversmoothing problem and exploding/vanishing gradients issues.
Incorporates matrix exponential into generative flows for improved performance.
problem Improving generative flow models for better density estimation.
method Integrates matrix exponential into generative flows, proposing new layers and modifying network architecture.
result The proposed model achieves great performance on density estimation.
Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.
problem Understanding how features evolve during training in deep neural networks, especially in the large-depth limit.
method Conditional Gaussian representations and SDE system with decoupled backward weights.
result Depth-induced suppression of forward-backward coupling in infinite-depth networks, leading to a decoupled forward-backward SDE system.
Paper proves CFlows can approximate any diffeomorphism and applies it in Bayesian optimization.
problem Proving the universality of CFlows in approximating diffeomorphisms.
method Deriving the universality of Para-CFlows through affine coupling layers and invertible linear transforms.
result Para-CFlows can approximate any diffeomorphism in C^k-norm.
Chaos in cerebellar cells enhances complexity of neural patterns.
problem Understanding how cerebellar granular layer represents complex information.
method Constructed a model of cerebellar granular layer with gap junctions, evaluated using reservoir computing.
result Chaotic dynamics in the cerebellar granular layer produce complex and diverse output patterns.
A new neural network architecture reduces parameters and improves performance.
problem Reducing model complexity and improving performance in neural networks.
method Coupled ensembles of neural networks with parallel branches and tighter coupling.
result Significant performance improvements with reduced parameter count.
Paper discusses conditions for deforming coupled Kähler-Einstein metrics.
problem Conditions for deforming coupled Kähler-Einstein metrics.
method Analyzes deformation of coupled Kähler-Einstein metrics on Fano manifolds.
result Necessary and sufficient condition for deformation of coupled Kähler-Einstein metrics.
Study neural networks learning from noisy examples via reverberation.
problem Learning from noisy examples in neural networks.
method Adapted Guerra's interpolation technique to provide statistical mechanics of supervised and unsupervised learning.
result Full phase diagrams and thresholds for learning are analytically obtained.
This work improves deep learning from noisy crowdsourced labels.
problem Learning label correction and neural classifier from noisy crowdsourced data.
method Coupled Cross-Entropy Minimization (CCEM) with identifiability and regularization.
result The CCEM criterion correctly identifies annotators' confusion and neural classifier under realistic conditions.
This paper shows how to approximate any log-concave distribution using well-conditioned affine coupling flows.
problem Understanding the representational power of affine coupling flows for log-concave distributions.
method Leveraging connections between affine coupling architectures, Langevin dynamics, and Hénon maps to prove log-concave approximation.
result Any log-concave distribution can be approximated using well-conditioned affine-coupling flows.
Unified Jacobi coupling construction for various geometric settings.
problem Constructing Jacobi structures on associated bundles.
method Extending Sternberg--Weinstein coupling to Jacobi geometry.
result Associated bundles inherit Jacobi structures from base spaces.
Study Poisson structures on fibered 5-manifolds with compatibility conditions.
problem Understanding Poisson structures on fibered 5-manifolds.
method Using almost coupling condition and bigraded factorization of the Jacobi identity.
result Describe global behavior and singularities of almost coupling Poisson tensors.
Many real-world complex systems across natural, social, and economical domains consist of manifold layers to form multiplex networks. The multiple network layers give rise to nonlinear effect for the emergent dynamics of systems. Especially, weak layers that can potentially play significant role in amplifying the vulne…
Numerical observations on martingale couplings are confirmed under certain conditions.
problem Understanding the validity of numerical observations on maximizers and minimizers of martingale couplings.
method Investigation of sufficient conditions and counterexamples for the property to hold.
result The non-decreasing property of martingale couplings is preserved for maximizers under specific conditions.
This paper analyzes deep and wide transformer training dynamics.
problem Understanding the training dynamics of infinitely deep and wide transformers.
method Develops a mean-field framework for gradient-based training of transformers, controlling a neural PDE.
result Establishes a rigorous foundation for gradient-based transformer training, proving convergence to global minima.
A new method for conditional sampling using paired Wasserstein Autoencoders.
problem Conditional sampling from complex data distributions.
method Derive a novel loss function for Wasserstein Autoencoders to enable sampling from OT-type couplings.
result Learned cost-optimal transport maps and conditional sampling from an OT-type coupling.
Develop a framework for barycentric projections of optimal transport plans on Riemannian manifolds.
problem Optimal transport couplings are probabilistic objects, while many learning pipelines require deterministic maps.
method Develop a framework for barycentric projections of transport couplings on Riemannian manifolds.
result The intrinsic projection maps each source point to the conditional Fréchet mean of its destination law and is shown to be the best deterministic representative under squared geodesic loss.
CF-INNs can approximate any invertible function, resolving a long-standing problem.
problem Whether CF-INNs can approximate any invertible function.
method Demonstrated CF-INNs are universal approximators for invertible functions by showing a convenient criterion.
result CF-INNs are universal approximators for invertible functions.
vOED-NFs uses normalizing flows to improve Bayesian OED without likelihood evaluations.
problem Optimizing experiments to maximize information gain in model parameters.
method vOED-NFs combines variational approximations with normalizing flows for efficient EIG estimation.
result vOED-NFs achieves lower EIG estimation bias compared to previous methods.
Study introduces a probabilistic framework for air-sea fluxes using neural networks.
problem Accurately quantifying air-sea fluxes for understanding interactions and improving weather/climate models.
method Gaussian distributions conditioned on input variables, artificial neural networks, eddy-covariance data, minimizing negative log-likelihood loss.
result Trained neural networks provide alternative mean flux estimates and quantify uncertainty.
We review coupled SU(3)-structures, also known in the literature as restricted half-flat structures, in relation to supersymmetry. In particular, we study special classes of examples admitting such structures and the behaviour of flows of SU(3)-structures with respect to the coupled condition.
KuramotoGNN uses Kuramoto model to prevent over-smoothing in graph neural networks.
problem Over-smoothing in graph neural networks where node features become indistinguishable.
method Integrates Kuramoto model to prevent phase synchronization and instead achieve frequency synchronization.
result KuramotoGNN reduces over-smoothing on various graph deep learning tasks.
Paper introduces supervised and unsupervised TAM models for binary neurons.
problem Learning and retrieval of structured triplets of patterns in neural networks.
method Extends Hebbian paradigm to supervised and unsupervised protocols, using glassy statistical mechanical techniques.
result Obtained self-consistency equations for critical dataset sizes and retrieval performance.
Two probability distributions μ and ν in second stochastic order can be coupled by a supermartingale, and in fact by many. Is there a canonical choice? We construct and investigate two couplings which arise as optimizers for constrained Monge-Kantorovich optimal transport problems where only supermartingales are al…
This paper uses linear rational splines for invertible modeling, offering a simpler inverse and similar costs.
problem Creating expressive invertible models with tractable Jacobian determinants.
method Replacing affine transformations with linear rational splines in coupling layers.
result Linear rational splines offer a simpler inverse and similar costs for inference and generation.
New method improves BLL models for complex datasets.
problem Limited expressive capacity of Gaussian priors in BLL models.
method Combines diffusion techniques and implicit priors for variational learning.
result Enhanced predictive accuracy and uncertainty quantification.
Proof confirms condition for Kähler-Einstein metrics on toric Fano manifolds.
problem Existence of Kähler-Einstein metrics on toric Fano manifolds.
method Condition in terms of barycenters of polytopes.
result Necessary and sufficient conditions for existence of coupled Kähler-Einstein metrics and soliton solutions.
BGG-equations are geometric overdetermined systems of PDEs on parabolic geometries. Normal solutions of BGG-equations are particularly interesting and we give a simple formula for the necessary and sufficient additional integrability conditions on a solution. We then discuss a procedure for coupling known solutions of …
We propose new types of canonical metrics on Kähler manifolds, called coupled Kähler-Einstein metrics, generalizing Kähler-Einstein metrics. We prove existence and uniqueness results in the cases when the canonical bundle is ample and when the manifold is Kähler-Einstein Fano. In the Fano case we also prove that existe…
Last-layer approximation improves UQ performance without sacrificing computational efficiency.
problem Epistemic uncertainty quantification for deep neural networks.
method Comparison of full-network and last-layer linearization using theoretical and empirical approaches.
result Last-layer approximation yields comparable UQ performance with improved computational efficiency.
Improved NER performance on imbalanced data.
problem Highly unbalanced training data in NER tasks.
method Adapted a neural architecture with CRF and BI-LSTM layers, using pre-trained embeddings. Introduced a two-class split to optimize performance.
result Significant improvement in performance for weak classes with minimal training data.
A new flow family reduces to Li-Yuan-Zhang's and can lead to convergence under certain conditions.
problem Establishing convergence of coupled flow equations under various conditions.
method Introducing a one-parameter family of coupled flows and applying C0 estimates and monotonicity of energy functionals. result Convergence of the flow can be established for κeq1 under suitable conditions. Study finds optimal martingale coupling between two distributions with minimal entropy.
problem Finding the optimal martingale coupling between two distributions with minimal relative entropy.
method Solving a dual problem to find the log-density of the optimal coupling, which represents the marginal and martingale constraints.
result The log-density of the optimal coupling is given by a triplet of real functions representing the marginal and martingale constraints.
ReDi improves few-step generation for discrete data models.
problem Slow sampling speeds in discrete flow-based models.
method Rectified Discrete Flow (ReDi) reduces factorization error by rectifying coupling.
result Empirically, ReDi reduces Conditional Total Correlation and enables few-step generation.
New framework explains normalizing flows' power and limitations.
problem Understanding the expressive power and limitations of normalizing flows.
method Theoretical framework for well-conditioned coupling-based normalizing flows and volume-preserving flows.
result RealNVP is distributionally universal, but volume-preserving flows are not.
Conditions for solutions to complex Monge-Ampère equations on Fano manifolds.
problem Existence of solutions to complex Monge-Ampère equations on Fano horosymmetric manifolds.
method Necessary and sufficient conditions derived from combinatorial data.
result Conditions for existence of solutions in terms of combinatorial data.
A coupling by reflection of a time-inhomogeneous diffusion process on a manifold are studied. The condition we assume is a natural time-inhomogeneous extension of lower Ricci curvature bounds. In particular, it includes the case of backward Ricci flow. As in time-homogeneous cases, our coupling provides a gradient esti…
We give sufficient conditions for the existence of a Dirac structure on the total space of a Poisson fiber bundle endowed with a compatible connection. We also show that Cartan and Cartan-Hannay-Berry connections give rise to coupling Dirac structures.
New neural network architectures use signed permutation representations for finite groups, improving performance.
problem Designing and optimizing neural networks for finite groups with signed permutation representations.
method Introduces G-invariant deep neural networks with densely connected layers and signed permutation representations. result Signed permutation representations lead to significantly better performance in classification tasks.