Minimal DAMs can recognize patterns in high noise, even with minimal data.
problem Pattern recognition in high noise conditions with limited data.
method Interpolating between DAMs and spin glasses, using minimal dense associative networks and extremizing quenched free-energy.
result Minimal DAMs can correctly recognize patterns even when the signal is very weak and noise is high.
Develops hyperparameter transfer methods for Dense Associative Memories.
problem Challenges in transferring hyperparameters for DenseAMs due to unique architecture and activation functions.
method Derives explicit prescriptions for hyperparameter transfer from small to large models.
result Excellent agreement between theoretical and empirical results.
The study finds dense clusters of solutions in a simple neural network model, providing bounds for their existence.
problem Exploring the existence of minimizers in a simple neural network model with binary weights.
method Formulating the learning problem as a constraint satisfaction problem and computing moment bounds for the existence of solutions.
result First rigorous steps toward proving the existence of dense clusters of solutions in certain parameter regimes.
Dense Associative Memories outperform classical networks in robustness and signal processing.
problem Improving neural network performance in adversarial attacks and weak signal processing.
method Relaxing replica symmetry in statistical mechanics of spin glasses to analyze unsupervised and supervised learning.
result Explicit analytical investigation of phase diagrams and storage capacities for Dense Associative Memories.
A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of mo…
We study the regularity properties of the value function associated with an affine optimal control problem with quadratic cost plus a potential, for a fixed final time and initial point. Without assuming any condition on singular minimizers, we prove that the value function is continuous on an open and dense subset of …
This paper introduces a hierarchical associative memory model with multiple layers.
problem Limitations of traditional associative memory models with only one hidden layer.
method Develops a fully recurrent model with arbitrary layers, including locally connected ones, and a corresponding energy function.
result The model can dynamically assemble memories using weights from lower layers and higher layers' rules.
A new theory explains large associative memory with biological plausibility.
problem Large associative memory in neurobiology and machine learning.
method Microscopic theory with hidden neurons and two-body interactions.
result Valid model of large associative memory with biological plausibility.
SSVI efficiently trains sparse Bayesian neural networks with minimal compression and performance loss.
problem Efficiently training Bayesian neural networks with uncertainty quantification.
method SSVI optimizes a sparse subspace basis selection and its parameters alternately, guided by weight distribution statistics.
result SSVI achieves significant compression (10-20x model size reduction) with minimal performance drop (under 3%) and FLOPs reduction (up to 20x) compared to dense Variational Inference.
Minimal variations guide unsupervised learning for better downstream tasks.
problem Efficiently describing raw data for various future tasks.
method Minimal variations as a guiding principle for unsupervised representation learning.
result Unveiling minimal variations as a principle behind unsupervised learning.
Energy Transformer integrates attention, energy models, and associative memory.
problem Lack of clear theoretical foundations in attention mechanisms and straightforward design of energy functions in energy-based models.
method Proposes Energy Transformer, a sequence of attention layers with a specifically engineered energy function.
result Obtained strong results on graph anomaly detection and classification tasks.
Detection of dense cycles in graphs reveals a gap between easy detection and hard recovery.
problem Detecting and recovering dense cycles in Erdős-Rényi graphs.
method Characterization of computational thresholds for detection and recovery using low-degree polynomial algorithms.
result A gap exists between the detection and recovery thresholds for certain parameter regimes.
Model pruning seeks to induce sparsity in a deep neural network's various connection matrices, thereby reducing the number of nonzero-valued parameters in the model. Recent reports (Han et al., 2015; Narang et al., 2017) prune deep networks at the cost of only a marginal loss in accuracy and achieve a sizable reduction…
Statistical network modeling has focused on representing the graph as a discrete structure, namely the adjacency matrix, and considering the exchangeability of this array. In such cases, the Aldous-Hoover representation theorem (Aldous, 1981;Hoover, 1979} applies and informs us that the graph is necessarily either dens…
Tensor Neural Networks improve pricing accuracy for interest rate derivatives.
problem Inaccurate pricing of Bermudan Swaptions using traditional methods.
method Leveraging Tensor Neural Networks to solve backward Stochastic Differential Equations.
result Tensor Neural Networks provide more accurate and robust prices than Dense Neural Networks.
A new associative memory uses Sinkhorn divergence for efficient pattern retrieval.
problem Efficiently retrieving patterns from large datasets of weighted point clouds.
method Derived retrieval dynamics as a SHK gradient flow, discretized for a deterministic algorithm.
result Proved basin invariance, geometric convergence, and robust recovery from perturbations.
Neural networks can detect weak patterns hidden in noise.
problem Detecting weak patterns in noisy data.
method Developed a three-layer Sejnowski machine with redundant representation, showing patterns can be stored and retrieved efficiently.
result Neural networks can retrieve information with intensity O(1) even in the presence of noise O(\sqrt{N}) in the large N limit.
Dense neural networks can't approximate all functions.
problem Approximation capabilities of dense neural networks.
method Model compression approach combining weak regularity lemma and graph neural networks.
result Existence of Lipschitz continuous functions not approximable by dense neural networks.
Study on horospheres in higher rank homogeneous spaces, proving density properties.
problem Density of horospheres in higher rank homogeneous spaces.
method Analyzing maximal horospherical subgroups and their minimal subsets in the context of Furstenberg boundary.
result Equivalence of horospherical limit points and density properties in higher rank homogeneous spaces.
The key idea of current deep learning methods for dense prediction is to apply a model on a regular patch centered on each pixel to make pixel-wise predictions. These methods are limited in the sense that the patches are determined by network architecture instead of learned from data. In this work, we propose the dense…
Dense neural networks learn efficiently with large datasets and noise.
problem Training neural networks with large, noisy datasets.
method Statistical mechanics and Monte Carlo simulations.
result Dense neural networks can handle large amounts of patterns and recognize patterns at high signal-to-noise ratios.
Improves generative model coverage of underrepresented modes.
problem Generative models miss underrepresented modes in data.
method Leverage score sampling for complete mode coverage.
result Significantly improves mode coverage compared to standard methods.
This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.
problem Understanding the loss landscape of sparse neural networks, especially one-hidden-layer networks.
method Analyzes sparse networks with dense and sparse final layers, focusing on linear and non-linear models.
result Sparse networks can have no spurious valleys under certain conditions, but spurious valleys and minima can exist for wide sparse networks.
In this paper we prove that, given an open Riemann surface M and an integer n≥3, the set of complete conformal minimal immersions M→Rn with X(M)=Rn forms a dense subset in the space of all conformal minimal immersions M→Rn endowed with the compact-open topology.…
Proves unique maps from certain spaces to others.
problem Uniqueness of equivariant harmonic maps into specific spaces.
method Analyzes maps into irreducible symmetric spaces and Euclidean buildings.
result Proves uniqueness of maps for certain actions.
Analyzes unsupervised neural networks using statistical mechanics and Monte Carlo simulations.
problem Understanding computational capabilities of unsupervised neural networks.
method Statistical mechanics approach and Monte Carlo simulations.
result Obtained a phase diagram summarizing network performance.
Neural networks are dense among Lipschitz functions with fixed Lipschitz constant.
problem Characterizing neural network approximations to Lipschitz functions.
method Analyzing L-Lipschitz neural networks and their density in L-Lipschitz functions. result One layer neural networks are dense in the set of all L-Lipschitz functions. This study analyzes cryptocurrency market crashes using complex network analysis.
problem Identifying and understanding dynamics of cryptocurrency market crashes.
method Complex network analysis of cryptocurrency market during pre-crash, crash, and post-crash periods.
result Network density and clustering coefficient spike during crashes, indicating uninformed panic sell-off.
Extends DAMs to Gaussian distributions for efficient pattern storage and retrieval.
problem Limited storage capacity and retrieval methods for non-vector pattern representations.
method Introduces a log-sum-exp energy function over Gaussian distributions, using optimal transport maps for retrieval dynamics.
result Proves exponential storage capacity and provides quantitative retrieval guarantees.
RDL-Net improves speech enhancement with fewer parameters and better performance.
problem Improving speech enhancement with fewer parameters and better performance.
method Proposes RDL-Net, a CNN combining residual and dense aggregations without over-allocating parameters.
result RDL-Net achieves higher speech enhancement performance with fewer parameters and lower computational requirements.
Enhanced image denoising with MWRDCNN using residual dense blocks.
problem Image denoising with improved performance and robustness.
method Multi-wavelet residual dense convolutional neural network (MWRDCNN) with residual dense blocks (RDBs).
result Significantly improved performance in image denoising compared to existing techniques.
New proof shows almost all surface group actions are dense.
problem Transitivity of normal subgroups on character varieties.
method Proved almost minimal action of non-trivial normal subgroups.
result Almost all points in character variety have dense orbits.
A dense amalgam connects boundaries of groups split by finite subgroups.
problem Understanding boundaries of groups split by finite subgroups.
method Introducing dense amalgam and applying it to EZ-boundaries. result Boundaries of groups split by finite subgroups have a dense amalgam structure.
New framework for dense weighted networks with community-specific patterns.
problem Dense networks with varying edge weights across communities.
method Proposes a new model with functions mapping node characteristics to edge weights, requiring few parameters.
result Developed a bootstrap methodology for generating new networks.
Generative model captures hubs and dense communities in social networks.
problem Capturing both hubs and dense communities in social networks.
method Graphon mixture model with a new condition on sparse graphs.
result Estimation of hub normalized degree and graphon for sparse components.
Let M be a compact, orientable, mean convex 3-manifold with boundary. We show that the set of all simple closed curves in the boundary of M which bound unique area minimizing disks in M is dense in the space of simple closed curves in the boundary of M which are nullhomotopic in M. We also show that the set of all simp…
Method generates dense fields from sparse measurements without needing spatial statistics or examples.
problem Generating dense physical fields from sparse measurements.
method Introduces a differentiable numerical simulator into neural network training.
result Superior results on fluid mechanics problems compared to statistical and neural network methods.
Polynomial-time test for detecting dense subgraphs in heterogeneous networks.
problem Detecting a planted community in heterogeneous networks.
method Proposes a polynomial-time test with a standard normal distribution null limiting distribution.
result The test is efficient and performs well in both simulations and real data.
Study shows mixing of flows on specific geometric spaces.
problem Mixing of one-parameter diagonal flows on Anosov homogeneous spaces.
method Proves local mixing for flows on $Γackslash G$ with deviations in transverse subspaces.
result Local mixing of flows on $Γackslash G$ for various directions.
New models explain residual and dilated dense neural networks using sparse coding.
problem Lack of theoretical understanding of residual and dilated dense neural networks.
method Proposed Res-CSC and MSD-CSC models, derived mathematical relationships, implemented ISTA.
result Mathematical understanding of residual and dilated dense neural networks.
New algorithm improves CRF inference and learning.
problem Efficient inference and learning for dense CRFs.
method Regularized Frank-Wolfe algorithm for nonconvex CRF optimization.
result Regularized Frank-Wolfe outperforms mean field and CNN baselines.
In this work, a dense recurrent convolutional neural network (DRCNN) was constructed to detect sleep disorders including arousal, apnea and hypopnea using Polysomnography (PSG) measurement channels provided in the 2018 Physionet challenge database. Our model structure is composed of multiple dense convolutional units (…
We find that cusp densities of hyperbolic knots in the 3-sphere are dense in [0,0.6826...] and those of links are dense in [0,0.853...]. We define a new invariant associated with cusp volume, the cusp crossing density, as the ratio between the cusp volume and the crossing number of a link, and show that cusp crossing d…
Paper proposes dense average network for improved power load forecasting.
problem Improving power load forecasting accuracy to save millions for the power industry.
method Introduces dense average connection and constructs dense average network for power load forecasting.
result Proposed model outperforms existing methods on public datasets.
We address feature interpretation and reproducibility issues in dense nets, proposing a modified loss function.
problem Feature interpretation and reproducibility issues in dense nets.
method Proposed a modified loss function to circumvent basis collapse.
result Substantially concise nets with 100x fewer parameters and lower MSE loss.
In this paper we construct complete simply connected minimal surfaces with a prescribed coordinate function. Moreover, we prove that these surfaces are dense in the space of all minimal surfaces with this coordinate function (with the topology of the smooth convergence on compact sets).
Sparse butterfly network replaces dense layers in neural networks, improving expressibility and performance.
problem Improving expressibility and performance of neural networks with dense layers.
method Replacing dense layers with a butterfly network architecture.
result The proposed architecture significantly reduces the number of weights from quadratic to nearly linear, with comparable or better performance.
Unified framework for network model assessment using maximum entropy.
problem Statistical inference for network models.
method Constrained entropy-maximization problem, Lagrange multipliers.
result Consistent goodness-of-fit and two-sample tests for network models.