Minimal networks minimize length and mass in certain configurations.
problem Finding minimal networks that minimize length and mass.
method Global and local calibrations to prove minimization properties.
result Minimal networks minimize mass and interfaces in partitions.
The paper extends minimal network theory to the sphere, proving local minimality.
problem Finding networks of minimal length on the sphere.
method Adapted spherical geometry, calibration method, and local metric perturbation estimates.
result Spherical minimal networks composed of great-circle arcs are locally length-minimizing within small geodesic balls.
The paper proves stability and convergence of minimal networks under curvature motion.
problem Stability and convergence of minimal networks under curvature motion.
method Proved Lojasiewicz-Simon gradient inequalities for minimal networks.
result Motion by curvature starting from networks close to minimal ones exists for all times and smoothly converges.
Sharpness minimization algorithms don't solely improve generalization.
problem Why do overparameterized neural networks generalize?
method Theoretical and empirical investigation of two-layer ReLU networks.
result Sharpness minimization algorithms do not always lead to better generalization.
Study minimal networks on spheres and balls near standard metrics.
problem Existence of minimal networks in spheres and balls with metrics close to standard.
method Finite-dimensional reduction method, inspired by configuration of networks and triods.
result Existence of minimal networks in spheres and balls for metrics close to standard.
We consider planar networks of three curves that meet at two junctions with prescribed equal angles, minimizing a combination of the elastic energy and the length functional. We prove existence and regularity of minimizers, and we show some properties of the minimal configurations.
A behavior of extreme networks under deformations of their boundary sets is investigated. It is shown that analyticity of a deformation of boundary set guarantees preservation of the networks types for minimal spanning trees, minimal fillings and so-called stable shortest trees in the Euclidean space.
ReLU networks implicitly favor low-rank solutions, but not as strongly as linear networks.
problem Understanding implicit regularization in ReLU networks for rank minimization.
method Analysis of gradient flow on ReLU networks, empirical testing.
result Gradient flow on ReLU networks does not necessarily minimize ranks, unlike in linear networks.
TSSM splits neural networks for parallel training with minimal accuracy loss.
problem Accuracy degradation in parallel training of deep neural networks.
method TSSM reformulates alternating minimization to achieve parallelism with minimal accuracy loss.
result TSSM achieves significant speedup without accuracy loss on multiple datasets.
This study explains how different training methods affect the minimizer of neural networks.
problem How training methods influence the minimizer of neural networks.
method Explains how initialization size, adaptive optimization (AdaGrad), and stochastic mini-batch training affect the minimizer.
result Different training methods lead to different minimizers, even in overparameterized networks.
Study finds minimal length networks connecting three points in Heisenberg group.
problem Finding minimal length networks connecting three points in the Heisenberg group.
method Proved existence of minimal horizontal triods, formulated curve shortening flow, used numerical experiments.
result Characterized and deformed minimal horizontal triods into critical points for length functional.
We study a problem of geometric graph theory: We determine the triply periodic graph in Euclidean 3-space which minimizes length among all graphs spanning a fundamental domain of 3-space with the same volume. The minimizer is the so-called srs network with quotient the complete graph on four vertices K4. The network…
The past decade has witnessed a successful application of deep learning to solving many challenging problems in machine learning and artificial intelligence. However, the loss functions of deep neural networks (especially nonlinear networks) are still far from being well understood from a theoretical aspect. In this pa…
Loss minimization leads to multicalibration for neural networks.
problem Ensuring fairness in predictions across multiple protected groups.
method Minimizing squared loss over neural networks of size n.
result Minimizing loss over neural nets of size n implies multicalibration for most values of n.
Noise in linear networks minimizes sharpness and leads to shrinkage-thresholding.
problem Minimizing sharpness in diagonal linear networks.
method Stochastic sharpness-aware minimization (SAM) with isotropic noise.
result Noise forces shrinkage-thresholding of true parameters.
Deep linear networks minimize sharpness, avoiding large eigenvalues.
problem Understanding optimization dynamics in deep linear networks for regression.
method Analyzing sharpness (largest eigenvalue of Hessian) of minimizers and gradient flow solutions.
result Gradient flow implicitly regularizes towards flat minima, with sharpness bounded by a constant.
Machine learning predicts minimal surfaces for knots, supporting a conjecture.
problem Predicting minimal surfaces for knots in hyperbolic space.
method Physics-Informed Neural Networks (PINNs) to solve minimal surface equation.
result Computational minimal surfaces align with Fine's Conjecture.
APD method decomposes neural network parameters into simple, faithful components.
problem Understanding the internal mechanisms learned by neural networks.
method Attribution-based Parameter Decomposition (APD) method.
result Demonstrated effectiveness in recovering features, separating computations, and identifying representations.
Wide deep neural networks are easy to optimize without constraints.
problem Optimizing wide deep neural networks.
method Analysis of optimization landscapes and empirical-risk minimization.
result Wide neural networks have no confined points, making optimization easier.
Generative networks minimize predictive scoring rules for probabilistic forecasting.
problem Evaluating and improving probabilistic forecasts using generative models.
method Training generative networks to minimize predictive-sequential scoring rules on temporal sequences.
result Our method outperforms adversarial approaches in probabilistic calibration.
ALMA improves clustering of multilayer networks.
problem Clustering multilayer networks with distinct layers and communities.
method Alternating minimization algorithm (ALMA) for simultaneous layer partition and community estimation.
result ALMA achieves higher accuracy than TWIST in clustering multilayer networks.
Researchers find optimal configurations of complex knots and links.
problem Finding the most efficient configurations of complex knots and links.
method Minimizing Möbius and Minimum Distance energies by describing them with a small number of free parameters.
result Optimal geometries for Hopf links, Borromean rings, and chain links are found.
Diagonal linear networks converge to lasso regularization path during training.
problem Understanding the regularization behavior of diagonal linear networks.
method Analyzing the training trajectory of diagonal linear networks and comparing it to the lasso regularization path.
result The training trajectory of diagonal linear networks is closely related to the lasso regularization path.
We consider the question of what functions can be captured by ReLU networks with an unbounded number of units (infinite width), but where the overall network Euclidean norm (sum of squares of all weights in the system, except for an unregularized bias term for each unit) is bounded; or equivalently what is the minimal …
Deep neural network with l_1-regularization achieves nearly optimal risk bounds.
problem Achieving optimal risk bounds in deep learning.
method Empirical risk minimization with l_1-regularization.
result Adaptively nearly-minimax risk bound across various function classes.
We introduce Minimal Achievable Sufficient Statistic (MASS) Learning, a training method for machine learning models that attempts to produce minimal sufficient statistics with respect to a class of functions (e.g. deep networks) being optimized over. In deriving MASS Learning, we also introduce Conserved Differential I…
Zero loss is achievable in overparametrized DL networks under specific conditions.
problem Achieving zero loss in overparametrized deep learning networks.
method Determine sufficient conditions for zero loss attainability and present an explicit construction of zero loss minimizers.
result Explicit minimizers for zero loss in overparametrized DL networks are constructed without gradient descent.
Deep neural network predicts molecular wave functions in minimal basis.
problem Improving accuracy and efficiency in quantum chemistry calculations.
method Adapted SchNet for Orbitals (SchNOrb) model in quasi-atomic minimal basis.
result Model accurately predicts molecular orbital energies and wavefunctions for large molecules.
Paper presents efficient algorithms for convolutional neural networks using Winograd minimal filtering.
problem Resource-efficient implementation of convolutional neural networks.
method Winograd minimal filtering trick applied to M-tap filters (M=3,5,7,9,11) for parallel hardware implementation.
result Approximately 30% reduction in multipliers for fully parallel hardware implementation.
The paper constructs upper bounds for cost minimization in shallow neural networks.
problem Cost minimization in underparametrized shallow ReLU networks.
method Explicit construction of upper bounds based on the geometric structure of classification data.
result An upper bound on the minimum of the cost function of order O(δP), with exact degenerate local minimum in the special case M=Q. The paper constructs minimizers for deep learning networks and analyzes their geometric structure.
problem Underparametrized deep learning networks and their minimizers.
method Direct construction of minimizers without gradient descent, considering specific settings.
result Explicit family of minimizers for the global minimum and a set of degenerate local minima.
This paper develops a novel deep recurrent neural network for sequential signal reconstruction.
problem Sequential signal reconstruction from low-dimensional measurements.
method Unfolding a reweighted ℓ1-ℓ1 minimization algorithm to design a deep recurrent neural network. result The proposed reweighted-RNN significantly outperforms existing RNN models in sequential frame reconstruction.
Symmetric critical points lead to symmetry breaking in neural networks.
problem Understanding symmetry in critical points of invariant functions.
method Analyzing the symmetry of critical points and their neighbors in invariant nonconvex functions.
result Symmetric critical points in invariant nonconvex functions are generically followed by symmetry breaking adjacent points.
Existence of minimizers proven for residual ANNs with ReLU activation.
problem Existence of minimizers in neural network optimization landscapes.
method Proof using closure of search space containing ANNs and additional discontinuous responses.
result Existence of minimizers proven for residual ANNs with ReLU activation.
Unified federated learning via GTV minimization.
problem Training local models for decentralized datasets with network structure.
method Formulated federated learning as GTV minimization, developed a decentralized algorithm.
result Upper bound on local model parameters deviation, revealing conditions for pooling homogeneous datasets.
DLM for BNNs fails to improve over ELBO optimization.
problem Performance of DLM for Bayesian Neural Networks (BNNs).
method Direct Loss Minimization (DLM) compared to ELBO optimization.
result DLM does not significantly improve over ELBO optimization for BNNs.
This paper proposes a new method to approximate posterior distributions using generative neural networks trained via scoring rule minimization.
problem Bayesian Likelihood-Free Inference for models with intractable likelihood.
method Approximate posterior with generative neural networks trained via scoring rule minimization, avoiding the instability of adversarial training.
result Scoring Rule minimization leads to better performance and uncertainty quantification compared to adversarial training.
The paper analyzes a simple neural network model with algebraic methods.
problem Finding minima of a ridge-regularized mean squared error for ReLU perceptrons.
method Developed a Divide-Enumerate-Merge strategy using computational algebra.
result Identifies both isolated and connected minima of the RR-MSE.
We propose a new deep recurrent neural network (RNN) architecture for sequential signal reconstruction. Our network is designed by unfolding the iterations of the proximal gradient method that solves the l1-l1 minimization problem. As such, our network leverages by design that signals have a sparse representation and t…
Classifies pinned p-elasticae and finds unique optimality exponents.
problem Classifying and understanding p-elasticae under pinned boundary conditions. method Classification and analysis of p-elasticae, proving uniqueness and existence. result Discovery of a unique exponent p≃1.5728 for full optimality. Improves deep neural networks using soft labels through alternating minimization.
problem Improving deep neural networks training with soft labels.
method Co-Learns DNNs and soft labels via Alternating Minimization of two objectives.
result COLAM achieves improved performance on many tasks with better testing classification accuracy.
New algorithm minimizes regret in multi-armed bandits with network interference.
problem Minimizing regret in online experiments with network interference.
method Sparse network interference model, discrete Fourier analysis, linear regression-based algorithms.
result Provable low regret algorithms for sparse interference networks.
We apply the network Lasso to classify partially labeled data points which are characterized by high-dimensional feature vectors. In order to learn an accurate classifier from limited amounts of labeled data, we borrow statistical strength, via an intrinsic network structure, across the dataset. The resulting logistic …
New framework minimizes interference and selection bias in network A/B testing.
problem Interference and selection bias in network A/B testing.
method Proposes a principled framework that jointly minimizes interference and selection bias using edge spillover probability and cluster matching.
result Significantly lower error in causal effect estimation compared to existing solutions.
New neural network criterion connects RH to minimization problem.
problem Riemann Hypothesis (RH) about zeta function zeros.
method Revisits and extends Nyman-Beurling criterion linking RH to neural networks.
result Establishes connection between RH and minimization problem involving neural networks.
New findings show modern neural networks have finite sample complexity in o-minimal structures.
problem Understanding the learnability of modern neural networks in a broad context.
method Analyzing feedforward neural networks definable in o-minimal structures.
result Modern neural networks, including MLPs, CNNs, GNNs, and transformers, have finite sample complexity in the agnostic PAC setting.
Neural networks minimize error with shallow ReLU models for function estimation.
problem Estimating unknown functions from noisy data.
method Minimizing squared errors plus weight decay regularization.
result Neural network estimators are minimax optimal up to logarithmic factors.
SGD and weight decay encourage neural networks to learn low-rank weight matrices.
problem The bias of SGD towards low-rank weight matrices in neural networks.
method The study investigates the effect of SGD and weight decay on the rank of weight matrices in neural networks, both theoretically and empirically.
result Training with SGD and weight decay induces a bias towards rank minimization in weight matrices, which becomes more pronounced with smaller batch sizes and stronger weight decay.